Im experimenting with some RSS reader/fetcher im writing at the moment. Everything is going smoothly except 1 thing. It's terribly slow.
Let me explain:
- I fetch the list of RSS feeds from the database
- I iterate every feed from this list, open it with cURL and parse it with SimpleXMLElement
- I check descriptions and title's of these feeds with a given keyword, to see if its already in database or not.
- If its not i add it to database.
For now i am looping through 11 feeds. Which gives me a page loading time of 18 seconds. This is without updating the database. When there are some new articles found, it goes up to 22 seconds (on localhost).
On a live webserver, my guess is that this will be even slower, and maybe goes beyond the limit php is setup to.
So my question is, what are your suggestions to improve speed.. and if this is not possible, whats the best way to break this down into multiples executions, like say 2 feeds at a time? I'd like to keep it all automated, dont want to click after every 2 feeds.
Hope you guys have some good suggestions for me!
If you want some code example let me know and ill paste some
Thanks!