2011-11-28 23:51
浏览 17

使用SimplePie使用RSS源进行优化的数据库插入 - 需要创意

So basically I have just benchmarked my update_feeds controller and found that the amount of sql queries that are run are shocking. I am looking for a way to optimise the process of fectching multiple feeds and then insert the data into a table (title and url)

Currently there are 193 Feeds in the DB that I fetch the URL's for and then I process these one by one checking and insert data of them into another table. The problem is simplepie goes through and inserts every item.

So I am ending up with QUERIES: 3297 Total Execution Time 202.8051

I am looking for a way to optimise this process does anyone have any tips? I will post some code. Thanks.

Controller for fetching the feeds

        //$this->simplepie->cache_location = BASEPATH .'cache';
        $feeds = $this->FeedModel->get_feed_update_urls();
        foreach ($feeds as $feed_id => $feed_url) {
            $items = $this->simplepie->get_items();
            foreach ($items as $item) {
                $this->FeedItemModel->load($feed_id, md5($item->get_id()));
                $this->FeedItemModel->link = $item->get_permalink();
                $this->FeedItemModel->title = $item->get_title();
                $this->FeedItemModel->created_time = $item->get_date('Y-m-d H:i:s');

Model for inserting feeds

function save() {
        if ($this->_id !== false) {
            $this->db->query('UPDATE feed_items SET link=?, title=?, created_time=? WHERE id=?', array($this->link, $this->title, $this->created_time, $this->_id));
        } else {
            $this->db->query('INSERT INTO feed_items(feed_id, remote_id, link, title, created_time, updated_time) VALUES (?, ?, ?, ?, ?, NOW()) ON DUPLICATE KEY UPDATE remote_id=remote_id', array($this->feed_id, $this->remote_id, $this->link, $this->title, $this->created_time, $this->remote_id));
            $this->_id = $this->db->insert_id();
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

1条回答 默认 最新

  • duanjuduo4573 2011-11-29 00:28

    Instead of loading every single time and then saving, you should instead do an update (and then insert when update doesn't find rows). That will reduce by a single query per feed item.

    打赏 评论

相关推荐 更多相似问题