dsizmmwnm56437180 2016-03-14 02:37
浏览 61
已采纳

Yii2:MongoDB选择跳过大值

I'm writing yii2 command controller, that exports mongodb data to sphinx (as csv). MongoDB controller contains about 9M lines, and I've added a while cycle.

  1. public function actionExportSphinx()
  2. {
  3. $query = new MongoQuery;
  4. $count = (int) $query->select(['_id'])
  5. ->from('test')
  6. ->count();
  7. $i = 0;
  8. while($i < $count) {
  9. $rows = self::getExportData($i);
  10. self::printMongoRow($rows);
  11. unset($rows);
  12. $i += 100000;
  13. }
  14. }

Each next cycle loop is more slower than previous one. And on $i = 500000 mongo excepts timeout... I know, that it is a mongo skip problem, but I don't know any solutions of this.

UPD: Added self::getExportData() and self::printMongoRow() methods.

  1. public function printMongoRow($rows)
  2. {
  3. foreach ($rows as $row) {
  4. printf("%d,\"%s\",\"%s\",\"%s\",\"%s\",%d,%d,%d
  5. ",
  6. $row['_id'],
  7. (isset($row['lastname']) ? str_replace('"', "", $row['lastname']) : ""),
  8. (isset($row['firstname']) ? str_replace('"', "", $row['firstname']) : ""),
  9. (isset($row['middlename']) ? str_replace('"', "", $row['middlename']) : ""),
  10. (isset($row['town']) ? str_replace('"', "", $row['town']['title']) : ""),
  11. (isset($row['birthday']) ? $row['birthday'] : 0),
  12. (isset($row['birthmonth']) ? $row['birthmonth'] : 0),
  13. (isset($row['birthyear']) ? $row['birthyear'] : 0)
  14. );
  15. }
  16. }
  17. public function getExportData($i)
  18. {
  19. $query = new MongoQuery;
  20. $query->select(['lastname', 'firstname', 'middlename', 'town', 'birthday', 'birthmonth', 'birthyear'])
  21. ->from('test')
  22. ->limit(100000)
  23. ->offset($i);
  24. return $query->all();
  25. }

展开全部

  • 写回答

1条回答 默认 最新

  • dtpf76658 2016-03-14 07:37
    关注

    The answer is given in comments

    In case when you iterate over the whole MongoDb collection it is preferrable to use mongo query cursor directly instead of the skip-limit approach.

    When cursor is used you perform a query search only once and the results will be fetched into memory iteratively.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
编辑
预览

报告相同问题?

悬赏问题

  • ¥15 PADS Logic 原理图
  • ¥15 PADS Logic 图标
  • ¥15 电脑和power bi环境都是英文如何将日期层次结构转换成英文
  • ¥20 气象站点数据求取中~
  • ¥15 如何获取APP内弹出的网址链接
  • ¥15 wifi 图标不见了 不知道怎么办 上不了网 变成小地球了
手机看
程序员都在用的中文IT技术交流社区

程序员都在用的中文IT技术交流社区

专业的中文 IT 技术社区,与千万技术人共成长

专业的中文 IT 技术社区,与千万技术人共成长

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

客服 返回
顶部