dsizmmwnm56437180 2016-03-14 10:37
浏览 61
已采纳

Yii2:MongoDB选择跳过大值

I'm writing yii2 command controller, that exports mongodb data to sphinx (as csv). MongoDB controller contains about 9M lines, and I've added a while cycle.

public function actionExportSphinx()
{
    $query = new MongoQuery;
    $count = (int) $query->select(['_id'])
                   ->from('test')
                   ->count();
    $i = 0;

    while($i < $count) {
        $rows = self::getExportData($i);
        self::printMongoRow($rows);
        unset($rows);
        $i += 100000;
    }
}

Each next cycle loop is more slower than previous one. And on $i = 500000 mongo excepts timeout... I know, that it is a mongo skip problem, but I don't know any solutions of this.

UPD: Added self::getExportData() and self::printMongoRow() methods.

public function printMongoRow($rows)
{
    foreach ($rows as $row) {
        printf("%d,\"%s\",\"%s\",\"%s\",\"%s\",%d,%d,%d
",
            $row['_id'],
            (isset($row['lastname']) ? str_replace('"', "", $row['lastname']) : ""),
            (isset($row['firstname']) ? str_replace('"', "", $row['firstname']) : ""),
            (isset($row['middlename']) ? str_replace('"', "", $row['middlename']) : ""),
            (isset($row['town']) ? str_replace('"', "", $row['town']['title']) : ""),
            (isset($row['birthday']) ? $row['birthday'] : 0),
            (isset($row['birthmonth']) ? $row['birthmonth'] : 0),
            (isset($row['birthyear']) ? $row['birthyear'] : 0)
            );
    }
}

public function getExportData($i)
{
    $query = new MongoQuery;
        $query->select(['lastname', 'firstname', 'middlename', 'town', 'birthday', 'birthmonth', 'birthyear'])
                ->from('test')
                ->limit(100000)
                ->offset($i);
        return $query->all();
}
  • 写回答

1条回答 默认 最新

  • dtpf76658 2016-03-14 15:37
    关注

    The answer is given in comments

    In case when you iterate over the whole MongoDb collection it is preferrable to use mongo query cursor directly instead of the skip-limit approach.

    When cursor is used you perform a query search only once and the results will be fetched into memory iteratively.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥100 关于加载卡的问题有能知道这个要怎么处理吗?
  • ¥100 rtmpose姿态评估
  • ¥15 java 通过反射找路径下的类,打包后就找不到
  • ¥15 通联支付网上收银统一下单接口
  • ¥15 angular有偿编写,
  • ¥15 centos7系统下abinit安装时make出错
  • ¥15 hbuildex运行微信小程序报错
  • ¥15 关于#python#的问题:我知道这个问题对你们来说肯定so easy
  • ¥15 wpf datagrid如何实现多层表头
  • ¥15 为啥画版图在Run DRC会出现Connect Error?可我Calibre的hostname和计算机的hostname已经设置成一样的了。