dqyuipw44576 2016-11-29 00:12
浏览 17
已采纳

长时间运行的控制台命令会变慢

Initial text

I created console in Laravel project. The command takes html data from one table, checks two pattern matches for each record by preg_match. If it returns true, updates are being done to other table's record that has the same attribute as record from the first table that is currently in focus in foreach loop. Number of records is cca 3500

After cca 150 iterations, command dramatically slows down, and I need one day for getting the command done.

I read all similar issues from this forum but they didn't help me. Not even the answer about forcing garbage collection.

Code is like following:

$ras = RecordsA::all();
$pattern = '/===this is the pattern===/';
foreach($ras as $ra){
    $html = $ra->html;
    $rb = RecordB::where("url", $ra->url)->first();
    $rb->phone = preg_match($pattern, $html, $matches) ? $matches[1] : $rb->phone;
    $rb->save();
}

I was searching for possible issue about preg_match performance but it was unsuccessful.

Did anybody meet such problem?


For MMMTroy update

I forgot to say I also tried custom but similar to your code:

$counter = DB::select("select count(*) as count from records_a")->first();
//Pattern for Wiktor Stribiżew :)
    $pattern = '/Telefon:([^<])+</';
for($i = 0; $i < $counter->count; $i+=150){
    $ras = RecordsA::limit(150)->offset($i);
    foreach($ras as $ra){
        $html = $ra->html;
        $rb = RecordB::where("url", $ra->url)->first();
        $rb->phone = preg_match($pattern, $html, $matches) ? $matches[1] : $rb->phone;
        $rb->save();
    }
}
  • 写回答

3条回答 默认 最新

  • douchong8393 2016-11-29 06:52
    关注

    "Pagination via OFFSET" is Order(N*N). You would be better off with Order(N), so "remember where you left off".

    More discussion.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(2条)

报告相同问题?

悬赏问题

  • ¥15 Python中的request,如何使用ssr节点,通过代理requests网页。本人在泰国,需要用大陆ip才能玩网页游戏,合法合规。
  • ¥100 为什么这个恒流源电路不能恒流?
  • ¥15 有偿求跨组件数据流路径图
  • ¥15 写一个方法checkPerson,入参实体类Person,出参布尔值
  • ¥15 我想咨询一下路面纹理三维点云数据处理的一些问题,上传的坐标文件里是怎么对无序点进行编号的,以及xy坐标在处理的时候是进行整体模型分片处理的吗
  • ¥15 CSAPPattacklab
  • ¥15 一直显示正在等待HID—ISP
  • ¥15 Python turtle 画图
  • ¥15 stm32开发clion时遇到的编译问题
  • ¥15 lna设计 源简并电感型共源放大器