doushansu9012 2014-10-30 08:30
浏览 55
已采纳

PHP数组插入/操作降级迭代

I am in the process of transferring data from one database to another. They are different dbs (mssql to mysql) so I cant do direct queries and am using PHP as an intermediary. Consider the following code. For some reason, each time it goes through the while loop it takes twice as much time as the time before.

$continue = true;
$limit = 20000;

while($continue){
    $i = 0;
    $imp->endTimer();
    $imp->startTimer("Fetching Apps");

    $qry = "THIS IS A BASIC SELECT QUERY";
    $data = $imp->src->dbQuery($qry, array(), PDO::FETCH_ASSOC);

    $inserts = array();
    $continue = (count($data) == $limit);

    $imp->endTimer();
    $imp->startTimer("Processing Apps " . memory_get_usage() );

    if($data == false){
        $continue = false;
    }
    else{
        foreach($data AS $row){

            // THERE IS SOME EXTREMELY BASIC IF STATEMENTS HERE

            $inserts[] = array(
                "paymentID"=>$paymentID,
                "ticketID"=>$ticketID,
                "applicationLink"=>$row{'ApplicationID'},
                "paymentLink"=>(int)($paymentLink),
                "ticketLink"=>(int)($ticketLink),
                "dateApplied"=>$row{'AddDate'},
                "appliedBy"=>$adderID,
                "appliedAmount"=>$amount,
                "officeID"=>$imp->officeID,
                "customerID"=>-1,
                "taxCollected"=>0
            );
            $i++;
            $minID = $row{'ApplicationID'};
        }
    }


    $imp->endTimer();
    $imp->startTimer("Inserting $i Apps");

    if(count($inserts) > 0){
         $imp->dest->dbBulkInsert("appliedPayments", $inserts);
    }

    unset($data);
    unset($inserts);
    echo "Inserted $i Apps<BR>";
}  

It doesn't matter what I set the limit to, the processing portion takes twice as long each time. I am logging each portion of the loop and selecting the data from the old database and inserting it into the new one take no time at all. The "processing portion" is doubling every time. Why? Here are the logs, if you do some quick math on the timestamps, each step labeled "Processing Apps" takes twice as long as the one before... (I stopped it a little early on this one, but it was taking a significantly longer time on the final iteration)

logs

  • 写回答

1条回答 默认 最新

  • douao1926 2014-10-30 13:28
    关注

    Well - so I don't know why this works, but if I move everything inside the while loop into a separate function, it DRAMATICALLY increases performance. Im guessing its a garbage collection / memory management issue and that having a function call end helps the Garbage collector know it can release the memory. Now when I log the memory usage, the memory usage stays constant between calls instead of growing... Dirty php...

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 使用ue5插件narrative时如何切换关卡也保存叙事任务记录
  • ¥20 软件测试决策法疑问求解答
  • ¥15 win11 23H2删除推荐的项目,支持注册表等
  • ¥15 matlab 用yalmip搭建模型,cplex求解,线性化处理的方法
  • ¥15 qt6.6.3 基于百度云的语音识别 不会改
  • ¥15 关于#目标检测#的问题:大概就是类似后台自动检测某下架商品的库存,在他监测到该商品上架并且可以购买的瞬间点击立即购买下单
  • ¥15 神经网络怎么把隐含层变量融合到损失函数中?
  • ¥15 lingo18勾选global solver求解使用的算法
  • ¥15 全部备份安卓app数据包括密码,可以复制到另一手机上运行
  • ¥20 测距传感器数据手册i2c