doufa5001 2018-05-11 13:36
浏览 9

php - 尝试从CSV下载URL,它随机URL停止

I've tried searching for this issue but couldn't find out why it is actually failing at random stages.

I have a CSV with 2 columns, ID and URL, there are 100 rows. I have the following code to loop through the CSV rows and use put_file_contents() to download the data at the URLs into files that are named [ID].jpg.

I am also outputting data to the screen each time around the loop and adding a line to "log.txt". For some reason it just stops working after a random URL and doesn't put anything in the error_log.

Sometimes it's after 17 URLs, sometimes it after 27, sometimes its after 49. I am at a loss as to why it fails.

<?php

set_time_limit(3600);

$data = array_map('str_getcsv', file('IDtoImageURL2.csv'));

$total = 0;

echo "Count total: ".count($data)."<br /><br />";

for($i = 1; $i < count($data); $i++)
{
    file_put_contents("log.txt", $data[$i][1]." - ", FILE_APPEND | LOCK_EX);
    $total++;

    $url = $data[$i][1];
    $img = "photos/".$data[$i][0].'.jpg';
    $img2 = "photos/named/".$data[$i][0].'.jpg';

    echo "<b>Getting ".$data[$i][0].":</b><br />";
    echo "URL: ".$data[$i][1]."<br />";
    echo "putting $img<br />";
    echo "putting $img2<br />";

    $imgData = file_get_contents($url);

    if($imgData)
    {
        file_put_contents($img, $imgData);
        file_put_contents($img2, $imgData);

        echo "<b>Done</b><br /><br />";

        file_put_contents("log.txt", "done".PHP_EOL, FILE_APPEND | LOCK_EX);
    }
    else
    {
        echo "<b>Failed</b><br /><br />";

        file_put_contents("log.txt", "failed".PHP_EOL, FILE_APPEND | LOCK_EX);
    }   
}

echo "Total: ".$total;

?>

Here is the CSV data that I'm working with.

What's strange is that sometimes it posts some of the echo output to the screen and sometimes it just silently errors with a blank scree. The URLs are looped through in the same order and so I don't understand why the results keep differing. Any input would be greatly appreciated.

Thank you for your time

  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥100 支付宝网页转账系统不识别账号
    • ¥15 基于单片机的靶位控制系统
    • ¥15 AT89C51控制8位八段数码管显示时钟。
    • ¥15 真我手机蓝牙传输进度消息被关闭了,怎么打开?(关键词-消息通知)
    • ¥15 下图接收小电路,谁知道原理
    • ¥15 装 pytorch 的时候出了好多问题,遇到这种情况怎么处理?
    • ¥20 IOS游览器某宝手机网页版自动立即购买JavaScript脚本
    • ¥15 手机接入宽带网线,如何释放宽带全部速度
    • ¥30 关于#r语言#的问题:如何对R语言中mfgarch包中构建的garch-midas模型进行样本内长期波动率预测和样本外长期波动率预测
    • ¥15 ETLCloud 处理json多层级问题