doulun1915 2015-09-19 20:37
浏览 23

PHP curl简历下载

I'm currently trying to download satellite images from esa's Copernicus / Sentinel project with curl. Unfortunately the download keeps stopping at around 90% and the php script returns an Internal Server Error (500).

Therefore I would like to resume the download at a specific byte number. It seems that the esa server just ignores the http-range-header (CURLOPT_RANGE) and CURLOPT_RESUME_FROM doesn't change anything either.

If I use Google Chrome to download the file manually, the download also interrupts but continues after some time.

So, if Google Chrome can resume the download, curl should be able to do that, too. I would appreciate any help on how to do that.

Some details:

The file I'm trying to download is here (420MB), to access it you need to register at scihub.esa.int/dhus/.
Content-Type is application/octet-stream

My code:

$save_file = fopen($save_filepath, "w+");
$open_file = curl_init(str_replace(" ","%20", $url));
curl_setopt($open_file, CURLOPT_USERPWD, $username.":".$password);
curl_setopt($open_file, CURLOPT_TIMEOUT, 300);
curl_setopt($open_file, CURLOPT_FILE, $save_file);
curl_setopt($open_file, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($open_file, CURLOPT_PROGRESSFUNCTION, "trackprogress");
curl_setopt($open_file, CURLOPT_NOPROGRESS, false);
curl_exec($open_file);
curl_close($open_file);
fclose($save_file);

It works perfectly for smaller files (I've tested it with some images and pdf-files) and I can also download most of the satellite image (the first 380MB are downloaded). I tried to increase the timeout value, too, but the script terminates long before the 5 minutes are reached.

I tried curl_setopt($open_file, CURLOPT_RESUME_FROM, 1048576); and curl_setopt($open_file, CURLOPT_RANGE, "1048576-"); but the file always starts with the same bytes.

EDIT:

I can't answer my question, but for this specific case I found a workaround. So, if anybody reads this and also wants to download these satellite images with cURL by chance, here is what I did:

When downloading not just the image file, but the zip-file with some additional data, the download still keeps stopping, however with curl_setopt($open_file, CURLOPT_RESUME_FROM, $bytes_already_loaded); it is possible to skip the bytes which had previously been loaded and resume the download (which isn't possible for the image file). Thus, use this link instead of the image file.

  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥100 set_link_state
    • ¥15 虚幻5 UE美术毛发渲染
    • ¥15 CVRP 图论 物流运输优化
    • ¥15 Tableau online 嵌入ppt失败
    • ¥100 支付宝网页转账系统不识别账号
    • ¥15 基于单片机的靶位控制系统
    • ¥15 真我手机蓝牙传输进度消息被关闭了,怎么打开?(关键词-消息通知)
    • ¥15 装 pytorch 的时候出了好多问题,遇到这种情况怎么处理?
    • ¥20 IOS游览器某宝手机网页版自动立即购买JavaScript脚本
    • ¥15 手机接入宽带网线,如何释放宽带全部速度