CURL请求不完整,可疑超时但不确定

I am currently using CURL via a php script running as daily cron to export product data in csv format from a site's admin area.

The normal way of exporting data will be to go to the Export page in a browser, and set the configuration, then click on "export data" button. But as the number of products I am exporting is very large, and it takes more than 5-10 mins to export the data, I've decided to use php's curl function to mimic this on a daily basis via cron.

Previously, it is working fine, but recently as I increased the number of products in the store by 500+, the script fails to return the exported data. Testing it manually via clicking on the "export" button in a browser, does return the data correctly. Thus there is no "timeout" issue with running the export in a browser manually.

I've tested and by removing/decreasing the number of products (thus the time needed), the php-curl script works fine again when run from cron.

So I suspect that it has something to do with timeouts issue, specifically with the curl function in php.

I've set both CURLOPT_TIMEOUT and CURLOPT_CONNECTTIMEOUT to '0' respectively to try. In the php-curl script, I've also set "set_time_limit(3000)". But still it does not work, and the request will timeout, with the script failing to return with a complete set of csv data.

Any help in helping me resolve/understand this issue will be much appreciated!

Edited: Added part of the code where the curl script is calling the export function.

$interface = new StoreInterface(); 
echo "Start exporting at " .  date('l jS \of F Y h:i:s A') . "
";
set_time_limit(3000);
$result_html = $interface->exportProducts();
//parse $result_html to only retain the csv format
preg_match('/<pre>(.*)<\/pre>/s',$result_html[0],$output);
if(strlen($output[1])<10) {   //debugging for now
echo "Export did not happen correctly. Quit
";
    die('Export unsuccessful');
}
file_put_contents($output_path,$output[1]);
echo "Script completed. Thank you! 
";
duanba2001
duanba2001 你检查过错误日志了吗?
大约 10 年之前 回复
dp411805872
dp411805872 没有特定的错误消息。是的,只有部分结果。当curl脚本在商店脚本中调用“export”函数时,脚本将csv数据作为临时文件输出到服务器中。如果成功,它应该将临时文件的完整内容返回到要处理的curl脚本。但在当前情况下,临时文件的内容突然中途停止,并且curl脚本将返回指示数据不完整。
大约 10 年之前 回复
doufencigui933699
啊啊啊小孔 “脚本超时”是什么意思?是否有特定的错误消息?你只得到部分结果吗?
大约 10 年之前 回复

1个回答



您可以使用以下curl选项将所有卷曲传输详细信息记录到日志文件中,并检查是否存在任何问题。</ p>

  $ fp = fopen('./ debug / transfer.log'); 
curl_setopt($ curl,CURLOPT_VERBOSE,TRUE);
curl_setopt($ curl,CURLOPT_STDERR,$ fp); \ n </ code> </ pre>

另外我认为CURLOPT_TIMEOUT选项不支持为无限期超时指定“0”值。 您需要在此处指定最大超时值。</ p>
</ div>

展开原文

原文

You can use the following curl options to log all the curl transfer details to a log file and check for any issues.

$fp = fopen('./debug/transfer.log');
curl_setopt($curl, CURLOPT_VERBOSE, TRUE);
curl_setopt($curl, CURLOPT_STDERR, $fp);

Also I believe that the CURLOPT_TIMEOUT option doesn't support specifying a "0" value for an indefinite timeout. You need to specify the maximum timeout value here.

duanjie5570
duanjie5570 我会尝试这种方法,看看如何。 谢谢。
大约 10 年之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问