duanchendu69495 2018-03-04 14:38
浏览 299

PHP并行cURL 502坏网关

I'm using parallel cURL to download webpages source code by URL faster then single curl, but sometimes i get random amount of 502 error. If i'm using single cURL instead of parallel, i get less 502. Any solution how can i fix this?

502 Bad Gateway
The server returned an invalid or incomplete response

Tested with:

  1. Localhost - XAMPP - PHP Version 7.0.13

  2. VPS - (LAMP) - PHP Version 7.0.25

PHP

    $curl_arr = array();
    $master = curl_multi_init();

    for($i = 0; $i < count($urls); $i++)
    {
        $random = rand(1,9999);

        $link = trim($urls[$i]);

        $link = $link."?".$random;

        $url = trim($link);
        $curl_arr[$i] = curl_init($url);
        curl_setopt($curl_arr[$i], CURLOPT_RETURNTRANSFER, true);
        curl_setopt($curl_arr[$i], CURLOPT_FRESH_CONNECT, TRUE);
        curl_setopt($curl_arr[$i], CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36');
        curl_setopt($curl_arr[$i], CURLOPT_HTTPHEADER, $headers);
        curl_setopt($curl_arr[$i], CURLOPT_SSL_VERIFYPEER, false);
        curl_setopt($curl_arr[$i], CURLOPT_SSL_VERIFYHOST, false);
        curl_setopt($curl_arr[$i], CURLOPT_ENCODING , "gzip");
        curl_setopt($curl_arr[$i], CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4 );

        curl_multi_add_handle($master, $curl_arr[$i]);
    }
    do 
    {
        curl_multi_exec($master,$running);
    } 
    while($running > 0);
  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥15 Matlab问题解答有两个问题
    • ¥50 Oracle Kubernetes服务器集群主节点无法访问,工作节点可以访问
    • ¥15 LCD12864中文显示
    • ¥15 在使用CH341SER.EXE时不小心把所有驱动文件删除了怎么解决
    • ¥15 gsoap生成onvif框架
    • ¥15 有关sql server business intellige安装,包括SSDT、SSMS。
    • ¥15 stm32的can接口不能收发数据
    • ¥15 目标检测算法移植到arm开发板
    • ¥15 利用JD51设计温度报警系统
    • ¥15 快手联盟怎么快速的跑出建立模型