duanju8308 2019-01-31 13:02
浏览 122

在收到的HTML中获取页面内容curl错误404

I am trying to get the page contents from an url.
This is the url i am trying to get the content of:

https://www.facebook.com/events/383049449109054/

At the moment i am doing it the following way:

function GetHtmlContents( $url )
{
    $user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36';

    $options = array(

        CURLOPT_CUSTOMREQUEST  =>"GET",        //set request type post or get
        CURLOPT_POST           => false,        //set to GET
        CURLOPT_USERAGENT      => $user_agent, //set user agent
        CURLOPT_COOKIEFILE     =>"cookie.txt", //set cookie file
        CURLOPT_COOKIEJAR      =>"cookie.txt", //set cookie jar
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

this works great for the most part but in the recieved html there is an 404 error for some content i need:

Network request encountered error HTTP 404 HTTP_CLIENT_ERROR: 404 Not Found

Not Found

The requested URL /api/graphql/ was not found on this server.

See images below the clarify the problem:

original page when visiting it from a browser as a normal user:
enter image description here
page when requesting it from curl in php:
enter image description here

How do i make sure this content is also returned by the curl request?
If more information is needed let me know so i can clarify!

  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥15 html5+css有人可以帮吗?
    • ¥30 Unity接入微信SDK 无法开启摄像头
    • ¥20 有偿 写代码 要用特定的软件anaconda 里的jvpyter 用python3写
    • ¥20 cad图纸,chx-3六轴码垛机器人
    • ¥15 移动摄像头专网需要解vlan
    • ¥20 access多表提取相同字段数据并合并
    • ¥20 基于MSP430f5529的MPU6050驱动,求出欧拉角
    • ¥20 Java-Oj-桌布的计算
    • ¥15 powerbuilder中的datawindow数据整合到新的DataWindow
    • ¥20 有人知道这种图怎么画吗?