cURL无法获取特定网站的内容

Question

I try to get the content of this website with cURL

www.mytischtennis.de/public/

but it gets no body response. With many other websites the code works:

<?php


$output = grabPage(
    "http://www.mytischtennis.de/public/"
  //"http://www.spiegel.de" //this page and many other pages are working
);

if (is_array($output)) {
    var_dump($output);
} else {
    echo $output;
}

function grabPage($url)
{
    $ch = curl_init();
    $cookiePath= dirname(__FILE__) . "\cookie.txt";

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($ch, CURLOPT_MAXREDIRS, 50);
    curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($ch, CURLOPT_TIMEOUT, 40);
    curl_setopt($ch, CURLOPT_COOKIE, 'CFID=c7a592d8-5798-4471-9af4-4c4d954d03cd; cfid=c7a592d8-5798-4471-9af4-4c4d954d03cd; MYTT_COOKIESOK=1; CFTOKEN0=; cftoken=0; SRV=74');
    curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiePath);
    curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiePath);

    $fpErrors = fopen(dirname(__FILE__) . '\errorlog.txt', 'w');

    curl_setopt($ch, CURLOPT_VERBOSE, 1);
    curl_setopt($ch, CURLOPT_STDERR, $fpErrors);

    curl_setopt($ch, CURLOPT_URL, $url);
    ob_start();
    $curl_exec = curl_exec($ch);
    ob_end_clean();


    if ($curl_exec === false) {
        echo 'Error: ' . curl_error($ch);
    } else {
        echo 'Success';
    }

    var_dump(curl_getinfo($ch));
    curl_close($ch);

    return $curl_exec;
}

I tried to read a fiddler/wireshark dump of a browser request to this website. But I can't figure out which of that many requests and which parameters are necessary to get the content. You can test cURL with the url www.mytischtennis.de/public/ also on this website: http://onlinecurl.com/

Answer 1

You need to accept gzip encoding in the response by sending the appropriate HTTP header in the request:

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept-Encoding: gzip'));

Now your answer from the server might or might not be gziped. The proper way to check that is to interpret the Content-Encoding HTTP header in the response. But you can also do it quick and dirty like this:

$content = @gzdecode($curl_exec);
return $content !== false ? $content : $curl_exec;