带有curl的简单html dom解析器在不同的URL上挂起太长时间

Question

I use simple HTML DOM parser together with curl (I do not have a big experience with curl) and I try to figure out why is hanging on different URL requests long. I have been trying to log with verbose but I did not get back any useful information. It seems like is a Caching problem because after long response all my other requests are acting the same till I clear Browser Cache

str_get_html(get_data($target));

function get_data($url)
{
 $ch = curl_init();
 $timeout = 30;
 curl_setopt($ch,CURLOPT_URL,$url);
 curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
 curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
 curl_setopt($ch, CURLOPT_USERAGENT, 'some useragent');
 $data = curl_exec($ch);
 curl_close($ch);
 return $data;
}

Answer 1

You are using CURLOPT_NOBODY curl option in your request. Are you sure what it does? It sends the HEAD request to the target url instead of the GET. There are lot of web servers available in the Internet which do accept the HEAD request and keep the request in stuck until the timeout occures. And this is what you are experiencing right now.