php脚本获取动态加载的网站计划

Question

I'm trying to make a php script which can take a specific tv channel schedule from a specific website written in Yii Framework. I'm trying to get the full HTML using php cURL but I can't find this section.

$curl = curl_init('http://port.hu/tv');
curl_setopt ($curl, CURLOPT_POSTFIELDS, 'id=tvchannel-3&date=2017-02-05'); // for example
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_POST, 1);
$content = curl_exec($curl);
echo $content;

I tried before with php file_get_contents with the same result. Any idea? Thanks.

Answer 1

You cannot scrape directly from the website page, because looks like the website is using ajax (I guess) to load the data onto the page. So what I did, I monitor the Network activity on the page using Chrome Developer Tools, and I found this API url:

http://port.hu/tvapi?channel_id=tvchannel-3&i_datetime_from=2017-02-05&i_datetime_to=2017-02-10

It returned JSON strings, and the dev does not secure the API. So no need to scrape anymore, just load the JSON API directly.