PHP和AJAX网址获取数据

Question

I have a problem for scrape a URL, if I use a dev tools of Chrome i see the URL and preview if content, but if I intent open URL from browser return me 404 not found.

This is URL with AJAX data . I Need the parte.of odds comparison: http://www.betexplorer.com/soccer/england/premier-league/wolves-manchester-city/Kz6hM15m/ This is a AJAX data URL : www.betexplorer.com/gres/ajax/matchodds.php?p=0&e=Kz6hM15m&b=1x2 this URL return 404 not found in browser but in fiesta URL show the odds . Is possible ger the data from secondo url?

Thank you.

Answer 1

Upon quick inspection of the site you have mentioned in your question, I found out it is checking for referer and user-agent, when handling the request.

Adding header "Referer" with value of "http://www.betexplorer.com/" and a random "browser-like" user agent, yields the json response with requested odds :)

For example:

<?php
$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, "http://www.betexplorer.com/gres/ajax/matchodds.php?p=0&e=Kz6hM15m&b=1x2"); 
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.12011-10-16 20:23:00");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_REFERER, "http://www.betexplorer.com/");
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
$output = curl_exec($ch); 
curl_close($ch);