scraperwiki：为什么我的刮刀适用于1个网址而不是另一个？

Question

This is my first scraper https://scraperwiki.com/scrapers/my_first_scraper_1/

I managed to scrape google.com but not this page.

http://subeta.net/pet_extra.php?act=read&petid=1014561

any reasons why?

I have followed the documentation from here.

https://scraperwiki.com/docs/php/php_intro_tutorial/

And there is no reason why the code should not work.

Answer 1

It looks like you are specifying to find a specific element. Elements change dependent on the site you are scraping. So if it doesn't find the element you are looking for you get no return. Also I would look into creating your own scraping/spidering tool with curl. Not only will you learn a lot but you will find out a lot about how to scrape sites.

Also a side not you might want to consider abiding by the robots.txt file on the website you are scraping from or ask permission before scraping as it is considered impolite.