使用php触发JavaScript滚动事件

Question

I'm trying to parse a HTML-document using XPath. I'm "fetching" the content of the HTML document using the file_get_contents() function within php.

The problem is that I will never get the full content of the page because there is some JavaScript implemented that requests more content when the user scrolls down. An example would be this page: demo.

I already read about some "browser components" mentioned in an answer on another stack exchange portal but I was unable to find something for my special needs.

Is there a way to trigger these scroll events with php? Are there any other alternatives how I can get the full page content?

Answer 1

I assume you want to scrape pages. I would avoid using php here. Scraping pages with PhantomJS could lead to a more solid approach.

Nevertheless you asked for a php solution.

The technique used is called ajax pagination. You can figure out the URL and parameters with a the web developer tool of your browser (look in the network section of the tool).

I prefer Firebug for Firefox.

After you got the correct parameters you can call this URL with file_get_contents(), by adapting the parameters you can load the following pages.