I'm building a php 'web crawler' to populate a database with housing records which I then want to use for analysis. The problem is that the website that I'm trying to access has 2 pages:
- on the first, I need to select an option from a drop-down list after which the page auto re-loads (the page is using a POST form submit)
- when the POST variable is filled, it loads the individual nodes that I'm interesting in
My question is whether I can directly access this 2nd page and read the data into a string using php? Normally I would just use file_get_contents() for this, but I don't know how I can add a POST header as additional input to this?
