使用requests库爬取公司网站的数据，用Fiddler观察正常获取数据和使用requests库爬取数据的Post请求，没看出有任何区别，可是却爬不到任何数据。

正常网页获取数据抓取到的Post请求如下：

POST http://10.245.0.225/REPORT_FM/MainController.do?method=queryDatas&type=SF&startTime=2021-01-26%2015:00:00&endTime=2021-01-27%2015:00:00&selectType=null HTTP/1.1
Host: 10.245.0.225
Connection: keep-alive
Content-Length: 390
Accept: */*
X-Requested-With: XMLHttpRequest
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36
__REQUEST_TYPE: AJAX_REQUEST
Origin: http://10.245.0.225
Referer: http://10.245.0.225/REPORT_FM/base/outStatistics/statisticsSF.jsp?globalUniqueID=D3AFCA5F9D40478F9EC64D28D1C26A40
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cookie: JSESSIONID=4D3C9ABEE95EE1860D281DDF7C8FE1CA

dtGridPager=%7B%22isExport%22%3Afalse%2C%22pageSize%22%3A300%2C%22startRecord%22%3A0%2C%22nowPage%22%3A1%2C%22recordCount%22%3A-1%2C%22pageCount%22%3A-1%2C%22parameters%22%3A%7B%22startTime%22%3A%222021-01-21+00%3A00%3A00%22%2C%22endTime%22%3A%222021-01-22+00%3A00%3A00%22%7D%2C%22fastQueryParameters%22%3A%7B%7D%2C%22advanceQueryConditions%22%3A%5B%5D%2C%22advanceQuerySorts%22%3A%5B%5D%7D

使用代码爬取网页抓取到的Post请求如下：

POST http://10.245.0.225/REPORT_FM/MainController.do?method=queryDatas&type=SF&startTime=2021-01-26%2011:00:00&endTime=2021-01-27%2011:00:00&selectType=null HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
__REQUEST_TYPE: AJAX_REQUEST
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
Cookie: JSESSIONID=6913FCE39C0D7C01D83904E1C5A2EA2F
Host: 10.245.0.225
Origin: http://10.245.0.225
Referer: http://10.245.0.225/REPORT_FM/base/outStatistics/statisticsSF.jsp?globalUniqueID=DD79A370C1FB4F23A8744E40DB12B700
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 390

dtGridPager=%7B%22isExport%22%3Afalse%2C%22pageSize%22%3A300%2C%22startRecord%22%3A0%2C%22nowPage%22%3A1%2C%22recordCount%22%3A-1%2C%22pageCount%22%3A-1%2C%22parameters%22%3A%7B%22startTime%22%3A%222021-01-21+00%3A00%3A00%22%2C%22endTime%22%3A%222021-01-22+00%3A00%3A00%22%7D%2C%22fastQueryParameters%22%3A%7B%7D%2C%22advanceQueryConditions%22%3A%5B%5D%2C%22advanceQuerySorts%22%3A%5B%5D%7D

可是得到的结果却截然相反，从网站正常获取有结果（Content-Length:6337)：

使用代码却没有任何结果(Content-Length:0)

请各位专家帮忙解决，万分感谢！

eclipse运行jsp文件web browser总显示此程序无法显示该网页

mlin_xu

4个回复