python request 库POST请求从ajax爬取数据遇到formdata不会处理了

Question

目标爬取网站为http://www.ccgp-shaanxi.gov.cn/notice/list.do?noticetype=3&province=province

form data为：

以下为我的代码：

import requests
from urllib.parse import urlencode

base_url = 'http://www.ccgp-shaanxi.gov.cn/notice/noticeaframe.do?noticetype='
noticetype = '3'
url = base_url + noticetype
headers = {
    'Host': 'www.ccgp-shaanxi.gov.cn',
    'Referer': 'http://www.ccgp-shaanxi.gov.cn/notice/list.do?noticetype=3&province=province',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36',
    'X-Requested-With': 'XMLHttpRequest'}

data = {
    'parameters[purcatalogguid]': '',
    'page.pageNum': '',
    'parameters[title]': '税',
    'parameters[startdate]': '',
    'parameters[enddate]': '',
    'parameters[regionguid]': '6101',
    'parameters[projectcode]': '',
    'province': '',
    'parameters[purmethod]': ''
}

data = urlencode(data)
print(data)
request = requests.post(url, data=data, headers=headers)
# print(request.text)
print(request.headers)

但是获取到的信息反应、感觉formdata貌似没有起作用，因本人属于新手小白，各位大神能指点指点吗？

可以执行一下帮忙分析分析吗？

Answer 1

刚看了一下你的代码：

改了一下，我加了查询日期，6月1日到6月11的。其他字段自己加上就行了：

import requests
url = "http://www.ccgp-shaanxi.gov.cn/notice/noticeaframe.do?noticetype=3"
headers ={
"Host": "www.ccgp-shaanxi.gov.cn",
"Connection": "keep-alive",
"Cache-Control": "max-age=0",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate",
"Accept-Language": "zh-CN,zh;q=0.9",
"Cookie": "JSESSIONID=7FE3D1F9A78DB03F2C5D49090F024A75"
}
form_data = [('parameters[purcatalogguid]', ''),
('parameters[title]', ''),
('parameters[startdate]', '2018-06-01'),
('parameters[enddate]', '2018-06-11'),
('parameters[regionguid]', '610001'),
('parameters[projectcode]', ''),
('province', ''),
('parameters[purmethod]', ''),
]
page = requests.post(url=url, headers=headers, data=form_data).text
print(page)