爬虫练习中json报错

Question

在练习爬取药监局数据中出现的问题，不知道怎么处理

代码如下：

import requests
if __name__ == "__main__":
    url = 'http://scxk.nmpa.gov.cn:81/xk/itownet/portalAction.do?hKHnQfLv=5WZOD3hXtU4QVUQZgOJtVpsflTwXNf6zZJQJEEdJtzwDAOJHs9ts.RbByiWHHepyVLIb6XSnTxuv5XOMij3QOsP_nzj6h2aZ_1x7d83Z7qNlwLxa.b9Fin.RXQAsf4z7iNf1Jj7rtfy4epYdq8BSdDzUiDpKXl6jCj6lb_50S7auZ_fFog5uuWsNt3lA41NY1bQlfdaMtqGTkgIEilyWG7L6nKggcvzmpnZm.ohOo5frrGrJfnEqufEVzjCwsXquOOM7gYvtc9P7FWQ8jrHLjMG&8X7Yi61c=4rEsQdayCx7AeK_SBYQz_u3iJuhgbE.jXLoi9cG5V3DnpvRH84mU2JmEl7UZyXP1XGFlFrm6jmuQkZZu4oc5uL.wEh76RsctqHUBxO48ktzcCP7ZKzeozeS48Wr8EF9Lw'
    data = {
        'on': 'true',
        'page': '1',
        'pageSize': '15',
        'productName':'',
         'conditionType': '1',
          'applyname':'',
           'applysn':'',
    }
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36'
    }
    id_list = []
    json_ids = requests.post(url=url, headers=headers,data=data).json()
    for dic in json_ids['list']:
        id_list.append(dic['ID'])
    post_url = 'http://scxk.nmpa.gov.cn:81/xk/itownet/portalAction.do?hKHnQfLv=5xWQ5q2qqWp1LWrDyWkSmOgydpifZ0TOO861I8niRMUk8uHxl.Zh6TPq3Fjp6XQ66g3OSN0yRXCOr6wtdHr60kRl36rNDrke0bzNMcj.PvC9Bp4m7.dmN5PVviJkTekiBMY4UtuiH9sGOsy8nwDPtmfNcS.NEOosLHkFA0xctyzfsBAxxB6ZIkBSolodOcC49qxJRDnhZlFpP2x8QkaNhaWEbyBhG975bFLcqC5mq69sHjgwU5LKDfXi0o2GO4r85JMIEJDNGB.XE5ZwsdDlEeq&8X7Yi61c=4qCdT8_THr9vGLpcQph2glB4PEWNMRQmZ6SXgJEbaf3hI_.ZUfcP.XhLnLBzbboTH1faF41xBdDuPiMknUR4tsQkXBU3HAzXBbmqTWFAxpZEij7sO.dfnaALlMDICHZrg'
    for id in id_list:
        data = {
        'id':id
        }
        detail_json = requests.post(url=post_url,headers=headers,data=data).json()
        print(detail_json)


报错代码如下：

`````` During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/pythonProject1/爬虫联系.py", line 60, in <module>
    json_ids = requests.post(url=url, headers=headers,data=data).json()
  File "C:\Users\13754\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 4 column 1 (char 6)

尝试将json转化为text 或更改url 未成功

代码应该怎么修改才可以运行

Answer 1

这问题的出现是因为你的代码25行:

json_ids = requests.post(url=url, headers=headers,data=data).json()

requests返回的数据不是标准的json数据引起的，可以检查该url的响应是否是标准的json数据，如果确定是的话，可以用 response.json() 来获取数据，如果不是的 可以用 response.text 查看数据在选择对应的处理方式