冷山集 2020-07-27 09:23 采纳率: 100%
浏览 174
已采纳

python爬取36kr时模拟Post请求返回的内容不是需要的内容

python爬取36kr时模拟Post请求返回的内容不是需要的内容

import requests
import json
import re



headers = {
'accept': '*/*',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'zh-CN,zh;q=0.9',
'content-length': '301',
'content-type': 'application/json',
'origin': 'https://36kr.com',
'referer': 'https://36kr.com/information/technology',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-site',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36'
}

data = {
    "partner_id":"web",
    # "timestamp":1595811108197,
    "param":{
        "subnavType":1,
        "subnavNick":"technology",
        "pageSize":30,"pageEvent":1,
        "pageCallback":"eyJmaXJzdElkIjozMDAwNzU0LCJsYXN0SWQiOjI5OTk4NDksImZpcnN0Q3JlYXRlVGltZSI6MTU5NTgxMTA5MjM5MywibGFzdENyZWF0ZVRpbWUiOjE1OTU2Mzg4MDAwMDB9",
        "siteId":1,
        "platformId":3
    }
}


Request_URL = 'https://gateway.36kr.com/api/mis/nav/ifm/subNav/flow'

r = requests.post(Request_URL, data=data, headers=headers, timeout=9).text
# json_list = json.loads(r)
# print(json_list)
print(type(r))
print(r)

返回的内容如下:

<class 'str'>
<!DOCTYPE html><html><head><title>Apache Tomcat/8.5.5 - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - </h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>java.lang.NullPointerException
</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.5.5 logs.</u></p><hr class="line"><h3>Apache Tomcat/8.5.5</h3></body></html>

求解接下来该怎么样编写代码

  • 写回答

4条回答 默认 最新

  • 放风喽 2020-07-27 20:16
    关注

    HTTP Status 500所以肯定什么地方错了
    我用网页打开也是500,所以你再看看接口的必要元素是不是少了

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(3条)

报告相同问题?

悬赏问题

  • ¥66 定制开发肯德基自动化网站下单软件
  • ¥20 vscode虚拟环境依赖包未安装
  • ¥15 odoo17关于owl开发js代码问题
  • ¥15 光纤中多普勒频移公式的推导
  • ¥15 怎么制作一个人脸识别门禁系统
  • ¥20 大华dss监控平台网络关闭登不进去
  • ¥15 请使用蚁群算法解决下列问题,并给出我完整的代码
  • ¥20 关于php录入完成后,批量更新数据库
  • ¥15 请教往复密封润滑问题
  • ¥15 cocos creator发布ios包