小白python爬网页错误：'gbk' codec can't decode byte 0x91 in position 1298: illegal multibyte sequence

Question

跟着视频学爬了一下网页。网页编码是gb2312，代码如下：

import requests
base_url = "https://search.sina.com.cn/?"
headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36',
}
params = {
    'q': '新型冠状病毒',
    'c': 'news',
    'from': 'channel',
    'ie': 'utf-8'
}
response = requests.get(base_url, headers=headers, params=params)
response_str = response.content.decode('gbk')

print(response.text)

with open('sina_news.html', 'w', encoding='gbk') as fp:
     fp.write(response_str)

然后就出现了这个错误：line 20, in
response_str = response.content.decode('gbk')
UnicodeDecodeError: 'gbk' codec can't decode byte 0x91 in position 1298: illegal multibyte sequence
查了很多方法都不太明白，用utf-8会乱码，用tryexcept和error ignore也是乱码。求助大佬们QAQ

Answer 1

https://www.cnblogs.com/mengyu/p/6638975.html