想请教一下python爬虫中文乱码问题

Question

小白最近在学爬虫，请问各位大哥们我想爬取的这句话“第一回·宴桃园豪杰三结义 斩黄巾英雄首立功”为啥是乱码呢
我用利用正则解析下面这个
第一回·宴桃园豪杰三结义 斩黄巾英雄首立功
代码如下，问题可能有些弱智，但恳请各位不吝赐教

import requests
import re

url = 'https://www.shicimingju.com/book/sanguoyanyi.html'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59"}
page_text = requests.get(url=url, headers=headers).text

ex = '<a href="/book/sanguoyanyi/1.html">(.*?)</a>'
text_data = re.findall(ex, page_text, re.S)
print(text_data)

这个是乱码情况：['第ä¸\x80å\x9b\x9e·宴æ¡\x83å\x9b\xad豪æ\x9d°ä¸\x89ç»\x93ä¹\x89 æ\x96©é»\x84å·¾è\x8b±é\x9b\x84é¦\x96ç«\x8bå\x8a\x9f']

Answer 1

教你一招，请求数据之前先右键查看网页源代码，看看最上面的charset是啥，一般都是utf-8或者gbk的，要按它给的写程序，你这个网页是utf8编码，所以可以这样写，参考：

import requests
import re
url = 'https://www.shicimingju.com/book/sanguoyanyi.html'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59"}
res = requests.get(url=url, headers=headers)
res.encoding='utf-8'
page_text = res.text
ex = '<a href="/book/sanguoyanyi/1.html">(.*?)</a>'
text_data = re.findall(ex, page_text, re.S)
print(text_data)