最近我在爬取小说的时候,出现了爬取到了小说最后一段,开头以及中间部分全部没有了,下面是我写的代码以及请求的内容:
URL = 'https://www.kankezw.com/du/23/23361/1633023.html'
head = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36 Edg/92.0.902.78'
}
html = requests.get(url=URL, headers=head)
html.encoding = 'utf-8'
page_txt = BeautifulSoup(html.text, 'html.parser')
html_txt = page_txt.find('div', attrs={'id': 'content1'})
print(html_txt.text)
站在原地望着少年那恍如与世隔绝的孤独背影,萧薰儿踌躇了一会,然后在身后一干嫉妒的狼嚎声中,快步追了上去,与少年并肩而行…