储存txt文件时报错'gbk' codec can't encode character '\u200b' in position 164: illegal multibyte sequence

Question

问题遇到的现象和发生背景

爬虫信息储存txt文件时报错'gbk' codec can't encode character '\u200b' in position 164: illegal multibyte sequence

用代码块功能插入代码，请勿粘贴截图

import requests
import re
import time
import csv
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36 SE 2.X MetaSr 1.0',
    'Cookie': 'BIDUPSID=A87FDC113E9F5C879F4BEA4D7D6F5A72; PSTM=1662346944; BD_UPN=12314753; newlogin=1; BDUSS=40SGNtOGUzSFh2NHFTSi0zZW9Pa0pIeE5NUnB6Ymt0RDdIUGdqVTVDaUpWMTlqRVFBQUFBJCQAAAAAAAAAAAEAAACJedgyQUHHo8POAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAInKN2OJyjdjSU; BDUSS_BFESS=40SGNtOGUzSFh2NHFTSi0zZW9Pa0pIeE5NUnB6Ymt0RDdIUGdqVTVDaUpWMTlqRVFBQUFBJCQAAAAAAAAAAAEAAACJedgyQUHHo8POAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAInKN2OJyjdjSU; BAIDUID=A87FDC113E9F5C878BD59310E3A6E04A:SL=0:NR=10:FG=1; ispeed_lsm=2; BDORZ=FFFB88E999055A3F8A630C64834BD6D0; sug=3; sugstore=1; ORIGIN=0; bdime=0; BAIDUID_BFESS=A87FDC113E9F5C878BD59310E3A6E04A:SL=0:NR=10:FG=1; Hm_lvt_aec699bb6442ba076c8981c6dc490771=1665671890,1665745286; Hm_lpvt_aec699bb6442ba076c8981c6dc490771=1665745286; delPer=0; BD_CK_SAM=1; PSINO=5; BA_HECTOR=212024a12ha5a0010h84886k1hkigg61a; ZFY=:BKhQhsIdmwipwi9PbQ4h5ytjGOPXDsCSQfVRTuUcXVE:C; baikeVisitId=af3dc6a3-770a-4941-ad83-ab9dd0ce59ae; COOKIE_SESSION=129_0_1_0_8_1_1_0_1_1_0_0_129_0_1_0_1665745416_0_1665745415%7C5%230_0_1665745415%7C1; H_PS_645EC=9813r29D6TCK%2BXRVz5TlZby%2BLvNs6AnvuSOkr76NyC4OTdjCvtetKIWOu%2FPQSLqexz77iV8tlV4L; BDRCVFR[C0p6oIjvx-c]=mk3SLVN4HKm; H_PS_PSSID=37568_36551_37551_37358_37396_36807_37405_36789_37538_37497_37508_22159_37570; BDSVRTM=955'
}
url = 'https://www.baidu.com/s?rtt=1&bsst=1&cl=2&tn=news&ie=utf-8&word=阿里巴巴'
res = requests.get(url, headers=headers)
res.encoding = 'utf-8'

# 爬取一个公司的多页
def sduxsyg(page):
    if page==0:
        url = 'https://www.view.sdu.edu.cn/xsyg.htm'
    else:
        url = 'https://www.view.sdu.edu.cn/xsyg/'+ str(177-page)+'.htm'

    res = requests.get(url, headers=headers)
    res.encoding = 'utf-8'
    res=res.text
    # 其他相关爬虫代码
    p_title = '
.*?'
    title = re.findall(p_title, res, re.S)
    print (title)
    p_href = '
.*?<a href="(.*?)" target'
    href = re.findall(p_href, res, re.S)
    print (href)
    p_date = '<dl>.*?<a href=.*?<dd class="dd1">(.*?)</dd>'
    date = re.findall(p_date, res, re.S)
    print (date)
    p_address = '<dl>.*?<a href=.*?<dd class="dd2">(.*?)</dd>'
    address = re.findall(p_address, res, re.S)
    print (address)

    data_list=list(zip(title, href, date, address))
    print(data_list)
    file1 = open('./学术预告.txt','a')  # 如果把a改成w的话，则每次生成txt的时候都会把原来的txt清空；如果出现乱码问题，则设置encoding参数为utf-8，写成file1 = open('E:\\数据挖掘报告.txt', 'a'，encoding='utf-8')

    for i in range(len(title)):  # range(len(title)),这里因为知道len(title) = 10，所以也可以写成for i in range(10)
        title[i] = title[i].strip()  # strip()函数用来取消字符串两端的换行或者空格，不过这里好像不太需要了
        title[i] = title[i].replace('\xa0', '')  #\xa0是不间断空白符 我们通常所用的空格是\x20，是在标准ASCII可见字符0x20~0x7e范围内。而\xa0属于latin1（ISO/IEC_8859-1）中的扩展字符集字符，代表空白符nbsp(non-breaking space)。latin1字符集向下兼容ASCII（0x20~0x7e）。通常我们见到的字符多数是latin1的，比如在MySQL数据库中
        title[i] = title[i].replace('\u200b', '')  # \u200b 是Unicode(万国码)中的零宽度字符,可以理解为不可见字符
        print('第'+str(page+1)+'页第'+str(i+1)+'条学术报告：'+'\t'+title[i]+'\t'+href[i]+'\t'+date[i]+'\t'+address[i]+'\n')
        file1.write('第'+str(page+1)+'页第'+str(i+1)+'条学术报告：'+'\t'+title[i]+'\t'+href[i]+'\t'+date[i]+'\t'+address[i]+'\n')
    file1.close()

    # data_list=[]
    # for a,b,c,d in zip(title, href, date, address):
    #     x={}
    #     x['标题']=a
    #     x['链接']=b
    #     x['日期']=c
    #     x['地址']=d
    #     data_list.append(x)
    # with open('./学术预告.csv','a', newline='') as f:  # 如果把a改成w的话，则每次生成txt的时候都会把原来的txt清空；如果出现乱码问题，则设置encoding参数为utf-8.
    #     write = csv.writer(f)  # 创建writer对象
    #     write.writerow(['标题','链接','日期','地址'] )
    #     for info in data_list:
    #         write.writerow(info.values())
    #



for i in range(10):  # 这里一共爬取了10页
    sduxsyg(i)  # i是从0开始的序号，所以要写成i+1表示第几页
    print('第' + str(i+1) + '页爬取成功')  # i是从0开始的序号，所以写i+1
    time.sleep(3)  # 不要爬太快

运行结果及报错内容

我想要达到的结果

想知道为啥错了，该怎么修改

Answer 1

Py文件头加个试试

# coding=utf-8

打开文件加

file1 = open('./学术预告.txt','a',encoding='utf-8')