问题遇到的现象和发生背景 :在B站尚硅谷爬虫代理课上学到了代理基本应用,代码是按照视频上写的,
但是老师用的是http而我的电脑浏览器只有https
问题相关代码,请勿粘贴截图 :
url = 'https://www.baidu.com/s?wd=ip%27
import urllib.request
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36 Edg/101.0.1210.53'}
#request = urllib.request.Request(url = url,headers = headers)
#请求对象定制
request = urllib.request.Request(url = url,headers=headers)
#模拟浏览器访问服务器
response= urllib.request.urlopen(request)
proxies ={'http':'118.24.219.151:16817'}
#handler build_opener open
handler = urllib.request.ProxyHandler(proxies = proxies)
opener = urllib.request.build_opener(handler)
response = opener.open(request)
#获取响应信息
content = response.read().decode('utf-8')
保存
with open('daili.html','w',encoding='utf-8')as fp:
fp.write(content)