木三136 2021-03-25 14:18 采纳率: 78.9%
浏览 123
已采纳

关于python的爬虫问题

在使用爬虫的代理时,遇到连接超时的问题,应该怎么解决啊

import requests

# 代理ip请求测试网站
url = "http://myip.ipip.net/"
# 封装请求头
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'}

# 封装代理ip参数值:协议/ip/端口
proxy = {'http': 'http://157.245.64.113:8080'}
# 发送请求
response = requests.get(url=url, headers=headers, proxies=proxy, timeout=10)
print(response.status_code)  # 打印请求返回的状态码
# 获取响应数据
response_data = response.text
# 把请求之后的获取到的响应数据保存到本地文件夹,以查看代理请求是否成功!
with open('ip.html', 'w', encoding='utf-8') as adi:
    adi.write(response_data)
exit()  # 代码执行完毕退出程序

这是报的错误

Traceback (most recent call last):
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 1198, in getresponse
    response.begin()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "D:\Learn\Anaconda3\envs\python35\lib\socket.py", line 576, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\util\retry.py", line 531, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\packages\six.py", line 735, in reraise
    raise value
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 337, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='157.245.64.113', port=8080): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/Learn/PyCharm/项目制作_1/火车票/test2.py", line 12, in <module>
    response = requests.get(url=url, headers=headers, proxies=proxy, timeout=10)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='157.245.64.113', port=8080): Read timed out. (read timeout=10)


 

  • 写回答

4条回答 默认 最新

  • coagenth 2021-03-25 19:00
    关注

    换代理,或在requests.get时加长超时设置,timeout=20。

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(3条)

报告相同问题?

悬赏问题

  • ¥20 wireshark抓不到vlan
  • ¥20 关于#stm32#的问题:需要指导自动酸碱滴定仪的原理图程序代码及仿真
  • ¥20 设计一款异域新娘的视频相亲软件需要哪些技术支持
  • ¥15 stata安慰剂检验作图但是真实值不出现在图上
  • ¥15 c程序不知道为什么得不到结果
  • ¥40 复杂的限制性的商函数处理
  • ¥15 程序不包含适用于入口点的静态Main方法
  • ¥15 素材场景中光线烘焙后灯光失效
  • ¥15 请教一下各位,为什么我这个没有实现模拟点击
  • ¥15 执行 virtuoso 命令后,界面没有,cadence 启动不起来