weixin_44385960
木三136
2021-03-25 14:18
采纳率: 60%
浏览 74

关于python的爬虫问题

在使用爬虫的代理时,遇到连接超时的问题,应该怎么解决啊

import requests

# 代理ip请求测试网站
url = "http://myip.ipip.net/"
# 封装请求头
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'}

# 封装代理ip参数值:协议/ip/端口
proxy = {'http': 'http://157.245.64.113:8080'}
# 发送请求
response = requests.get(url=url, headers=headers, proxies=proxy, timeout=10)
print(response.status_code)  # 打印请求返回的状态码
# 获取响应数据
response_data = response.text
# 把请求之后的获取到的响应数据保存到本地文件夹,以查看代理请求是否成功!
with open('ip.html', 'w', encoding='utf-8') as adi:
    adi.write(response_data)
exit()  # 代码执行完毕退出程序

这是报的错误

Traceback (most recent call last):
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 1198, in getresponse
    response.begin()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "D:\Learn\Anaconda3\envs\python35\lib\http\client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "D:\Learn\Anaconda3\envs\python35\lib\socket.py", line 576, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\util\retry.py", line 531, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\packages\six.py", line 735, in reraise
    raise value
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\urllib3\connectionpool.py", line 337, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='157.245.64.113', port=8080): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/Learn/PyCharm/项目制作_1/火车票/test2.py", line 12, in <module>
    response = requests.get(url=url, headers=headers, proxies=proxy, timeout=10)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "D:\Learn\PyCharm\项目制作_1\venv\lib\site-packages\requests\adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='157.245.64.113', port=8080): Read timed out. (read timeout=10)


 

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

3条回答 默认 最新

  • funny123
    coagenth 2021-03-25 19:00
    已采纳

    换代理,或在requests.get时加长超时设置,timeout=20。

    点赞 评论
  • weixin_44385960
    木三136 2021-03-25 14:20

    这是提示超时

    点赞 评论
  • Eric_Liu_Xin
    澈丹丶 2021-03-25 15:27

    代理有问题,你本地可以正常访问代理吗

    点赞 评论

相关推荐