阿肆si 2021-05-10 01:54 采纳率: 66.7%
浏览 348
已采纳

scrapy如何手动停止爬虫?

没用代理ip前我都是ctrl+c停下爬虫,用了代理,ctrl+c停不下来,下面是ctrl+c停止后的信息

2021-05-10 09:46:03 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.b2b168.com/c168-19142095.html> (failed 1 times): [<twisted.pyth
on.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
2021-05-10 09:46:03 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.b2b168.com/c168-19146912.html> (failed 1 times): [<twisted.pyth
on.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
2021-05-10 09:46:03 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.b2b168.com/c168-19185028.html> (failed 1 times): An error occur
red while connecting: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other sid
e was lost in a non-clean fashion: Connection lost.

2021-05-10 09:46:03 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.b2b168.com/c168-19146450.html> (failed 1 times): [<twisted.pyth
on.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
2021-05-10 09:46:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): dps.kdlapi.com:443
2021-05-10 09:46:54 [urllib3.connectionpool] DEBUG: https://dps.kdlapi.com:443 "GET /api/getdps/?orderid=962036615546243&num=1&pt=1&format=json&sep=1 HTT
P/1.1" 200 None
2021-05-10 09:47:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): dps.kdlapi.com:443
2021-05-10 09:47:54 [urllib3.connectionpool] DEBUG: https://dps.kdlapi.com:443 "GET /api/getdps/?orderid=962036615546243&num=1&pt=1&format=json&sep=1 HTT
P/1.1" 200 None
2021-05-10 09:48:54 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): dps.kdlapi.com:443
2021-05-10 09:48:55 [urllib3.connectionpool] DEBUG: https://dps.kdlapi.com:443 "GET /api/getdps/?orderid=962036615546243&num=1&pt=1&format=json&sep=1 HTT
P/1.1" 200 None
2021-05-10 09:49:55 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): dps.kdlapi.com:443
2021-05-10 09:49:55 [urllib3.connectionpool] DEBUG: https://dps.kdlapi.com:443 "GET /api/getdps/?orderid=962036615546243&num=1&pt=1&format=json&sep=1 HTT
P/1.1" 200 None
  • 写回答

1条回答 默认 最新

  • 考古学家lx(李玺) python领域优质创作者 2021-05-10 06:36
    关注

    Ctrl+C 只是终止主线程,你的其他线程没有守护,所以 Ctrl+C 它们继续运行。

     

    另外scrapy中的 Ctrl+C 是暂停,并不是完全停止,Ctrl+C 是断点续爬的基础。

     

     

    <font face="-apple-system, SF UI Text, Arial, PingFang SC, Hiragino Sans GB, Microsoft YaHei, WenQuanYi Micro Hei, sans-serif, SimHei, SimSun">可以 </font>Ctrl+C 然后<font face="-apple-system, SF UI Text, Arial, PingFang SC, Hiragino Sans GB, Microsoft YaHei, WenQuanYi Micro Hei, sans-serif, SimHei, SimSun"> </font>Ctrl+ZCtrl+C

     

     

     

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
编辑
预览

报告相同问题?

悬赏问题

  • ¥15 全志t113i启动qt应用程序提示internal error
  • ¥15 ensp可以看看嘛.
  • ¥80 51单片机C语言代码解决单片机为AT89C52是清翔单片机
  • ¥60 优博讯DT50高通安卓11系统刷完机自动进去fastboot模式
  • ¥15 minist数字识别
  • ¥15 在安装gym库的pygame时遇到问题,不知道如何解决
  • ¥20 uniapp中的webview 使用的是本地的vue页面,在模拟器上显示无法打开
  • ¥15 网上下载的3DMAX模型,不显示贴图怎么办
  • ¥15 关于#stm32#的问题:寻找一块开发版,作为智能化割草机的控制模块和树莓派主板相连,要求:最低可控制 3 个电机(两个驱动电机,1 个割草电机),其次可以与树莓派主板相连电机照片如下:
  • ¥15 潜在扩散模型的Unet特征提取
手机看
程序员都在用的中文IT技术交流社区

程序员都在用的中文IT技术交流社区

专业的中文 IT 技术社区,与千万技术人共成长

专业的中文 IT 技术社区,与千万技术人共成长

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

客服 返回
顶部