2 zengkunling zengkunling 于 2016.03.27 17:49 提问

请教python 爬虫 ip代理 的问题

我在做一个从淘宝上抓照片的爬虫,然后在应对重定向问题的时候用ip代理,使用代理部分的代码如下,ip代理是从西刺代理上抓下来并检验的,高匿的,但还是没能解决这个问题。请问下这是怎么个情况,该如何解决。如果说这样用代理是没有错的话,那是怎么被反爬虫了?

```proxies = [
{'http':'222.83.14.145:3128'},
{'http':'211.144.76.58:9000'},
{'http':'101.200.138.85:3128'},
{'http':'114.40.79.27:8080'},
{'http':'119.29.53.211:80'},
{'http':'58.49.144.208:8090'},
{'http':'42.96.197.187:80'},
{'http':'119.29.171.208:80'},
{'http':'124.160.194.71:80'},
{'http':'27.10.109.113:8118'},
{'http':'58.23.182.211:8118'},
{'http':'119.188.94.145:80'},
{'http':'113.243.132.120:8080'},
{'http':'122.96.59.107:843'},
{'http':'42.233.18.143:8118'},
{'http':'122.237.107.30:80'},
{'http':'117.69.6.181:8118'},
{'http':'118.113.7.48:8118'},
{'http':"222.83.14.145:3128"},
{'http':"115.160.137.178:8088"}
]
IPNum = ipcount%len(proxies)
print IPNum
print proxies[IPNum]
html = requests.get(each,headers = header,proxies = proxies[IPNum])


3个回答

oyljerry
oyljerry   Ds   Rxr 2016.03.27 18:18

只能说这些代理也被加到人家的爬虫黑名单了 所以还是被识别出爬虫了

zengkunling
zengkunling 额。。。那请问如何应对呢?
一年多之前 回复
CSDNXIAOD
CSDNXIAOD   2016.03.30 11:27

爬虫 代理问题
python IP代理爬虫,download 代理IP
----------------------biu~biu~biu~~~在下问答机器人小D,这是我依靠自己的聪明才智给出的答案,如果不正确,你来咬我啊!

qq_25970551
qq_25970551   2016.05.16 15:34

使用收费的代理ip?我记得有一个专门讲爬虫的教程有专门讲过代理ip等反反爬虫的,你可以看看学习学习:http://blog.csdn.net/youmumzcs/article/details/51396283

Csdn user default icon
上传中...
上传图片
插入图片
准确详细的回答,更有利于被提问者采纳,从而获得C币。复制、灌水、广告等回答会被删除,是时候展现真正的技术了!