- 爬取的过程之中连续多次返回以下结果,困扰了许多天,百度了许多,都为找到合理的解决方案,感谢大神解惑
<html>
<head>
<script>window.location.href='https://www.qichacha.com/index_verify?type=companysearch&back=/search?key=%E7%A6%8F%E5%BB%BA%E9%9B%8D%E6%A0%BC%E5%BB%BA%E8%AE%BE%E5%B7%A5%E7%A8%8B%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8';</script>
</head>
<body></body>
</html>
-
爬虫使用的是HttpURLConnection 和蘑菇代理,访问的页面
URL url = new URL(targetUrl); Authenticator.setDefault(new ProxyAuthenticator(proxyUser, proxyPass)); // 创建代理服务器地址对象 InetSocketAddress addr = new InetSocketAddress(proxyServer, proxyPort); // 创建HTTP类型代理对象 Proxy proxy = new Proxy(Proxy.Type.HTTP, addr); // 设置通过代理访问目标页面 HttpURLConnection connection = (HttpURLConnection) url.openConnection(proxy);
