爬虫在执行EntityUtils.toString时报socket的问题

Question

如题
在高并发前提下执行爬虫任务，在日志中有大量的SocketTimeoutException异常出现，定位是html = EntityUtils.toString(response.getEntity(), charset);这句报错，异常如下：
2018-01-12 10:00:21 ERROR com.prnasia.util.http.StringUtil.getEntityAccordChar(StringUtil.java:83) - 网页转换编码出现问题,出现问题的url是:http://www.hotelschools.cn/zhuanke/shanghai/149.html出现的错误原因是:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:198)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.Reader.read(Reader.java:140)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:268)
at com.prnasia.util.http.StringUtil.getEntityAccordChar(StringUtil.java:66)
at com.prnasia.spider.basic.BasicCrawl.getEntity(BasicCrawl.java:45)
at com.prnasia.spider.pageUrl.PageUrl.getPageUrlByXpath(PageUrl.java:45)
at com.prnasia.task.consumer.TaskConsumerThread.run(TaskConsumerThread.java:51)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
网页是可以正常访问的，但是程序运行时会有很多类似异常抛出，求各位大神答疑解惑。

Answer 1

爬虫高并发? 要么是你把人家的服务器给抓死了, 要么是触发防火墙规则人家把你给墙了, 不能这么干啊