Jsoup写的一个简单的爬虫出现502

Question

我写了一个爬虫，用线程池执行，报502的错误

如何解决？

for (int i = 1; i <= 402; i++) {
                int finalI = i;
                threadPoolExecutor.execute(() -> {
                    try {
                        DownloadSource.download("https://xxxxxxx/page/"+ finalI);
                        System.out.println("--------------------------------第"+ finalI +"页 加载完毕------------------------------------------");
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                });
            }

报错：

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=502, URL=https://xxxxx
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:762)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:707)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:297)
    at org.jsoup.helper.HttpConnection.get(HttpConnection.java:286)
    at com.DownloadSource.download(DownloadSource.java:25)
    at com.DownloadSource.lambda$main$0(DownloadSource.java:53)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Answer 1

• 这篇博客也许可以解决你的问题👉 ：Jsoup简单理解笔记