Java 无法爬取TB商品

Question

Java 爬取指定店铺的所有商品信息只能获取外层div，商品列表应该是动态生成的，抓取不到。哪位谁可以帮帮忙

只能到取到选中的div内容取不到

Answer 1

淘宝现在抓商品可不好抓哦，很多反爬虫措施。我用python试过。
至于java，我觉得你应该直接用淘宝页面的url，然后分析页面。

public static void main(String[] args) throws Exception {
 String html = sendGetRequest(url);
}
/***
     * get请求
     * @param url
     * @return
     * @throws IOException
     */
    private static String sendGetRequest(String url) throws IOException {
        URL obj = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) obj.openConnection();

        // 设置请求方法为GET
        connection.setRequestMethod("GET");

        // 获取响应码
        int responseCode = connection.getResponseCode();
        System.out.println("Response Code: " + responseCode);

        // 读取响应内容
        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String inputLine;
        StringBuilder response = new StringBuilder();

        while ((inputLine = in.readLine()) != null) {
            response.append(inputLine);
        }
        in.close();

        return response.toString();
    }