PHP如何采集指定的数据(爬虫)？

Question

PHP初学者，现在遇到了瓶颈。
http://www.iwencai.com/unifiedwap/result?w=bipv,%E5%88%9B%E4%B8%9A%E6%9D%BF&querytype=&issugs
这个网址下的结果

它的接口是：
http://www.iwencai.com/unifiedwap/unified-wap/v2/result/get-robot-data?question=bipv%2C%E5%88%9B%E4%B8%9A%E6%9D%BF&perpage=100&page=1&secondary_intent=&log_info=%7B%22input_type%22%3A%22click%22%7D&source=Ths_iwencai_Xuangu&version=2.0&query_area=&block_list=&add_info=%7B%22urp%22%3A%7B%22scene%22%3A1%2C%22company%22%3A1%2C%22business%22%3A1%7D%2C%22contentType%22%3A%22json%22%7D
试了很多次都不成功，最后被JS跳转到了指定页面。
烦请大神帮忙指点，要带上源码。
在此叩谢了。

Answer 1

请求头中有个必须的参数：hexin-v，并且它有一定时效性，大概20分钟就会失效，这个需要继续研究。其他都是一些正常查询参数

$client = new \GuzzleHttp\Client([
            'headers' => [
                'hexin-v' => 'XXXXXXXNu8wdEBD5WjrHlUm13ehGqgF8i95lUA9SCWp_XXXXXXX',
                'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36'
            ]
        ]);

        $response = $client->post('http://www.iwencai.com/unifiedwap/unified-wap/v2/result/get-robot-data', [
            'form_params' => [
                'question' => 'bipv,创业板',
                'secondary_intent' => 'stock',
                'perpage' => 100,
                'page' => 1,
                'sort_key' => '最新涨跌幅',
                'sort_order' => 'desc',
                'token' => '0ac952a816257243739761551',
                'fund_class' => '',
                'show_indexes' => [
                    "最新价",
                    "最新涨跌幅",
                    "概念解析",
                    "概念资讯",
                    "所属概念数量",
                    "a股市值(不含限售股)",
                    "注册地址",
                    "经营范围",
                    "所属概念",
                    "上市板块"
                ],
                'source' => 'Ths_iwencai_Xuangu',
                'version' => '2.0',
                'query_area' => '',
                'block_list' => '',
                'add_info' => [
                    "urp" =>
                        [
                            [
                                "scene" => 1,
                                "company" => 1,
                                "business" => 1
                            ],
                            "contentType" => "json"
                        ],
                ]
            ]
        ]);

        $content = $response->getBody()->getContents();
//        var_dump($content);
        $array = json_decode($content, true, 512, JSON_UNESCAPED_UNICODE);
//        var_dump($array);
        var_dump($array['data']['answer'][0]['txt'][0]['content']['components'][0]['data']['datas']);
         /*
          Result : array(14) {
          [0]=>
          array(14) {
         ...
            ["code"]=>
            string(6) "300040"
            ["最新价"]=>
            string(5) "11.30"
            ["注册地址"]=>
            string(56) "黑龙江省哈尔滨市松北区九洲路609号1#厂房"
            ["最新涨跌幅"]=>
            string(6) "19.958"
            ["market_code"]=>
            string(2) "33"
            ["股票代码"]=>
            string(9) "300040.SZ"
            ...
         */