doujiu9307 2014-12-22 06:44
浏览 10
已采纳

如何使用ajax一次刮一页，返回下一页链接然后再去

Question:

I have a php scraping function and code that all works well, however it times out because its trying to load 60 different pages...

I was thinking of using AJAX to load one page at a time in a loop. Since i'm very new to AJAX im having some trouble.

This is what I have so far, I can get it to loop through the links if I provide them, however I want it to scrape page 1, return the next page link and then scrape the next page on a continuous loop until there are no more pages. As it stands it goes into infinite loop mode...

Any ideas guys?

Here is my code which i took from a youtube video which was using an array (i am only passing through a string)

<?php
ini_set('display_errors',1);
//error_reporting(E_ALL);
set_time_limit(0);

require_once 'scrape_intrepid.php';

//posted to this page
if(isset($_POST['id'])) {

    //get the id
    $id = $_POST['id'];

    //this returns the next page link successfully, i just cant get it back into the function
    $ids = scrapeSite($id);
    echo $ids;
    echo "<br>";
    $data = $id . " - DONE";
    echo json_encode($data);

    exit();

} else {

    $ids = 'http://www.intrepidtravel.com/search/trip?page=1';
}
?>
<html>
    <head>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
    <script>
        $(function() {

            function update() {
                ids = <?=json_encode($ids);?>;
                if(ids){
                    var id = ids;
                    $.post("index.php",{id:id}).done(function(msg){
                        console.log(ids,msg);
                        update();
                    });
                } else {
                    console.log("done");
                    $("#log").html("Completed!");
                }
            }

            $("#go").click(function() {
                $("#go").html("Loading...");
                update();
            });
        });

    </script>
</head>
<body>
    <button id="go">Go button</button>
    <div id="log">Results</div>
</body>

  • 写回答

1条回答 默认 最新

  • doujiang1832 2014-12-22 10:23
    关注

    Ended up solving this in another way: The function I am calling to function.php runs the script and returns the next URL to scrape. which is the msg value, so the refresh is called again once this is validated. Just processed 60 pages each taking 38 seconds each :S

    <script>
$(document).ready(function() {

    refresh('http://www.intrepidtravel.com/search/trip?');

    function refresh(url) {
        $.ajax({
            type: "GET",
            url: "function.php",
            data: 'url=' + url,
            success: function(msg){
                $('#result').append('--->Completed! <br>Next Page: is ' + msg);
                console.log(msg);
                if ($.trim(msg) == 'lastpage'){
                    $('#result').append('--->Last page - DONE!');
                }
                else {
                    refresh(msg);
                }
            }

        }); // Ajax Call
    } //refresh

}); //document.ready
</script>

    And the function.php file:

    require_once 'scrape_intrepid.php';

    if ($_GET['url']){
    $url = $_GET['url'];
    if ($url=="lastpage"){
        echo $url;
    } else {
    $nextlink = scrapeSite($url);
    echo($nextlink);
    }
}
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题？

悬赏问题

  • ¥15 安卓adb backup备份应用数据失败
  • ¥15 eclipse运行项目时遇到的问题
  • ¥15 关于#c##的问题：最近需要用CAT工具Trados进行一些开发
  • ¥15 南大pa1 小游戏没有界面，并且报了如下错误，尝试过换显卡驱动，但是好像不行
  • ¥15 没有证书，nginx怎么反向代理到只能接受https的公网网站
  • ¥50 成都蓉城足球俱乐部小程序抢票
  • ¥15 yolov7训练自己的数据集
  • ¥15 esp8266与51单片机连接问题(标签-单片机|关键词-串口)（相关搜索：51单片机|单片机|测试代码）
  • ¥15 电力市场出清matlab yalmip kkt 双层优化问题
  • ¥30 ros小车路径规划实现不了，如何解决？(操作系统-ubuntu)