dongpa5277 2016-01-12 13:21
浏览 43
已采纳

Curl Web Scraper问题,错误的数组

i have Problem with my Scraping Script. I like to select the value from my sql by id in a while and save the the scraped content in my database. its works but the curl script send a wrong result to my database, everytime the content from the first request.

My Code:

<?php
error_reporting(E_ALL);
include('db.php');

$i = 1;

while ($i <= 5)
{

$sql = "SELECT * FROM `plz` WHERE `id` = '$i'";
    $row = mysql_fetch_assoc( mysql_query($sql) );

$plz = $row['plz'];


//create array of data to be posted
$post_data['adv_plz'] = "$plz";
$post_data['finda'] = 'adv';
$post_data['lang'] = 'de_DE';

//traverse array and prepare data for posting (key1=value1)
foreach ( $post_data as $key => $value) {
    $post_items[] = $key . '=' . $value;
}

//create the final string to be posted using implode()
$post_string = implode ('&', $post_items);

//create cURL connection
$curl_connection = curl_init('https://www.domain.de/');

//set options
curl_setopt($curl_connection, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($curl_connection, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($curl_connection, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_connection, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl_connection, CURLOPT_FOLLOWLOCATION, 1);

//set data to be posted
curl_setopt($curl_connection, CURLOPT_POSTFIELDS, $post_string);

//perform our request
$result = curl_exec($curl_connection);

//show information regarding the request
$result = utf8_encode($result);

mysql_query("UPDATE plz SET content = '$result' WHERE id = '$i'");


    $i++;
}


?>

Here is the Content of the echo $post_string

adv_plz=01000&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE&adv_plz=01003&finda=adv&lang=de_DE

adv_plz=01000&finda=adv&lang=de_DE&adv_plz=01001&finda=adv&lang=de_DE&adv_plz=01002&finda=adv&lang=de_DE&adv_plz=01003&finda=adv&lang=de_DE&adv_plz=01004&finda=adv&lang=de_DE

I hope somebody can help me. If i try the script not in a while it works perfectly.

</div>
  • 写回答

1条回答 默认 最新

  • dongzhen7108 2016-01-12 13:45
    关注

    Try resetting the $post_items variable before entering the foreach loop.

    $post_items = array();
    
    //traverse array and prepare data for posting (key1=value1)
    foreach ( $post_d ....
    
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 使用ue5插件narrative时如何切换关卡也保存叙事任务记录
  • ¥20 软件测试决策法疑问求解答
  • ¥15 win11 23H2删除推荐的项目,支持注册表等
  • ¥15 matlab 用yalmip搭建模型,cplex求解,线性化处理的方法
  • ¥15 qt6.6.3 基于百度云的语音识别 不会改
  • ¥15 关于#目标检测#的问题:大概就是类似后台自动检测某下架商品的库存,在他监测到该商品上架并且可以购买的瞬间点击立即购买下单
  • ¥15 神经网络怎么把隐含层变量融合到损失函数中?
  • ¥15 lingo18勾选global solver求解使用的算法
  • ¥15 全部备份安卓app数据包括密码,可以复制到另一手机上运行
  • ¥20 测距传感器数据手册i2c