循环多卷曲请求PHP的最佳方法

Question

Hi guys I am making a tool which will fetch telephone numbers from the database and make a curl/api request to validate the number and get it's info then update certain fields in the database based from the API response.

So I have a table called phones which has

->id
->number
->network
->country

So in my table only the id and number which has values, network and country is null. Which is the reason I will use API that will update those fields depending on the number. However there's a problem, so basically what will happen is I will loop on all those numbers like so:

$phone = Phone::all();
foreach ($phone as $key => $value) 
{
  // Call the API that will get the details in the current number
  // and update the details in the table for the current number 

  /** Some code for API Call **/

  //Update Script
  $update = Phone::find($value->id);
  $update->network = $network;
  $update->country = $country;
  $update->country_prefix = $country_prefix;
  $update->status = $status;
  $update->remarks = $remarks;
  $update->save();
}

That will work fine and do my task, but problem is this is very slow when I looped in let's say, 50,000 records, coz before it can send the next curl request, it must wait for the response of the previous one right? Question is how can I make it a 20 request per loop count? Coz the API I'm using supports 20 request per second so I wan't to maximize it.

I know my loop will change coz I need to get 20 records at a time and not repeating the same records again.

Answer 1

This is fairly easy with a library like GuzzleHttp. I don't know what your object structure or validation routine looks like, but this example should get you started:

use GuzzleHttp\Client;
use GuzzleHttp\Promise;

$client = new Client(['base_uri' => 'http://example.com/api/']);

$updates = [];
$phone = Phone::all();
foreach ($phone as $key => $value)
{

    $update = Phone::find($value->id);
    $update->network = $network;
    $update->country = $country;
    $update->country_prefix = $country_prefix;
    $update->status = $status;
    $update->remarks = $remarks;

    // Load updates into array for processing
    $updates[$update->id] = $update;

    if (count($updates) == 20) {

        // Setting up the async requests
        $promises = [];

        foreach ($updates as $u) {
            // This is posting to http://example.com/api/phone, appending to the 'base_uri' above.
            // This will send a json body, but you can change the format as necessary
            $promises[$u->id] = $client->postAsync('/phone', ['json' => ['phone' => $u->number]]);
        }

        // Waits for the requests to complete
        $results = Promise\unwrap($promises);

        // Saves each number with a 200 response
        foreach ($results as $id => $result) {
            if ($result->getStatusCode() == 200) {
                $updates[$id]->save();
            }
        }

        // Clear processed records from array
        $updates = [];
    }
}

You can read through the documentation for more details.

You can also do this with curl_multi_*, but the implementation is much more complex.