php memcached设置功能锁

Question

I use php memcached to implement token the code is below:

    function addTokenKey($token)
    {
       $allTokens = $this->memcache->get("AllTokens");


        if(gettype($allTokens) == "boolean")
        {
            $array = array();
            array_push($array,$token);

            $this->memcache->set("AllTokens",$array);

            echo "addTokenKey 1.2:".count($array)."<br>";
        }
        else{                
            echo "addTokenKey 2.1:".count($allTokens)."<br>";

            array_push($allTokens,$token);
            $this->memcache->set("AllTokens",$allTokens);   

            echo "addTokenKey 2.2:".count($allTokens)."<br>";
        }

     } 

I send mulitple request to call this function at the same time

but sometime I get the same result,Ex:

request result

addTokenKey 2.1:5

addTokenKey 2.2:6

another request result

addTokenKey 2.1:5

addTokenKey 2.2:6

How to avoid this case happen? lock or ..?

---

refer to:https://github.com/zerkalica/Semaphore

I use this library to try to do lock & release,the code is below:

    function addTokenKey($token)
    {
       $adapter   = new MemcachedAdapter($this->memcache);
       $semaphore = new SemaphoreManager($adapter); 


       $ttl = 60; // Time in seconds, used, if script dies and release never called.
       $handle = $semaphore->acquire('addTokenKey_lock_key', $ttl);

       $allTokens = $this->memcache->get("AllTokens");

        if($allTokens == false)
        {
            //array_push($allTokens,$token);


            $array = array();
            array_push($array,$token);

            $this->memcache->set("AllTokens",$array);

            echo "addTokenKey 1.2:".count($array)."<br>";
        }
        else{

            echo "addTokenKey 2.1:".count($allTokens)."<br>";

            array_push($allTokens,$token);
            $result = $this->memcache->set("AllTokens",$allTokens);     

            echo "addTokenKey 2.2:".count($allTokens)." ".$result."<br>";
        }
        $semaphore->release($handle);
     }

but I always got two error

Fatal error: Uncaught exception 'ErrorException' with message 'Can't acquire lock for millwright_semaphoreaddTokenKey_lock_key' in /xxxxxxx/Server/lib/Semaphore/SemaphoreManager.php on line 50

Fatal error: Uncaught exception 'LogicException' with message 'Call ::acquire('millwright_semaphoremillwright_semaphoreaddTokenKey_lock_key') first' in /xxxxxxx/Server/lib/Semaphore/SemaphoreManager.php on line 65

I already fix this error in SemaphoreManager.php by removing "$this->prefix ." code

but still have miss array count problem.

I modify some code below to try,

I send 100 request,finally allTokens number is only 50,

others will show "Unable to set"

    function addTokenKey($token)
    {            
        // initialize lock
        $lock = FALSE;
        // initialize configurable parameters
        $tries = 0;
        $max_tries = 1000;
        $lock_ttl = 10;

        $allTokens = $this->memcache->get("AllTokens");

        while($lock === FALSE && $tries < $max_tries ) {
            if( $allTokens == false ) {            
                $allTokens = array();
                array_push($allTokens,$token);
                $this->memcache->set("AllTokens",$allTokens);
                echo "addTokenKey 1.2:".count($allTokens)."<br>"; 
                return;
            } 
            $count = count($allTokens) ;
            // add() will return false if someone raced us for this lock
            // ALWAYS USE add() FOR CUSTOM LOCKS
            $lock = $this->memcache->add("lock_".$count, 1, $lock_ttl);
            $tries++;
            usleep(100*($tries%($max_tries/10))); // exponential backoff style of sleep        
        }
        if($lock === FALSE && $tries >= $max_tries) {    
            print("Unable to set");
        } else {        
            echo "addTokenKey 2.1:".count($allTokens)."<br>";
            array_push($allTokens,$token);
            $this->memcache->set("AllTokens",$allTokens);   
            echo "addTokenKey 2.2:".count($allTokens)."<br>";        
        }            
     }

---

finally I use memcached getAllKeys function to fix the problem,don't DIY to record allTokens but this function only can use in linux memcached, windows memcache don't support getAllKeys

Answer 1

In normal senario, this issue will not be visible but it will only be creating problem when there will be n number of concurrent requests. And that is because the memecache update is not atomic with its normal get/set. Always use memcached increment/decrement for insuring the atomicity for setting integer valued keys for senario where there will be concurrency in the requests. Since memcached increment() is atomic by itself, we need not put any locking mechanism. Yes , but for acheiveing the atomicity for any other race conditions , you will have to apply the custom locking etc to insure the atomicity for concurrent requests.

Try like below and check it:

$mem = new Memcache;
$mem->addServer("127.0.0.1", 11211);

function incrementUserVisits($userIdFromRequest) {
    global $mem;    
    $key = "visit_".$userIdFromRequest;
    $count = $mem->increment($key, 1);
    if( $count === FALSE ) {
        $count = $mem->add($key, 1, 0, 0);
        if($count === FALSE) {          
            $count = $mem->increment($key, 1);
            if($count === FALSE) {              
                return FALSE;
            }
            else {              
                return TRUE;
           }
       }
       else {           
           return TRUE;
       }
   }
   else {       
      return TRUE;
   }
}
incrementUserVisits($userIdFromRequest);

You can try the below code ( i have managed to combine/build after a bit of research) but i have not tested it even for a syntax error but feels that it will help you in achieving the custom lock to handle the race conditions.

$mem = new Memcache;
$mem->addServer("127.0.0.1", 11211);
function addTokenKey($token) {
    global $mem;

    // initialize lock
    $lock = FALSE;
    // initialize configurable parameters
    $tries = 0;
    $max_tries = 1000;
    $lock_ttl = 10;

    $allTokens = $mem->get("AllTokens");

    while($lock === FALSE && $tries < $max_tries ) {
        if( gettype($allTokens) == "boolean" ) {            
            $allTokens = array();
            array_push($allTokens,$token);
            $mem->set("AllTokens",$allTokens);
            echo "addTokenKey 1.2:".count($allTokens)."<br>";                        
        } 
        $count = count($allTokens) ;
        // add() will return false if someone raced us for this lock
        // ALWAYS USE add() FOR CUSTOM LOCKS
        $lock = $mem->add("lock_".$count, 1, 0, $lock_ttl);
        $tries++;
        usleep(100*($tries%($max_tries/10))); // exponential backoff style of sleep        
    }
    if($lock === FALSE && $tries >= $max_tries) {    
        print("Unable to set");
    } else {        
        echo "addTokenKey 2.1:".count($allTokens)."<br>";
        array_push($allTokens,$token);
        $mem->set("AllTokens",$allTokens, 0, 0);   
        echo "addTokenKey 2.2:".count($allTokens)."<br>";        
    }
}
addTokenKey('XXX111');

Apology for any error but i think you can play with it and can achieve what you are looking for.