如何通过MongoDB同步在两个不同服务器上运行的两个应用程序

Question

I am developing a web application in golang and use single MongoDB instance as a datastorage. I have code that should be executed exclusively. Since my web application runs on two different servers I can't use golang synchronization tools for that.

Idea is to use MongoDB for that by locking document, but I don't know whether it is possible and if it is, how to do that?

Answer 1

Note beforehand: Using Redis would be a better and more efficient choice for distributed locking.

But if you still want to use MongoDB for this, read on.

Some notes to the solutions below:

• All solutions below are safe and work even if you have multiple MongoDB servers (a shared cluster), because none of the solutions below rely on simple reads; and all writes (e.g. insert or update) go to the master instance.

• If a goroutine fails to obtain a lock, it may decide to sleep a little (e.g. 1 second), then retry obtaining the lock. There should be a max retry count before giving up.

---

Using the existence of a document as the lock

Simplest would be to rely on MongoDB not allowing 2 documents with the same ID to exist (in the same collection).

So to acquire a lock, simply insert a document into a designated collection (e.g. locks) with the lock ID. If insertion succeeds, you successfully acquired the lock. If insertion fails, you did not. To release the lock, simply delete (remove) the document.

Some things to note: you MUST release the lock, because if you fail to do so, all code that attempts to acquire this lock will never succeed. So releasing the lock should be done using a deferred function (defer). Unfortunately this won't ensure the release in case of some communication error (network failure).

To have guarantee about lock release, you may create an index that specifies document expiration, so the locks would be deleted automatically after some time, should any problems arise in the Go app while it holds the lock.

Example:

Lock documents are not inserted prior. But an index is required:

db.locks.createIndex({lockedAt: 1}, {expireAfterSeconds: 30})

Acquiring the lock:

sess := ... // Obtain a MongoDB session
c := sess.DB("").C("locks")

err := c.Insert(bson.M{
    "_id":      "l1",
    "lockedAt": time.Now(),
})
if err == nil {
    // LOCK OBTAINED! DO YOUR STUFF
}

Releasing the lock:

err := c.RemoveId("l1")

Pros: Simplest solution.

Cons: You can only specify the same timeout for all locks, harder to change it later (must drop and recreate the index).

Note that this last statement is not entirely true, because you are not forced to set the current time to the lockedAt field. E.g. if you set a timestamp pointing 5 seconds in the past, the lock will auto-expire after 25 seconds. If you set it 5 seconds to the future, the lock will expire after 35 seconds.

Also note that if a goroutine obtains a lock, and without any problems it needs to hold it for longer than 30 seconds, it could be done by updating the lockedAt field of the lock document. E.g. after 20 seconds if the goroutine does not encounter any problem but needs more time to finish its work holding the lock, it may update the lockedAt field to the current time preventing it to get auto-deleted (and thus giving green light to other goroutines waiting for that lock).

Using pre-created lock documents and update()

Another solution could be to have a collection with pre-created lock documents. Locks could have an ID (_id), and a state telling if it is locked or not (locked).

Creating a lock prior:

db.locks.insert({_id:"l1", locked:false})

To obtain a lock, use the Collection.Update() method, where in the selector you must filter by ID and the locked state, where state must be unlocked. And the update value should be a $set operation, setting the locked state to true.

err := c.Update(bson.M{
    "_id":    "l1",
    "locked": false,
}, bson.M{
    "$set": bson.M{"locked": true},
})
if err == nil {
    // LOCK OBTAINED! DO YOUR STUFF
}

How does this work? If multiple Go instances (or even multiple goroutines in the same Go app) try to obtain the lock, only one will succeed, because the selector for the rest will return mgo.ErrNotFound, because the one that prevails sets the locked field to true.

Once you did your stuff holding the lock, you must release the lock:

err := c.UpdateId("l1", bson.M{
    "$set": bson.M{"locked": false},
})

To have guarantee of the lock release, you may include a timestamp in the lock documents when it was locked. And when attempting to acquire a lock, the selector should also accept locks that are locked but are older than a given timeout (e.g. 30 seconds). In this case the update should also set the locked timestamp.

Example guaranteeing lock release with timeout:

The lock document:

db.locks.insert({_id:"l1", locked:false})

Acquiring the lock:

err := c.Update(bson.M{
    "_id": "l1",
    "$or": []interface{}{
        bson.M{"locked": false},
        bson.M{"lockedAt": bson.M{"$lt": time.Now().Add(-30 * time.Second)}},
    },
}, bson.M{
    "$set": bson.M{
        "locked":   true,
        "lockedAt": time.Now(),
    },
})
if err == nil {
    // LOCK OBTAINED! DO YOUR STUFF
}

Releasing the lock:

err := c.UpdateId("l1", bson.M{
    "$set": bson.M{ "locked": false},
})

Pros: You can use different timeout for different locks, or even for the same locks at different places (although this would be bad practice).

Cons: Slightly more complex.

Note that to "extend the lifetime" of a lock, the same technique can be used that is described above, that is, if the lock expiration is nearing and the goroutine needs more time, it may update the lockedAt field of the lock document.