创建MGO TTL索引以有选择地删除文档

Question

I'm working with Golang and MongoDB. I have a collection which needs to keep a document which can be persistent or volatile. Hence, if it's set an expire date (as the example expireAt) the document is considered volatile and deleted otherwise it'll be kept in the collection unless it'll be manually deleted.

Reading this doc I've found an index that might work as I need it to.

Basically I need to replicate this kind of index in mgo:

db.log_events.createIndex( { "expireAt": 1 }, { expireAfterSeconds: 0 } )

db.log_events.insert( {
  "expireAt": new Date('July 22, 2013 14:00:00'),
  "logEvent": 2,
  "logMessage": "Success!"
} )

I've read (I'm searching back for the source of this information) that if the expireAt is not a valid date the deletion won't be trigger. Thus I think that all I need to do is to set the expireDate to a valid date when I need it, otherwise I'll leave it to the Go time.Time zero value.

This is my codebase

type Filter struct {
    Timestamp time.Time     `bson:"createdAt"`
    ExpireAt  time.Time     `bson:"expireAt"`
    Body      string        `bson:"body"`
}

// Create filter from data received via REST API.
var filter Filter
timestamp := time.Now()

if theUserAction == "share" { // This is action will set the document as volatile
    filter.ExpireAt = time.Now().Add(24 * time.Hour * 14)
}

filter.Timestamp = timestamp
filter.Body = "A BODY"

// Store filter in database
session, err := mdb.GetMgoSession() // This is a wrapping method that returns a valid mgo session
if err != nil {
    return NewErrorInternal("Error connecting to database", err)
}
defer session.Close()


// Get db with global data for legent
collection := session.DB(database).C(filtersCollection)

My question is: how can I set the index thus that it'll delete the document IF the expireAt key is valid? Reading the mgo documentation about Index Type it doesn't seems like there's a way to replicate the previously stated index, since the library only provides the ExpireAfter field..

Also, it's valid to assume that a zerovalue could be interpreted by mongodb as an invalid date?

From the docs it is January 1, year 1, 00:00:00.000000000 UTC which actually seems like a valid date..

What I've thought so far is doing something like this:

filtIdx := mgo.Index{
    Key:        []string{"expireAt"},
    Unique:     false,
    Background: true,
    Sparse:     false,
    ExpireAfter: 0,
}

Answer 1

How can I set the index thus that it'll delete the document IF the expireAt key is valid?

An example to set a TTL index using mgo.v2 is as below:

index := mgo.Index{
    Key:         []string{"expireAt"},
    ExpireAfter: time.Second * 120, 
}
err = coll.EnsureIndex(index)

Where the above example sets to 120 seconds of expiration. See also Expire Data from Collections by Setting TTL.

Is it still possible to make some documents to not expire at all? Since this is the behaviour I'm looking forward to obtain a collection where some documents do expire while other remain persistent

You can specify omitempty flag for ExpireAt struct field as below:

type Filter struct {
    Timestamp time.Time `bson:"createdAt"`
    Body      string    `bson:"body"`
    ExpireAt  time.Time `bson:"expireAt,omitempty"`
}

Which essentially only include the field if it's not set to a zero value. See more info mgo.v2 bson.Marshal

Now, for example you can insert two documents where one would expire and the other persists. Code example:

var foo Filter
foo.Timestamp = timestamp
foo.Body = "Will be deleted per TTL index"
foo.ExpireAt = time.Now()
collection.Insert(foo)

var bar Filter
bar.Timestamp = timestamp
bar.Body = "Persists until expireAt value is set"
collection.Insert(bar)

Later on, you can set the expireAt field with an Update(), as an example:

newValue := bson.M{"$set": bson.M{"expireAt": time.Now()}}
err = collection.Update(queryFilter, newValue)

Setting a valid time value for expireAt field, would make it qualify for the TTL index. i.e. no longer persists.

Depending on your use case, alternatively you may as well Remove() the document instead of updating and relying on TTL index.