使用golang从MongoDB批量获取记录

Question

How can one fetch records in batches from a mongo database using golang? I know mongoDB itself has something called cursor.batchSize(), but im trying to find an example using a golang driver. From what I've seen the golang library mgo has a function called Batch, but Im trying to figure out how to us it.

Ideally I'm looking for something like this:

const cursor = useDb.collection(mycollection).find().batchSize(10000);
for (let doc = await cursor.next(); doc != null; doc = await cursor.next()) {
    if (doc._id % divisor === 0) {
        counter++;
    }
}

I have something like this so far:

err := c.Find(nil).Batch(1).All(&items)

But it's not working like expected.

Links: https://docs.mongodb.com/manual/reference/method/cursor.batchSize/ https://godoc.org/github.com/globalsign/mgo#Query.Batch

Answer 1

The batchSize() specifies the number of documents to return in each batch of the response. You will still all results. For example, if the total number of returned docs is 100 and you specify batch size as 20, you will have 1 find and 4 getMore commands sent from the client to the mongod to complete the request.

One little trick you can use to verify. Use db.setProfilingLevel(0, 0) from mongo shell to log everything in the log file. Execute your test app and you will see find and getMore recorded in the log.