Your approach is perfectly fine, in fact, it is the best way on AppEngine.
Querying subsequent entities by setting a start cursor will not give you duplicate results if a new record is inserted which would be the first for example.
Why? Because the cursor contains the key of the last returned entity encoded, and not the number of previously returned entities.
So if you set a cursor, the datastore will start listing and returning entities that come after the key encoded in the cursor. If a new entity is saved that comes after the cursor, then that entity will be returned when reached.
Also using the for
and append()
is the best way. You might optimize it a little by creating a big enough slice beforehand:
var is = make([]Item, 0, limit)
But note that I made it with 0
length and limit
capacity on purpose: there is no guarantee that there will be enough entities to fill the full slice.
Another optimization would be to allocate it to be limit
length:
var is = make([]Item, limit)
and when datastore.Done
is reached, reslice it if it is not filled fully, for example:
for idx := 0; ; idx++ {
var i Item
_, err := t.Next(&i)
if err == datastore.Done {
if idx < len(is) {
is = is[:idx] // Reslice as it is not filled fully
}
break
}
is[idx] = i
}
Batch operations
GetMulti
, PutMulti
and DeleteMulti
are batch versions of the Get
, Put
and Delete
functions. They take a []*Key
instead of a *Key
, and may return an appengine.MultiError
when encountering partial failure.
Batch operations are not a replacement or alternative to queries. GetMulti
for example requires you to already have all the keys prepared for which you want to get the complete entities. And as such, there is no sense of a cursor for these batch operations.
Batch operations return you all the requested information (or do all the requested operation). There is no sequence of entities or operations which would/could be terminated and continued later on.
Queries and batch operations are for different things. You shouldn't worry about query and cursor performance. They do quite good, and what's important, they (the Datastore) scale good. A cursor will not slow the execution of a query, a query with a cursor will run just as fast as a query without a cursor, and also previously returned entities do not affect query execution time: it doesn't matter if you run a query without a cursor or with a cursor which you acquired after getting a million entities (which is only possible with several iterations).