使用Go处理Google数据存储中的架构更改？

Question

I added a new attribute to Go struct, persisted in a datastore entity kind.

I tried to load entities using a filter on the new attribute:

q := datastore.NewQuery("Person").Filter("Employed =", false)

This only worked as intended for persons created after the new attribute was added.

I had expected persons created before the attribute was added to be incleded in the filter, but they didnt have the attribute at all, so they were be excluded by the filter.

I have thought of two ways to handle this:

• First load all entities and filter in a second step with a loop. This adds complexity to the code.
• Batch load and resave all entities, which adds the property and sets it to false. I have to remember to do this every time I add a new attribute.

Is there a batter way to handle these types of entity schema changes?

Answer 1

The proper way is to re-save old entities if you can afford it.

You can use Python map/reduce library to update old entities.

If you want to stick to Go you can go with simple batch processing using task queues & query cursors.

In our Python app we prefer "in-flight hot update" approach whenever it is possible - new version loads entities and check for missing properties. If some props are obsolete we would keep them for a month or to and then add logic to remove them. This allows to migrate gradually and often rollback without issues (shit happens) or at least limit impact to small % of records. Also it's more cost effective as you do not pay for additional reads/writes. Once we sure we ok we can run a mapreduce. This does not involve downtime. You may need or don't need downtime - it's really depend on your change.

With Go it may be a bit tricky as Go would panic if you try to load entity that has no corresponding fields on a struct. There should be some workarounds about it (I remember something about special interface or a struct where you can load arbitrary entity) but I have no much practical experience with Go on GAE yet.