当某些字段为只读字段而另一些字段为可空值时，如何使用Golang结构在API中执行CRUD？

Question

I am writing an API that is used to perform basic CRUD operations (struct <=> mysql table basically). Here is an example struct that maps to a table in my database. I use pointers for the fields so that I can treat nil as NULL/absent:

type Foo struct {
  Id           *int32
  Name         *string
  Description  *string
  CreateDate   *string
}

The Id field is an autoincrement field that should be assigned by the database. The Name field is writable and required. The Description field is writable and nullable. The CreateDate field is assigned by MySQL on insert and should not be written to.

When the user POSTs a new Foo to create, the request body looks like this in JSON:

POST /Foo
{"Name": "test name", "Description": "test description"}

It's easy to decode this and hydrate a Foo struct using a

foo := Foo{}
json.NewDecoder(requestBody).Decode(&foo)

I'm using the https://github.com/coopernurse/gorp library to simplify inserts/updates/deletes, but my issue still holds even if I'm writing raw sql if I wish to generalize the query creation using reflection on fields.

gorpDbMap.Insert(&foo)

My first problem arises if the user provides an read-only field. If this request body is POSTed, the struct happily accepts Id and when I do the insert it overrides the autoincrement value. I know this is somewhat my fault for using an ORM rather than manually writing a SQL insert but my hope was that I could in some way enforce when hydrating the struct that only those writable fields should be decoded and any others ignored (or causing an error):

POST /Foo
{"Id": 1, "Name": "test name"}

I cannot find a simple way other than manually examining the hydrated struct and unsetting any read-only fields that I didn't want the user to provide.

The second problem I am experiencing is determining when a user is unsetting a value (passing NULL for a field to update) vs when the value was not provided. This is a partial update/PATCH in RESTful terminology.

For example, suppose a Foo with id=1 exists. The user now wishes to update the Name from test name to new name and the Description from test description to NULL.

PATCH /Foo/1
{"Name": "new name", "Description": NULL}

Since my struct uses pointers for its fields I can determine if the Description should be set to null on create if foo.Description == nil. But in this partial update, how can I differentiate between the case where Description was not provided (and should thus be left as-is) and the case above where the caller wishes to set the value of Description to NULL?

I know there are ways to solve this by writing a lot of custom code around each struct I define, but I was hoping for a general solution that doesn't require so much boilerplate. I've also seen solutions that adopt a different body format for PATCH requests, but I have to meet the existing contract so I cannot adopt a different format for partial updates.

I'm considering a couple options but neither satisfy me.

1. Use interface-typed maps and write code to examine each field and assert types as necessary. This way I can determine if a field and NULL vs not provided at all. Seems like a lot of work.

2. Define multiple structs for each scenario. This feels a little cleaner, but also a little unnecessarily verbose. And it only resolves one of the two problems I have (enforcing read-only) but not determining when a NULLable field is actually nulled out on partial update or just not provided.

e.g.

type FooWrite struct {
  Name        *string
  Description *string
}

type FooRead struct {
  FooWrite
  Id         int32
  CreateDate string
}

This article addresses part of the issue and got me this far, but doesn't address the two problems I'm having now: https://willnorris.com/2014/05/go-rest-apis-and-pointers

Most suggestions I've seen revolve around changing the design of my schema and avoiding NULLs, but I do not have the ability to modify that as it is already in use by other consumers.

Answer 1

One option here would be to use a custom type that special cases JSON marshalling. For instance, if you want an integer that is read only in JSON, you could do something like this:

type JsonReadOnlyInt int32

func (i JsonReadOnlyInt) MarshalJSON() ([]byte, error) {
    return json.Marshal(int32(i))
}

func (i *JsonReadOnlyInt) UnmarshalJSON([]byte) error {
    return nil // ignore attempts to set
}

If you use this type in one of your structures, the integer will be able to be marshalled to JSON but will be ignored in the reverse direction: http://play.golang.org/p/lW7xuXR6y0

It will require a bit more work to make this type work with GORP though. It looks like that package uses the standard library database conversion interfaces, so you would need to implement Scanner from database/sql and Valuer from database/sql/driver. Something like this:

func (i *JsonReadOnlyInt) Scan(value interface{}) error {
    // And maybe also cases for string/[]byte, depending on the driver
    v, ok := value.(int64)
    if !ok {
        return errors.New("Could not scan")
    }
    *i = JsonReadOnlyInt(v)
    return nil
}

func (i JsonReadOnlyInt) Value() (driver.Value, error) {
    return int64(i), nil
}

Now you should be able to round trip values of this type to the database.

As far as the patch question goes, there are two options you could try:

1. Just decode into a struct holding the old values for the record. Any fields missing from the JSON object will not be updated, and your read only fields can be protected using the above strategy.

2. Use a custom struct type to represent your field rather than a simple integer like above. Make its zero value correspond to unset, and make its UnmarshalJSON method set a flag to say that it has been set.

Which one is more appropriate will probably depend on the rest of your code.