如何更改json对象数组的字段名称

Question

So I have a project with lots of incoming data about 15 sources in total, of course there are inconsistencies in how each label there data available in their rest api's. I need to Change some of their field names to be consistent with the others, but I am at a loss on how to do this when the data sources are json object arrays. A working example of what I am trying to do is found here playground and below

however I seem to lack the knowledge as to how to make this work when the data is not a single json object , but instead and array of objects that I am unmarshaling.

Another approach is using Maps like in this example but the result is the same, works great as is for single objects, but I can not seem to get it to work with json object arrays. Iteration through arrays is not a possibility as I am collecting about 8,000 records every few minutes.

package main

import (
    "encoding/json"
    "os"
)

type omit bool

type Value interface{}

type CacheItem struct {
    Key    string `json:"key"`
    MaxAge int    `json:"cacheAge"`
    Value  Value  `json:"cacheValue"`
}

func NewCacheItem() (*CacheItem, error) {
    i := &CacheItem{}
    return i, json.Unmarshal([]byte(`{
      "key": "foo",
      "cacheAge": 1234,
      "cacheValue": {
        "nested": true
      }
    }`), i)
}

func main() {
    item, _ := NewCacheItem()

    json.NewEncoder(os.Stdout).Encode(struct {
        *CacheItem

        // Omit bad keys
        OmitMaxAge omit `json:"cacheAge,omitempty"`
        OmitValue  omit `json:"cacheValue,omitempty"`

        // Add nice keys
        MaxAge int    `json:"max_age"`
        Value  *Value `json:"value"`
    }{
        CacheItem: item,

        // Set the int by value:
        MaxAge: item.MaxAge,

        // Set the nested struct by reference, avoid making a copy:
        Value: &item.Value,
    })
}

Answer 1

It appears your desired output is JSON. You can accomplish the conversion by unmarshaling into a slice of structs, and then iterating through each of those to convert them to the second struct type (your anonymous struct above), append them into a slice and then marshal the slice back to JSON:

package main

import (
    "fmt"
    "encoding/json"
)

type omit bool

type Value interface{}

type CacheItem struct {
    Key    string `json:"key"`
    MaxAge int    `json:"cacheAge"`
    Value  Value  `json:"cacheValue"`
}

type OutGoing struct {
    // Omit bad keys
    OmitMaxAge omit `json:"cacheAge,omitempty"`
    OmitValue  omit `json:"cacheValue,omitempty"`

    // Add nice keys
    Key    string `json:"key"`
    MaxAge int    `json:"max_age"`
    Value  *Value `json:"value"`
}

func main() {
    objects := make([]CacheItem, 0)
    sample := []byte(`[
    {
      "key": "foo",
      "cacheAge": 1234,
      "cacheValue": {
        "nested": true
      }},
    {
      "key": "baz",
      "cacheAge": 123,
      "cacheValue": {
        "nested": true
    }}]`)

    json.Unmarshal(sample, &objects)

    out := make([]OutGoing, 0, len(objects))
    for _, o := range objects {
        out = append(out, OutGoing{Key:o.Key, MaxAge:o.MaxAge, Value:&o.Value})
    }
    s, _ := json.Marshal(out)
    fmt.Println(string(s))
}

This outputs

[{"key":"foo","max_age":1234,"value":{"nested":true}},{"key":"baz","max_age":123,"value":{"nested":true}}]

You could probably skip this iteration and conversion code if you wrote custom MarshalJSON and UnmarshalJSON methods for your CacheItem type, instead of relying on struct field tags. Then you could pass the same slice to both Unmarshal and Marshal.

To me there's no obvious performance mistake with these approaches -- contrast with building a string in a loop using the + operator -- and when that's the case it's often best to just get the software to work and then test for performance rather than ruling out a solution based on fears of performance issues without actually testing.

If there's a performance problem with the above approaches, and you really want to avoid marshal and unmarshal completely, you could look into byte replacement in the JSON data (e.g. regexp). I'm not recommending this approach, but if your changes are very simple and the inputs are very consistent it could work, and it would give another approach you could performance test, and then you could compare performance test results.