使用Golang将两片相似的数据合并到JSON中

Question

I'm a Go newbie, so go easy on me. I'm coming from Rubyland and struggling to understand some of this new terrain.

Context: We have a service with two endpoints that hit multiple API's, massage the data and send it back as json. They're serving data on the same objects, but they do distinctly different things. The only common attribute is uid.

They're sharing a model, which looks like this:

type Item struct {
  Uid     string      `json:"uid"`
  Val1    string      `json:"param1,omitempty"`
  Val2    []NestedVal `json:"param2,omitempty"`
  Val5    int         `json:"param5,omitempty"`
  Val6    string      `json:"param6,omitempty"`
  Val7    string      `json:"param7,omitempty"`
}

type NestedVal struct {
  Val3  int    `json:"val3,omitempty"`
  Val4  string `json:"val4,omitempty"`
}

An example response from endpoint A would be:

[
 {
   "uid": "123",
   "val1": "foobar"
   "val2": [
     {"val3": 666, "val4": "qux"},
     {"val3": 666, "val4": "qux"}
   ]
 },
 {
   "uid": "456",
   "val1": "foobar"
   "val2": [
     {"val3": 666, "val4": "qux"},
     {"val3": 666, "val4": "qux"}
   ]
 }
]

An example response from endpoint B would be:

[
 {
   "uid": "123",
   "val5": 999
   "val6": "bar"
   "val7": "baz"
 },
 {
   "uid": "456",
   "val5": 999
   "val6": "bar"
   "val7": "baz"
 }
]

Right now these exist as stand-alone services because they serve slightly different needs, but now we need to offer them up as a combined dataset, something like:

[
 {
   "uid": "123",
   "val1": "foobar"
   "val2": [
     {"val3": 666, "val4": "qux"},
     {"val3": 666, "val4": "qux"}
   ],
   "val5": 999
   "val6": "bar"
   "val7": "baz"
 },
 {
   "id": "456",
   "val1": "foobar"
   "val2": [
     {"val3": 666, "val4": "qux"},
     {"val3": 666, "val4": "qux"}
   ],
   "val5": 999
   "val6": "bar"
   "val7": "baz"
 }
]

For what it's worth, this is how they're returned (basically the same for both endpoints):

func FetchA(uids []string) []Item {
  var collection []Item

  *... fetching, parsing, and appending to collection ...*

  return collection
}

func HandlerA(response http.ResponseWriter, request *http.Request) {
  *... pull uids out of qs ...*
  collection := fetchA(uids)
  responseData, _ := json.Marshal(collection)
  *... write the response ...*
}

Bottom line: I have two data sets, both of type []Item. I just want to combine them on their shared uid keys. Obviously I'm approaching this as a Ruby dev, thinking about this problem as "I have two arrays of hashes and i want to merge them on a shared key" - I know thats the wrong frame of mind here, but I'm pretty stuck and I know the answer is right under my nose.

I've looked around and none of the answers I've seen have been particularly helpful. I checked out mergeo and go-merge but couldn't find a way around overwriting the merge-ee.

I think my real problem here is a lack of good understanding of Go data structures. This is how I'd combine the two in Ruby:

(responseA + responseB).group_by{|h| h[:uid]}.map{|k,v| v.reduce(:merge)}

EDIT: Here's a rough simulation on a Go playground https://play.golang.org/p/J0bJMjiM8DR

Answer 1

I think you are on the right track with "I have two arrays of hashes and i want to merge them on a shared key"

However, in Go you are going to be responsible for doing the merging yourself as their is no notion of a group_by type function / method.

func merge(As, Bs []Item) []Item {
    if len(As) == 0 {
        return Bs
    }
    bMap := make(map[string]Item)
    for _, b := range Bs {
        bMap[b.Uid] = b
    }
    merged := make([]Item, len(As))
    for i, a := range As {
        if b, ok := bMap[a.Uid]; ok {
            a.Val5 = b.Val5    // this and the next two lines could be done
            a.Val6 = b.Val6    // using reflection, but if you know the fields
            a.Val7 = b.Val7    // then this would be my preferred way.
        }
        merged[i] = a
    }
    return merged

}

Here is a playground of the results

Note: This wouldn't account for an empty slice of As as you would get nothing in the merge. Since I don't know the specifics of your application I can't say whether that is something that can happen, but if it can you will need to account for it.

For what it's worth, I think this is approximately the approach Verran was suggesting:

https://play.golang.org/p/3EpWlFdB8c6

Basically, let the results be returned on a channel and have them merged as you get both results.