从Golang中的嵌套动态键json获取数据

Question

I am new to go , I am trying to extract specific key from the json structure as below.

{
    "cluster_name": "escluster",
    "nodes": {
        "Sd2AvEXsswjTn6ErRYjg": {
            "timestamp": 1460624696217,
            "name": "master1",
            "transport_address": "10.0.0.1:9300",
            "host": "10.0.0.1:9300",
            "ip": [
                "10.0.0.1:9300",
                "NONE"
            ],
            "attributes": {
                "data": "false",
                "master": "true"
            },
            "os": {
                "timestamp": 1460624696217,
                "cpu_percent": 0,
                "load_average": 0,
                "mem": {
                    "total_in_bytes": 163987664,
                    "free_in_bytes": 136357264,
                    "used_in_bytes": 26629400,
                    "free_percent": 84,
                    "used_percent": 16
                },
                "swap": {
                    "total_in_bytes": 0,
                    "free_in_bytes": 0,
                    "used_in_bytes": 0
                }
            }
        },
        "yzabB-OaTfOqvgAELIMq1Q": {
            "timestamp": 1460624938213,
            "name": "data_1",
            "transport_address": "10.0.0.2:9300",
            "host": "10.0.0.2",
            "ip": [
                "10.0.0.2:9300",
                "NONE"
            ],
            "attributes": {
                "master": "false"
            },
            "os": {
                "timestamp": 1460624938213,
                "cpu_percent": 0,
                "load_average": 0.29,
                "mem": {
                    "total_in_bytes": 623840000,
                    "free_in_bytes": 4127648,
                    "used_in_bytes": 666012352,
                    "free_percent": 1,
                    "used_percent": 99
                },
                "swap": {
                    "total_in_bytes": 0,
                    "free_in_bytes": 0,
                    "used_in_bytes": 0
                }
            }
        },
        "q5CdgUF2TRewujr-0RPMgQ": {
            "timestamp": 1460624934417,
            "name": "master_0",
            "transport_address": "10.0.0.3:9300",
            "host": "10.0.0.2",
            "ip": [
                "10.0.0.3:9300",
                "NONE"
            ],
            "attributes": {
                "data": "false",
                "master": "true"
            },
            "os": {
                "timestamp": 1460624934417,
                "cpu_percent": 0,
                "load_average": 0,
                "mem": {
                    "total_in_bytes": 163898764,
                    "free_in_bytes": 139705616,
                    "used_in_bytes": 24194588,
                    "free_percent": 85,
                    "used_percent": 15
                },
                "swap": {
                    "total_in_bytes": 0,
                    "free_in_bytes": 0,
                    "used_in_bytes": 0
                }
            }
        }
    }
}      

As the node names are dynamic, I would like to retrieve mem for each node. I found this to retrieve data but its not dynamic key. How can we do this in go.

Answer 1

A good reference to learn how to process data in Go: http://blog.golang.org/json-and-go

Regarding your problem it seems to me like your data is relatively well structured... I have set an example extracting the mem part of each node in the following playground example:

http://play.golang.org/p/0O5U-3N_tA

The gist of it is that whatever is strongly specified can be encoded as structs. What is dynamic (e.g. your node names) can be encoded as maps. Both can be arbitrarily intermingled, so that this:

type Node struct {
    Timestamp        uint64            `json:timestamp`
    Name             string            `json:name`
    TransportAddress string            `json:transport_address`
    Host             string            `json:host`
    Ip               []string          `json:ip`
    Attributes       map[string]string `json:attributes`
    Os               *OsInfo           `json:os`
}

type Payload struct {
    Name  string           `json:cluster_name`
    Nodes map[string]*Node `json:nodes`
}

Can capture both the general structure of a node (via its struct definition) and the fact that they're dynamically indexed (via the fact that nodes themselves are stored in a dynamic map.

Once unmarshalled, it's then trivial to process your data to extract your memory info: it's just well-known structs and maps all the way down:

var p Payload
if err := json.Unmarshal([]byte(payload), &p); err != nil {
    log.Fatal(err)
}
for k, node := range p.Nodes {
    fmt.Printf("%s: %s
", k, node.Os.Mem)
}

Outputs as expected:

Sd2AvEXsswjTn6ErRYjg: map[total_in_bytes:%!s(uint64=163987664) free_in_bytes:%!s(uint64=136357264) used_in_bytes:%!s(uint64=26629400) free_percent:%!s(uint64=84) used_percent:%!s(uint64=16)]
yzabB-OaTfOqvgAELIMq1Q: map[used_percent:%!s(uint64=99) total_in_bytes:%!s(uint64=623840000) free_in_bytes:%!s(uint64=4127648) used_in_bytes:%!s(uint64=666012352) free_percent:%!s(uint64=1)]
q5CdgUF2TRewujr-0RPMgQ: map[total_in_bytes:%!s(uint64=163898764) free_in_bytes:%!s(uint64=139705616) used_in_bytes:%!s(uint64=24194588) free_percent:%!s(uint64=85) used_percent:%!s(uint64=15)]

(You are of course free to extract/reformat your data first and ditch the Payload object as soon as this is complete)