从网址转到lang解析JSON

Question

Here is my situation, i requested to a server with a GET and received a JSON format. i'm using go lang, and implemented it in Go Lang in beego web framework.

So i have implemented it like this one,

func (d *Tom) Get() {
    //passengersFile, err := http.Get("https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=MSFT&interval=1min&apikey=KLMH2VFJ0LCFNOX5")
    resp, err := http.Get("https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=MSFT&interval=1min&apikey=KLMH2VF0LCFNOX5")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    //fmt.Printf("%#v
", resp)

    dec := json.NewDecoder(resp.Body)
    if dec == nil {
        panic("Failed to start decoding JSON data")
    }

    json_map := make(map[string]interface{})

    err = dec.Decode(&json_map)
        for k, v := range json_map {
            if k == "Meta Data"{
                    continue
            }
            fmt.Printf("key[%s] value[%s]
", k, v)
        }
    if err != nil {
        panic(err)
    }

        d.Data["json"] = &json_map
    d.ServeJSON()
}

the format of the incoming json is like that ....

{
  "Meta Data": {
    "1. Information": "Intraday (1min) prices and volumes",
    "2. Symbol": "MSFT",
    "3. Last Refreshed": "2018-05-25 09:31:00",
    "4. Interval": "1min",
    "5. Output Size": "Compact",
    "6. Time Zone": "US/Eastern"
  },
  "Time Series (1min)": {
    "2018-05-24 14:23:00": {
      "1. open": "98.1432",
      "2. high": "98.1661",
      "3. low": "98.1238",
      "4. close": "98.1500",
      "5. volume": "19106"
    },
    "2018-05-24 14:24:00": {
      "1. open": "98.1500",
      "2. high": "98.1700",
      "3. low": "98.1400",
      "4. close": "98.1650",
      "5. volume": "18279"
    },
    "2018-05-24 14:25:00": {
      "1. open": "98.1650",
      "2. high": "98.2000",
      "3. low": "98.1600",
      "4. close": "98.1900",
      "5. volume": "32085"
    }
  }
}

Now i want to get the "Time Series (1min)" values and iterate on them to get each value of the "Date Time" values, like "1. open" ... etc. And of course save them on a single json and return it for those who requested it. Any help will be greatly appreciated!

Answer 1

The actual JSON object is quite, unusual - and it does not lend itself to a simple way of turning it into a struct.

For demonstration, I will use JSONGen, a great utility for turning JSON into Go structs.

For the problem at hand, I would probably use a two step approach.

First, parse the whole document (assuming msft.json contains an API response):

$ cat msft.json | JSONGen
type _ struct {
    MetaData struct {
        Information   string `json:"1. Information"`
        Interval      string `json:"4. Interval"`
        LastRefreshed string `json:"3. Last Refreshed"`
        OutputSize    string `json:"5. Output Size"`
        Symbol        string `json:"2. Symbol"`
        TimeZone      string `json:"6. Time Zone"`
    } `json:"Meta Data"`
    TimeSeries1min struct {
        _ struct {
            Close  string `json:"4. close"`
            High   string `json:"2. high"`
            ...

The problem is the repeated elements, keyed by datetime, which would probably be better modelled as a list. Anyway, with jq we can parse out a relevant piece and generate another struct:

$ cat msft.json | jq '.["Time Series (1min)"]["2018-05-24 15:47:00"]' | JSONGen
type _ struct {
    Close  string `json:"4. close"`
    High   string `json:"2. high"`
    Low    string `json:"3. low"`
    Open   string `json:"1. open"`
    Volume string `json:"5. volume"`
}

Now we can combine the two structs into one. Here is a complete program to parse the JSON input.

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "os"
)

type Item struct {
    Close  string `json:"4. close"`
    High   string `json:"2. high"`
    Low    string `json:"3. low"`
    Open   string `json:"1. open"`
    Volume string `json:"5. volume"`
}

type Response struct {
    MetaData struct {
        Information   string `json:"1. Information"`
        Interval      string `json:"4. Interval"`
        LastRefreshed string `json:"3. Last Refreshed"`
        OutputSize    string `json:"5. Output Size"`
        Symbol        string `json:"2. Symbol"`
        TimeZone      string `json:"6. Time Zone"`
    } `json:"Meta Data"`
    TimeSeries1min map[string]Item `json:"Time Series (1min)"`
}

We can model the time series as a map of OHLC items. The parsing now gets really simple:

func main() {
    resp := Response{}
    if err := json.NewDecoder(os.Stdin).Decode(&resp); err != nil {
        log.Fatal(err)
    }
    for k, v := range resp.TimeSeries1min {
        fmt.Printf("%s\t%s\t%s\t%s\t%s\t%s\t%s
",
            resp.MetaData.Symbol, resp.MetaData.LastRefreshed, k,
            v.Open, v.High, v.Low, v.Close)
    }
}

While will output something like:

$ go run main.go < msft.json
MSFT    2018-05-25 10:53:00 2018-05-25 10:49:00 98.6292 98.6292 98.5700 98.5750
MSFT    2018-05-25 10:53:00 2018-05-25 10:40:00 98.8700 98.8701 98.7900 98.8300
MSFT    2018-05-25 10:53:00 2018-05-25 10:22:00 98.6460 98.6500 98.6000 98.6300

...