duancan7914
2018-05-13 09:05
浏览 579
已采纳

为什么解组此API响应会返回意外的EOF?

Problem

I'm creating a microservice in Go, using protocol buffers and gRPC. It interacts with a third-party API (Snooth) and I'm trying to unmarshal the JSON response into a protobuf struct I've created, using the proto package.

Unmarshalling returns an unexpected EOF error.

What I've Tried

  • Using json.newDecoder instead of json.unmarshal
  • jsonpb.Unmarshal instead of proto.Unmarshal (returns a bad value in StructValue for key error)
  • Limiting the response read with io.LimitReader

I've also read something about prefixing the proto types with a size tag or something? But I'm not sure what that is or if it's relevant. Here's the repo on Github.

Question

What is causing this unexpected EOF error and how do I fix it, so that the API response is successfully unmarshalled into the proto struct?

Side note: I am new to Go and would also appreciate any feedback/improvements on the following code. Thanks!

Code

Proto

message Response {
    message Meta {
        int32 results = 1;
        int32 returned = 2;
        string errmsg = 3;
        int32 status = 4;
    }

    Meta meta = 1;
    repeated google.protobuf.Struct wines = 2;
    repeated google.protobuf.Struct actions = 3;
}

main.go

func fetchFromSnooth(e string, qs string, c chan response) {
    defer func() {
        if r := recover(); r != nil {
            log.Printf("Error fetching from Snooth: %s", r)
            errmsg := fmt.Sprint(r)
            c <- response{nil, snoothApiError{errmsg}}
        }
    }()

    v := url.Values{"akey": {os.Getenv("SNOOTH_API_KEY")}}
    requestUrl := fmt.Sprintf("%s%s/?%s%s", snoothRoot, e, v.Encode(), qs)
    log.Printf("Fetching: %s", requestUrl)

    res, err := httpClient.Get(requestUrl)
    if err != nil {
        panic(err)
    }
    defer res.Body.Close()

    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
        panic(err)
    }

    result := &pb.Response{}
    if err := proto.Unmarshal(body, result); err != nil {
        panic(err)
    }
    c <- response{result, snoothApiError{""}}
}

func (s *snoothApiService) searchWines(params *pb.Parameters_WineSearch) response {
    c := make(chan response)
    defer close(c)

    v := url.Values{}
    go fetchFromSnooth("wines", v.Encode(), c)
    return <-c
}

func main() {
    snooth := snoothApiService{}
    resp := snooth.searchWines(nil)
    fmt.Println(resp)
}

Edit

Here's an example of the type of API response I'm trying to unmarshal:

    {
    "meta": {
        "results": 1489442,
        "returned": 5,
        "errmsg": "",
        "status": 1
    },
    "wines": [
        {
            "name": "Conway Deep Sea Chardonnay la Costa Wine Co",
            "code": "conway-deep-sea-chardonnay-la-costa-wine-co-2008-1",
            "region": "USA > California > Central Coast",
            "winery": "Conway Family Wines",
            "winery_id": "conway-family-wines",
            "varietal": "Chardonnay",
            "price": "21.99",
            "vintage": "2008",
            "type": "White Wine",
            "link": "http:\/\/www.snooth.com\/wine\/conway-deep-sea-chardonnay-la-costa-wine-co-2008-1\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/0\/2\/8\/image_787698_square.jpeg",
            "snoothrank": 3,
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 10
        },
        {
            "name": "Olmaia Cabernet di Toscana",
            "code": "olmaia-cabernet-di-toscana",
            "region": "Italy > Tuscany > Toscana Igt",
            "winery": "Col D Orcia",
            "winery_id": "col-d-orcia",
            "varietal": "Cabernet Sauvignon",
            "price": "0.00",
            "vintage": "",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/olmaia-cabernet-di-toscana\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/d\/e\/e\/image_790198_square.jpeg",
            "snoothrank": 3.5,
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 25
        },
        {
            "name": "Dominio Dostares Prieto Picudo Vino de la Tierra de Castilla Y León Cumal",
            "code": "dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2006",
            "region": "Spain > Castilla y León > Vino de la Tierra de Castilla y León",
            "winery": "Bischöfliches Priesterseminar Trier",
            "winery_id": "bischofliches-priesterseminar-trier",
            "varietal": "Prieto Picudo",
            "price": "15.89",
            "vintage": "2006",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2006\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/d\/0\/4\/image_336595_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 1
        },
        {
            "name": "Dominio Dostares Prieto Picudo Vino de la Tierra de Castilla Y León Cumal",
            "code": "dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2005",
            "region": "Spain > Castilla y León > Vino de la Tierra de Castilla y León",
            "winery": "Bischöfliches Priesterseminar Trier",
            "winery_id": "bischofliches-priesterseminar-trier",
            "varietal": "Prieto Picudo",
            "price": "38.99",
            "vintage": "2005",
            "type": "Red Wine",
            "link": "http:\/\/www.snooth.com\/wine\/dominio-dostares-prieto-picudo-vino-de-la-tierra-de-castilla-y-leon-cumal-2005\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/1\/d\/a\/image_336596_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 1
        },
        {
            "name": "The Little Penguin Chardonnay Premier",
            "code": "the-little-penguin-chardonnay-premier-2010",
            "region": "South East Australia",
            "winery": "The Little Penguin",
            "winery_id": "the-little-penguin",
            "varietal": "Chardonnay",
            "price": "11.99",
            "vintage": "2010",
            "type": "White Wine",
            "link": "http:\/\/www.snooth.com\/wine\/the-little-penguin-chardonnay-premier-2010\/",
            "tags": "",
            "image": "https:\/\/ei.isnooth.com\/multimedia\/2\/c\/4\/image_826282_square.jpeg",
            "snoothrank": "n\/a",
            "available": 0,
            "num_merchants": 0,
            "num_reviews": 7
        }
    ]
}
  • 写回答
  • 好问题 提建议
  • 关注问题
  • 收藏
  • 邀请回答

2条回答 默认 最新

  • drtldt55533 2018-05-14 21:12
    已采纳

    --Update--

    I've now got the response unmarshalling as desired, using the jsonpb.Unmarshal method. To do so though I had to unmarshal and marshal the response with the regular json library first, in order to get around some escaped values in the response (I was receiving a 'bad value in Struct' error:

    resJson, err := ioutil.ReadAll(res.Body)
    
    j := make(map[string]interface{})
    if err := json.Unmarshal(resJson, &j); err != nil {
        panic(err)
    }
    
    jbytes, err := json.Marshal(j)
    
    result := &pb.Response{}
    r := strings.NewReader(string(jbytes))
    if err := jsonpb.Unmarshal(r, result); err != nil {
        panic(err)
    }
    
    已采纳该答案
    评论
    解决 无用
    打赏 举报
  • dsnrixf6765 2018-05-14 07:46

    It DOES parse with then encoding/json package: https://play.golang.org/p/IQzMm2tDI7w

    The protoc-generated code's Unmarshal parses a Protocol Buffers encoded byte stream, NOT JSON!

    评论
    解决 无用
    打赏 举报

相关推荐 更多相似问题