dosf40815
dosf40815
采纳率100%
2014-03-01 00:41

如何使用Golang的net / http包读取流式响应正文？

已采纳

我正在尝试连接到一个执行json数据流的端点。我想知道如何使用Go的net/http包执行基本请求，并在响应出现时读取响应？目前，我只能在连接关闭时读取响应。 

resp, err := http.Get("localhost:8080/stream")
if err != nil {
    ...
}
...
// perform work while connected and getting data

任何帮助都将不胜感激！谢谢!

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

2条回答

  • doure8758 doure8758 7年前

    The way to do streaming JSON parsing is with a Decoder: 

    json.NewDecoder(resp.Body).Decode(&yourStuff)

    For a streaming API where it's a bunch of objects coming back (a la Twitter), that should stream great with this model and the built-in encoding/json API. But if it's a large response where you have an object that's got a giant array with 10MB of stuff, you probably need to write your own Decoder to pull those inner pieces out and return them. I'm running into that problem with a library I've written.

    点赞 评论 复制链接分享
  • dtrotfd1012 dtrotfd1012 7年前

    The answer provided by Eve Freeman is the correct way to read json data. For reading any type of data, you can use the method below:

    resp, err := http.Get("http://localhost:3000/stream")
...

reader := bufio.NewReader(resp.Body)
for {
    line, err := reader.ReadBytes('
')
    ...

    log.Println(string(line))
}
    点赞 评论 复制链接分享
提交

相关推荐

  • 如何使用Golang的net / http包读取流式响应正文？
    7年前
    回答 2 已采纳 我正在尝试连接到一个执行json数据流的端点。我想知道如何使用Go的net/http包执行基本请求，并在响应出现时读取响应？目前，我只能在连接关闭时读取响应。 resp, err := http.Get("localhost:8080/stream") if err != nil { ... } ... // perform work while connected and getting data 任何帮助都将不胜感激！谢谢!
  • golang中访问GET参数net / http的问题
    7年前
    回答 2 已采纳 Following is my go program to extract GET parameters. (URL: /mysql?hostname=example.com) package main import ( "net/http" "fmt" //"encoding/json" //"html" "github.com/kr/pretty" ); func main(){ http.HandleFunc("/", foo) http.ListenAndServe(":80", nil) } func foo(w http.ResponseWriter, r * http.Request){ w.Header().Set("Server","A Go WebServer") w.Header().Set("Content-Type", "text/html") hostname := r.URL.Query()["hostname"] //w.Write([]byte(hostname[0])) fmt.Printf("%# v", pretty.Formatter(hostname[0])) w.Write([]byte(hostname)) } Error shown on console. "example.com"2014/09/01 02:57:22 http: panic serving 172.17.92.14:49411: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42dc41) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f8b5) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410381) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x410228) /root/go/src/api/main.go:34 (0x400d77) foo: fmt.Printf("%# v", pretty.Formatter(hostname[0])) /usr/lib/go/src/pkg/net/http/server.go:690 (0x421bed) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422a6f) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x421a00) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d9bb) 2014/09/01 02:57:22 http: panic serving 172.17.92.14:49412: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42dc41) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f8b5) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410381) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x410228) /root/go/src/api/main.go:34 (0x400d77) foo: fmt.Printf("%# v", pretty.Formatter(hostname[0])) /usr/lib/go/src/pkg/net/http/server.go:690 (0x421bed) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422a6f) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x421a00) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d9bb) 2014/09/01 02:57:22 http: panic serving 172.17.92.14:49413: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42dc41) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f8b5) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410381) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x410228) /root/go/src/api/main.go:34 (0x400d77) foo: fmt.Printf("%# v", pretty.Formatter(hostname[0])) /usr/lib/go/src/pkg/net/http/server.go:690 (0x421bed) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422a6f) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x421a00) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d9bb) I am having trouble with determining the data type of hostname variable. I should get it as a string. When I try to write to http response writer w.Write([]byte(hostname)) following build error is shown go/src/api/main.go:38: cannot convert hostname (type []string) to type []byte If I treat it as a string array build is successful but there is run time error newhost := hostname[0]; w.Write([]byte(newhost)) Error: 2014/09/01 04:42:40 http: panic serving 172.17.92.14:50404: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42db27) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f79b) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410267) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x41010e) /root/go/src/api/main.go:38 (0x400d63) foo: newhost := hostname[0]; /usr/lib/go/src/pkg/net/http/server.go:690 (0x421ad3) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422955) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x4218e6) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d8a1) 2014/09/01 04:42:40 http: panic serving 172.17.92.14:50405: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42db27) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f79b) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410267) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x41010e) /root/go/src/api/main.go:38 (0x400d63) foo: newhost := hostname[0]; /usr/lib/go/src/pkg/net/http/server.go:690 (0x421ad3) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422955) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x4218e6) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d8a1) 2014/09/01 04:42:40 http: panic serving 172.17.92.14:50406: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42db27) _func_003: buf.Write(debug.Stack()) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:1443 (0x40f79b) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:128 (0x410267) /home/michael/DPKG/golang/src/pkg/runtime/runtime.c:85 (0x41010e) /root/go/src/api/main.go:38 (0x400d63) foo: newhost := hostname[0]; /usr/lib/go/src/pkg/net/http/server.go:690 (0x421ad3) HandlerFunc.ServeHTTP: f(w, r) /usr/lib/go/src/pkg/net/http/server.go:926 (0x422955) (*ServeMux).ServeHTTP: mux.handler(r).ServeHTTP(w, r) /usr/lib/go/src/pkg/net/http/server.go:656 (0x4218e6) (*conn).serve: handler.ServeHTTP(w, w.req) /home/michael/DPKG/golang/src/pkg/runtime/proc.c:271 (0x40d8a1) I also checked length of newhost as @VonC suggested and it gave expected results. fmt.Print(" ") fmt.Println((len(newhost))) Output on console "example.com" 11 2014/09/01 05:05:15 http: panic serving 172.17.92.14:50779: runtime error: index out of range /usr/lib/go/src/pkg/net/http/server.go:576 (0x42dd33)
  • Golang的“ net / http”中的客户端HTTP请求和服务器HTTP请求有什么区别
    2年前
    回答 1 已采纳 I have seen people use NewRequest() method of the "net/http" package for testing APIs. Why not use the NewRequest() method from "net/http/httptesting"? What's the difference? Documentation advises the following: // To generate a client HTTP request instead of a server request, see // the NewRequest function in the net/http package. What would be the difference in handling cookies, for example? Both seem to be very similar.
  • 如何在golang net / smtp.sendmail中定义发件人名称？
    4年前
    回答 1 已采纳 My question is short, I have a golang application from which I am trying to send an email using net/smtp.sendmail: This is the command which I am going to use smtp.SendMail(server, auth, **from**, to, msg) No usually people pass the sender's email address to from. Is there anyway I can pass a sender's name as well? So the recipient will get an email from: Foo Bar <foo@bar.com> instead of an email from <foo@bar.com> I checked the official documentation and dozens of examples online but could not find anything. Thank you.
  • 扫描github.com/golang/protobuf/proto/testdata时出错：找不到包“。”
    3年前
    回答 1 已采纳 When running glide install on my project, I get the following error: [ERROR] Error scanning github.com/golang/protobuf/proto/testdata: cannot find package "." in: /Users/bevernie/.glide/cache/src/https-github.com-golang-protobuf/proto/testdata [ERROR] Failed to retrieve a list of dependencies: Error resolving imports When checking protobuf's source code, I can in fact see that there is no such package. I however don't directly use protobuf, so the error must come from one of the dependencies I use. When running glide tree on my project, there is only one instance of github.com/golang/protobuf/proto/testdata: |-- github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto) | |-- github.com/golang/protobuf/proto/test_proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto/test_proto) | | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto) | |-- github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any) | | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto) github.com/golang/protobuf/proto/testdata (glide get github.com/golang/protobuf/proto/testdata) |-- github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any) | |-- github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto) | | |-- github.com/golang/protobuf/proto/test_proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto/test_proto) | | | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto) | | |-- (Recursion) github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any) which doesn't really help me pinpoint the source of the problem. Do you have any suggestions as of how to fix the issues? My project was compiling just fine until a week or two ago (I use Docker to deploy in production, so the glide install was run every time and never failed before that, and I haven't added any new dependency recently).
  • Gorilla WebSocket与golang.org/x/net/websocket的比较
    6年前
    回答 1 已采纳 According Gorilla Websockets Project it is not possible to send pings and pongs using golang.org/x/net/websocket. At the same time, the following is on the project page of golang.org/x/net/websocket: Package websocket implements a client and server for the WebSocket protocol as specified in RFC 6455. I am a little confused. golang.org/x/net/websocket implements RFC 6455 but can not send control frames (cancel, ping, pong) although this is specified in RFC 6455 - Section Control Frames So what will happen if I use golang.org/x/net/websocket package. Will the connection abort after a timeout? In other words, how is it ensured here that the connection does not break off.
  • 导入错误：带有html的golang.org/x/net/html
    2年前
    回答 2 已采纳 I'm using "html" and "golang.org/x/net/html" in golang. It's got error by same name.So I named other name like this. net_html "golang.org/x/net/html" and also I tried like this _ "golang.org/x/net/html" But it wasn't effect for me. Do you know how to solve this problem?
  • 使用Golang中的encoding / xml包制作soap xml xml
    3年前
    回答 2 已采纳 I want to make a sample xml that repeats a field depending on the value of the slice. But the final result is not as desired. XML that I want to create recipients = ["num1","num2",...] //Start XML . . . <SendDetail> <item><Mobile>num1</Mobile></item> <item><Mobile>num2</Mobile></item> ... </SendDetail> The number of repetitions of depends on the length of the slice of recipients and its values. My Code type item struct { Mobile []string `xml:"Mobile"` } type sendDetail struct { Item *[]item `xml:"item"` } SendDetail: &sendDetail{ Item:&[]item{ item{ Mobile:recipients, }, }, }, Result <SendDetail> <item> <Mobile>num1</Mobile> <Mobile>num2</Mobile> </item> </SendDetail> Any pointers will be appreciated.
  • 如何从Golang的http响应中读取压缩数据
    3年前
    回答 2 已采纳 I have a http response which is gzipped. resp, err := client.Do(req) if err != nil { return "", err } defer resp.Body.Close() if resp.StatusCode == http.StatusOK { var buf bytes.Buffer } How can I ungzipped it and parse it into my struct? I saw a question like this: Reading gzipped HTTP response in Go but it output the response into a standard output. Also the example runs into error, the reader, err = gzip.NewReader(response.Body) returns err as "EOF". How can I debug this?
  • 如何避免找不到包“ github.com/golang/protobuf/jsonpb”错误 docker
    3年前
    回答 1 已采纳 I want to put my code inside a docker container, I have created dockerfile and when I run, I got an error internal/server/handlers.go:16:2: cannot find package "github.com/lib/pq" in any of: /usr/local/go/src/github.com/lib/pq (from $GOROOT) /go/src/github.com/lib/pq (from $GOPATH) but when I launch my code locally without docker by typing go run main.go everything is fine
  • golang 读取 http包读取流式响应正文_Golang net/rpc 包的深度解读和学习
    4月前
    weixin_39732609的博客 Golang 提供了一个开箱即用的RPC服务，实现方式简约而不简单。本文对net/rpc 包做深度解读和学习实战。RPC 简单介绍远程过程调用 (Remote Procedure Call，RPC) 是一种计算机通信协议。允许运行在一台计算机的程序调用另一个地址空间的子程序(一般是开放网络中的一台计算机)，而程序员就像调用调用本地程序一样，无需额外做交互编程。RPC 是一种 CS (Cl...
  • golang 读取 http包读取流式响应正文_Golang net 包学习和实战
    5月前
    Salton Z的博客 golang 的 net 包，相关接口和结构比较多，今天做个简单的梳理。网络模型在总结 net 包之前，还需要温习模糊的网络模型知识。下图是大学课本上的网络模型图：模型图中可以看到，OSI 的七层模型，每一层实现的是与对端相应层的通信接口。但是实际应用中，我们把会话层、表示层、应用层统称为应用层。因此，就变成了TCP/IP 的五层模型。 其中网络层包含了 ip,arp,icmp 等协议，传输层包含...
  • golang 读取 http包读取流式响应正文_gRPC 流式调用
    4月前
    枢密副使太子少保的博客 gRPC 使用 Protocol buffers 作为接口定义语言(IDL)来描述服务接口和输入输出消息的结构，目前支持 4 种定义服务方法类型：类型说明简单 RPC客户端传入一个请求对象，服务端返回一个结果对象客户端流式 RPC客户端传入多个请求对象，服务端返回一个结果对象服务端流式 RPC客户端传入一个请求对象，服务端返回多个结果对象双向流式 RPC客户端传入多个请求对象，服务端返回...
  • golang 读取 http包读取流式响应正文_Go net/rpc 包的深度解读和学习
    4月前
    weixin_39712969的博客 Golang 提供了一个开箱即用的RPC服务，实现方式简约而不简单。本文对net/rpc 包做深度解读和学习实战。RPC 简单介绍远程过程调用 (Remote Procedure Call，RPC) 是一种计算机通信协议。允许运行在一台计算机的程序调用另一个地址空间的子程序(一般是开放网络中的一台计算机)，而程序员就像调用调用本地程序一样，无需额外做交互编程。RPC 是一种 CS (Cl...
  • 译】golang net/http timeout 完整指南
    4月前
    埋刀封剑的博客 在Go中编写HTTP服务器或客户端时，超时是最容易，也最细微的出错点之一：有很多可供选择的选项，并且一个错误在很长一段时间内不会有任何后果，直到网络出现故障并且进程挂起。 HTTP是一个复杂的多阶段协议，所以没有适合所有超时的解决方案。考虑以下不同情况，比如流式传输端点、JSON API或者Comet端点，默认设置通常不是你想要的。 在这篇文章中，我将分解您可能需要应用超时的各个阶段，并在服务器和客户端上查看执行此操作的不同方法。 SetDeadline 首先，您需要了解Go对外暴露的实现超时的网络
  • 关于Go net/http 超时完全指南
    1年前
    衣舞晨风的博客 翻译自：The complete guide to Go net/http timeouts地址：https://blog.cloudflare.com/the-complete-gui...
  • golang mysql大量写入_Golang 实现分片读取http超大文件流和并发控制
    3月前
    weixin_39954698的博客 分片读取http超大文件流Golang中的HTTP发送get请求，在获取内容有两种情况。Golang发送http get请求方式resp, err := http.Get(sendUrl)if err != nil {fmt.Println("出错", err)return}第一种方式是直接全部读取出来，这种方式在小数据量的时候很方便。body变量直接全部接收resp响应内容body, err2 ...
  • golang HTTP Response Body的内存泄漏问题
    2年前
    喝可乐_耍酒疯的博客 在爬虫类的场景中， 我们需要做大量的并发http请求，所以经常需要打开和关闭大量http连接。 如果没有正确处理http连接， 很容易导致内存的泄漏。 package main import ( "fmt" "net/http" "io/ioutil" ) func main() { resp, err := http.Get("https://api....
  • go net.conn读取请求头信息_只因掌握它，便轻松使用Go构建微服务
    4月前
    一枚优就业的学员的博客 点击上方蓝色“Go语言中文网”关注我们，设个星标，每天学习Go语言图片描述本教程提供了 Go 使用 gRPC 的基础教程在教程中你将会学到如何：在.proto文件中定义一个服务。使用 protocol buffer 编译器生成客户端和服务端代码。使用 gRPC 的 Go API 为你的服务写一个客户端和服务器。继续之前，请确保你已经对 gRPC 概念有所了解，并且熟悉 protoco...
  • go net.conn读取请求头信息_Go 语言中的 gRPC 基础入门
    4月前
    赵余的博客 01为什么使用 gRPC？借助 gRPC，我们可以在 .proto 文件中一次定义我们的服务，并以 gRPC 支持的任何语言生成客户端和服务器代码，无论是在大型数据中心内的服务器，还是在个人的电脑的环境中，这些客户端和服务器代码都可以运行 – gRPC 可以为您处理不同语言和环境之间的通信。我们还获得了使用 protocol buffers 的所有优点，包括有效的序列化，简单的 IDL...