I wrote a little web crawler and had known that the Response is a zip file.
In my limited experience with golang programing, I only know how to unzip a existing file.
Can I unzip the Response.Body in memory without saving it in hard disk in advance?
golang解压缩Response.Body
- 写回答
- 好问题 0 提建议
- 追加酬金
- 关注问题
- 邀请回答
-
1条回答 默认 最新
- dsfhe34889 2018-05-26 04:30关注
Updating answer for handling Zip file response body in-memory.
Note: Ensure you have sufficient memory for handling zip file.
package main import ( "archive/zip" "bytes" "fmt" "io/ioutil" "log" "net/http" ) func main() { resp, err := http.Get("zip file url") if err != nil { log.Fatal(err) } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { log.Fatal(err) } zipReader, err := zip.NewReader(bytes.NewReader(body), int64(len(body))) if err != nil { log.Fatal(err) } // Read all the files from zip archive for _, zipFile := range zipReader.File { fmt.Println("Reading file:", zipFile.Name) unzippedFileBytes, err := readZipFile(zipFile) if err != nil { log.Println(err) continue } _ = unzippedFileBytes // this is unzipped file bytes } } func readZipFile(zf *zip.File) ([]byte, error) { f, err := zf.Open() if err != nil { return nil, err } defer f.Close() return ioutil.ReadAll(f) }
By default Go HTTP client handles Gzip response automatically. So do typical read and close of response body.
However there is a catch in it.
// Reference https://github.com/golang/go/blob/master/src/net/http/transport.go // // DisableCompression, if true, prevents the Transport from // requesting compression with an "Accept-Encoding: gzip" // request header when the Request contains no existing // Accept-Encoding value. If the Transport requests gzip on // its own and gets a gzipped response, it's transparently // decoded in the Response.Body. However, if the user // explicitly requested gzip it is not automatically // uncompressed. DisableCompression bool
What it means is; If you add a header
Accept-Encoding: gzip
manually in the request then you have to handle Gzip response body by yourself.For Example -
reader, err := gzip.NewReader(resp.Body) if err != nil { log.Fatal(err) } defer reader.Close() body, err := ioutil.ReadAll(reader) if err != nil { log.Fatal(err) } fmt.Println(string(body))
本回答被题主选为最佳回答 , 对您是否有帮助呢?解决 无用评论 打赏 举报
悬赏问题
- ¥20 docker里部署springboot项目,访问不到扬声器
- ¥15 netty整合springboot之后自动重连失效
- ¥15 悬赏!微信开发者工具报错,求帮改
- ¥20 wireshark抓不到vlan
- ¥20 关于#stm32#的问题:需要指导自动酸碱滴定仪的原理图程序代码及仿真
- ¥20 设计一款异域新娘的视频相亲软件需要哪些技术支持
- ¥15 stata安慰剂检验作图但是真实值不出现在图上
- ¥15 c程序不知道为什么得不到结果
- ¥40 复杂的限制性的商函数处理
- ¥15 程序不包含适用于入口点的静态Main方法