在Go中，如何有效地将流式HTTP响应正文写入文件中的搜索位置？

Question

I have a program that combines multiple http responses and writes to the respective seek positions on a file. I am currently doing this by

client := new(http.Client)
req, _ := http.NewRequest("GET", os.Args[1], nil)
resp, _ := client.Do(req)
defer resp.Close()
reader, _ := ioutil.ReadAll(resp.Body) //Reads the entire response to memory
//Some func that gets the seek value someval
fs.Seek(int64(someval), 0)
fs.Write(reader)

This sometimes results in a large memory usage because of the ioutil.ReadAll.

I tried bytes.Buffer as

buf := new(bytes.Buffer)
offset, _ := buf.ReadFrom(resp.Body) //Still reads the entire response to memory.
fs.Write(buf.Bytes())

but it was still the same.

My intention was to use a buffered write to the file, then seek to the offset again, and to continue write again till the end of stream is received (and hence capturing the offset value from buf.ReadFrom). But it was also keeping everything in the memory and writing at once.

What is the best way to write a similar stream directly to the disk, without keeping the entire content in buffer?

An example to understand would be much appreciated.

Thank you.

Answer 1

Use io.Copy to copy the response body to the file:

resp, _ := client.Do(req)
defer resp.Close()
//Some func that gets the seek value someval
fs.Seek(int64(someval), 0)
n, err := io.Copy(fs, resp.Body)
// n is number of bytes copied