http将服务器处理大文件

Question

Please consider this code posted here, it handles http PUT requests. It works OK, with small files:

$ curl  -v http://192.168.1.10:9193/ -T red.tx
*   Trying 192.168.1.10...
* Connected to 192.168.1.10 (192.168.1.10) port 9193 (#0)
> PUT /red.tx HTTP/1.1
> Host: 192.168.1.10:9193
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 19697247
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Date: Fri, 26 Jul 2019 00:43:43 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host 192.168.1.10 left intact

However - send it a large-ish file > 1G size, it runs out of memory. Understandably so since the entire body is read in as contents, err := ioutil.ReadAll(r.Body)

runtime: out of memory: cannot allocate 536870912-byte block (537919488 in use)
fatal error: out of memory    
runtime stack:
runtime.throw(0x259a35, 0xd)
        /usr/lib/go-1.7/src/runtime/panic.go:566 +0x78
runtime.largeAlloc(0x1ffffe00, 0x10616f01, 0x10637afc)
        /usr/lib/go-1.7/src/runtime/malloc.go:776 +0xc8
runtime.mallocgc.func1()
        /usr/lib/go-1.7/src/runtime/malloc.go:669 +0x34
runtime.systemstack(0x10aa8200)
        /usr/lib/go-1.7/src/runtime/asm_arm.s:247 +0x80
runtime.mstart()
        /usr/lib/go-1.7/src/runtime/proc.go:1079

Please show me the right way to read from r.Body, and write it to the file. Thanks!

Answer 1

Try to copy the request payload into file directly.

f, err := os.Create(temporaryFilename)
if err != nil {
    return err
}
defer f.Close()

_, err := io.Copy(f, r.Body)

If you need to do something with the copied file, simply manipulate the created file object (in above example it's f).