2016-06-21 05:30


  • rest
  • gzip

I need to implement web service in go that processes tar.gz files and I wonder what is the correct way, what content type I need to define, etc.

plus, I found that a lot of things are handled automatically - on the client side I just post a gzip reader as request body and Accept-Encoding: gzip header is added automatically, and on the server side - I do not need to gunzip the request body, it is already extracted to tar. does that make sense?

Can I rely that it would be like this with any client?


func main() {

    router := mux.NewRouter().StrictSlash(true)
    router.Handle("/results", dataupload.NewUploadHandler()).Methods("POST")
    log.Fatal(http.ListenAndServe(*address, router))


package dataupload

import (
    log "github.com/Sirupsen/logrus"

// UploadHandler responds to /results http request, which is the result-service rest API for uploading results
type UploadHandler struct {
    uploader Uploader

// NewUploadHandler creates UploadHandler instance
func NewUploadHandler() *UploadHandler {

    return &UploadHandler{
        uploader: TarUploader{},

func (uh UploadHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) {

    retStatus := http.StatusOK
    body, err := getBody(request)
    if err != nil {
        retStatus = http.StatusBadRequest
        log.Error("Error fetching request body. ", err)
    } else {
        _, err := uh.uploader.Upload(body)

func getBody(request *http.Request) (io.ReadCloser, error) {

    requestBody := request.Body
    if requestBody == nil {
        return nil, errors.New("Empty request body")
    var err error
    // this part is commented out since somehow the body is already gunzipped - no need to extract it.
    /*if strings.Contains(request.Header.Get("Accept-Encoding"), "gzip") {
        requestBody, err = gzip.NewReader(requestBody)

    return requestBody, err


func main() {

    f, err := os.Open("test.tar.gz")
    if err != nil {
        log.Fatalf("error openning file %s", err)
    defer f.Close()
    client := new(http.Client)
    reader, err := gzip.NewReader(f)
    if err != nil {
        log.Fatalf("error gzip file %s", err)
    request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)
    _, err = client.Do(request)
    if err != nil {
        log.Fatalf("error uploading file %s", err)
  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答


  • dpq755012465 dpq755012465 5年前

    The code you've written for the client is just sending the tarfile directly because of this code:

    reader, err := gzip.NewReader(f)
    request, err := http.NewRequest("POST", "http://localhost:8080/results", reader)

    If you sent the .tar.gz file content directly, then you would need to gunzip it on the server. E.g.:

    request, err := http.NewRequest(..., f)

    I think that's closer to the behavior you should expect third-party clients to exhibit.

    点赞 评论 复制链接分享
  • duanfei1268 duanfei1268 5年前

    Claerly not, but maybe...

    Golang provides a very good support for the http client (and server). This is one of the first language to support http2 and the design of the API clearly shows their concern on having a fast http.

    This is why they add Accept-Econding: gzip automatically. That will dramatically reduce the size of the server response and then optimize the transfer.

    But the gzip remains an option in http 1 and not all of the client will push this header to your server.

    Note that the Content-Type describes the type of data you are sending (here a tar.gz but could be application/json, test/javascript, ...), when the Accept-Encoding describes the way the data has been encoded for the transport

    Go will take care of transparently handling the Accept-Encoding for you because it is responsible of the transport of the data. Then it will be up to you to handle the Content-Type because only you know how to give a sense to the content you received

    点赞 评论 复制链接分享