golang：内存问题

Question

I have trouble with memory. I don't understand why Go uses more and more memory (never freeing it) when my program runs for a long time.

After the first allocation, program uses nearly 9 MB of memory. Then after 12 hours it starts to use more memory exponentially, until 800 MB.

//.....code.....
if bol {
    // Assignment Struct.Var
    Struct_VastScript.TxtNoticeTop = JsonStruct_S.Options.TxtNoticeTop
    Struct_VastScript.TxtNoticeBottom = JsonStruct_S.Options.TxtNoticeBottom
    Struct_VastScript.Loop = JsonStruct_S.Options.Loop

    Struct_Image, err := getImage(Struct_VastScript.Video)
    if err == nil {
        if mobile == "true" {
            Struct_VastScript.Image = Struct_Image.URL360
        }
    }
    //open and parse a template file
    fi = path.Join("templates/VastPlayer", "TempVastPlayer.txt")
    tmpl, err := template.ParseFiles(fi)

    if err != nil {
        job_1.Complete(health.Panic)
        return false, err
    }
    //substitute fields in the template 'tmpl', with values from 'XmlStruct_V' and write it out to 'buf'
    var buf bytes.Buffer
    if err := tmpl.Execute(&buf, Struct_VastScript); err != nil {
        //if  err := tmpl.Execute(w, XmlStruct_V); err != nil {
        job_1.Complete(health.Panic)
        return false, err
    }

    // Call Func randString() : return alphanum random
    dir := randString(12)
    fpath := "http://creative2.xxx.io/api/html/" + dir

    // Create a new EndPoint to write the generated 'template' on 'w' http.ResponseWriter
    routeHtml := "/api/html/" + dir
    http.HandleFunc(routeHtml, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        //writes Template to 'w' http.ResponseWriter
        fmt.Fprintf(w, buf.String())
        fmt.Println("successfull Operation 2 !!")
        fmt.Println("")
        job_2.Complete(health.Success)
    }))


    //Call Func JsonReply(): return the finale Json response
    str := JsonReply(fpath, JsonStruct_S.Options.Animated, JsonStruct_S.Options.Responsive, JsonStruct_S.Options.Clickurl, JsonStruct_S.Options.Width, JsonStruct_S.Options.Height, adid, campaignid, JsonStruct_S.Type, JsonStruct_S.Options.Aspectratio, mobile)
    w.Header().Set("Content-Type", "application/json")
    //writes FinaleJson to 'w' http.ResponseWriter(it contains the link of the second endpoint "/api/html/")
    fmt.Fprint(w, str)
    fmt.Println("successfull Operation !!")
    fmt.Println("")
    job_1.Complete(health.Success)
    return true, nil
} else {
    return false, nil
}

For each call,my service need to generate a new template with the params that I receive,as you see I create a new endpoint for each call, I don't know if it's a good idea, I think the problem comes from this part of code but Im not sure because I don't know how GO manage it.

Answer 1

Obviously, you should not create handler every time request appears. They never free the memory so you will end up having out of memory exception.

Instead, put the handler endpoint into array (slice) and use ONE handler that responds to the request by looking the URL in this slice and then removing the item from the slice with it is not needed any longer.

So basically, instead of

routeHtml := "/api/html/" + dir
http.HandleFunc(routeHtml, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    //writes Template to 'w' http.ResponseWriter
    fmt.Fprintf(w, buf.String())
    fmt.Println("successfull Operation 2 !!")
    fmt.Println("")
    job_2.Complete(health.Success)
}))

do

type JobInfo struct {
    Path string
    // some data here
}

// maybe global context
var jobs []JobInfo

// initialisation
jobs = make([]JobInfo, 0)

http.HandleFunc("/api/html/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    path := r.URL.Path
    var job *JobInfo
    for _, j := range jobs {
        if j.Path == path {
            job = &j
            break
        }
    }

    if job != nil {
        // handle job request here
    }
}))

// and then in the jobs' loop
handlers = append(handlers, JobInfo{"/api/html/" + dir, ...})

It will work because:

Patterns name fixed, rooted paths, like "/favicon.ico", or rooted subtrees, like "/images/" (note the trailing slash). Longer patterns take precedence over shorter ones, so that if there are handlers registered for both "/images/" and "/images/thumbnails/", the latter handler will be called for paths beginning "/images/thumbnails/" and the former will receive requests for any other paths in the "/images/" subtree.

Do not forget to clean the array jobs, of course.