关闭
dpecb06062 2014-01-30 14:54
浏览 89
已采纳

golang读取XML内存泄漏?

We've been decoding a lot of XML lately using golang and encoding/xml. We noticed that, after quite a few files, our boxes run out of memory, start swapping, and generally die an unhappy death. So we made a test program. Here it is:

package main

import (
    "encoding/xml"
    "io/ioutil"
    "log"
    "time"
)

// this XML is for reading AWS SQS messages
type message struct {
    Body          []string `xml:"ReceiveMessageResult>Message>Body"`
    ReceiptHandle []string `xml:"ReceiveMessageResult>Message>ReceiptHandle"`
}

func main() {
    var m message

    readTicker := time.NewTicker(5 * time.Millisecond)

    body, err := ioutil.ReadFile("test.xml")

    for {
        select {
        case <-readTicker.C:
            err = xml.Unmarshal(body, &m)
            if err != nil {
                log.Println(err.Error())
            }
        }
    }
}

All it does is repeatedly decode an XML file over and over again. Our boxes show the same symptom: the memory usage of the binary grows without bound, until the box starts swapping.

We also added in some profiling code, which fires after 20s into the above script, and got the following from pprof's top100:

(pprof) top100
Total: 56.0 MB
    55.0  98.2%  98.2%     55.0  98.2% encoding/xml.copyValue
     1.0   1.8% 100.0%      1.0   1.8% cnew
     0.0   0.0% 100.0%      0.5   0.9% bytes.(*Buffer).WriteByte
     0.0   0.0% 100.0%      0.5   0.9% bytes.(*Buffer).grow
     0.0   0.0% 100.0%      0.5   0.9% bytes.makeSlice
     0.0   0.0% 100.0%     55.5  99.1% encoding/xml.(*Decoder).Decode
...

Running this later on, before the box runs out of memory, yields a higher total but pretty much the same percentages. Can anyone help us out? What are we missing?

Thanks in advance!

展开全部

  • 写回答

2条回答 默认 最新

  • duanputian5341 2014-01-30 15:26
    关注

    Try printing out your message each time. It will continue to append the fields onto the original struct.

    You need to reset the message with m = message{} after you do what you need to do with it, to clear it out, otherwise it will continue to grow.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)
编辑
预览

报告相同问题?

手机看
程序员都在用的中文IT技术交流社区

程序员都在用的中文IT技术交流社区

专业的中文 IT 技术社区,与千万技术人共成长

专业的中文 IT 技术社区,与千万技术人共成长

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

客服 返回
顶部