dongqiao5573
2013-11-22 19:30 阅读 37

Golang:在消息包和JSON之间解析基准测试

We are working on a TCP server which takes simple textbased commands over TCP (similar to redis)

We are tossing up between using raw text command, JSON or message pack (http://msgpack.org/)

An example of a command could be:

text command: LOCK some_random_key 1000

JSON command: {"command":"LOCK","key":"some_random_key","timeout":1000}

messagePack: \x83\xA7command\xA4LOCK\xA3key\xAFsome_random_key\xA7timeout\xCD\x03\xE8

Question:

EDIT: I have figured out my own question which is the speed comparison between parsing JSON and MsgPack. Please see results in my answer

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

4条回答 默认 最新

  • 已采纳
    doulaopu2343 doulaopu2343 2013-11-24 04:58

    Parsing Speed Comparison:

    BenchmarkJSON     100000         17888 ns/op
    BenchmarkMsgPack      200000         10432 ns/op
    

    My benchmarking code:

    package benchmark
    
    import (
        "encoding/json"
        "github.com/vmihailenco/msgpack"
        "testing"
    )
    
    var in = map[string]interface{}{"c": "LOCK", "k": "31uEbMgunupShBVTewXjtqbBv5MndwfXhb", "T/O": 1000, "max": 200}
    
    func BenchmarkJSON(b *testing.B) {
        for i := 0; i < b.N; i++ {
            jsonB := EncodeJSON(in)
            DecodeJSON(jsonB)
        }
    }
    
    func BenchmarkMsgPack(b *testing.B) {
        for i := 0; i < b.N; i++ {
            b := EncodeMsgPack(in)
            DecodeMsgPack(b)
        }
    }
    
    func EncodeMsgPack(message map[string]interface{}) []byte {
        b, _ := msgpack.Marshal(message)
        return b
    }
    
    func DecodeMsgPack(b []byte) (out map[string]interface{}) {
        _ = msgpack.Unmarshal(b, &out)
        return
    }
    
    func EncodeJSON(message map[string]interface{}) []byte {
        b, _ := json.Marshal(message)
        return b
    }
    
    func DecodeJSON(b []byte) (out map[string]interface{}) {
        _ = json.Unmarshal(b, &out)
        return
    }
    
    点赞 评论 复制链接分享
  • dongyu1614 dongyu1614 2013-11-22 19:49

    I would suggest to do some benchmarks on the kind of data that the machines will be talking to each other.

    I would suggest to try Protocol Buffers (Encoding) + Snappy (compression)

    点赞 评论 复制链接分享
  • duangouhui0446 duangouhui0446 2013-11-23 00:18

    msgpack only promises to be shorter than json, not faster to parse. In both cases, your test string is so short and simple that your benchmarking may simply be testing the maturity of the particular implementation, rather than the underlying algorithms.

    If all your messages really are this short, parsing speed may be the least of your problems. I'd suggest designing your server such that the parsing bits are easily replaceable and actually profiling the code in action.

    Donald Knuth made the following statement on optimization:

    "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil"

    Lastly if you really want to know what is going on, you need to profile the code. See

    http://blog.golang.org/profiling-go-programs

    for an example of how to profile code with go.

    点赞 评论 复制链接分享
  • duanpo2813 duanpo2813 2013-11-23 05:14

    Also, your test cases are reversed BenchmarkJSON actually calls MsgPack and BenchmarkMsgPack calls Json

    could that have something to do with it?

    点赞 评论 复制链接分享

相关推荐