douwu5009 2018-11-28 01:53
浏览 48
已采纳

去二进制编码变体与固定切片长度

I need to encode integer keys as byte slices for a KV database. I want to make the encoding smaller and cut the zero padding. I thought the variant encoding from the binary package would be the way to go.

But in both cases, variant and fixed, the byte slice length is the same. Just different bits arrangement since first bit is used as a flag. I assumed the variant encoding would cut the "extra fat". No.

package main

import (
    "encoding/binary"
    "fmt"
)

func main() {

    x := 16
    y := 106547

    fmt.Println(x)
    fmt.Println(y)

    // Variant
    bvx := make([]byte, 8)
    bvy := make([]byte, 8)

    xbts := binary.PutUvarint(bvx, uint64(x))
    ybts := binary.PutUvarint(bvy, uint64(y))

    fmt.Println("Variant bytes written x: ", xbts)
    fmt.Println("Variant bytes written y: ", ybts)

    fmt.Println(bvx)
    fmt.Println(bvy)

    fmt.Println("bvx length: ", len(bvx))
    fmt.Println("bvy length: ", len(bvy))

    // Fixed
    bfx := make([]byte, 8)
    bfy := make([]byte, 8)

    binary.LittleEndian.PutUint64(bfx, uint64(x))
    binary.LittleEndian.PutUint64(bfy, uint64(y))

    fmt.Println(bfx)
    fmt.Println(bfy)

    fmt.Println("bfx length: ", len(bfx))
    fmt.Println("bfy length: ", len(bfy))

}

My question is. Do I have to splice the byte slice manually with variant encoding to get rid of the extra bytes? Since put PutUvariant returns the number of bytes written, I can just splice the byte slice.

Is this the right way to do it? If not, what is the correct way to make the slices smaller?

Thanks

  • 写回答

1条回答 默认 最新

  • dtyz76562 2018-11-28 02:29
    关注

    Package binary

    import "encoding/binary"
    

    func PutUvarint

    func PutUvarint(buf []byte, x uint64) int
    

    PutUvarint encodes a uint64 into buf and returns the number of bytes written. If the buffer is too small, PutUvarint will panic.


    Fix your code:

    bvx := make([]byte, binary.MaxVarintLen64)
    bvy := make([]byte, binary.MaxVarintLen64)
    bvx = bvx[:binary.PutUvarint(bvx[:cap(bvx)], uint64(x))]
    bvy = bvy[:binary.PutUvarint(bvy[:cap(bvy)], uint64(y))]
    

    package main
    
    import (
        "encoding/binary"
        "fmt"
    )
    
    func main() {
    
        x := 16
        y := 106547
    
        fmt.Println(x)
        fmt.Println(y)
    
        // Variant
        bvx := make([]byte, binary.MaxVarintLen64)
        bvy := make([]byte, binary.MaxVarintLen64)
    
        bvx = bvx[:binary.PutUvarint(bvx[:cap(bvx)], uint64(x))]
        bvy = bvy[:binary.PutUvarint(bvy[:cap(bvy)], uint64(y))]
    
        fmt.Println("Variant bytes written x: ", len(bvx))
        fmt.Println("Variant bytes written y: ", len(bvy))
    
        fmt.Println(bvx)
        fmt.Println(bvy)
    
        fmt.Println("bvx length: ", len(bvx))
        fmt.Println("bvy length: ", len(bvy))
    
        // Fixed
        bfx := make([]byte, 8)
        bfy := make([]byte, 8)
    
        binary.LittleEndian.PutUint64(bfx, uint64(x))
        binary.LittleEndian.PutUint64(bfy, uint64(y))
    
        fmt.Println(bfx)
        fmt.Println(bfy)
    
        fmt.Println("bfx length: ", len(bfx))
        fmt.Println("bfy length: ", len(bfy))
    }
    

    Playground: https://play.golang.org/p/XN46KafMY23

    Output:

    16
    106547
    Variant bytes written x:  1
    Variant bytes written y:  3
    [16]
    [179 192 6]
    bvx length:  1
    bvy length:  3
    [16 0 0 0 0 0 0 0]
    [51 160 1 0 0 0 0 0]
    bfx length:  8
    bfy length:  8
    
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 求糕手!用matlab求解目标规划模型🙏
  • ¥15 画出的分类图不对,求解答一下
  • ¥20 springboot和springcloud版本问题
  • ¥15 ps2手柄控制树莓派小车
  • ¥30 C#:vsto powerpoint的外接程序
  • ¥30 stata将do文件代码转化为ado文件
  • ¥15 两个同维数组相比,不同位置、出现重复比无意义,而不同位置、不出现重复比,则有意义。把有意义的两个数组放入新的集合MK中。
  • ¥15 可以远程电脑安装nvm
  • ¥15 写一个可直接调用的函数,将32位有符号数转成另一个无符号的数
  • ¥15 CMAKE+VS2019+QT5.15.2组合进行二次编译