In a recent Go project I need to read a binary data file generated by Python, but due to padding, binary.Read
in Go doesn't read it properly. Below is a minimal example of my problem.
The struct I deal with if of the following format
type Index struct{
A int32
B int32
C int32
D int64
}
As you can see the size of the struct is 4+4+4+8=20, but Python added an extra 4 bytes for alignment. So the size is actually 24.
Below is the runnable Python code I use to write this struct:
#!/usr/bin/env python
# encoding=utf8
import struct
if __name__ == '__main__':
data = range(1, 13)
format = 'iiiq' * 3
content = struct.pack(format, *data)
with open('index.bin', 'wb') as f:
f.write(content)
the iiiq
format means there are three 32 bit integers and one 64 bit integer in the struct, which is the same with the Index
struct I defined earlier. And running this code will generate a file named index.bin
of size 72, which equals to 24 * 3.
And below is the Go code I use to read index.bin
:
package main
import (
"encoding/binary"
"fmt"
"os"
"io"
"unsafe"
)
type Index struct {
A int32
B int32
C int32
D int64
}
func main() {
indexSize := unsafe.Sizeof(Index{})
fp, _ := os.Open("index.bin")
defer fp.Close()
info, _ := fp.Stat()
fileSize := info.Size()
entryCnt := fileSize / int64(indexSize)
fmt.Printf("entry cnt: %d
", entryCnt)
readSlice := make([]Index, entryCnt)
reader := io.Reader(fp)
_ = binary.Read(reader, binary.LittleEndian, &readSlice)
fmt.Printf("After read:
%#v
", readSlice)
}
And this is the output:
entry cnt: 3
After read:
[]main.Index{main.Index{A:1, B:2, C:3, D:17179869184}, main.Index{A:0, B:5, C:6, D:7}, main.Index{A:8, B:0, C:9, D:47244640266}}
Obviously the output is messed up when reading from the Python generated file.
So my question is, how can I read the python generated file(with padding) in Go properly?