I am not an Go unsafe package expert - Neither am I a seasoned C programmer. I am trying to read a huge file > 1G using mmap syscall in go. There are a number of reasons I do mmap and munmap as opposed to read, write I/O. That is beside the point - I can write to the file in a test, when I read from the file, I can ascertain that bytes length matches up, but I cannot read contents of this string file :( Can someone suggest some reading? I need to do to go a little further, here's some code I cooked up for sample test:
filename := "/tmp/dd_file.db"
f, err := os.OpenFile(filename, os.O_RDWR, 0666)
defer f.Close()
if err != nil {
fmt.Printf("error opening file: %v", err)
}
stat, _ := f.Stat()
size := stat.Size()
fmt.Printf("[READ-ONLY] : size was : %+v
", size)
got := make([]byte, size)
if _, err := f.ReadAt(got, 0); err != nil && err != io.EOF {
panic(err)
}
want, err := ioutil.ReadFile(filename)
if err != nil {
fmt.Printf("[READ-ONLY] : ioutil.ReadFile: %v", err)
}
// going to change the file size now, punch in a few things
t := unsafe.Sizeof("")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
_, err = f.Seek(int64(t-1), 0)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
_, err = f.Write([]byte(" "))
if err != nil {
fmt.Println(err)
os.Exit(1)
}
mmap, err := syscall.Mmap(int(f.Fd()), 0, int(t), syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_SHARED)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// not too sure on reading data on string - doesnt work as expected.
map_array := (*[10000]string)(unsafe.Pointer(&mmap[0]))
map_array[0] = "yellow!"
err = syscall.Munmap(mmap)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
newStat, _ := f.Stat()
newSize := newStat.Size()
fmt.Printf("[mmap( ) RW] : size was : %+v
", newSize)
got = make([]byte, newSize)
if _, err := f.ReadAt(got, 0); err != nil && err != io.EOF {
panic(err)
}
if len(got) == len(want) {
fmt.Println("well the lengths are equal atleast??!")
}
if !bytes.Equal(got, want) {
fmt.Printf("
[mmap( ) RW] : works! got %d
want %d", len(got), len(want))
}
This obviously works as expected - but what if I wanted to read via mmap( ) on an mmapped file, how do I read string out of these bytes (I have a sense there is an encoding package somewhere that I might have to put to use perhaps but then StringHeader on unsafe documentation confused me).
Suggestions.
