如何在Golang中删除大文件的前N个字节？ [重复]

Question

This question already has an answer here:

• delete first N bytes from a text file 1 answer

I have a big file about 10G, and i want to delete the first 10 bytes of this file.

If I use ioutil.ReadFile(), the whole file will be allocated to memory. This can't be done!

Another idea is that I read this file line by line. And after removing the data, I should write the remain data line by line. In this way, the memory can be saved, but if there exists a better way? Like split the reader or split the file?

</div>

Answer 1

On most filesystems you can't "cut" a part out from the beginning or from the middle of a file, you can only truncate it at the end.

The easiest to achieve what you want is to open the source file, skip the part you want to strip off (use seeking), open the destination file and simply copy from the source to the destination.

For seeking (skipping), use File.Seek(). For copying between the files, use io.Copy().

This is how it can be done:

fin, err := os.Open("source.txt")
if err != nil {
    panic(err)
}
defer fin.Close()

fout, err := os.Create("dest.txt")
if err != nil {
    panic(err)
}
defer fout.Close()

// Offset is the number of bytes you want to exclude
_, err = fin.Seek(10, io.SeekStart)
if err != nil {
    panic(err)
}

n, err := io.Copy(fout, fin)
fmt.Printf("Copied %d bytes, err: %v", n, err)

Note that the above code will get the result file you want in a new file. If you want the "new" to be the old (meaning you don't want a different file), after the above operation (if it succeeds) delete the original file and rename the new one to the old.

This is how you can do that final step:

if err := os.Remove("source.txt"); err != nil {
    panic(err)
}

if err := os.Rename("dest.txt", "source.txt"); err != nil {
    panic(err)
}