dongyunque2511 2016-11-01 18:49
浏览 72
已采纳

如何有效地在文件中插入字节片?

I am building a simple key-value store for fun. Right now, I am looking for an efficient way to insert a slice in a file.

My current approach:

  • find the desired offset

  • store in a buffer the blocks that precede the desired insert point

  • append the byte slice to that buffer

  • append the rest of the file

  • Write to disk

Problem is:

  • It is not a given that the whole file can fit in memory

  • It is inefficient

I have looked into available libraries and sadly the best match I have found os.WriteAt overwrites the following blocks. Example:

import "os"

func main() {
    pathToFile := "./tmp"
    bufferToWrite := []byte{255, 255, 255, 255, 255}

    f, _ := os.OpenFile(pathToFile, os.O_CREATE|os.O_RDWR, os.PermMode)
    defer f.Close()
    f.Write(bufferToWrite)

So, at this point the content of tmp will be (after $: xxd -g 1 -b tmp):

11111111 11111111 11111111 (x) 11111111 11111111

Let's try to insert something with offset = 3 (x):

    bufferToInsert := []byte{0, 0}
    f.WriteAt(bufferToInsert, 3)
}

Output will be:

11111111 11111111 11111111 00000000 00000000

And I want it to be:

11111111 11111111 11111111 00000000 00000000 11111111 11111111

Any ideas?

  • 写回答

1条回答 默认 最新

  • duanbairan4235 2016-11-01 19:00
    关注

    Instead of inventing your own file format, you could copy the pack file format from Git.

    The basic idea is to have an index file and a data file. When you want to insert a slice you just append it to the data file. Then you update the index file, which is usually smaller. Note that the pack file is not designed for real-time updates, but accompanied by individual object files.

    Or have a look at the Berkeley DB file format.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 基于RTKLIB框架写的精密单点定位-AR
  • ¥15 PFENet的预训练权重
  • ¥15 程序哪有错误怎么改?
  • ¥15 交换机和交换机之间的链路带宽以及主机带宽的理解
  • ¥15 ai创想家对战模式代码
  • ¥15 集合A由3个2行4列二维数组构成,从集合A中任意取一个二维数组元素、如果该二维数组元素的对应列位置的上、下两数都是奇数,而且仅有2个列是奇数/奇数,则该数组有意义,并放入集合B中打印输出。
  • ¥15 电信IPV6 无法外网访问吗
  • ¥15 有偿求效果比较好的遥感影像匹配的c++代码
  • ¥15 博主,你好,我下载了你的智能网联汽车辅助驾驶安全信息检测系统,现在不会运行,可以教我吗,
  • ¥15 怎么在excle输入下列公式