新手：在GO中适当调整[] byte大小（压缩）

Question

Go Newbie alert!

Not quite sure how to do this - I want to make a "file chunker" where I grab fixed slices out of a binary file for later upload as a learning project.

I currently have this:

    type (
       fileChunk  []byte
       fileChunks []fileChunk
    )


    func NumChunks(fi os.FileInfo, chunkSize int) int {
      chunks :=  fi.Size() / int64(chunkSize)
      if rem := fi.Size() % int64(chunkSize) != 0; rem {
        chunks++
      }
      return int(chunks)
    }

    // left out err checks for brevity
    func chunker(filePtr *string) fileChunks {
      f, err := os.Open(*filePtr)
      defer f.Close()

      // create the initial container to hold the slices
      file_chunks := make(fileChunks, 0)


      fi, err := f.Stat()  
      // show me how big the original file is   
      fmt.Printf("File Name: %s,  Size: %d
", fi.Name(), fi.Size())

      // let's partition it into 10000 byte pieces
      chunkSize := 10000
      chunks :=  NumChunks(fi, chunkSize)

      fmt.Printf("Need %d chunks for this file", chunks)

      for i := 0; i < chunks; i++ {
        b := make(fileChunk, chunkSize) // allocate a chunk, 10000 bytes

        n1, err := f.Read(b)
        fmt.Printf("Chunk: %d, %d bytes read
", i, n1)

            // add chunk to "container"
        file_chunks = append(file_chunks, b)
      }

      fmt.Println(len(file_chunks))

      return  file_chunks
    }

This all works mostly fine, but here's what happens if my fize size is 31234 bytes, then I'll end up with three slices full of the first 30000 bytes from the file, the final "chunk" will consist of 1234 "file bytes" followed by "padding" to the 10000 byte chunk size - I'd like the "remainder" filechunk ([]byte) to be sized to 1234, not the full capacity - what would the proper way to do this be? On the receiving side I would then "stitch" together all the pieces to recreate the original file.

Answer 1

You need to re-slice the remainder chunk to be just the length of the last chunk read:

n1, err := f.Read(b)
fmt.Printf("Chunk: %d, %d bytes read
", i, n1)
b = b[:n1]

This does the re-slicing for all chunks. Normally, n1 will be 10000 for all the non-remainder chunks, but there is no guarantee. The docs say "Read reads up to len(b) bytes from the File." So it's good to pay attention to n1 all the time.