douji3623
2017-05-30 13:36
浏览 60
已采纳

base64解码器(io.Reader实现)的不良行为

I have tried, within a for loop, to re-declare/assign a base64 decoder and used the os.Seek function to go back to the beginning of the file at the end of the loop before this, in order for the called function (in this test case PrintBytes) to be able to process the file from beginning to end time and time again throughout the for loop.

Here is my (I'm sure terribly un-idiomatic) code, which fails to read the 2nd byte into the []byte of length 2 and capacity 2 during the second iteration of the main for loop in main():

package main

import (
    "encoding/base64"
    "io"
    "log"
    "net/http"
    "os"
)

var (
    remote_file string = "http://cryptopals.com/static/challenge-data/6.txt"
    local_file  string = "secrets_01_06.txt"
)

func main() {
    f, err := os.Open(local_file)
    if err != nil {
        DownloadFile(local_file, remote_file)
        f, err = os.Open(local_file)
        if err != nil {
            log.Fatal(err)
        }
    }
    defer f.Close()

    for blocksize := 1; blocksize <= 5; blocksize++ {
        decoder := base64.NewDecoder(base64.StdEncoding, f)
        PrintBytes(decoder, blocksize)
        _, err := f.Seek(0, 0)
        if err != nil {
            log.Fatal(err)
        }
    }
}

func PrintBytes(reader io.Reader, blocksize int) {
    block := make([]byte, blocksize)
    for {
        n, err := reader.Read(block)
        if err != nil && err != io.EOF {
            log.Fatal(err)
        }
        if n != blocksize {
            log.Printf("n=%d\tblocksize=%d\tbreaking...", n, blocksize)
            break
        }
        log.Printf("%x\tblocksize=%d", block, blocksize)
    }
}

func DownloadFile(local string, url string) {
    f, err := os.Create(local)
    if err != nil {
        log.Fatal(err)
    }
    defer f.Close()

    resp, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    _, err = io.Copy(f, resp.Body)
    if err != nil {
        log.Fatal(err)
    }
}

The output from this code can be viewed here https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085

It is this behaviour that I don't understand: https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085#file-bad_reader_log-L5758

I was expecting it to simply read the file 2 bytes at a time into the 2-byte slice, from beginning to end. For what reason does it only read 1 byte here?

  • 写回答
  • 好问题 提建议
  • 追加酬金
  • 关注问题
  • 邀请回答

1条回答 默认 最新

  • doulong6761 2017-05-30 14:28
    最佳回答

    It is not the problem of encoding/base64. When using io.Reader, it's not guaranteed that number of bytes read exactly equal to the buffer size (i.e. blocksize in your example code). The documentation states:

    Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

    In your example, change PrintBytes to

    func PrintBytes(reader io.Reader, blocksize int) {
        block := make([]byte, blocksize)
        for {
            n, err := reader.Read(block)
            //Process the data if n > 0, even when err != nil
            if n > 0 {
                log.Printf("%x\tblocksize=%d", block[:n], blocksize)
            }
    
            //Check for error
            if err != nil {
                if err != io.EOF {
                    log.Fatal(err)
                } else if err == io.EOF {
                    break
                }
            } else if n == 0 {
                //Considered as nothing happened
                log.Printf("WARNING: read return 0,nil")
            }
        }
    }
    

    Update:

    Correct usage of io.Reader, modify code to always process the data if n > 0 even when error occurs.

    评论
    解决 无用
    打赏 举报

相关推荐 更多相似问题