douji3623 2017-05-30 13:36
浏览 61
已采纳

base64解码器(io.Reader实现)的不良行为

I have tried, within a for loop, to re-declare/assign a base64 decoder and used the os.Seek function to go back to the beginning of the file at the end of the loop before this, in order for the called function (in this test case PrintBytes) to be able to process the file from beginning to end time and time again throughout the for loop.

Here is my (I'm sure terribly un-idiomatic) code, which fails to read the 2nd byte into the []byte of length 2 and capacity 2 during the second iteration of the main for loop in main():

package main

import (
    "encoding/base64"
    "io"
    "log"
    "net/http"
    "os"
)

var (
    remote_file string = "http://cryptopals.com/static/challenge-data/6.txt"
    local_file  string = "secrets_01_06.txt"
)

func main() {
    f, err := os.Open(local_file)
    if err != nil {
        DownloadFile(local_file, remote_file)
        f, err = os.Open(local_file)
        if err != nil {
            log.Fatal(err)
        }
    }
    defer f.Close()

    for blocksize := 1; blocksize <= 5; blocksize++ {
        decoder := base64.NewDecoder(base64.StdEncoding, f)
        PrintBytes(decoder, blocksize)
        _, err := f.Seek(0, 0)
        if err != nil {
            log.Fatal(err)
        }
    }
}

func PrintBytes(reader io.Reader, blocksize int) {
    block := make([]byte, blocksize)
    for {
        n, err := reader.Read(block)
        if err != nil && err != io.EOF {
            log.Fatal(err)
        }
        if n != blocksize {
            log.Printf("n=%d\tblocksize=%d\tbreaking...", n, blocksize)
            break
        }
        log.Printf("%x\tblocksize=%d", block, blocksize)
    }
}

func DownloadFile(local string, url string) {
    f, err := os.Create(local)
    if err != nil {
        log.Fatal(err)
    }
    defer f.Close()

    resp, err := http.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    _, err = io.Copy(f, resp.Body)
    if err != nil {
        log.Fatal(err)
    }
}

The output from this code can be viewed here https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085

It is this behaviour that I don't understand: https://gist.github.com/tomatopeel/b8e2f04179c7613e2a8c8973a72ec085#file-bad_reader_log-L5758

I was expecting it to simply read the file 2 bytes at a time into the 2-byte slice, from beginning to end. For what reason does it only read 1 byte here?

  • 写回答

1条回答 默认 最新

  • doulong6761 2017-05-30 14:28
    关注

    It is not the problem of encoding/base64. When using io.Reader, it's not guaranteed that number of bytes read exactly equal to the buffer size (i.e. blocksize in your example code). The documentation states:

    Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

    In your example, change PrintBytes to

    func PrintBytes(reader io.Reader, blocksize int) {
        block := make([]byte, blocksize)
        for {
            n, err := reader.Read(block)
            //Process the data if n > 0, even when err != nil
            if n > 0 {
                log.Printf("%x\tblocksize=%d", block[:n], blocksize)
            }
    
            //Check for error
            if err != nil {
                if err != io.EOF {
                    log.Fatal(err)
                } else if err == io.EOF {
                    break
                }
            } else if n == 0 {
                //Considered as nothing happened
                log.Printf("WARNING: read return 0,nil")
            }
        }
    }
    

    Update:

    Correct usage of io.Reader, modify code to always process the data if n > 0 even when error occurs.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 envi深度学习模块错误原因
  • ¥15 孟德尔随机化混杂因素
  • ¥15 关于react-hook组件用函数控制是否渲染的及时性问题。
  • ¥50 Linux下的软件,要做模块化拆分。进程间通信是否有开源框架可以借用?
  • ¥100 修改原有的MYSQL存储代码,在最右边添加多列数据
  • ¥20 Open Interpreter 使用时报错: still has pending operation at deallocation, the process may crash
  • ¥15 qt中链接动态链接库,调用其中的函数,该函数的参数需要传入回调函数,自己创建的回调函数无法作为参数传递进去
  • ¥15 matlab svm二分类代码问题
  • ¥40 求一款能支持ios15以上的屏蔽越狱插件。比较好用的
  • ¥15 C++ QT对比内存字符(形式不定)