duanqianruan8448 2018-02-16 03:21
浏览 130
已采纳

big.Float SetPrec怪异的行为

After doing some calculations using big.Float in golang, I am setting the precision to 2.

And even thou the number is just a simple 10, after setting the precision it is 8.

package main

import (
    "fmt"
    "math/big"
)

func main() {
    cost := big.NewFloat(10)
    fmt.Println("COST NOW", cost)

    perKWh := big.NewFloat(0)
    cost.Add(cost, perKWh)
    fmt.Println("COST ", cost.String())

    perMinute := big.NewFloat(0)
    cost.Add(cost, perMinute)
    fmt.Println("COST ", cost.String())

    discountAmount := big.NewFloat(0)
    cost.Sub(cost, discountAmount)
    floatCos, _ := cost.Float64()
    fmt.Println(fmt.Sprintf("COST FLOAT %v", floatCos))
    cost.SetPrec(2)

    fmt.Println("COST ", cost.String())
}

Check playground example here: https://play.golang.org/p/JmCRXkD5u49

Would like to understand why

  • 写回答

2条回答 默认 最新

  • dongwei6700 2018-02-16 04:10
    关注

    From the fine manual:

    type Float
    [...]
    Each Float value also has a precision, rounding mode, and accuracy. The precision is the maximum number of mantissa bits available to represent the value. The rounding mode specifies how a result should be rounded to fit into the mantissa bits, and accuracy describes the rounding error with respect to the exact result.

    And big.Float is represented internally as:

    sign × mantissa × 2**exponent
    

    When you call SetPrec you're setting the number of bits available for the mantissa, not the number of digits of precision in the decimal representation of the number.

    You can't represent decimal 10 (1010 binary) in two bits of mantissa so it rounds to decimal 8 (1000 binary) which can fit into 2 bits. You need at least three bits to store the 101 part of decimal 10. 8 can fit into a single bit of mantissa so you'll see the same 8 if you say cost.SetPrec(1).

    You need to be thinking in terms of binary when using the big package.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)

报告相同问题?