duanqianruan8448 2018-02-16 03:21
浏览 128
已采纳

big.Float SetPrec怪异的行为

After doing some calculations using big.Float in golang, I am setting the precision to 2.

And even thou the number is just a simple 10, after setting the precision it is 8.

package main

import (
    "fmt"
    "math/big"
)

func main() {
    cost := big.NewFloat(10)
    fmt.Println("COST NOW", cost)

    perKWh := big.NewFloat(0)
    cost.Add(cost, perKWh)
    fmt.Println("COST ", cost.String())

    perMinute := big.NewFloat(0)
    cost.Add(cost, perMinute)
    fmt.Println("COST ", cost.String())

    discountAmount := big.NewFloat(0)
    cost.Sub(cost, discountAmount)
    floatCos, _ := cost.Float64()
    fmt.Println(fmt.Sprintf("COST FLOAT %v", floatCos))
    cost.SetPrec(2)

    fmt.Println("COST ", cost.String())
}

Check playground example here: https://play.golang.org/p/JmCRXkD5u49

Would like to understand why

  • 写回答

2条回答 默认 最新

  • dongwei6700 2018-02-16 04:10
    关注

    From the fine manual:

    type Float
    [...]
    Each Float value also has a precision, rounding mode, and accuracy. The precision is the maximum number of mantissa bits available to represent the value. The rounding mode specifies how a result should be rounded to fit into the mantissa bits, and accuracy describes the rounding error with respect to the exact result.

    And big.Float is represented internally as:

    sign × mantissa × 2**exponent
    

    When you call SetPrec you're setting the number of bits available for the mantissa, not the number of digits of precision in the decimal representation of the number.

    You can't represent decimal 10 (1010 binary) in two bits of mantissa so it rounds to decimal 8 (1000 binary) which can fit into 2 bits. You need at least three bits to store the 101 part of decimal 10. 8 can fit into a single bit of mantissa so you'll see the same 8 if you say cost.SetPrec(1).

    You need to be thinking in terms of binary when using the big package.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)

报告相同问题?

悬赏问题

  • ¥15 seatunnel-web使用SQL组件时候后台报错,无法找到表格
  • ¥15 fpga自动售货机数码管(相关搜索:数字时钟)
  • ¥15 用前端向数据库插入数据,通过debug发现数据能走到后端,但是放行之后就会提示错误
  • ¥30 3天&7天&&15天&销量如何统计同一行
  • ¥30 帮我写一段可以读取LD2450数据并计算距离的Arduino代码
  • ¥15 飞机曲面部件如机翼,壁板等具体的孔位模型
  • ¥15 vs2019中数据导出问题
  • ¥20 云服务Linux系统TCP-MSS值修改?
  • ¥20 关于#单片机#的问题:项目:使用模拟iic与ov2640通讯环境:F407问题:读取的ID号总是0xff,自己调了调发现在读从机数据时,SDA线上并未有信号变化(语言-c语言)
  • ¥20 怎么在stm32门禁成品上增加查询记录功能