Ok so, I did not really found the answer to my question, but as I did find a workaround I'll share it anyway.
Memory profiling
As @MichaelHampton suggested, I memory profiled my program. top 10
command revealed the following:
(pprof) top 10
Showing nodes accounting for 256MB, 99.88% of 256.30MB total
Dropped 15 nodes (cum <= 1.28MB)
flat flat% sum% cum cum%
256MB 99.88% 99.88% 256MB 99.88% github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/scrypt.Key
0 0% 99.88% 256MB 99.88% github.com/ethereum/go-ethereum/accounts/abi/bind.NewTransactor
0 0% 99.88% 256MB 99.88% github.com/ethereum/go-ethereum/accounts/keystore.DecryptKey
0 0% 99.88% 256MB 99.88% github.com/ethereum/go-ethereum/accounts/keystore.decryptKeyV3
0 0% 99.88% 256MB 99.88% github.com/ethereum/go-ethereum/accounts/keystore.getKDFKey
0 0% 99.88% 256.01MB 99.88% main.main
0 0% 99.88% 256.30MB 100% runtime.main
As you can see, the memory consumption comes from the Key
function in package scrypt
, which is called indirectly by bind.NewTransactor(...)
. From the doc:
Key derives a key from the password, salt, and cost parameters, returning a byte slice of length keyLen that can be used as cryptographic key.
In particular, this corresponds to how a private key is generated from a JSON wallet file. Indeed, the function itself allocates a lot of memory for cryptographic computations. What I do not understand though is why this huge memory allocation seems to persist once the key has been generated (the persistence is observed when I start the server with log.Fatal(http.ListenAndServe(":8080", nil))
just after).
Workaround
To confirm my suspicion, I used another way to generate my *TransactOpts
, which consists in getting the key directly form its hexadecimal representation instead of generating it from a wallet file:
func main() {
privateKey, err := crypto.HexToECDSA("myKeyInHex")
if err != nil {
log.Fatal(err)
}
auth := bind.NewKeyedTransactor(privateKey)
_ = auth
}
Although bind.NewTransactor
and bind.NewKeyedTransactor
return exactly the same object (the only difference being how the key was generated), using bind.NewTransactor
results in a 256MB persisting memory allocation, instead of a few KB for bind.NewKeyedTransactor
as shown below:
(pprof) top 10
Showing nodes accounting for 11.04kB, 100% of 11.04kB total
Showing top 10 nodes out of 19
flat flat% sum% cum cum%
6.83kB 61.90% 61.90% 6.83kB 61.90% time.initLocalFromTZI
4.21kB 38.10% 100% 4.21kB 38.10% github.com/ethereum/go-ethereum/crypto/sha3.(*state).clone (inline)
0 0% 100% 4.21kB 38.10% github.com/ethereum/go-ethereum/accounts/abi/bind.NewKeyedTransactor
0 0% 100% 4.21kB 38.10% github.com/ethereum/go-ethereum/crypto.Keccak256
0 0% 100% 4.21kB 38.10% github.com/ethereum/go-ethereum/crypto.PubkeyToAddress
0 0% 100% 4.21kB 38.10% github.com/ethereum/go-ethereum/crypto/sha3.(*state).Sum
0 0% 100% 6.83kB 61.90% github.com/pkg/profile.Start
0 0% 100% 6.83kB 61.90% github.com/pkg/profile.Start.func2
0 0% 100% 6.83kB 61.90% log.(*Logger).Output
0 0% 100% 6.83kB 61.90% log.(*Logger).formatHeader
Thus, for my simulation I will generate my private keys from my JSON wallets and store them in text files upstream, and then use bind.NewKeyedTransactor(...)
. I know this is not safe howsoever but for my simulation purpose it will be sufficient.
However, I'm pretty sure bind.NewTransactor
does not have the expected behavior concerning memory consumption, so I am going to open an issue on the go-ethereum
repository.