2016-10-26 00:13


  • caching
  • dns

I am building a test crawler and wanted to know if Go (golang) caches DNS queries. I don't see anything about caching in the dnsclient. This seems like an important thing to add to any crawler to prevent lots of extra DNS queries.

Does Go (1.4+) cache DNS lookups?

If not, does debian/ubuntu/linux, windows, or darwin/OSX do any caching at the network level Go benefits from?

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答


  • dongqun5769 dongqun5769 5年前

    The answer to your question is no. There is no built-in dns caching in the std lib resolver. Would it be helpful? Maybe in some cases. Our org runs a local dns cache on each server and points resolv.conf there. So it wouldn't necessarily help us much to have caching in the language.

    There are some solutions that could help you. This package seems to have a pretty good solution. From the snippet in their readme you could even do:

    http.DefaultClient.Transport = &http.Transport {
      MaxIdleConnsPerHost: 64,
      Dial: func(network string, address string) (net.Conn, error) {
        separator := strings.LastIndex(address, ":")
        ip, _ := dnscache.FetchString(address[:separator])
        return net.Dial("tcp", ip + address[separator:])

    To enable it for all http requests from http.Get and friends.

    点赞 评论 复制链接分享
  • drcx71276 drcx71276 5年前

    The Go resolver does not do any in-process caching. While it would be possible to roll your own, your best bet is probably to run a system-wide DNS cache on each machine. (My favourite being dnsmasq.)

    点赞 评论 复制链接分享