weixin_39565777 2020-11-29 16:52
浏览 0

hubwatcher cache removal

Description of change

This is the start of some steps to remove the Watcher and HubWatcher current revno cache. This frequently shows up in production as being the primary source of memory consumption. It is currently necessary, because when you start a Watch() on a document, you pass in the revno of the document you are watching, and we then need to know if you want to see a change for that document (because it has become newer than when you first looked at it).

Once you start a Watch(), then you assume that the document hasn't changed if you haven't gotten an event.

This changes the BaseWatcher contract. Now Watch/WatchMulti/WatchCollection/WatchCollectionWithFilter are synchronous. So once the function returns, you know that the watch is there and will inform your channel of any changes. However, they no longer send a 'no-op' first event. Most watchers in state/watcher.go already had some sort of "initialization" where they had to check if there was anything to be sent as their first event, so all of that continues to hold true.

QA steps


$ juju bootstrap lxd
# deploy and operate as normal, everything should still get the same set of events
# only now it should consume less memory when there are lots of models.
$ juju ssh -m controller 0
$$ juju_heap_profile > heap.pprof
# should also not show HubWatcher as a big source of memory.

Documentation changes

Should not be user visible, aside from slow memory growth.

Bug reference

https://bugs.launchpad.net/bugs/1813867

该提问来源于开源项目:juju/juju

  • 写回答

12条回答 默认 最新

  • weixin_39565777 2020-11-29 16:52
    关注

    , , I do believe this patch is now officially ready for review. The test suites appear to pass and I'm on to doing manual testing. I've reviewed the diff and hopefully cleaned up things as this branch evolved over a fair bit of time.

    If you'd like to jump on a HO to discuss it with me, I'd be happy to do so.

    评论

报告相同问题?