2021-01-12 12:07 阅读 0

Multiple brokers unsupported

When starting a Faust worker with a broker configuration shaped like "kafka://broker1:9092;broker2:9092" (which should be supported according to the docs), a fatal error is raised:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/code/src/mypackage/__main__.py", line 15, in <module>
  File "/usr/local/lib/python3.6/site-packages/faust/app/base.py", line 400, in main
  File "/usr/local/lib/python3.6/site-packages/faust/app/base.py", line 337, in finalize
    id = self.conf.id
  File "/usr/local/lib/python3.6/site-packages/faust/app/base.py", line 1026, in conf
  File "/usr/local/lib/python3.6/site-packages/faust/app/base.py", line 964, in _configure
    conf = self._load_settings(silent=silent)
  File "/usr/local/lib/python3.6/site-packages/faust/app/base.py", line 976, in _load_settings
    return Settings(appid, **self._prepare_compat_settings(conf))
  File "/usr/local/lib/python3.6/site-packages/faust/types/settings.py", line 319, in __init__
    self.broker = url or broker or BROKER_URL
  File "/usr/local/lib/python3.6/site-packages/faust/types/settings.py", line 458, in broker
    self._broker = URL(broker)
  File "/usr/local/lib/python3.6/site-packages/yarl/__init__.py", line 170, in __new__
  File "/usr/local/lib/python3.6/urllib/parse.py", line 169, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: '9092;broker2:9092'

Culprit appears to be the Settings type, the broker property of which attempts to convert the provided field to a single URI even if it consists of multiple brokers.


  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

4条回答 默认 最新

  • weixin_39612297 weixin_39612297 2021-01-12 12:07

    The problem is within the yarl module. It does not parse multiple urls, just one

    点赞 评论 复制链接分享
  • weixin_39612297 weixin_39612297 2021-01-12 12:07

    Are there plans to correct this bug?

    点赞 评论 复制链接分享
  • weixin_39887183 weixin_39887183 2021-01-12 12:07

    I would also like to see support for multiple brokers implemented. If that's a longer term goal, it would be nice to update the settings documentation to inform users that multiple brokers are not yet supported.

    点赞 评论 复制链接分享
  • weixin_39611413 weixin_39611413 2021-01-12 12:07

    At least the documentation ought to be fixed.

    点赞 评论 复制链接分享