Kafka:客户的可用经纪人用完了

更新:事实证明我在Docker中的端口有问题。 不知道为什么能解决这个问题。</ p>

我相信我遇到了一个奇怪的错误。 我正在使用 Sarama 库,并且能够成功创建使用者。 </ p>

  func main(){
config = sarama.NewConfig()
config.ClientID =“ go-kafka-consumer”
config.Consumer.Return.Errors = true
//创建新的消费者
主服务器,err:= sarama.NewConsumer(“ localhost:9092”,config)
if err!= nil {
panic(err)
}

defer func(){
如果err:= master.Close(); err!= nil {
panic(err)
}
}()

partitionConsumer,err:= master.ConsumePartition(“ myTopic”,0,
sarama.OffsetOldest)
如果err!= nil {
panic(err)
}
}
</ code> </ pre>

一旦我将这段代码分解并移出主例程,我就会遇到 错误:</ p>


kafka:客户端的可用代理用尽了(您的集群是否可以访问?)</ p>
</ blockquote>
\ n

我将代码拆分如下​​:以前的main()方法现在已使用称为NewConsumer()的方法转换为使用者程序包,而我的新main()则如下所示调用NewConsumer():</ p >

  c:= Consumer.NewConsumer()
</ code> </ pre>

恐慌语句在sarama行中被触发 .NewConsumer </ code>并打印出 kafka:客户端的可用代理用完了(与您的集群是否可达?)</ code> </ p>

为什么会中断 我的代码以此方式触发Sarama无法成为消费者? Sarama是否需要直接从main运行?</ p>
</ div>

展开原文

原文

UPDATE: It turned out I had an issue with my ports in Docker. Not sure why that fixed this phenomenon.

I believe I have come across a strange error. I am using the Sarama library and am able to create a consumer successfully.

func main() {
 config = sarama.NewConfig()
 config.ClientID = "go-kafka-consumer"
 config.Consumer.Return.Errors = true
 // Create new consumer
 master, err := sarama.NewConsumer("localhost:9092", config)
 if err != nil {
    panic(err)
 }

 defer func() {
     if err := master.Close(); err != nil {
         panic(err)
     }
 }()

 partitionConsumer, err := master.ConsumePartition("myTopic",0, 
 sarama.OffsetOldest)
 if err != nil {
     panic(err)
 }
}

As soon as I break this code up and move outside the main routine, I run into the error:

kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

I have split my code up as follows: the previous main() method I have now converted into a consumer package with a method called NewConsumer() and my new main() calls NewConsumer() like so:

c := consumer.NewConsumer()

The panic statement is getting triggered in the line with sarama.NewConsumer and prints out kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

Why would breaking up my code this way trigger Sarama to fail to make the consumer? Does Sarama need to be run directly from main?

1个回答



认为</ em>,您可以通过这种方式创建2个或更多消费者,并将其分为一个组(可能是去-卡夫卡消费者</代码>)。 您的经纪人有一个带有1个分区的主题,因此分配了一个组,另一个生成了此错误消息。 如果您将该主题的分区提高到2,该错误将消失。
但是我认为您的问题是,您以某种方式实例化了比以前更多的消费者。</ p>

来自果壳中的卡夫卡:</ p>

\ n

也可以将给定主题的消费者分为多个消费者组-组中的每个消费者都从唯一的分区中读取内容,整个组将整个主题中的所有消息都消费掉。 如果使用者的使用者多于分区,则某些使用者将因为它们没有要读取的分区而处于空闲状态。</ strong>如果使用者的分区多于使用者,则使用者将从多个分区接收消息。 如果您具有相同数量的使用者和分区,则每个使用者都从一个分区中按顺序读取消息。</ p>
</ blockquote>

它们不会精确地产生错误,因此 Sarama的问题。</ p>
</ div>

展开原文

原文

I think you create this way 2 or more consumers that get grouped into a single group (probably go-kafka-consumer). Your Broker has a Topic with 1 Partition, so one of Group gets assigned, the other one produces this error message. If you would raise the Partitions of that Topic to 2 the error would go away. But I think your problem is that you somehow have instantiated more consumers than before.

From Kafka in a Nutshell:

Consumers can also be organized into consumer groups for a given topic — each consumer within the group reads from a unique partition and the group as a whole consumes all messages from the entire topic. If you have more consumers than partitions then some consumers will be idle because they have no partitions to read from. If you have more partitions than consumers then consumers will receive messages from multiple partitions. If you have equal numbers of consumers and partitions, each consumer reads messages in order from exactly one partition.

They would not exactly produce an Error, so that would be an issue with Sarama.

dongluanguang4774
dongluanguang4774 因为仅仅向前的端口还不足以让客户在容器rmoff.net/2018/08/02/kafka-listeners-explaned中与Kafka通信
一年多之前 回复
douvcpx6526
douvcpx6526 感谢您的回答。 原来这是我的kafka docker镜像端口配置的问题。 为什么解决了我并不完全确定。
一年多之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问