Kafka 0.11 / Golang Sarama版本支持

Question

I spent some time finding that my Go application connecting to a Kafka 0.11 cluster was using the old 0.8.2 version of the library, which is missing the Timestamp value in the response.

I then found that Kafka 0.11.x API/version wasn't supported (but they are working on it).

I have two solutions for now.

First is to set the required version explicitly in my apps. Second is to "tune" the Sarama code to use version 0.10.x as minimum version, enabling me to use all the 0.10.x API/functionality.

I was still wondering why the version is not taken from the Kafka broker I'm connecting to?

I can't understand how it's supposed to work from the code... I clearly see the version that is set or defined in the sarama.Config.Version, but I can't find anything to update this value once connected to a broker?

I know that Python is doing it this way:

from kafka import BrokerConnection
broker=BrokerConnection("localhost",9092,0)
broker.connect()
broker.check_version()

(0, 11, 0)

What am I missing?

Answer 1

As far as I've gone, I'm not sure Sarama handle the search of the broker version by itself.

From my point of view, one need to define the API version you want to use in the config parameters of the Producer/Broker/Client like :

config := sarama.NewConfig()
config.Version=sarama.V0_10_2_0

Also, Sarama does not support 0.11.0 yet (Sept 2017), so use 0.10.2.0 to access the latest API.

Finaly, to use the second solution, edit the file Shopify/sarama/utils.go and add your versions at the end :

V0_11_0_0  = newKafkaVersion(0, 11, 0, 0)
V0_11_0  = newKafkaVersion(0, 11, 0, 0)
minVersion = V0_11_0_0