Kafka multi-broker in Go
In this article, we are going to study the configuration of Kafka multi-broker cluster on local and use it in Go. We will create 5 more brokers in Kafka-cluster
Kafka Installation
Installation information is available on Link
Let's get started with Kafka!
For production applications, There are multiple Kafka brokers are running with one or more clusters.
Kafka Multi-Broker Cluster setup
For creating multiple brokers, Move to the directory where Kafka is installed. In my case, It is in “usr/local/kafka” then cd config and makes 5 copies of config/server.properties file with the name server-1.properties, server-2.properties, and so on.
Add the below two lines with incrementing the port number :9093 in server-{{N}}.properties.
broker.id=1
listeners=PLAINTEXT://:9093
log.dir=/tmp/kafka-logs-1
Now let's start the Kafka servers with a separate terminal, Refer to the below commands.
./bin/kafka-server-start.sh config/server.properties
./bin/kafka-server-start.sh config/{{server properties file name}}
Refer to the below image.
Now create the topic with 6 replicas and 1 partition.
Note: The replication factor should always be less than the current broker count.
Check out the repository Link and run the producer and consumer to publish the message.
Thank you.