EDUCBA

EDUCBA

MENUMENU
  • Free Tutorials
  • Free Courses
  • Certification Courses
  • 360+ Courses All in One Bundle
  • Login
Home Data Science Data Science Tutorials Kafka Tutorial Kafka Listener
Secondary Sidebar
Kafka Tutorial
  • Basic
    • What is Kafka?
    • Kafka Applications
    • Kafka Version
    • Kafka Use Cases
    • Kafka Consumer Group
    • Kafka Tools
    • Kafka Architecture
    • Kafka MirrorMaker
    • Kafka Console Producer
    • Kafka Console Consumer
    • Kafka Node
    • Kafka Listener
    • Kafka Cluster
    • Kafka Partition
    • Kafka Event
    • Kafka Replication
    • Kafka Monitoring
    • Kafka Zookeeper
    • Kafka Connect
    • Kafka Partition Key
    • Kafka Topic
    • Kafka burrow
    • Kafka Delete Topic
    • Kafka Replication Factor
    • Kafka Interview Questions
    • Kafka Alternatives
    • Kafka Queue
    • Kafka message
    • Kafka offset
    • Kafka Manager
    • Kafka Rebalance
    • Kafka Port
    • Kafka JDBC Connector
    • Kafka Security
    • Kafka confluent
    • Kafka Consumer
    • Kafka Producer
    • Kafka Client
    • Kafka Producer Config
    • Kafka Exporter
    • Kafka WebSocket
    • Kafka REST Proxy
    • Kafka Protocol
    • Kafka Producer Example

Related Courses

All in One Data Science Course

Pig Certification Course

Scala Certification Course

Kafka Listener

By Priya PedamkarPriya Pedamkar

Kafka Listener

Introduction to Kafka Listener

The Kafka Listener is work on the publish and subscribe model. The Apache Kafka is nothing but a massaging protocol. The Kafka broker will receive the number of messages by the Kafka topics. We need to set the listener configuration correctly. While working with the Kafka listeners, we need to set the “advertised.listeners” property. With the help of this property, the external client will able to connect correctly. If the listener will not set with the current configuration then it will try to connect with the internal host only. In the normal count of Kafka broker i.e. 3 then there is no issues will arrive. But when the network or the Kafka architecture ( multiple brokers ) is too complex then we need to define the correct property of the “advertised.listeners” property.

Syntax of Kafka Listener

As such, there is no specific syntax available. Generally, we are using the configuration command for the Kafka Listener.

  • We are using the core Kafka commands and Kafka listener command for the troubleshooting front.
  • At the time of Kafka Listener configuration; we are using the CLI method. But generally, we are using the UI tool only.

How Kafka Listener Works?

We need to majorly focus on the server.properties configuration. We need to majorly work on the below properties like

  • KAFKA_LISTENERS
  • KAFKA_ADVERTISED_LISTENERS
  • KAFKA _ LISTENER _ SECURITY _ PROTOCOL _ MAP

In the Kafka config, the KAFKA_LISTENERS is nothing but a comma separated list of listeners. In this, there is a combination of hostname, IP address and ports. It will help for the Kafka bind for the listener. When we are dealing with the complex network and multiple we need to set the default is 0.0.0.0 i.e. listing on all the present interfaces.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

All in One Data Science Bundle(360+ Courses, 50+ projects)
Python TutorialMachine LearningAWSArtificial Intelligence
TableauR ProgrammingPowerBIDeep Learning
Price
View Courses
360+ Online Courses | 50+ projects | 1500+ Hours | Verifiable Certificates | Lifetime Access
4.7 (86,408 ratings)
  • We need to set the listeners value.

In the Kafka config, the KAFKA _ ADVERTISED _ LISTENERS is a comma-separated list of listeners. There is a combination of hostname, IP address and ports. The KAFKA _ ADVERTISED _ LISTENERS is the metadata that’s passed back to clients.

  • We need to set the advertised. listeners value

In the Kafka config, the KAFKA _ LISTENER _ SECURITY _ PROTOCOL _ MAP will define the key and value pairs for the security protocol. It will use per listener name.

  • We need to set the listener.security.protocol.map value

Below are the lists of modifier, element type that are compatible with the Kafka Listener.

Modifier and Type Optional Element Explanation
java.lang.String autoStartup We can set the value of this property is true or false. We are able to change the default value in the container factory config.
java.lang.String beanRef In SpEL expression, the pseudo bean name used within the annotation. With this property, the current bean value listener is defined.
java.lang.String clientIdPrefix With the help of this value, we can override, provided the client id property in the consumer factory config.
java.lang.String concurrency It will help to override the container factory’s concurrency setting in the listener config.
java.lang.String containerFactory In the Kafka Listener ContainerFactory, the bean name to use the create message listener container. It will help to serve this endpoint.
java.lang.String containerGroup If we have configured this value, the listener container will be added to a bean (of type Collection ( <MessageListenerContainer> )
java.lang.String errorHandler If we have received an exception then we need to set the Kafka ListenerErrorHandler property/bean name value property.
java.lang.String groupId In the consumer factory, this option will help to Override the group.id property.
java.lang.String id With the help of this property, The unique identifier of the container for this listener.
boolean idIsGroup When the groupId is not provided, use the boolean idIsGroup as the group.id property for the Kafka consumer.
java.lang.String[] properties In the Kafka consumer properties, we can add the java.lang.String[]. In the consumer factory, it will succeed any properties with the same name defined in the configuration. It is a Boolean splitIterables ( if the consumer factory supports property overrides ).

When the value of this value is false then it an Iterable return the result.

TopicPartition[] topicPartitions The TopicPartition[] value is useful to define the listener when it will use the manual topic or the partition assignment.
java.lang.String topicPattern This property is useful for the topic pattern for this listener.
java.lang.String[] topics This property is useful for the topics for this listener.

Examples to Implement or Check Kafka Listener

Following are the examples are given below:

1. Check the Status of Kafka Listener

In the Kafka environment, it plays an important role in message delivery. We can configure the Kafka environment in a different way i.e. the single node Kafka environment or the multi-node Kafka environment. As per the environment, the Kafka listener value will be change ( with the combination of hostname and port ).

Note: The default port of the Kafka listener in the cluster mode is “6667”.

Command:

lsof -i TCP:6667

Explanation: As per the above command, we are able to listen to the Kafka listener on “6667” port.

Output:

Kafka Listener

2. Change the Kafka Listener Configuration

In Kafka, the listener configuration is present by default. As per the requirement and the needed configuration, we can change the Kafka listener properties.

Command: We need to do the Kafka listener configuration from Ambari UI and change the listeners property.

Explanation: As per Screenshot 1 ( A ), we are having the default configuration of Kafka listeners with port no “6667”. In Screenshot 1 ( B ), we are changing the hostname and the port no i.e. hostname “10.10.132.152” & port “6667”. After doing changes in the Kafka listener, we are not able to listen to the previous port “6667” ( Please refer Screenshot 1 ( C ). In Screenshot 1 ( D ), we are able to listen on the Kafka listener on the new port i.e. “6670”.

Output:

Kafka Listener - configuration

Screenshot 1 ( A )

Kafka Listener - configuration 1

Screenshot 1 ( B )

lsof -i TCP:6667

Kafka Listener-1.4-1

Screenshot 1 ( C )

lsof -i TCP:6670

Kafka Listener-1.4-2

Screenshot 1 ( D )

Conclusion

We have seen the uncut concept of “Kafka Listener” with the proper example, explanation and methods with different outputs. As per the requirement, we can do the changes in the Kafka Listener. It is very important to get the messages from the Kafka topics ( with the help of producer as well as consumer ).

Recommended Articles

This is a guide to Kafka Listener. Here we also discuss the introduction and how kafka listener works? along with different examples and its code implementation. You may also have a look at the following articles to learn more –

  1. ZeroMQ vs RabbitMQ
  2. ZeroMQ vs Kafka
  3. Kafka Listener
  4. RabbitMQ Alternatives
Popular Course in this category
Apache Kafka Training (1 Course, 1 Project)
  1 Online Courses |  1 Hands-on Project |  7+ Hours |  Verifiable Certificate of Completion
4.5
Price

View Course

Related Courses

All in One Data Science Bundle (360+ Courses, 50+ projects)4.9
Apache Pig Training (2 Courses, 4+ Projects)4.8
Scala Programming Training (3 Courses,1Project)4.7
0 Shares
Share
Tweet
Share
Primary Sidebar
Footer
About Us
  • Blog
  • Who is EDUCBA?
  • Sign Up
  • Live Classes
  • Corporate Training
  • Certificate from Top Institutions
  • Contact Us
  • Verifiable Certificate
  • Reviews
  • Terms and Conditions
  • Privacy Policy
  •  
Apps
  • iPhone & iPad
  • Android
Resources
  • Free Courses
  • Database Management
  • Machine Learning
  • All Tutorials
Certification Courses
  • All Courses
  • Data Science Course - All in One Bundle
  • Machine Learning Course
  • Hadoop Certification Training
  • Cloud Computing Training Course
  • R Programming Course
  • AWS Training Course
  • SAS Training Course

ISO 10004:2018 & ISO 9001:2015 Certified

© 2022 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA
Free Data Science Course

SPSS, Data visualization with Python, Matplotlib Library, Seaborn Package

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA Login

Forgot Password?

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA
Free Data Science Course

Hadoop, Data Science, Statistics & others

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

Let’s Get Started

By signing up, you agree to our Terms of Use and Privacy Policy.

This website or its third-party tools use cookies, which are necessary to its functioning and required to achieve the purposes illustrated in the cookie policy. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to our Privacy Policy

Loading . . .
Quiz
Question:

Answer:

Quiz Result
Total QuestionsCorrect AnswersWrong AnswersPercentage

Explore 1000+ varieties of Mock tests View more