Introduction to ActiveMQ and Kafka
Apache ActiveMQ is an open-source, multi-protocol, Java-based messaging server. It implements the JMS (Java Message Service) API and is able to support various messaging protocols including AMQP, STOMP, and MQTT. It is commonly used for sending messages between applications/services. In this topic, we are going to learn about ActiveMQ vs Kafka.
On the other hand, Apache Kafka is an open-source stream-processing software developed by LinkedIn (and later donated to Apache) to effectively manage their growing data and switch to real-time processing from batch-processing. It is written in Scala and Java and based on the publish-subscribe model of messaging.
Head to Head Comparison Between ActiveMQ and Kafka (Infographics)
Below are the top differences between ActiveMQ vs Kafka
Key Differences Between ActiveMQ and Kafka
ActiveMQ and Kafka are designed for different purposes. Following are the key differences:
Kafka is a distributed streaming platform that offers high horizontal scalability. Also, it provides high throughput and that’s why it’s used for real-time data processing. ActiveMQ is a general-purpose messaging solution that supports various messaging protocols. Kafka is way faster than ActiveMQ. It can handle millions of messages per sec.
ActiveMQ supports both message-queues and publishes/subscribe messaging systems. Kafka, on the other hand, is based on publish/subscribe but does have certain advantages of message-queues.
ActiveMQ guarantees that a message will be delivered, but with Kafka, there is a probability (however low it is) that a message might not get delivered.
4.8 (8,975 ratings)
View Course
Message loss in Kafka can happen in the following scenario:
- It can happen while consuming messages in parallel. Consider a situation where 2 messages come to consumers: X and Y. The two messages are processed in parallel. While processing the messages, Y was successful and committed the offset. However, while handling the message, X produced an error. Considering the message B has a larger offset, Kafka will save the latest offset and the message A never comes back to the consumer.
It’s fairly easier to implement exactly-once message delivery in ActiveMQ than it is in Kafka. Duplicate message delivery in Kafka can happen in the following scenario:
- The consumer has consumed the messages successfully and then committed the messages to its local store, but it crashes and couldn’t commit the offset to Kafka before it has crashed. When the consumer restarts, Kafka will deliver the messages from the last offset.
In Kafka, a message is basically a key-value pair. The payload of the message is the value. Key, on the other hand, is generally used for partitioning purposes and must contain a business-specific key in order to place related messages on the same partition.
In ActiveMQ, the message consists of metadata (headers and properties) and body (which is the payload).
ActiveMQ vs Kafka Comparison Table
Let’s discuss the top 10 difference between ActiveMQ vs Kafka
| ActiveMQ | Kafka |
It is a traditional messaging system that deals with a small amount of data. It has the following use cases:
|
It is a distributed system meant for processing huge amount of data. It has the following use cases:
|
It has transaction support. The two levels of transactions support are:
It uses TransactionStore to handle transactions. TransactionStore will cache all messages and ACKS until commit or rollback occurs. |
Kafka initially didn’t support transactions, but since its 0.11 release, it does support transactions to some extent. |
| It maintains the delivery state of every message resulting in lower throughput. | Kafka producers don’t wait for acknowledgments from the Brokers. So, brokers can write messages at a very high rate resulting in higher throughput |
| In ActiveMQ, it’s the responsibility of the producers to ensure that messages have been delivered. | In Kafka, it’s the responsibility of the consumers to consume all the messages they are supposed to consume. |
| It cannot ensure that messages are received in the same order they were sent. | It can ensure that messages are received in the order they were sent at the partition level. |
| There is something called JMS API message selector, which allows a consumer to specify the messages it is interested in. So, the work of filtering messages is upto the JMS and not the applications. | Kafka doesn’t have any concept of filters at the brokers that can ensure that messages that are picked up by consumers match a certain criterion. The filtering has to be done by the consumers or by the applications. |
| It is a push-type messaging platform where the providers push the messages to the consumers. | It is a pull-type messaging platform where the consumers pull the messages from the brokers. |
| It is not possible to scale horizontally. There is also no concept of replication. | It is highly scalable. Due to replications of partitions, it offers higher availability too. |
| The performance of both queue and topic degrades as the number of consumers rises.
|
It doesn’t slow down with the addition of new consumers. |
| It doesn’t provide checksums to detect corruption of messages out of the box. | It includes checksums to detect corruption of messages in storage and has a comprehensive set of security features. |
Conclusion
We have seen that Kafka and ActiveMQ have different use cases. A company will go for Kafka if it has to process a huge amount of data in real-time and can bear message loss to some extent. Whereas, ActiveMQ would be the proper choice if it cares about one-time delivery and messages are valuable (like in financial transactions).
Recommended Article
This is a guide to ActiveMQ vs Kafka. Here we discuss the key differences with infographics and comparison table. You may also have a look at the following articles to learn more –
