O'Reilly logo

Apache Kafka by Nishant Garg

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Multithreaded consumer for multipartition topics

The previous example is a very basic example of a consumer who consumes messages from a single broker with no explicit partitioning of messages within the topic. Let's jump to the next level and write another program, which consumes messages from multiple partitions connecting to single/multiple topics.

A multithreaded high-level consumer-API-based design is usually based on the number of partitions in the topic and follows a one-to-one mapping approach between the thread and the partitions within the topic. For example, if four partitions are defined for any topic, as a best practice, only four threads should be initiated with the consumer application to read the data; otherwise some conflicting ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required