Class: Karafka::Pro::Routing::Features::ParallelSegments
- Inherits:
-
Base
- Object
- Routing::Features::Base
- Base
- Karafka::Pro::Routing::Features::ParallelSegments
- Defined in:
- lib/karafka/pro/routing/features/parallel_segments.rb,
lib/karafka/pro/routing/features/parallel_segments/topic.rb,
lib/karafka/pro/routing/features/parallel_segments/config.rb,
lib/karafka/pro/routing/features/parallel_segments/builder.rb,
lib/karafka/pro/routing/features/parallel_segments/consumer_group.rb,
lib/karafka/pro/routing/features/parallel_segments/contracts/consumer_group.rb
Overview
Feature that allows parallelizing message processing within a single consumer group by creating multiple consumer group instances. It enables processing messages from each partition in parallel by distributing them to separate consumer group instances based on a partitioning key. Useful for both CPU and IO bound operations.
Each parallel segment operates as an independent consumer group instance, processing messages that are assigned to it based on the configured partitioner and reducer. This allows for better resource utilization and increased processing throughput without requiring changes to the topic’s partition count.
Defined Under Namespace
Modules: Builder, ConsumerGroup, Contracts, Topic Classes: Config