Skip to content

Upgrading to Karafka 2.5

Unreleased Version

Karafka 2.5 has not yet been officially released, and this upgrade guide is a work in progress. The features, interfaces, and migration steps described in this document may change before the final release. This documentation is an early preview to help you prepare for the upcoming changes but should not be used for production migrations until Karafka 2.5 is officially released.

Karafka 2.5 introduces two breaking changes that align naming conventions with other Kafka ecosystem tools, such as Kafka Streams and Apache Flink. These low-risk changes focus on configuration alignment and naming conventions without affecting consumer group names or processing flows.

Pro & Enterprise Upgrade Support

If you're gearing up to upgrade to the latest Karafka version and are a Pro or Enterprise user, remember you've got a dedicated lifeline! Reach out via the dedicated Slack channel for direct support to ensure everything has been covered.

As always, please make sure you have upgraded to the most recent version of 2.3 before upgrading to 2.4.

Also, remember to read and apply our standard upgrade procedures.

DLQ and Piping Header Prefix Change

The prefix for DLQ (Dead Letter Queue) dispatched and piped messages headers has been changed from original_ to source_:

  • original_topic → source_topic
  • original_partition → source_partition
  • original_offset → source_offset
  • original_consumer_group → source_consumer_group
  • original_key → source_key
  • original_attempts → source_attempts

This change aligns Karafka's naming conventions with Kafka Streams and Apache Flink for better ecosystem consistency and future compatibility.

Recurring Tasks Topic Configuration Structure (Pro)

The scheduled jobs topics configuration has been restructured to use a nested format:

Before:

config.recurring_tasks.topics.schedules = "karafka_recurring_tasks_schedules"
config.recurring_tasks.topics.logs = "karafka_recurring_tasks_logs"

After:

config.recurring_tasks.topics.schedules.name = "karafka_recurring_tasks_schedules"
config.recurring_tasks.topics.logs.name = "karafka_recurring_tasks_logs"

This change aligns with the Web UI's topic namespacing pattern of using topic.name.

Impact Assessment

  • Low Risk: These changes are purely naming-related and don't affect consumer group names, processing logic, or system stability.
  • Migration: Simple search and replace operations should be sufficient for most codebases.
  1. Search for uses of original_ in your message header access code and replace with source_
  2. For Pro users, update the recurring tasks topic configuration structure
  3. Run your test suite to verify everything works as expected

These changes enhance future compatibility without significantly reworking your processing flows or message-handling logic.