Skip to content

Rdkafka Changelog

0.14.1 (2023-12-02)

  • [Feature] Add Admin#metadata (mensfeld)
  • [Feature] Add Admin#create_partitions (mensfeld)
  • [Feature] Add Admin#delete_group utility (piotaixr)
  • [Feature] Add Create and Delete ACL Feature To Admin Functions (vgnanasekaran)
  • [Enhancement] Improve error reporting on unknown_topic_or_part and include missing topic (mensfeld)
  • [Enhancement] Improve error reporting on consumer polling errors (mensfeld)

0.14.0 (2023-11-17)

  • [Enhancement] Bump librdkafka to 2.3.0
  • [Enhancement] Increase the #lag and #query_watermark_offsets default timeouts from 100ms to 1000ms. This will compensate for network glitches and remote clusters operations.

0.13.9 (2023-11-07)

  • [Enhancement] Expose alternative way of managing consumer events via a separate queue.
  • [Enhancement] Allow for setting statistics_callback as nil to reset predefined settings configured by a different gem.

0.13.8 (2023-10-31)

  • [Enhancement] Get consumer position (thijsc & mensfeld)

0.13.7 (2023-10-31)

  • [Change] Drop support for Ruby 2.6 due to incompatibilities in usage of ObjectSpace::WeakMap
  • [Fix] Fix dangling Opaque references.

0.13.6 (2023-10-17)

  • [Feature] Support transactions API in the producer
  • [Enhancement] Add raise_response_error flag to the Rdkafka::AbstractHandle.
  • [Enhancement] Provide #purge to remove any outstanding requests from the producer.
  • [Enhancement] Fix #flush does not handle the timeouts errors by making it return true if all flushed or false if failed. We do not raise an exception here to keep it backwards compatible.

0.13.5

  • Fix DeliveryReport create_result#error being nil despite an error being associated with it

0.13.4

  • Always call initial poll on librdkafka to make sure oauth bearer cb is handled pre-operations.

0.13.3

  • Bump librdkafka to 2.2.0

0.13.2

  • Ensure operations counter decrement is fully thread-safe
  • Bump librdkafka to 2.1.1

0.13.1

  • Add offsets_for_times method on consumer (timflapper)

0.13.0 (2023-07-24)

  • Support cooperative sticky partition assignment in the rebalance callback (methodmissing)
  • Support both string and symbol header keys (ColinDKelley)
  • Handle tombstone messages properly (kgalieva)
  • Add topic name to delivery report (maeve)
  • Allow string partitioner config (mollyegibson)
  • Fix documented type for DeliveryReport#error (jimmydo)
  • Bump librdkafka to 2.0.2 (lmaia)
  • Use finalizers to cleanly exit producer and admin (thijsc)
  • Lock access to the native kafka client (thijsc)
  • Fix potential race condition in multi-threaded producer (mensfeld)
  • Fix leaking FFI resources in specs (mensfeld)
  • Improve specs stability (mensfeld)
  • Make metadata request timeout configurable (mensfeld)
  • call_on_partitions_assigned and call_on_partitions_revoked only get a tpl passed in (thijsc)
  • Support #assignment_lost? on a consumer to check for involuntary assignment revocation (mensfeld)
  • Expose #name on the consumer and producer (mensfeld)
  • Introduce producer partitions count metadata cache (mensfeld)
  • Retry metadta fetches on certain errors with a backoff (mensfeld)
  • Do not lock access to underlying native kafka client and rely on Karafka granular locking (mensfeld)

0.12.3

  • Include backtrace in non-raised binded errors.
  • Include topic name in the delivery reports

0.12.2

  • Increase the metadata default timeout from 250ms to 2 seconds. This should allow for working with remote clusters.

0.12.1

  • Bumps librdkafka to 2.0.2 (lmaia)
  • Add support for adding more partitions via Admin API

0.12.0 (2022-06-17)

  • Bumps librdkafka to 1.9.0
  • Fix crash on empty partition key (mensfeld)
  • Pass the delivery handle to the callback (gvisokinskas)

0.11.0 (2021-11-17)

  • Upgrade librdkafka to 1.8.2
  • Bump supported minimum Ruby version to 2.6
  • Better homebrew path detection

0.10.0 (2021-09-07)

  • Upgrade librdkafka to 1.5.0
  • Add error callback config

0.9.0 (2021-06-23)

  • Fixes for Ruby 3.0
  • Allow any callable object for callbacks (gremerritt)
  • Reduce memory allocations in Rdkafka::Producer#produce (jturkel)
  • Use queue as log callback to avoid unsafe calls from trap context (breunigs)
  • Allow passing in topic configuration on create_topic (dezka)
  • Add each_batch method to consumer (mgrosso)

0.8.1 (2020-12-07)

  • Fix topic_flag behaviour and add tests for Metadata (geoff2k)
  • Add topic admin interface (geoff2k)
  • Raise an exception if @native_kafka is nil (geoff2k)
  • Option to use zstd compression (jasonmartens)

0.8.0 (2020-06-02)

  • Upgrade librdkafka to 1.4.0
  • Integrate librdkafka metadata API and add partition_key (by Adithya-copart)
  • Ruby 2.7 compatibility fix (by Geoff TheĢ)A
  • Add error to delivery report (by Alex Stanovsky)
  • Don't override CPPFLAGS and LDFLAGS if already set on Mac (by Hiroshi Hatake)
  • Allow use of Rake 13.x and up (by Tomasz Pajor)

0.7.0 (2019-09-21)

  • Bump librdkafka to 1.2.0 (by rob-as)
  • Allow customizing the wait time for delivery report availability (by mensfeld)

0.6.0 (2019-07-23)

  • Bump librdkafka to 1.1.0 (by Chris Gaffney)
  • Implement seek (by breunigs)

0.5.0 (2019-04-11)

  • Bump librdkafka to 1.0.0 (by breunigs)
  • Add cluster and member information (by dmexe)
  • Support message headers for consumer & producer (by dmexe)
  • Add consumer rebalance listener (by dmexe)
  • Implement pause/resume partitions (by dmexe)

0.4.2 (2019-01-12)

  • Delivery callback for producer
  • Document list param of commit method
  • Use default Homebrew openssl location if present
  • Consumer lag handles empty topics
  • End iteration in consumer when it is closed
  • Add support for storing message offsets
  • Add missing runtime dependency to rake

0.4.1 (2018-10-19)

  • Bump librdkafka to 0.11.6

0.4.0 (2018-09-24)

  • Improvements in librdkafka archive download
  • Add global statistics callback
  • Use Time for timestamps, potentially breaking change if you rely on the previous behavior where it returns an integer with the number of milliseconds.
  • Bump librdkafka to 0.11.5
  • Implement TopicPartitionList in Ruby so we don't have to keep track of native objects.
  • Support committing a topic partition list
  • Add consumer assignment method

0.3.5 (2018-01-17)

  • Fix crash when not waiting for delivery handles
  • Run specs on Ruby 2.5

0.3.4 (2017-12-05)

  • Bump librdkafka to 0.11.3

0.3.3 (2017-10-27)

  • Fix bug that prevent display of RdkafkaError message

0.3.2 (2017-10-25)

  • add_topic now supports using a partition count
  • Add way to make errors clearer with an extra message
  • Show topics in subscribe error message
  • Show partition and topic in query watermark offsets error message

0.3.1 (2017-10-23)

  • Bump librdkafka to 0.11.1
  • Officially support ranges in add_topic for topic partition list.
  • Add consumer lag calculator

0.3.0 (2017-10-17)

  • Move both add topic methods to one add_topic in TopicPartitionList
  • Add committed offsets to consumer
  • Add query watermark offset to consumer

0.2.0 (2017-10-13)

  • Some refactoring and add inline documentation

0.1.x (2017-09-10)

  • Initial working version including producing and consuming