[//]: # (This file is auto-generated by bin/refresh_remote_content)
[//]: # (Do not edit manually - changes will be overwritten)

# WaterDrop changelog

## WaterDrop changelog / 2.10.8 (2026-10-25)
- [Maintenance] Document that a transaction timeout also raises `:fenced`, indistinguishable from a takeover, and when removing `:fenced` from `non_reloadable_errors` is safe.
- [Maintenance] Drop unused `byebug` from the test bundle (Gemfile, Gemfile.lock), aligning with karafka (#3432) and karafka-web (#1354).
- [Maintenance] Fix test races and skip the fork test on JRuby, resolving unstable behavior in `test/` only.
- [Fix] Stop `Producer#close` from raising `Errno::EPIPE` on JRuby when its close signal races the poller closing the pipe.
- [Fix] Abort a failed transaction only once: raise an abortable commit error as is instead of a follow-up `Operation not valid in state Ready` error, and do not abort after a failed `begin_transaction`.

## WaterDrop changelog / 2.10.7 (2026-10-19)
- [Change] Require `karafka-rdkafka` `>=` `0.31.0` so delivery reports carry the message persistence status, latency and broker id.
- [Maintenance] Stop the transaction purge instrumentation spec from flaking when a slow runner lets librdkafka deliver the message before the abort, which leaves nothing to purge.

## WaterDrop changelog / 2.10.6 (2026-10-12)
- **[EOL]** Drop Ruby 3.2 support; require Ruby 3.3+.
- [Maintenance] Document that a middleware chain run is not atomic, so an in-place step followed by a raising step is applied again on the next `#flush`.
- [Maintenance] Stop the sync delivery error spec from flaking when librdkafka rejects the produce to the invalid topic inline instead of through the delivery report.

## WaterDrop changelog / 2.10.5 (2026-09-23)
- [Fix] Stop `#flush` from running middleware twice on messages re-buffered after a failed flush, which double-processed payloads and duplicated headers on retry (regression of #474).
- [Fix] Raise `ProducerClosedError` instead of silently losing a message buffered while the producer is closing.
- [Fix] Restore both buffers when `#flush` fails before anything is dispatched, instead of discarding the whole batch.

## WaterDrop changelog / 2.10.4 (2026-08-26)
- [Fix] Avoid a `FrozenError` on `Producer#close` when the producer is configured with a frozen string id (for example a frozen string literal, `config.id = "rspec"`).
- [Fix] Emit `connection_pool.shutdown` only once when shutting down the global connection pool.
- [Fix] Retry a produce that races a concurrent idempotent fatal-error reload instead of leaking a raw `Rdkafka::ClosedProducerError`/`Rdkafka::ClosedInnerError`.
- [Maintenance] Stop the `#partition_count when topic does not exist` spec from flaking on slow CI runners by waiting for authoritative broker metadata before asserting the count.

## WaterDrop changelog / 2.10.3 (2026-07-15)
- [Feature] Add `wait_timeout_on_transaction_abort` (disabled by default) to avoid a fatal `INVALID_TXN_STATE` on abort ([librdkafka#4849](https://github.com/confluentinc/librdkafka/issues/4849)). When enabled, the awaited message is delivered as aborted instead of purged.

## WaterDrop changelog / 2.10.2 (2026-06-15)
- [Feature] Expose `Producer#current_variant`, returning the variant active for the current dispatch, for use in middleware and instrumentation listeners.
- [Enhancement] Stop allocating one interpolated string per message in `LoggerListener` batch produce handlers.
- [Enhancement] Use `Array#concat` in `Producer#buffer_many` instead of appending messages one by one.
- [Enhancement] Skip building the `message.acknowledged` instrumentation payload in the delivery callback when nothing is subscribed to that event.
- [Enhancement] Resolve the fiber-local variant once per `#produce` call and once per `#produce_many_sync` wait phase instead of re-resolving it for every usage.
- [Enhancement] Do not allocate the fiber-local variants hash on the `Producer#current_variant` read path when variants are never used.
- [Enhancement] Cache the variant validation contract in a constant instead of instantiating a new `Contracts::Variant` on every `Producer#with` / `Producer#variant` call.
- [Enhancement] Cache the tombstone validation contract in a constant instead of instantiating a new `Contracts::Tombstone` per tombstone message.
- [Enhancement] Replace explicit `Warning[:performance]` opt-in with a dynamic approach using `Warning.categories` to enable all stable opt-in warning categories in the test suite.
- [Fix] Prevent a deadlock between a transactional single-message dispatch and `#close`.
- [Fix] Prevent a deadlock when closing an idempotent producer with `reload_on_idempotent_fatal_error` enabled whose final flush hits a fatal error.
- [Fix] Make concurrent idempotent fatal-error reload thread-safe so a second thread's reload can no longer raise `NoMethodError` after the first reset `@client`.
- [Fix] Stop `#flush_async` / `#flush_sync` from silently dropping valid buffered messages when the dispatch fails; unsent messages are now re-buffered so they can be retried instead of being lost.
- [Fix] Make `Producer#close` fork-safe so the GC finalizer inherited by a forked child can no longer flush and close the parent's client.
- [Fix] Prevent a concurrent flush, purge or close from dropping a message buffered with `#buffer` or `#buffer_many`.
- [Fix] Stop a nested variant call on the same producer from overriding the outer variant inside a variant `transaction` block.
- [Fix] Stop `ConnectionPool#shutdown` and `#reload` from dropping in-flight messages by closing producers gracefully. Use `pool.shutdown(force: true)` for the previous force-and-purge behavior.
- [Fix] Fix a race that could leave a newly registered producer permanently unpolled by the FD poller.

## WaterDrop changelog / 2.10.1 (2026-05-25)
- [Fix] Prevent `Producer#close` from raising `ThreadError: can't be called from trap context` when called from a signal trap (e.g. Puma's `after_stopped` hook in single mode). It now closes from a background thread and waits for it (#866).

## WaterDrop changelog / 2.10.0 (2026-05-07)
- [Fix] Clean up the native client, instrumentation callbacks and poller registration when `init_transactions` fails during producer construction, so failed attempts no longer leak threads and file descriptors.
- **[Breaking]** Skip librdkafka statistics when nothing subscribes to `statistics.emitted` before the rdkafka client is built, to save allocations. To use statistics, subscribe a listener BEFORE the first producer use.
- **[Breaking]** Raise `WaterDrop::Errors::StatisticsNotEnabledError` when subscribing to `statistics.emitted` on a monitor whose librdkafka statistics were disabled at client build time, instead of silently doing nothing.
- [Feature] Add tombstone API (`#tombstone_sync`, `#tombstone_async`, `#tombstone_many_sync`, `#tombstone_many_async`) for producing tombstone records (nil-payload messages) with required key and partition validation. Works with variants.
- [Fix] Add `ensure_same_process!` to `Poller#unregister` for fork safety, so a child that inherited a pre-fork producer no longer deadlocks on `producer.close`.

## WaterDrop changelog / 2.9.0 (2026-04-08)
- [Fix] Use `delete` in the variant ensure block to avoid leaving stale nil entries in `Fiber.current.waterdrop_clients` and prevent memory leaks in long-running processes (#836).
- [Fix] Exclude test files, `.github/`, and `log/` directories from gem releases to reduce package size.
- **[Breaking]** Switch default polling mode from `:thread` to `:fd`. If you experience any issues, you can revert to the previous behavior by setting `config.polling.mode = :thread`. The `:thread` mode will be deprecated in 2.10 and removed in 2.11.
- **[Breaking]** Decorate only the statistics keys used by the built-in Datadog listener (`tx`, `txretries`, `txerrs`, `rxerrs`). If you use other `_d` or `_fd` keys, set a custom `config.statistics_decorator`.
- [Feature] Add a `config.statistics_decorator` setting to use a custom `StatisticsDecorator` (e.g. with different `only_keys` and `excluded_keys`).
- [Change] Upscale default timeout values 3x closer to librdkafka defaults to prevent intermediate timeouts during node recovery (`message.timeout.ms`: 50s → 150s, `transaction.timeout.ms`: 55s → 165s, `max_wait_timeout`: 60s → 180s).

**Upgrade Notes**: https://karafka.io/docs/Upgrades-WaterDrop-2.9/

## WaterDrop changelog / 2.8.16 (2026-02-25)
- [Feature] Add FD-based polling (`config.polling.mode = :fd`): a single Ruby thread with `IO.select` instead of the default thread mode, with 39-54% higher throughput, lower memory usage and fewer threads.
- [Feature] Add `config.polling.fd.max_time` setting (default: 100ms) to control maximum polling time per producer per cycle. This enables per-producer priority differentiation.
- [Feature] Add `Producer#queue_size` (alias: `#queue_length`) to return the count of messages dispatched to librdkafka but not yet sent to the Kafka broker.
- [Enhancement] Add `poll_nb` and `poll_drain` methods to rdkafka Producer for efficient non-blocking polling without GVL release overhead.
- [Enhancement] Add `poller.producer_registered` and `poller.producer_unregistered` instrumentation events for FD mode.
- [Change] Update to use `max_wait_timeout_ms` parameter in rdkafka `AbstractHandle#wait` for consistency with librdkafka conventions.
- [Change] Require `karafka-rdkafka` `>=` `0.24.0` due to `max_wait_timeout_ms` API change and FD polling support.

## WaterDrop changelog / 2.8.15 (2025-11-24)
- [Enhancement] Skip statistics decoration and emission when no listeners are subscribed to `statistics.emitted` events to reduce overhead.
- [Enhancement] Support late subscription to `statistics.emitted` by checking for listeners on each emission (every 5 seconds).

## WaterDrop changelog / 2.8.14 (2025-11-14)
- [Fix] Fatal error replacement not working without exact error unwind from the fatal envelope
- [Testing] Add `WaterDrop::Producer::Testing` module for injecting and querying librdkafka fatal errors in tests.
- [Testing] Add `#trigger_test_fatal_error(error_code, reason)` method to simulate fatal errors without requiring actual broker-side conditions.
- [Testing] Add `#fatal_error` method to query current fatal error state for validation in tests.
- [Testing] Add comprehensive test coverage for fatal error injection, reload behavior, and event instrumentation with real librdkafka errors.
- [Change] Require `karafka-rdkafka` `>=` `0.23.1` due to error handling fixes.

## WaterDrop changelog / 2.8.13 (2025-10-31)
- [Enhancement] Make `fenced` error skip-reload behavior configurable via new `non_reloadable_errors` setting (defaults to `[:fenced]` for backward compatibility).
- [Enhancement] Add `producer.reload` event allowing config modification before reload to escape fencing loops (#706).
- [Enhancement] Do not early initialize the new instance on reload.

## WaterDrop changelog / 2.8.12 (2025-10-10)
- [Enhancement] Introduce `reload_on_idempotent_fatal_error` to automatically reload librdkafka producer after fatal errors on idempotent (non-transactional) producers.
- [Enhancement] Add backoff and retry limits for fatal error reloads to prevent infinite reload loops (defaults in brackets): `wait_backoff_on_idempotent_fatal_error` (5000ms), `max_attempts_on_idempotent_fatal_error` (5), `wait_backoff_on_transaction_fatal_error` (1000ms) and `max_attempts_on_transaction_fatal_error` (10).
- [Enhancement] Ensure `error.occurred` is instrumented before idempotent fatal error reload for visibility.
- [Enhancement] Automatically reset fatal error reload attempts counter on successful produce/transaction to allow recovery.
- [Refactor] Extract idempotence-related logic into separate `WaterDrop::Producer::Idempotence` module.
- [Refactor] Initialize `@idempotent` and `@transactional` instance variables in Producer#initialize for consistent Ruby object shapes optimization.
- [Refactor] Add `idempotent_reloadable?` and `idempotent_retryable?` methods to encapsulate idempotent fatal error reload checks.
- [Refactor] Add `transactional_retryable?` method to encapsulate transactional fatal error reload retry checks.
- [Fix] Waterdrop `config.kafka` errors on frozen hash.
- [Fix] `Producer#transactional?` method now correctly computes transactional status when `@transactional` is initialized to nil.

## WaterDrop changelog / 2.8.11 (2025-09-27)
- [Enhancement] Provide fast-track for middleware-less flows (20% faster) for single message, 5000x faster for batches.
- [Enhancement] Optimize middlewares application by around 20%.
- **[EOL]** Remove Ruby `3.1` according to the EOL schedule.
- [Fix] Connection pool timeout parameter now accepts milliseconds instead of seconds for consistency with other WaterDrop timeouts. The default timeout has been changed from `5` seconds to `5000` milliseconds (equivalent value).

## WaterDrop changelog / 2.8.10 (2025-09-25)
- [Enhancement] Add `#close` alias for `WaterDrop::ConnectionPool#shutdown` to align with producer API for consistent interface across both individual producers and connection pools.
- [Enhancement] Add `WaterDrop.monitor` as the preferred alias for `WaterDrop.instrumentation`, matching the per-producer API. `instrumentation` remains as a deprecated alias.

## WaterDrop changelog / 2.8.9 (2025-09-23)
- [Enhancement] Add connection pool lifecycle events to global instrumentation: `connection_pool.created`, `connection_pool.setup`, `connection_pool.shutdown`, `connection_pool.reload` and `connection_pool.reloaded`.
- [Enhancement] Add default connection pool transactional direct API.

## WaterDrop changelog / 2.8.8 (2025-09-23)
- [Feature] Add `WaterDrop::ConnectionPool` for efficient connection pooling using the proven `connection_pool` gem.
- [Feature] Add class-level `WaterDrop.instrumentation` with `producer.created` and `producer.configured` events, so libraries (e.g. Datadog tracing) can configure producers without instance references.
- **[EOL]** Remove Ruby `3.1` specs according to the EOL schedule.

## WaterDrop changelog / 2.8.7 (2025-09-02)
- [Enhancement] Disable Nagle algorithm by default (improves latency / aligned with librdkafka)
- [Change] Normalize how libs and dependencies are required (no functional change for the end user)

## WaterDrop changelog / 2.8.6 (2025-08-18)
- [Feature] Add `idle_disconnect_timeout` config option to automatically disconnect idle producers after a configurable timeout period.
- [Feature] Add support for [async](https://github.com/socketry/async) gems ecosystem with proper fiber yielding during blocking operations.
- [Feature] Add integration testing infrastructure with `bin/integrations` runner for testing external ecosystem compatibility.
- [Enhancement] Introduce the `WaterDrop::Producer#disconnect` so users can write custom logic to save on connections then producer is only used from time to time.
- [Enhancement] Introduce `WaterDrop::Producer#inspect` that is mutex-safe.
- [Enhancement] Raise errors on detected Ruby warnings.
- [Enhancement] Optimize producer for Ruby shapes.
- [Enhancement] Add integration spec to validate fiber yielding behavior with async gems.
- [Change] Require `karafka-rdkafka` `>=` `0.20.0`.
- [Change] Add new CI action to trigger auto-doc refresh.

## WaterDrop changelog / 2.8.5 (2025-06-23)
- [Enhancement] Normalize topic + partition logs format (single place).
- [Fix] A producer is not idempotent unless the enable.idempotence config is `true` (ferrous26).

## WaterDrop changelog / 2.8.4 (2025-05-23)
- [Change] Require `karafka-rdkafka` `>= 0.19.2` due to new partition count caching usage.
- [Change] Move to trusted-publishers and remove signing since no longer needed.

## WaterDrop changelog / 2.8.3 (2025-04-08)
- [Enhancement] Support producing messages with arrays of strings in headers (KIP-82).
- [Refactor] Introduce a `bin/verify_topics_naming` script to ensure proper test topics naming convention.
- [Refactor] Remove factory bot and active support requirement in tests/dev.
- [Change] Require `karafka-rdkafka` `>= 0.19.1` due to KIP-82.

## WaterDrop changelog / 2.8.2 (2025-02-13)
- [Feature] Allow for tagging of producer instances similar to how consumers can be tagged.
- [Refactor] Ensure all test topics in the test suite start with "it-" prefix.
- [Refactor] Introduce a `bin/verify_kafka_warnings` script to clean Kafka from temporary test-suite topics.

## WaterDrop changelog / 2.8.1 (2024-12-26)
- [Enhancement] Raise `WaterDrop::ProducerNotTransactionalError` when attempting to use transactions on a non-transactional producer.
- [Fix] Disallow closing a producer from within a transaction.
- [Fix] WaterDrop should prevent opening a transaction using a closed producer.

## WaterDrop changelog / 2.8.0 (2024-09-16)

This release contains **BREAKING** changes. Make sure to read and apply upgrade notes.

- **[Breaking]** Require Ruby `3.1+`.
- **[Breaking]** Remove ability to abort transactions using `throw(:abort)`. Please use `raise WaterDrop::Errors::AbortTransaction`.
- **[Breaking]** Disallow (similar to ActiveRecord) exiting transactions with `return`, `break` or `throw`.
- **[Breaking]** License changed from MIT to LGPL with an additional commercial option. Note: there is no commercial code in this repository. The commercial license is available for companies unable to use LGPL-licensed software for legal reasons.
- [Enhancement] Make variants fiber safe.
- [Enhancement] In transactional mode do not return any `dispatched` messages as none will be dispatched due to rollback.
- [Enhancement] Align the `LoggerListener` async messages to reflect, that messages are delegated to the internal queue and not dispatched.
- [Fix] Ensure, that `:dispatched` key for `#produce_many_sync` always contains delivery handles (final) and not delivery reports.

## WaterDrop changelog / 2.7.4 (2024-07-04)
- [Maintenance] Alias `WaterDrop::Errors::AbortTransaction` with `WaterDrop::AbortTransaction`.
- [Maintenance] Lower the precision reporting to 100 microseconds in the logger listener.
- [Fix] Consumer consuming error: Local: Erroneous state (state) post break flow in transaction.
- [Change] Require 'karafka-core' `>= 2.4.3`

## WaterDrop changelog / 2.7.3 (2024-06-09)
- [Enhancement] Introduce `reload_on_transaction_fatal_error` to reload the librdkafka after transactional failures
- [Enhancement] Flush on fatal transactional errors.
- [Enhancement] Add topic scope to `report_metric` (YadhuPrakash)
- [Enhancement] Cache middleware reference saving 1 object allocation on each message dispatch.
- [Enhancement] Provide `#idempotent?` similar to `#transactional?`.
- [Enhancement] Provide alias to `#with` named `#variant`.
- [Fix] Prevent from creating `acks` altering variants on idempotent producers.

## WaterDrop changelog / 2.7.2 (2024-05-09)
- [Fix] Fix missing requirement of `delegate` for non-Rails use-cases. Always require delegate for variants usage (samsm)

## WaterDrop changelog / 2.7.1 (2024-05-09)
- **[Feature]** Support context-base configuration with low-level topic settings alterations producer variants.
- [Enhancement] Prefix random default `SecureRandom.hex(6)` producers ids with `waterdrop-hex` to indicate type of object.

## WaterDrop changelog / 2.7.0 (2024-04-26)

This release contains **BREAKING** changes. Make sure to read and apply upgrade notes.

- **[Feature]** Support custom OAuth providers.
- **[Breaking]** Drop Ruby `2.7` support.
- **[Breaking]** Change default timeouts so final delivery `message.timeout.ms` is less that `max_wait_time` so we do not end up with not final verdict.
- **[Breaking]** Update all the time related configuration settings to be in `ms` and not mixed.
- **[Breaking]** Remove no longer needed `wait_timeout` configuration option.
- **[Breaking]** Do **not** validate or morph (via middleware) messages added to the buffer prior to `flush_sync` or `flush_async`.
- [Enhancement] Provide `WaterDrop::Producer#transaction?` that returns only when producer has an active transaction running.
- [Enhancement] Introduce `instrument_on_wait_queue_full` (defaults to `true`) to control whether retryable queue full errors are instrumented in the error pipeline.
- [Enhancement] Protect critical `rdkafka` thread executable code sections.
- [Enhancement] Treat the queue size as a gauge rather than a cumulative stat (isturdy).
- [Fix] Fix a case where purge on non-initialized client would crash.
- [Fix] Middlewares run twice when using buffered produce.
- [Fix] Validations run twice when using buffered produce.

## WaterDrop changelog / 2.6.14 (2024-02-06)
- [Enhancement] Instrument `producer.connected` and `producer.closing` lifecycle events.

## WaterDrop changelog / 2.6.13 (2024-01-29)
- [Enhancement] Expose `#partition_count` for building custom partitioners that need to be aware of number of partitions on a given topic.

## WaterDrop changelog / 2.6.12 (2024-01-03)
- [Enhancement] Provide ability to label message dispatches for increased observability.
- [Enhancement] Provide ability to commit offset during the transaction with a consumer provided.
- [Change] Change transactional message purged error type from `message.error` to `librdkafka.dispatch_error` to align with the non-transactional error type.
- [Change] Remove usage of concurrent ruby.

## WaterDrop changelog / 2.6.11 (2023-10-25)
- [Enhancement] Return delivery handles and delivery report for both dummy and buffered clients with proper topics, partitions and offsets assign and auto-increment offsets per partition.
- [Fix] Fix a case where buffered test client would not accumulate messages on failed transactions

## WaterDrop changelog / 2.6.10 (2023-10-24)
- [Improvement] Introduce the `message.purged` event for a transactional message purged before delivery to Kafka. Non-transactional producers report purges via `error.occurred` instead.
- [Fix] Fix a case where `message.acknowledged` would not have `caller` key.
- [Fix] Fix a bug where critical errors (like `IRB::Abort`) would not abort the ongoing transaction.

## WaterDrop changelog / 2.6.9 (2023-10-23)
- [Improvement] Introduce a `transaction.finished` event to indicate that transaction has finished whether it was aborted or committed.
- [Improvement] Use `transaction.committed` event to indicate that transaction has been committed.

## WaterDrop changelog / 2.6.8 (2023-10-20)
- **[Feature]** Introduce transactions support.
- [Improvement] Expand `LoggerListener` to inform about transactions (info level).
- [Improvement] Allow waterdrop to use topic as a symbol or a string.
- [Improvement] Enhance both `message.acknowledged` and `error.occurred` (for `librdkafka.dispatch_error`) with full delivery_report.
- [Improvement] Provide `#close!` that will force producer close even with outgoing data after the ma wait timeout.
- [Improvement] Provide `#purge` that will purge any outgoing data and data from the internal queues (both WaterDrop and librdkafka).
- [Fix] Fix the `librdkafka.dispatch_error` error dispatch for errors with negative code.

## WaterDrop changelog / 2.6.7 (2023-09-01)
- [Improvement] early flush data from `librdkafka` internal buffer before closing.
- [Maintenance] Update the signing cert as the old one expired.

## WaterDrop changelog / 2.6.6 (2023-08-03)
- [Improvement] Provide `log_messages` option to `LoggerListener` so the extensive messages data logging can disabled.

## WaterDrop changelog / 2.6.5 (2023-07-22)
- [Fix] Add cause to the errors that are passed into instrumentation (konalegi)

## WaterDrop changelog / 2.6.4 (2023-07-11)
- [Improvement] Use original error `#inspect` for `WaterDrop::Errors::ProduceError` and `WaterDrop::Errors::ProduceManyError` instead of the current empty string.

## WaterDrop changelog / 2.6.3 (2023-06-28)
- [Change] Use `Concurrent::AtomicFixnum` to track operations in progress to prevent potential race conditions on JRuby and TruffleRuby (not yet supported but this is for future usage).
- [Change] Require `karafka-rdkafka` `>= 0.13.2`.
- [Change] Require 'karafka-core' `>= 2.1.1`

## WaterDrop changelog / 2.6.2 (2023-06-21)
- [Refactor] Introduce a counter-based locking approach to make sure, that we close the producer safely but at the same time not to limit messages production with producing lock.
- [Refactor] Make private methods private.
- [Refactor] Validate that producer is not closed only when attempting to produce.
- [Refactor] Improve one 5 minute long spec to run in 10 seconds.
- [Refactor] clear client assignment after closing.

## WaterDrop changelog / 2.6.1 (2023-06-19)
- [Refactor] Remove no longer needed patches.
- [Fix] Fork detection on a short lived processes seems to fail. Clear the used parent process client reference not to close it in the finalizer (#356).
- [Change] Require `karafka-rdkafka` `>= 0.13.0`.
- [Change] Require 'karafka-core' `>= 2.1.0`

## WaterDrop changelog / 2.6.0 (2023-06-11)
- [Improvement] Introduce `client_class` setting for ability to replace underlying client with anything specific to a given env (dev, test, etc).
- [Improvement] Introduce `Clients::Buffered` useful for writing specs that do not have to talk with Kafka (id-ilych)
- [Improvement] Make `#produce` method private to avoid confusion and make sure it is not used directly (it is not part of the official API).
- [Change] Change `wait_on_queue_full` from `false` to `true` as a default.
- [Change] Rename `wait_on_queue_full_timeout` to `wait_backoff_on_queue_full` to match what it actually does.
- [Enhancement] Introduce `wait_timeout_on_queue_full`: the time after which the error is raised despite backoff, so never-deliverable messages no longer cause an infinite loop.
- [Fix] Provide `type` for queue full errors that references the appropriate public API method correctly.

## WaterDrop changelog / 2.5.3 (2023-05-26)
- [Enhancement] Include topic name in the `error.occurred` notification payload.
- [Enhancement] Include topic name in the `message.acknowledged` notification payload.
- [Maintenance] Require `karafka-core` `2.0.13`

## WaterDrop changelog / 2.5.2 (2023-04-24)
- [Fix] Require missing Pathname (#345)

## WaterDrop changelog / 2.5.1 (2023-03-09)
- [Feature] Introduce a configurable backoff upon `librdkafka` queue full (false by default).

## WaterDrop changelog / 2.5.0 (2023-03-04)
- [Feature] Pipe **all** the errors including synchronous errors via the `error.occurred`.
- [Improvement] Pipe delivery errors that occurred not via the error callback using the `error.occurred` channel.
- [Improvement] Introduce `WaterDrop::Errors::ProduceError` and `WaterDrop::Errors::ProduceManyError` for any inline raised errors that occur. You can get the original error by using the `#cause`.
- [Improvement] Include `#dispatched` messages handler in the `WaterDrop::Errors::ProduceManyError` error, to be able to understand which of the messages were delegated to `librdkafka` prior to the failure.
- [Maintenance] Remove the `WaterDrop::Errors::FlushFailureError` in favour of correct error that occurred to unify the error handling.
- [Maintenance] Rename `Datadog::Listener` to `Datadog::MetricsListener` to align with Karafka (#329).
- [Fix] Do **not** flush when there is no data to flush in the internal buffer.
- [Fix] Wait on the final data flush for short-lived producers to make sure, that the message is actually dispatched by `librdkafka` or timeout.

## WaterDrop changelog / 2.4.11 (2023-02-24)
- Replace the local rspec locator with generalized core one.
- Make `::WaterDrop::Instrumentation::Notifications::EVENTS` list public for anyone wanting to re-bind those into a different notification bus.

## WaterDrop changelog / 2.4.10 (2023-01-30)
- Include `caller` in the error instrumentation to align with Karafka.

## WaterDrop changelog / 2.4.9 (2023-01-11)
- Remove empty debug logging out of `LoggerListener`.
- Do not lock Ruby version in Karafka in favour of `karafka-core`.
- Make sure `karafka-core` version is at least `2.0.9` to make sure we run `karafka-rdkafka`.

## WaterDrop changelog / 2.4.8 (2023-01-07)
- Use monotonic time from Karafka core.

## WaterDrop changelog / 2.4.7 (2022-12-18)
- Add support to customizable middlewares that can modify message hash prior to validation and dispatch.
- Fix a case where upon not-available leader, metadata request would not be retried
- Require `karafka-core` 2.0.7.

## WaterDrop changelog / 2.4.6 (2022-12-10)
- Set `statistics.interval.ms` to 5 seconds by default, so the defaults cover all the instrumentation out of the box.

## WaterDrop changelog / 2.4.5 (2022-12-10)
- Fix invalid error scope visibility.
- Cache partition count to improve messages production and lower stress on Kafka when `partition_key` is on.

## WaterDrop changelog / 2.4.4 (2022-12-09)
- Add temporary patch on top of `rdkafka-ruby` to mitigate metadata fetch timeout failures.

## WaterDrop changelog / 2.4.3 (2022-12-07)
- Support for librdkafka 0.13
- Update Github Actions
- Change auto-generated id from `SecureRandom#uuid` to `SecureRandom#hex(6)`
- Remove shared components that were moved to `karafka-core` from WaterDrop

## WaterDrop changelog / 2.4.2 (2022-09-29)
- Allow sending tombstone messages (#267)

## WaterDrop changelog / 2.4.1 (2022-08-01)
- Replace local statistics decorator with the one extracted to `karafka-core`.

## WaterDrop changelog / 2.4.0 (2022-07-28)
- Small refactor of the DataDog/Statsd listener to align for future extraction to `karafka-common`.
- Replace `dry-monitor` with home-brew notification layer (API compatible) and allow for usage with `ActiveSupport::Notifications`.
- Remove all the common code into `karafka-core` and add it as a dependency.

## WaterDrop changelog / 2.3.3 (2022-07-18)
- Replace `dry-validation` with home-brew validation layer and drop direct dependency on `dry-validation`.
- Remove indirect dependency on dry-configurable from DataDog listener (no changes required).

## WaterDrop changelog / 2.3.2 (2022-07-17)
- Replace `dry-configurable` with home-brew config and drop direct dependency on `dry-configurable`.

## WaterDrop changelog / 2.3.1 (2022-06-17)
- Update rdkafka patches to align with `0.12.0` and `0.11.1` support.

## WaterDrop changelog / 2.3.0 (2022-04-03)
-  Rename StdoutListener to LoggerListener (#240)

## WaterDrop changelog / 2.2.0 (2022-02-18)
- Add Datadog listener for metrics + errors publishing
- Add Datadog example dashboard template
- Update Readme to show Dd instrumentation usage
- Align the directory namespace convention with gem name (waterdrop => WaterDrop)
- Introduce a common base for validation contracts
- Drop CI support for ruby 2.6
- Require all `kafka` settings to have symbol keys (compatibility with Karafka 2.0 and rdkafka)

## WaterDrop changelog / 2.1.0 (2022-01-03)
- Ruby 3.1 support
- Change the error notification key from `error.emitted` to `error.occurred`.
- Normalize error tracking and make all the places publish errors into the same notification endpoint (`error.occurred`).
- Start semantic versioning WaterDrop.

## WaterDrop changelog / 2.0.7 (2021-12-03)
- Source code metadata url added to the gemspec
- Replace `:producer` with `:producer_id` in events and update `StdoutListener` accordingly. This change aligns all the events in terms of not publishing the whole producer object in the events.
- Add `error.emitted` into the `StdoutListener`.
- Enable `StdoutLogger` in specs for additional integration coverage.

## WaterDrop changelog / 2.0.6 (2021-12-01)
- #218 - Fixes a case, where dispatch of callbacks the same moment a new producer was created could cause a concurrency issue in the manager.
- Fix some unstable specs.

## WaterDrop changelog / 2.0.5 (2021-11-28)

### WaterDrop changelog / 2.0.5 (2021-11-28) / Bug fixes

- Fixes an issue where multiple producers would emit stats of other producers causing the same stats to be published several times (as many times as a number of producers). This could cause invalid reporting for multi-kafka setups.
- Fixes a bug where emitted statistics would contain their first value as the first delta value for first stats emitted.
- Fixes a bug where decorated statistics would include a delta for a root field with non-numeric values.

### WaterDrop changelog / 2.0.5 (2021-11-28) / Changes and features
- Introduces support for error callbacks instrumentation notifications with `error.emitted` monitor emitted key for tracking background errors that would occur on the producer (disconnects, etc).
- Removes the `:producer` key from `statistics.emitted` and replaces it with `:producer_id` not to inject whole producer into the payload
- Removes the `:producer` key from `message.acknowledged` and replaces it with `:producer_id` not to inject whole producer into the payload
- Cleanup and refactor of callbacks support to simplify the API and make it work with Rdkafka way of things.
- Introduces a callbacks manager concept that will also be within in Karafka `2.0` for both statistics and errors tracking per client.
- Sets default Kafka `client.id` to `waterdrop` when not set.
- Updates specs to always emit statistics for better test coverage.
- Adds statistics and errors integration specs running against Kafka.
- Replaces direct `RSpec.describe` reference with auto-discovery
- Patches `rdkafka` to provide functionalities that are needed for granular callback support.

## WaterDrop changelog / 2.0.4 (2021-09-19)
- Update `dry-*` to the recent versions and update settings syntax to match it
- Update Zeitwerk requirement

## WaterDrop changelog / 2.0.3 (2021-09-05)
- Remove rdkafka patch in favour of spec topic pre-creation
- Do not close client that was never used upon closing producer

## WaterDrop changelog / 2.0.2 (2021-08-13)
- Add support for `partition_key`
- Switch license from `LGPL-3.0` to  `MIT`
- Switch flushing on close to sync

## WaterDrop changelog / 2.0.1 (2021-06-05)
- Remove Ruby 2.5 support and update minimum Ruby requirement to 2.6
- Fix the `finalizer references object to be finalized` warning issued with 3.0

## WaterDrop changelog / 2.0.0 (2020-12-13)
- Redesign of the whole API (see `README.md` for the use-cases and the current API)
- Replace `ruby-kafka` with `rdkafka`
- Switch license from `MIT` to `LGPL-3.0`
- #113 - Add some basic validations of the kafka scope of the config (Azdaroth)
- Global state removed
- Redesigned metrics that use `rdkafka` internal data + custom diffing
- Restore JRuby support

## WaterDrop changelog / 1.4.0 (2020-08-25)
- Release to match Karafka 1.4 versioning.

## WaterDrop changelog / 1.3.4 (2020-02-17)
- Support for new `dry-configurable`

## WaterDrop changelog / 1.3.3 (2019-01-06)
- #119 - Support exactly once delivery and transactional messaging (kylekthompson)
- #119 - Support delivery_boy 1.0 (kylekthompson)

## WaterDrop changelog / 1.3.2 (2019-26-12)
- Ruby 2.7.0 support
- Fix missing `delegate` dependency on `ruby-kafka`

## WaterDrop changelog / 1.3.1 (2019-10-21)
- Ruby 2.6.5 support
- Expose setting to optionally verify hostname on ssl certs #109 (tabdollahi)

## WaterDrop changelog / 1.3.0 (2019-09-09)
- Drop Ruby 2.4 support

## WaterDrop changelog / 1.3.0.rc1 (2019-07-31)
- Drop Ruby 2.3 support
- Drop support for Kafka 0.10 in favor of native support for Kafka 0.11.
- Ruby 2.6.3 support
- Support message headers
- `sasl_over_ssl` support
- Unlock Ruby Kafka + provide support for 0.7 only
- #60 - Rename listener to StdoutListener
- Drop support for Kafka 0.10 in favor of native support for Kafka 0.11.
- Support ruby-kafka 0.7
- Support message headers
- `sasl_over_ssl` support
- `ssl_client_cert_key_password` support
- #87 - Make stdout listener as instance
- Use Zeitwerk for gem code loading
- #93 - zstd compression support
- #99 - schemas are renamed to contracts
- Bump delivery_boy (0.2.7 => 0.2.8)

## WaterDrop changelog / 1.2.5
- Bump dependencies to match Karafka
- drop jruby support
- drop ruby 2.2 support

## WaterDrop changelog / 1.2.4
- Due to multiple requests, unlock of 0.7 with an additional post-install message

## WaterDrop changelog / 1.2.3
- Lock ruby-kafka to 0.6 (0.7 support targeted for WaterDrop 1.3)

## WaterDrop changelog / 1.2.2
- #55 - Codec settings unification and config applier

## WaterDrop changelog / 1.2.1
- #54 - compression_codec api sync with king-konf requirements

## WaterDrop changelog / 1.2.0
- #45 - Allow specifying a create time for messages
- #47 - Support SCRAM once released
- #49 - Add lz4 support once merged and released
- #50 - Potential message loss in async mode
- Ruby 2.5.0 support
- Gem bump to match Karafka framework versioning
- #48 - ssl_ca_certs_from_system
- #52 - Use instrumentation compatible with Karafka 1.2

## WaterDrop changelog / 1.0.1
- Added high level retry on connection problems

## WaterDrop changelog / 1.0.0

- #37 - ack level for producer
- Gem bump
- Ruby 2.4.2 support
- Raw ruby-kafka driver is now replaced with delivery_boy
- Sync and async producers
- Complete update of the API
- Much better validations for config details
- Complete API remodel - please read the new README
- Renamed send_messages to deliver

## WaterDrop changelog / 0.4
- Bump to match Karafka
- Renamed ```hosts``` to ```seed_brokers```
- Removed the ```ssl``` scoping for ```kafka``` config namespace to better match Karafka conventions
- Added ```client_id``` option on a root config level
- Added ```logger``` option on a root config level
- Auto Propagation of config down to ruby-kafka

## WaterDrop changelog / 0.3.2
- Removed support for Ruby 2.1.*
- ~~Ruby 2.3.3 as default~~
- Ruby 2.4.0 as default
- Gem dump x2
- Dry configurable config (#20)
- added .rspec for default spec helper require
- Added SSL capabilities
- Coditsu instead of PG dev tools for quality control

## WaterDrop changelog / 0.3.1
- Dev tools update
- Gem update
- Specs updates
- File naming convention fix from waterdrop to water_drop + compatibility file
- Additional params (partition, etc) that can be passed into producer

## WaterDrop changelog / 0.3.0
- Driver change from Poseidon (not maintained) to Ruby-Kafka

## WaterDrop changelog / 0.2.0
- Version dump - this WaterDrop version no longer relies on Aspector to work
- #17 - Logger for Aspector - WaterDrop no longer depends on Aspector
- #8 - add send date as a default value added to a message - wont-fix. Should be implemented on a message level since WaterDrop just transports messages without adding additional stuff.
- #11 - same as above

## WaterDrop changelog / 0.1.13
- Resolved bug #15. When you use waterdrop in aspect way, message will be automatically parse to JSON.

## WaterDrop changelog / 0.1.12
- Removed default to_json casting because of binary/other data types incompatibility. This is an incompatibility. If you use WaterDrop, please add a proper casting method to places where you use it.
- Gem dump

## WaterDrop changelog / 0.1.11
- Poseidon options extractions and tweaks

## WaterDrop changelog / 0.1.10
- Switched raise_on_failure to ignore all StandardError failures (or not to), not just specific once
- Reloading inside connection pool connection that seems to be broken (one that failed) - this should prevent from multiple issues (but not from single one) that are related to the connection

## WaterDrop changelog / 0.1.9
- Required acks and set to -1 (most secure but slower)
- Added a proxy layer to to producer so we could replace Kafka with other messaging systems
- Gem dump

## WaterDrop changelog / 0.1.8
- proper poseidon clients names (not duplicated)

## WaterDrop changelog / 0.1.7
- kafka_host, kafka_hosts and kafka_ports settings consistency fix

## WaterDrop changelog / 0.1.6
- Added null-logger gem

## WaterDrop changelog / 0.1.5
- raise_on_failure flag to ignore (if false) that message was not sent

## WaterDrop changelog / 0.1.4
- Renamed WaterDrop::Event to WaterDrop::Message to follow Apache Kafka naming convention

## WaterDrop changelog / 0.1.3
- Gems cleanup
- Requirements fix

## WaterDrop changelog / 0.1.2
- Initial gem release


---

*Last modified: 2026-10-26 12:54:13*