Module: Karafka::Pro::Cleaner

Defined in:
lib/karafka/pro/cleaner.rb,
lib/karafka/pro/cleaner/errors.rb,
lib/karafka/pro/cleaner/messages/message.rb,
lib/karafka/pro/cleaner/messages/messages.rb,
lib/karafka/pro/cleaner/messages/metadata.rb

Overview

Feature that introduces a granular memory management for each message and messages iterator

It allows for better resource allocation by providing an API to clear payload and raw payload from a message after those are no longer needed but before whole messages are freed and removed by Ruby GC.

This can be useful when processing bigger batches or bigger messages one after another and wanting not to have all of the data loaded into memory.

Can yield significant memory savings (up to 80%).

Defined Under Namespace

Modules: Errors, Messages

Class Method Summary collapse

Class Method Details

.post_setup(_config) ⇒ Object

Parameters:

  • _config (Karafka::Core::Configurable::Node) —

    root node config



28
29
30
 
# File 'lib/karafka/pro/cleaner.rb', line 28

def post_setup(_config)
  true
end

.pre_setup(_config) ⇒ Object

Parameters:

  • _config (Karafka::Core::Configurable::Node) —

    root node config



21
22
23
24
25
 
# File 'lib/karafka/pro/cleaner.rb', line 21

def pre_setup(_config)
  ::Karafka::Messages::Message.prepend(Messages::Message)
  ::Karafka::Messages::Metadata.prepend(Messages::Metadata)
  ::Karafka::Messages::Messages.prepend(Messages::Messages)
end