Skip to content

Kafka Cluster Configuration

Kafka Configuration Variability

The defaults and exact list of cluster configuration options may differ between various Kafka versions. For the most accurate information, please refer to the documentation for the specific Kafka version.

Names Default Value Read-only Sensitive Description
advertised.listeners PLAINTEXT://127.0.0.1:9092 Listeners to publish to ZooKeeper for clients to use, if different than the listeners config property.
allow.everyone.if.no.acl.found Indicates whether a request should be allowed if no ACL is found for the resource.
alter.config.policy.class.name A class name that implements the AlterConfigPolicy interface, used for custom policies when altering configurations.
alter.log.dirs.replication.quota.window.num 11 Number of samples to use for tracking alter log dirs replication quotas.
alter.log.dirs.replication.quota.window.size.seconds 1 Time span for each sample used for tracking alter log dirs replication quotas.
authorizer.class.name org.apache.kafka.metadata.authorizer.StandardAuthorizer Class name of the authorizer used for ACL-based authorization.
auto.create.topics.enable true Enable auto creation of topics.
auto.include.jmx.reporter true Enable automatic inclusion of JMX reporter.
auto.leader.rebalance.enable true Enable auto leader balancing.
background.threads 10 Number of threads to use for various background processing tasks.
broker.heartbeat.interval.ms 2000 The interval at which the broker sends heartbeats to ZooKeeper.
broker.id 1 Unique ID of the broker.
broker.id.generation.enable true Enable automatic generation of broker IDs.
broker.rack Rack ID of the broker.
broker.session.timeout.ms 9000 The timeout used to detect broker failures when using ZooKeeper.
client.quota.callback.class A class name that implements the ClientQuotaCallback interface.
compression.type producer Specify the compression type for data generated by the broker.
connection.failed.authentication.delay.ms 100 Time in milliseconds to delay an authentication failure.
connections.max.idle.ms 600000 Idle time after which connections are closed.
connections.max.reauth.ms 0 Maximum time to delay re-authentication of connections.
control.plane.listener.name Listener name for the control plane.
controlled.shutdown.enable true Enable controlled shutdown of the broker.
controlled.shutdown.max.retries 3 Maximum number of retries for controlled shutdown.
controlled.shutdown.retry.backoff.ms 5000 Backoff time between retries during controlled shutdown.
controller.listener.names CONTROLLER Listener names for the controller.
controller.quorum.append.linger.ms 25 Time in milliseconds to delay appending to the controller quorum log.
controller.quorum.election.backoff.max.ms 1000 Maximum backoff time in milliseconds for controller quorum election.
controller.quorum.election.timeout.ms 1000 Timeout in milliseconds for controller quorum election.
controller.quorum.fetch.timeout.ms 2000 Timeout in milliseconds for fetching from the controller quorum.
controller.quorum.request.timeout.ms 2000 Timeout in milliseconds for controller quorum requests.
controller.quorum.retry.backoff.ms 20 Backoff time in milliseconds between retries for controller quorum operations.
controller.quorum.voters 1@127.0.0.1:9093 Voters in the controller quorum.
controller.quota.window.num 11 Number of samples to use for tracking controller quotas.
controller.quota.window.size.seconds 1 Time span for each sample used for tracking controller quotas.
controller.socket.timeout.ms 30000 Socket timeout for controller connections.
create.topic.policy.class.name A class name that implements the CreateTopicPolicy interface.
default.replication.factor 1 Default replication factor for automatically created topics.
delegation.token.expiry.check.interval.ms 3600000 Interval in milliseconds for delegation token expiry checks.
delegation.token.expiry.time.ms 86400000 Time in milliseconds for delegation token expiry.
delegation.token.master.key Master key for signing delegation tokens.
delegation.token.max.lifetime.ms 604800000 Maximum lifetime for delegation tokens.
delegation.token.secret.key Secret key for signing delegation tokens.
delete.records.purgatory.purge.interval.requests 1 Interval in requests to purge the delete records purgatory.
delete.topic.enable true Enable topic deletion.
early.start.listeners Listeners to start early during broker startup.
fetch.max.bytes 57671680 Maximum bytes fetched per request.
fetch.purgatory.purge.interval.requests 1000 Interval in requests to purge the fetch purgatory.
group.initial.rebalance.delay.ms 3000 Initial rebalance delay for consumer groups.
group.max.session.timeout.ms 1800000 Maximum session timeout for consumer groups.
group.max.size 2147483647 Maximum size of consumer groups.
group.min.session.timeout.ms 6000 Minimum session timeout for consumer groups.
initial.broker.registration.timeout.ms 60000 Timeout for initial broker registration.
inter.broker.listener.name PLAINTEXT Listener name for inter-broker communication.
inter.broker.protocol.version 3.6-IV2 Protocol version for inter-broker communication.
kafka.metrics.polling.interval.secs 10 Polling interval for Kafka metrics.
kafka.metrics.reporters List of classes implementing the MetricsReporter interface.
leader.imbalance.check.interval.seconds 300 Interval in seconds to check for leader imbalance.
leader.imbalance.per.broker.percentage 10 Percentage of imbalance tolerated per broker.
listener.security.protocol.map CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT Mapping of listener names to security protocols.
listeners PLAINTEXT://:9092,CONTROLLER://:9093 Listeners for broker connections.
log.cleaner.backoff.ms 15000 Backoff time between log cleaner operations.
log.cleaner.dedupe.buffer.size 134217728 Size of the buffer used for log cleaning.
log.cleaner.delete.retention.ms 86400000 Retention time for deleted records.
log.cleaner.enable true Enable log cleaner.
log.cleaner.io.buffer.load.factor 0.9 Load factor for the log cleaner IO buffer.
log.cleaner.io.buffer.size 524288 Size of the log cleaner IO buffer.
log.cleaner.io.max.bytes.per.second 1.7976931348623157E308 Maximum bytes per second for log cleaner IO.
log.cleaner.max.compaction.lag.ms 9223372036854775807 Maximum compaction lag for log cleaner.
log.cleaner.min.cleanable.ratio 0.5 Minimum cleanable ratio for log cleaner.
log.cleaner.min.compaction.lag.ms 0 Minimum compaction lag for log cleaner.
log.cleaner.threads 1 Number of threads for log cleaner.
log.cleanup.policy delete Policy for log cleanup.
log.dir /tmp/kafka-logs Directory for log files.
log.dirs /var/lib/kafka/data Directories for log files.
log.flush.interval.messages 9223372036854775807 Interval in messages for log flush.
log.flush.interval.ms Interval in milliseconds for log flush.
log.flush.offset.checkpoint.interval.ms 60000 Interval in milliseconds for log offset checkpoint.
log.flush.scheduler.interval.ms 9223372036854775807 Interval in milliseconds for log flush scheduler.
log.flush.start.offset.checkpoint.interval.ms 60000 Interval in milliseconds for log flush start offset checkpoint.
log.index.interval.bytes 4096 Interval in bytes for log index.
log.index.size.max.bytes 10485760 Maximum size in bytes for log index.
log.local.retention.bytes -2 Local retention size in bytes for log.
log.local.retention.ms -2 Local retention time in milliseconds for log.
log.message.downconversion.enable true Enable downconversion of log messages.
log.message.format.version 3.0-IV1 Format version for log messages.
log.message.timestamp.after.max.ms 9223372036854775807 Maximum timestamp difference for log messages.
log.message.timestamp.before.max.ms 9223372036854775807 Maximum timestamp before log messages.
log.message.timestamp.difference.max.ms 9223372036854775807 Maximum timestamp difference for log messages.
log.message.timestamp.type CreateTime Type of timestamp for log messages.
log.preallocate false Preallocate log segments.
log.retention.bytes -1 Retention size in bytes for log.
log.retention.check.interval.ms 300000 Interval in milliseconds for log retention check.
log.retention.hours 168 Retention time in hours for log.
log.retention.minutes Retention time in minutes for log.
log.retention.ms Retention time in milliseconds for log.
log.roll.hours 168 Roll interval in hours for log segments.
log.roll.jitter.hours 0 Jitter for log roll interval in hours.
log.roll.jitter.ms Jitter for log roll interval in milliseconds.
log.roll.ms Roll interval in milliseconds for log segments.
log.segment.bytes 1073741824 Size of log segments in bytes.
log.segment.delete.delay.ms 60000 Delay in milliseconds before deleting log segments.
max.connection.creation.rate 2147483647 Maximum rate of connection creation.
max.connections 2147483647 Maximum number of connections.
max.connections.per.ip 2147483647 Maximum number of connections per IP.
max.connections.per.ip.overrides Overrides for maximum connections per IP.
max.incremental.fetch.session.cache.slots 1000 Maximum cache slots for incremental fetch sessions.
message.max.bytes 1048588 Maximum size of messages.
metadata.log.dir Directory for metadata log.
metadata.log.max.record.bytes.between.snapshots 20971520 Maximum bytes between metadata log snapshots.
metadata.log.max.snapshot.interval.ms 3600000 Maximum interval between metadata log snapshots.
metadata.log.segment.bytes 1073741824 Size of metadata log segments in bytes.
metadata.log.segment.ms 604800000 Interval in milliseconds for metadata log segments.
metadata.max.idle.interval.ms 500 Maximum idle interval for metadata.
metadata.max.retention.bytes 104857600 Maximum retention size for metadata.
metadata.max.retention.ms 604800000 Maximum retention time for metadata.
metric.reporters List of metric reporter classes.
metrics.num.samples 2 Number of metric samples.
metrics.recording.level INFO Recording level for metrics.
metrics.sample.window.ms 30000 Sample window for metrics.
min.insync.replicas 1 Minimum in-sync replicas.
node.id 1 ID of the node.
num.io.threads 8 Number of IO threads.
num.network.threads 3 Number of network threads.
num.partitions 1 Number of partitions.
num.recovery.threads.per.data.dir 1 Number of recovery threads per data directory.
num.replica.alter.log.dirs.threads Number of threads for altering replica log dirs.
num.replica.fetchers 1 Number of replica fetchers.
offset.metadata.max.bytes 4096 Maximum bytes for offset metadata.
offsets.commit.required.acks -1 Required acknowledgments for offset commits.
offsets.commit.timeout.ms 5000 Timeout for offset commits.
offsets.load.buffer.size 5242880 Buffer size for loading offsets.
offsets.retention.check.interval.ms 600000 Interval for checking offset retention.
offsets.retention.minutes 10080 Retention time in minutes for offsets.
offsets.topic.compression.codec 0 Compression codec for offsets topic.
offsets.topic.num.partitions 50 Number of partitions for offsets topic.
offsets.topic.replication.factor 1 Replication factor for offsets topic.
offsets.topic.segment.bytes 104857600 Segment size for offsets topic.
password.encoder.cipher.algorithm AES/CBC/PKCS5Padding Cipher algorithm for password encoder.
password.encoder.iterations 4096 Iterations for password encoder.
password.encoder.key.length 128 Key length for password encoder.
password.encoder.keyfactory.algorithm Key factory algorithm for password encoder.
password.encoder.old.secret Old secret for password encoder.
password.encoder.secret Secret for password encoder.
principal.builder.class org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder Class name of the principal builder.
process.roles broker,controller Roles of the process.
producer.id.expiration.ms 86400000 Expiration time for producer IDs.
producer.purgatory.purge.interval.requests 1000 Purge interval for producer purgatory.
queued.max.request.bytes -1 Maximum request bytes in the queue.
queued.max.requests 500 Maximum number of requests in the queue.
quota.window.num 11 Number of samples for quota tracking.
quota.window.size.seconds 1 Time span for each quota sample.
remote.log.manager.task.interval.ms 30000 Interval for remote log manager tasks.
remote.log.manager.thread.pool.size 10 Thread pool size for remote log manager.
remote.log.metadata.custom.metadata.max.bytes 128 Maximum custom metadata bytes for remote log metadata.
remote.log.metadata.manager.class.name org.apache.kafka.server.log.remote.metadata.storage.TopicBasedRemoteLogMetadataManager Class name of the remote log metadata manager.
remote.log.metadata.manager.class.path Class path for the remote log metadata manager.
remote.log.metadata.manager.impl.prefix rlmm.config. Prefix for remote log metadata manager implementation.
remote.log.metadata.manager.listener.name Listener name for the remote log metadata manager.
remote.log.reader.max.pending.tasks 100 Maximum pending tasks for remote log reader.
remote.log.reader.threads 10 Number of threads for remote log reader.
remote.log.storage.manager.class.name Class name of the remote log storage manager.
remote.log.storage.manager.class.path Class path for the remote log storage manager.
remote.log.storage.manager.impl.prefix rsm.config. Prefix for remote log storage manager implementation.
remote.log.storage.system.enable false Enable remote log storage system.
replica.fetch.backoff.ms 1000 Backoff time for replica fetch.
replica.fetch.max.bytes 1048576 Maximum bytes for replica fetch.
replica.fetch.min.bytes 1 Minimum bytes for replica fetch.
replica.fetch.response.max.bytes 10485760 Maximum response bytes for replica fetch.
replica.fetch.wait.max.ms 500 Maximum wait time for replica fetch.
replica.high.watermark.checkpoint.interval.ms 5000 Interval for high watermark checkpointing.
replica.lag.time.max.ms 30000 Maximum lag time for replica.
replica.selector.class Class name of the replica selector.
replica.socket.receive.buffer.bytes 65536 Receive buffer size for replica socket.
replica.socket.timeout.ms 30000 Timeout for replica socket.
replication.quota.window.num 11 Number of samples for replication quota.
replication.quota.window.size.seconds 1 Time span for each replication quota sample.
request.timeout.ms 30000 Timeout for requests.
reserved.broker.max.id 1000 Maximum reserved broker ID.
sasl.client.callback.handler.class Class name of the SASL client callback handler.
sasl.enabled.mechanisms GSSAPI Enabled SASL mechanisms.
sasl.jaas.config JAAS configuration for SASL.
sasl.kerberos.kinit.cmd /usr/bin/kinit Kerberos kinit command.
sasl.kerberos.min.time.before.relogin 60000 Minimum time before Kerberos relogin.
sasl.kerberos.principal.to.local.rules DEFAULT Rules for Kerberos principal to local mapping.
sasl.kerberos.service.name Kerberos service name.
sasl.kerberos.ticket.renew.jitter 0.05 Jitter for Kerberos ticket renewal.
sasl.kerberos.ticket.renew.window.factor 0.8 Window factor for Kerberos ticket renewal.
sasl.login.callback.handler.class Class name of the SASL login callback handler.
sasl.login.class Class name of the SASL login module.
sasl.login.connect.timeout.ms Connection timeout for SASL login.
sasl.login.read.timeout.ms Read timeout for SASL login.
sasl.login.refresh.buffer.seconds 300 Buffer time for SASL login refresh.
sasl.login.refresh.min.period.seconds 60 Minimum period for SASL login refresh.
sasl.login.refresh.window.factor 0.8 Window factor for SASL login refresh.
sasl.login.refresh.window.jitter 0.05 Jitter for SASL login refresh window.
sasl.login.retry.backoff.max.ms 10000 Maximum backoff for SASL login retry.
sasl.login.retry.backoff.ms 100 Backoff time for SASL login retry.
sasl.mechanism.controller.protocol GSSAPI SASL mechanism for controller protocol.
sasl.mechanism.inter.broker.protocol GSSAPI SASL mechanism for inter-broker protocol.
sasl.oauthbearer.clock.skew.seconds 30 Clock skew for OAuthBearer.
sasl.oauthbearer.expected.audience Expected audience for OAuthBearer.
sasl.oauthbearer.expected.issuer Expected issuer for OAuthBearer.
sasl.oauthbearer.jwks.endpoint.refresh.ms 3600000 JWKS endpoint refresh interval for OAuthBearer.
sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms 10000 Maximum backoff for JWKS endpoint retry.
sasl.oauthbearer.jwks.endpoint.retry.backoff.ms 100 Backoff time for JWKS endpoint retry.
sasl.oauthbearer.jwks.endpoint.url JWKS endpoint URL for OAuthBearer.
sasl.oauthbearer.scope.claim.name scope Scope claim name for OAuthBearer.
sasl.oauthbearer.sub.claim.name sub Sub claim name for OAuthBearer.
sasl.oauthbearer.token.endpoint.url Token endpoint URL for OAuthBearer.
sasl.server.callback.handler.class Class name of the SASL server callback handler.
sasl.server.max.receive.size 524288 Maximum receive size for SASL server.
security.inter.broker.protocol PLAINTEXT Security protocol for inter-broker communication.
security.providers List of security providers.
socket.connection.setup.timeout.max.ms 30000 Maximum setup timeout for socket connection.
socket.connection.setup.timeout.ms 10000 Setup timeout for socket connection.
socket.listen.backlog.size 50 Listen backlog size for socket.
socket.receive.buffer.bytes 102400 Receive buffer size for socket.
socket.request.max.bytes 104857600 Maximum request size for socket.
socket.send.buffer.bytes 102400 Send buffer size for socket.
ssl.cipher.suites Cipher suites for SSL.
ssl.client.auth none Client authentication setting for SSL.
ssl.enabled.protocols TLSv1.2,TLSv1.3 Enabled protocols for SSL.
ssl.endpoint.identification.algorithm https Endpoint identification algorithm for SSL.
ssl.engine.factory.class Class name of the SSL engine factory.
ssl.key.password Password for SSL key.
ssl.keymanager.algorithm SunX509 Key manager algorithm for SSL.
ssl.keystore.certificate.chain Certificate chain for SSL keystore.
ssl.keystore.key Key for SSL keystore.
ssl.keystore.location Location of the SSL keystore.
ssl.keystore.password Password for SSL keystore.
ssl.keystore.type JKS Type of the SSL keystore.
ssl.principal.mapping.rules DEFAULT Principal mapping rules for SSL.
ssl.protocol TLSv1.3 Protocol for SSL.
ssl.provider Provider for SSL.
ssl.secure.random.implementation Implementation of secure random for SSL.
ssl.trustmanager.algorithm PKIX Trust manager algorithm for SSL.
ssl.truststore.certificates Certificates for SSL truststore.
ssl.truststore.location Location of the SSL truststore.
ssl.truststore.password Password for SSL truststore.
ssl.truststore.type JKS Type of the SSL truststore.
transaction.abort.timed.out.transaction.cleanup.interval.ms 10000 Interval for cleaning up timed out transactions.
transaction.max.timeout.ms 900000 Maximum timeout for transactions.
transaction.partition.verification.enable true Enable partition verification for transactions.
transaction.remove.expired.transaction.cleanup.interval.ms 3600000 Interval for cleaning up expired transactions.
transaction.state.log.load.buffer.size 5242880 Buffer size for loading transaction state logs.
transaction.state.log.min.isr 1 Minimum in-sync replicas for transaction state logs.
transaction.state.log.num.partitions 50 Number of partitions for transaction state logs.
transaction.state.log.replication.factor 1 Replication factor for transaction state logs.
transaction.state.log.segment.bytes 104857600 Segment size for transaction state logs.
transactional.id.expiration.ms 604800000 Expiration time for transactional IDs.
unclean.leader.election.enable false Enable unclean leader election.
zookeeper.clientCnxnSocket Client connection socket for ZooKeeper.
zookeeper.connect ZooKeeper connection string.
zookeeper.connection.timeout.ms Connection timeout for ZooKeeper.
zookeeper.max.in.flight.requests 10 Maximum in-flight requests for ZooKeeper.
zookeeper.metadata.migration.enable false Enable metadata migration to ZooKeeper.
zookeeper.session.timeout.ms 18000 Session timeout for ZooKeeper.
zookeeper.set.acl false Set ACL for ZooKeeper.
zookeeper.ssl.cipher.suites Cipher suites for ZooKeeper SSL.
zookeeper.ssl.client.enable false Enable ZooKeeper SSL for clients.
zookeeper.ssl.crl.enable false Enable CRL for ZooKeeper SSL.
zookeeper.ssl.enabled.protocols Enabled protocols for ZooKeeper SSL.
zookeeper.ssl.endpoint.identification.algorithm HTTPS Endpoint identification algorithm for ZooKeeper SSL.
zookeeper.ssl.keystore.location Location of the keystore for ZooKeeper SSL.
zookeeper.ssl.keystore.password Password for the keystore for ZooKeeper SSL.
zookeeper.ssl.keystore.type Type of the keystore for ZooKeeper SSL.
zookeeper.ssl.ocsp.enable false Enable OCSP for ZooKeeper SSL.
zookeeper.ssl.protocol TLSv1.2 Protocol for ZooKeeper SSL.
zookeeper.ssl.truststore.location Location of the truststore for ZooKeeper SSL.
zookeeper.ssl.truststore.password Password for the truststore for ZooKeeper SSL.
zookeeper.ssl.truststore.type Type of the truststore for ZooKeeper SSL.

Legend

  • Names: The name of the parameter or setting with its synonyms.
  • Default Value: The initial value assigned to the parameter if not explicitly set.
  • Read-Only: Indicates if the parameter is immutable and cannot be modified.
  • Sensitive: Specifies if the parameter contains sensitive information that will not be accessible or visible using Karafka.
  • Description: A detailed explanation of the parameter's purpose and usage.