Admin ACLs (Access Control Lists) API

Apache Kafka ACLs (Access Control Lists) provide a robust mechanism to control permissions and access rights for Kafka resources. They are crucial for ensuring data security, managing consumer and producer interactions, and maintaining overall cluster integrity. Karafka extends these capabilities with a simplified, Ruby-friendly API.

The Karafka Admin ACLs API provides a structured and easy-to-use interface for managing Kafka ACLs. It allows developers to create, delete, and describe ACLs with Ruby symbol-based definitions, enhancing readability and ease of use compared to the direct usage of librdkafka types.

This documentation provides an overview of Kafka ACLs, how to use the ACLs with Karafka, primary use cases, and code samples to get you started.

What are Kafka ACLs?

Kafka ACLs are rules that determine how users and applications can interact with Kafka resources, such as topics, consumer groups, and brokers. Each ACL entry specifies the allowed or denied operations for a particular principal (user or client) on a given resource. Operations can include reading from a topic, writing to a topic, or creating a consumer group.

ACLs ensure that only authorized entities can access Kafka's functionalities, which is vital for maintaining data security and operational integrity.

Types

The Karafka Admin ACLs API defines several "types" that categorize and specify details for ACL management in a Kafka environment. These types are crucial for setting up detailed and secure access controls.

These types play a critical role in the Karafka Admin ACLs API, providing a structured and comprehensive way to manage access controls within a Kafka environment. Understanding and utilizing these types enables you to secure and regulate operations in your Kafka clusters effectively.

Resource Types

Resource types are entities within Kafka for which ACLs can be defined. They specify the scope of the ACL.

Type	Description
`:any`	Used for lookups, not for setting permissions.
`:topic`	Represents a single Kafka topic.
`:consumer_group`	Represents a single Kafka consumer group.
`:broker`	Represents a given Kafka broker.

Resource Pattern Types

Resource pattern types dictate how ACLs are applied to resources, defining the precision and scope of access rules.

Pattern Type	Description
`:any`	For lookups only; not for setting permissions.
`:match`	Targets resources with a pattern matching for broader control.
`:literal`	Applies ACLs to a specifically named resource.
`:prefixed`	Applies ACLs to all resources with a common name prefix.

Operation Types

Operations are the actions that can be permitted or denied on Kafka resources, defining what a user or service can do.

Operation Type	Description
`:any`	For lookups, indicating no specific operation type.
`:all`	Allows all operations (complete access).
`:read`	Grants read access to a resource.
`:write`	Allows writing data to a resource.
`:create`	Permits the creation of resources.
`:delete`	Enables the deletion of resources.
`:alter`	Allows modifications to resources.
`:describe`	Grants the ability to view resource details.
`:cluster_action`	Permits actions related to the Kafka cluster.
`:describe_configs`	Allows viewing configurations for resources.
`:alter_configs`	Enables modification of resource configurations.
`:idempotent_write`	Grants the ability for idempotent writes.

Permission Types

Permission types indicate the nature of the access being granted or denied, essentially determining whether an operation is allowed.

Permission Type	Description
`:any`	Used for lookups, indicating no specific permission type.
`:allow`	Grants the specified operations, enabling actions on the resource.
`:deny`	Blocks the specified operations, preventing actions on the resource.

Usage

When initializing an ACL in Karafka, you'll use several parameters to define these rules. Here's a breakdown of each argument you'll provide:

Argument	Description
`resource_type`	Determines the type of Kafka resource you're securing, such as a topic (`:topic`) or consumer group (`:consumer_group`). You can specify this as a symbol from `RESOURCE_TYPES_MAP` for readability or use a direct numerical type from `rdkafka`. Choose the resource type that aligns with the item you wish to control access to.
`resource_name`	The specific name of the resource, like the name of a topic. This can sometimes be `nil`, mainly when your resource pattern type doesn't require a particular name. Use this to pinpoint the exact resource you're setting the ACL for.
`resource_pattern_type`	Defines how the ACL applies to the resource. You might set it to a literal match for a specific resource or a prefixed pattern to cover a group of resources. This is specified using a symbol from `RESOURCE_PATTERNS_TYPE_MAP` or a direct numerical type.
`principal`	The principal (usually a user or client identity) the ACL is for. This specifies who the ACL will apply to. It can sometimes be `nil` if you're defining a more general rule that isn't principal-specific.
`host`	Indicates the host from which the principal can access the resource. It defaults to `*` (all hosts), but can be set to a specific IP or hostname to restrict access further.
`operation`	What action the principal can or cannot perform, like read, write, or create. This is chosen from the `OPERATIONS_MAP` and can be a descriptive symbol or a numerical type. Select the operation that best fits the action you wish to allow or prevent.
`permission_type`	Specifies whether to allow or deny the operation defined. This is where you enforce the rule, granting or restricting access as needed. Choose 'allow' to enable the operation for the principal or 'deny' to block it.

Creating an ACL

To create an ACL, you need to instantiate a Karafka::Admin::Acl object and use the #create method:

acl = Karafka::Admin::Acl.new(
  resource_type: :topic,
  resource_name: 'my_topic',
  resource_pattern_type: :literal,
  principal: 'user:Bob',
  host: '*',
  operation: :write,
  permission_type: :allow
)

Karafka::Admin::Acl.create(acl)

Deleting an ACL

To delete an ACL, you can use the #delete method with an existing ACL object:

acls = Karafka::Admin::Acl.all

Karafka::Admin::Acl.delete(acls.first)

or you may explicitly define the ACL to remove:

acl = Karafka::Admin::Acl.new(
  resource_type: :topic,
  resource_name: 'my_topic',
  resource_pattern_type: :literal,
  principal: 'user:Bob',
  host: '*',
  operation: :write,
  permission_type: :allow
)

Karafka::Admin::Acl.delete(acl)

Describing ACLs

To retrieve details about existing ACLs, use the #describe method. It returns all ACLs matching the provided criteria:

acl_match = Karafka::Admin::Acl.all.first

acls = Karafka::Admin::Acl.describe(acl_match)

acls.each do |acl|
  puts acl.inspect
end

Listing All ACLs

To list all ACLs within the Kafka cluster, you can use the #all method:

all_acls = Karafka::Admin::Acl.all

all_acls.each do |acl|
  puts acl.inspect
end

Example Use Cases and When to Use It

Topic Access Control: Restrict read/write operations on specific topics to certain users or applications.
Consumer Group Management: Control which principals can create or interact with consumer groups.
Administrative Restriction: Limit who can create, alter, or delete topics within the Kafka cluster.
Security: Ensure that only authorized entities can perform operations, maintaining data integrity and security.
Securing Data: Whenever you need to secure your Kafka data, ensure that only authorized users and services can access or modify it.
Multi-tenant Systems: In systems where multiple users or services interact with Kafka, you must enforce strict access controls.
Compliance and Auditing: Your application must comply with security standards or require auditing capabilities for access and operations.

Summary

Karafka's Admin ACLs API provides a powerful yet user-friendly way to manage Kafka ACLs, ensuring secure and authorized access to Kafka resources. By leveraging Ruby symbols and a structured API, it simplifies the process of ACL management, making it more accessible and less error-prone for Ruby developers.

Whether securing a small project or an enterprise-scale system, understanding and utilizing Kafka ACLs through Karafka can significantly enhance your application's security and data governance.