# Enterprise Architecture Consultation

Karafka Enterprise includes **4 hours per year** of dedicated architecture consultation with Maciej Mensfeld, the creator and lead maintainer of Karafka ecosystem. These sessions provide direct access to deep Kafka and Karafka expertise that can help you navigate complex architectural decisions, optimize performance, and avoid common pitfalls that can cost weeks of engineering time.

## Enterprise Architecture Consultation / Why Architecture Consultation Matters

Kafka-based systems present unique challenges that aren't immediately obvious from documentation alone. The difference between a well-architected Kafka implementation and a problematic one often comes down to subtle configuration choices, partition strategies, and processing patterns that only become apparent under production load.

Common issues that seem straightforward but have significant implications include:

- **Message duplication vs. message loss trade-offs** - Understanding delivery guarantees and their real-world impact on your specific use case
- **Partition strategy decisions** - Choosing between fewer partitions with virtual partitioning vs. more physical partitions
- **Consumer group topology** - Balancing connection overhead against processing parallelism
- **Rebalancing behavior** - Minimizing disruption during deployments and scaling events
- **Resource allocation** - Right-sizing thread pools, connection counts, and memory buffers

A single architectural decision made early can save (or cost) your team weeks of debugging and refactoring later.

## Enterprise Architecture Consultation / What I Can Help With

### Enterprise Architecture Consultation / What I Can Help With / Karafka Application Architecture

Designing your Karafka application for maintainability and performance:

- **Routing structure design** - Organizing consumers, topics, and subscription groups for clarity and efficiency
- **Consumer class organization** - Balancing single-responsibility principles with practical code structure
- **Dead Letter Queue design** - Structuring error handling flows and retry strategies
- **Transactional processing patterns** - Coordinating Kafka transactions with database operations
- **Multi-app topology** - Splitting responsibilities across multiple Karafka applications

### Enterprise Architecture Consultation / What I Can Help With / Pro Features Strategy

Maximizing value from Karafka Pro capabilities:

- **Virtual Partitions** - When they help vs. when physical partitions are better for your workload
- **Long-Running Jobs** - Handling jobs that exceed poll intervals without losing progress
- **Enhanced Dead Letter Queue** - Advanced retry patterns and error categorization
- **Scheduled Messages** - Implementing delayed processing and time-based workflows
- **Filtering API** - Building efficient filters for high-volume topics
- **Periodic Jobs** - Scheduling recurring tasks within Karafka
- **Multiplexing and Piping** - Complex message routing between topics
- **Adaptive Iterator** - Processing large batches with memory efficiency

### Enterprise Architecture Consultation / What I Can Help With / Web UI Deployment and Operations

Getting the most out of Karafka's Web UI:

- **Deployment architecture** - Standalone vs. embedded deployment, multi-app visibility
- **Topic configuration optimization** - Retention, compaction, and partition settings for Web UI topics
- **Multi-cluster monitoring** - Configuring Web UI for visibility across multiple Kafka clusters
- **Access control and security** - Authentication and authorization strategies for production dashboards
- **Operational cost optimization** - Minimizing network and storage costs while maintaining visibility
- **Explorer and data inspection** - Using Web UI for debugging and message inspection

### Enterprise Architecture Consultation / What I Can Help With / Integration Patterns

Connecting Karafka with your existing stack:

- **Rails integration** - Proper initialization, connection management, and request/background separation
- **ActiveJob backend** - When to use Karafka as a job backend vs. dedicated message processing
- **ActiveRecord connections** - Managing database connection pools with concurrent consumers
- **Testing strategies** - Unit testing consumers, integration testing with Kafka, and CI/CD setup
- **Monitoring integration** - DataDog, AppSignal, StatsD, and custom instrumentation

### Enterprise Architecture Consultation / What I Can Help With / Performance Optimization

Production systems often exhibit unexpected behavior:

- Latency spikes during batch processing that don't correlate with load
- Consumer lag accumulating despite adequate resources
- Memory growth patterns that don't match expected consumption
- Throughput plateaus that resist straightforward scaling attempts
- Network utilization suggesting inefficient batching or compression

I can review your configuration, identify bottlenecks, and recommend specific tuning parameters based on your workload characteristics.

### Enterprise Architecture Consultation / What I Can Help With / Scaling and Deployment

Planning for growth and production operations:

- **Horizontal scaling** - Partition assignment strategies and consumer group sizing
- **Swarm mode** - Multi-process deployments with copy-on-write optimization
- **Kubernetes patterns** - Container orchestration, health checks, and graceful shutdown
- **Graceful deployments** - Minimizing rebalancing impact during releases
- **Multi-cluster setup** - Disaster recovery and geographic distribution

### Enterprise Architecture Consultation / What I Can Help With / Migration Planning

Moving to Karafka from another framework or upgrading major versions:

- Migration path from Racecar, ruby-kafka, or custom implementations
- Upgrade strategies between Karafka major versions
- Transitioning from OSS to Pro features with minimal disruption
- Rolling deployment approaches that minimize rebalancing impact

## Enterprise Architecture Consultation / How to Use Your Consultation Hours

The 4 hours can be used flexibly throughout your subscription year:

- **Single deep-dive session** - Use all 4 hours for comprehensive architecture review
- **Multiple shorter sessions** - Split into 1-2 hour sessions as questions arise
- **Mixed approach** - Initial 2-hour planning session, with remaining hours for follow-up

Sessions are conducted remotely via video call. You can share your screen to walk through code, configuration, and monitoring dashboards for real-time analysis.

## Enterprise Architecture Consultation / Terms & Conditions

1. **Karafka Enterprise Subscription**: You must have an active paid Karafka Enterprise subscription.
2. **Advance Notice**: Schedule sessions at least one week in advance.
3. **Session Duration**: Minimum 1-hour blocks; 4 hours total per subscription year.
4. **Carryover**: Unused hours do not carry over to the next subscription year.
5. **Scope**: Consultation covers Karafka, WaterDrop, and related Kafka architecture. Custom application development is not included.
6. **Confidentiality**: All discussions and shared materials are treated as confidential.

---

## Enterprise Architecture Consultation / See Also

- [Enterprise](https://karafka.io/docs/Pro-Enterprise.md) - For comprehensive information about Karafka Enterprise
- [Enterprise Workshop Session](https://karafka.io/docs/Pro-Enterprise-Workshop-Session.md) - For details about the included training workshop
- [Support](https://karafka.io/docs/Pro-Support.md) - For details about enterprise support offerings


---

*Last modified: 2026-02-10 13:02:27*