Architecture

Crabka is a Kafka-compatible system built from one broker core and a set of operational services around it. The broker speaks the Kafka wire protocol, stores data in Kafka-compatible log segments, and uses KRaft for metadata. The operator, Schema Registry, gateway, rebalancer, replicator, and Rust clients connect to that core as ordinary cluster components.

Runtime Map

flowchart TD
  subgraph Kubernetes [Kubernetes control plane]
    Operator[crabka-operator]
    Kafka[Kafka cluster]
    Pool[KafkaNodePool]
    Topic[KafkaTopic]
    User[KafkaUser]
    Rebal[KafkaRebalance]
    SR[SchemaRegistry]
  end
  subgraph Broker [crabka-broker]
    Dispatch[Kafka request dispatch]
    Log[Log segments]
    Groups[Group and transaction coordinators]
    Quorum[KRaft metadata quorum]
  end
  Clients[Kafka clients and kafka-*.sh tools] --> Dispatch
  Dispatch --> Log
  Dispatch --> Groups
  Dispatch <--> Quorum
  Operator --> Kafka
  Operator --> Pool
  Operator --> Topic
  Operator --> User
  Operator --> Rebal
  Operator --> SR
  Operator -.reconciles.-> Broker
  Registry[crabka-schema-registry] --> Dispatch
  Gateway[crabka-grpc-gateway] --> Dispatch
  Rebalancer[crabka-rebalancer] --> Dispatch
  Replicator[crabka-replicator] --> Dispatch

Core Components

Broker. crabka-broker owns Kafka protocol handling, log storage, replication, group coordination, transactions, quotas, authorization, metrics, and the KRaft metadata quorum.

Operator. crabka-operator turns Kubernetes Custom Resources into StatefulSets, Services, ConfigMaps, Secrets, certificates, broker configuration, topic/user operations, rebalances, gateways, and Schema Registry deployments.

Clients and services. The native Rust clients, Schema Registry, gRPC / Connect-RPC gateway, rebalancer, and replicator all use the Kafka-compatible surface rather than private broker internals.

Reference generation. Exact CRD fields, broker settings, topic settings, protocol tables, and consensus failure diagrams are generated from source during the docs build. That keeps reference pages close to implementation reality.

Documentation Map

Use the docs by intent:

• Run locally: Quickstart.
• Run on Kubernetes: Operator Deployment.
• Add schemas: Schema Registry Deployment.
• Build stream processors: Streams and Data Formats.
• Inspect exact generated fields and keys: Reference.
• Browse crate APIs: Rust API.

Generated Reference

The generated reference is organized by component:

• Operator CRDs document Kubernetes resource fields such as Kafka, KafkaNodePool, KafkaUser, KafkaTopic, SchemaRegistry, and KafkaRebalance.
• Broker reference documents server configuration, topic configuration, and supported Kafka protocol APIs.
• Concepts document KRaft failure behavior with simulator-generated diagrams.

Reach for those pages when you need an exact field name, config key, API version, or rustdoc signature.