Operator Deployment

The Crabka Operator turns declarative Kubernetes resources into a running Kafka-compatible cluster. It owns the operational lifecycle: data-directory formatting, broker configuration, Services, StatefulSets, Secrets, certificates, topic/user reconciliation, broker rolls, and rebalances.

Use the operator for real clusters. Use the local Quickstart only for development and evaluation.

How the operator works

The operator is purpose-built for Crabka. It is not a Strimzi plugin or fork. A single operator process can watch the namespaces it is granted and reconcile every Crabka resource it finds.

A Kafka resource owns the cluster. One or more KafkaNodePool resources supply brokers. Each node pool becomes a StatefulSet, and the total broker count is the sum of the pool replica counts. Scale by editing replicas or adding another pool.

Custom Resources

Exact fields are generated into the Operator CRD reference.

Architecture

flowchart TD
  Operator[Operator process] -->|watches| CRDs[Kafka / NodePool / Topic / User / Rebalance / SchemaRegistry]
  CRDs --> Recon[Per-resource reconcilers]
  Recon --> Pools[KafkaNodePool reconciler]
  Pools --> STS[StatefulSets — one broker per pod]
  Recon --> Cluster[Kafka reconciler]
  Cluster --> Svc[Cluster Service]
  Cluster --> CM[ConfigMap]
  Cluster --> Sec[Secrets + CA]

Prerequisites

• A Kubernetes cluster running version 1.28 or higher.
• Helm 3 installed locally.
• kubectl configured to point to your cluster.

1. Install Prometheus CRDs

The operator uses the Prometheus Operator's PodMonitor and ServiceMonitor CRDs for metrics scraping. If they are not already installed, apply them:

PROM_OP_TAG=v0.79.2
kubectl apply -f "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/${PROM_OP_TAG}/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml"
kubectl apply -f "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/${PROM_OP_TAG}/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml"

2. Install Crabka Custom Resource Definitions (CRDs)

Apply the Crabka CRDs from this repository:

kubectl apply -f deploy/crds/

3. Install the operator

Install the operator into its own namespace with the chart in this repository:

helm install operator charts/crabka-operator \
  --namespace crabka-operator \
  --create-namespace

For the published chart repository and provenance verification, see the Helm chart documentation.

Private registries

If your operator or broker images live in a private registry, create a pull secret:

kubectl create secret docker-registry my-registry-secret \
  --docker-server=ghcr.io \
  --docker-username=<username> \
  --docker-password=<token> \
  --namespace=crabka-operator

Then reference it from the chart values:

helm upgrade operator charts/crabka-operator \
  --namespace crabka-operator \
  --set image.tag=0.3.7 \
  --set brokerImage.tag=0.3.7 \
  --set imagePullSecrets[0].name=my-registry-secret

4. Create a cluster

Apply a Kafka resource and a matching KafkaNodePool:

# kafka-cluster.yaml
apiVersion: crabka.io/v1alpha1
kind: Kafka
metadata:
  name: demo
  namespace: default
spec:
  kafkaVersion: "0.3.7"
  config:
    log.retention.hours: "24"
---
apiVersion: crabka.io/v1alpha1
kind: KafkaNodePool
metadata:
  name: brokers
  namespace: default
  labels:
    crabka.io/cluster: demo
spec:
  roles: [Controller, Broker]
  replicas: 1
  nodeIdStart: 0
  storage:
    type: PersistentClaim
    size: 1Gi
    deleteClaim: true
  resources:
    requests:
      cpu: "500m"
      memory: 512Mi
    limits:
      cpu: "2000m"
      memory: 2Gi

Apply the manifest:

kubectl apply -f kafka-cluster.yaml

Wait for the broker pod to become ready:

kubectl rollout status statefulset/demo-brokers --timeout=300s

5. Use the cluster

Use the bootstrap Service created by the operator with standard Kafka clients and tools. The exact Service names depend on the listener configuration in the Kafka resource.

Next:

• Create topics and users with the generated KafkaTopic and KafkaUser resources.
• Add Schema Registry with Schema Registry Deployment.
• Look up broker config keys in Server Configuration.