Kafka

When Remote WAL is enabled, the GreptimeDB cluster writes its write-ahead log to Kafka instead of local disks. Datanodes append WAL entries to Kafka topics, and Metasrv prunes entries that are no longer needed for recovery.

Until a region flushes to object storage, Kafka may hold the only durable copy of those writes. Treat it as a stateful dependency of the database, not as a transport buffer. Getting the retention policy wrong is the most common way to lose data here: Kafka deletes segments on its own schedule, and a WAL entry deleted before GreptimeDB replays it cannot be recovered.

• Deploying Kafka Cluster — installation, verification, monitoring, and uninstallation with the Bitnami Kafka Helm chart.
• Remote WAL Configuration — the Metasrv and Datanode [wal] options, plus the required settings and limitations the Kafka cluster must satisfy.
• Configure Remote WAL — a complete Helm chart example on Kubernetes.