From 061552ac74d131be9bd8f54496d67a9c2c16866f Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Tue, 17 Mar 2026 14:00:24 -0700 Subject: [PATCH 1/4] feat(DOC-2012): document Kafka 4.x client compatibility and KIP-890 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Explicitly state Kafka 4.x clients are compatible with Redpanda - Add KIP-890 (Transactions Server-Side Defense) to unsupported features: Redpanda does not implement the server-side portion; Kafka 4.x clients fall back to the standard transaction protocol when connecting to Redpanda - Add Apache Kafka® trademark on first mention - Remove "currently" from unsupported features intro - Fix "reach out" phrasing and mid-sentence line break Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 3f7815ded2..2b72b590fc 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -5,11 +5,11 @@ :description: Kafka clients, version 0.11 or later, are compatible with Redpanda. Validations and exceptions are listed. // tag::single-source[] -Redpanda is compatible with Apache Kafka versions 0.11 and later, with specific exceptions noted on this page. +Redpanda is compatible with Apache Kafka® versions 0.11 and later, with specific exceptions noted on this page. == Kafka client compatibility -Clients developed for Kafka versions 0.11 or later are compatible with Redpanda. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. +Clients developed for Kafka versions 0.11 or later are compatible with Redpanda, including Kafka 4.x clients. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. TIP: Redpanda Data recommends always using the latest supported version of a client. @@ -42,12 +42,11 @@ a| Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations below (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). -If you find a client that is not -supported, reach out to the Redpanda team in the community https://redpanda.com/slack[Slack^]. +If you find an unsupported client, contact the Redpanda team in the community https://redpanda.com/slack[Slack^]. == Unsupported Kafka features -Redpanda does not currently support the following Apache Kafka features: +Redpanda does not support the following Kafka features: * Multiple SCRAM mechanisms simultaneously for SASL users; for example, a user having both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user, either `SCRAM-SHA-256` or `SCRAM-SHA-512`. + @@ -66,6 +65,8 @@ ifndef::env-cloud[] + * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] ++ +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^]. From 30ce8138d8a04b0c30ea6f231fe0c7361613550f Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Mon, 23 Mar 2026 18:42:11 -0700 Subject: [PATCH 2/4] style(DOC-2012): apply style guide audit to kafka-clients.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add page-topic-type, personas, and learning objectives metadata - "Redpanda Data recommends" → direct imperative "Always use..." - "limitations below" → "limitations in the next section" - Fix "See the ... guide for details" → "For details, see xref:..." - "you can file an issue" → imperative construction - Break SCRAM bullet semicolon chain into two sentences - (pandaproxy) → monospace code term - "contact the Redpanda team in the community Slack" → cleaner phrasing Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 29 ++++++++++++++++-------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 2b72b590fc..eea2b9d6d0 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -3,15 +3,24 @@ :page-categories: Clients, Development, Kafka Compatibility :pp: {plus}{plus} :description: Kafka clients, version 0.11 or later, are compatible with Redpanda. Validations and exceptions are listed. +:page-topic-type: reference +:personas: developer +:learning-objective-1: Identify which Kafka clients are validated with Redpanda +:learning-objective-2: Identify unsupported Kafka features when integrating with Redpanda // tag::single-source[] Redpanda is compatible with Apache Kafka® versions 0.11 and later, with specific exceptions noted on this page. +Use this reference to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} + == Kafka client compatibility Clients developed for Kafka versions 0.11 or later are compatible with Redpanda, including Kafka 4.x clients. Modern clients auto-negotiate protocol versions or use an earlier protocol version accepted by Redpanda brokers. -TIP: Redpanda Data recommends always using the latest supported version of a client. +TIP: Always use the latest supported version of a Kafka client. The following clients have been validated with Redpanda. @@ -34,29 +43,29 @@ The following clients have been validated with Redpanda. | https://github.com/kafka-rust/kafka-rust[kafka-rust^] | Node.js -a| -* https://kafka.js.org[KafkaJS^] +a| +* https://kafka.js.org[KafkaJS^] * https://github.com/confluentinc/confluent-kafka-javascript[confluent-kafka-javascript^] |=== -Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations below (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). +Clients that have not been validated by Redpanda Data, but use the Kafka protocol, remain compatible with Redpanda subject to the limitations in the next section (particularly those based on librdkafka, such as confluent-kafka-dotnet or confluent-python). -If you find an unsupported client, contact the Redpanda team in the community https://redpanda.com/slack[Slack^]. +If you find an unsupported client, reach out in the https://redpanda.com/slack[Redpanda community Slack^]. == Unsupported Kafka features Redpanda does not support the following Kafka features: -* Multiple SCRAM mechanisms simultaneously for SASL users; for example, a user having both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user, either `SCRAM-SHA-256` or `SCRAM-SHA-512`. +* Multiple SCRAM mechanisms simultaneously for SASL users. For example, a user cannot have both a `SCRAM-SHA-256` and a `SCRAM-SHA-512` credential. Redpanda supports only one SASL/SCRAM mechanism per user: either `SCRAM-SHA-256` or `SCRAM-SHA-512`. + ifndef::env-cloud[] -See the xref:manage:security/authentication.adoc#sasl[Configure Authentication] guide for details. +For details, see xref:manage:security/authentication.adoc#sasl[Configure SASL authentication]. endif::[] ifdef::env-cloud[] -See the xref:security:cloud-authentication.adoc[Authentication] guide for details. +For details, see xref:security:cloud-authentication.adoc[Authentication]. endif::[] -* HTTP Proxy (pandaproxy): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions. +* HTTP Proxy (`pandaproxy`): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions. ifdef::env-cloud[] + * The `delete.retention.ms` topic configuration in Kafka is not supported. Tombstone markers are not removed for topics with a `compact` xref:get-started:config-topics.adoc#change-the-cleanup-policy[cleanup policy]. Redpanda only deletes tombstone markers when topics with a cleanup policy of `compact,delete` have reached their xref:get-started:create-topic.adoc[retention limits]. @@ -68,6 +77,6 @@ endif::[] + * https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. -If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^]. +If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team. // end::single-source[] From 9eeb533f715e6da9ef1ae89ff0fcb3a565d1dbd4 Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Tue, 24 Mar 2026 17:43:39 -0700 Subject: [PATCH 3/4] docs(DOC-2012): explain why KIP-890 is not needed in Redpanda Add context that KIP-890 addresses transaction errors specific to Kafka replication, and that Redpanda Raft-based replication is not susceptible to this issue. Addresses bashtanov review comment. Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index eea2b9d6d0..183da1426a 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -75,7 +75,7 @@ ifndef::env-cloud[] * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] + -* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. Kafka 4.x clients use per-transaction epoch bumping to protect against erroneous transaction commits, but fall back to the standard transaction protocol when connecting to Redpanda. +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. Kafka 4.x clients use per-transaction epoch bumping and fall back to the standard transaction protocol when connecting to Redpanda. If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team. From 3f7a73983ac0ba8cede69e8dc59fdb3927bf5c01 Mon Sep 17 00:00:00 2001 From: Michael Ernest Date: Thu, 26 Mar 2026 15:02:46 -0700 Subject: [PATCH 4/4] fix(DOC-2012): clarify KIP-890 fallback behavior per bashtanov review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Kafka 4.x clients only use per-transaction epoch bumping when the server supports Transactions V2. Since Redpanda doesn't implement V2, clients fall back to the original transaction protocol — epoch bumping doesn't apply in this case. Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/kafka-clients.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index 183da1426a..f40fef0722 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -75,7 +75,7 @@ ifndef::env-cloud[] * Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. endif::[] + -* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. Kafka 4.x clients use per-transaction epoch bumping and fall back to the standard transaction protocol when connecting to Redpanda. +* https://cwiki.apache.org/confluence/display/KAFKA/KIP-890[KIP-890^] (Transactions Server-Side Defense): Redpanda does not implement the server-side portion of KIP-890. KIP-890 addresses a class of transaction errors specific to Kafka's replication model. Redpanda's Raft-based replication is not susceptible to this issue. When connecting to Redpanda, Kafka 4.x clients detect that Transactions V2 is not supported and fall back to the original transaction protocol. Per-transaction epoch bumping is part of Transactions V2 and does not apply in this case. If you find an unsupported feature or incompatibility, https://github.com/redpanda-data/redpanda/issues/new[file an issue^] with the Redpanda team.