Resources on Jikkou

Kafka Brokers

Mon, 01 Jan 0001 00:00:00 +0000

This section describes the resource definition format for kafkabrokers entities, which can be used to define the brokers you plan to manage on a specific Kafka cluster.

Listing `KafkaBroker`

You can retrieve the state of Kafka Consumer Groups using the jikkou get kafkabrokers (or jikkou get kb) command.

Usage

Usage:

Get all 'KafkaBroker' resources.

jikkou get kafkabrokers [-hV] [--default-configs] [--dynamic-broker-configs]
 [--list] [--static-broker-configs]
 [--logger-level=<level>] [-o=<format>]
 [-s=<expressions>]...

DESCRIPTION:

Use jikkou get kafkabrokers when you want to describe the state of all
resources of type 'KafkaBroker'.

OPTIONS:

 --default-configs Describe built-in default configuration for configs
 that have a default value.
 --dynamic-broker-configs
 Describe dynamic configs that are configured as
 default for all brokers or for specific broker in
 the cluster.
 -h, --help Show this help message and exit.
 --list Get resources as ResourceListObject.
 --logger-level=<level>
 Specify the log level verbosity to be used while
 running a command.
 Valid level values are: TRACE, DEBUG, INFO, WARN,
 ERROR.
 For example, `--logger-level=INFO`
 -o, --output=<format> Prints the output in the specified format. Allowed
 values: json, yaml (default yaml).
 -s, --selector=<expressions>
 The selector expression used for including or
 excluding resources.
 --static-broker-configs
 Describe static configs provided as broker properties
 at start up (e.g. server.properties file).
 -V, --version Print version information and exit.

(The output from your current Jikkou version may be different from the above example.)

Kafka Consumer Groups

Mon, 01 Jan 0001 00:00:00 +0000

This section describes the resource definition format for KafkaConsumerGroup entities, which can be used to define the consumer groups you plan to manage on a specific Kafka cluster.

Listing `KafkaConsumerGroup`

You can retrieve the state of Kafka Consumer Groups using the jikkou get kafkaconsumergroups (or jikkou get kcg) command.

Usage

$ jikkou get kafkaconsumergroups --help

Usage:

Get all 'KafkaConsumerGroup' resources.

jikkou get kafkaconsumergroups [-hV] [--list] [--offsets]
 [--logger-level=<level>] [-o=<format>]
 [--in-states=PARAM]... [-s=<expressions>]...

DESCRIPTION:

Use jikkou get kafkaconsumergroups when you want to describe the state of all
resources of type 'KafkaConsumerGroup'.

OPTIONS:

 -h, --help Show this help message and exit.
 --in-states=PARAM If states is set, only groups in these states will be
 returned. Otherwise, all groups are returned. This
 operation is supported by brokers with version
 2.6.0 or later
 --list Get resources as ResourceListObject.
 --logger-level=<level>
 Specify the log level verbosity to be used while
 running a command.
 Valid level values are: TRACE, DEBUG, INFO, WARN,
 ERROR.
 For example, `--logger-level=INFO
 -o, --output=<format> Prints the output in the specified format. Allowed
 values: json, yaml (default yaml).
 --offsets Specify whether consumer group offsets should be
 described.
 -s, --selector=<expressions>
 The selector expression used for including or
 excluding resources.
 -V, --version Print version information and exit

(The output from your current Jikkou version may be different from the above example.)

Kafka Topics

Mon, 01 Jan 0001 00:00:00 +0000

KafkaTopic resources are used to define the topics you want to manage on your Kafka Cluster(s). A KafkaTopic resource defines the number of partitions, the replication factor, and the configuration properties to be associated to a topics.

`KafkaTopic`

Specification

Here is the resource definition file for defining a KafkaTopic.

apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaTopic" # The resource kind (required)
metadata:
 name: <The name of the topic>  # (required)
 labels: { }
 annotations: { }
spec:
 partitions: <Number of partitions>  # (optional)
 replicas: <Number of replicas>  # (optional)
 configs:
 <config_key>: <Config Value>  # The topic config properties keyed by name to override (optional)
 configMapRefs: [ ] # The list of ConfigMap to be applied to this topic (optional)

The metadata.name property is mandatory and specifies the name of the kafka topic.

Kafka Users

Mon, 01 Jan 0001 00:00:00 +0000

This section describes the resource definition format for KafkaUser entities, which can be used to manage SCRAM Users for Apache Kafka.

Definition Format of `KafkaUser`

Below is the overall structure of the KafkaUser resource.

---
apiVersion: kafka.jikkou.io/v1  # The api version (required)
kind: KafkaUser  # The resource kind (required)
metadata:
 name: <string>
 annotations:
 # force update
 kafka.jikkou.io/force-password-renewal: <boolean>
spec:
 authentications:
 - type: <enum> # or 
 password: <string>  # leave empty to generate secure password

See below for details about all these fields.

Kafka Share Groups

Mon, 01 Jan 0001 00:00:00 +0000

This section describes the resource definition format for KafkaShareGroup entities, which can be used to manage the share groups (Queues, KIP-932) of a specific Kafka cluster.

Broker requirement

Share groups require a Kafka 4.x cluster with share groups enabled on the brokers (group.share.enable=true and the share.version feature). On older clusters the share group operations are not available.

Listing `KafkaShareGroup`

You can retrieve the state of Kafka Share Groups using the jikkou get kafkasharegroups (or jikkou get ksg) command.

Kafka Authorizations

Mon, 01 Jan 0001 00:00:00 +0000

KafkaPrincipalAuthorization resources are used to define Access Control Lists (ACLs) for principals authenticated to your Kafka Cluster.

Jikkou can be used to describe all ACL policies that need to be created on Kafka Cluster

`KafkaPrincipalAuthorization`

Specification

---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalAuthorization"
metadata:
 name: "User:Alice"
spec:
 roles: [ ] # List of roles to be added to the principal (optional)
 acls: # List of KafkaPrincipalACL (required)
 - resource:
 type: <The type of the resource>  # (required)
 pattern: <The pattern to be used for matching resources>  # (required) 
 patternType: <The pattern type>  # (required) 
 type: <The type of this ACL>  # ALLOW or DENY (required) 
 operations: [ ] # Operation that will be allowed or denied (required) 
 host: <HOST>  # IP address from which principal will have access or will be denied (optional)

For more information on how to define authorization and ACLs, see the official Apache Kafka documentation: Security

Kafka Quotas

Mon, 01 Jan 0001 00:00:00 +0000

KafkaClientQuota resources are used to define the quota limits to be applied on Kafka consumers and producers. A KafkaClientQuota resource can be used to apply limit to consumers and/or producers identified by a client-id or a user principal.

`KafkaClientQuota`

Specification

Here is the resource definition file for defining a KafkaClientQuota.

apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaClientQuota" # The resource kind (required)
metadata: # (optional)
 labels: { }
 annotations: { }
spec:
 type: <The quota type> # (required) 
 entity:
 clientId: <The id of the client>  # (required depending on the quota type).
 user: <The principal of the user>  # (required depending on the quota type).
 configs:
 requestPercentage: <The quota in percentage (%) of total requests>  # (optional)
 producerByteRate: <The quota in bytes for restricting data production>  # (optional)
 consumerByteRate: <The quota in bytes for restricting data consumption> # (optional)

Quota Types

The list below describes the supported quota types:

Kafka Table Records

Mon, 01 Jan 0001 00:00:00 +0000

A KafkaTableRecord resource can be used to produce a key/value record into a given compacted topic, i.e., a topic with cleanup.policy=compact (a.k.a. KTable).

`KafkaTableRecord`

Specification

Here is the resource definition file for defining a KafkaTableRecord.

apiVersion: "kafka.jikkou.io/v1beta1" # The api version (required)
kind: "KafkaTableRecord" # The resource kind (required)
metadata:
 labels: { }
 annotations: { }
spec:
 type: <string>  # The topic name (required) 
 headers: # The list of headers
 - name: <string>
 value: <string>
 key: # The record-key (required)
 type: <string> # The record-key type. Must be one of: BINARY, STRING, JSON (required)
 data: # The record-key in JSON serialized form.
 $ref: <url or path>  # Or an url to a local file containing the JSON string value.
 value: # The record-value (required)
 type: <string> # The record-value type. Must be one of: BINARY, STRING, JSON (required)
 data: # The record-value in JSON serialized form.
 $ref: <url or path>  # Or an url to a local file containing the JSON string value.

Usage

The KafkaTableRecord resource has been designed primarily to manage reference data published and shared via Kafka. Therefore, it is highly recommended to use this resource only with compacted Kafka topics containing a small amount of data.

Resources on Jikkou

Kafka Brokers

Listing KafkaBroker

Usage

Kafka Consumer Groups

Listing KafkaConsumerGroup

Usage

Kafka Topics

KafkaTopic

Specification

Kafka Users

Definition Format of KafkaUser

Kafka Share Groups

Broker requirement

Listing KafkaShareGroup

Kafka Authorizations

KafkaPrincipalAuthorization

Specification

Kafka Quotas

KafkaClientQuota

Specification

Quota Types

Kafka Table Records

KafkaTableRecord

Specification

Usage

Listing `KafkaBroker`

Listing `KafkaConsumerGroup`

`KafkaTopic`

Definition Format of `KafkaUser`

Listing `KafkaShareGroup`

`KafkaPrincipalAuthorization`

`KafkaClientQuota`

`KafkaTableRecord`