Look here for precise, descriptive information: command flags, configuration keys, resource specifications, and provider options. Reference is for looking things up — if you want to learn or accomplish a task, see the Tutorials and How-to Guides.
This is the multi-page printable view of this section. Click here to print.
Reference
- 1: Jikkou CLI
- 1.1: Overview
- 1.2: Configuration
- 1.3: Commands
- 1.3.1: jikkou action
- 1.3.2: jikkou api-extensions
- 1.3.3: jikkou api-resources
- 1.3.4: jikkou apply
- 1.3.5: jikkou config
- 1.3.6: jikkou config current-context
- 1.3.7: jikkou config get-contexts
- 1.3.8: jikkou config set-context
- 1.3.9: jikkou config use-context
- 1.3.10: jikkou config view
- 1.3.11: jikkou create
- 1.3.12: jikkou delete
- 1.3.13: jikkou diff
- 1.3.14: jikkou get
- 1.3.15: jikkou health
- 1.3.16: jikkou patch
- 1.3.17: jikkou prepare
- 1.3.18: jikkou replace
- 1.3.19: jikkou server-info
- 1.3.20: jikkou update
- 1.3.21: jikkou validate
- 2: Jikkou API Server Documentation
- 2.1: Configurations
- 2.1.1: API Server
- 2.1.2: Authentication
- 2.1.2.1: Basic Auth
- 2.1.2.2: JWT
- 2.1.3: CLI Proxy Mode
- 2.2: Jikkou - API References
- 3: Extension Providers
- 3.1: Core Extensions
- 3.1.1: Resource Repositories
- 3.1.1.1: Github Resource Repository
- 3.1.1.2: Local Resource Repository
- 3.1.2: Resources
- 3.1.2.1: ConfigMap
- 3.1.2.2: ValidatingResourcePolicy
- 3.2: Apache Kafka
- 3.2.1: Configuration
- 3.2.2: Resources
- 3.2.2.1: Kafka Brokers
- 3.2.2.2: Kafka Consumer Groups
- 3.2.2.3: Kafka Topics
- 3.2.2.4: Kafka Users
- 3.2.2.5: Kafka Share Groups
- 3.2.2.6: Kafka Authorizations
- 3.2.2.7: Kafka Quotas
- 3.2.2.8: Kafka Table Records
- 3.2.3: Transformations
- 3.2.3.1: KafkaTopicMaxNumPartitions
- 3.2.3.2: KafkaTopicMaxRetentionMs
- 3.2.3.3: KafkaTopicMinInSyncReplicas
- 3.2.3.4: KafkaTopicMinReplicas
- 3.2.3.5: KafkaTopicMinRetentionMs
- 3.2.4: Validations
- 3.2.5: Annotations
- 3.2.6: Actions
- 3.2.6.1: KafkaConsumerGroupsResetOffsets
- 3.2.6.2: KafkaShareGroupsResetOffsets
- 3.2.7: Compatibility
- 3.3: Apache Kafka Connect
- 3.3.1: Configuration
- 3.3.2: Resources
- 3.3.2.1: KafkaConnectors
- 3.3.3: Validations
- 3.3.4: Annotations
- 3.3.5: Labels
- 3.3.6: Actions
- 3.3.6.1: KafkaConnectRestartConnectors
- 3.4: Schema Registry
- 3.4.1: Configuration
- 3.4.2: Resources
- 3.4.2.1: Schema Registry Subjects
- 3.4.3: Validations
- 3.4.4: Annotations
- 3.5: Aiven
- 3.5.1: Configuration
- 3.5.2: Resources
- 3.5.2.1: ACL for Aiven Apache Kafka®
- 3.5.2.2: Quotas for Aiven Apache Kafka®
- 3.5.2.3: ACL for Aiven Schema Registry
- 3.5.2.4: Subject for Aiven Schema Registry
- 3.5.3: Validations
- 3.5.4: Annotations
- 3.6: AWS
- 3.6.1: Configuration
- 3.6.2: Resources
- 3.6.2.1: Schema for AWS Glue Schema Registry
- 3.6.3: Validations
- 3.6.4: Annotations
- 3.7: Apache Iceberg
- 3.7.1: Configuration
- 3.7.2: Resources
- 3.7.2.1: Iceberg Namespace
- 3.7.2.2: Iceberg Table
- 3.7.2.3: Iceberg View
- 3.7.3: Annotations
- 3.8: Confluent Cloud
- 3.8.1: Configuration
- 3.8.2: Resources
- 3.8.2.1: Role Bindings for Confluent Cloud
- 3.8.3: Annotations
1 - Jikkou CLI
Hands-on: Try the Jikkou Get Started tutorials.
The Jikkou CLI (jikkou) is the primary interface for managing infrastructure resources. This reference covers:
- CLI Features — Global options, version check, and shell tab-completion
- CLI Configuration — Contexts, configuration files, and HOCON settings
- Commands — Complete reference for every command
To automate Jikkou in CI/CD, see the Automating guide.
1.1 - Overview
The command line interface to Jikkou is the jikkou command, which accepts a variety of subcommands such as
jikkou apply or jikkou validate.
To view a list of the commands available in your current Jikkou version, run jikkou with no additional arguments:
Usage:
jikkou [-hV] [--logger-level=<level>] [COMMAND]
Jikkou CLI:: A command-line client designed to provide an efficient and easy way to manage, automate, and provision resources.
Find more information at: https://www.jikkou.io/.
OPTIONS:
-h, --help Show this help message and exit.
--logger-level=<level>
Specify the log level verbosity to be used while running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN, ERROR.
For example, `--logger-level=INFO`
-V, --version Print version information and exit.
CORE COMMANDS:
apply Update the resources as described by the resource definition files.
create Create resources from the resource definition files (only non-existing resources will be created).
delete Delete resources that are no longer described by the resource definition files.
diff Show resource changes required by the current resource definitions.
get Display one or many specific resources.
patch Execute all changes for the specified reconciliation mode.
prepare Prepare the resource definition files for validation.
replace Replace all resources.
update Create or update resources from the resource definition files
validate Check whether the resources definitions meet all validation requirements.
SYSTEM MANAGEMENT COMMANDS:
action List/execute actions.
health Print or describe health indicators.
ADDITIONAL COMMANDS:
api-extensions Print the supported API extensions
api-resources Print the supported API resources
config Sets or retrieves the configuration of this client
generate-completion Generate bash/zsh completion script for jikkou.
help Display help information about the specified command.
See 'jikkou --help' for more information about a command.
(The output from your current Jikkou version may be different than the above example.)
For detailed options and usage examples for each command, see the Commands Reference.
Checking Jikkou Version
Run the jikkou --version to display your current installation version:
Jikkou version "0.37.0" 2026-02-17
JVM: 25.0.1 (GraalVM Community Substrate VM 25.0.1+8)
Shell Tab-completion
It is recommended to install the bash/zsh completion script jikkou_completion.
The completion script can be downloaded from the project Github repository:
wget https://raw.githubusercontent.com/streamthoughts/jikkou/main/jikkou_completion -O jikkou_completion
or alternatively, you can run the following command to generate it.
source <(jikkou generate-completion)
1.2 - Configuration
Configuration
To set up the configuration settings used by Jikkou CLI, you will need create a jikkou config file, which is created automatically when you create a configuration context using:
jikkou config set-context <context-name> [--config-file=<config-file>] [--config-props=<config-value>]
By default, the configuration of jikkou is located under the path $HOME/.jikkou/config.
This jikkou config file defines all the contexts that can be used by jikkou CLI.
For example, below is the config file created during the Getting Started.
{
"currentContext": "localhost",
"localhost": {
"configFile": null,
"configProps": {
"provider.kafka.client.bootstrap.servers": "localhost:9092"
}
}
}
Most of the time, a context does not directly contain the configuration properties to be used, but rather points to a
specific HOCON (Human-Optimized Config Object Notation) through the configFile property.
Then, the configProps allows you to override some of the property define by this file.
In addition, if no configuration file path is specified, Jikkou will lookup for an application.conf to
those following locations:
./application.conf$HOME/.jikkou/application.conf
Finally, Jikkou always fallback to a reference.conf file that you can use as a template to define your own configuration.
reference.conf:
jikkou {
# Configure Jikkou Proxy Mode
# proxy {
# url = "http://localhost:8080"
# }
# Configure Extension Providers
extension {
}
# Core
provider.core {
enabled = true
}
# Apache Kafka Provider
provider.kafka {
enabled = true
type = io.jikkou.kafka.KafkaExtensionProvider
config = {
# The default Kafka Client configuration
client {
bootstrap.servers = "localhost:9092"
bootstrap.servers = ${?JIKKOU_DEFAULT_KAFKA_BOOTSTRAP_SERVERS}
}
brokers {
# If 'True'
waitForEnabled = true
waitForEnabled = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_ENABLED}
# The minimal number of brokers that should be alive for the CLI stops waiting.
waitForMinAvailable = 1
waitForMinAvailable = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_MIN_AVAILABLE}
# The amount of time to wait before verifying that brokers are available.
waitForRetryBackoffMs = 1000
waitForRetryBackoffMs = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_RETRY_BACKOFF_MS}
# Wait until brokers are available or this timeout is reached.
waitForTimeoutMs = 60000
waitForTimeoutMs = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_TIMEOUT_MS}
}
}
}
# Schema Registry Provider
provider.schemaregistry {
enabled = true
type = io.jikkou.schema.registry.SchemaRegistryExtensionProvider
config = {
url = "http://localhost:8081"
url = ${?JIKKOU_DEFAULT_SCHEMA_REGISTRY_URL}
}
}
# The default custom transformations to apply on any resources.
transformations = []
# The default custom validations to apply on any resources.
validations = [
{
name = "topicMustHaveValidName"
type = io.jikkou.kafka.validation.TopicNameRegexValidation
priority = 100
config = {
topicNameRegex = "[a-zA-Z0-9\\._\\-]+"
topicNameRegex = ${?VALIDATION_DEFAULT_TOPIC_NAME_REGEX}
}
},
{
name = "topicMustHavePartitionsEqualsOrGreaterThanOne"
type = io.jikkou.kafka.validation.TopicMinNumPartitionsValidation
priority = 100
config = {
topicMinNumPartitions = 1
topicMinNumPartitions = ${?VALIDATION_DEFAULT_TOPIC_MIN_NUM_PARTITIONS}
}
},
{
name = "topicMustHaveReplicasEqualsOrGreaterThanOne"
type = io.jikkou.kafka.validation.TopicMinReplicationFactorValidation
priority = 100
config = {
topicMinReplicationFactor = 1
topicMinReplicationFactor = ${?VALIDATION_DEFAULT_TOPIC_MIN_REPLICATION_FACTOR}
}
}
]
# The default custom reporters to report applied changes.
reporters = [
# Uncomment following lines to enable default kafka reporter
# {
# name = "default"
# type = io.jikkou.kafka.reporter.KafkaChangeReporter
# config = {
# event.source = "jikkou/cli"
# kafka = {
# topic.creation.enabled = true
# topic.creation.defaultReplicationFactor = 1
# topic.name = "jikkou-resource-change-event"
# client = ${jikkou.kafka.client} {
# client.id = "jikkou-reporter-producer"
# }
# }
# }
# }
]
jinja {
enableRecursiveMacroCalls = false
}
}
Managing Contexts
For detailed usage of all configuration commands, see the Commands Reference:
- jikkou config set-context - Create or update a configuration context
- jikkou config get-contexts - List all configured contexts
- jikkou config current-context - Show the current context
- jikkou config use-context - Switch to a different context
- jikkou config view - Show the merged configuration
Tips
To debug the configuration used by Jikkou, you can run the following command:jikkou config view --comments
or jikkou config view --debug1.3 - Commands
This section contains the complete reference for every Jikkou CLI command, including synopsis, options, usage examples, and related commands.
You can also run jikkou <command> --help from the terminal to view built-in help for any command.
Global Options
The following options are available on all commands:
| Flag | Description |
|---|---|
--logger-level=<level> | Specify the log level verbosity. Valid values: TRACE, DEBUG, INFO, WARN, ERROR |
-h, --help | Show help message and exit |
-V, --version | Print version information and exit |
Core Commands
Commands for reconciling and inspecting resources against your target platform.
Reconciliation
These commands apply changes to your target platform based on resource definition files. Each command corresponds to a specific reconciliation mode.
| Command | Mode | Description |
|---|---|---|
| jikkou apply | FULL | Create, update, and delete resources to match the desired state |
| jikkou create | CREATE | Create only non-existing resources |
| jikkou update | UPDATE | Create new and update existing resources (no deletions) |
| jikkou delete | DELETE | Delete resources no longer described in definition files |
| jikkou patch | (explicit) | Reconcile with an explicitly specified mode |
| jikkou replace | Delete and recreate all resources from scratch |
Inspect & Validate
| Command | Description |
|---|---|
| jikkou diff | Preview resource changes without applying them |
| jikkou get | Display one or many resources from the target platform |
| jikkou validate | Check resource definitions against all validation requirements |
| jikkou prepare | Render templates and apply transformations without validating |
System Management Commands
| Command | Description |
|---|---|
| jikkou action | List and execute provider actions |
| jikkou health | Print or describe health indicators for target environments |
| jikkou server-info | Display Jikkou API server information (proxy mode only) |
Configuration & Discovery Commands
| Command | Description |
|---|---|
| jikkou config | Manage CLI configuration contexts |
| jikkou api-resources | List the supported API resource types and their verbs |
| jikkou api-extensions | List and inspect registered API extensions |
1.3.1 - jikkou action
List and execute actions.
Synopsis
The action command manages Jikkou actions. Actions are named operations that can be executed on the target platform.
Like the get command, action uses dynamic subcommands based on the actions available from your configured providers.
Each action is registered as a subcommand under jikkou action, and has an execute subcommand to run it.
jikkou action <action-name> execute [flags]
Examples
# List all available actions (shown as subcommands in help)
jikkou action --help
# Execute an action
jikkou action KafkaConsumerGroupsResetOffsets execute --options topic=my-topic --options group=my-group
# Execute an action with YAML output
jikkou action KafkaConsumerGroupsResetOffsets execute -o YAML
# Execute an action targeting a specific provider
jikkou action KafkaConsumerGroupsResetOffsets execute --provider kafka-prod --options topic=my-topic
Options (execute subcommand)
| Flag | Short | Default | Description |
|---|---|---|---|
--output | -o | YAML | Output format. Valid values: JSON, YAML |
--provider | Select a specific provider instance |
Additional options may be available depending on the action. These are dynamically registered based on the
action’s ApiOptionSpec definitions. Use jikkou api-extensions get <action-name> to view options for a
specific action.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou api-extensions - List and inspect available extensions
- jikkou get - Display resources from the platform
1.3.2 - jikkou api-extensions
Print the supported API extensions.
Synopsis
The api-extensions command lists and inspects the API extensions registered in Jikkou. Extensions include resource
collectors, transformations, validations, actions, health indicators, and controllers.
This command has two subcommands:
api-extensions list- List all extensionsapi-extensions get- Get details about a specific extension
Subcommands
jikkou api-extensions list
Print a summary table of all supported API extensions.
jikkou api-extensions list [flags]
Examples
# List all extensions
jikkou api-extensions list
# List extensions of a specific category
jikkou api-extensions list --category Transformation
# List extensions from a specific provider
jikkou api-extensions list --provider kafka
# List extensions that support a specific resource kind
jikkou api-extensions list --kind KafkaTopic
Options
| Flag | Default | Description |
|---|---|---|
--category | Limit to extensions of the specified category | |
--provider | Limit to extensions of the specified provider | |
--kind | Limit to extensions that support the specified resource kind |
jikkou api-extensions get
Print detailed information about a specific API extension, including its title, description, configuration options, and usage examples.
jikkou api-extensions get <name> [flags]
Examples
# Get details about an extension in WIDE format (default)
jikkou api-extensions get KafkaTopicMaxRetentionMs
# Get details in JSON format
jikkou api-extensions get KafkaTopicMaxRetentionMs -o JSON
# Get details in YAML format
jikkou api-extensions get KafkaTopicMaxRetentionMs -o YAML
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--output | -o | WIDE | Output format. Valid values: JSON, YAML, WIDE |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou api-resources - List available resource types
- jikkou get - Display resources of a given type
1.3.3 - jikkou api-resources
Print the supported API resources.
Synopsis
List the API resources supported by the Jikkou CLI or Jikkou API Server (in proxy mode). This command shows the resource name, short names, API version, kind, and supported verbs for each resource type.
Use this command to discover which resource types are available with your current configuration and providers.
jikkou api-resources [flags]
Examples
# List all available API resources
jikkou api-resources
# List resources for a specific API group
jikkou api-resources --api-group kafka.jikkou.io
# List resources that support specific verbs
jikkou api-resources --verbs LIST,GET
# List resources that support CREATE
jikkou api-resources --verbs CREATE
# Combine filters
jikkou api-resources --api-group kafka.jikkou.io --verbs LIST
Sample output
NAME SHORTNAMES APIVERSION KIND VERBS
kafkatopics kt kafka.jikkou.io/v1beta2 KafkaTopic CREATE, DELETE, GET, LIST, UPDATE
kafkaconsumergroups kafka.jikkou.io/v1beta2 KafkaConsumerGroup DELETE, GET, LIST
Options
| Flag | Default | Description |
|---|---|---|
--api-group | Limit to resources in the specified API group | |
--verbs | Limit to resources that support the specified verbs (comma-separated) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou api-extensions - List and inspect available extensions
- jikkou get - Display resources of a given type
1.3.4 - jikkou apply
Update the resources aand as described by the resource definition files.
Synopsis
Reconciles the target platform so that the resources match the resource definition files passed as arguments. This command uses the FULL reconciliation mode, meaning it will create, update, and delete resources as needed to match the desired state defined in your resource files.
jikkou apply [flags]
Examples
# Apply resources defined in a YAML file
jikkou apply -f my-resources.yaml
# Apply resources from a directory
jikkou apply -f ./resources/
# Apply resources with dry-run to preview changes
jikkou apply -f my-resources.yaml --dry-run
# Apply resources with specific selector
jikkou apply -f my-resources.yaml --selector 'metadata.name IN (my-topic)'
# Apply resources with template values
jikkou apply -f my-resources.yaml --values-files values.yaml
# Apply with inline template variable
jikkou apply -f my-resources.yaml -v topicName=my-topic
# Apply with custom labels
jikkou apply -f my-resources.yaml -l environment=production
# Apply with controller options
jikkou apply -f my-resources.yaml --options delete-orphans=true
# Apply targeting a specific provider
jikkou apply -f my-resources.yaml --provider kafka-prod
# Apply with JSON output format
jikkou apply -f my-resources.yaml -o JSON --pretty
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou create - Create resources (CREATE mode only)
- jikkou update - Create or update resources (UPDATE mode only)
- jikkou delete - Delete resources (DELETE mode only)
- jikkou diff - Show changes without applying
- jikkou validate - Validate resource definitions
1.3.5 - jikkou config
Sets or retrieves the configuration of this client.
Synopsis
The config command manages Jikkou CLI configuration contexts. A context defines the configuration settings
(config file path, inline properties, provider bindings) used when running Jikkou commands.
Configuration is stored in the Jikkou config file, located by default at $HOME/.jikkou/config.
Subcommands
| Command | Description |
|---|---|
| jikkou config set-context | Configure a context with the provided arguments |
| jikkou config get-contexts | List all configured contexts |
| jikkou config current-context | Display the current context |
| jikkou config use-context | Switch to a specified context |
| jikkou config view | Show merged configuration settings |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- Configuration - Learn how to configure Jikkou CLI
1.3.6 - jikkou config current-context
Display the current context.
Synopsis
Displays the current context used by Jikkou CLI, including the configuration file path and inline configuration properties associated with it.
jikkou config current-context
Examples
# Show the current context
$ jikkou config current-context
Using context 'localhost'
KEY VALUE
ConfigFile
ConfigProps {"provider.kafka.config.client.bootstrap.servers":"localhost:9092"}
Options
This command has no specific options.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou config get-contexts - List all contexts
- jikkou config use-context - Switch to a different context
- jikkou config view - Show the merged configuration
- jikkou config - Config command overview
1.3.7 - jikkou config get-contexts
List all configured contexts.
Synopsis
Get all contexts defined in the Jikkou config file. The current context is marked with an asterisk (*).
jikkou config get-contexts
Examples
# List all contexts
$ jikkou config get-contexts
NAME
localhost *
development
staging
production
Options
This command has no specific options.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou config current-context - Show the current context details
- jikkou config use-context - Switch to a context
- jikkou config set-context - Create or update a context
- jikkou config - Config command overview
1.3.8 - jikkou config set-context
Configure a context with the provided arguments.
Synopsis
Configures the specified context with the provided arguments. A context stores a reference to a configuration file and/or inline configuration properties. If the context already exists, it will be updated.
After setting a context, use jikkou config use-context to switch to it.
jikkou config set-context <context-name> [flags]
Examples
# Create a context with inline Kafka bootstrap server
jikkou config set-context localhost \
--config-props=provider.kafka.config.client.bootstrap.servers=localhost:9092
# Create a context pointing to a configuration file
jikkou config set-context production --config-file=/path/to/production.conf
# Create a context with a properties file
jikkou config set-context staging --config-props-file=/path/to/staging.properties
# Create a context with provider-scoped properties
jikkou config set-context dev \
--provider kafka \
--config-props=client.bootstrap.servers=kafka-dev:9092
# Create a context with a config prefix
jikkou config set-context dev \
--config-prefix=provider.kafka.config \
--config-props=client.bootstrap.servers=kafka-dev:9092
Options
| Flag | Default | Description |
|---|---|---|
--config-file | Path to a Jikkou configuration file | |
--config-props | Inline configuration properties as key=value pairs (repeatable) | |
--config-props-file | Path(s) to one or more configuration properties files (repeatable) | |
--provider | Name of the provider to which this configuration should be attached | |
--config-prefix | Prefix to apply to all configuration property keys |
Arguments
| Argument | Description |
|---|---|
context-name | (required) The name of the context to configure |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou config use-context - Switch to a context
- jikkou config get-contexts - List all contexts
- jikkou config - Config command overview
1.3.9 - jikkou config use-context
Switch to a specified context.
Synopsis
Configures Jikkou to use the specified context. All subsequent commands will use the configuration associated with this context.
jikkou config use-context <context-name>
Examples
# Switch to the production context
$ jikkou config use-context production
Using context 'production'
# If already using the specified context
$ jikkou config use-context production
Already using context production
Arguments
| Argument | Description |
|---|---|
context-name | (required) The name of the context to switch to |
Options
This command has no specific options.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou config get-contexts - List all available contexts
- jikkou config current-context - Display the current context
- jikkou config set-context - Create or update a context
- jikkou config - Config command overview
1.3.10 - jikkou config view
Show merged configuration settings.
Synopsis
Show the merged Jikkou configuration settings for the current context (or a named context). The configuration is rendered in HOCON format, showing all resolved values from the config file, inline properties, and defaults.
Use --debug to see the origin of each setting, or --comments to include human-written comments from the
configuration files.
jikkou config view [flags]
Examples
# View the current configuration
jikkou config view
# View configuration for a specific context
jikkou config view --name production
# View configuration with origin comments (useful for debugging)
jikkou config view --debug
# View configuration with human-written comments
jikkou config view --comments
Options
| Flag | Default | Description |
|---|---|---|
--name | The name of the context configuration to view (defaults to current context) | |
--debug | false | Print configuration with the origin of each setting as comments |
--comments | false | Print configuration with human-written comments |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou config current-context - Display the current context
- jikkou config set-context - Create or update a context
- jikkou config - Config command overview
1.3.11 - jikkou create
Create resources from the resource definition files (only non-existing resources will be created).
Synopsis
Reconcile the target platform by creating all non-existing resources that are described by the resource definition files passed as arguments. This command uses the CREATE reconciliation mode, meaning only new resources will be created. Existing resources will not be updated or deleted.
jikkou create [flags]
Examples
# Create resources defined in a YAML file
jikkou create -f my-resources.yaml
# Preview what would be created
jikkou create -f my-resources.yaml --dry-run
# Create resources from a directory
jikkou create -f ./resources/
# Create resources with template values
jikkou create -f my-resources.yaml --values-files values.yaml
# Create resources targeting a specific provider
jikkou create -f my-resources.yaml --provider kafka-dev
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes (FULL mode)
- jikkou update - Create or update resources (UPDATE mode)
- jikkou delete - Delete resources (DELETE mode)
- jikkou diff - Show changes without applying
1.3.12 - jikkou delete
Delete resources that are no longer described by the resource definition files.
Synopsis
Reconcile the target platform by deleting all existing resources that are no longer described by the resource definition files passed as arguments. This command uses the DELETE reconciliation mode, meaning only deletions will be performed. No resources will be created or updated.
jikkou delete [flags]
Examples
# Delete resources no longer defined in a YAML file
jikkou delete -f my-resources.yaml
# Preview what would be deleted
jikkou delete -f my-resources.yaml --dry-run
# Delete resources from a directory
jikkou delete -f ./resources/
# Delete with selector to target specific resources
jikkou delete -f my-resources.yaml -s 'metadata.name IN (old-topic)'
# Delete resources targeting a specific provider
jikkou delete -f my-resources.yaml --provider kafka-dev
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes (FULL mode)
- jikkou create - Create resources only (CREATE mode)
- jikkou update - Create or update resources (UPDATE mode)
- jikkou diff - Show changes without applying
1.3.13 - jikkou diff
Show resource changes required by the current resource definitions.
Synopsis
Generates a speculative reconciliation plan, showing the resource changes Jikkou would apply to reconcile the resource definitions. This command does not actually perform the reconciliation actions.
Use diff to preview what changes would be made before running apply, create, update, or delete.
jikkou diff [flags]
Examples
# Show changes required by a resource definition file
jikkou diff -f my-resources.yaml
# Show only resources that would be created
jikkou diff -f my-resources.yaml --filter-resource-op CREATE
# Show only resources that would be created or deleted
jikkou diff -f my-resources.yaml --filter-resource-op CREATE,DELETE
# Filter changes to show only update operations
jikkou diff -f my-resources.yaml --filter-change-op UPDATE
# Output as a resource list
jikkou diff -f my-resources.yaml --list
# Output in JSON format
jikkou diff -f my-resources.yaml -o JSON
# Diff with a specific provider
jikkou diff -f my-resources.yaml --provider kafka-prod
# Diff with selector
jikkou diff -f my-resources.yaml -s 'metadata.name MATCHES (my-topic-.*)'
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | YAML | Output format. Valid values: JSON, YAML |
--filter-resource-op | Filter resources by operation (comma-separated). Valid values: NONE, CREATE, DELETE, REPLACE, UPDATE | ||
--filter-change-op | Filter state changes by operation (comma-separated). Valid values: NONE, CREATE, DELETE, REPLACE, UPDATE | ||
--list | false | Output resources as an ApiResourceChangeList |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes
- jikkou validate - Validate resource definitions
- jikkou prepare - Prepare resources for validation
1.3.14 - jikkou get
Display one or many specific resources.
Synopsis
Display one or many resources of a given type from the target platform. The get command groups resources under
their provider (e.g. kafka, schemaregistry, aiven), so invocations follow the form
jikkou get <provider> <kind>. Use jikkou api-providers list to discover the available providers and
jikkou api-resources to discover the available resource types.
When a resource name is specified via --name, it retrieves that specific resource. Otherwise, it lists all
resources of the given type.
jikkou get <provider> <resource-type> [flags]
Resource naming. Under a provider, each resource is exposed by its provider-local name when one is declared
(e.g. topics under kafka, subjects under schemaregistry). The full plural name (e.g. kafkatopics,
schemaregistrysubjects) and any short names (e.g. kt, sr) remain valid aliases inside the provider scope.
Backward compatibility. The flat form jikkou get <resource-type> is still supported but prints a deprecation
notice on stderr and will be removed in a future release. Prefer the qualified form
jikkou get <provider> <resource-type>.
Examples
# List all Kafka topics (canonical form)
jikkou get kafka topics
# Same, using the plural name as alias
jikkou get kafka kafkatopics
# Same, using the short name as alias
jikkou get kafka kt
# Get a specific Kafka topic by name
jikkou get kafka topics --name my-topic
# List resources with a selector
jikkou get kafka topics -s 'metadata.name MATCHES (my-.*)'
# List resources with JSON output
jikkou get kafka topics -o JSON
# List resources as a ResourceListObject
jikkou get kafka topics --list
# List resources with custom options
jikkou get kafka topics --options describe-default-configs=true
# List resources targeting a specific provider instance
jikkou get kafka topics --provider kafka-prod
# List schema registry subjects
jikkou get schemaregistry subjects
# Discover available resource types
jikkou api-resources --verbs LIST,GET
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--output | -o | YAML | Output format. Valid values: JSON, YAML |
--list | false | Output resources as a ResourceListObject | |
--provider | Select a specific provider instance |
Additional options may be available depending on the resource type. These are dynamically registered based on the
provider’s ApiOptionSpec definitions. Use jikkou api-extensions get <extension-name> to view options for a
specific resource collector.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou api-resources - List available resource types
- jikkou api-extensions - List and inspect extensions
- jikkou diff - Show changes between desired and actual state
1.3.15 - jikkou health
Print or describe health indicators.
Synopsis
The health command provides information about the health of target environments. It has two subcommands:
health get- Retrieve health status for one or all indicatorshealth get-indicators- List all available health indicators
Subcommands
jikkou health get
Get health information for a specific indicator or all indicators.
jikkou health get <indicator|all> [flags]
Examples
# Get health for all indicators
jikkou health get all
# Get health for a specific indicator
jikkou health get kafka
# Get health with a custom timeout
jikkou health get all --timeout-ms 5000
# Get health in JSON format
jikkou health get all -o JSON
# Get health targeting a specific provider
jikkou health get kafka --provider kafka-prod
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--output | -o | YAML | Output format. Valid values: JSON, YAML |
--timeout-ms | 2000 | Timeout in milliseconds for retrieving health indicators | |
--provider | Select a specific provider instance |
The command exits with code 0 if the health status is UP, or a non-zero code otherwise.
jikkou health get-indicators
List all available health indicators. This command takes no options and displays the name and description of each registered health indicator.
jikkou health get-indicators
Examples
# List all health indicators
jikkou health get-indicators
Options
This subcommand has no specific options.
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou api-extensions - List and inspect extensions
- jikkou server-info - Display server information (proxy mode)
1.3.16 - jikkou patch
Execute all changes for the specified reconciliation mode.
Synopsis
Reconcile resources by applying all the changes as defined in the resource descriptor files passed through the
arguments. Unlike the other reconciliation commands (apply, create, update, delete), patch requires you to
explicitly specify the reconciliation mode using the --mode flag.
The patch command applies changes to existing resources on the platform by computing a diff between the current
and desired states, then executing only the operations allowed by the selected mode.
jikkou patch --mode <mode> [flags]
Examples
# Patch resources using FULL reconciliation mode
jikkou patch --mode FULL -f my-resources.yaml
# Patch resources - create only
jikkou patch --mode CREATE -f my-resources.yaml
# Patch resources - update only
jikkou patch --mode UPDATE -f my-resources.yaml
# Preview patches without applying
jikkou patch --mode FULL -f my-resources.yaml --dry-run
# Patch with a specific provider
jikkou patch --mode UPDATE -f my-resources.yaml --provider kafka-prod
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--mode | (required) The reconciliation mode. Valid values: CREATE, DELETE, UPDATE, FULL | ||
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes (FULL mode)
- jikkou replace - Replace resources by deleting and recreating
- jikkou diff - Show changes without applying
1.3.17 - jikkou prepare
Prepare the resource definition files for validation.
Synopsis
Prepare the resource definition files specified through the command line arguments for validation. This command applies all configured transformations to the resource definitions and outputs the result, without running validation rules or performing reconciliation.
Use prepare to inspect how your resources look after template rendering and transformations, before validation and
reconciliation happen.
jikkou prepare [flags]
Examples
# Prepare resources from a YAML file
jikkou prepare -f my-resources.yaml
# Prepare resources from a directory
jikkou prepare -f ./resources/
# Prepare with template values
jikkou prepare -f my-resources.yaml --values-files values.yaml
# Prepare with JSON output
jikkou prepare -f my-resources.yaml -o JSON
# Prepare with selector
jikkou prepare -f my-resources.yaml -s 'metadata.name IN (my-topic)'
# Prepare targeting a specific provider
jikkou prepare -f my-resources.yaml --provider kafka-prod
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | YAML | Output format. Valid values: JSON, YAML |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou validate - Validate prepared resources
- jikkou apply - Apply changes
- jikkou diff - Show changes without applying
1.3.18 - jikkou replace
Replace all resources.
Synopsis
Replaces resources by deleting and (re)creating all the resources as defined in the resource descriptor files passed
through the arguments. Unlike apply, which performs incremental updates, replace performs a full replacement of
resources: existing resources are deleted first and then recreated from scratch.
Use replace when you need to ensure resources exactly match the desired state without any leftover configuration
from previous versions.
jikkou replace [flags]
Examples
# Replace resources defined in a YAML file
jikkou replace -f my-resources.yaml
# Preview what would be replaced
jikkou replace -f my-resources.yaml --dry-run
# Replace resources from a directory
jikkou replace -f ./resources/
# Replace resources targeting a specific provider
jikkou replace -f my-resources.yaml --provider kafka-prod
# Replace with JSON output
jikkou replace -f my-resources.yaml -o JSON --pretty
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes (FULL mode)
- jikkou patch - Patch resources with a specified mode
- jikkou diff - Show changes without applying
1.3.19 - jikkou server-info
Display Jikkou API server information.
Synopsis
Print information about the Jikkou API server. This command is only available when Jikkou is configured in
proxy mode (i.e., when jikkou.proxy.url is set in your configuration).
jikkou server-info [flags]
Examples
# Get server information in YAML format (default)
jikkou server-info
# Get server information in JSON format
jikkou server-info -o JSON
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--output | -o | YAML | Output format. Valid values: JSON, YAML |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou health - Check health of target environments
- Configuration - Learn how to configure proxy mode
1.3.20 - jikkou update
Create or update resources from the resource definition files.
Synopsis
Reconcile the target platform by creating or updating resources that are described by the resource definition files passed as arguments. This command uses the UPDATE reconciliation mode, meaning new resources will be created and existing resources will be updated. Resources that exist on the platform but are not described in the resource files will not be deleted.
jikkou update [flags]
Examples
# Update resources defined in a YAML file
jikkou update -f my-resources.yaml
# Preview what would be created or updated
jikkou update -f my-resources.yaml --dry-run
# Update resources from a directory
jikkou update -f ./resources/
# Update resources with selector to filter specific resources
jikkou update -f my-resources.yaml -s 'metadata.name MATCHES (my-topic-.*)'
# Update resources targeting a specific provider
jikkou update -f my-resources.yaml --provider kafka-prod
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | TEXT | Output format. Valid values: TEXT, COMPACT, JSON, YAML |
--pretty | false | Pretty print JSON output | |
--dry-run | false | Execute command in dry-run mode (preview changes without applying) |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou apply - Apply all changes (FULL mode)
- jikkou create - Create resources only (CREATE mode)
- jikkou delete - Delete resources (DELETE mode)
- jikkou diff - Show changes without applying
1.3.21 - jikkou validate
Check whether the resources definitions meet all validation requirements.
Synopsis
Validate the resource definition files specified through the command line arguments.
Validate runs all the user-defined validation requirements after performing any relevant resource transformations. Validation rules are applied only to resources matching the selectors passed through the command line arguments.
If validation passes, the command outputs the validated resources. If validation fails, the command prints the validation errors and exits with a non-zero status code.
jikkou validate [flags]
Examples
# Validate resources defined in a YAML file
jikkou validate -f my-resources.yaml
# Validate resources from a directory
jikkou validate -f ./resources/
# Validate with a selector to filter specific resources
jikkou validate -f my-resources.yaml -s 'metadata.name MATCHES (my-topic-.*)'
# Validate with template values
jikkou validate -f my-resources.yaml --values-files values.yaml
# Validate with JSON output
jikkou validate -f my-resources.yaml -o JSON
# Validate targeting a specific provider
jikkou validate -f my-resources.yaml --provider kafka-prod
Options
| Flag | Short | Default | Description |
|---|---|---|---|
--files | -f | Resource definition file or directory locations (one or more required) | |
--file-name | -n | **/*.{yaml,yml} | Glob pattern to filter resource files when using directories |
--values-files | Template values file locations (one or more) | ||
--values-file-name | **/*.{yaml,yml} | Glob pattern to filter values files | |
--set-label | -l | Set labels on resources (key=value, repeatable) | |
--set-annotation | Set annotations on resources (key=value, repeatable) | ||
--set-value | -v | Set template variables for the built-in Values object (key=value, repeatable) | |
--selector | -s | Selector expression for including or excluding resources | |
--selector-match | ALL | Selector matching strategy. Valid values: ALL, ANY, NONE | |
--options | Controller configuration options (key=value, repeatable) | ||
--provider | Select a specific provider instance | ||
--output | -o | YAML | Output format. Valid values: JSON, YAML |
Options inherited from parent commands
| Flag | Description |
|---|---|
--logger-level=<level> | Log level: TRACE, DEBUG, INFO, WARN, ERROR |
SEE ALSO
- jikkou prepare - Prepare resources for validation
- jikkou apply - Apply changes after validation
- jikkou diff - Show changes without applying
2 - Jikkou API Server Documentation
Jikkou API Server provides a REST interface to any platform supported by Jikkou, making it even easier to manage, automate and visualise all your data platform assets.
Jikkou CLI can be used in combination with Jikkou API Server by configuring it in proxy mode. In this mode, the CLI no longer connects directly to your various platforms, but forwards all operations to the API server. This deployment method allows you to enhance the overall security of the platforms managed through Jikkou.
2.1 - Configurations
Jikkou API Server is built with Micronaut Framework.
The default configuration file is located in the installation directory of you server under the
path /etc/application.yaml.
You can either modify this configuration file directly or create a new one.
Then, your configuration file path can be targeted through the MICRONAUT_CONFIG_FILES environment variable.
A YAML Configuration file example can be found here: application.yaml
Note
For more information about how to configure the application, we recommend you to read the official Micronaut documentation (see: Application Configuration).2.1.1 - API Server
Running Server on a Specific Port
By default, the server runs on port 28082. However, you can set the server to run on a specific port:
# ./etc/application.yaml
micronaut:
server:
port: 80 # Port used to access APIs
endpoints:
all:
port: 80 # Port used to access Health endpoints
Enabling Specific Extension Providers
By default, the server is configured to run only with the core and kafka extension providers.
However, you can enable (or disable) additional providers:
jikkou:
# The providers
provider:
# Core
core:
enabled: true
type: io.jikkou.core.CoreExtensionProvider
# Default configuration for Apache Kafka
kafka:
enabled: true
type: io.jikkou.kafka.KafkaExtensionProvider
config:
client:
bootstrap.servers: localhost:9092
# Default configuration for Schema Registry
schemaregistry:
enabled: true
type: io.jikkou.schema.registry.SchemaRegistryExtensionProvider
config:
url: http://localhost:8081
# Default configuration for Kafka Connect
kafkaconnect:
enabled: true
type: io.jikkou.kafka.connect.KafkaConnectExtensionProvider
config:
clusters:
- name: localhost
url: http://localhost:8083
2.1.2 - Authentication
Enable Security
To enable secure access to the API Server:
Configuration File
Update the configuration file (i.e., application.yaml) of the server with:
micronaut:
security:
enabled: true
Environment Variable
As an alternative, you can set the following environment variable MICRONAUT_SECUTIRY_ENABLED=true.
Note
For more information about how Micronaut binds environment variables and configuration property: https://docs.micronaut.io/latest/guide/index.html#_property_value_binding).Unauthorized Access
When accessing a secured path, the server will return the following response if access is not authorized:
{
"message": "Unauthorized",
"errors": [
{
"status": 401,
"error_code": "authentication_user_unauthorized",
"message": "Unauthorized"
}
]
}
2.1.2.1 - Basic Auth
Jikkou API Server can be secured using a Basic HTTP Authentication Scheme.
RFC7617 defines the “Basic” Hypertext Transfer Protocol (HTTP) authentication scheme, which transmits credentials as user-id/password pairs, encoded using Base64.
Basic Authentication should be used over a secured connection using HTTPS.
Configure Basic HTTP Authentication
Step1: Enable security
Add the following configuration to your server configuration.
# ./etc/application.yaml
micronaut:
security:
enabled: true
Step2: Configure the list of users
The list of username/password authorized to connect to the API server can be configured as follows:
# ./etc/application.yaml
jikkou:
security:
basic-auth:
- username: "admin"
password: "{noop}password"
For production environment, password must not be configured in plaintext. Password can be passed encoded
in bcrypt, scrypt, argon2, and sha256.
Example
echo -n password | sha256sum
5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
# ./etc/application.yaml
jikkou:
security:
basic-auth:
- username: "admin"
password: "{sha256}5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8"
Step3: Validate authentication
Encode credentials
echo -n "admin:password" | base64
YWRtaW46cGFzc3dvcmQ=
Send request
curl -IX GET http://localhost:28082/apis/kafka.jikkou.io/v1beta2/kafkabrokers \
-H "Accept: application/json" \
-H "Authorization: Basic YWRtaW46cGFzc3dvcmQ"
HTTP/1.1 200 OK
Content-Type: application/hal+json
content-length: 576
2.1.2.2 - JWT
Jikkou API Server can be secured using JWT (JSON Web Token) Authentication.
Configure JWT
Step1: Set JWT signature secret
Add the following configuration to your server configuration.
# ./etc/application.yaml
micronaut:
security:
enabled: true
authentication: bearer <1>
token:
enabled: true
jwt:
signatures:
secret:
generator:
secret: ${JWT_GENERATOR_SIGNATURE_SECRET:pleaseChangeThisSecretForANewOne} <2>
- <1> Set authentication to bearer to receive a JSON response from the login endpoint.
- <2> Change this to your own secret and keep it safe (do not store this in your VCS).
Step2: Generate a Token
Generate a valid JSON Web Token on https://jwt.io/ using your secret.
Example with pleaseChangeThisSecretForANewOne as signature secret.
TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.6cD3MnZmX2xyEAWyh-GgGD11TX8SmvmHVLknuAIJ8yE
Step3: Validate authentication
$ curl -I -X GET http://localhost:28082/apis/kafka.jikkou.io/v1beta2/kafkabrokers \
-H "Accept: application/json" \
-H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.6cD3MnZmX2xyEAWyh-GgGD11TX8SmvmHVLknuAIJ8yE"
HTTP/1.1 200 OK
Content-Type: application/hal+json
content-length: 576
2.1.3 - CLI Proxy Mode
Configuration
Step 1: Enable Proxy Mode
To enable proxy mode so that the CLI communicates directly with your API Server, add the following parameters to your configuration:
jikkou {
# Proxy Configuration
proxy {
# Specify whether proxy mode is enabled (default: false).
enabled = true
# URL of the API Server
url = "http://localhost:28082"
# Specifcy whether HTTP request debugging should be enabled (default: false)
debugging = false
# The connect timeout in millisecond (if not configured used ` default-timeout` ).
connect-timeout = 10000
# The read timeout in millisecond (if not configured used ` default-timeout` ).
read-timeout = 10000
# The write timeout in millisecond (if not configured used ` default-timeout` ).
write-timeout = 10000
# The default timeout (i.e., for read/connect) in millisecond (default: 10000)
default-timeout = 10000
# Security settings to authenticate to the API Server.
security = {
# For Token based Authentication.
# access-token = ""
# For Username/Password Basic-Authentication.
# basic-auth = {
# username = ""
# password = ""
# }
}
}
}
Step 2: Check connection
When enabling Proxy Mode, Jikkou CLI provides the additional command server-info. You can use it to verify the
connectivity with teh server.
$ jikkou server-info -o JSON | jq
{
"version": "0.31.0",
"build_time": "2023-11-15T10:35:22+0100",
"commit_id": "f3384d38e606fb32599c175895d0cbef28258540"
}
2.2 - Jikkou - API References
3 - Extension Providers
The section helps you learn more about the built-in Extension Providers for Jikkou.
3.1 - Core Extensions
Here, you will find information to use the Core extensions.
More information:
3.1.1 - Resource Repositories
Here, you will find the list of core Resource Repositories supported for Jikkou.
Core Repositories
More information:
3.1.1.1 - Github Resource Repository
Overview
The GithubResourceRepository can be used to load resources from a public or private GitHub repository.
Configuration
jikkou {
repositories = [
{
# Name of your local repositories
name = "<string>"
# The fully qualified class name (FQCN) of the repository
type = io.jikkou.core.repository.GithubResourceRepository
config {
# Specify the GitHub repository in the format 'owner/repo'
repository = "<string>"
# Specify the branch or ref to load resources from
branch = main
# Specify the paths/directories in the repository containing the resource definitions
paths = [] # Default: **/*.{yaml,yml}
# Specify the pattern used to match YAML file paths.
# Pattern should be passed in the form of 'syntax:pattern'. The "glob" and "regex" syntaxes are supported (e.g.: **/*.{yaml,yml}).
# If no syntax is specified the 'glob' syntax is used.
file-pattern = "**/*.{yaml,yml}"
# Specify the locations of the values-files containing the variables to pass into the template engine built-in object 'Values'.
values-files = []
# Specify the pattern used to match YAML file paths when one or multiple directories are given through the `values-files` property.
# Pattern should be passed in the form of 'syntax:pattern'. The "glob" and "regex" syntaxes are supported (e.g.: **/*.{yaml,yml}).
# If no syntax is specified the 'glob' syntax is used.
values-file-name = "<string>" # Default: **/*.{yaml,yml}
# The labels to be added to all resources loaded from the repository
labels {
<label_key> = <label_value>
}
}
}
]
}
3.1.1.2 - Local Resource Repository
Overview
The LocalResourceRepository can be used to load resources from local files or directories.
Configuration
jikkou {
repositories = [
{
# Name of your local repositories
name = "<string>"
# The fully qualified class name (FQCN) of the repository
type = io.jikkou.core.repository.LocalResourceRepository
config {
# Specify the locations containing the definitions for resources in a YAML file, a directory.
files = []
# Specify the pattern used to match YAML file paths when one or multiple directories are given through the `files` property.
# Pattern should be passed in the form of 'syntax:pattern'. The "glob" and "regex" syntaxes are supported (e.g.: **/*.{yaml,yml}).
# If no syntax is specified the 'glob' syntax is used.
file-name = "<string>" # Default: **/*.{yaml,yml}
# Specify the locations of the values-files containing the variables to pass into the template engine built-in object 'Values'.
values-files = []
# Specify the pattern used to match YAML file paths when one or multiple directories are given through the `values-files` property.
# Pattern should be passed in the form of 'syntax:pattern'. The "glob" and "regex" syntaxes are supported (e.g.: **/*.{yaml,yml}).
# If no syntax is specified the 'glob' syntax is used.
values-file-name = "<string>" # Default: **/*.{yaml,yml}
# The labels to add to all resources loaded from the repository
labels {
<label_key> = <label_value>
}
}
}
]
}
3.1.2 - Resources
Here, you will find the list of core resources supported for Jikkou.
Core Resources
More information:
3.1.2.1 - ConfigMap
You can use a ConfigMap to define reusable data in the form of key/value pairs that can then be referenced and used by
other resources.
Specification
---
apiVersion: "core.jikkou.io/v1beta2"
kind: ConfigMap
metadata:
name: '<CONFIG-MAP-NAME>' # Name of the ConfigMap (required)
data: # Map of key-value pairs (required)
<KEY_1>: "<VALUE_1>"
Example
For example, the below ConfigMap show how to define default config properties namedcKafkaTopicConfig that can then
reference and used to define multiple KafkaTopic. resources.
---
apiVersion: "core.jikkou.io/v1beta2"
kind: ConfigMap
metadata:
name: 'KafkaTopicConfig'
data:
cleanup.policy: 'delete'
min.insync.replicas: 2
retention.ms: 86400000 # (1 day)
3.1.2.2 - ValidatingResourcePolicy
The ValidatingResourcePolicy resource is used to define validation rules applied to resources or resource changes before they are applied by Jikkou.
It allows enforcing organizational policies, validating constraints, or filtering out undesired operations.
Each policy can select one or more resource kinds and define rules expressed in Google CEL (Common Expression Language).
Rules can either fail the execution or filter the invalid resources, depending on the configured failurePolicy.
Specification
apiVersion: core.jikkou.io/v1
kind: ValidatingResourcePolicy
metadata:
name: <string> # Required. Unique policy name.
spec:
failurePolicy: <string> # Required. One of: FAIL | FILTER
selector:
matchingStrategy: <string> # Optional. One of: ALL | ANY (default: ALL)
matchResources:
- apiVersion: <string> # Optional. API version to match (e.g., core.jikkou.io/v1)
kind: <string> # Required. Resource kind (e.g., KafkaTopic)
matchLabels:
- key: <string> # Label key to match
operator: <string> # One of: In | NotIn | Exists | DoesNotExist
values: [<string>] # Optional list of values
matchExpressions:
- <string> # CEL expression
rules:
- name: <string> # Required. Rule identifier.
expression: <string> # Required. A CEL expression evaluated against the resource.
message: <string> # Optional. Static message returned when the rule fails.
messageExpression: <string> # Optional. CEL expression to generate a dynamic error message.
Fields
| Field | Type | Required | Description |
|---|---|---|---|
spec.failurePolicy | string | Yes | Defines the policy behavior when validation fails. Possible values: • FAIL → stop execution with an error.• FILTER → skip the invalid resource(s) but continue processing others. |
spec.selector.matchingStrategy | string | No | Strategy for combining multiple selectors. Possible values: • ALL → resource must match all conditions.• ANY → resource must match at least one condition.Default: ALL. |
spec.selector.matchResources | list | No | Selects resources by API version and kind. |
spec.selector.matchLabels | list | No | Selects resources based on labels, using operators (In, NotIn, Exists, DoesNotExist). |
spec.selector.matchExpressions | list | No | Selects resources using CEL expressions for advanced filtering. |
spec.rules | list | Yes | A list of validation rules. |
spec.rules[].name | string | Yes | A unique identifier for the rule. |
spec.rules[].expression | string | Yes | A CEL expression evaluated against the resource. The rule fails when the expression evaluates to true. |
spec.rules[].message | string | No | Static error message returned when validation fails. |
spec.rules[].messageExpression | string | No | CEL expression returning a dynamic error message string. |
Resource Selection
Policies define which resources they apply to using a selector.
A selector can combine multiple strategies to target resources based on:
- Resource metadata (kind, apiVersion).
- Labels (with operators like
In,NotIn,Exists,DoesNotExist). - CEL expressions (arbitrary conditions on resource content).
Matching Strategy
| Strategy | Description |
|---|---|
ALL | The resource must match all specified selectors (matchResources, matchLabels, and matchExpressions). |
ANY | The resource is selected if it matches at least one of the specified selectors. |
Default: ALL
matchResources
Selects resources by API version and/or kind.
matchResources:
- apiVersion: core.jikkou.io/v1
kind: KafkaTopic
apiVersion→ Optional. Restricts matching to a specific API group/version.kind→ Required. Matches the resource kind (e.g.KafkaTopic,KafkaTopicChange).
matchLabels
Selects resources based on their metadata labels using operators.
matchLabels:
- key: environment
operator: In
values: ["prod", "staging"]
- key: team
operator: NotIn
values: ["test"]
- key: critical
operator: Exists
Supported operators:
| Operator | Description |
|---|---|
In | Matches if the label value is in the list of values. |
NotIn | Matches if the label value is not in the list of values. |
Exists | Matches if the label key is defined (value doesn’t matter). |
DoesNotExist | Matches if the label key is not defined. |
matchExpressions
Selects resources using CEL expressions for maximum flexibility.
matchExpressions:
- "resource.metadata.name.startsWith('topic-')"
- "resource.spec.partitions > 10"
Examples:
- Match resources with names starting with
topic-. - Match topics with more than 10 partitions.
Examples
Example 1: Filtering DELETE operations on KafkaTopic resources
apiVersion: core.jikkou.io/v1
kind: ValidatingResourcePolicy
metadata:
name: KafkaTopicPolicy
spec:
failurePolicy: FILTER
selector:
matchResources:
- kind: KafkaTopicChange
rules:
- name: FilterDeleteOperation
expression: "size(resource.spec.changes) > 0 && resource.spec.op == 'DELETE'"
messageExpression: "'Operation ' + resource.spec.op + ' on topics is not authorized'"
This policy prevents delete operations on Kafka topics from being executed by filtering them out.
Example 2: Validating partitions count for KafkaTopic
apiVersion: core.jikkou.io/v1
kind: ValidatingResourcePolicy
metadata:
name: KafkaTopicPolicy
spec:
failurePolicy: FAIL
selector:
matchResources:
- kind: KafkaTopic
rules:
- name: MaxTopicPartitions
expression: "resource.spec.partitions >= 50"
messageExpression: "'Topic partition MUST be inferior to 50, but was: ' + string(resource.spec.partitions)"
- name: MinTopicPartitions
expression: "resource.spec.partitions < 3"
message: "Topic must have at-least 3 partitions"
This policy enforces a minimum of 3 partitions and a maximum of 49 partitions for Kafka topics.
Example 3: Match only KafkaTopic in prod environment
selector:
matchingStrategy: ALL
matchResources:
- kind: KafkaTopic
matchLabels:
- key: environment
operator: In
values: ["prod"]
Example 4: Match any KafkaTopic OR resources with label critical=true
selector:
matchingStrategy: ANY
matchResources:
- kind: KafkaTopic
matchLabels:
- key: critical
operator: In
values: ["true"]
Example 5: Match using CEL expression
selector:
matchExpressions:
- "resource.spec.replicationFactor < 3"
Use cases
- Preventing destructive operations (e.g., deleting topics, removing configs).
- Enforcing resource limits (e.g., partition count, replication factor).
- Ensuring naming conventions or metadata compliance.
- Dynamically generating error messages with contextual information.
3.2 - Apache Kafka
Here, you will find information to use the Apache Kafka extensions.
More information:
3.2.1 - Configuration
Here, you will find the list of resources supported for Apache Kafka.
Configuration
The Apache Kafka extension is built on top of the Kafka Admin Client. You can configure the properties to be passed to
kafka client through the Jikkou client configuration property jikkou.provider.kafka.
Example:
jikkou {
provider.kafka {
enabled = true
type = io.jikkou.kafka.KafkaExtensionProvider
config = {
client {
bootstrap.servers = "localhost:9092"
security.protocol = "SSL"
ssl.keystore.location = "/tmp/client.keystore.p12"
ssl.keystore.password = "password"
ssl.keystore.type = "PKCS12"
ssl.truststore.location = "/tmp/client.truststore.jks"
ssl.truststore.password = "password"
ssl.key.password = "password"
}
}
}
}
In addition, the extension support configuration settings to wait for at least a minimal number of brokers before processing.
jikkou {
provider.kafka {
enabled = true
type = io.jikkou.kafka.KafkaExtensionProvider
config = {
brokers {
# If 'True'
waitForEnabled = true
waitForEnabled = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_ENABLED}
# The minimal number of brokers that should be alive for the CLI stops waiting.
waitForMinAvailable = 1
waitForMinAvailable = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_MIN_AVAILABLE}
# The amount of time to wait before verifying that brokers are available.
waitForRetryBackoffMs = 1000
waitForRetryBackoffMs = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_RETRY_BACKOFF_MS}
# Wait until brokers are available or this timeout is reached.
waitForTimeoutMs = 60000
waitForTimeoutMs = ${?JIKKOU_KAFKA_BROKERS_WAIT_FOR_TIMEOUT_MS}
}
}
}
}
3.2.2 - Resources
Here, you will find the list of resources supported for Apache Kafka.
Apache Kafka Resources
More information:
3.2.2.1 - Kafka Brokers
This section describes the resource definition format for kafkabrokers entities, which can be used to define the
brokers you plan to manage on a specific Kafka cluster.
Listing KafkaBroker
You can retrieve the state of Kafka Consumer Groups using the jikkou get kafkabrokers (or jikkou get kb) command.
Usage
Usage:
Get all 'KafkaBroker' resources.
jikkou get kafkabrokers [-hV] [--default-configs] [--dynamic-broker-configs]
[--list] [--static-broker-configs]
[--logger-level=<level>] [-o=<format>]
[-s=<expressions>]...
DESCRIPTION:
Use jikkou get kafkabrokers when you want to describe the state of all
resources of type 'KafkaBroker'.
OPTIONS:
--default-configs Describe built-in default configuration for configs
that have a default value.
--dynamic-broker-configs
Describe dynamic configs that are configured as
default for all brokers or for specific broker in
the cluster.
-h, --help Show this help message and exit.
--list Get resources as ResourceListObject.
--logger-level=<level>
Specify the log level verbosity to be used while
running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN,
ERROR.
For example, `--logger-level=INFO`
-o, --output=<format> Prints the output in the specified format. Allowed
values: json, yaml (default yaml).
-s, --selector=<expressions>
The selector expression used for including or
excluding resources.
--static-broker-configs
Describe static configs provided as broker properties
at start up (e.g. server.properties file).
-V, --version Print version information and exit.
(The output from your current Jikkou version may be different from the above example.)
Examples
(command)
$ jikkou get kafkabrokers --static-broker-configs
(output)
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaBroker"
metadata:
name: "101"
labels: {}
annotations:
kafka.jikkou.io/cluster-id: "xtzWWN4bTjitpL3kfd9s5g"
spec:
id: "101"
host: "localhost"
port: 9092
configs:
advertised.listeners: "PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092"
authorizer.class.name: "org.apache.kafka.metadata.authorizer.StandardAuthorizer"
broker.id: "101"
controller.listener.names: "CONTROLLER"
controller.quorum.voters: "101@kafka:29093"
inter.broker.listener.name: "PLAINTEXT"
listener.security.protocol.map: "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT"
listeners: "PLAINTEXT://kafka:29092,PLAINTEXT_HOST://0.0.0.0:9092,CONTROLLER://kafka:29093"
log.dirs: "/var/lib/kafka/data"
node.id: "101"
offsets.topic.replication.factor: "1"
process.roles: "broker,controller"
transaction.state.log.replication.factor: "1"
zookeeper.connect: ""
3.2.2.2 - Kafka Consumer Groups
This section describes the resource definition format for KafkaConsumerGroup entities, which can be used to define the
consumer groups you plan to manage on a specific Kafka cluster.
Listing KafkaConsumerGroup
You can retrieve the state of Kafka Consumer Groups using the jikkou get kafkaconsumergroups (or jikkou get kcg) command.
Usage
$ jikkou get kafkaconsumergroups --help
Usage:
Get all 'KafkaConsumerGroup' resources.
jikkou get kafkaconsumergroups [-hV] [--list] [--offsets]
[--logger-level=<level>] [-o=<format>]
[--in-states=PARAM]... [-s=<expressions>]...
DESCRIPTION:
Use jikkou get kafkaconsumergroups when you want to describe the state of all
resources of type 'KafkaConsumerGroup'.
OPTIONS:
-h, --help Show this help message and exit.
--in-states=PARAM If states is set, only groups in these states will be
returned. Otherwise, all groups are returned. This
operation is supported by brokers with version
2.6.0 or later
--list Get resources as ResourceListObject.
--logger-level=<level>
Specify the log level verbosity to be used while
running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN,
ERROR.
For example, `--logger-level=INFO
-o, --output=<format> Prints the output in the specified format. Allowed
values: json, yaml (default yaml).
--offsets Specify whether consumer group offsets should be
described.
-s, --selector=<expressions>
The selector expression used for including or
excluding resources.
-V, --version Print version information and exit
(The output from your current Jikkou version may be different from the above example.)
Examples
(command)
$ jikkou get kafkaconsumergroups --in-states STABLE --offsets
(output)
---
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConsumerGroup"
metadata:
name: "my-group"
labels:
kafka.jikkou.io/is-simple-consumer: false
status:
state: "STABLE"
members:
- memberId: "console-consumer-b103994e-bcd5-4236-9d03-97065057e594"
clientId: "console-consumer"
host: "/127.0.0.1"
assignments:
- "my-topic-0"
offsets:
- topic: "my-topic"
partition: 0
offset: 0
coordinator:
id: "101"
host: "localhost"
port: 9092
Managing consumer group configuration
Since Kafka 4.x, consumer groups support dynamic group-level configuration. You can manage these
declaratively through the spec.configs map, which is reconciled against the Kafka GROUP
configuration resource (drift detection + incremental alter), just like topic configs.
apiVersion: "kafka.jikkou.io/v1"
kind: "KafkaConsumerGroup"
metadata:
name: "my-group"
spec:
configs:
consumer.session.timeout.ms: 50000
By default, group configs present on the cluster but absent from the resource are removed
(config-delete-orphans=true). Set it to false to leave externally-managed configs untouched.
3.2.2.3 - Kafka Topics
KafkaTopic resources are used to define the topics you want to manage on your Kafka Cluster(s). A KafkaTopic resource defines the number of partitions, the replication factor, and the configuration properties to be associated to a topics.
KafkaTopic
Specification
Here is the resource definition file for defining a KafkaTopic.
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaTopic" # The resource kind (required)
metadata:
name: <The name of the topic> # (required)
labels: { }
annotations: { }
spec:
partitions: <Number of partitions> # (optional)
replicas: <Number of replicas> # (optional)
configs:
<config_key>: <Config Value> # The topic config properties keyed by name to override (optional)
configMapRefs: [ ] # The list of ConfigMap to be applied to this topic (optional)
The metadata.name property is mandatory and specifies the name of the kafka topic.
To use the cluster default values for the number of partitions and replicas you can set the property
spec.partitions and spec.replicas to -1.
Example
Here is a simple example that shows how to define a single YAML file containing two topic definition using
the KafkaTopic resource type.
file: kafka-topics.yaml
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: KafkaTopic
metadata:
name: 'my-topic-p1-r1' # Name of the topic
labels:
environment: example
spec:
partitions: 1 # Number of topic partitions (use -1 to use cluster default)
replicas: 1 # Replication factor per partition (use -1 to use cluster default)
configs:
min.insync.replicas: 1
cleanup.policy: 'delete'
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: KafkaTopic
metadata:
name: 'my-topic-p2-r1' # Name of the topic
labels:
environment: example
spec:
partitions: 2 # Number of topic partitions (use -1 to use cluster default)
replicas: 1 # Replication factor per partition (use -1 to use cluster default)
configs:
min.insync.replicas: 1
cleanup.policy: 'delete'
See official Apache Kafka documentation for details about the topic-level configs.
--- lines.KafkaTopicList
If you need to define multiple topics (e.g. using a template), it may be easier to use a KafkaTopicList resource.
Specification
Here the resource definition file for defining a KafkaTopicList.
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaTopicList" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
items: [ ] # An array of KafkaTopic
Example
Here is a simple example that shows how to define a single YAML file containing two topic definitions using
the KafkaTopicList resource type. In addition, the example uses a ConfigMap object to define the topic configuration
only once.
file: kafka-topics.yaml
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: KafkaTopicList
metadata:
labels:
environment: example
items:
- metadata:
name: 'my-topic-p1-r1'
spec:
partitions: 1
replicas: 1
configMapRefs: [ "TopicConfig" ]
- metadata:
name: 'my-topic-p2-r1'
spec:
partitions: 2
replicas: 1
configMapRefs: [ "TopicConfig" ]
---
apiVersion: "core.jikkou.io/v1beta2"
kind: ConfigMap
metadata:
name: 'TopicConfig'
data:
min.insync.replicas: 1
cleanup.policy: 'delete'
3.2.2.4 - Kafka Users
This section describes the resource definition format for KafkaUser entities, which can be used to
manage SCRAM Users for Apache Kafka.
Definition Format of KafkaUser
Below is the overall structure of the KafkaUser resource.
---
apiVersion: kafka.jikkou.io/v1 # The api version (required)
kind: KafkaUser # The resource kind (required)
metadata:
name: <string>
annotations:
# force update
kafka.jikkou.io/force-password-renewal: <boolean>
spec:
authentications:
- type: <enum> # or
password: <string> # leave empty to generate secure password
See below for details about all these fields.
Metadata
metadata.name [required]
The name of the User.
kafka.jikkou.io/force-password-renewal [optional]
Specification
spec.authentications [required]
The list of authentications to manage for the user.
spec.authentications[].type [required]
The authentication type:
scram-sha-256scram-sha-512
spec.authentications[].password [required]
The password of the user.
Examples
The following is an example of a resource describing a User:
---
# Example: file: kafka-scram-users.yaml
apiVersion: "kafka.jikkou.io/v1"
kind: "User"
metadata:
name: "Bob"
spec:
authentications:
- type: scram-sha-256
password: null
- type: scram-sha-512
password: null
Listing Kafka Users
You can retrieve the SCRAM users of a Kafka cluster using the jikkou get kafkausers (or jikkou get ku) command.
Usage
$ jikkou get kc --help
Usage:
Get all 'KafkaUser' resources.
jikkou get kafkausers [-hV] [--list] [--logger-level=<level>] [--name=<name>]
[-o=<format>]
[--selector-match=<selectorMatchingStrategy>]
[-s=<expressions>]...
DESCRIPTION:
Use jikkou get kafkausers when you want to describe the state of all resources
of type 'KafkaUser'.
OPTIONS:
-h, --help Show this help message and exit.
--list Get resources as ResourceListObject (default: false).
--logger-level=<level>
Specify the log level verbosity to be used while
running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN,
ERROR.
For example, `--logger-level=INFO`
--name=<name> The name of the resource.
-o, --output=<format> Prints the output in the specified format. Valid
values: JSON, YAML (default: YAML).
-s, --selector=<expressions>
The selector expression used for including or
excluding resources.
--selector-match=<selectorMatchingStrategy>
The selector matching strategy. Valid values: NONE,
ALL, ANY (default: ALL)
-V, --version Print version information and exit.
(The output from your current Jikkou version may be different from the above example.)
Examples
(command)
$ jikkou get ku
(output)
apiVersion: "kafka.jikkou.io/v1"
kind: "KafkaUser"
metadata:
name: "Bob"
labels: {}
annotations:
kafka.jikkou.io/cluster-id: "xtzWWN4bTjitpL3kfd9s5g"
spec:
authentications:
- type: "scram-sha-256"
iterations: 8192
- type: "scram-sha-512"
iterations: 8192
3.2.2.5 - Kafka Share Groups
This section describes the resource definition format for KafkaShareGroup entities, which can be used to
manage the share groups (Queues, KIP-932)
of a specific Kafka cluster.
Broker requirement
Share groups require a Kafka 4.x cluster with share groups enabled on the brokers (group.share.enable=true and the share.version feature). On older clusters the share group
operations are not available.Listing KafkaShareGroup
You can retrieve the state of Kafka Share Groups using the jikkou get kafkasharegroups (or jikkou get ksg) command.
Examples
(command)
$ jikkou get kafkasharegroups --in-states STABLE --offsets
(output)
---
apiVersion: "kafka.jikkou.io/v1"
kind: "KafkaShareGroup"
metadata:
name: "my-share-group"
spec:
configs:
share.auto.offset.reset: "earliest"
status:
state: "STABLE"
members:
- memberId: "share-consumer-b103994e-bcd5-4236-9d03-97065057e594"
clientId: "share-consumer"
host: "/127.0.0.1"
rackId: "rack-1"
assignments:
- "my-topic-0"
offsets:
- topic: "my-topic"
partition: 0
offset: 0
coordinator:
id: "101"
host: "localhost"
port: 9092
Managing share group configuration
The spec.configs map lets you declaratively manage the dynamic group configuration of a share group
(e.g. share.auto.offset.reset, share.record.lock.duration.ms, share.isolation.level). These are applied
against the Kafka GROUP configuration resource and are reconciled like topic configs (drift detection +
incremental alter).
apiVersion: "kafka.jikkou.io/v1"
kind: "KafkaShareGroup"
metadata:
name: "orders-queue"
spec:
configs:
share.record.lock.duration.ms: 30000
share.auto.offset.reset: "earliest"
Apply it with:
$ jikkou apply --files orders-queue.yaml
By default, group configs present on the cluster but absent from the resource are removed
(config-delete-orphans=true). Set it to false to leave externally-managed configs untouched.
Deleting KafkaShareGroup
A share group can be deleted by reconciling a resource flagged for deletion (jikkou delete),
which removes the group via the AdminClient deleteShareGroups API.
Resetting share group offsets
See the KafkaShareGroupsResetOffsets action
to reset the Share-Partition Start Offset (SPSO) of one or more share groups.
3.2.2.6 - Kafka Authorizations
KafkaPrincipalAuthorization resources are used to define Access Control Lists (ACLs) for principals authenticated to your Kafka Cluster.
Jikkou can be used to describe all ACL policies that need to be created on Kafka Cluster
KafkaPrincipalAuthorization
Specification
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalAuthorization"
metadata:
name: "User:Alice"
spec:
roles: [ ] # List of roles to be added to the principal (optional)
acls: # List of KafkaPrincipalACL (required)
- resource:
type: <The type of the resource> # (required)
pattern: <The pattern to be used for matching resources> # (required)
patternType: <The pattern type> # (required)
type: <The type of this ACL> # ALLOW or DENY (required)
operations: [ ] # Operation that will be allowed or denied (required)
host: <HOST> # IP address from which principal will have access or will be denied (optional)
For more information on how to define authorization and ACLs, see the official Apache Kafka documentation: Security
Operations
The list below describes the valid values for the spec.acls.[].operations property :
READWRITECREATEDELETEALTERDESCRIBECLUSTER_ACTIONDESCRIBE_CONFIGSALTER_CONFIGSIDEMPOTENT_WRITECREATE_TOKENDESCRIBE_TOKENSALL
For more information see official Apache Kafka documentation: Operations in Kafka
Resource Types
The list below describes the valid values for the spec.acls.[].resource.type property :
TOPICGROUPCLUSTERUSERTRANSACTIONAL_ID
For more information see official Apache Kafka documentation: Resources in Kafka
Pattern Types
The list below describes the valid values for the spec.acls.[].resource.patternType property :
LITERAL: Use to allow or denied a principal to have access to a specific resource name.MATCH: Use to allow or denied a principal to have access to all resources matching the given regex.PREFIXED: Use to allow or denied a principal to have access to all resources having the given prefix.
Example
---
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaPrincipalAuthorization" # The resource kind (required)
metadata:
name: "User:Alice"
spec:
acls:
- resource:
type: 'topic'
pattern: 'my-topic-'
patternType: 'PREFIXED'
type: "ALLOW"
operations: [ 'READ', 'WRITE' ]
host: "*"
- resource:
type: 'topic'
pattern: 'my-other-topic-.*'
patternType: 'MATCH'
type: 'ALLOW'
operations: [ 'READ' ]
host: "*"
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalAuthorization"
metadata:
name: "User:Bob"
spec:
acls:
- resource:
type: 'topic'
pattern: 'my-topic-'
patternType: 'PREFIXED'
type: 'ALLOW'
operations: [ 'READ', 'WRITE' ]
host: "*"
KafkaPrincipalRole
Specification
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaPrincipalRole" # The resource kind (required)
metadata:
name: <Name of role> # The name of the role (required)
spec:
acls: [ ] # A list of KafkaPrincipalACL (required)
Example
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalRole"
metadata:
name: "KafkaTopicPublicRead"
spec:
acls:
- type: "ALLOW"
operations: [ 'READ' ]
resource:
type: 'topic'
pattern: '/public-([.-])*/'
patternType: 'MATCH'
host: "*"
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalRole"
metadata:
name: "KafkaTopicPublicWrite"
spec:
acls:
- type: "ALLOW"
operations: [ 'WRITE' ]
resource:
type: 'topic'
pattern: '/public-([.-])*/'
patternType: 'MATCH'
host: "*"
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalAuthorization"
metadata:
name: "User:Alice"
spec:
roles:
- "KafkaTopicPublicRead"
- "KafkaTopicPublicWrite"
---
apiVersion: "kafka.jikkou.io/v1beta2"
kind: "KafkaPrincipalAuthorization"
metadata:
name: "User:Bob"
spec:
roles:
- "KafkaTopicPublicRead"
3.2.2.7 - Kafka Quotas
KafkaClientQuota resources are used to define the quota limits to be applied on Kafka consumers and producers.
A KafkaClientQuota resource can be used to apply limit to consumers and/or producers identified by a client-id or a
user principal.
KafkaClientQuota
Specification
Here is the resource definition file for defining a KafkaClientQuota.
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaClientQuota" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
spec:
type: <The quota type> # (required)
entity:
clientId: <The id of the client> # (required depending on the quota type).
user: <The principal of the user> # (required depending on the quota type).
configs:
requestPercentage: <The quota in percentage (%) of total requests> # (optional)
producerByteRate: <The quota in bytes for restricting data production> # (optional)
consumerByteRate: <The quota in bytes for restricting data consumption> # (optional)
Quota Types
The list below describes the supported quota types:
USERS_DEFAULT: Set default quotas for all users.USER: Set quotas for a specific user principal.USER_CLIENT: Set quotas for a specific user principal and a specific client-id.USER_ALL_CLIENTS: Set default quotas for a specific user and all clients.CLIENT: Set default quotas for a specific client.CLIENTS_DEFAULT: Set default quotas for all clients.
Example
Here is a simple example that shows how to define a single YAML file containing two quota definitions using
the KafkaClientQuota resource type.
file: kafka-quotas.yaml
---
apiVersion: 'kafka.jikkou.io/v1beta2'
kind: 'KafkaClientQuota'
metadata:
labels: { }
annotations: { }
spec:
type: 'CLIENT'
entity:
clientId: 'my-client'
configs:
requestPercentage: 10
producerByteRate: 1024
consumerByteRate: 1024
---
apiVersion: 'kafka.jikkou.io/v1beta2'
kind: 'KafkaClientQuota'
metadata:
labels: { }
annotations: { }
spec:
type: 'USER'
entity:
user: 'my-user'
configs:
requestPercentage: 10
producerByteRate: 1024
consumerByteRate: 1024
KafkaClientQuotaList
If you need to define multiple topics (e.g. using a template), it may be easier to use a KafkaClientQuotaList
resource.
Specification
Here the resource definition file for defining a KafkaTopicList.
apiVersion: "kafka.jikkou.io/v1beta2" # The api version (required)
kind: "KafkaClientQuotaList" # The resource kind (required)
metadata: # (optional)
name: <The name of the topic>
labels: { }
annotations: { }
items: [ ] # An array of KafkaClientQuota
Example
Here is a simple example that shows how to define a single YAML file containing two KafkaClientQuota definition using
the KafkaClientQuotaList resource type.
apiVersion: 'kafka.jikkou.io/v1beta2'
kind: 'KafkaClientQuotaList'
metadata:
labels: { }
annotations: { }
items:
- spec:
type: 'CLIENT'
entity:
clientId: 'my-client'
configs:
requestPercentage: 10
producerByteRate: 1024
consumerByteRate: 1024
- spec:
type: 'USER'
entity:
user: 'my-user'
configs:
requestPercentage: 10
producerByteRate: 1024
consumerByteRate: 1024
3.2.2.8 - Kafka Table Records
A KafkaTableRecord resource can be used to produce a key/value record into a given compacted topic, i.e., a topic
with cleanup.policy=compact (a.k.a. KTable).
KafkaTableRecord
Specification
Here is the resource definition file for defining a KafkaTableRecord.
apiVersion: "kafka.jikkou.io/v1beta1" # The api version (required)
kind: "KafkaTableRecord" # The resource kind (required)
metadata:
labels: { }
annotations: { }
spec:
type: <string> # The topic name (required)
headers: # The list of headers
- name: <string>
value: <string>
key: # The record-key (required)
type: <string> # The record-key type. Must be one of: BINARY, STRING, JSON (required)
data: # The record-key in JSON serialized form.
$ref: <url or path> # Or an url to a local file containing the JSON string value.
value: # The record-value (required)
type: <string> # The record-value type. Must be one of: BINARY, STRING, JSON (required)
data: # The record-value in JSON serialized form.
$ref: <url or path> # Or an url to a local file containing the JSON string value.
Usage
The KafkaTableRecord resource has been designed primarily to manage reference data published and shared via Kafka.
Therefore, it
is highly recommended to use this resource only with compacted Kafka topics containing a small amount of data.
Examples
Here are some examples that show how to a KafkaTableRecord using the different supported data type.
STRING:
---
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaTableRecord"
spec:
topic: "my-topic"
headers:
- name: "content-type"
value: "application/text"
key:
type: STRING
data: |
"bar"
value:
type: STRING
data: |
"foo"
JSON:
---
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaTableRecord"
spec:
topic: "my-topic"
headers:
- name: "content-type"
value: "application/text"
key:
type: STRING
data: |
"bar"
value:
type: JSON
data: |
{
"foo": "bar"
}
BINARY:
---
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaTableRecord"
spec:
topic: "my-topic"
headers:
- name: "content-type"
value: "application/text"
key:
type: STRING
data: |
"bar"
value:
type: BINARY
data: |
"eyJmb28iOiAiYmFyIn0K"
3.2.3 - Transformations
Here, you will find information to use the built-in transformations for Apache Kafka resources.
More information:
3.2.3.1 - KafkaTopicMaxNumPartitions
This transformation can be used to enforce a maximum value for the number of partitions of kafka topics.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
maxNumPartitions | Int | maximum value for the number of partitions to be used for Kafka Topics |
Example
jikkou {
transformations: [
{
type = io.jikkou.kafka.transform.KafkaTopicMaxNumPartitions
priority = 100
config = {
maxNumPartitions = 50
}
}
]
}
3.2.3.2 - KafkaTopicMaxRetentionMs
This transformation can be used to enforce a maximum value for the retention.ms property of kafka topics.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
maxRetentionMs | Int | Minimum value of retention.ms to be used for Kafka Topics |
Example
jikkou {
transformations: [
{
type = io.jikkou.kafka.transform.KafkaTopicMinRetentionMsTransformation
priority = 100
config = {
maxRetentionMs = 2592000000 # 30 days
}
}
]
}
3.2.3.3 - KafkaTopicMinInSyncReplicas
This transformation can be used to enforce a minimum value for the min.insync.replicas property of kafka topics.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
minInSyncReplicas | Int | Minimum value of min.insync.replicas to be used for Kafka Topics |
Example
jikkou {
transformations: [
{
type = io.jikkou.kafka.transform.KafkaTopicMinInSyncReplicasTransformation
priority = 100
config = {
minInSyncReplicas = 2
}
}
]
}
3.2.3.4 - KafkaTopicMinReplicas
This transformation can be used to enforce a minimum value for the replication factor of kafka topics.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
minReplicationFactor | Int | Minimum value of replication factor to be used for Kafka Topics |
Example
jikkou {
transformations: [
{
type = io.jikkou.kafka.transform.KafkaTopicMinReplicasTransformation
priority = 100
config = {
minReplicationFactor = 3
}
}
]
}
3.2.3.5 - KafkaTopicMinRetentionMs
This transformation can be used to enforce a minimum value for the retention.ms property of kafka topics.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
minRetentionMs | Int | Minimum value of retention.ms to be used for Kafka Topics |
Example
jikkou {
transformations: [
{
type = io.jikkou.kafka.transform.KafkaTopicMinRetentionMsTransformation
priority = 100
config = {
minRetentionMs = 604800000 # 7 days
}
}
]
}
3.2.4 - Validations
Jikkou ships with the following built-in validations:
Topics
NoDuplicateTopicsAllowedValidation
(auto registered)
TopicConfigKeysValidation
(auto registered)
type = io.jikkou.kafka.validation.TopicConfigKeysValidation
The TopicConfigKeysValidation allows checking if the specified topic configs are all valid.
TopicMinNumPartitions
type = io.jikkou.kafka.validation.TopicMinNumPartitionsValidation
The TopicMinNumPartitions allows checking if the specified number of partitions for a topic is not less than the minimum required.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicMinNumPartitions | Int | Minimum number of partitions allowed |
TopicMaxNumPartitions
type = io.jikkou.kafka.validation.TopicMaxNumPartitions
The TopicMaxNumPartitions allows checking if the number of partitions for a topic is not greater than the maximum configured.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicMaxNumPartitions | Int | Maximum number of partitions allowed |
TopicMinReplicationFactor
type = io.jikkou.kafka.validation.TopicMinReplicationFactor
The TopicMinReplicationFactor allows checking if the specified replication factor for a topic is not less than the minimum required.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicMinReplicationFactor | Int | Minimum replication factor allowed |
TopicMaxReplicationFactor
type = io.jikkou.kafka.validation.TopicMaxReplicationFactor
The TopicMaxReplicationFactor allows checking if the specified replication factor for a topic is not greater than the maximum configured.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicMaxReplicationFactor | Int | Maximum replication factor allowed |
TopicNamePrefix
type = io.jikkou.kafka.validation.TopicNamePrefix
The TopicNamePrefix allows checking if the specified name for a topic starts with one of the configured suffixes.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicNamePrefixes | List | List of topic name prefixes allows |
TopicNameSuffix
type = io.jikkou.kafka.validation.TopicNameSuffix
The TopicNameSuffix allows checking if the specified name for a topic ends with one of the configured suffixes.
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
topicNameSuffixes | List | List of topic name suffixes allows |
ACLs
NoDuplicateUsersAllowedValidation
(auto registered)
NoDuplicateRolesAllowedValidation
(auto registered)
Quotas
QuotasEntityValidation
(auto registered)
3.2.5 - Annotations
Here, you will find information about the annotations provided the Apache Kafka extension for Jikkou.
List of built-in annotations
kafka.jikkou.io/cluster-id
Used by jikkou.
The annotation is automatically added by Jikkou to a describe object part of an Apache Kafka cluster.
3.2.6 - Actions
Here, you will find the list of actions provided by the Extension Provider for Apache Kafka.
Apache Kafka Action
More information:
3.2.6.1 - KafkaConsumerGroupsResetOffsets
The KafkaConsumerGroupsResetOffsets action allows resetting offsets of consumer group.
It supports one consumer group at the time, and group should be in EMPTY state.
Usage (CLI)
Usage:
Execute the action.
jikkou action KafkaConsumerGroupsResetOffsets execute [-hV] [--all] [--dry-run]
[--to-earliest] [--to-latest] [--group=PARAM] [--logger-level=<level>]
[-o=<format>] [--to-datetime=PARAM] [--to-offset=PARAM] [--excludes=PARAM]...
[--groups=PARAM]... [--includes=PARAM]... --topic=PARAM [--topic=PARAM]...
DESCRIPTION:
Reset offsets of consumer group. Supports multiple consumer groups, and groups
should be in EMPTY state.
You must choose one of the following reset specifications: to-datetime,
by-duration, to-earliest, to-latest, to-offset.
OPTIONS:
--all Specifies to act on all consumer groups.
--dry-run Only show results without executing changes on
Consumer Groups.
--excludes=PARAM List of patterns to match the consumer groups that
must be excluded from the reset-offset action.
--group=PARAM The consumer group to act on.
--groups=PARAM The consumer groups to act on.
-h, --help Show this help message and exit.
--includes=PARAM List of patterns to match the consumer groups that
must be included in the reset-offset action.
--logger-level=<level>
Specify the log level verbosity to be used while
running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN,
ERROR.
For example, `--logger-level=INFO`
-o, --output=<format> Prints the output in the specified format. Allowed
values: JSON, YAML (default YAML).
--to-datetime=PARAM Reset offsets to offset from datetime. Format:
'YYYY-MM-DDTHH:mm:SS.sss'
--to-earliest Reset offsets to earliest offset.
--to-latest Reset offsets to latest offset.
--to-offset=PARAM Reset offsets to a specific offset.
--topic=PARAM The topic whose partitions must be included in the
reset-offset action.
-V, --version Print version information and exit.
Examples
Reset Single Consumer Group to the earliest offsets
jikkou action kafkaconsumergroupresetoffsets execute \
--group my-group \
--topic test \
--to-earliest
(output)
---
kind: "ApiActionResultSet"
apiVersion: "core.jikkou.io/v1"
metadata:
labels: {}
annotations:
configs.jikkou.io/to-earliest: "true"
configs.jikkou.io/group: "my-group"
configs.jikkou.io/dry-run: "false"
configs.jikkou.io/topic:
- "test"
results:
- status: "SUCCEEDED"
errors: []
data:
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConsumerGroup"
metadata:
name: "my-group"
labels:
kafka.jikkou.io/is-simple-consumer: false
annotations: {}
status:
state: "EMPTY"
members: []
offsets:
- topic: "test"
partition: 1
offset: 0
- topic: "test"
partition: 0
offset: 0
- topic: "test"
partition: 2
offset: 0
- topic: "--test"
partition: 0
offset: 0
coordinator:
id: "101"
host: "localhost"
port: 9092
Reset All Consumer Groups to the earliest offsets
jikkou action kafkaconsumergroupresetoffsets execute \
--all \
--topic test \
--to-earliest
3.2.6.2 - KafkaShareGroupsResetOffsets
This section describes the KafkaShareGroupsResetOffsets action, used to reset the
Share-Partition Start Offset (SPSO) of one or more Kafka share groups.
Broker requirement
Requires a Kafka 4.x cluster with share groups enabled (group.share.enable=true).Usage
You must choose one of the following reset specifications: to-earliest, to-latest, to-offset,
or delete-offsets.
| Option | Description |
|---|---|
--group | The share group to act on. |
--groups | The share groups to act on. |
--all | Act on all share groups. |
--topic | Topics to include. Each entry is topic (all partitions) or topic:partition. |
--to-earliest | Reset offsets to the earliest offset. |
--to-latest | Reset offsets to the latest offset. |
--to-offset | Reset offsets to a specific offset. |
--delete-offsets | Delete the share-partition offsets for the given topics. |
--includes | Patterns of share groups to include. |
--excludes | Patterns of share groups to exclude. |
--dry-run | Only show results without executing changes. |
Examples
Reset a single share group to the earliest offset for a topic:
$ jikkou action KafkaShareGroupsResetOffsets execute \
--group my-share-group \
--topic my-topic \
--to-earliest
Delete the share-partition offsets for a topic:
$ jikkou action KafkaShareGroupsResetOffsets execute \
--group my-share-group \
--topic my-topic \
--delete-offsets
3.2.7 - Compatibility
The Apache Kafka extension for Jikkou utilizes the Kafka Admin Client which is compatible with any Kafka infrastructure, such as :
- Aiven
- Apache Kafka
- Confluent Cloud
- MSK
- Redpanda
- etc.
In addition, Kafka Protocol has a “bidirectional” client compatibility policy. In other words, new clients can talk to old servers, and old clients can talk to new servers.
3.3 - Apache Kafka Connect
Here, you will find information to use the Kafka Connect extension for Jikkou.
More information:
3.3.1 - Configuration
This section describes how to configure the Kafka Connect extension.
Configuration
You can configure the properties to be used to connect the Kafka Connect cluster
through the Jikkou client configuration property: jikkou.provider.kafkaconnect.
Example:
jikkou {
provider.kafkaconnect {
enabled = true
type = io.jikkou.kafka.connect.KafkaConnectExtensionProvider
config = {
# Array of Kafka Connect clusters configurations.
clusters = [
{
# Name of the cluster (e.g., dev, staging, production, etc.)
name = "localhost"
# URL of the Kafka Connect service
url = "http://localhost:8083"
# Method to use for authenticating on Kafka Connect. Available values are: [none, basicauth, ssl]
authMethod = none
# Use when 'authMethod' is 'basicauth' to specify the username for Authorization Basic header
basicAuthUser = null
# Use when 'authMethod' is 'basicauth' to specify the password for Authorization Basic header
basicAuthPassword = null
# Enable debug logging
debugLoggingEnabled = false
# Ssl Config: Use when 'authMethod' is 'ssl'
# The location of the key store file.
sslKeyStoreLocation = "/certs/registry.keystore.jks"
# The file format of the key store file.
sslKeyStoreType = "JKS"
# The password for the key store file.
sslKeyStorePassword = "password"
# The password of the private key in the key store file.
sslKeyPassword = "password"
# The location of the trust store file.
sslTrustStoreLocation = "/certs/registry.truststore.jks"
# The file format of the trust store file.
sslTrustStoreType = "JKS"
# The password for the trust store file.
sslTrustStorePassword = "password"
# Specifies whether to ignore the hostname verification.
sslIgnoreHostnameVerification = true
# HTTP proxy: route requests to the Kafka Connect REST API through a forward proxy.
# Proxy URL, e.g. 'http://proxy.example.com:3128'. When empty, JVM proxy system properties are used.
proxyUrl = "http://proxy.example.com:3128"
# Username for proxy Basic authentication (optional).
proxyUsername = null
# Password for proxy Basic authentication (optional).
proxyPassword = null
# Comma-separated hosts that bypass the proxy, e.g. 'localhost,127.0.0.1,*.internal'.
nonProxyHosts = "localhost,127.0.0.1"
}
]
}
}
}
HTTP proxy
Jikkou can reach the Kafka Connect REST API through an HTTP/HTTPS forward proxy.
| Property | Description |
|---|---|
proxyUrl | Proxy URL, e.g. http://proxy.example.com:3128. When empty, JVM proxy system properties are used. |
proxyUsername | Username for proxy Basic authentication (optional). |
proxyPassword | Password for proxy Basic authentication (optional). |
nonProxyHosts | Comma-separated hosts that bypass the proxy, e.g. localhost,127.0.0.1,*.internal. |
If proxyUrl is not set, Jikkou honors the standard JVM proxy system properties
(-Dhttps.proxyHost, -Dhttp.proxyHost, -Dhttp.proxyUser, -Dhttp.proxyPassword,
-Dhttp.nonProxyHosts), which can be supplied via JAVA_TOOL_OPTIONS.
Note
The OS-levelhttp_proxy / https_proxy environment variables are not read by the JVM
and have no effect. Use proxyUrl or the JVM system properties above.3.3.2 - Resources
Here, you will find the list of resources supported by the Kafka Connect Extension.
Kafka Connect Resources
More information:
3.3.2.1 - KafkaConnectors
This section describes the resource definition format for KafkaConnector entities, which can be used to define the
configuration and status of connectors you plan to create and manage on specific Kafka Connect clusters.
Definition Format of KafkaConnector
Below is the overall structure of the KafkaConnector resource.
---
apiVersion: "kafka.jikkou.io/v1beta1" # The api version (required)
kind: "KafkaConnector" # The resource kind (required)
metadata:
name: <string> # The name of the connector (required)
labels:
# Name of the Kafka Connect cluster to create the connector instance in (required).
kafka.jikkou.io/connect-cluster: <string>
annotations:
# Override client properties to connect to Kafka Connect cluster (optional).
jikkou.io/config-override: |
<json>
spec:
connectorClass: <string> # Name or alias of the class for this connector.
tasksMax: <integer> # The maximum number of tasks for the Kafka Connector.
config: # Configuration properties of the connector.
<key>: <value>
state: <string> # The state the connector should be in. Defaults to running.
See below for details about all these fields.
Metadata
metadata.name [required]
The name of the connector.
labels.kafka.jikkou.io/connect-cluster [required]
The name of the Kafka Connect cluster to create the connector instance in.
The cluster name must be configured through the kafkaConnect.clusters[] Jikkou’s configuration setting (see: Configuration).
jikkou.io/config-override: [optional]
The JSON client configurations to override for connecting to the Kafka Connect cluster. The configuration properties passed through this annotation override any cluster properties defined in the Jikkou’s configuration setting (see: Configuration).
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConnector"
metadata:
name: "my-connector"
labels:
kafka.jikkou.io/connect-cluster: "my-connect-cluster"
annotations:
jikkou.io/config-override: |
{ "url": "http://localhost:8083" }
Specification
spec.connectorClass [required]
The name or alias of the class for this connector.
spec.tasksMax [optional]
The maximum number of tasks for the Kafka Connector. Default is 1.
spec.config [required]
The connector’s configuration properties.
spec.state [optional]
The state the connector should be in. Defaults to running.
Below are the valid values:
running: Transition the connector and its tasks to RUNNING state.paused: Pause the connector and its tasks, which stops message processing until the connector is resumed.stopped: Completely shut down the connector and its tasks. The connector config remains present in the config topic of the cluster (if running in distributed mode), unmodified.
Examples
The following is an example of a resource describing a Kafka connector:
---
# Example: file: kafka-connector-filestream-sink.yaml
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConnector"
metadata:
name: "local-file-sink"
labels:
kafka.jikkou.io/connect-cluster: "my-connect-cluster"
spec:
connectorClass: "FileStreamSink"
tasksMax: 1
config:
file: "/tmp/test.sink.txt"
topics: "connect-test"
state: "RUNNING"
Listing KafkaConnector
You can retrieve the state of Kafka Connector instances running on your Kafka Connect clusters using the jikkou get kafkaconnectors (or jikkou get kc) command.
Usage
$jikkou get kc --help
Usage:
Get all 'KafkaConnector' resources.
jikkou get kafkaconnectors [-hV] [--expand-status] [-o=<format>]
[-s=<expressions>]...
Description:
Use jikkou get kafkaconnectors when you want to describe the state of all
resources of type 'KafkaConnector'.
Options:
--expand-status Retrieves additional information about the status of
the connector and its tasks.
-h, --help Show this help message and exit.
-o, --output=<format> Prints the output in the specified format. Allowed
values: json, yaml (default yaml).
-s, --selector=<expressions>
The selector expression use for including or
excluding resources.
-V, --version Print version information and exit.
(The output from your current Jikkou version may be different from the above example.)
Examples
(command)
$ jikkou get kc --expand-status
(output)
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConnector"
metadata:
name: "local-file-sink"
labels:
kafka.jikkou.io/connect-cluster: "localhost"
spec:
connectorClass: "FileStreamSink"
tasksMax: 1
config:
file: "/tmp/test.sink.txt"
topics: "connect-test"
state: "RUNNING"
status:
connectorStatus:
name: "local-file-sink"
connector:
state: "RUNNING"
worker_id: "localhost:8083"
tasks:
id: 1
state: "RUNNING"
worker_id: "localhost:8083"
The status.connectorStatus provides the connector status, as reported by the Kafka Connect REST API.
3.3.3 - Validations
Jikkou ships with the following built-in validations:
3.3.4 - Annotations
This section lists a number of well known annotations, that have defined semantics. They can be attached
to KafkaConnect resources through the metadata.annotations field and consumed as needed by extensions (i.e., validations, transformations, controller,
collector, etc.).
List of built-in annotations
3.3.5 - Labels
This section lists a number of well known labels, that have defined semantics. They can be attached
to KafkaConnect resources through the metadata.labels field and consumed as needed by extensions (i.e., validations, transformations, controller,
collector, etc.).
Labels
kafka.jikkou.io/connect-cluster
# Example
---
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConnector"
metadata:
labels:
kafka.jikkou.io/connect-cluster: 'my-connect-cluster'
The value of this label defined the name of the Kafka Connect cluster to create the connector instance in.
The cluster name must be configured through the kafkaConnect.clusters[] Jikkou’s configuration setting (see: Configuration).
3.3.6 - Actions
Here, you will find the list of actions provided by the Extension Provider for Kafka Connect.
Kafka Connect Action
More information:
3.3.6.1 - KafkaConnectRestartConnectors
The KafkaConnectRestartConnectors action allows a user to restart all or just the
failed Connector and Task instances for one or multiple named connectors.
Usage (CLI)
Usage:
Execute the action.
jikkou action KafkaConnectRestartConnectors execute [-hV] [--include-tasks]
[--only-failed] [--connect-cluster=PARAM] [--logger-level=<level>]
[-o=<format>] [--connector-name=PARAM]...
DESCRIPTION:
The KafkaConnectRestartConnectors action a user to restart all or just the
failed Connector and Task instances for one or multiple named connectors.
OPTIONS:
--connect-cluster=PARAM
The name of the connect cluster.
--connector-name=PARAM
The connector's name.
-h, --help Show this help message and exit.
--include-tasks Specifies whether to restart the connector instance
and task instances (includeTasks=true) or just the
connector instance (includeTasks=false)
--logger-level=<level>
Specify the log level verbosity to be used while
running a command.
Valid level values are: TRACE, DEBUG, INFO, WARN,
ERROR.
For example, `--logger-level=INFO`
-o, --output=<format> Prints the output in the specified format. Allowed
values: JSON, YAML (default YAML).
--only-failed Specifies whether to restart just the instances with
a FAILED status (onlyFailed=true) or all instances
(onlyFailed=false)
-V, --version Print version information and exit.
Examples
Restart all connectors for all Kafka Connect clusters.
jikkou action kafkaconnectrestartconnectors execute
(output)
---
kind: "ApiActionResultSet"
apiVersion: "core.jikkou.io/v1"
metadata:
labels: {}
annotations: {}
results:
- status: "SUCCEEDED"
data:
apiVersion: "kafka.jikkou.io/v1beta1"
kind: "KafkaConnector"
metadata:
name: "local-file-sink"
labels:
kafka.jikkou.io/connect-cluster: "my-connect-cluster"
annotations: {}
spec:
connectorClass: "FileStreamSink"
tasksMax: 1
config:
file: "/tmp/test.sink.txt"
topics: "connect-test"
state: "RUNNING"
status:
connectorStatus:
name: "local-file-sink"
connector:
state: "RUNNING"
workerId: "connect:8083"
tasks:
- id: 0
state: "RUNNING"
workerId: "connect:8083"
Restart all connectors with a FAILED status on all Kafka Connect clusters.
jikkou action kafkaconnectrestartconnectors execute \
--only-failed
Restart specific connector and tasks for on Kafka Connect cluster
jikkou action kafkaconnectrestartconnectors execute \
--cluster-name my-connect-cluster
--connector-name local-file-sink \
--include-tasks
3.4 - Schema Registry
Here, you will find information to use the Schema Registry extensions.
More information:
3.4.1 - Configuration
Here, you will find the list of resources supported for SchemaRegistry.
Configuration
You can configure the properties to be used to connect the SchemaRegistry service
through the Jikkou client configuration property jikkou.provider.schemaregistry.
Example:
jikkou {
provider.schemaregistry {
enabled = true
type = io.jikkou.schema.registry.SchemaRegistryExtensionProvider
config = {
# Comma-separated list of URLs for schema registry instances that can be used to register or look up schemas
url = "http://localhost:8081"
# The name of the schema registry implementation vendor - can be any value
vendor = generic
# Method to use for authenticating on Schema Registry. Available values are: [none, basicauth, ssl]
authMethod = none
# Use when 'schemaRegistry.authMethod' is 'basicauth' to specify the username for Authorization Basic header
basicAuthUser = null
# Use when 'schemaRegistry.authMethod' is 'basicauth' to specify the password for Authorization Basic header
basicAuthPassword = null
# Enable debug logging
debugLoggingEnabled = false
# Ssl Config: Use when 'authMethod' is 'ssl'
# The location of the key store file.
sslKeyStoreLocation = "/certs/registry.keystore.jks"
# The file format of the key store file.
sslKeyStoreType = "JKS"
# The password for the key store file.
sslKeyStorePassword = "password"
# The password of the private key in the key store file.
sslKeyPassword = "password"
# The location of the trust store file.
sslTrustStoreLocation = "/certs/registry.truststore.jks"
# The file format of the trust store file.
sslTrustStoreType = "JKS"
# The password for the trust store file.
sslTrustStorePassword = "password"
# Specifies whether to ignore the hostname verification.
sslIgnoreHostnameVerification = true
# HTTP proxy: route requests to the Schema Registry through a forward proxy.
# Proxy URL, e.g. 'http://proxy.example.com:3128'. When empty, JVM proxy system properties are used.
proxyUrl = "http://proxy.example.com:3128"
# Username for proxy Basic authentication (optional).
proxyUsername = null
# Password for proxy Basic authentication (optional).
proxyPassword = null
# Comma-separated hosts that bypass the proxy, e.g. 'localhost,127.0.0.1,*.internal'.
nonProxyHosts = "localhost,127.0.0.1"
}
}
}
HTTP proxy
Jikkou can reach the Schema Registry REST API through an HTTP/HTTPS forward proxy.
| Property | Description |
|---|---|
proxyUrl | Proxy URL, e.g. http://proxy.example.com:3128. When empty, JVM proxy system properties are used. |
proxyUsername | Username for proxy Basic authentication (optional). |
proxyPassword | Password for proxy Basic authentication (optional). |
nonProxyHosts | Comma-separated hosts that bypass the proxy, e.g. localhost,127.0.0.1,*.internal. |
If proxyUrl is not set, Jikkou honors the standard JVM proxy system properties
(-Dhttps.proxyHost, -Dhttp.proxyHost, -Dhttp.proxyUser, -Dhttp.proxyPassword,
-Dhttp.nonProxyHosts), which can be supplied via JAVA_TOOL_OPTIONS.
Note
The OS-levelhttp_proxy / https_proxy environment variables are not read by the JVM
and have no effect. Use proxyUrl or the JVM system properties above.3.4.2 - Resources
Here, you will find the list of resources supported for Schema Registry.
Schema Registry Resources
More information:
3.4.2.1 - Schema Registry Subjects
SchemaRegistrySubject resources are used to define the subject schemas you want to manage on your SchemaRegistry. A SchemaRegistrySubject resource defines the schema, the compatibility level, and the references to be associated with a subject version.
Definition Format of SchemaRegistrySubject
Below is the overall structure of the SchemaRegistrySubject resource.
apiVersion: "schemaregistry.jikkou.io/v1beta2" # The api version (required)
kind: "SchemaRegistrySubject" # The resource kind (required)
metadata:
name: <The name of the subject> # (required)
labels: { }
annotations: { }
spec:
schemaRegistry:
vendor: <vendor_name> # (optional) The vendor of the SchemaRegistry, e.g., Confluent, Karapace, etc
compatibilityLevel: <compatibility_level> # (optional) The schema compatibility level for this subject.
schemaType: <The schema format> # (required) Accepted values are: AVRO, PROTOBUF, JSON
schema:
$ref: <url or path> #
references: # Specifies the names of referenced schemas (optional array).
- name: <string> # The name for the reference.
subject: <string> # The subject under which the referenced schema is registered.
version: <string> # The exact version of the schema under the registered subject.
]
Metadata
The metadata.name property is mandatory and specifies the name of the Subject.
Specification
To use the SchemaRegistry default values for the compatibilityLevel you can omit the property.
Example
Here is a simple example that shows how to define a single subject AVRO schema for type using
the SchemaRegistrySubject resource type.
file: subject-user.yaml
---
apiVersion: "schemaregistry.jikkou.io/v1beta2"
kind: "SchemaRegistrySubject"
metadata:
name: "User"
labels: { }
annotations:
schemaregistry.jikkou.io/normalize-schema: true
spec:
compatibilityLevel: "FULL_TRANSITIVE"
schemaType: "AVRO"
schema:
$ref: ./user-schema.avsc
file: user-schema.avsc
---
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null,
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string" ],
"default": null
}
]
}
Alternatively, we can directly pass the Avro schema as follows :
file: subject-user.yaml
---
apiVersion: "schemaregistry.jikkou.io/v1beta2"
kind: "SchemaRegistrySubject"
metadata:
name: "User"
labels: { }
annotations:
schemaregistry.jikkou.io/normalize-schema: true
spec:
compatibilityLevel: "FULL_TRANSITIVE"
schemaType: "AVRO"
schema: |
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string"],
"default": null
}
]
}
SchemaRegistrySubjectList
If you need to manage multiple Schemas at once (e.g. using a template), it may be more suitable to use the resource collection SchemaRegistrySubjectList.
Specification
Here the resource definition file for defining a SchemaRegistrySubjectList.
apiVersion: "schemaregistry.jikkou.io/v1beta2" # The api version (required)
kind: "SchemaRegistrySubjectList" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
items: [ ] # The array of SchemaRegistrySubject
3.4.3 - Validations
Jikkou ships with the following built-in validations:
Subject
SchemaCompatibilityValidation
type = io.jikkou.schema.registry.validation.SchemaCompatibilityValidation
The SchemaCompatibilityValidation allows testing the compatibility of the schema with the latest
version already registered in the Schema Registry using the provided compatibility-level.
AvroSchemaValidation
The AvroSchemaValidation allows checking if the specified Avro schema matches some specific avro schema definition
rules;
type = io.jikkou.schema.registry.validation.AvroSchemaValidation
Configuration
| Name | Type | Description | Default |
|---|---|---|---|
fieldsMustHaveDoc | Boolean | Verify that all record fields have a doc property | false |
fieldsMustBeNullable | Boolean | Verify that all record fields are nullable | false |
fieldsMustBeOptional | Boolean | Verify that all record fields are optional | false |
3.4.4 - Annotations
Here, you will find information about the annotations provided by the Schema Registry extension for Jikkou.
List of built-in annotations
schemaregistry.jikkou.io/url
Used by jikkou.
The annotation is automatically added by Jikkou to describe the SchemaRegistry URL from which a subject schema is retrieved.
schemaregistry.jikkou.io/schema-version
Used by jikkou.
The annotation is automatically added by Jikkou to describe the version of a subject schema.
schemaregistry.jikkou.io/schema-id
Used by jikkou.
The annotation is automatically added by Jikkou to describe the version of a subject id.
schemaregistry.jikkou.io/normalize-schema
Used on: schemaregistry.jikkou.io/v1beta2:SchemaRegistrySubject
This annotation can be used to normalize the schema on SchemaRegistry server side. Note, that Jikkou will attempt to normalize AVRO and JSON schema.
See: Confluent SchemaRegistry API Reference
schemaregistry.jikkou.io/permanante-delete
Used on: schemaregistry.jikkou.io/v1beta2:SchemaRegistrySubject
The annotation can be used to specify a hard delete of the subject, which removes all associated metadata including the schema ID. The default is false. If the flag is not included, a soft delete is performed. You must perform a soft delete first, then the hard delete.
See: Confluent SchemaRegistry API Reference
schemaregistry.jikkou.io/use-canonical-fingerprint
This annotation can be used to use a canonical fingerprint to compare schemas (only supported for Avro schema).
3.5 - Aiven
Here, you will find information to use the Aiven for Kafka extensions.
More information:
3.5.1 - Configuration
Here, you will find the list of resources supported by the extension for Aiven.
Configuration
You can configure the properties to be used to connect the Aiven service
through the Jikkou client configuration property jikkou.provider.aiven.
Example:
jikkou {
provider.aiven {
enabled = true
type = io.jikkou.extension.aiven.AivenExtensionProvider
config = {
# Aiven project name
project = "http://localhost:8081"
# Aiven service name
service = generic
# URL to the Aiven REST API.
apiUrl = "https://api.aiven.io/v1/"
# Aiven Bearer Token. Tokens can be obtained from your Aiven profile page
tokenAuth = null
# HTTP proxy URL, e.g. 'http://proxy.example.com:3128'. When empty, JVM proxy system properties are used.
proxyUrl = "http://proxy.example.com:3128"
# Username for proxy Basic authentication (optional).
proxyUsername = null
# Password for proxy Basic authentication (optional).
proxyPassword = null
# Comma-separated hosts that bypass the proxy, e.g. 'localhost,127.0.0.1,*.internal'.
nonProxyHosts = "localhost,127.0.0.1"
# Enable debug logging
debugLoggingEnabled = false
}
}
}
HTTP proxy
Jikkou can reach the Aiven REST API through an HTTP/HTTPS forward proxy.
| Property | Description |
|---|---|
proxyUrl | Proxy URL, e.g. http://proxy.example.com:3128. When empty, JVM proxy system properties are used. |
proxyUsername | Username for proxy Basic authentication (optional). |
proxyPassword | Password for proxy Basic authentication (optional). |
nonProxyHosts | Comma-separated hosts that bypass the proxy, e.g. localhost,127.0.0.1,*.internal. |
If proxyUrl is not set, Jikkou honors the standard JVM proxy system properties
(-Dhttps.proxyHost, -Dhttp.proxyHost, -Dhttp.proxyUser, -Dhttp.proxyPassword,
-Dhttp.nonProxyHosts), which can be supplied via JAVA_TOOL_OPTIONS.
Note
The OS-levelhttp_proxy / https_proxy environment variables are not read by the JVM
and have no effect. Use proxyUrl or the JVM system properties above.3.5.2 - Resources
Here, you will find the list of resources supported by the extensions for Aiven.
Aiven for Apache Kafka Resources
More information:
3.5.2.1 - ACL for Aiven Apache Kafka®
The KafkaTopicAclEntry resources are used to manage the Access Control Lists in Aiven for Apache Kafka®. A
KafkaTopicAclEntry resource defines the permission to be granted to a user for one or more kafka topics.
KafkaTopicAclEntry
Specification
Here is the resource definition file for defining a KafkaTopicAclEntry.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "KafkaTopicAclEntry" # The resource kind (required)
metadata:
labels: { }
annotations: { }
spec:
permission: <> # The permission. Accepted values are: READ, WRITE, READWRITE, ADMIN
username: <> # The username
topic: <> # Topic name or glob pattern
Example
Here is a simple example that shows how to define a single ACL entry using
the KafkaTopicAclEntry resource type.
file: kafka-topic-acl-entry.yaml
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaTopicAclEntry"
metadata:
labels: { }
annotations: { }
spec:
permission: "READWRITE"
username: "alice"
topic: "public-*"
KafkaTopicAclEntryList
If you need to define multiple ACL entries (e.g. using a template), it may be easier to use a KafkaTopicAclEntryList resource.
Specification
Here the resource definition file for defining a KafkaTopicList.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "KafkaTopicAclEntryList" # The resource kind (required)
metadata: # (optional)
name: <The name of the topic>
labels: { }
annotations: { }
items: [ ] # An array of KafkaTopicAclEntry
Example
Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using
the KafkaTopicAclEntryList resource type.
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaTopicAclEntryList"
items:
- spec:
permission: "READWRITE"
username: "alice"
topic: "public-*"
- spec:
permission: "READ"
username: "bob"
topic: "public-*"
3.5.2.2 - Quotas for Aiven Apache Kafka®
The KafkaQuota resources are used to manage the Quotas in Aiven for Apache Kafka® service.
For more details, see https://docs.aiven.io/docs/products/kafka/concepts/kafka-quotas
KafkaQuota
Specification
Here is the resource definition file for defining a KafkaQuota.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "KafkaQuota" # The resource kind (required)
metadata:
labels: { }
annotations: { }
spec:
user: <string> # The username: (Optional: 'default' if null)
clientId: <string> # The client-id
consumerByteRate: <number> # The quota in bytes for restricting data consumption
producerByteRate: <number> # The quota in bytes for restricting data production
requestPercentage: <number>
Example
Here is a simple example that shows how to define a single ACL entry using
the KafkaQuota resource type.
file: kafka-quotas.yaml
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaQuota"
spec:
user: "default"
clientId: "default"
consumerByteRate: 1048576
producerByteRate: 1048576
requestPercentage: 25
KafkaQuotaList
If you need to define multiple Kafka quotas (e.g. using a template), it may be easier to use a KafkaQuotaList resource.
Specification
Here the resource definition file for defining a KafkaTopicList.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "KafkaQuotaList" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
items: [ ] # An array of KafkaQuotaList
Example
Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using
the KafkaQuotaList resource type.
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaQuotaList"
items:
- spec:
user: "default"
clientId: "default"
consumerByteRate: 1048576
producerByteRate: 1048576
requestPercentage: 5
- spec:
user: "avnadmin"
consumerByteRate: 5242880
producerByteRate: 5242880
requestPercentage: 25
3.5.2.3 - ACL for Aiven Schema Registry
The SchemaRegistryAclEntry resources are used to manage the Access Control Lists in Aiven for Schema Registry. A
SchemaRegistryAclEntry resource defines the permission to be granted to a user for one or more Schema Registry
Subjects.
SchemaRegistryAclEntry
Specification
Here is the resource definition file for defining a SchemaRegistryAclEntry.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "SchemaRegistryAclEntry" # The resource kind (required)
metadata:
labels: { }
annotations: { }
spec:
permission: <> # The permission. Accepted values are: READ, WRITE
username: <> # The username
resource: <> # The Schema Registry ACL entry resource name pattern
NOTE: The resource name pattern should be Config: or Subject:<subject_name> where subject_name must consist of
alpha-numeric characters, underscores, dashes, dots and glob characters * and ?.
Example
Here is an example that shows how to define a simple ACL entry using
the SchemaRegistryAclEntry resource type.
file: schema-registry-acl-entry.yaml
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistryAclEntry"
spec:
permission: "READ"
username: "Alice"
resource: "Subject:*"
SchemaRegistryAclEntryList
If you need to define multiple ACL entries (e.g. using a template), it may be easier to use
a SchemaRegistryAclEntryList resource.
Specification
Here the resource definition file for defining a SchemaRegistryAclEntryList.
---
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "SchemaRegistryAclEntryList" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
items: [ ] # An array of SchemaRegistryAclEntry
Example
Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using
the SchemaRegistryAclEntryList resource type.
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistryAclEntryList"
items:
- spec:
permission: "READ"
username: "alice"
resource: "Config:"
- spec:
permission: "WRITE"
username: "alice"
resource: "Subject:*"
3.5.2.4 - Subject for Aiven Schema Registry
SchemaRegistrySubject resources are used to define the subject schemas you want to manage on your Schema Registry. A SchemaRegistrySubject resource defines the schema, the compatibility level, and the references to be associated with a subject version.
SchemaRegistrySubject
Specification
Here is the resource definition file for defining a SchemaRegistrySubject.
apiVersion: "kafka.aiven.io/v1beta1" # The api version (required)
kind: "SchemaRegistrySubject" # The resource kind (required)
metadata:
name: <The name of the subject> # (required)
labels: { }
annotations: { }
spec:
schemaRegistry:
vendor: 'Karapace' # (optional) The vendor of the Schema Registry
compatibilityLevel: <compatibility_level> # (optional) The schema compatibility level for this subject.
schemaType: <The schema format> # (required) Accepted values are: AVRO, PROTOBUF, JSON
schema:
$ref: <url or path> #
references: # Specifies the names of referenced schemas (optional array).
- name: <> # The name for the reference.
subject: <> # The subject under which the referenced schema is registered.
version: <> # The exact version of the schema under the registered subject.
]
The metadata.name property is mandatory and specifies the name of the Subject.
To use the SchemaRegistry default values for the compatibilityLevel you can omit the property.
Example
Here is a simple example that shows how to define a single subject AVRO schema for type using
the SchemaRegistrySubject resource type.
file: subject-user.yaml
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistrySubject"
metadata:
name: "User"
labels: { }
annotations:
schemaregistry.jikkou.io/normalize-schema: true
spec:
compatibilityLevel: "FULL_TRANSITIVE"
schemaType: "AVRO"
schema:
$ref: ./user-schema.avsc
file: user-schema.avsc
---
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null,
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string" ],
"default": null
}
]
}
Alternatively, we can directly pass the Avro schema as follows :
file: subject-user.yaml
---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistrySubject"
metadata:
name: "User"
labels: { }
annotations:
schemaregistry.jikkou.io/normalize-schema: true
spec:
compatibilityLevel: "FULL_TRANSITIVE"
schemaType: "AVRO"
schema: |
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string"],
"default": null
}
]
}
3.5.3 - Validations
Jikkou ships with the following built-in validations:
No validation
3.5.4 - Annotations
Here, you will find information about the annotations provided by the Aiven extension for Jikkou.
List of built-in annotations
kafka.aiven.io/acl-entry-id
Used by jikkou.
The annotation is automatically added by Jikkou to describe the ID of an ACL entry.
3.6 - AWS
Here, you will find information to use the AWS extensions (Jikkou v0.36).
More information:
3.6.1 - Configuration
Here, you will find the list of resources supported by the extension for AWS.
Configuration
You can configure the properties to be used to connect the AWS services
through the Jikkou client configuration property jikkou.providers.
Example:
jikkou {
# AWS
provider.aws {
enabled = true
config = {
# The AWS S3 Region, e.g. us-east-1
aws.client.region = ""
# The AWS Access Key ID.
aws.client.accessKeyId = ""
# The AWS Secret Access Key.
aws.client.secretAccessKey = ""
# The AWS session token.
aws.client.sessionToken = ""
# The endpoint with which the SDK should communicate allowing you to use a different S3 compatible service
aws.client.endpointOverride = ""
# The name of the registries. Used only for lookup.
aws.glue.registryNames = ""
}
}
}
3.6.2 - Resources
Here, you will find the list of resources supported by the extensions for AWS.
AWS Resources
More information:
3.6.2.1 - Schema for AWS Glue Schema Registry
AwsGlueSchema resources are used to define the schemas you want to manage on your AWS Glue Schema registry. A AwsGlueSchema resource defines the schema, and the compatibility mode to be associated with a subject definition.
AwsGlueSchema
Specification
Here is the resource definition file for defining a AwsGlueSchema.
apiVersion: "aws.jikkou.io/v1" # The api version (required)
kind: "AwsGlueSchema" # The resource kind (required)
metadata:
name: <The name of the subject> # The schema name (required)
labels:
glue.aws.amazon.com/registry-name: # The registry name (required)
annotations: { }
spec:
compatibility: <compatibility> # The schema compatibility level for this subject (required).
dataFormat: <The data format> # Accepted values are: AVRO, PROTOBUF, JSON (required).
schemaDefinition:
$ref: <url or path> #
]
The metadata.name property is mandatory and specifies the name of the Subject.
Compatibility
Supported compatibility mode are:
BACKWARD(recommended) — Consumer can read both current and previous version.BACKWARD_ALL— Consumer can read current and all previous versions.FORWARD— Consumer can read both current and subsequent version.FORWARD_ALL— Consumer can read both current and all subsequent versions.FULL— Combination of Backward and Forward.FULL_ALL— Combination of Backward all and Forward all.NONE— No compatibility checks are performed.DISABLED— Prevent any versioning for this schema
Example
Here is a simple example that shows how to define a single subject AVRO schema for type using
the AwsGlueSchema resource type.
file: subject-user.yaml
---
apiVersion: "aws.jikkou.io/v1"
kind: "AwsGlueSchema"
metadata:
name: "User"
labels:
glue.aws.amazon.com/registry-name: Test
annotations:
glue.aws.amazon.com/normalize-schema: true
spec:
compatibility: "BACKWARD"
dataFormat: "AVRO"
schema:
$ref: ./user-schema.avsc
file: user-schema.avsc
---
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null,
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string" ],
"default": null
}
]
}
Alternatively, we can directly pass the Avro schema as follows :
file: subject-user.yaml
---
apiVersion: "aws.jikkou.io/v1"
kind: "AwsGlueSchema"
metadata:
name: "User"
labels: { }
annotations:
glue.aws.amazon.com/normalize-schema: true
spec:
compatibility: "BACKWARD"
dataFormat: "AVRO"
schema: |
{
"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{
"name": "name",
"type": [ "null", "string" ],
"default": null
},
{
"name": "favorite_number",
"type": [ "null", "int" ],
"default": null
},
{
"name": "favorite_color",
"type": [ "null", "string"],
"default": null
}
]
}
3.6.3 - Validations
Jikkou ships with the following built-in validations:
No validation
3.6.4 - Annotations
Here, you will find information about the annotations provided by the AWS extension for Jikkou.
List of built-in annotations
glue.aws.amazon.com/created-time
The date and time the schema was created.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/updated-time
The date and time the schema was updated.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/registry-name
The name of the registry.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/registry-arn
The Amazon Resource Name (ARN) of the registry.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/schema-arn
The Amazon Resource Name (ARN) of the schema.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/schema-version-id
The SchemaVersionId of the schema version.
The annotation is automatically added by Jikkou.
glue.aws.amazon.com/use-canonical-fingerprint
This annotation can be used to use a canonical fingerprint to compare schemas (only supported for Avro schema).
3.7 - Apache Iceberg
Here, you will find information to use the Apache Iceberg extensions.
More information:
3.7.1 - Configuration
Here, you will find the configuration properties for the Apache Iceberg extension.
Configuration
The Apache Iceberg extension connects to an Iceberg catalog. You configure it through the Jikkou
client configuration property jikkou.provider.iceberg.
Example (JDBC catalog with PostgreSQL):
jikkou {
provider.iceberg {
enabled = true
type = io.jikkou.iceberg.IcebergExtensionProvider
config = {
# Required — type of Iceberg catalog.
# Accepted values: rest, hive, jdbc, glue, nessie, hadoop
catalogType = "jdbc"
# The catalog name used to identify this catalog instance.
catalogName = "default"
# The URI of the catalog endpoint (REST URL, Hive Metastore URI, JDBC URL, etc.)
catalogUri = "jdbc:postgresql://localhost:5432/iceberg"
# The warehouse root location (e.g., local path, S3 bucket path, HDFS path)
warehouse = "/tmp/iceberg-warehouse"
# Extra catalog-specific properties passed directly to CatalogUtil.buildIcebergCatalog()
catalogProperties {
jdbc.user = "iceberg"
jdbc.password = "iceberg"
}
# Enable verbose debug logging for catalog operations (default: false)
debugLoggingEnabled = false
}
}
}
Configuration Properties
| Property | Type | Required | Default | Description |
|---|---|---|---|---|
catalogType | String | yes | — | Iceberg catalog type: rest, hive, jdbc, glue, nessie, hadoop |
catalogName | String | no | default | The catalog instance name |
catalogUri | String | no | — | Catalog endpoint URI (REST API URL, Hive Metastore thrift URI, JDBC URL, Nessie server URL) |
warehouse | String | no | — | Warehouse root location (e.g., s3://bucket/warehouse, /tmp/iceberg) |
catalogProperties | Map | no | — | Additional catalog properties forwarded verbatim to the Iceberg CatalogUtil |
debugLoggingEnabled | Boolean | no | false | Enable debug-level logging for catalog operations |
Catalog Types
JDBC Catalog (PostgreSQL)
Stores catalog metadata (namespaces, table specs) in a relational database. The PostgreSQL JDBC driver is bundled in the Jikkou CLI distribution.
config = {
catalogType = "jdbc"
catalogUri = "jdbc:postgresql://localhost:5432/iceberg"
warehouse = "/tmp/iceberg-warehouse"
catalogProperties {
jdbc.user = "iceberg"
jdbc.password = "iceberg"
}
}
REST Catalog
Connects to any Iceberg REST Catalog API (e.g., Polaris, Gravitino, Unity Catalog):
config = {
catalogType = "rest"
catalogUri = "https://polaris.example.com/api/catalog"
warehouse = "s3://my-bucket/warehouse"
catalogProperties {
rest.signing-name = "execute-api"
rest.signing-region = "us-east-1"
}
}
Hive Metastore
Connects to an Apache Hive Metastore (requires iceberg-hive-metastore on the classpath):
config = {
catalogType = "hive"
catalogUri = "thrift://hive-metastore:9083"
warehouse = "hdfs://namenode:8020/user/hive/warehouse"
}
AWS Glue
Connects to AWS Glue Data Catalog (requires iceberg-aws on the classpath):
config = {
catalogType = "glue"
warehouse = "s3://my-bucket/warehouse"
catalogProperties {
glue.region = "us-east-1"
}
}
Nessie
Nessie exposes a standard Iceberg REST catalog endpoint at /iceberg. Using
catalogType = "rest" is recommended because it relies only on iceberg-core
(always bundled in the Jikkou CLI). The catalogType = "nessie" variant requires
the optional iceberg-nessie JAR on the classpath.
# Recommended: use Nessie's built-in Iceberg REST endpoint
config = {
catalogType = "rest"
catalogUri = "http://nessie:19120/iceberg"
warehouse = "s3://my-bucket/warehouse"
catalogProperties {
prefix = "main" # Nessie branch
}
}
Controller Settings
The table and view controllers expose additional options to control reconciliation behaviour.
These are set inside the provider config block.
Table Controller
| Property | Type | Default | Description |
|---|---|---|---|
delete-orphans | Boolean | false | Drop tables that exist in the catalog but are not defined in any resource |
delete-orphan-columns | Boolean | false | Drop columns present in the live table but absent from the spec |
delete-purge | Boolean | false | Purge underlying data files when dropping a table (irreversible) |
tables.deletion.exclude | List<Pattern> | [] | Regex patterns — matching table names are never deleted |
View Controller
| Property | Type | Default | Description |
|---|---|---|---|
delete-orphans | Boolean | false | Drop views that exist in the catalog but are not defined in any resource |
views.deletion.exclude | List<Pattern> | [] | Regex patterns — matching view names are never deleted |
Note
Thedelete-orphans property applies independently to tables and views.
The IcebergNamespaceController always uses delete-orphans = false to prevent accidental
deletion of namespaces that may still contain tables. To delete a namespace, use the DELETE
reconciliation mode explicitly.Example:
jikkou {
provider.iceberg {
enabled = true
type = io.jikkou.iceberg.IcebergExtensionProvider
config = {
catalogType = "rest"
catalogUri = "http://localhost:8181"
warehouse = "s3://my-bucket/warehouse"
# Table reconciliation safety settings
delete-orphans = false
delete-orphan-columns = false
delete-purge = false
# Never delete tables whose name starts with "audit_"
tables.deletion.exclude = ["^audit_.*"]
# Never delete views whose name starts with "v_core_"
views.deletion.exclude = ["^v_core_.*"]
}
}
}
3.7.2 - Resources
Here, you will find the list of resources supported by the extensions for Apache Iceberg.
Iceberg Resources
The Apache Iceberg extension provides the following resource types:
| Resource | Description |
|---|---|
IcebergNamespace | Manage namespaces (databases) in an Iceberg catalog |
IcebergTable | Manage tables with schema evolution, partitioning, and sort order |
IcebergView | Manage SQL views backed by one or more dialect-specific queries |
More information:
3.7.2.1 - Iceberg Namespace
IcebergNamespace resources are used to define the namespaces (databases) you want to manage in your
Iceberg catalog. A namespace groups tables and carries metadata as key/value properties.
IcebergNamespace
Specification
Here is the resource definition file for defining an IcebergNamespace.
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergNamespace" # The resource kind (required)
metadata:
name: <namespace name> # Dot-separated namespace path (required)
labels: { }
annotations: { }
spec:
properties: # Namespace-level metadata (optional)
<key>: <value>
The metadata.name property is mandatory. Nested namespaces are expressed using dot notation —
for example, analytics.events creates a namespace events inside analytics.
Example
file: iceberg-namespaces.yaml
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergNamespace"
metadata:
name: "analytics"
spec:
properties:
owner: "data-team"
environment: "production"
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergNamespace"
metadata:
name: "analytics.events"
spec:
properties:
owner: "data-team"
team: "platform"
---.IcebergNamespaceList
If you need to define multiple namespaces (e.g., using a template), it may be easier to use
an IcebergNamespaceList resource.
Specification
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergNamespaceList" # The resource kind (required)
metadata: { }
items: [ ] # An array of IcebergNamespace
Example
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergNamespaceList"
items:
- metadata:
name: "analytics"
spec:
properties:
owner: "data-team"
- metadata:
name: "analytics.events"
spec:
properties:
owner: "platform-team"
3.7.2.2 - Iceberg Table
IcebergTable resources are used to define the tables you want to manage in your Iceberg catalog.
An IcebergTable resource defines the schema, partition layout, sort order, and table-level
properties. Jikkou performs safe schema evolution — adding, renaming, updating, and dropping
columns — without data loss.
IcebergTable
Specification
Here is the resource definition file for defining an IcebergTable.
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergTable" # The resource kind (required)
metadata:
name: <namespace>.<table> # Fully qualified table name (required)
labels: { }
annotations: { }
spec:
location: <storage path> # Override the default table location (optional)
schema:
identifierFields: # Primary-key columns for MERGE/UPSERT semantics (optional)
- <column name>
columns: # Ordered list of columns (required)
- name: <column name> # Column name (required)
type: <type> # Column type (required) — see type reference below
required: <true|false> # Whether the column is non-nullable (default: false)
doc: <description> # Documentation string (optional)
default: <value> # Initial default value, immutable after creation (optional)
writeDefault: <value> # Write-default for absent values, updatable (optional)
previousName: <old name> # Triggers a safe rename instead of drop+add (optional)
partitionFields: # Partition layout (optional)
- sourceColumn: <column> # Source column to partition on (required)
transform: <transform> # Partition transform (required) — see transforms below
name: <partition field name> # Custom partition field name (optional)
sortFields: # Default write sort order (optional)
- column: <column> # Column name (mutually exclusive with term)
term: <expression> # Sort expression e.g. bucket[16](user_id) (mutually exclusive with column)
direction: <asc|desc> # Sort direction (default: asc)
nullOrder: <first|last> # Null placement (default: last)
properties: # Table-level metadata properties (optional)
<key>: <value>
Column Types
Jikkou maps a set of type strings to the underlying Iceberg types:
| Type string | Iceberg type | Notes |
|---|---|---|
boolean | BooleanType | |
int / integer | IntegerType | |
long | LongType | |
float | FloatType | |
double | DoubleType | |
date | DateType | |
time | TimeType | |
timestamp | TimestampType (without tz) | |
timestamptz | TimestampType (with tz) | |
string | StringType | |
uuid | UUIDType | |
binary | BinaryType | |
fixed[N] | FixedType(N) | e.g. fixed[16] |
decimal(P,S) | DecimalType(P,S) | e.g. decimal(18,2) |
Complex types (struct, list, map) can be expressed as nested objects using the following format:
Struct:
type:
type: "struct"
fields:
- name: "field_name"
type: "string" # Any type (primitive or nested)
required: true # default: false
doc: "description" # optional
List:
type:
type: "list"
elementType: "string" # Any type (primitive or nested)
elementRequired: false # default: false
Map:
type:
type: "map"
keyType: "string" # Any type (primitive or nested)
valueType: "long" # Any type (primitive or nested)
valueRequired: false # default: false
Partition Transforms
| Transform | Example | Description |
|---|---|---|
identity | identity | Partition by the exact column value |
year | year | Extract year from a date/timestamp column |
month | month | Extract year-month |
day | day | Extract calendar date |
hour | hour | Extract date-hour |
bucket[N] | bucket[16] | Hash-bucket into N buckets |
truncate[W] | truncate[8] | Truncate string or integer to width W |
void | void | Always-null partition (marks dropped fields) — not yet supported |
Schema Evolution
Jikkou applies schema changes in a safe, deterministic order:
- Incompatible change check — verify annotation before proceeding
- Renames — processed first to preserve Iceberg field IDs
- Column additions — new columns appended
- Column updates — type promotion, documentation, nullability, write-default changes
- Column deletions — processed after updates to avoid conflicts
- Identifier field changes
- Schema commit — all schema changes are committed atomically
- Partition spec replacement
- Sort order replacement
- Properties update
Safe Column Rename
To rename a column without losing its Iceberg field ID (which would break existing readers),
set the previousName field to the old column name:
columns:
- name: "user_identifier" # new name
previousName: "user_id" # old name — triggers a rename, not drop+add
type: "long"
required: true
Incompatible Changes
By default, Jikkou rejects type changes that are not safe promotions (e.g., int → long is
safe; string → int is not). To allow incompatible changes on a specific resource, set the
annotation iceberg.jikkou.io/allow-incompatible-changes: "true".
Warning
Incompatible schema changes may corrupt existing data readers. Use this annotation with caution and only when you are certain all consumers can handle the new schema.Examples
Simple table with day partitioning
file: iceberg-page-views.yaml
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergTable"
metadata:
name: "analytics.events.page_views"
spec:
schema:
columns:
- name: "event_id"
type: "uuid"
required: true
doc: "Unique event identifier"
- name: "user_id"
type: "long"
required: true
doc: "The user who triggered the event"
- name: "page_url"
type: "string"
required: true
doc: "URL of the viewed page"
- name: "event_time"
type: "timestamptz"
required: true
doc: "Timestamp when the event occurred (UTC)"
- name: "duration_ms"
type: "long"
doc: "Time spent on the page in milliseconds"
partitionFields:
- sourceColumn: "event_time"
transform: "day"
sortFields:
- column: "event_time"
direction: "asc"
nullOrder: "last"
properties:
write.format.default: "parquet"
write.parquet.compression-codec: "zstd"
Table with bucket partitioning and identifier fields
file: iceberg-orders.yaml
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergTable"
metadata:
name: "analytics.events.orders"
annotations:
iceberg.jikkou.io/allow-incompatible-changes: "false"
spec:
schema:
columns:
- name: "order_id"
type: "long"
required: true
doc: "Unique order identifier"
- name: "customer_id"
type: "long"
required: true
doc: "Customer who placed the order"
- name: "order_date"
type: "date"
required: true
doc: "Date the order was placed"
- name: "status"
type: "string"
required: true
doc: "Order status"
- name: "total_amount"
type: "decimal(18,2)"
required: true
doc: "Total order amount"
identifierFields:
- "order_id"
partitionFields:
- sourceColumn: "order_date"
transform: "month"
- sourceColumn: "customer_id"
transform: "bucket[16]"
sortFields:
- column: "order_date"
direction: "desc"
nullOrder: "last"
- column: "customer_id"
direction: "asc"
nullOrder: "last"
properties:
write.format.default: "parquet"
IcebergTableList
If you need to define multiple tables (e.g., using a template), it may be easier to use an
IcebergTableList resource.
Specification
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergTableList" # The resource kind (required)
metadata: { }
items: [ ] # An array of IcebergTable
Example
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergTableList"
items:
- metadata:
name: "analytics.raw.clicks"
spec:
schema:
columns:
- name: "click_id"
type: "uuid"
required: true
- name: "ts"
type: "timestamptz"
required: true
partitionFields:
- sourceColumn: "ts"
transform: "day"
- metadata:
name: "analytics.raw.impressions"
spec:
schema:
columns:
- name: "impression_id"
type: "uuid"
required: true
- name: "ts"
type: "timestamptz"
required: true
partitionFields:
- sourceColumn: "ts"
transform: "day"
3.7.2.3 - Iceberg View
IcebergView resources are used to define SQL views in your Iceberg catalog.
A view is a logical definition backed by one or more SQL queries — the output schema
is inferred by the engine and populated on collect.
IcebergView
Specification
Here is the resource definition file for defining an IcebergView.
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergView" # The resource kind (required)
metadata:
name: <namespace>.<view> # Fully qualified view name (required)
labels: { }
annotations: { }
spec:
schema: # Output schema (read-only, inferred by the engine)
columns:
- name: <column name>
type: <type>
required: <true|false>
doc: <description>
queries: # SQL query definitions (at least one required)
- sql: <SQL SELECT statement> # The SQL defining the view (required)
dialect: <dialect> # SQL dialect e.g. 'spark', 'trino', 'presto', 'hive' (required)
defaultNamespace: <namespace> # Default namespace for unqualified table references (optional)
defaultCatalog: <catalog> # Default catalog for unqualified table references (optional)
properties: # View-level metadata properties (optional)
<key>: <value>
Examples
Simple view with daily aggregation
file: iceberg-daily-page-stats.yaml
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergView"
metadata:
name: "analytics.events.daily_page_stats"
spec:
queries:
- sql: >-
SELECT
CAST(event_time AS DATE) AS view_date,
page_url,
COUNT(*) AS view_count,
COUNT(DISTINCT user_id) AS unique_users
FROM analytics.events.page_views
GROUP BY CAST(event_time AS DATE), page_url
dialect: "spark"
defaultNamespace: "analytics.events"
properties:
comment: "Daily page view statistics aggregated from raw events"
View with multiple SQL dialects
file: iceberg-multi-dialect-view.yaml
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergView"
metadata:
name: "analytics.events.active_users"
spec:
queries:
- sql: >-
SELECT user_id, COUNT(*) AS event_count
FROM analytics.events.page_views
WHERE event_time >= current_date() - INTERVAL 30 DAYS
GROUP BY user_id
dialect: "spark"
- sql: >-
SELECT user_id, COUNT(*) AS event_count
FROM analytics.events.page_views
WHERE event_time >= current_date - INTERVAL '30' DAY
GROUP BY user_id
dialect: "trino"
defaultNamespace: "analytics.events"
properties:
comment: "Users active in the last 30 days"
IcebergViewList
If you need to define multiple views (e.g., using a template), it may be easier to use an
IcebergViewList resource.
Specification
apiVersion: "iceberg.jikkou.io/v1beta1" # The api version (required)
kind: "IcebergViewList" # The resource kind (required)
metadata: { }
items: [ ] # An array of IcebergView
Example
---
apiVersion: "iceberg.jikkou.io/v1beta1"
kind: "IcebergViewList"
items:
- metadata:
name: "analytics.events.daily_page_stats"
spec:
queries:
- sql: >-
SELECT CAST(event_time AS DATE) AS view_date, page_url, COUNT(*) AS view_count
FROM analytics.events.page_views
GROUP BY CAST(event_time AS DATE), page_url
dialect: "spark"
defaultNamespace: "analytics.events"
- metadata:
name: "analytics.events.active_users"
spec:
queries:
- sql: >-
SELECT user_id, COUNT(*) AS event_count
FROM analytics.events.page_views
WHERE event_time >= current_date() - INTERVAL 30 DAYS
GROUP BY user_id
dialect: "spark"
defaultNamespace: "analytics.events"
3.7.3 - Annotations
Here, you will find information about the annotations provided by the Apache Iceberg extension for Jikkou.
List of built-in annotations
iceberg.jikkou.io/allow-incompatible-changes
Controls whether incompatible schema changes are allowed when reconciling an IcebergTable.
By default, Jikkou rejects type changes that cannot be safely promoted (for example, changing
a column from string to int). Setting this annotation to "true" on a specific table
resource lifts that restriction for that resource only.
metadata:
annotations:
iceberg.jikkou.io/allow-incompatible-changes: "true"
Warning
Incompatible schema changes may break existing readers. Only enable this annotation when you are certain all consumers of the table can handle the new schema.iceberg.jikkou.io/namespace-location
Read-only annotation populated by the namespace collector. Contains the storage location of a namespace as reported by the catalog. This annotation is set automatically when collecting existing namespaces and does not need to be specified in resource files.
iceberg.jikkou.io/table-location
Reserved annotation for the storage location of a table. Currently, the table location
is stored in the spec.location field rather than as an annotation. This annotation key
is defined for future use.
iceberg.jikkou.io/view-location
Read-only annotation populated by the view collector. Contains the storage location of a view as reported by the catalog. This annotation is set automatically when collecting existing views and does not need to be specified in resource files.
3.8 - Confluent Cloud
Here, you will find information to use the Confluent Cloud extensions.
More information:
3.8.1 - Configuration
Here, you will find the configuration for the Confluent Cloud extension.
Configuration
You can configure the properties to connect to Confluent Cloud
through the Jikkou client configuration property jikkou.provider.confluent-cloud.
Example:
jikkou {
provider.confluent-cloud {
enabled = true
type = io.jikkou.extension.confluent.ConfluentCloudExtensionProvider
config = {
# URL to the Confluent Cloud REST API (default: https://api.confluent.cloud)
apiUrl = "https://api.confluent.cloud"
# Confluent Cloud API Key (must be a Cloud API Key, not a Cluster API Key)
apiKey = ${CONFLUENT_CLOUD_API_KEY}
# Confluent Cloud API Secret
apiSecret = ${CONFLUENT_CLOUD_API_SECRET}
# CRN pattern used to scope role binding list operations
crnPattern = ${CONFLUENT_CLOUD_CRN_PATTERN}
# HTTP proxy URL, e.g. 'http://proxy.example.com:3128' (optional)
proxyUrl = "http://proxy.example.com:3128"
# Username for proxy Basic authentication (optional)
proxyUsername = null
# Password for proxy Basic authentication (optional)
proxyPassword = null
# Comma-separated hosts that bypass the proxy (optional)
nonProxyHosts = "localhost,127.0.0.1"
# Enable debug logging (default: false)
debugLoggingEnabled = false
}
}
}
Configuration Properties
| Property | Type | Required | Default | Description |
|---|---|---|---|---|
apiUrl | String | No | https://api.confluent.cloud | URL to the Confluent Cloud REST API. |
apiKey | String | Yes | Cloud API Key. Must be a Cloud API Key, not a Cluster API Key. | |
apiSecret | String | Yes | Cloud API Secret. | |
crnPattern | String | Yes | CRN pattern to scope role binding list operations. | |
proxyUrl | String | No | HTTP proxy URL, e.g. http://proxy.example.com:3128. When empty, JVM proxy system properties are used. | |
proxyUsername | String | No | Username for proxy Basic authentication. | |
proxyPassword | String | No | Password for proxy Basic authentication. | |
nonProxyHosts | String | No | Comma-separated hosts that bypass the proxy, e.g. localhost,*.internal. | |
debugLoggingEnabled | Boolean | No | false | Enable debug logging for REST API calls. |
If
proxyUrlis not set, Jikkou honors the standard JVM proxy system properties (-Dhttps.proxyHost,-Dhttp.proxyHost,-Dhttp.proxyUser,-Dhttp.proxyPassword,-Dhttp.nonProxyHosts), which can be supplied viaJAVA_TOOL_OPTIONS. The OS-levelhttp_proxy/https_proxyenvironment variables are not read by the JVM and have no effect.
Creating a Cloud API Key
Cloud API Keys can be created using the Confluent Cloud CLI:
confluent api-key create --resource cloud --description "Jikkou role binding management"
Important: You must use a Cloud API Key (organization-level), not a Cluster API Key. Cluster API Keys will result in a
401 Unauthorizederror.
CRN Pattern
The crnPattern property is required and scopes all list operations to a specific part of your organization hierarchy. Examples:
| Scope | CRN Pattern |
|---|---|
| Organization | crn://confluent.cloud/organization=org-abc123 |
| Environment | crn://confluent.cloud/organization=org-abc123/environment=env-def456 |
| Kafka Cluster | crn://confluent.cloud/organization=org-abc123/environment=env-def456/cloud-cluster=lkc-789 |
3.8.2 - Resources
Here, you will find the list of resources supported by the extension for Confluent Cloud.
3.8.2.1 - Role Bindings for Confluent Cloud
The RoleBinding resources are used to manage RBAC role bindings in Confluent Cloud. A
RoleBinding resource defines which role is granted to a principal for a specific scope (identified by a CRN pattern).
RoleBinding
Specification
Here is the resource definition file for defining a RoleBinding.
---
apiVersion: "iam.confluent.cloud/v1" # The api version (required)
kind: "RoleBinding" # The resource kind (required)
metadata:
labels: { }
annotations: { }
spec:
principal: <> # The principal (e.g., User:sa-abc123 or User:u-xyz789)
roleName: <> # The role name (e.g., CloudClusterAdmin, DeveloperRead)
crnPattern: <> # The Confluent Resource Name pattern (e.g., crn://confluent.cloud/...)
Fields
| Field | Type | Required | Description |
|---|---|---|---|
principal | String | Yes | The principal. Pattern: User:<user-id> or Group:<group-name>. |
roleName | String | Yes | The role to bind. See Confluent Cloud RBAC roles. |
crnPattern | String | Yes | The Confluent Resource Name (CRN) pattern defining the scope of the binding. |
Common Role Names
| Role | Description |
|---|---|
OrganizationAdmin | Full access to the organization. |
EnvironmentAdmin | Full access to an environment. |
CloudClusterAdmin | Full access to a Kafka cluster. |
DeveloperManage | Manage topics and schemas. |
DeveloperRead | Read from topics and view schemas. |
DeveloperWrite | Write to topics and manage schemas. |
ResourceOwner | Full access to a specific resource. |
Example
Here is a simple example that shows how to define a single role binding using
the RoleBinding resource type.
file: role-binding.yaml
---
apiVersion: "iam.confluent.cloud/v1"
kind: "RoleBinding"
metadata:
labels: { }
annotations: { }
spec:
principal: "User:sa-abc123"
roleName: "CloudClusterAdmin"
crnPattern: "crn://confluent.cloud/organization=org-123/environment=env-456/cloud-cluster=lkc-789"
Usage
# List all role bindings
jikkou get ccloud-rbs
# Apply role bindings from a file
jikkou apply --files ./role-binding.yaml
# Delete orphan role bindings not defined in the file
jikkou apply --files ./role-binding.yaml -o delete-orphans=true
# Dry-run to preview changes without applying
jikkou diff --files ./role-binding.yaml
Metadata Labels
When listing role bindings, Jikkou automatically enriches each resource with metadata labels to help identify principals:
| Label | Description |
|---|---|
confluent.cloud/principal-name | The display name of the user or service account. |
confluent.cloud/principal-email | The email of the user (not set for service accounts). |
Example output:
apiVersion: "iam.confluent.cloud/v1"
kind: "RoleBinding"
metadata:
labels:
confluent.cloud/principal-name: "Florian Hussonnois"
confluent.cloud/principal-email: "florian@example.com"
annotations:
confluent.cloud/role-binding-id: "rb-NBl9kE"
spec:
principal: "User:u-rrnm2g9"
roleName: "OrganizationAdmin"
crnPattern: "crn://confluent.cloud/organization=d497af93-23f5-434a-a008-0547797be410"
RoleBindingList
If you need to define multiple role bindings (e.g., using a template), it may be easier to use a RoleBindingList resource.
Specification
Here is the resource definition file for defining a RoleBindingList.
---
apiVersion: "iam.confluent.cloud/v1" # The api version (required)
kind: "RoleBindingList" # The resource kind (required)
metadata: # (optional)
labels: { }
annotations: { }
items: [ ] # An array of RoleBinding
Example
---
apiVersion: "iam.confluent.cloud/v1"
kind: "RoleBindingList"
items:
- spec:
principal: "User:sa-abc123"
roleName: "CloudClusterAdmin"
crnPattern: "crn://confluent.cloud/organization=org-123/environment=env-456/cloud-cluster=lkc-789"
- spec:
principal: "User:sa-abc123"
roleName: "DeveloperRead"
crnPattern: "crn://confluent.cloud/organization=org-123/environment=env-456/cloud-cluster=lkc-789/kafka=lkc-789/topic=my-topic"
- spec:
principal: "User:u-xyz789"
roleName: "OrganizationAdmin"
crnPattern: "crn://confluent.cloud/organization=org-123"
Note: Role bindings are immutable in the Confluent Cloud API. If you need to change a role binding, delete the old one and create a new one. Jikkou only supports
CREATEandDELETEoperations (noUPDATE).
3.8.3 - Annotations
Here, you will find information about the annotations provided by the Confluent Cloud extension for Jikkou.
List of built-in annotations
confluent.cloud/role-binding-id
Used by Jikkou.
The annotation is automatically added by Jikkou to store the Confluent Cloud role binding ID (e.g., rb-NBl9kE).
This ID is used internally when deleting role bindings.