This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Aiven

Lean how to use the Jikkou Extensions Providers for Aiven.

Here, you will find information to use the Aiven for Kafka extensions.

More information:

1 - Configuration

Learn how to configure the extensions for Aiven.

Here, you will find the list of resources supported by the extension for Aiven.

Configuration

You can configure the properties to be used to connect the Aiven service through the Jikkou client configuration property jikkou.provider.aiven.

Example:

jikkou {
  provider.aiven {
    enabled = true
    type = io.streamthoughts.jikkou.extension.aiven.AivenExtensionProvider
    config = {
      # Aiven project name
      project = "http://localhost:8081"
      # Aiven service name
      service = generic
      # URL to the Aiven REST API.
      apiUrl = "https://api.aiven.io/v1/"
      # Aiven Bearer Token. Tokens can be obtained from your Aiven profile page
      tokenAuth = null
      # Enable debug logging
      debugLoggingEnabled = false
    }
  }
}

2 - Resources

Learn how to use the built-in resources provided by the extensions for Aiven.

Here, you will find the list of resources supported by the extensions for Aiven.

Aiven for Apache Kafka Resources

More information:

2.1 - ACL for Aiven Apache Kafka®

Learn how to manage Access Control Lists (ACLs) in Aiven for Apache Kafka®

The KafkaTopicAclEntry resources are used to manage the Access Control Lists in Aiven for Apache Kafka®. A KafkaTopicAclEntry resource defines the permission to be granted to a user for one or more kafka topics.

KafkaTopicAclEntry

Specification

Here is the resource definition file for defining a KafkaTopicAclEntry.

---
apiVersion: "kafka.aiven.io/v1beta1"   # The api version (required)
kind: "KafkaTopicAclEntry"             # The resource kind (required)
metadata:
  labels: { }
  annotations: { }
spec:
  permission: <>               # The permission. Accepted values are: READ, WRITE, READWRITE, ADMIN
  username: <>                 # The username
  topic: <>                    # Topic name or glob pattern

Example

Here is a simple example that shows how to define a single ACL entry using the KafkaTopicAclEntry resource type.

file: kafka-topic-acl-entry.yaml

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaTopicAclEntry"
metadata:
  labels: { }
  annotations: { }
spec:
  permission: "READWRITE"
  username: "alice"
  topic: "public-*"

KafkaTopicAclEntryList

If you need to define multiple ACL entries (e.g. using a template), it may be easier to use a KafkaTopicAclEntryList resource.

Specification

Here the resource definition file for defining a KafkaTopicList.

---
apiVersion: "kafka.aiven.io/v1beta1"    # The api version (required)
kind: "KafkaTopicAclEntryList"          # The resource kind (required)
metadata: # (optional)
  name: <The name of the topic>
  labels: { }
  annotations: { }
items: [ ]                             # An array of KafkaTopicAclEntry

Example

Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using the KafkaTopicAclEntryList resource type.

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaTopicAclEntryList"
items:
  - spec:
      permission: "READWRITE"
      username: "alice"
      topic: "public-*"
  - spec:
      permission: "READ"
      username: "bob"
      topic: "public-*"

2.2 - Quotas for Aiven Apache Kafka®

Learn how to manage Quotas in Aiven for Apache Kafka®

The KafkaQuota resources are used to manage the Quotas in Aiven for Apache Kafka® service. For more details, see https://docs.aiven.io/docs/products/kafka/concepts/kafka-quotas

KafkaQuota

Specification

Here is the resource definition file for defining a KafkaQuota.

---
apiVersion: "kafka.aiven.io/v1beta1"   # The api version (required)
kind: "KafkaQuota"                     # The resource kind (required)
metadata:
  labels: { }
  annotations: { }
spec:
  user: <string>                     # The username: (Optional: 'default' if null)
  clientId: <string>                 # The client-id
  consumerByteRate: <number>         # The quota in bytes for restricting data consumption
  producerByteRate: <number>         # The quota in bytes for restricting data production
  requestPercentage: <number>

Example

Here is a simple example that shows how to define a single ACL entry using the KafkaQuota resource type.

file: kafka-quotas.yaml

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaQuota"
spec:
  user: "default"
  clientId: "default"
  consumerByteRate: 1048576
  producerByteRate: 1048576
  requestPercentage: 25

KafkaQuotaList

If you need to define multiple Kafka quotas (e.g. using a template), it may be easier to use a KafkaQuotaList resource.

Specification

Here the resource definition file for defining a KafkaTopicList.

---
apiVersion: "kafka.aiven.io/v1beta1"    # The api version (required)
kind: "KafkaQuotaList"                  # The resource kind (required)
metadata: # (optional)
  labels: { }
  annotations: { }
items: [ ]                             # An array of KafkaQuotaList

Example

Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using the KafkaQuotaList resource type.

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "KafkaQuotaList"
items:
  - spec:
      user: "default"
      clientId: "default"
      consumerByteRate: 1048576
      producerByteRate: 1048576
      requestPercentage: 5
  - spec:
      user: "avnadmin"
      consumerByteRate: 5242880
      producerByteRate: 5242880
      requestPercentage: 25

2.3 - ACL for Aiven Schema Registry

Learn how to manage Access Control Lists (ACLs) in Aiven for Schema Registry

The SchemaRegistryAclEntry resources are used to manage the Access Control Lists in Aiven for Schema Registry. A SchemaRegistryAclEntry resource defines the permission to be granted to a user for one or more Schema Registry Subjects.

SchemaRegistryAclEntry

Specification

Here is the resource definition file for defining a SchemaRegistryAclEntry.

---
apiVersion: "kafka.aiven.io/v1beta1"   # The api version (required)
kind: "SchemaRegistryAclEntry"         # The resource kind (required)
metadata:
  labels: { }
  annotations: { }
spec:
  permission: <>               # The permission. Accepted values are: READ, WRITE
  username: <>                 # The username
  resource: <>                 # The Schema Registry ACL entry resource name pattern

NOTE: The resource name pattern should be Config: or Subject:<subject_name> where subject_name must consist of alpha-numeric characters, underscores, dashes, dots and glob characters * and ?.

Example

Here is an example that shows how to define a simple ACL entry using the SchemaRegistryAclEntry resource type.

file: schema-registry-acl-entry.yaml

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistryAclEntry"
spec:
  permission: "READ"
  username: "Alice"
  resource: "Subject:*"

SchemaRegistryAclEntryList

If you need to define multiple ACL entries (e.g. using a template), it may be easier to use a SchemaRegistryAclEntryList resource.

Specification

Here the resource definition file for defining a SchemaRegistryAclEntryList.

---
apiVersion: "kafka.aiven.io/v1beta1"    # The api version (required)
kind: "SchemaRegistryAclEntryList"      # The resource kind (required)
metadata: # (optional)
  labels: { }
  annotations: { }
items: [ ]                              # An array of SchemaRegistryAclEntry

Example

Here is a simple example that shows how to define a single YAML file containing two ACL entry definitions using the SchemaRegistryAclEntryList resource type.

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistryAclEntryList"
items:
  - spec:
      permission: "READ"
      username: "alice"
      resource: "Config:"
  - spec:
      permission: "WRITE"
      username: "alice"
      resource: "Subject:*"

2.4 - Subject for Aiven Schema Registry

Learn how to manage Schema Registry Subject Schema in Aiven.

SchemaRegistrySubject resources are used to define the subject schemas you want to manage on your Schema Registry. A SchemaRegistrySubject resource defines the schema, the compatibility level, and the references to be associated with a subject version.

SchemaRegistrySubject

Specification

Here is the resource definition file for defining a SchemaRegistrySubject.

apiVersion: "kafka.aiven.io/v1beta1"  # The api version (required)
kind: "SchemaRegistrySubject"                   # The resource kind (required)
metadata:
  name: <The name of the subject>               # (required)
  labels: { }
  annotations: { }
spec:
  schemaRegistry:
    vendor: 'Karapace'                          # (optional) The vendor of the Schema Registry
  compatibilityLevel: <compatibility_level>     # (optional) The schema compatibility level for this subject.
  schemaType: <The schema format>               # (required) Accepted values are: AVRO, PROTOBUF, JSON
  schema:
    $ref: <url or path>  # 
  references:                                   # Specifies the names of referenced schemas (optional array).
    - name: <>                                  # The name for the reference.
      subject: <>                               # The subject under which the referenced schema is registered.
      version: <>                               # The exact version of the schema under the registered subject.
]

The metadata.name property is mandatory and specifies the name of the Subject.

To use the SchemaRegistry default values for the compatibilityLevel you can omit the property.

Example

Here is a simple example that shows how to define a single subject AVRO schema for type using the SchemaRegistrySubject resource type.

file: subject-user.yaml

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistrySubject"
metadata:
  name: "User"
  labels: { }
  annotations:
    schemaregistry.jikkou.io/normalize-schema: true
spec:
  compatibilityLevel: "FULL_TRANSITIVE"
  schemaType: "AVRO"
  schema:
    $ref: ./user-schema.avsc

file: user-schema.avsc

---
{
  "namespace": "example.avro",
  "type": "record",
  "name": "User",
  "fields": [
    {
      "name": "name",
      "type": [ "null", "string" ],
      "default": null,
    },
    {
      "name": "favorite_number",
      "type": [ "null", "int" ],
      "default": null
    },
    {
      "name": "favorite_color",
      "type": [ "null", "string" ],
      "default": null
    }
  ]
}

Alternatively, we can directly pass the Avro schema as follows :

file: subject-user.yaml

---
apiVersion: "kafka.aiven.io/v1beta1"
kind: "SchemaRegistrySubject"
metadata:
  name: "User"
  labels: { }
  annotations:
    schemaregistry.jikkou.io/normalize-schema: true
spec:
  compatibilityLevel: "FULL_TRANSITIVE"
  schemaType: "AVRO"
  schema: |
    {
      "namespace": "example.avro",
      "type": "record",
      "name": "User",
      "fields": [
        {
          "name": "name",
          "type": [ "null", "string" ],
          "default": null
        },
        {
          "name": "favorite_number",
          "type": [ "null", "int" ],
          "default": null
        },
        {
          "name": "favorite_color",
          "type": [ "null", "string"],
          "default": null
        }
      ]
    }

3 - Validations

Learn how to use the built-in validations provided by the extensions for Aiven.

Jikkou ships with the following built-in validations:

No validation

4 - Annotations

Learn how to use the metadata annotations provided by the extensions for Aiven.

Here, you will find information about the annotations provided by the Aiven extension for Jikkou.

List of built-in annotations

kafka.aiven.io/acl-entry-id

Used by jikkou.

The annotation is automatically added by Jikkou to describe the ID of an ACL entry.