Manage Apache Kafka® topics

Create a topic

  1. Go to the Clusters page in the console.

  2. Select the cluster whose topics you want to manage.

  3. On the cluster page, go to the Topics tab.

  4. Click Create.

  5. Under Topic Settings, specify the connection properties:

    • Cleanup policy

      Specifies what to do with old log messages. It can have one of the following values:

      • Delete: Deletes log segments when their retention time or log size reaches the limit.

      • Compact: Compresses messages in the log. The service keeps only one log file for each primary log key in this case.

      • Compact and delete: Both compacts and deletes log segments.

      Warning

      Note that topics of the Compact and Compact and delete types require that you specify keys for your messages.

    • Compression Type

      Specifies the codec used for message compression:

      • Uncompressed: Compression is disabled

      • Zstd: The zstd codec

      • Lz4: The lz4 codec

      • Snappy: The snappy codec

      • Gzip: The gzip codec

      • Producer: The codec is set on the producer side

    • Retention Bytes

      The maximum size of a partition in bytes. When the partition reaches this size, Apache Kafka® deletes the old log segments. The setting applies if the cleanup policy is Delete.

    • Retention Ms

      Time in milliseconds for Apache Kafka® to keep log segment files. This setting applies if the cleanup policy is Delete. After the specified time, the segment file is deleted.

  6. Specify the Basic Settings:

    • Name

      A topic name. It must be unique in the Apache Kafka® cluster.

    • Partitions

      A number of topic partitions.

    • Replication factor

      Specifies the number of copies of a topic in a cluster. This parameter value should not exceed the number of brokers in the cluster.

  7. Click Submit.

Use the TopicService create method and pass the following parameters:

  • cluster_id - the ID of the cluster in which you want to create a topic. To find the cluster ID, get a list of clusters in the project.

  • topic_spec - the specifications for your new topic, according to the doublecloud.kafka.v1.TopicSpec model.

Manage a topic

  1. Go to the Clusters page in the console and select the cluster whose topics you want to manage.

  2. On the cluster's page, go to the Topics tab.

  3. Click → Edit.

  4. On the edit page, you can change all settings listed in the Create a topic section.

  5. Click Submit.

To manage your Apache Kafka® topic, use the TopicService update method and pass the following parameters:

  • cluster_id - the ID of the cluster in which you want to create a topic. To find the cluster ID, get a list of clusters in the project.

  • topic_name - the name of the topic you want to update. To find the topic name, get a list of topic in your Apache Kafka® cluster.

  • topic_spec - the new specifications for your topic, according to the doublecloud.kafka.v1.TopicSpec model.

Delete a topic

  1. Go to the Clusters page in the console and select the cluster whose topics you want to manage.

  2. Select the cluster whose topics you want to manage.

  3. On the cluster's page, go to the Topics tab.

  4. Click → Delete.

Use the TopicService delete method and pass the following parameters:

  • cluster_id the ID of the cluster from which you want to delete a topic. To find the cluster ID, get a list of clusters in the project.

  • topic_name - the name of the topic you want to delete. To find the topic name, get a list of topic in your Apache Kafka® cluster.

See also

Previous
Delete a cluster
Next
View a list of operations