Schema registry in DoubleCloud Managed Service for Apache Kafka®
Apache Kafka® uses a binary format for storing and passing messages. The content of the message doesn't contain any information about its structure. To ensure the correct interpretation of the binary data, the consumer needs a data format schema describing the data input or output format.
Based on the data format schema, the producer generates, and the consumer interprets messages from topics. If data format schemas differ between producers and consumers, the application may throw errors because of incorrect message interpretation.
The developers of the producer and consumer must:
-
Update data format schemas regularly and on time.
-
If necessary, enable the producer and consumer to support several data format schema versions.
A data format schema registry enables automation of the data format schema handling. It simplifies data management, especially when a schema changes over time. The registry automatically checks data version compatibility and ensures the backward compatibility of schema versions.
Registry of data format schemas
Producers and consumers use the data format registry as follows:
-
A producer transmits data format schemas to the registry.
When placed in the registry:
-
A unique version number is assigned to a schema.
-
The schema and its version are saved in an Apache Kafka® service topic.
-
-
When sending a message, a producer specifies the version number of the required schema.
-
Upon receiving a message, a consumer extracts the version number of the data format schema.
-
If the required data format schema is missing from the local cache, the consumer looks it up in the registry.
-
After getting the necessary schema, it interprets the received message accordingly.
Managed Schema registry
DoubleCloud Managed Service for Apache Kafka® features a built-in Managed Schema Registry data format schema registry. The registry is deployed on each cluster's broker host and is accessible via HTTPS on port 443.
We support the following data schema formats: