Schema Registry provides a centralized repository for managing and validating schemas for topic message data, and for serialization and deserialization of the data over the network.

Producers and consumers of topics (e.g. Apache Kafka) can use schemas to ensure data consistency and compatibility as schemas evolve.

Schema Registry is a key component for data governance, helping to ensure data quality, adherence to standards, visibility into data lineage, audit capabilities, collaboration across teams, efficient application development protocols, and system performance.

Schema Registry provides a serving layer for your metadata, offering a RESTful interface for storing and retrieving schemas. It’s available in different implementations:

Key features across implementations:

RESTful interface for storing and retrieving schemasVersioned history of all schemas based on specified subject name strategiesMultiple compatibility settingsSchema Evolution according to configured compatibility settingsEnhanced schema type support
For example, services can return diffs between two Modern ValueStore versions, showing new and deprecated columns

Confluent Schema Registry provides a serving layer for your metadata with Apache Avro schemas support. It lives outside of and separately from your Kafka brokers. While producers and consumers communicate with Kafka to publish and read data (messages) to topics, they can concurrently interact with Schema Registry to send and retrieve schemas describing the data models for the messages.

Key design aspects of Kafka Schema Registry

Assigns globally unique, monotonically increasing IDs to each registered schemaUses Kafka as durable backend and write-ahead changelogDistributed architecture with single-primary designZooKeeper/Kafka coordinates primary election based on configuration

Origin: Analytics API
Related notes:
Created: 2021-10-12