Second Brain: Crafted, Curated, Connected, Compounded on 10月02日
Avro:数据序列化系统解析
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文解析了开源数据序列化系统Avro,介绍了其数据模型、格式以及Schema Evolution功能,并指出其在数据湖文件格式中的应用。

Avro is an open-source data serialization system that helps with data exchange between systems, programming languages, and processing frameworks. Avro helps define a binary format for your data, as well as map it to the programming language of your choice.

Avro has a JSON-like data model, but can be represented as either JSON or in a compact binary form. It comes with a very sophisticated schema description language that describes data.

Avro also supports Schema Evolution (same as Apache Parquet and Delta Lake do) using an external registry which will allow you to change the schema for your ingested data relatively easily.


Origin: Data Lake File Formats
References: Apache Avro
Created 2022-08-15

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

Avro 数据序列化 Schema Evolution 数据湖 文件格式
相关文章