5.6 The merits of schemas

Protocol Buffers' and Avro's schema languages are much simpler than XML Schema or JSON Schema, which support detailed validation rules ("must match this regex", "must be between 0 and 100"). Being simpler to implement and use, they've gained support across a wide range of languages.

These ideas are not new. ASN.1 — a schema definition language first standardized in 1984 — used to define network protocols; its binary encoding (DER) is still used to encode SSL certificates (X.509). It supports schema evolution using tag numbers, similar to Protocol Buffers. But it's very complex and badly documented — probably not a good choice for new applications.

Also worth noting: most relational databases have a proprietary binary network protocol, with a vendor-supplied driver (ODBC/JDBC) decoding responses into in-memory data structures.

Four properties of schema-driven binary encodings:

1. Much more compact than "binary JSON" variants, since they omit field names from the encoded data
2. The schema is valuable documentation — and because the schema is REQUIRED for decoding, you can be sure it is up to date (manually maintained documentation easily diverges from reality)
3. A database of schemas lets you check forward and backward compatibility of changes BEFORE anything is deployed
4. Code generation from the schema enables compile-time type checking for statically typed languages

Schema evolution gives the same flexibility as schemaless/schema-on-read JSON databases, while also providing better guarantees about your data and better tooling.

Still, keep the number of concurrent schema formats to a minimum to keep operations simple.

---