Avro

• Avro serialises a record as a compact binary encoding with no field names in the payload. Fields are written in schema order, so the reader needs the schema to interpret the bytes — and in exchange the bytes carry no key strings and no delimiters (Serialization: Objects to Bytes).
• In an object container file, the schema is written once in the file header as JSON, followed by blocks of records with a sync marker between blocks. The sync marker is what makes the file splittable: a reader can seek to an arbitrary offset, scan forward to the next marker, and start reading whole records from there.
• On a message broker, embedding the full schema in every message would defeat the purpose, so the standard arrangement is a schema registry: the payload carries a small schema identifier, and consumers resolve the id to a schema and cache it (Schema Registry).
• The property Avro is genuinely built around is schema resolution. A reader supplies its own expected schema, the data carries the writer's, and the library reconciles them: fields present in the writer's schema and absent from the reader's are skipped, and fields absent from the writer's but present in the reader's with a default are filled with that default (Schema Evolution).
• That reconciliation is what makes the compatibility rules concrete rather than aspirational. Adding a field with a default is backward compatible because old data reads under a new schema. Removing a field that had a default is forward compatible because new data reads under an old schema. Renaming is neither, because nothing connects the two names (Backward Compatibility).
• It is row-oriented, which is the whole trade. A record is contiguous, so reading one record is one contiguous read — and reading one field across a million records means touching all million records (Row vs Column Storage).

AVRO OBJECT CONTAINER FILE
┌─────────────────────────────────────────────────────────┐
│ HEADER                                                  │
│   magic "Obj\x01"                                       │
│   metadata:                                             │
│     avro.schema = { "type":"record","name":"OrderPlaced",│
│                     "fields":[ ... ] }   <- JSON, once   │
│     avro.codec  = "deflate" | "snappy" | "null"          │
│   sync marker (16 random bytes)                          │
├─────────────────────────────────────────────────────────┤
│ BLOCK: record count, byte size, compressed records       │
│ sync marker                    <- a reader can seek here │
├─────────────────────────────────────────────────────────┤
│ BLOCK: record count, byte size, compressed records       │
│ sync marker                                              │
└─────────────────────────────────────────────────────────┘

ONE RECORD, THREE WAYS

  JSON      {"order_id":1001,"country":"DE","revenue":41.90}
            -> field names present, per record, every record

  AVRO      <varint 1001><len 2>"DE"<double 41.90>
            -> values only, in schema order; the schema is elsewhere

  ON KAFKA  <magic 0><schema id: 4 bytes><avro-encoded body>
            -> a schema IDENTIFIER travels with the data,
               not the schema. The registry is now a dependency.

• Use Avro (or an equivalent schema-carrying binary format) for the transport and landing of event data, and convert to a columnar format for the analytical layer. The two formats are not competing for the same job (The Raw Landing Zone).
• Give every field a default from the day the schema is created. Defaults are the mechanism that makes both directions of compatibility possible, and adding them retroactively does not help data already written (Nullability & Defaults).
• Run a schema registry with compatibility enforcement in CI, so an incompatible schema is rejected before a producer deploys rather than after (Contract Enforcement).
• Use logical types — date, timestamp-millis, decimal — rather than encoding those as strings or floats. A monetary amount as a float is a correctness problem that no format will save you from (Data Contracts).
• Keep the schema and the event's meaning versioned together, and record the schema id on every landed record so a replay can tell which version wrote each row (Replay from the Log).
• Do not reach for Avro because a diagram had it. If your events land straight into a warehouse and are never replayed, JSON with a contract check may be the whole answer (ELT: Load First, Transform Where the Data Lives).

• The data is self-describing in a container file: the writer schema is in the header, so a file is interpretable with nothing external. On a broker it is *not* self-describing — it is registry-describing, and a lost registry is lost data.
• Schema resolution guarantees a deterministic reconciliation of writer and reader schemas by field name and by default, and it will refuse rather than guess when the two are irreconcilable.
• Container files are splittable at sync markers, so parallel reads are possible without a footer or an index.
• Nothing is guaranteed about column-level access. Reading one field means deserialising the record that contains it, which is the defining cost of a row format.
• Compatibility rules are enforced by the registry, not by the format. A registry configured for no compatibility checking will happily accept a breaking change (Schema Registry).