File Formats & Compression
Parquet, Avro, ORC and the text formats they replaced. Row groups, column chunks, statistics, encodings — what a format actually stores, and what that lets a reader skip.
Columns of one type with repeating values compress in ways rows of mixed types cannot — and the chain from fewer bytes to a faster query has three places it can break.
Four type-aware encodings, what redundancy each one exploits, and the column property — cardinality, sortedness, range — that decides whether it does anything at all.
A self-describing, columnar, splittable file whose footer tells a reader what it can skip — which is a different claim from "it is smaller".
Row groups, column chunks, pages and the footer — where statistics come from, why they are only useful when the data is sorted, and how nesting is stored without abandoning columns.
Follow `SELECT country, revenue FROM events WHERE date = '2026-08-25'` from a directory listing to decoded values, and count what got skipped at each of the four gates.
Row-oriented binary records with the schema travelling alongside the data — built for exchange and evolution rather than for scanning one column across a billion rows.
A sibling columnar design with the same goals and different specifics: stripes instead of row groups, row-index strides for finer skipping, and row-level ACID in the ecosystem it grew up in.
Not a rivalry. One is built for reading a few columns across many rows, the other for handling whole records one at a time — and most pipelines use both, in that order.
No types, no schema, no statistics, ambiguous quoting and — for CSV — a splittability problem that has no clean fix. And still the right answer for interchange, small data and human inspection.