Frontend

Parsing

Tokens to structure. Recursive descent and Pratt parsing by hand, LL and LR as families, and what a parser generator buys and costs.

What a Parser Actually Does
▶ lab

A parser turns a flat token list into a tree whose shape is dictated entirely by the grammar. It decides what is applied to what — and it is not allowed to decide whether any of it means anything.

Q · What is a parser responsible for, and what is it not allowed to decide?
Parse Tree vs Abstract Syntax Tree
▶ lab

Two trees for the same input `1 + 2 * 3`: one with a node for every grammar rule and every comma, one with five nodes. The difference is not tidiness — it decides which tools you can build.

Q · What is the difference between a parse tree and an AST, and why do compilers build one and IDEs the other?
Recursive Descent
▶ lab

One function per grammar rule, the call stack as the parse stack. It is the technique most production compilers actually use — and the one that loops forever if you hand it a left-recursive grammar.

Q · How do I write a parser by hand, and why does my expression rule recurse forever?
Pratt Parsing
▶ lab

Replace the tower of precedence functions with one loop and a table of binding powers. Prefix handlers, infix handlers, and a single comparison that decides whether to keep going — this is the parsing technique most worth actually knowing.

Q · How do I parse expressions with fifteen precedence levels without writing fifteen functions?
LL Parsing, FIRST and FOLLOW
▶ lab

Left-to-right scan, leftmost derivation, k tokens of lookahead. The FIRST and FOLLOW sets are the mechanical answer to "which production do I pick", and the reason some grammars simply cannot be parsed top-down.

Q · What does the "LL" in LL(1) actually mean, and how does a parser decide which production to use?
LR Parsing
▶ lab

Bottom-up: never choose a production until the whole right-hand side is on the stack. It handles left recursion natively, accepts a strictly larger class of grammars than LL — and reports its problems as "conflict in state 143".

Q · What makes LR parsing more powerful than LL, and what is a shift/reduce conflict actually telling me?
Shift and Reduce, Step by Step
▶ lab

Two actions, one stack. Walk `1 + 2` through a bottom-up parser one move at a time and watch the tree assemble itself from the leaves upward — then see exactly what a conflict is.

Q · What do "shift" and "reduce" actually do to the stack, and when does the parser not know which to pick?
LL vs LR: Why Production Compilers Chose the Weaker One
▶ lab

LR accepts a strictly larger class of grammars. Clang, rustc, Roslyn, V8 and Go all hand-write recursive descent anyway. The reason is not ignorance or inertia — it is error messages, incremental reparse, and what an IDE needs.

Q · If LR is more powerful, why does almost every compiler I use hand-write a recursive-descent parser?
Parser Generators
▶ lab

ANTLR, Bison, tree-sitter and LALRPOP turn a grammar file into a parser. What you buy is a machine-checked grammar and cheap change; what you pay is diagnostics, debuggability and a build step.

Q · Should I write a grammar file and generate a parser, or write the parser myself?