Loweringspec

Syntactic Sugar

Sugar changes how a program is written without changing what the language can express. That makes it cheap to add and easy to underrate — the cost is not in the semantics, it is in the grammar, the diagnostics and the number of ways to say the same thing.

The question

What actually makes a feature "just syntactic sugar", and does calling it that mean it does not matter?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute

What the program is here

Source text, and the AST the parser builds from it. A sugared construct is one that has a meaning-preserving expansion into a smaller *core* language, so the representation question is where that expansion lives: sugar expanded inside the parser never exists as a node at all, while sugar that survives as its own AST node exists until some later pass rewrites it. The tree is the record of that choice, and it is the only record — by the time anything reaches the IR both designs look identical.

What this phase may assume or do

A construct is sugar only if a rewriting into the core language preserves observable behavior for *every* program, not for the examples in the proposal. That means preserving evaluation order, the number of times each subexpression is evaluated, short-circuiting, and which traps or exceptions occur. a += b is sugar for a = a + b only where the language guarantees the left operand is evaluated once: with arr[i++] += b the naive expansion increments i twice and is a different program.

Key points

Sugar is a falsifiable claim: there is a smaller core language into which the construct expands with no change in observable behavior.
The expansion must preserve evaluation order, evaluation count, short-circuiting and trapping — not just the resulting value.
Compound assignment evaluates its left operand once; the naive textual expansion evaluates it twice and is wrong.
Sugar is cheap for the back end and expensive for the grammar, the diagnostics and the reader.
A construct that requires new storage, a split function or a new control-flow mechanism is a primitive, not sugar.
Expanding in the parser makes every later phase simpler and every error message worse.

A claim about expressive power, not about taste

Calling a feature sugar is a precise claim: there exists a smaller language, a subset of this one, into which the feature can be translated without changing what any program does. The subset is the *core language*, and everything outside it is convenience. The claim is falsifiable — if you cannot write the expansion, the feature is not sugar, it is a new primitive.

The distinction earns its keep in implementation cost. A new primitive has to be understood by the type checker, the optimizer, the code generator and the debugger. Sugar has to be understood by the parser and by whatever pass performs the expansion, and by nothing else — every phase downstream sees only the core. That asymmetry is why languages accumulate sugar much faster than they accumulate primitives.

It is also why "just sugar" is a bad dismissal. Notation is what people read. arr.map(f).filter(g) and a hand-written loop compile to code of similar shape, and one of them can be checked by a reviewer at a glance. Expressive power and usability are different axes, and sugar moves only the second one — which is the whole reason it exists.

Sugar and the core construct it expands totypical

Sugar	Core form	What the expansion must preserve
for item in collectionspec	Get an iterator, loop calling next, bind, check for exhaustion	The iterator is obtained once; exhaustion ends the loop rather than raising
obj?.fieldspec	Test the receiver against null, then either produce null or read the field	The receiver is evaluated exactly once, and the whole chain short-circuits, not just one link
a += bspec	a = a + b	The left operand is evaluated once even when it has side effects
Interpolated string	Concatenation, or a call to a formatting function with the pieces as arguments	Left-to-right evaluation of the embedded expressions, and the conversion each value gets
x && yspec	A branch: evaluate x, and evaluate y only if x was truthy	That y is not evaluated at all when x decides the answer
a ?? bspec	Test a against null/undefined, produce a or b	That the test is against nullishness specifically, not against falsiness

The expansion is about behaviour, not about text

specThe "evaluated only once" rule for compound assignment is in the C and C++ standards, and JavaScript, Python, Rust and Go all specify equivalent behaviour for their own compound-assignment operators. What differs is what happens for an *overloaded* operator: in C++ operator+= may be a separate user-defined function with no relationship to operator+, so the expansion is not available at all, and in Python __iadd__ may mutate in place where __add__ would return a new object — the sugar and the core form are then genuinely different programs.

Every sugar has a naive textual expansion, and for several of them the naive expansion is wrong. This is the most common way a language implementation gets sugar subtly incorrect: the rewrite is derived by substituting text rather than by preserving the operational meaning, and the difference only shows up when a subexpression has a side effect.

The compound assignment case is the canonical one, and C says so explicitly: E1 op= E2 is equivalent to E1 = E1 op E2 except that the lvalue E1 is evaluated only once. That exception is the entire content of the rule. A correct expansion has to introduce a temporary holding the address, not the value, and then use it twice.

Optional chaining has the same shape with a different twist. a?.b.c does not mean (a?.b).c — the short circuit covers the whole chain, so if a is null the .c is never attempted either. A rewrite that expands one link at a time produces a program that throws where the source promised it would not.

The naive expansion of a compound assignment, and the correct one

1arr[i++] += 5;
2 
3/* Wrong: substitutes the text of the lvalue twice. */
4arr[i++] = arr[i++] + 5;   /* i incremented twice, two different elements touched */
5 
6/* Right: evaluate the lvalue once, keep the address. */
7{ int *p = &arr[i++]; *p = *p + 5; }

The correct expansion needs a temporary the source language never mentions. That is normal — lowering introduces names, and the names it introduces are exactly the things the source was implicitly holding on to.

What sugar costs, since it is not semantics

The bill arrives in four places, none of them the runtime. The grammar grows, and every new form has to be unambiguous against every existing one — which is why languages that added optional chaining had to resolve a?.b against the conditional operator and against a ternary whose consequent starts with a dot. The parser gets bigger and the ambiguity analysis gets harder, and both are permanent.

Diagnostics degrade, and this is the cost people notice. If the sugar is expanded in the parser, the type error is reported against a construct nobody wrote. A reader who typed for x in xs and receives an error about a missing __next__ method on line 4 is being shown the compiler's internal rewrite. Languages that care about this keep the sugared node in the tree and desugar later, which is the subject of [[desugaring]].

Then there is the cost that has no technical name: more than one way to say the same thing. Every added form is a decision every reader has to make and every style guide has to legislate. This is a real design cost and it compounds, and it is the argument that most often loses to "but it is only sugar".

Finally, sugar interacts. Two features that are each obviously fine can produce a combination that is ambiguous, surprising, or expensive to check. Ambiguity is not a property of a production; it is a property of the grammar as a whole — see [[ambiguous-grammars]].

Grammar surface: every form must be unambiguous against every other form, forever.
Diagnostics: an error reported against the expansion names constructs the author never wrote.
Debugging: a stepper that steps through the expansion shows source positions that do not correspond to anything.
Teaching and review: N ways to write the same thing is N choices per line of code.
Interaction: features are fine individually and ambiguous in combination.

Where the line actually falls

Some constructs look like sugar and are not. async/await is often described as sugar over callbacks, and it is not: the rewrite requires splitting a function at every suspension point and storing the live locals somewhere that survives the frame, which is a genuine change of representation rather than a local rewriting — see [[async-lowering]]. Exceptions are not sugar over error returns, because they unwind frames the intervening code did not agree to. Closures are not sugar, because they change where a variable can live.

The test that actually works: can you write the expansion as a *local* tree rewriting, using only constructs the language already has, without changing any storage decision or any control-flow mechanism? If yes it is sugar. If the rewrite has to allocate something, split a function, or introduce a mechanism the language did not previously need, it is a feature — and the rest of this module is about those.

How it works

The steps, in the order the compiler takes them.

The grammar gains a production for the sugared form, which must be checked for ambiguity against the whole existing grammar rather than against the neighbouring rules.
The parser either builds a node for the sugared form, or builds the expansion directly — a decision that determines what every later phase and every diagnostic can see.
If a node is built, a later pass rewrites it into core constructs, introducing temporaries for any subexpression the source guaranteed would be evaluated once.
Spans from the sugared source are attached to the introduced nodes so that diagnostics and debug information can still point at what the author wrote.
From the point of expansion onward, no phase distinguishes the sugared program from a program written in the core form by hand.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

A type error names a method or a trait the author never mentioned, because the message was generated against the expansion — the reader has to reverse-engineer the desugaring before they can read the error.
A subexpression with a side effect runs twice: an index is incremented twice, a counter double-counts, a network call is made twice, and the code reads as if it happens once.
A debugger steps into a line that does not exist, or reports a variable the author never declared, because the introduced temporary carried a real span.
A short-circuit is lost in the expansion: an expression whose right half was supposed to be skipped is evaluated, and a null dereference appears in code that visibly guards against null.
A new sugar is ambiguous against an existing one only for a rare input, so it ships, and the parse of a?.b:c becomes a compatibility hazard nobody can fix.

When it helps

Notation that matches how practitioners already talk about the domain, where the core form obscures the intent behind mechanism.
Removing a repeated boilerplate shape whose every occurrence is identical — iteration protocols, null checks, resource cleanup.
Making a correctness-relevant intention explicit in the syntax, so a reviewer sees it: a null-safe access reads differently from an unchecked one.

When it hurts

When the expansion is not actually behavior-preserving in the corner cases, which is where sugar bugs live and where nobody tests.
When the same operation gains a third spelling and every code review acquires a style argument.
When the sugar is expanded early and the error messages become messages about the compiler rather than about the program.
When the construct is presented as sugar to win the design argument, and the implementation then discovers it needs new storage or new control flow.

What it costs

Every one of these is paid by something.

Expanding in the parser buys a smaller core for every later phase — fewer node kinds in the checker, the optimizer and the code generator — and pays in diagnostic quality, because the checker can only report against nodes that exist.
Keeping the sugared node in the tree buys precise error messages and a debugger that steps over what the author wrote, and pays in node kinds: every pass must handle the sugared form or be careful to run after the desugaring pass.
Each added sugar buys concision at the point of use and pays a permanent grammar-ambiguity obligation plus one more decision for every reader of the language, forever.
Sugar defined by rewriting to a user-overloadable operation buys extensibility and pays predictability: the expansion is only meaning-preserving if every overload obeys a contract the compiler cannot check.

What else you could do

What a different compiler or language does instead, and when that is better.

Add it as a real primitive with its own typing rule and its own lowering, which costs more implementation and buys precise diagnostics and freedom to optimize it specially — this is what languages do for iteration once performance matters.
Provide it as a library function instead of syntax. Option.map gets much of what ?. gets, keeps the grammar untouched, and reads worse when it is nested three deep.
Provide a macro system and let users define their own sugar, which is Lisp's and Rust's answer. It moves the cost from the language designer to the reader, who now has to know which forms are macros.
Say no. Scheme and Go both have long records of refusing sugar on the grounds that the core is the feature, and the resulting languages are notably easy to write tools for.

See it for yourself

The flag, dump or tool that shows you this directly.

Python: python -m ast (or ast.dump(ast.parse(src))) shows which constructs survive as nodes. for survives; a += b becomes an AugAssign node; dis.dis then shows the bytecode the expansion produced.
Rust: cargo expand (a cargo subcommand) shows the program after macro expansion, and rustc -Z unpretty=hir on nightly shows the HIR, which is the tree after most desugaring.
JavaScript: any of the standard transpilers with a target that predates the feature will print the expansion. Compare a?.b.c before and after and look for how many temporaries it introduced.
C and C++: clang -Xclang -ast-dump shows a CompoundAssignOperator node with separate computation and result types — the node exists precisely because the expansion is not a textual one.
Our own desugaring viewer at /compilers/lowering runs the real AtlasLang desugarer and shows the tree before and after, with spans preserved.

Plausible wrong readings

Stated the way a confident engineer states them.

"It is only sugar, so it is free." It is free for the back end and expensive for the grammar, the diagnostics and every future reader of the language.
"Sugar means the compiler literally substitutes the text." It substitutes a tree, and for several constructs the correct tree contains temporaries the text does not.
"If two forms produce the same bytecode they are the same feature." They produce the same bytecode for the cases you tested. Side-effecting operands, overloaded operators and short-circuiting are where they diverge.
"async/await is sugar over promises." It requires splitting the function and moving locals off the frame, which no local rewriting can do — see [[async-lowering]].

Misconceptions

The claim, and what is actually true.

Sugar makes the language more powerful.

By definition it does not — that is what distinguishes it from a primitive. It makes the language more usable, which is a different and equally real property.

Removing sugar would make programs slower.

The expansion is what runs either way. Removing sugar makes programs longer to write and read, not slower.

Anything that can be written as a function call is not sugar worth having.

Sugar that captures short-circuiting, binding or control flow cannot be a function call in a strict language, because a call evaluates its arguments.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

Some parts of a language are conveniences: the compiler translates them into simpler parts of the same language before doing anything else. A for loop over a list becomes an ordinary loop that asks the list for the next item. a += b becomes a = a + b. Nothing new becomes possible; the code just gets shorter to write and, usually, easier to read.

practical

The place this bites you is error messages. If a message mentions a method, a trait or a variable you never wrote, you are looking at a diagnostic generated against the expansion, and the fix is to find out what the construct expands to. Every mainstream language has a dump flag that shows you — learn the one for your language and the class of confusing error disappears. The second place it bites is side effects in operands: if the thing on the left of a compound assignment or inside an optional chain does anything other than name a variable, check what the specification says about how many times it is evaluated.

advanced

The genuinely interesting design question is not whether to add a sugar but *when to expand it*, and the answer has moved over the last two decades. Early compilers expanded in the parser because the core was small and memory was tight. Modern frontends keep a high-level tree — Rust's HIR, Swift's and Kotlin's equivalents, TypeScript's AST — because the frontend is now also a language server, a formatter, a linter and a refactoring engine, and every one of those must answer questions about the text the user typed. Expansion is therefore pushed as late as it can go, and the cost of that decision is that every pass in the frontend must understand the sugared forms. This is the same trade in a different currency as [[concrete-syntax-tree]]: information the compiler does not need is retained because the tools built on the compiler do.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

specWhich constructs are defined by expansion is a per-language question settled by the specification. Python defines for in terms of the iterator protocol and a += b in terms of __iadd__ with a fallback to __add__; Rust defines for in terms of IntoIterator and compound assignment in terms of the AddAssign trait; C++ leaves compound assignment on class types entirely to a user-defined operator with no required relationship to +. The shape is the same in all three and the guarantees are not.

typicalMainstream compilers expand short-circuit operators and compound assignment during or immediately after parsing, and keep loop and pattern constructs as nodes for longer so that borrow checking, exhaustiveness and lint passes can see the source form. This is a convention rather than a rule, and a language server for the same language usually keeps everything as nodes because it must answer questions about the text.

implementationJavaScript transpiler output for optional chaining differs by tool and target: some emit a single guarded expression with one temporary, some emit nested conditionals per link, and some emit a helper call. All three are correct; they differ in code size and in what a stepping debugger shows, so an intuition formed from one tool's output does not transfer.

If you were asked this in an interview

Give me a construct in a language you know that is sugar, and one that looks like sugar and is not. What distinguishes them?
Why is a += b not simply a = a + b?
A user reports a confusing type error that names a trait they never wrote. What do you ask them first?

Connections

API Designapi-design-principles sdk-design