Toolingimplementation

The Language Server

A compiler frontend rebuilt under three constraints a batch compiler never has: it must be incremental, it must produce answers about code that does not compile, and it may never throw information away. That is a different engineering problem, not the same one with a socket attached.

The question

Why can I not just run the compiler in a loop and send its output to my editor?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute
What the program is here

A long-lived in-memory model of a whole project: a full-fidelity syntax tree per file, a symbol index spanning all of them, a resolution and type cache keyed by what it was computed from, and a set of *unsaved* buffer contents that differ from what is on disk. Where a batch compiler holds one program and exits, a language server holds a continuously mutating program plus enough dependency information to know exactly which cached answers a keystroke invalidated.

What this phase may assume or do

A language server may assume almost nothing that a batch compiler assumes. It may not assume the file parses — most of the time, mid-keystroke, it does not. It may not assume names resolve, that types are consistent, or that the project builds. What it *is* entitled to assume is its own invalidation contract: a cached result stays valid exactly as long as every input it was derived from is unchanged, which is why every result must record its dependencies. Violating that contract is the characteristic language-server bug — an answer computed from a stale tree, served confidently, and wrong in a way the user experiences as "restart the language server".

Key points

  • A language server is a compiler frontend under three constraints: incremental, error-tolerant, and forbidden from discarding intermediate representations.
  • Source in an editor is syntactically invalid much of the time, and completion is needed at exactly the moment it does not parse.
  • The architecture inverts from a pipeline to demand-driven queries with memoization and recorded dependencies.
  • Early cutoff — recomputing a result and finding it unchanged, so dependents stay valid — is what makes an edit cost work proportional to the edit rather than the project.
  • Find-references is the reverse of name resolution and no compilation computes it, so it requires a separately maintained project-wide index.
  • Every feature needs a different representation to still be alive, which is why the server keeps all of them at once.
  • The characteristic bug is a stale answer served confidently, caused by an unrecorded dependency; the universal workaround is a cache flush.
  • Knowing how a file is compiled comes from the build system, and that boundary is the largest source of an IDE disagreeing with the compiler.

The three constraints

A batch compiler has an easy life, and it is worth spelling out how easy before describing what replaces it. It reads a fixed set of files once. It may fail on the first error and exit. It allocates freely, because the process ends in seconds and the OS reclaims everything. It computes things in a fixed order and never has to answer a question about a partial result. Every one of those properties is unavailable to a language server.

It must be incremental. The user types a character and expects diagnostics, completions and hover types within a frame or two. Recompiling the project is not an option, so the server must know precisely which of its cached results the edit invalidated — which is a dependency-tracking problem across the entire frontend, not a caching optimization bolted on afterwards.

It must be resilient to broken code. The single most important measurement in this area is that source in an editor is syntactically invalid a large fraction of the time, because that is what typing is. Completion is *most* needed right after you type ., which is precisely the moment the expression does not parse. A frontend that reports an error and stops has nothing to offer at the exact instant the user needs it. So the parser must produce a complete tree with error and missing nodes, and every later phase must tolerate holes — see [[error-recovery]] and [[parser-synchronization]].

It may never let information die. A compiler discards each representation as soon as the next one is built; that is good engineering and [[information-loss]] describes it. A language server is asked questions about every representation at once: "what does this token mean" (tokens), "what is this expression's type" (typed AST), "where is this symbol defined" (symbol table), "who calls this" (a project-wide index that no compilation ever needed). It must keep them all, keep them consistent, and keep them for a project rather than a translation unit.

The same frontend under two sets of requirementstypical
PropertyBatch compilerLanguage server
LifetimeSeconds; exits and frees everythingDays; memory is a permanent budget
InputFiles on disk, read onceEditor buffers that differ from disk, changing per keystroke
On a parse errorReport and, at some point, stopProduce a complete tree with error nodes and keep answering
Work per editn/a — one run per invocationProportional to what the edit actually invalidated
Order of workFixed pipeline, front to backDemand-driven: compute what this query needs and nothing else
Intermediate resultsDiscarded as soon as consumedRetained and indexed; several are the product
ScopeOne translation unit at a timeThe whole project, plus dependencies, plus reverse references
Failure modeNon-zero exit codeA stale answer served with full confidence

Demand-driven, not pipeline-driven

implementationQuery-based architectures are one family, not the only one. rust-analyzer (salsa) and the Rust compiler itself are query-driven end to end; the TypeScript language service is closer to versioned reuse with an incremental parser; clangd reuses preamble ASTs — the precompiled prefix of headers — and reparses the rest of the file, which works because C++ headers dominate the cost. Each is matched to its language's cost profile, and copying one into another language usually fails on the axis that language happens to be expensive on.

The architectural consequence of those constraints is that the frontend gets inverted. Instead of running phases in order and producing everything, the server computes results *on demand*, memoizes them, and records what each one depended on. A query such as "the type of the expression at line 40" pulls on "the resolved symbol for this name", which pulls on "the symbol table for this scope", which pulls on "the parse tree for this file", which pulls on "the text of this file". Nothing above the pulled path is computed at all.

This is the design behind rust-analyzer's salsa framework, and it is worth understanding as a general idea rather than a Rust one: every computation is a pure function of its inputs, results are cached with their dependency set, and an input change marks dependents as *maybe*-dirty rather than dirty. On the next query the framework re-runs a maybe-dirty computation and compares its result to the cached one; if it is unchanged — as it usually is, since most edits change one function body and nothing about the signatures anyone else depends on — everything downstream stays valid. That last refinement, sometimes called firewalling or early cutoff, is what makes editing a large project feel local.

The TypeScript language service reaches a similar place by different means: a document registry keyed by file version, source files reused wholesale when unchanged, and an incremental parser that reuses nodes from the previous tree for unedited regions. Roslyn does it with red-green trees plus a compilation object that can be cheaply forked from a previous one. Different mechanisms, one idea: the unit of reuse must be finer than the file, and something must record what depends on what.

The shape of a demand-driven frontend
1query parse(file)        -> Tree      { depends on: text(file) }
2query item_tree(file)    -> Items     { depends on: parse(file) }        // signatures only
3query resolve(name, ctx) -> Symbol    { depends on: item_tree(*) }
4query type_of(expr)      -> Type      { depends on: resolve(...), body(fn) }
5query diagnostics(file)  -> Diag[]    { depends on: type_of(...) }
6 
7// Edit inside a function body:
8//   text(f) changed -> parse(f) recomputed -> item_tree(f) recomputed
9//   ... but item_tree is UNCHANGED (signatures did not move),
10//   so resolve() in every other file stays valid and nothing else reruns.

The early-cutoff comparison at item_tree is the whole trick. Without it, editing one line invalidates every downstream result in the project and the server is a batch compiler with extra steps. With it, a body edit costs one file's worth of work no matter how large the project is — which is also exactly the argument for separating signatures from bodies in [[interface-files]] and [[incremental-compilation]].

What it actually has to answer

The feature list looks like a grab bag until you notice that each one needs a different representation to still be alive, which is the argument of this lesson stated as a table of requirements.

And several of them have latency budgets that decide the architecture rather than following from it. Completion after . must return in tens of milliseconds or the user has typed past it; diagnostics may take a second; find-references across a large project may take longer still and is expected to show progress. A server that computes everything eagerly meets none of these budgets; a server that computes nothing until asked meets all of them for the common case and needs a background index for the project-wide ones.

  • Diagnostics — the compiler's errors and warnings, but computed for a buffer that is not on disk and republished after every change, unsolicited.
  • Completion — needs the type of the expression before the cursor, at a moment when the expression is incomplete. Frequently the single hardest feature to make both fast and correct.
  • Hover — needs the resolved symbol plus its type plus its documentation comment, which means trivia had to survive into the tree.
  • Go to definition — needs the symbol table and a span; the classic case that name resolution exists for, exposed directly to the user.
  • Find references — the *reverse* of resolution, which no compilation ever computes. It requires a project-wide index built and maintained in the background.
  • Rename — find-references plus a validity check, and the subject of [[semantic-refactoring]].
  • Signature help — needs the callee resolved and the current argument index, inside a call expression that is by definition unfinished.
  • Semantic highlighting, inlay hints, code lens — need types for spans that the user is not even looking at, computed for the visible range only if you want it to be fast.

Where they go wrong

The failure modes are specific to this architecture and, once you know them, immediately recognisable from the user side.

Staleness. A dependency was not recorded, so a cached result outlived one of its inputs. The user sees an error on a line they fixed thirty seconds ago, or completion offering a field they deleted. The universal workaround — "restart the language server" — is a full cache flush, and its popularity is a measure of how hard the invalidation contract is to get exactly right.

Unbounded memory. Never letting information die is the requirement; not letting it grow forever is the engineering. Every server needs eviction — least-recently-used file trees, dropping type caches for files not open, bounding the index — and the ones that skip it are the ones that need restarting after an afternoon.

The cold-start cliff. Opening a large project means indexing it, and until the index exists, find-references is wrong and completion is incomplete. Every server has a strategy here (persist the index to disk, index dependencies lazily, serve syntactic results while semantic ones warm up) and every strategy leaks into the user experience.

The build-system boundary. A server cannot analyse a file without knowing how it is compiled: the include paths, the feature flags, the target, the macro definitions, the module graph. That information lives in a build system the server does not own, which is why compile_commands.json exists for C++, why cargo check metadata drives rust-analyzer, and why an unconfigured monorepo produces a server that is confidently wrong about everything. This boundary is the single largest source of "the IDE says it is broken but it compiles fine".

How it works

The steps, in the order the compiler takes them.

  • The client sends the full text of a file on open and a sequence of incremental changes thereafter, so the server's copy of the buffer is authoritative and may differ from disk.
  • The server parses with an error-tolerant parser into a full-fidelity tree, inserting error and missing nodes rather than aborting.
  • Every derived result — item signatures, resolved names, types, diagnostics — is computed lazily by a query, memoized, and stamped with the set of inputs it read.
  • A text change bumps a revision counter and marks dependents maybe-dirty; nothing is recomputed until a query asks.
  • On a query, maybe-dirty results are recomputed and compared with their previous value; an unchanged value stops the invalidation from propagating further.
  • Project-wide reverse information — references, implementations, subtypes — is built by a background indexer and updated as files change, since no forward computation produces it.
  • Results are cached with an eviction policy, because the process must survive days of editing within a bounded memory budget.
  • Compilation settings — include paths, features, target, module layout — are obtained from the build system, and every analysis is parameterised by them.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

  • The editor shows an error on a line the user already fixed, because a cached result outlived an input whose dependency was never recorded.
  • Completion returns nothing at the moment it is most needed, because the expression before the cursor did not parse and the frontend gave up instead of producing an error node.
  • The server's memory grows through a day of editing until the editor becomes unresponsive, and the fix everyone learns is to restart it.
  • Find-references misses call sites in a file that has not been indexed yet, and a rename based on it produces a project that no longer compiles.
  • The IDE reports hundreds of errors in a project that builds cleanly, because the build configuration it inferred is wrong — the wrong feature flags, a missing include path, the wrong target.
  • Every keystroke triggers a full project re-analysis because early cutoff is missing, so typing in a large file lags by seconds.
  • Two clients open the same file and the server serves an answer computed from the other client's buffer version, producing an off-by-a-few-characters span.

When it helps

  • Any language with more than one editor, since the alternative is one integration per editor and none of them equally good.
  • Large projects, where the entire value proposition is that answers cost the size of the edit rather than the size of the codebase.
  • Refactoring and navigation features, which need semantic information that no text-based tool can produce correctly — see [[semantic-refactoring]].
  • Teaching and onboarding: inlay hints, hover types and go-to-definition make an unfamiliar codebase legible in a way that reading files does not.

When it hurts

  • Small scripts and single files, where a batch compile is fast enough and the server's memory and start-up cost buy nothing.
  • Languages whose semantics are genuinely dynamic, where the server can offer syntax but very little true resolution, and confident-looking completion is misleading.
  • Build systems that cannot describe how a file is compiled, where the server is guessing and its diagnostics disagree with the real build.
  • Environments with a hard memory ceiling — remote development on a small container, a large monorepo on a laptop — where the retained model does not fit and eviction thrashes.

What it costs

Every one of these is paid by something.

  • Demand-driven queries buy work proportional to the edit and pay a pervasive architectural constraint: every computation must be a pure function with declared inputs, which rules out the convenient mutable global state a batch frontend uses freely.
  • Memoizing everything buys latency and pays memory that grows for as long as the process lives, forcing an eviction policy that will occasionally evict the thing you were about to need.
  • Error-tolerant parsing buys answers on broken input and pays a parser that is substantially harder to write, harder to test, and capable of producing a tree that later phases must all be hardened against.
  • A background project index buys find-references and rename and pays a cold-start period during which those features are silently incomplete, plus the memory to hold it.
  • Sharing the compiler's frontend buys exact agreement between IDE and build and pays by constraining the compiler's architecture to be incremental and error-tolerant everywhere — a cost the compiler team pays for the IDE team's benefit.
  • Reimplementing a separate frontend for the IDE buys freedom to optimise for latency and pays with two implementations that disagree, which users experience as errors that appear in one place and not the other.

What else you could do

What a different compiler or language does instead, and when that is better.

  • A batch compiler invoked on save, with its diagnostics parsed from stderr. Simple, honest, and it offers no completion, no hover and no navigation — which is what most editors did before language servers.
  • Purely syntactic tooling: tree-sitter for highlighting, folding and structural navigation with no name resolution at all. Fast, robust on broken code, language-agnostic, and unable to answer any semantic question.
  • Tags-based navigation (ctags, gtags): an index of names to definitions built by regex. Instant, works everywhere, and wrong for anything involving scope, overloading or generics.
  • An editor-specific integration compiled into the IDE, which is what Visual Studio and IntelliJ did for years. Deeper integration and better latency, at the cost of one implementation per editor per language — the M×N problem [[lsp]] exists to solve.
  • Running the analysis in the cloud on a prebuilt index, which is what code-search products do. Excellent for reading a project you have not cloned, useless for the buffer you are editing.

See it for yourself

The flag, dump or tool that shows you this directly.

  • clangd: clangd --check=path/to/file.cc runs the whole pipeline on one file and prints timings and diagnostics without an editor — the fastest way to find out why the IDE disagrees with the build. clangd --log=verbose shows the LSP traffic, and it needs a compile_commands.json to be correct at all.
  • rust-analyzer: rust-analyzer analysis-stats . prints how much of the project it could resolve and where it failed; RA_LOG=info for logs; the VS Code commands "Show Syntax Tree" and "Status" expose the internal model directly.
  • TypeScript: set "typescript.tsserver.log": "verbose" in VS Code and open "TypeScript: Open TS Server log", or run tsserver directly. tsc --generateTrace traceDir plus the analyzer in @typescript/analyze-trace shows where check time actually goes.
  • gopls: gopls check ./... from the command line, gopls stats for memory and cache statistics, and -rpc.trace for the protocol traffic.
  • Any server, any editor: turn on the client-side trace ("<lang>.trace.server": "verbose" in VS Code, :LspLog in Neovim) and read the JSON-RPC. Most "the IDE is broken" reports are visible there in under a minute.
  • Memory: most servers expose a status or statistics command; use it before assuming the editor is at fault, since a server holding a whole project is frequently the largest process on the machine.

Plausible wrong readings

Stated the way a confident engineer states them.

  • "A language server is the compiler with a socket on it." It is the frontend rebuilt around three requirements the compiler does not have. Toolchains that tried the socket approach ended up rewriting the frontend anyway.
  • "Completion is a text-matching feature." Useful completion needs the type of the expression before the cursor, which needs resolution, which needs a tree — at the moment the code does not parse. That is why it is the hardest feature and the one that most often degrades to identifier matching.
  • "If it is slow, the machine is too small." Usually a dependency is over-broad, so an edit invalidates far more than it should. The fix is in the invalidation graph, not in the hardware.
  • "The IDE errors are wrong; the build is what matters." Both are right about different configurations. Almost always the server inferred different compilation settings, and the fix is to give it the real ones.
  • "Restarting the language server fixes bugs." It flushes a cache that had gone stale. The bug is a missing dependency edge, and it is still there.

Misconceptions

The claim, and what is actually true.

The language server and the compiler should obviously share code.
They should, and doing so forces the compiler to be incremental and error-tolerant throughout — a large cost paid by the compiler team. Several ecosystems maintain two frontends instead, and users pay for it in disagreements between the two.
Find-references is just go-to-definition run backwards.
Resolution is a forward computation the compiler already performs. The reverse map exists nowhere and must be built and maintained by a background indexer over the whole project.
Incrementality is a performance optimization.
It is the architecture. A frontend that is not incremental from the ground up cannot be made so by caching, because the caches have no dependency information to invalidate against.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

Your editor asks questions a compiler is never asked: what type is this expression while I am halfway through typing it, where is this defined, who calls it. Answering them needs the compiler's frontend, but under three conditions it was never built for — the answer must come back in milliseconds after every keystroke, the code usually does not parse yet, and every intermediate result has to be kept around because some feature asks about each of them. That is why a language server is a genuinely different program rather than the compiler with a network port.

practical

When the IDE disagrees with the build, check the compilation settings first — compile_commands.json for clangd, the Cargo metadata for rust-analyzer, the tsconfig.json include set for TypeScript. That accounts for most of it. When the server is slow, look at whether an edit is invalidating more than it should rather than at the machine: analysis-stats, gopls stats and the TypeScript trace analyzer all show where the time goes. And when you find yourself restarting the server regularly, file it — the restart is flushing a cache that went stale, and a stale-cache bug that nobody reports is one nobody fixes.

advanced

The deep design decision is the granularity of the dependency graph, and it is a direct trade against the cost of the comparison that implements early cutoff. Track dependencies per file and an edit anywhere invalidates everything downstream of that file. Track them per item — this function's signature, that struct's fields — and a body edit invalidates almost nothing, which is what makes editing a large project feel local; but now every query key is finer, the graph is larger, and the bookkeeping itself costs memory and time. The separation of an "item tree" (signatures only, deliberately independent of bodies) from the bodies themselves is the standard resolution, and it is the same idea as [[interface-files]] in a batch compiler, arrived at from the opposite direction. Which is the real lesson: incremental compilation and interactive tooling are the same problem with different latency budgets, and the architectures converge.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

implementationEvery architectural detail here is server-specific. rust-analyzer is query-driven through salsa; the TypeScript language service uses versioned document reuse with an incremental parser; clangd caches a preamble AST because C++ header processing dominates its cost; gopls keeps a package-level cache keyed by file content hashes. The three constraints are common to all of them; the mechanism chosen to satisfy them is matched to what each language happens to be expensive at, and does not transfer.
typicalThe claim that source in an editor is invalid much of the time is an observation about interactive editing, not a measured constant, and it varies a great deal by language: a whitespace-sensitive language with few delimiters spends less time in an unparseable state than one with nested brackets and generics. What is uniform is the *timing* — the invalid moments cluster exactly where completion is requested, which is what makes error tolerance non-negotiable rather than merely nice.
specNothing in the LSP specification requires any of this architecture; the protocol only fixes the messages. A conforming server may recompile the world on every keystroke and be correct but unusable, and several early servers did exactly that. The three constraints come from the interaction budget and from what users ask for, not from the standard — which is why "it speaks LSP" says nothing about whether it will be good.

If you were asked this in an interview

  • Name the three constraints a language server has that a batch compiler does not, and say what each one forces architecturally.
  • Why is completion the hardest feature to make both fast and correct?
  • What is early cutoff in an incremental frontend, and what happens without it?

Connections

DSAhash-tabledirected-graphn-ary-treetopological-sort
Performancelatency-fundamentalsprofiling-basics
Domains that do not exist yet
  • Programming Languages & Runtime Internals — Memory management for a long-lived process holding a large in-memory model
    A language server's hardest operational problem is that it must retain everything and still fit in a budget for days, which is an allocation, retention and eviction problem rather than a compilation one. How a runtime's allocator and collector behave under that workload is owned there; why the model cannot simply be discarded is ours.
  • DevOps / Production Engineering — Making build configuration machine-readable so tooling and CI agree
    The largest source of an IDE disagreeing with the build is that the server inferred the compilation settings instead of being told them. Emitting compile_commands.json or equivalent metadata from the build is a build-engineering practice owned there, and it is the precondition for any of this analysis being correct.