Inlining
Replace a call with the callee's body. The direct saving — a call and a return — is the least interesting part; the value is that every other optimization can now see across a boundary it could not cross. The cost is code size, compile time and instruction-cache pressure, and it is a budget rather than a rule.
When does inlining a function actually make the program faster, and when does it make it slower?
IR for the caller with a call instruction, plus IR for the callee — two functions, each analysed in isolation. Inlining is the transformation that turns two representations into one, which is the entire point: analyses that are intraprocedural by construction become interprocedural for free once the boundary is gone. Everything downstream then operates on a single, larger CFG.
The callee's body must be available, its parameters must be bound to the argument values, its local names must be renamed to avoid capture, its returns must become jumps to a continuation, and the resulting program must observe the same sequence of effects. Correctness is rarely the hard part: barring varargs, recursion without a base case in the specializer, setjmp/longjmp, and functions whose address identity is observed, inlining is almost always *legal*. The hard question is whether it is *wise*, and that question has no legality answer — only a cost model.
Key points
- The direct saving is a call, a return, argument marshalling and a prologue; on a modern core that is small.
- The real value is that optimizations which are intraprocedural by construction can now see across the boundary — constants flow in, branches collapse, loads become eliminable.
- The costs are code size, instruction-cache pressure, compile time and debuggability, and all four are measurable.
- Because it is nearly always legal, the interesting question is a cost model, not a precondition — which makes it unlike every other lesson in this module.
- Inlining one function makes it a worse inlining candidate itself, so real inliners work bottom-up on the call graph with a budget.
- What a compiler can inline depends on what it can see: translation-unit boundaries limit it, LTO removes the limit at link-time cost, and a JIT sidesteps it with a guard.
What is actually saved, and what is actually gained
The mechanical saving from inlining is small and easy to overstate: a call instruction, a return, the argument marshalling required by the calling convention, and the prologue and epilogue that save and restore registers. On a modern out-of-order core with a return-address predictor, a predictable direct call costs a handful of cycles. If that were all inlining bought, it would be a minor optimization.
The real gain is that the boundary disappears. Before inlining, the caller must assume the callee may clobber every caller-saved register, may write to any memory the callee could reach, and returns a value about which nothing is known. After inlining, all of that is visible: constants flow into the body, branches on those constants become literal, dead arms vanish, redundant loads across the former call become eliminable, and the callee's own code specializes to the one call site it was inlined into.
That cascade is why inlining is usually described as the most important optimization in an optimizing compiler. It is not doing very much itself; it is what makes everything else possible. It is also why the effect is so uneven: inlining a function whose arguments are all run-time values into a cold path buys the call overhead and nothing else, while inlining a two-line accessor with a constant argument can delete an entire branch.
static int scale(int v, int factor) {
if (factor == 0) return 0;
return v * factor;
}
int use(int v) { return scale(v, 1); }// after inlining and the passes it unblocked:
int use(int v) { return v; }The callee is available, has no varargs, does not observe its own address, and binding factor to the literal 1 preserves the argument's value. Once the body is in place, constant propagation makes the condition 1 == 0, branch simplification removes the taken-never arm, strength reduction removes v * 1, and dead code elimination clears the remains. Every step has its own precondition; inlining supplied none of them and enabled all of them.
The callee's behavior depends on the call boundary itself. A function calling alloca, whose storage is released when its frame is popped, changes lifetime when inlined. A varargs function has no fixed parameter list to bind. A function whose address is compared for identity, or that inspects its own return address for a stack trace, observes the frame it was given. And in a JIT, inlining a speculatively devirtualized target requires a guard and a deoptimization path — see [[guards]] and [[deoptimization]].
The costs, all of which are real
-finline-limit and a set of parameters. The inline keyword in C and C++ is not among the strong signals: it is primarily a linkage rule about multiple definitions, and compilers treat it as a weak hint at best.Every inlined copy of a function body is another copy of that body in the binary. Inline a 40-instruction helper at 200 call sites and the function has been paid for 200 times. That is not merely disk space: instruction cache is a small, shared, fixed resource, and a working set that no longer fits in it converts every hot loop iteration into instruction fetches from further out in the memory hierarchy.
Compile time grows superlinearly with aggressive inlining, because the passes that run after it now run on much larger functions, and many of them are worse than linear. This is a routine cause of a build that becomes minutes slower after someone turns a header into a template or marks a large function inline. Debug builds avoid most of it, which is one reason the gap between debug and release build times is what it is.
Debuggability suffers in a specific and familiar way. There is no frame for the inlined function, so a naive backtrace does not show it; DWARF records inlined subroutine information so a good debugger can reconstruct it, and profilers frequently cannot, which is why a flame graph sometimes attributes a callee's time to its caller — see [[debugging-optimized-code]].
And it interacts badly with itself. Inlining A into B makes B bigger, which makes B less likely to be inlined into C. Every real inliner therefore works with a budget, a size threshold, a call-site hotness estimate, and a bottom-up traversal of the call graph, and the ordering of those decisions matters as much as the decisions themselves.
| Signal | Pushes toward inlining | Pushes against |
|---|---|---|
| Callee size | Small enough that the body costs less than the call sequence | Large body duplicated at many sites |
| Call-site count | Called exactly once — inlining costs nothing in size and lets the original be deleted | Hundreds of sites, so the body is duplicated hundreds of times |
| Constant arguments | A literal argument that will collapse a branch or specialize the body | All arguments unknown, so nothing specializes |
| Profile data | Site is measurably hot — [[profile-guided-optimization]] uses exactly this | Site is measurably cold; inlining spends size on code that rarely runs |
| Loop context | Call is inside a hot loop, so the overhead is paid per iteration | Call is on an error path |
| Recursion | Bounded depth, or the recursion becomes a loop after inlining | Unbounded — inlining cannot terminate without an arbitrary depth limit |
| Visibility | Callee is static or has internal linkage, so the out-of-line copy can be removed | Callee is exported and the standalone copy must exist anyway |
Where the decision is made, and by whom
A traditional compiler can only inline what it can see, which in C and C++ means the callee must be in the same translation unit — hence header-only libraries, static inline in headers, and templates, all of which exist partly to make bodies visible. [[link-time-optimization]] removes that limitation by deferring code generation until link time, when every body is available; the cost is a link step that is now a whole-program compile.
A JIT has a different set of constraints and a better set of facts. It knows which call sites are hot because it counted them, it knows the actual receiver types because it observed them, and it can inline speculatively behind a guard and undo the decision if the guard fails. That is why virtual calls in Java and JavaScript are frequently inlined and equivalent C++ virtual calls frequently are not — not because the languages are faster or slower, but because the information arrives at a different time. See [[devirtualization]] and [[speculative-optimization]].
Programmer-facing controls exist and are mostly weaker than they look. inline in C and C++ is a linkage keyword. __attribute__((always_inline)) and [[gnu::always_inline]] are genuine directives and are how you force it when you have measured. #[inline] in Rust is a hint that also makes the body available across crate boundaries, which is the part that matters. @inline annotations in JVM-hosted languages are advisory to a compiler that will make its own decision at run time regardless.
How it works
The steps, in the order the compiler takes them.
- Choose a call site, usually bottom-up over the call graph so that callees are already in their final form.
- Estimate the cost: callee size after simplification with the actual arguments, against a threshold adjusted by call-site hotness and by whether the callee has other callers.
- Clone the callee's blocks into the caller, renaming every value so nothing captures a caller name.
- Bind parameters to argument values — usually as copies, which
[[copy-propagation]]removes immediately afterwards. - Rewrite each
returnas a jump to a continuation block, with a phi merging the returned values if there was more than one return. - Re-run simplification on the enlarged function: propagation, folding, branch simplification and dead code elimination all now have new material.
- If the callee has no remaining callers and is not externally visible, delete it.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- A binary grows by tens of percent after an inlining threshold change, the hot working set stops fitting in the instruction cache, and throughput drops with no change in instruction count per operation. The profile shows front-end stalls rather than any single slow function.
- A build that took ninety seconds takes six minutes after a widely-used helper moves into a header, because every translation unit now inlines it and every later pass runs on larger functions.
- A stack trace from a crash is missing frames, and a profiler attributes all the time in a hot helper to its caller. The function is present in source and absent from the frame layout.
- A recursive function is inlined to the compiler's depth limit, producing a very large function that then spills heavily; the code is slower than the original call-based version.
- A change to a small function has no effect until every dependent translation unit is rebuilt, because the old body was inlined into all of them — the incremental-build hazard that makes header-only libraries expensive to iterate on.
When it helps
- Small accessors, wrappers, iterators and lambdas, where the body is smaller than the call sequence and the abstraction was free only if it is removed.
- Call sites with constant arguments, where inlining is the enabler for specialization that removes far more than the call.
- Hot loops containing a call, where the overhead is paid every iteration and removing it also lets the loop be unrolled or vectorized — a vectorizer will usually refuse a loop with a call in the body, so inlining is a precondition for
[[compiler-vectorization]]. - Functions called exactly once, where inlining is close to free in size and lets the standalone copy be deleted.
When it hurts
- Large bodies at many call sites, where the size cost is multiplied and the instruction cache pays for it — the case where measured performance goes down.
- Cold paths, especially error handling, where inlining spends binary size and cache footprint on code that essentially never runs.
[[gnu::cold]]andunlikelyannotations exist to say so. - Anything you need to see in a profile or a backtrace, where losing the frame costs more in diagnosis time than the call cost.
What it costs
Every one of these is paid by something.
- Inlining buys removed call overhead and, far more importantly, cross-boundary optimization; it pays in binary size, which converts directly into instruction-cache pressure on hot paths and into slower incremental builds.
- Making bodies visible so they *can* be inlined — headers, templates,
#[inline], LTO — buys the opportunity and costs compile time, build parallelism and incremental rebuild granularity: a change to an inlined body invalidates every consumer. - A more aggressive threshold buys speed on the sites where specialization pays and costs it back on the sites where it does not, and the compiler cannot tell them apart without a profile — which is the entire argument for
[[profile-guided-optimization]].
What else you could do
What a different compiler or language does instead, and when that is better.
- Do not call in the first place: restructuring so the work happens once outside a loop beats optimizing the call away, and does not depend on any compiler decision.
- Link-time optimization gets cross-module inlining without header-only distribution, at the cost of a much heavier link —
[[link-time-optimization]]. - Profile-guided optimization tells the inliner which sites are hot instead of letting it guess, which is the single largest improvement available to a static inliner —
[[pgo-tradeoffs]]. - A JIT defers the decision until the hot sites are known by measurement rather than estimate, and can inline speculatively behind a guard — an option unavailable to a static compiler —
[[jit-compilation]]. - Manual specialization: write the specialized version yourself when you know the important case. Verbose, and it does not depend on a threshold you do not control.
See it for yourself
The flag, dump or tool that shows you this directly.
- Clang:
-Rpass=inlinereports each successful inline and-Rpass-missed=inlinereports each refusal with the cost and threshold that decided it. This is the single most useful flag in the lesson. - GCC:
-fopt-info-inlineand-fopt-info-inline-misseddo the same;-fdump-ipa-inline-detailsprints the full decision process. - Measure the size cost directly:
size a.outbefore and after, andnm --size-sortto find which functions grew. - Check whether inlining is why something is fast: recompile the hot function with
__attribute__((noinline))and re-measure. If the difference vanishes, it was the specialization and not the call overhead. - For a JIT:
-XX:+PrintInliningon HotSpot, and--trace-turbo-inliningin V8, both show run-time decisions a static compiler could not have made.
Plausible wrong readings
Stated the way a confident engineer states them.
- "Marking a function
inlinemakes it inline." In C and C++ that keyword is about linkage and multiple definitions. The compiler decides, andalways_inlineis the attribute that actually directs it. - "Inlining removes function-call overhead, so more is better." The overhead removed is small; the size added is not. Past the point where the hot code stops fitting in the instruction cache, more inlining is measurably worse.
- "A function too big to inline cannot be optimized well." It is optimized normally — it just is not specialized to any one caller. Partial inlining of a hot early-return path is often the better answer anyway.
- "Small functions are always inlined." Not across translation units without LTO, not through function pointers or virtual calls without devirtualization, and not when the compiler's cost model disagrees with your estimate of small.
Misconceptions
The claim, and what is actually true.
inline keyword controls inlining.always_inline and noinline are the directives; inline is at most a weak hint.-Rpass-missed=inline will tell you the cost and the threshold, and the answer is often that the body genuinely is too large to duplicate.Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
Instead of jumping to a function and back, the compiler pastes the function's code where the call was. That saves the jump, but the bigger effect is that the compiler can now see the function and the calling code together and simplify across the join. The price is that the code is now in the program more than once.
practical
Do not guess whether something was inlined — ask with -Rpass=inline or -fopt-info-inline. If you need it inlined, use always_inline and measure both time and binary size. If a build got slow or a binary got fat, look at inlining thresholds and at what recently moved into a header. And if a profile has a suspiciously heavy caller, suspect that a callee was inlined into it and check the debug info.
advanced
The hard part is that the decision is not local. Inlining A into B changes whether B should be inlined into C, so the problem is a traversal of the call graph under a global budget, and it is NP-hard in the general formulation. Real inliners approximate: bottom-up ordering on the strongly-connected components of the call graph, per-function and per-module budgets, cost estimation that simulates the simplification the inline would enable rather than measuring the callee as written, and — with a profile — hotness weighting. Partial inlining, where only the early-exit prologue of a large function is inlined and the cold rest stays out of line, is the refinement that captures most of the benefit at a fraction of the size, and it is why some functions appear both inlined and not in the same binary.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
inline keyword all differ between GCC, Clang and MSVC, so "will this be inlined" is answerable only by asking the specific compiler with -Rpass=inline or its equivalent.If you were asked this in an interview
- What does inlining actually buy, beyond removing the call and return?
- Give me a case where inlining measurably slows a program down, and say what you would measure to confirm it.
- Why can a JIT inline a virtual call that a C++ compiler will not?
Connections
- Programming Languages & Runtime Internals — Inlining performed by a running virtual machine, and the frame bookkeeping that lets it be undoneA JIT inlines using measured hotness and observed types, which a static compiler cannot have. The compiler-side half — the guard, the state map, the deoptimization point — is ours; the runtime that maintains and consults them is theirs.