Interoperability Is a Language Design Decision
A language that must call existing code has already had its data representation, its error model and its threading model partly decided for it. The C ABI is the lingua franca not because it is good but because everything already speaks it.
What does it cost my language to be able to call, and be called by, code written in something else?
The boundary as a contract expressed in machine terms rather than in source terms: a calling convention, a memory layout for every type that crosses, an ownership rule for every pointer, and an agreement about what happens when something fails. Neither side sees the other's source, so everything the two agree on must be expressible in the object file — which is why [[abi]] and [[name-mangling]] are the concrete form of a design decision made here.
A value may cross the boundary only if both sides agree on its representation exactly: the same size, the same alignment, the same field order, the same padding, and the same convention for who frees it. A compiler may not apply any layout optimization — field reordering, niche packing, tail-call elision that changes stack behavior — to a type that crosses, unless the language provides an explicit annotation freezing the layout. Rust's repr(C) is exactly that annotation, and applying Rust's default layout to a struct handed to C is undefined behavior on both sides.
Key points
- The C ABI is the shared vocabulary by default, and speaking it means giving up layout optimization, adopting an out-of-band ownership convention and converting every error at the boundary.
- The expensive mismatches are memory management, error models and blocking behavior — none of which the ABI can express.
- Exceptions and panics must not unwind through a foreign frame; each side converts at the boundary and loses information doing it.
- Being callable is much harder than being able to call, and requires a runtime that is embeddable, re-entrant and willing not to own the process.
- Not sharing an address space removes every one of these problems and replaces them with serialisation and a process boundary, which is frequently the better trade.
The C ABI is the only shared vocabulary
Nearly every language's interoperability story is "we speak C". Not because C's conventions are good — they carry no ownership information, no error channel and no type information at run time — but because every operating system, every driver, every database client and every cryptographic library exposes one, and because the calling convention is defined per platform in a document everyone already implements.
What crossing that boundary actually requires is a specific list, and every item on it is a design decision the language must have made. The data layout must be expressible as C structs, which forbids the layout optimizations most languages want to perform. Ownership must be conveyed out of band, in documentation, because the ABI has no way to say who frees a pointer. Errors must be conveyed as return codes or out-parameters, because C has no error channel. And strings must be agreed on: null-terminated bytes on one side, and on the other, whatever your language decided — which is why every language has a string conversion at its C boundary and why the conversion is usually a copy.
| Language | Layout at the boundary | Ownership across it | Errors across it |
|---|---|---|---|
| Rust | repr(C) freezes field order and padding; default layout is unspecified and may not cross | Manual, inside unsafe. CString and raw pointers make the transfer explicit | Converted at the boundary; a panic unwinding into C is undefined and must be caught |
| Go | cgo generates conversions; Go pointers may not be stored in C memory | Pinning rules enforced at run time, because the collector may move or free | Errors converted at the boundary; a C call blocks a scheduler thread |
| Java | JNI, or the newer foreign-function API with explicit memory layouts | Explicit local and global references, because the collector must know what C holds | Exceptions checked for explicitly after every JNI call; they do not propagate |
| Python | C extension structs, or ctypes/cffi describing the layout at run time | Reference counting is manual and exact in the C API; one missed decrement is a leak | Errors are a thread-local indicator plus a NULL return, checked at every call |
| C++ | extern "C" disables mangling; classes with virtual functions or non-trivial layout cannot cross | Manual; smart pointers do not survive the boundary | Exceptions must not propagate into C; they are caught and converted |
The three mismatches that actually hurt
Calling convention and layout are the easy half; tooling handles them. The expensive mismatches are the ones where the two languages have incompatible models of something the ABI cannot express.
Memory management. A garbage-collected language handing a pointer to C has a problem: the collector may move the object or free it while C holds the pointer. Every managed language solves this with pinning, handles or explicit global references, and every one of those mechanisms is a place where forgetting produces a crash that is hard to attribute. In the other direction, a pointer C allocated must be freed by C's allocator, which means the managed side must remember to call back rather than let its own mechanism handle it.
Error models. Exceptions do not cross an ABI boundary. Unwinding through a frame compiled by a language that does not know about unwinding is undefined, which is why Rust must catch panics at extern "C" boundaries and C++ must not let exceptions escape one. The general rule is that each side converts to the other's model at the boundary, and every conversion is a place where information — a stack trace, a cause chain, a typed error — is lost.
Threading and blocking. A language with a lightweight-task scheduler has a problem when foreign code blocks: the scheduler thread is gone until the call returns. Go handles this by detaching the thread from the scheduler around cgo calls, which is why cgo calls are much more expensive than Go calls and why a program making many of them behaves quite differently. A language with a global interpreter lock has the mirror problem: foreign code must release it to allow concurrency and must reacquire it before touching anything of the language's own.
Being callable is harder than being able to call
Most languages get the outbound direction working early and find the inbound direction much harder, because being callable means your runtime must be startable, embeddable and re-entrant on a thread you did not create.
The requirements compound. A garbage collector must be able to find roots on a stack that mostly belongs to foreign code. A runtime that assumes it owns the process — signal handlers, thread-local state, the main thread — must be made to coexist. Initialisation must be idempotent and thread-safe, because the host may call in from several threads at once. And the exported surface must have stable, unmangled names, which returns to [[abi-stability]] and to the question of what a released binary promises.
This is why languages with excellent outbound FFI often have poor embeddability, and why the languages that are genuinely good at being embedded — C, Lua, and to a degree Rust — either have no runtime to speak of or made it explicitly re-entrant from the start. It is a design decision, and it is nearly impossible to retrofit.
The alternative: do not share an address space
The whole category of problem exists because two languages are sharing memory in one process. Give that up and it dissolves: a subprocess with a pipe, a socket, or a message queue has no layout agreement, no ownership question and no unwinding hazard, because nothing crosses but bytes.
The costs are real and are the ordinary ones — serialisation on every call, a process boundary's latency, and a supervision problem — but for coarse-grained integration they are usually smaller than the FFI bill, and they buy fault isolation that an in-process boundary can never provide. A crash on one side of a pipe is an error code; a crash on one side of an FFI call is a crash.
WebAssembly is the interesting middle: a shared process, an isolated linear memory, and an interface described by types rather than by a calling convention. It gives up direct pointer sharing — which is exactly the thing that makes FFI both fast and dangerous — and gets sandboxing and a checkable interface in exchange. See [[wasm-model]].
How it works
The steps, in the order the compiler takes them.
- The language provides an annotation that freezes a type's layout to the platform C layout, disabling its own field reordering and packing for that type.
- It provides a linkage annotation that disables name mangling so the symbol is findable by a plain name.
- It defines what happens to its own error mechanism at the boundary — catch and convert, or undefined — and enforces it.
- A managed runtime provides pinning, handles or global references so foreign code can hold a pointer the collector will not invalidate.
- A scheduled runtime detaches or replaces its worker thread around a foreign call that may block.
- Tooling generates the binding stubs from a header or an interface description, because writing them by hand is where the layout mistakes come from.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- A struct passed to C works on one platform and corrupts data on another, because the language reordered its fields or chose different padding and nobody applied the layout annotation.
- A managed object is collected or moved while foreign code holds a pointer to it, and the crash appears in the foreign library, which is then blamed.
- An exception unwinds into a frame compiled by a language that does not understand unwinding, and the process terminates with no useful information at all.
- A program making many small foreign calls is an order of magnitude slower than expected, because each call costs a scheduler detach or a lock acquisition that a same-language call does not.
- A library is upgraded, its struct gains a field, and every program built against the old header reads garbage — the ABI break that a source-compatible change caused. See
[[abi-stability]]. - A memory leak grows slowly in production, and it is one missing reference decrement on an error path through the C extension boundary.
When it helps
- Deciding what a new language must be able to reach on day one, which determines its string representation, its struct layout rules and its error model far more than aesthetics do.
- Choosing between in-process FFI and a process boundary for an integration, where the FFI bill is usually underestimated and the serialisation bill overestimated.
- Diagnosing crashes that appear inside a third-party native library, which are very often an ownership or lifetime mistake on the calling side.
When it hurts
- Designing the whole language around interoperability. A language that adopts C's data model wholesale inherits its weaknesses and gains nothing its users could not have had by writing C.
- Using FFI for fine-grained calls. The per-call overhead is real and can exceed the work being done, which converts an integration into a performance problem.
What it costs
Every one of these is paid by something.
- Speaking the C ABI buys access to every existing library on the platform, and costs the ability to optimize the layout of any type that crosses, plus an ownership convention that exists only in documentation and is therefore violated.
- Making the language embeddable buys adoption as a scripting or extension language, and costs a runtime that must be re-entrant, must not own process-wide resources, and must find its roots on a foreign stack — constraints that touch the collector, the scheduler and the initialisation path.
- Generating bindings automatically buys correctness at the layout level, and costs a tool that must track the foreign language's type system and a build step that now depends on parsing foreign headers.
- Choosing a process boundary instead buys fault isolation and no layout agreement at all, and costs serialisation on every call, latency, and a supervision and lifecycle problem that did not previously exist.
What else you could do
What a different compiler or language does instead, and when that is better.
- Communicate over a protocol rather than a call: a subprocess with a pipe, a local socket, or gRPC. Slower per call, isolated on failure, and free of every layout and ownership question.
- Compile both sides to a common target — WebAssembly, or the JVM, or the CLR — so the boundary is inside one type system rather than across two. Buys checkable interfaces, costs the ability to use anything not available on that target.
- Reimplement rather than bind, which is often cheaper than it sounds for small libraries and removes a permanent maintenance dependency on someone else's ABI.
- Generate the binding from a shared interface description rather than from a header, which is what protocol definition languages do for the process-boundary case —
[[dsl]].
See it for yourself
The flag, dump or tool that shows you this directly.
- What actually crosses:
nm -gD libfoo.solists the exported symbols, and mangled names tell you immediately whether the boundary is C or C++. - Whether your layout matches:
paholeon the object file, orclang -Xclang -fdump-record-layouts, prints field offsets and padding for every struct so both sides can be compared directly. - Whether the ABI changed between library versions:
abidiffcompares two builds and reports layout and signature changes that a source diff will not show. - What a foreign call costs: benchmark a no-op call across the boundary against a no-op native call. In Go with cgo, and in any language with a lock to release, the ratio is instructive.
- Ownership mistakes: run the whole program under AddressSanitizer or Valgrind. Boundary bugs are memory bugs and these find them.
Plausible wrong readings
Stated the way a confident engineer states them.
- "FFI is just calling a function." It is agreeing on a calling convention, a memory layout, an ownership discipline, an error protocol and a threading model, only the first of which the compiler checks.
- "If it compiles, the layout matches." Nothing checks that the header you compiled against describes the library you loaded. That mismatch is the classic ABI break and it produces silent corruption.
- "Garbage collection makes memory easier at the boundary." It makes it harder: the collector may move or free an object that foreign code holds, which is a problem manual languages simply do not have.
- "We can add embeddability later." Embeddability constrains the collector, the scheduler, signal handling and initialisation. Languages that did not design for it have generally not retrofitted it.
Misconceptions
The claim, and what is actually true.
unsafe; languages that do not mark it still have the same hole.Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
To call code written in another language, both sides must agree on how arguments are passed, how data is laid out in memory, who frees what, and what happens when something fails. C is the common vocabulary because everything speaks it, and speaking it means giving up several things your language would otherwise do to your data.
practical
Freeze the layout of every type that crosses with the explicit annotation, generate bindings rather than writing them, and run the whole thing under AddressSanitizer before believing it works. Convert errors at the boundary explicitly and never let an exception or panic unwind through foreign code. And measure the per-call cost before designing a fine-grained interface across the boundary, because in several runtimes it is far more than a function call.
advanced
The design question underneath all of this is what a language is willing to say about its own representations. Layout freedom, moving collection, unwinding and lightweight scheduling are all things a language does because nobody outside can see inside — and every one of them becomes a boundary problem the moment somebody can. The languages with the least painful interoperability are the ones that decided early which representations are public: C has no private ones, Rust made publication explicit with repr(C) and extern, and the managed languages made it a runtime negotiation with handles and pinning. The expensive position is the implicit one, where a representation was never declared public and everybody depends on it anyway, which is how an ABI break becomes an ecosystem event.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
[[calling-conventions]].If you were asked this in an interview
- What must two languages agree on before one can call the other in the same process, and which of those does the compiler check?
- Why is a garbage collector a problem at an FFI boundary, and what are the standard mitigations?
- When would you choose a subprocess and a pipe over an in-process FFI binding?
Connections
- Programming Languages & Runtime Internals — Pinning, handle tables, safepoint interaction with foreign frames, and embedding a runtimeThis lesson owns the boundary contract the compiler must emit and enforce. The runtime machinery that makes a collector and a scheduler survive a foreign call is theirs, and it is where the hardest of these bugs live.