Floating Point: Trading Precision for Range
A float is scientific notation in binary: a sign, an exponent that slides the point, and a fraction. That design buys an enormous range from a fixed number of bits, and it pays for it with precision that varies depending on how large the value is.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Three fields, and a sliding point
Fixed-point representation splits the bits at a fixed position: so many for the integer part, so many for the fraction. It is simple, exact within its range, and useless for scientific work because that range is tiny. Floating point instead stores where the point goes, which is what the name means.
The layout is a sign bit, an exponent field and a fraction field. The exponent is stored biased — an offset is added so the stored value is always non-negative, avoiding a second sign — and the fraction has an implicit leading 1 that is not stored, because a normalised binary number always starts with one. That trick buys a free extra bit of precision.
The consequence to internalise is in the last column below. Precision is a fixed number of significant bits, not a fixed absolute step. Single precision gives roughly seven decimal digits and double roughly fifteen — at any magnitude. So near 1.0 the neighbouring representable values are extremely close together, and near 10^15 they are far apart, which is why the same computation can be exact in one range and hopeless in another.
| Format | Sign | Exponent | Fraction | Approximate decimal digits |
|---|---|---|---|---|
| binary16 (half) | 1 bit | 5 bits | 10 bits | about 3 |
binary32 (single, float) | 1 bit | 8 bits | 23 bits | about 7 |
binary64 (double, double) | 1 bit | 11 bits | 52 bits | about 15 |
| bfloat16 | 1 bit | 8 bits | 7 bits | about 2 |
Reading an actual value out of the bits
Working through one example makes the structure concrete. The pattern 0x3F800000 is exactly 1.0 in single precision, and it is worth being able to see why: sign 0, exponent field 127 which after removing the bias of 127 is 0, and a fraction of zero which with the implicit leading one gives 1.0 × 2^0.
The exponent bias exists so comparisons work. Because the exponent is stored as a non-negative number and sits above the fraction, two positive floats can be compared as if they were plain integers — the bit patterns order the same way the values do. That is not an accident; it is a deliberate property of the format that makes sorting and comparison hardware simpler.
The extremes of the exponent field are reserved for special values rather than numbers: all zeros means zero or a subnormal (values closer to zero than normalisation allows, at reduced precision), and all ones means infinity or NaN. NaN is the one to know, because it compares unequal to everything including itself, which is exactly why a sort can behave bizarrely if NaN enters the data.
0x3F800000 -> 0 01111111 00000000000000000000000
sign exponent fraction
0 127 0
exponent - bias = 127 - 127 = 0
significand = 1 + 0 = 1.0 (leading 1 is implicit)
value = 1.0 x 2^0 = 1.0
0x40490FDB -> 0 10000000 10010010000111111011011
exponent - bias = 128 - 127 = 1
significand ~ 1.5707963...
value ~ 3.14159274 (approximately pi, not exactly)
Reserved exponent patterns:
exponent all 0s -> zero (fraction 0) or subnormal (fraction != 0)
exponent all 1s -> infinity (fraction 0) or NaN (fraction != 0)
NaN != NaN. This is required by the standard, and it is why a
comparison-based sort can misbehave if NaN reaches the data.What the trade actually buys, and what it costs
The scale below shows the gap between adjacent representable doubles at different magnitudes. Near 1.0 the spacing is around 2×10^−16, which is far finer than most measurements. By 10^16 the spacing has grown to about 2 — meaning integers above roughly 2^53 cannot all be represented, and adding 1 to such a value can genuinely change nothing.
This is why money is not stored in floating point. Not because floats are "inaccurate" in some vague sense, but because the representable values are powers of two scaled, and 0.01 is not one of them — the same reason 1/3 has no exact decimal expansion. Currency uses integer minor units or a decimal type precisely to avoid it.
It is also why the JavaScript Number.MAX_SAFE_INTEGER limit at 2^53−1 exists: beyond it, consecutive integers stop being distinguishable in a double. Any system passing large identifiers through JSON meets this, usually as an ID that silently changes value in transit.
Key points
- A float is binary scientific notation: sign, biased exponent and fraction with an implicit leading one.
- Precision is a fixed count of significant bits, so absolute spacing between representable values grows with magnitude.
- The exponent bias makes positive floats comparable as if they were integers, which simplifies the hardware.
- Reserved exponent patterns encode zero, subnormals, infinity and NaN — and NaN compares unequal to itself.
- Beyond 2^53 a double cannot represent every integer, which is the origin of the JavaScript safe-integer limit.
Floating-Point Explorer
Change an input and watch which number moves — and which one refuses to.
The stored value is not the decimal you typed. A finite binary fraction cannot represent most decimal fractions exactly, in the same way base ten cannot write 1/3 exactly. Every arithmetic operation starts from this approximation, which is why accumulated error grows and why equality comparison on floats is a trap.
Try 0.1, 0.5, 0.3, 1e300, and 0.1 + 0.2 — the classic result comes from exactly this representation.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Value → normalisation: the number is scaled so its significand lies in [1, 2), and the leading one is then dropped as implicit.
- 2Exponent → bias: the scale factor is stored with an offset added so it is never negative, keeping ordering intact.
- 3Fields → register: sign, exponent and fraction are packed into a single 32- or 64-bit pattern.
- 4FPU → result: arithmetic aligns exponents, operates on significands, renormalises and rounds to the nearest representable value.
- 5Rounding → error: every operation may introduce a small error, and those errors accumulate across a computation.
- • Concluding floats are "inaccurate" in general; they are exactly specified and deterministic, just not able to represent every decimal.
- • Assuming a small tolerance works at every magnitude — an absolute epsilon that suits values near 1.0 is meaningless near 10^12.
- • Expecting floating-point addition to be associative, which silently breaks parallel reductions that regroup the operations.
- • Treating a double as safe for arbitrarily large integer identifiers.
Consequences, controls and cost
- • Most decimal fractions, including 0.1 and 0.01, have no exact binary representation.
- • Adding a sufficiently small value to a large one can leave the large one unchanged.
- • Floating-point addition is not associative, so changing the summation order changes the result.
- • NaN propagates through arithmetic and breaks comparison-based logic including sorting.
- • Use integers or a decimal type for money and any quantity where exactness at a fixed scale is required.
- • Compare with a tolerance appropriate to the magnitude involved, rather than testing equality.
- • Sum from smallest to largest, or use a compensated summation algorithm, when accumulating many values.
- • Check for NaN explicitly at the point data enters a computation rather than discovering it downstream.
- • Print values in hex to see the exact stored pattern rather than a rounded decimal rendering.
- • Compute the same sum in different orders and compare; a difference proves accumulated rounding is significant for your data.
- • Use a higher-precision type as a reference to estimate the error in the working precision.
- • Double precision costs twice the memory and bandwidth of single, which matters in large arrays and on GPUs.
- • Decimal types are exact at a fixed scale but slower and usually unsupported by vector hardware.
- • Reduced precision formats save enormous memory and bandwidth in machine learning, at accuracy that must be validated per model — see [[model-memory-and-quantization]].
Scope
§224 — what these claims are specific to.
- GENERALIEEE-754 binary32 and binary64 are implemented essentially universally, so bit patterns and rounding behaviour are portable in a way little else in this domain is.
- PLATFORM-SPECIFICExtended intermediate precision, fused multiply-add contraction and flush-to-zero handling of subnormals vary by compiler flags and hardware, so identical source can produce slightly different results across targets.