FP16 / bfloat16 / FP32 Precision Converter

Understanding the FP16, bfloat16 and FP32 converter — concept and principle

ML and graphics hardware often store numbers in 16 bits to save memory and bandwidth. The two common 16-bit formats split their bits differently: IEEE half precision (FP16) keeps more fraction bits, while bfloat16 (bf16) keeps the full FP32 exponent range. This converter rounds a decimal number to the nearest FP32, bfloat16 and FP16 value, shows the FP16 bit pattern in hex and lists each format's machine epsilon.

How it is calculated

• FP32: 1 sign bit, 8 exponent bits (bias 127), 23 fraction bits. Value = (−1)ˢ × 1.f × 2^(e − 127).
• bfloat16: the top 16 bits of FP32 (1 sign, 8 exponent, 7 fraction). The lower 16 bits are rounded to nearest, ties to even.
• FP16: 1 sign, 5 exponent bits (bias 15), 10 fraction bits. Magnitudes of 65520 or more become infinity; numbers below 2⁻¹⁴ are stored as subnormals in steps of 2⁻²⁴.
• Machine epsilon: the gap between 1 and the next larger value: 2⁻¹⁰ ≈ 9.77 × 10⁻⁴ (FP16), 2⁻⁷ ≈ 7.81 × 10⁻³ (bf16), 2⁻²³ ≈ 1.19 × 10⁻⁷ (FP32).

The principle

A floating-point number is scientific notation in base 2: the exponent sets the scale, the fraction sets the precision. With p fraction bits, numbers between 2ᵉ and 2ᵉ⁺¹ are spaced 2^(e − p) apart, so the relative rounding error is at most half the epsilon. FP16 spends more bits on the fraction and has about 3 significant decimal digits but a maximum of 65504. bfloat16 has only about 2–3 digits but reaches about 3.4 × 10³⁸ like FP32, which helps neural-network training avoid overflow.

Worked example

1. Input: 3.14159. It lies between 2 and 4, so the exponent is 1.
2. FP32 keeps 23 fraction bits, pattern 0x40490FD0, which is 3.141590118.
3. bfloat16 keeps the upper half 0x4049; the dropped half 0x0FD0 is below 0x8000, so it rounds down to 2 × (1 + 73/128) = 3.140625.
4. FP16: fraction = round((3.14159 ÷ 2 − 1) × 1024) = round(584.29) = 584 = 0x248, biased exponent 1 + 15 = 16, giving bits 0 10000 1001001000 = 0x4248.
5. Its value is 2 × (1 + 584/1024) = 3.140625. Since 584/1024 = 73/128, both 16-bit formats give the same value here.

How to read the result

The difference from your input is the rounding error: 3.14159 − 3.140625 ≈ 0.000965, a relative error of about 3 × 10⁻⁴. Near 3, FP16 values are 2⁻⁹ ≈ 0.00195 apart and bf16 values 2⁻⁶ ≈ 0.0156 apart, so other inputs in this range will show larger bf16 errors.

Common mistakes

• Assuming bf16 is more precise than FP16 because it has a wider range; it has fewer fraction bits.
• Storing values above 65504 in FP16 and getting infinity.
• Adding a small number to a large one in 16 bits: 2048 + 1 in FP16 stays 2048.

Frequently asked questions

What is the largest FP16 number?

65504; the smallest positive normal value is 2⁻¹⁴ ≈ 6.10 × 10⁻⁵.

Why do GPUs offer bfloat16?

It halves memory like FP16 but keeps the FP32 exponent range, so training needs less loss scaling.

Is FP16 the same as half precision?

Yes, it is the IEEE 754 binary16 format.

Sources: IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic · Intel, BFLOAT16 – Hardware Numerics Definition, 2018