FLOPS (Floating-Point Operations per Second)

FLOPS (floating-point operations per second) is the standard measure of a processor's raw arithmetic throughput — how many floating-point calculations it can perform each second. Because training and running neural networks is overwhelmingly floating-point matrix math, FLOPS is the headline number vendors quote when comparing AI accelerators. It scales with familiar prefixes: a GFLOP is 10⁹, a TFLOP 10¹², and a PFLOP 10¹⁵ operations per second. Like any headline number, it rewards careful reading more than face-value belief.

Precision changes the number

A FLOPS figure is meaningless without its precision. The same chip delivers very different throughput at FP64 (double precision), FP32 (single), FP16/BF16 (half), and FP8 — with each step down in precision roughly doubling the operations per second, because narrower numbers mean more of them per cycle and per byte moved. Modern AI accelerators advertise their largest numbers at the lowest precisions, since that is what neural networks actually use, and those low-precision peaks come from dedicated matrix hardware — Tensor Cores in NVIDIA's case — rather than the general-purpose cores. Some datasheets quietly quote figures that assume structured sparsity, doubling the number again under conditions real models may not meet. The rule for reading spec sheets: always compare like precision for like, and check the footnotes.

Why FLOPS is only half the story

Peak FLOPS describes the compute ceiling, but real workloads rarely touch it. Training runs typically achieve a fraction of peak — model-FLOPs utilization commonly lands in the 30–50% range — because the arithmetic units stall waiting for data. For inference, and especially the token-by-token generation pattern of chat, the binding constraint is usually memory bandwidth, not arithmetic: generating each token requires streaming essentially the whole model's weights out of memory for a comparatively small amount of math. The hardware spends its time moving numbers, not multiplying them. This is why a card's real-world tokens per second tracks its memory bandwidth far better than its TFLOPS rating, and why quantization — shrinking the weights so fewer bytes must move — speeds up generation even though it barely changes the FLOPS being performed.

FLOPS versus hashrate

Readers of this site know a sibling metric: hashrate. The comparison is instructive. A Bitcoin ASIC performs no floating-point work at all — SHA-256 is pure integer and bitwise logic, which is exactly why mining silicon and AI silicon diverged into different species. Both numbers measure raw operations per second; both are ceilings rather than promises; and both industries learned the same lesson, that the honest figure of merit is work per joule, not work per second. An efficiency-first reading — J/TH for miners, performance-per-watt for accelerators — is what separates an operator from a spec-sheet shopper in either world.

Using FLOPS when sizing local hardware

When choosing hardware for a local LLM, treat FLOPS as one leg of a tripod. VRAM capacity decides which models you can load at all; memory bandwidth decides how fast they generate; FLOPS decides how fast the compute-heavy phases run — prompt processing (chewing through a long context window), batch workloads, and fine-tuning. A card can be generously supplied with FLOPS and still feel slow in chat because its bandwidth is the bottleneck, or modest on paper and perfectly adequate for a quantized model that fits comfortably. Read all three numbers together, and weight them by what you will actually run: for a self-hoster, capacity and bandwidth are almost always the constraints that bind first.