FLOPS (Floating-Point Operations Per Second)

FLOPS stands for floating-point operations per second and is the standard unit for expressing the raw arithmetic throughput of a processor. Modern AI accelerators are rated in TFLOPS (10¹² operations per second) or, at cluster scale, PFLOPS (10¹⁵), almost always measured on the dense matrix multiplication (GEMM) that dominates neural-network workloads. The figure is to AI hardware what hashrate is to a miner: the headline capacity number — and, like hashrate, it means little until you know the conditions under which it was measured.

FLOPS versus FLOPs: the letter that changes everything

The case of the final letter carries real meaning. FLOPS (capital S) is a rate — operations per second — describing how fast hardware runs. FLOPs (lowercase s) is a count — the total number of floating-point operations a workload requires, with no time dimension. Training a model requires some fixed number of FLOPs; your hardware delivers some number of FLOPS; dividing one by the other (adjusted for utilization) gives wall-clock time. Confusing the two is one of the most common errors in capacity planning, and the distinction is worth internalizing before reading any hardware review or research paper.

Peak versus sustained, and the precision asterisk

Vendors quote peak theoretical FLOPS, derived by multiplying clock speed, unit count, and operations per cycle — a number the silicon can touch only when every execution unit is fed every cycle. Real workloads rarely get close: sustained throughput of 30–60% of peak is typical, throttled by memory traffic, communication, and software overhead. Precision matters just as much. A modern accelerator's FP16 or BF16 rate is many times its FP64 rate, and FP8 or INT8 figures — often quoted with sparsity assumptions layered on top — can be higher still. Dedicated matrix hardware such as the tensor core is what delivers these elevated low-precision rates. A FLOPS number without its precision attached is marketing, not specification.

Why FLOPS alone won't predict your token speed

For an operator self-hosting language models, the uncomfortable truth is that FLOPS is often the wrong number to shop by. Token-by-token generation performs relatively little arithmetic per byte of model weights read — a low arithmetic intensity workload — so decode speed on a single user's chat session is usually set by memory bandwidth, not by compute. A GPU with modest TFLOPS but fast memory can out-generate a compute monster that starves. FLOPS starts mattering when arithmetic intensity rises: prompt processing, batched serving, training, and fine-tuning are compute-hungry phases where the rating genuinely predicts performance. Knowing which regime your workload lives in is the whole game.

Judging hardware like a craftsman

The convention for counting also trips people up: a fused multiply-accumulate is universally counted as two floating-point operations, which is why peak figures are always even multiples of unit count and clock. And comparisons across vendors need care with sparsity — some headline numbers assume structured-sparse weights that double the effective rate, an assumption real dense models do not meet. Read the footnotes on a spec sheet the way you would read a miner's efficiency claim: conditions attached.

The right mental tool is the roofline model, which plots achievable performance against arithmetic intensity and shows at a glance whether a workload will be limited by the memory ceiling or the compute ceiling. Actual efficiency against theoretical peak is tracked as Model FLOPs Utilization. The sovereign approach to hardware — the same one that applies to evaluating a miner's joules per terahash — is to distrust headline numbers, identify your bottleneck, and measure on your own workload. FLOPS is a real and useful figure; it is simply one axis of a two-axis problem.