TOPS (Tera Operations Per Second)

TOPS (Tera Operations Per Second) is the headline throughput rating stamped on neural processing units (NPUs), edge AI accelerators, and the AI engines built into modern systems-on-chip. It counts how many trillions of integer operations the silicon can perform every second, where one "operation" is typically a single multiply-accumulate (MAC) step inside a matrix multiplication. Almost every published TOPS figure is measured at INT8 precision, because 8-bit integer math packs the most operations through a fixed amount of hardware — and because it makes the number on the box bigger.

How TOPS is calculated

Peak TOPS comes straight from the hardware design: roughly 2 × (number of MAC units) × (clock frequency), divided by a trillion. The factor of two appears because a multiply-accumulate is counted as two operations — one multiply, one add. This is a theoretical ceiling under ideal conditions: every MAC unit busy every cycle, perfect data feeding, no memory stalls. Real workloads never hit it. Some vendors inflate further by quoting structured-sparsity figures, where the hardware skips zeroed weights — a legitimate feature, but one that can double the advertised number for models that may not benefit at all. Miners will recognize the pattern: it is the same relationship as nameplate hashrate versus what a machine actually delivers on the wall, except the AI industry's gap between datasheet and reality is usually wider.

Why precision matters

A TOPS rating is meaningless without its precision. The same chip rated at 40 TOPS for INT8 might deliver roughly half that for FP16, because lower-bit formats let more operations flow through the same datapath per cycle. When sizing hardware, compare TOPS at the precision you actually intend to run — which for local LLM work usually means whatever your quantization scheme demands. See FP16/INT8 precision for how these formats trade accuracy against throughput.

The memory-bandwidth reality check

Here is the trap that catches most first-time buyers of "AI PCs" and edge boards: LLM text generation is rarely limited by TOPS at all. Producing each token requires streaming essentially the entire model through the compute units, so the binding constraint is memory bandwidth and capacity, not arithmetic. A modest GPU with fast, ample VRAM will often outrun a high-TOPS NPU strapped to slow shared memory for local LLM inference. TOPS matters most for compute-dense workloads — vision models, prompt prefill, convolution-heavy pipelines — while sustained token generation lives and dies on bytes per second. For sovereign Bitcoiners speccing an air-gapped inference rig, the buying checklist is therefore: memory capacity first (does the model fit?), memory bandwidth second (how fast can it stream?), and TOPS third (can the chip keep up with the memory?). A spec sheet that shouts TOPS while whispering bandwidth is telling you where to look.

Reading a spec sheet like a miner

Miners already own the mental model this market needs. Nobody buys an ASIC on terahash alone; the number that matters is efficiency — joules per terahash — and the same discipline applies here, where TOPS per watt is the honest figure of merit for edge silicon. Sustained versus peak is the second familiar trap: a fanless board that bursts to its rated TOPS and then throttles is the AI equivalent of a miner spec'd at the wall in a walk-in freezer. Apply the checklist: demand the precision alongside the TOPS figure, ask whether sparsity inflated it, find the memory bandwidth in the fine print, and estimate the sustained number under your real thermal envelope. Vendors count on buyers comparing single headline integers; a buyer who cross-examines the datasheet the way they would a hashboard listing is very hard to disappoint.