Definition
Arithmetic intensity is the ratio of arithmetic work to data movement, usually expressed as floating-point operations per byte transferred (FLOPs/byte). It is the single most useful number for predicting how a workload will perform on a given accelerator, because it tells you whether the bottleneck will be the compute units or the memory system.
Why the ratio decides everything
Every operation needs operands fetched from memory and results written back. If a kernel does many calculations per byte moved (high arithmetic intensity), the compute units stay busy and performance is limited by peak FLOPS. If it does little arithmetic per byte (low arithmetic intensity), the chip spends most of its time waiting on memory and performance is capped by bandwidth, no matter how many TFLOPS the spec sheet claims.
Examples in AI
A large GEMM with good tiling reuses each loaded value across an entire row or tile, giving high arithmetic intensity — this is why training utilizes accelerators well. By contrast, AI inference on a single request is often dominated by streaming huge weight matrices through once per token, giving low arithmetic intensity and making memory bandwidth, not FLOPS, the deciding factor. This is precisely why a sovereign Bitcoiner picking a local-inference machine should weigh memory bandwidth heavily.
Arithmetic intensity is the x-axis of the roofline chart. See the roofline model and the closely-related framing of operational intensity to put this number to work.
In Simple Terms
Arithmetic intensity is the ratio of arithmetic work to data movement, usually expressed as floating-point operations per byte transferred (FLOPs/byte). It is the single most…