NVLink

NVLink is NVIDIA's proprietary high-speed interconnect for direct GPU-to-GPU communication. When a model is too large to fit on a single accelerator, the GPUs holding its pieces must constantly exchange activations, gradients, and weights; NVLink provides a dedicated, low-latency path between them that is many times wider than the standard PCIe bus, keeping multi-GPU systems computing instead of waiting on data movement. It is one of the quiet foundations of the large-model era: without an interconnect in this class, the biggest neural networks could not be trained or served at practical speed at all.

Bandwidth across generations

NVLink has scaled aggressively: the first generation offered 160 GB/s of bidirectional bandwidth on the Tesla P100, the A100's third generation reached 600 GB/s, the H100's fourth generation hit 900 GB/s across 18 links, and the fifth generation in Blackwell-class GPUs delivers 1.8 TB/s per GPU. For comparison, a PCIe Gen 5 x16 slot tops out near 63 GB/s bidirectional — an order of magnitude less. Paired with NVSwitch fabric, NVLink extends beyond point-to-point pairs into all-to-all topologies where every GPU in a rack-scale system can talk to every other at full link speed, which is what lets a tray of accelerators behave like one enormous device with pooled memory.

Why the interconnect is the bottleneck

The reason this matters comes down to how big models are split. Tensor parallelism carves individual layers across GPUs, forcing synchronization at every step; pipeline and data parallelism ship activations and gradients between devices continuously. In all cases, compute units stall whenever the interconnect cannot feed them — the same class of problem, one level up, as a single GPU stalling on memory bandwidth. Raw FLOPS figures assume the data arrives on time; the interconnect decides whether it does. NVLink exists because at multi-GPU scale, moving numbers is often harder than multiplying them.

The centralization angle

NVLink is also a vendor-specific technology, and that has consequences beyond the spec sheet. Building a cluster around it ties you to NVIDIA's hardware, drivers, and ecosystem, reinforcing the same lock-in dynamics that surround CUDA — and full-fat NVLink connectivity ships on datacenter-class parts, not consumer cards, placing large-scale model training firmly in the territory of well-capitalized organizations. Open alternatives and industry consortium efforts exist to standardize accelerator interconnects, but the highest-bandwidth path today runs through one company. Bitcoiners will recognize the pattern: whoever controls the chokepoint infrastructure shapes who gets to participate. It is a useful lens for understanding why frontier AI is centralized and why sovereign AI efforts focus on what individuals can run without permission.

What it means for the self-hoster

Here is the liberating part: for most people running a local LLM, NVLink is simply irrelevant. A single card serving an open-weights model — especially with quantization shrinking the weights to fit in VRAM — involves no GPU-to-GPU traffic at all, and inference at home is bounded by memory bandwidth and model size, not interconnect. Even modest multi-GPU home setups typically split models in ways where PCIe is a tolerable, if imperfect, link. NVLink becomes decisive only when scaling out serious multi-GPU training or high-throughput serving in distributed compute territory. Knowing where that line sits keeps you from over-buying hardware for a problem you do not have — and from underestimating what your single card can do.