Definition
Local LLM is a large language model deployed on hardware under your direct control — a personal computer, a server in your facility, or a purpose-built inference device — running inference without sending data to any external server. "Local" refers to the location of computation, not the origin of the model weights.
Also known as: on-device LLM, self-hosted LLM, offline AI, private AI.
How it works
A large language model is a neural network with billions of parameters trained to predict and generate text. Running one locally means loading those parameters into your own hardware's memory — VRAM for GPU inference, system RAM for CPU or NPU inference — and performing the forward pass without any network call. Open-weight models, whose parameters are released publicly by their developers, make this possible without a vendor relationship or ongoing API subscription. Quantization reduces model size by representing weights at lower numerical precision (typically 4-bit or 8-bit), enabling large models to run on consumer hardware that could not hold the full-precision version.
Why it matters for sovereignty
A local LLM cannot be switched off by a policy decision, a government order, or an API pricing change. Your queries and their results remain on your hardware. For operators building a full sovereign stack, a local LLM is the AI layer — the same philosophy that underlies running your own Bitcoin node rather than trusting a third-party service. D-Central's local LLM Canada page covers hardware selection and model options for Canadian deployments.
Related terms: Sovereign AI, On-Premise AI, Compute Sovereignty
In Simple Terms
Local LLM is a large language model deployed on hardware under your direct control — a personal computer, a server in your facility, or a…