Local LLM Model Database

An open-weight LLM (large language model) is an AI model whose trained weights (the numerical parameters that encode its knowledge) are publicly released and downloadable. Unlike closed models such as GPT-4 or Claude — which are only accessible via APIs owned by their developers — open-weight models can be run on your own hardware without any API call, subscription, or data transmission to a third-party server. “Open-weight” is distinct from “open source”: the weights are free to download, but the training code and training data may or may not be published. OLMo 2 (Allen AI) is currently the most fully open family, releasing weights, training data, training code, and intermediate checkpoints under Apache 2.0.

The US CLOUD (Clarifying Lawful Overseas Use of Data) Act of 2018 allows US authorities to compel US-headquartered technology companies to hand over data stored on servers anywhere in the world, including Canada. When Canadian organisations use AI APIs hosted by US companies (OpenAI, Anthropic, Google, Microsoft Azure), their prompts and generated outputs may be subject to this obligation. Running an open-weight model on your own Canadian-controlled hardware means no US company is in custody of your data, eliminating this exposure. This is particularly relevant for law firms, healthcare organisations, financial institutions, and government bodies handling sensitive information. Note: this is informational, not legal advice — consult a privacy lawyer for your specific regulatory context.

Quantisation reduces the precision (bit-width) used to store each model parameter, trading a small amount of quality for dramatically lower memory requirements. FP16 (16-bit floating point) is full precision — the most accurate but requires the most VRAM (approximately 2 GB per billion parameters). Q8 (8-bit integer quantisation) halves the memory to roughly 1 GB per billion parameter with minimal quality loss. Q4_K_M (4-bit quantisation with mixed-precision handling) further halves it to roughly 0.5–0.6 GB per billion parameter, with a small but measurable quality reduction on demanding tasks. For most chat, writing, and coding tasks Q4_K_M is indistinguishable from FP16 in practice. Ollama applies Q4_K_M by default when you run ollama pull <model> without a quantisation tag.

Ollama is an open-source tool (MIT license) that makes running open-weight LLMs on macOS, Linux, and Windows as simple as a single command. It handles model downloading, quantisation selection, GPU/CPU offloading, and exposes a local REST API compatible with the OpenAI API format. To run a model: install Ollama from ollama.com, then run ollama run llama3.1:8b (for example) in a terminal. The Ollama tag column in this dataset gives you the exact string to pass to ollama pull or ollama run. Ollama is not the only way to run these models — llama.cpp, LM Studio, vLLM, and text-generation-webui are popular alternatives.

The answer depends on your hardware and use case. For a 6–8 GB GPU: Phi-3.5 mini 3.8B (2.2 GB Q4, MIT, 128K context) or Mistral Nemo 12B (7.1 GB Q4, Apache 2.0, 128K context) are strong starting points. For a 24 GB GPU: Qwen2.5 32B or Qwen3 30B-A3B (MoE, inference speed of a 3B model). For a 70B-class model on multi-GPU or a 48 GB card: Llama 3.3 70B or DeepSeek-R1-Distill-Llama-70B for reasoning-heavy tasks. For maximum commercial licence freedom: Apache 2.0 models (Qwen3, Gemma 4, OLMo 2, Mistral AI models) carry the fewest restrictions. For a fully self-hosted Bitcoin mining / AI sovereignty setup, D-Central’s AI Sovereignty Consulting can help design the right stack.

A Mixture of Experts (MoE) model partitions its parameters into “experts” — sub-networks specialised for different inputs. For each token, a router selects only a small subset of experts to activate (e.g. 2 out of 8). This means the model can have many more total parameters than a standard dense model while using the same compute per token as a smaller one. The catch: all expert weights must be loaded into VRAM simultaneously, even though most are idle at any given moment. So Mixtral 8x7B (46.7B total params, 12.9B active) infers at the speed of a 13B model but requires ~26 GB VRAM at Q4. Similarly Qwen3 30B-A3B requires ~19 GB VRAM but infers as fast as a 3B model. MoE models are excellent for latency-sensitive deployments where you have sufficient VRAM.

No — Command R (35B) and Command R+ (104B) from Cohere are released under CC-BY-NC-4.0, which prohibits commercial use without a separate licence from Cohere. Both are marked with a ✗ in the “Commercial” column of this dataset. If you need a commercially licensed model in the 35B+ range, consider Qwen2.5 32B (Apache 2.0), Qwen3 32B (Apache 2.0), or Llama 3.1 70B (Llama Community License, commercial OK). Always read the full licence text before production deployment. This dataset is informational, not legal advice.

The dataset is manually curated and verified against Hugging Face model cards and the Ollama library. The current version covers models as of June 2026. The open-weight LLM space moves rapidly — new model families, updated versions, and licence changes appear frequently. We update this database when major new releases occur. The REST API at /wp-json/dc/v1/local-llm-models always reflects the current version. If you spot an error or omission, the dataset is available under CC BY 4.0 — cite D-Central and contribute corrections. Last verified: June 2026.