Run Mistral Locally in Canada: A Practical Setup Guide (2026)

Why run a local LLM in Canada at all?

Most commercial AI APIs route your prompts through servers in the United States. The US CLOUD Act allows US authorities to compel American cloud providers to disclose data stored anywhere in the world, including data from Canadian clients. Quebec Law 25 (in full force since September 2024) restricts the transfer of personal information outside Quebec without adequate protective measures — and the Commission d’accès à l’information has clarified that uploading documents to cloud AI interfaces can constitute such a transfer.

Running an open-weight model locally removes that exposure entirely. Your data stays on hardware you control, in the jurisdiction you choose. There are no usage logs, no model-training opt-out forms to file, and no per-token costs that compound as your usage scales.

For a broader look at the sovereignty argument, see our Canadian digital sovereignty primer and our full comparison of cloud vs. local AI total cost of ownership.

The Mistral open-weight family — what is available as of 2026

Mistral AI (Paris, France) has been releasing open-weight models since 2023, building a reputation for strong per-parameter efficiency. As of early 2026 the publicly downloadable lineup includes several tiers. Verify current availability and licence details at mistral.ai and on the Hugging Face hub — the model landscape changes quickly.

Model	Size	Architecture	Licence	Best for
Ministral 3B	3B params	Dense transformer	Apache 2.0	Edge devices, very low VRAM, fast drafts
Ministral 8B	8B params	Dense transformer	Apache 2.0	General assistant on consumer GPU
Ministral 14B	14B params	Dense transformer	Apache 2.0	Stronger reasoning, coding, document analysis
Mistral Nemo 12B	12B params	Dense transformer	Apache 2.0	128k context, built with NVIDIA, strong instruction follow
Mistral Small 3 (24B)	24B params	Dense transformer	Apache 2.0	Flagship “small” model; multimodal, function calling
Devstral (24B)	24B params	Dense transformer	Apache 2.0	Agentic coding workflows, software engineering tasks
Codestral 22B	22B params	Dense transformer	Non-commercial (MNPL)*	Code completion across 80+ languages
Mistral Large 3	675B total / 41B active	Sparse MoE	Apache 2.0	Frontier-scale tasks; requires serious multi-GPU hardware

* Codestral 22B was released under the Mistral Non-Production Licence (MNPL), which restricts commercial use. Verify the current licence before deploying it in a production environment. All other models in the table above carry Apache 2.0, which permits commercial self-hosting. Voxtral (Mistral’s TTS model) is CC BY-NC 4.0 and is not listed here as it targets a different use case.

The Ministral 3 family (3B, 8B, 14B) and Mistral Large 3 were released in December 2025. They also include base, instruct, and reasoning variants — check Hugging Face for the specific checkpoint you need.

Credit where it is due: the open-weight community — including the researchers at Mistral AI, NVIDIA (Mistral Nemo was a joint release), the Ollama project, bartowski and TheBloke (community GGUF quantizers on Hugging Face), and the broader llama.cpp and vLLM ecosystems — makes all of this possible. D-Central is building on their work, not competing with it.

For a side-by-side look at how Mistral compares to Llama, Qwen, and Gemma, see our open-weight AI Canada comparison.

Hardware you need — VRAM reality check

The figures below are community-sourced estimates for common quantization levels. Actual VRAM usage depends on context window size, batch size, and your specific inference backend. Treat these as planning numbers, not guarantees — verify for your exact setup using our local LLM VRAM calculator and at ollama.com/library.

Model	Q4_K_M VRAM (est.)	FP16 VRAM (est.)	Minimum GPU (consumer)	Comfortable GPU
Ministral 3B	~2–3 GB	~6 GB	RTX 3060 (12 GB) or even 8 GB cards	Any RTX 30/40/50 series
Ministral 8B	~4–5 GB	~16 GB	RTX 3060 12 GB, RX 6700 XT 12 GB	RTX 3080 10 GB / RTX 4060 Ti 16 GB
Mistral Nemo 12B	~7–8 GB	~24 GB	RTX 3080 10 GB (tight at full context)	RTX 3080 Ti 12 GB / RTX 4070 12 GB
Ministral 14B	~8–10 GB	~28 GB	RTX 3080 Ti 12 GB	RTX 4070 Ti 16 GB / RTX 4080 16 GB
Mistral Small 3 (24B)	~13–14 GB	~48 GB	RTX 3090 24 GB / RTX 4090 24 GB	RTX 4090 / A5000 24 GB
Devstral (24B)	~13–14 GB	~48 GB	RTX 4090 24 GB	RTX 4090 / dual RTX 3090
Codestral 22B	~13 GB	~44 GB	RTX 4090 24 GB	RTX 4090 / A5000 24 GB
Mistral Large 3 (675B)	~400 GB+ (Q4 est.)	Not feasible on consumer hardware	Multi-GPU server or DGX-class system	8× H100 / 8× A100 80 GB

No GPU? Models up to ~8B can also run on CPU + RAM (expect slower throughput — 2–8 tokens/second on a modern desktop). Apple Silicon Macs can use their unified memory efficiently; an M3 Max with 64 GB handles Mistral Small 24B Q4 acceptably. AMD ROCm support through Ollama is available but support varies by GPU generation — check the Ollama GitHub for confirmed ROCm cards.

For a deeper explanation of what Q4, Q8, and FP16 mean and when each matters, see our AI quantization guide (INT4, INT8, FP16).

Step-by-step: run Mistral locally with Ollama

Ollama is the most accessible path for most users — a single binary that handles model download, GGUF quantization, and a local HTTP API. Credit goes to the Ollama project and contributors for making local LLM deployment this approachable.

1. Install Ollama

macOS / Linux:

curl -fsSL https://ollama.com/install.sh | sh

Windows: Download the installer from ollama.com/download. Requires Windows 10 or later; NVIDIA GPU support is automatic if your drivers are current.

After installation, Ollama runs a background service that listens on http://localhost:11434.

2. Pull and run a Mistral model

Choose the model that fits your hardware from the table above, then run:

# Ministral 8B — general assistant, fits in 8 GB VRAM at Q4
ollama run ministral:8b

# Mistral Nemo 12B — 128k context, good balance of size and quality
ollama run mistral-nemo:12b

# Ministral 14B — stronger reasoning, good for document work
ollama run ministral:14b

# Mistral Small 3 (24B) — flagship small model, needs 24 GB VRAM at Q4
ollama run mistral-small:24b

# Devstral 24B — for coding and agentic workflows
ollama run devstral:24b

The first run downloads the model (several GB; expect a few minutes on a typical Canadian residential connection). Subsequent runs load from disk cache.

Important: Model tags on Ollama evolve. Always check ollama.com/library for the current tag name before running — tags listed here were accurate as of mid-2026 but may change.

3. List downloaded models and switch between them

# List what you have locally
ollama list

# Remove a model to free disk space
ollama rm mistral-nemo:12b

# Pull a specific quantization (example: Q5_K_M for better quality on 12B)
ollama pull mistral-nemo:12b-instruct-2407-q5_K_M

4. Use the local API in your applications

Ollama exposes an OpenAI-compatible REST endpoint. Any application that supports the OpenAI API can point to your local Ollama instance:

curl http://localhost:11434/api/chat 
  -H "Content-Type: application/json" 
  -d '{
    "model": "ministral:8b",
    "messages": [
      {"role": "user", "content": "Summarize this contract clause for me."}
    ]
  }'

For a streaming response add "stream": true. This same endpoint works with LangChain, LlamaIndex, Open WebUI, and dozens of other tools.

5. Add a web interface (optional)

Open WebUI gives you a ChatGPT-style browser interface connected to your local Ollama. Install it with Docker:

docker run -d -p 3000:80 
  --add-host=host.docker.internal:host-gateway 
  -v open-webui:/app/backend/data 
  --name open-webui 
  ghcr.io/open-webui/open-webui:main

Then open http://localhost:3000 in your browser. All inference still happens on your local machine.

Beyond Ollama — other local inference options

Ollama is the right starting point for most users, but it is not the only option:

• LM Studio — graphical interface for macOS and Windows; good for non-technical team members; downloads GGUF models from Hugging Face with a search interface.
• llama.cpp — the underlying engine Ollama uses; gives you fine-grained control over quantization, context length, and GPU layer offloading; command-line only.
• vLLM — production-grade inference server designed for high-throughput deployments; requires Linux + NVIDIA GPU; supports tensor parallelism across multiple GPUs (useful for Mistral Large 3).
• Jan — open-source desktop app similar to LM Studio; fully offline; good for privacy-focused users.

For a full comparison of Ollama vs. vLLM vs. llama.cpp (performance, use cases, memory overhead), see our dedicated guide: Ollama vs. vLLM vs. llama.cpp.

Mistral’s EU-sovereignty roots — relevant for Canadian users

Mistral AI is a French company, founded in 2023 by researchers who previously worked at DeepMind and Meta. It processes API traffic in EU data centres and operates under GDPR. In January 2026, France’s Ministry of the Armed Forces awarded Mistral a framework agreement to run AI across all branches of the French military — on French-controlled infrastructure.

This is relevant context for Canadian businesses, but it is not the same as Canadian sovereignty. When you self-host Mistral weights on Canadian infrastructure, you gain something beyond what even a European cloud provider can offer: the model runs entirely within your perimeter. No party — French, American, or otherwise — has access to your prompts or outputs.

The Apache 2.0 licence on Mistral’s models reinforces this. You can inspect the model card, run the weights offline, fine-tune on your own data, and redistribute internally — without reporting back to Mistral AI or agreeing to a usage policy that could change.

Quebec Law 25 and local LLMs

Quebec Law 25 (Act to modernize legislative provisions as regards the protection of personal information) has been fully in force since September 2024. Section 17 restricts the transfer of personal information outside Quebec without adequate protection. The Commission d’accès à l’information has clarified that uploading documents to cloud AI interfaces can constitute such a transfer, triggering consent, PIA, and documentation requirements.

Running Mistral locally means the personal information in your prompts never leaves your infrastructure. That does not automatically make you Law 25 compliant — you still need proper access controls, data retention policies, and governance — but it removes the cross-border transfer question entirely. For a full breakdown, see our guide on Quebec Law 25 and on-premise LLMs.

For a broader look at the Canadian regulatory landscape and local AI, see our local LLM Canada hub page.

Where the cloud still wins

This guide is not an argument that local always beats cloud. There are real trade-offs:

• Frontier model quality. GPT-4o, Claude Sonnet, and Gemini Ultra still outperform every publicly available open-weight model on complex multi-step reasoning tasks. If your use case demands the absolute frontier, cloud APIs remain the practical choice — just manage your data accordingly.
• Setup and maintenance cost. Running a local model requires hardware investment, driver management, and ongoing maintenance. For a team that needs an AI assistant tomorrow morning, a cloud API is faster.
• Scale-out. Serving hundreds of concurrent users efficiently requires serious infrastructure (vLLM, Kubernetes, load balancers). Cloud providers have already built that. Replicating it locally has a real cost.
• Multimodal and specialised APIs. Speech, image generation, and real-time voice are still dominated by proprietary APIs; the open-weight equivalents are catching up but are not yet peer.

The honest answer for most Canadian businesses: a hybrid approach makes sense. Use self-hosted Mistral for workflows involving sensitive client data, internal documents, or regulated information. Use cloud APIs for tasks where quality outweighs sovereignty concerns and the data is not sensitive. See our cloud vs. local AI TCO comparison for a cost model to guide that decision.

Standing on the shoulders of giants

This page would not exist without the work of others. We credit:

• Mistral AI — for pioneering commercial-grade open-weight models and releasing them under Apache 2.0, making self-hosting a real option for businesses that cannot afford to give their data to a cloud.
• The Ollama project — for making local model deployment genuinely accessible to non-ML engineers.
• The llama.cpp project (Georgi Gerganov and contributors) — for the quantization and inference engine that underpins most consumer-grade local LLM tooling.
• The GGUF quantization community (bartowski, TheBloke, and others on Hugging Face) — for producing and maintaining the quantized model variants that make running large models on consumer hardware practical.
• The Open WebUI project — for the open-source chat interface that makes local models accessible to non-technical team members.

D-Central’s role here is integration and deployment guidance — helping Canadian businesses and Bitcoin infrastructure operators benefit from work the open-source community has already done. For air-gapped environments (no internet connection during inference), see our companion guide on air-gapped AI coding in Canada. For a full list of open-weight models available for self-hosting, including non-Mistral options, see open-weight AI Canada comparison.

Frequently asked questions

Is Mistral really free to use commercially?

Most Mistral models — including the entire Ministral 3 family, Mistral Nemo, Mistral Small 3, Devstral, and Mistral Large 3 — are released under the Apache 2.0 licence, which explicitly permits commercial use, including self-hosting in a production environment. The exception is Codestral 22B, which carries the Mistral Non-Production Licence (MNPL) — verify the current licence at mistral.ai before using it commercially. Voxtral (text-to-speech) is CC BY-NC 4.0 (non-commercial).

What is the minimum GPU to run Mistral locally?

Ministral 3B runs on virtually any GPU with 4+ GB VRAM at Q4 quantization. Ministral 8B fits comfortably in 6–8 GB at Q4_K_M, making it practical on an RTX 3060 12 GB or similar. Mistral Small 3 (24B) needs approximately 13–14 GB VRAM at Q4_K_M — an RTX 3090 or RTX 4090 is the typical consumer choice. Mistral Large 3 (675B total parameters) is not consumer hardware territory; it requires multi-GPU server infrastructure. All figures are estimates — verify with our VRAM calculator.

Does Ollama support AMD GPUs?

Yes, Ollama includes ROCm support for AMD GPUs on Linux. Support varies by GPU generation and driver version. Check the official Ollama documentation and GitHub issues for your specific card before purchasing hardware for this purpose. NVIDIA CUDA support is more broadly tested and documented.

Can I run Mistral on a Mac?

Yes. Ollama supports Apple Silicon (M1, M2, M3, M4 series) natively. Apple Silicon’s unified memory architecture means VRAM and system RAM are shared — a Mac with 64 GB unified memory can load Mistral Small 3 24B at Q4_K_M with headroom for an active macOS session. Performance is typically 10–25 tokens/second depending on chip generation and model size.

Does self-hosting Mistral make me Quebec Law 25 compliant?

Self-hosting eliminates the cross-border data transfer question that makes cloud AI risky under Law 25 Section 17. But compliance is not automatic — you still need proper consent mechanisms, privacy impact assessments for automated decision-making, access controls, and data retention policies. Self-hosting is a necessary step for many Law 25 use cases; it is not a sufficient one on its own. See our full Law 25 on-premise LLM guide for what else is required.

What is the difference between Ministral and Mistral?

“Ministral” is the name Mistral AI gave to the small-end models in their December 2025 Mistral 3 release: the 3B, 8B, and 14B variants. “Mistral” (without the “ini”) refers to the broader brand and to the older Mistral 7B model (released 2023). The Ministral 3 family uses the same Apache 2.0 licence as the earlier Mistral models, but features updated architecture and reasoning capabilities.

How is Mistral different from DeepSeek for local use?

Both Mistral and DeepSeek release Apache 2.0 open-weight models that can be self-hosted. The key differences are origin (Mistral is French/EU, DeepSeek is Chinese), architecture choices (DeepSeek R1/V3 use MoE at scale; Mistral mixes dense and MoE depending on the model), and community ecosystem (Mistral has deeper integration with European enterprise tooling). For Canadian sovereignty purposes, both can be run entirely on-premise — the choice often comes down to benchmark performance on your specific task. See our run DeepSeek locally Canada guide for a parallel setup walkthrough.

Where can I get help setting up Mistral for my Canadian business?

D-Central offers AI sovereignty consulting for Canadian businesses, with particular focus on Quebec Law 25 compliance architectures, hardware selection, and private deployment of open-weight models. See our AI sovereignty consulting page for service details, or visit our local LLM Canada hub for self-serve resources.