Which AI Models Fit on 128GB? DGX Spark & Strix Halo Model Fit Guide (2026)

Last updated: April 2026 · Tested configs: NVIDIA DGX Spark (GB10, 128GB LPDDR5X, CUDA 12.8, TensorRT-LLM / Ollama) & AMD Strix Halo (Ryzen AI Max+ 395, 128GB LPDDR5X, ROCm 6.3 / Vulkan, Ollama + llama.cpp) · Quantization: GGUF Q4_K_M baseline, with Q8_0 and FP16 reference. Context measured at 32K unless noted.

If you are shopping a personal AI supercomputer in 2026, the headline spec that matters is not TFLOPS ??? it is how much model you can hold in fast memory without swapping. Both the NVIDIA DGX Spark (GB10 Grace Blackwell) and the AMD Strix Halo (Ryzen AI Max+ 395) ship with 128GB of unified LPDDR5X. That one number decides whether a 70B model leaves you 60GB of headroom or whether gpt-oss-120B still fits with a usable context window. Discrete GPUs with 24GB VRAM cannot even load gpt-oss-120B at Q4; these two boxes can, because CPU and GPU share the same pool with no PCIe copy.

This guide gives you the straight answer for 24 current open-weight models ??? with real GGUF footprints at Q4, Q8 and FP16, the extra cost of a 32K context window, and a clear Fits / Does Not Fit verdict for 128GB. If you are comparing DGX Spark vs Strix Halo, read the speed section after the table: capacity is identical, latency and software stack are not.

128GB Unified Memory: What It Means

On a desktop with a 24GB RTX 4090, “VRAM” and “system RAM” are two separate pools. A model must fit entirely in VRAM to run fast; spill to system RAM over PCIe and tokens per second collapses. Unified memory collapses that distinction. On both DGX Spark and Strix Halo, the CPU and GPU address the same 128GB LPDDR5X ??? typically 256-bit, ~200???256 GB/s ??? with zero copy between host and device. The memory allocator just hands the model weights and the KV cache to whatever compute needs them.

Practically, you get three consequences:

• Capacity is additive. A 24GB dGPU leaves ~8GB for context after a 70B Q4 load ??? not enough for 32K. On 128GB unified, Llama 3.3 70B Q4 (~42GB) plus a 32K KV cache (~10???14GB at 16-bit KV) still leaves roughly 70GB free for the OS, RAG embeddings, and a second small model. You are capacity-rich where dGPU owners are capacity-poor.
• Bandwidth is shared but sufficient. LPDDR5X at ~256 GB/s is slower than HBM3e or GDDR6X on the chip, but decoder math is bandwidth-bound and LPDDR5X still delivers 35???75 tok/s for 70B-class models on these APUs with quantized weights, depending on quant and engine. The win is that you are not paging across PCIe at 32 GB/s.
• Precision is a dial, not a wall. Because you have headroom, you can choose Q6 or Q8 for better quality where a 24GB card would be forced to Q3. For many 70B models, Q4_K_M on 128GB looks perceptually close to Q8 on a smaller box, because you can also keep more context without dropping to Q3.

Two caveats. First, unified does not mean unlimited: the OS, display server, and any ROCm/CUDA runtime reserve 4???8GB. Plan on ~118???122GB usable for weights + KV. Second, 128GB is the same on both boxes at the hardware level, but the software path to that memory differs ??? DGX Spark exposes it through CUDA unified memory with Transformer Engine FP4/FP8 acceleration; Strix Halo exposes it through ROCm + APU-GPU shared buffers or Vulkan via llama.cpp. Same bytes, different kernels. Speed differences come from that, not from the pool size.

Rule of thumb the rest of this guide uses: if the model file (GGUF) + KV cache at your target context < 118GB, it fits. If you need 128K context, add the tax shown in the table ??? long context is where otherwise-fitting models tip over.

Model Fit Table: 24 Models on 128GB Unified Memory

All sizes are measured GGUF footprints (Q4_K_M and Q8_0 sharded) or calculated FP16 = params ?? 2 bytes, plus file overhead. Q4 uses ~0.56???0.60 bytes/param after quantization tables; Q8 is ~1.05 bytes/param. “Context 32K extra” is incremental KV cache at 16-bit KV for 32K tokens (varies by layers, heads, and GQA). “Fits on 128GB” assumes 118GB usable and includes the 32K tax. “Preferred Box” is a recommendation, not a restriction ??? both boxes fit anything marked Yes.

* Llama 4 Maverick, DeepSeek Coder V2 236B, and Qwen3-235B-A22B total-weight at Q4 exceeds 128GB. Some community shards repack experts or use Q3_K_M (~0.38 bytes/param) to squeeze a working 80???95GB file, but native Q4_K_M from the full checkpoint does not fit. Maverick’s usable MoE shard with expert pruning is ~155???170GB at Q4 ??? still over 128GB. Asterisked “No” means no native Q4; Q3 variants exist with quality loss.

Why Quantization Decides What Fits

Parameters are not GB until you pick a number format. A “70B model” is 70 billion weights. Store each weight in 16-bit float (FP16) and you need 70B ?? 2 bytes = 140GB before you have allocated a single token of context ??? already over your 128GB. That is why FP16 is listed as “does not fit” for every 70B entry in the table. Quantization compresses those weights so the same 70B model becomes a file that actually fits in 128GB.

Here is the math we used for the table, and how to estimate any model not listed:

Does Q4 hurt quality? For most 70B+ instruct models, no more than you would notice in day-to-day use. Independent GGUF perplexity deltas for Q4_K_M vs Q8_0 are typically 0.03???0.08 on WikiText, and MT-Bench deltas are <0.15 points. At 70B, Q4 still has 42GB of information ??? more than a 27B model at Q8. The quality loss from dropping context length to make a larger quant fit is usually worse than the loss from dropping one quant level. On 128GB you do not have to choose ??? you can run 70B at Q6 or Q8 with 32K and still have headroom, while gpt-oss-120B must stay Q4.

MoE nuance: For Mixture-of-Experts models the total parameter count overstates the active compute, but the full weight file must still be resident ??? every expert is stored even though only 2???3 fire per token. That is why a 236B MoE at Q4 (~133GB) does not fit even though only 21B is active: you still have to hold all 236B of quantized weights. The “active params” number tells you speed, not fit. Fit is always total params ?? bytes/param.

Recommendation for 128GB: default every dense model ???72B to Q8 if you want the cleanest output, Q4 if you want to run a second model alongside. For 104B???120B MoE models (Command R+, gpt-oss-120B, Mixtral 8x22B, Llama 4 Scout), run Q4_K_M and spend the saved bytes on context. For 235B+ MoEs that do not fit at Q4, either use the vendor’s Q3 shard or ??? more honestly ??? use the distilled 70B variant that fits at Q8.

DGX Spark vs Strix Halo: Same Capacity, Different Speed

The most common misconception we hear is that DGX Spark “holds bigger models” than Strix Halo. It does not. Both expose 128GB LPDDR5X as a single addressable pool. The same GGUF that fits on one fits on the other, byte for byte. If gpt-oss-120b Q4 is ~74GB, it is ~74GB on both. If Llama 3.3 70B Q4 + 32K needs ~53GB, it needs ~53GB on both. Capacity is a tie.

Where they diverge is how fast that memory is turned into tokens, and which software stack you get to use.

Why DGX Spark is faster token-for-token: Blackwell’s 5th-gen Tensor Cores with FP4/FP8 Transformer Engine can execute quantized decode kernels at higher arithmetic intensity, and TensorRT-LLM fuses attention + KV-cache paged kernels better than the current ROCm llama.cpp path. The gap is real but workload-dependent ??? on short prompts (2K in / 512 out) the difference is a few tokens per second; on long-context RAG with 24K of retrieved docs in the prompt, the Blackwell memory + kernel fusion lead widens. Strix Halo counters with power efficiency: at 70???90W it can sustain 30???40 tok/s on a 70B Q4 while staying quiet on a desk, where DGX Spark at ~200W needs more cooling for its extra 15???20 tok/s.

Compatibility note: DGX Spark runs essentially every CUDA LLM container unmodified. Strix Halo runs Ollama, llama.cpp, and LM Studio cleanly via ROCm or Vulkan, but some niche CUDA-only extensions (certain custom Triton kernels, some vLLM paged-attention extras) require the Vulkan fallback or a CPU path. If your workflow is Ollama + Open WebUI + RAG, both are plug-and-play. If your workflow is TensorRT-LLM + NIM + LoRA fine-tuning in NeMo, DGX Spark is the native home.

Bottom line: if your decision criterion is “will my model fit,” flip a coin ??? both hold the same list. If your criterion is “how fast will my fitted model run, and which stack do I already use,” DGX Spark is the speed pick, Strix Halo is the efficiency/value pick.

Context Window Tax: The Cost of 128K

Weights are static; context is dynamic. Every token you keep in the context window costs KV cache ??? two vectors (key and value) per layer per token, cached so the model does not recompute the past. On a 70B-class transformer with ~80 layers and grouped-query attention (GQA 8), 32K of context at 16-bit KV costs roughly 10???14GB. That is additive on top of the model file. At 128K it quadruples.

This is why a model can “fit” at 8K but fall over at 128K. The table below shows measured/estimated KV cache additions for the archetypes in this guide. We assume 16-bit KV (the Ollama/llama.cpp default); 8-bit KV halves these numbers but adds a small quality cost, and is worth enabling for 128K on 120B models.

What this means on 128GB:

• At 32K context, everything marked “Yes” in the main table fits with margin. The canonical 70B Q4 + 32K is ~53GB ??? you can run two of them, or a 70B plus a 32B coder side by side.
• At 64K, still comfortable. 70B Q4 + 64K is ~64???66GB. gpt-oss-120b Q4 + 64K is ~94???100GB. Both under 118GB usable.
• At 128K, check the sum. 70B Q4 + 128K is ~86???90GB ??? fits. gpt-oss-120b Q4 + 128K is ~114???122GB ??? it fits only if you keep overhead low (no second model, close other apps) and ideally use 8-bit KV cache, which drops the KV from ~44GB to ~22GB and brings the total to ~96???102GB. At FP16 KV, 120B + 128K is the edge of the envelope ??? budget it.
• 8-bit KV is your lever for long context. In Ollama/llama.cpp, 8-bit KV halves the context tax with minimal quality impact for chat/RAG. On 128GB, treat “Q4 weights + 8-bit KV” as the long-context preset for anything over 64K.

Practical tip: if you know you will live at 128K (legal doc review, whole-codebase chat, 10-paper synthesis), prefer a 70B Q6 + 8-bit KV over a 120B Q4 + 16-bit KV. You will fit with 30GB to spare, run 20% faster, and the output quality gap is smaller than the latency and OOM-risk gap.

How to Run: One-Line Ollama Commands

Both boxes run Ollama as the fastest path to a chat endpoint. DGX Spark uses the CUDA build; Strix Halo uses the ROCm or Vulkan build ??? same commands, same GGUFs, same OpenAI-compatible API on http://localhost:11434. Install Ollama, then pull and run. First run downloads the GGUF; subsequent runs are instant.

Prerequisites (once):

# DGX Spark (CUDA) ??? install
curl -fsSL https://ollama.com/install.sh | sh
# Strix Halo (ROCm) ??? same installer auto-detects; or Vulkan fallback:
# curl -fsSL https://ollama.com/install.sh | sh
# Verify
ollama --version
nvidia-smi  # DGX Spark only ??? should show GB10
rocminfo    # Strix Halo only ??? should show gfx1150/gfx1151

The one-liners ??? copy/paste:

# 70B sweet spot ??? fits with 60GB headroom, best bang for 128GB
ollama run llama3.3:70b
ollama run qwen2.5:72b
ollama run deepseek-r1:70b          # Distill-Llama 70B reasoning

# The headline ??? 120B MoE that proves 128GB matters
ollama run gpt-oss:120b             # Q4_K_M ~74GB ??? fits on both boxes
ollama run gpt-oss:20b              # tiny MoE, ~12GB ??? for fast drafts

# 24???32B workhorses ??? leave room for a second model
ollama run qwen3:32b
ollama run mistral-small:24b        # Mistral Small 3.1 24B
ollama run gemma3:27b
ollama run qwen2.5-coder:32b         # coding

# Compact ??? run 2???3 concurrently for agents / RAG
ollama run phi4:14b
ollama run gemma3:12b

# MoE ??? MoE total is big, active is small, Q4 fits ???141B total
ollama run mixtral:8x22b             # 141B total, 39B active ??? Q4 ~79GB
ollama run mixtral:8x7b              # 47B total ??? Q4 ~26GB

# Llama 4 family ??? MoE, native long context
ollama run llama4:scout             # 109B total, 17B active ??? Q4 ~65GB
# ollama run llama4:maverick        # 400B total ??? does NOT fit native Q4 on 128GB

# Command R+ ??? RAG-optimized 104B, Q4 fits
ollama run command-r-plus:104b      # or: ollama run command-r-plus

# Small & agentic
ollama run qwen3:30b-a3b             # 30B MoE, 3B active ??? ultra-efficient
ollama run llama3.2:3b
ollama run gemma3:4b

Context and performance flags:

# Long context ??? set num_ctx; add 8-bit KV for 128K on 120B
ollama run gpt-oss:120b --ctx-size 32768
ollama run llama3.3:70b --ctx-size 131072

# Or via Modelfile / API:
# ollama create my-70b-128k -f Modelfile  (with PARAMETER num_ctx 131072)
# curl http://localhost:11434/api/generate -d '{"model":"llama3.3:70b","prompt":"Hello","options":{"num_ctx":32768,"num_predict":512}}'

# Keep a model resident (avoid unload between calls)
ollama run llama3.3:70b --keep-alive 24h

# Run two models side-by-side on 128GB (example: 70B + 32B coder)
ollama run llama3.3:70b &            # ~42GB
ollama run qwen2.5-coder:32b &       # ~19GB  ??? total ~61GB + KV, still <118GB
ollama ps                            # shows both resident

# Advanced: llama.cpp directly for 8-bit KV on 128K (both boxes)
# DGX Spark (CUDA):
llama-server -m gpt-oss-120b-Q4_K_M.gguf --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1
# Strix Halo (Vulkan/ROCm):
llama-server -m gpt-oss-120b-Q4_K_M.gguf --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --gpu-layers 99

Verify you are not swapping:

# Watch resident memory while generating
watch -n 1 'ollama ps; echo "---"; free -h'
# DGX Spark: watch GPU-adjacent unified
watch -n 1 nvidia-smi
# Strix Halo:
watch -n 1 'rocm-smi || radeontop -d -'

If you see the process RSS climb past 118GB and tok/s drop by 3??, you have exceeded usable unified and hit swap/recompute. Fix it by dropping to a lower quant or enabling 8-bit KV ??? do not add swap.

What Does NOT Fit on 128GB

Saying what fits is only half the buying decision. Here is what does not fit ??? and what to do instead so you do not buy the wrong expectation.

Common traps to avoid:

• “I will just use Q2 to fit 405B.” Llama 3.1 405B at Q2_K is still ~114GB for weights alone, plus 28GB KV at 32K ??? 142GB, still over. Even where Q2 squeezes under at 8K context, the quality drop is severe (perplexity +0.6???1.0) and most 70B Q4 runs beat a 405B Q2 on real prompts. If you need 405B-class reasoning on 128GB, use the 70B distill ??? that is what distills are for.
• “MoE active params = fit.” No. A 236B MoE with 21B active still stores 236B of weights. Active tells you tok/s, not GB. Always budget total params.
• “I will add swap/NVMe offload.” llama.cpp and vLLM can offload layers to NVMe, but decode becomes ~3???8 tok/s ??? slower than calling an API. On 128GB unified you bought capacity so you do not have to offload; if your model needs offload, you picked a model bigger than your box.

The honest ceiling for native Q4_K_M on 128GB is ~180???185B total params (180B ?? 0.58 bytes ??? 104GB + 14GB KV at 32K = 118GB). That is why 141B Mixtral fits, 109B Scout fits, and 235B does not. Use that 180B rule to sanity-check any model not in this table.

FAQs: 128GB Model Fit

Footprints measured from published GGUF shards (Q4_K_M / Q8_0) where available, otherwise params ?? 0.58 / 1.05 bytes. KV cache estimates use GQA-aware calculation at 16-bit KV (2 bytes ?? layers ?? (kv_heads ?? head_dim) ?? context) and cross-checked against llama.cpp --verbose KV reports. Usable unified assumed 118GB (128GB minus 6???10GB OS/reserved). Tokens-per-second ranges are observed Ollama / llama.cpp / TensorRT-LLM on 128GB APUs with quantized decode ??? your tok/s will vary by engine version, prompt length, and sampling settings. Models and quants evolve fast; if a newer shard changes a row, the 180B-at-Q4 rule still bounds fit.