ONNX

ONNX (Open Neural Network Exchange) is a community-driven open standard for representing machine-learning models in a portable way. It defines a common computation graph format and a versioned set of built-in operators so that a model trained in one framework can be exported once and executed in another, breaking the vendor and toolchain lock-in that otherwise ties a model to the software it was created in. For a sovereign stack, that portability is the point: a model you depend on should not be hostage to a single runtime's continued existence, licensing, or business model.

How the format works

An ONNX model is a directed acyclic graph of nodes, where each node is a call to a standardized operator, matrix multiplication, convolution, attention primitives, activation functions, with defined input and output tensor types. The operator set is specified centrally and versioned as "opsets," so a runtime that implements a given opset can execute any model targeting it, regardless of which framework produced the file. Because the operators are specified once and shared, implementations stay interoperable rather than each tool quietly reinventing incompatible semantics. The effect is the same one a common document standard has for text: different programs, one file, predictable results. Model weights travel inside the same file as the graph, making an .onnx file a self-contained artifact you can archive, hash, and verify.

ONNX Runtime and running on what you own

The companion project, ONNX Runtime, executes these models efficiently across CPUs, GPUs, and specialized accelerators through pluggable "execution providers." That flexibility is valuable when your compute is whatever you already own rather than whatever a cloud offers: the same exported model can run on a workshop server's GPU, a mini-PC's CPU, or an embedded board, with the runtime choosing the best available backend. Quantized ONNX models extend the reach further down, into edge devices with tight memory budgets, which is why ONNX shows up so often in constrained and embedded deployments where a full training framework would be absurd overhead.

Where ONNX fits next to GGUF

The local LLM world has largely standardized on GGUF, the quantization-friendly format consumed by llama.cpp and its descendants, so ONNX is usually not the format you reach for to run a chat model. Its strength is everything else in a self-hosted AI stack: embedding models for retrieval pipelines, speech-to-text and text-to-speech, image classification and object detection, OCR, and the long tail of small task-specific models. A practical sovereign setup often runs both side by side, a GGUF language model for reasoning and a handful of ONNX models doing the perceptual work around it. Models in either format are commonly distributed through the Hugging Face Hub, and exporting a framework-native model to ONNX is a standard, well-documented step.

Why an open interchange format matters

Formats are where lock-in hides. A model trapped in a proprietary serialization is only as durable as the company that defined it; a model in an open, publicly specified format can outlive its creator, its framework, and the fashion cycle of ML tooling. ONNX gives self-hosters the same guarantee that open firmware gives a miner: the artifact you rely on is documented, inspectable, and runnable by independent implementations, so no single vendor can revoke your ability to use what you already have. In a stack built on the principle of verify, don't trust, the file format is part of the trust surface, and ONNX keeps that surface open.

The operational caveats are versioning ones. A model file targets a specific opset, and a runtime must implement that opset to execute it, so a self-hoster archiving models for the long term should record the opset and a known-good runtime version alongside the file, and verify the file's hash after download. These are the same habits used for firmware images, applied to model artifacts, and they cost minutes now against hours of debugging later.