MLOps

MLOps (Machine Learning Operations) is the set of practices, tooling, and culture that takes a trained machine learning model out of a notebook and runs it reliably in production over time. It borrows heavily from DevOps, but adds the data and model concerns that traditional software pipelines never had to manage: training data versioning, reproducible experiments, automated retraining, and the slow degradation of accuracy as the world changes around a model.

What MLOps covers

A mature MLOps practice spans the full lifecycle. Data and feature pipelines feed reproducible training runs; finished models are tracked in a model registry; approved versions are pushed to serving infrastructure; and the running system is watched continuously for performance regressions. Continuous integration and continuous delivery (CI/CD) concepts apply, but with an extra loop: continuous training, where fresh data triggers automated retraining when quality slips.

Why sovereign operators care

For anyone running models on their own hardware rather than renting a black-box API, MLOps is the difference between a one-off experiment and a dependable service. Self-hosting an inference workload means you own the uptime, the version history, and the monitoring — nobody upstream silently swaps your model or changes its behavior. The same self-reliance ethic that drives running your own Bitcoin node applies here: control the full stack, audit every change, and keep a reproducible record.

At D-Central we treat AI infrastructure as another layer to decentralize. To go deeper on the building blocks, see our entries on model serving, the model registry, and model monitoring.

A Minimal Self-Hosted MLOps Stack

You do not need a platform team to get the core disciplines. Version everything that determines behavior: the model file (record its exact checksum — model repositories can and do republish files under the same name), the runtime version, the prompt templates, and the configuration. Pin them together so “what is running right now” has one answer. Keep an evaluation set — even thirty representative inputs with known-good outputs — and run it whenever anything changes; that turns “the new model feels better” into a measurement. Add basic service hygiene: the inference server under a supervisor that restarts it, logs retained, disk and VRAM monitored. That handful of habits is most of the value of MLOps at home-lab scale.

Drift: Why a Working Model Stops Working

Models degrade in production without a single line of code changing, and MLOps names the two mechanisms. Data drift: the inputs shift away from what the model was built against — new hardware models appear in your repair logs, new jargon enters the tickets — so accuracy erodes at the edges first. Concept drift: the relationship itself changes — the same symptom now has a different most-likely cause because the fleet composition changed. Monitoring output quality over time, not just uptime, is what catches both; the operational tell is a slow rise in corrections, retries, or human overrides rather than a dramatic failure.

The Sovereignty Dividend

Every MLOps discipline pays double when you self-host, because you are also immune to the failure mode no cloud customer can control: silent upstream change. Hosted models get updated, deprecated, and re-aligned under stable API names, so a pipeline that worked in January can behave differently in June with nothing in your changelog. A pinned local model with a recorded checksum literally cannot change behind your back — reproducibility is a property of the setup rather than a promise from a vendor. Pair a versioned model with a versioned runtime like llama.cpp and the whole stack becomes auditable the same way a node's software is — which is precisely the point of self-hosting it.