Definition
Overfitting describes a model that has learned its training data too well, capturing not just the underlying patterns but also the random noise and quirks specific to that dataset. The result is a model that scores high on the data it was trained on yet performs poorly on new, unseen inputs. It is one of the central failure modes in machine learning, and the mirror image of underfitting.
How to recognize it
The classic signature of overfitting is a large gap between training and validation performance: the model keeps improving on the training set while its accuracy on held-out data plateaus or degrades. Overfitting is more likely when a model has many parameters relative to the amount of training data, when training runs for too many passes over the data, or when the data is noisy. Larger models trained on small datasets are especially vulnerable.
Why it matters for sovereign builders
If you are fine-tuning or training a model on your own hardware rather than renting someone else's API, overfitting is a practical concern, not an abstraction. A model overfit to a narrow corpus will give brittle, overconfident answers outside that corpus. Common countermeasures include holding out a validation set, stopping training early, gathering more diverse data, and applying constraints that discourage excessive complexity.
For more on the techniques used to fight it, see our entries on regularization and synthetic data, and the opposite failure described in underfitting.
Pick a model to fine-tune in the GPU–LLM fit dataset.
In Simple Terms
Overfitting describes a model that has learned its training data too well, capturing not just the underlying patterns but also the random noise and quirks…