Definition
The bias-variance tradeoff is the central tension that governs how well a model generalizes. Bias is error from a model being too simple — it makes strong assumptions and misses real structure, the hallmark of underfitting. Variance is error from a model being too sensitive to the particular training data it saw, so it captures noise and fails on new inputs, the hallmark of overfitting. Total generalization error decomposes into bias, variance, and irreducible noise.
The tradeoff
The two pull in opposite directions. Increasing model capacity (more parameters, more flexibility) lowers bias but raises variance; constraining the model raises bias but lowers variance. Plotted against complexity, training error falls steadily while test error forms a U-shape — high on the left from bias, high on the right from variance, with a sweet spot in the middle. Finding that sweet spot is much of the practical work of training a good model. (Very large modern networks complicate the classic picture, but the framework remains the working intuition.)
Managing it
The levers are familiar: add data or regularization to cut variance, increase capacity or better features to cut bias, and use a held-out validation set to watch where you actually sit on the curve. For a sovereign builder fine-tuning on a modest private dataset, the bias-variance lens is the quickest way to diagnose whether a disappointing model needs more capacity or more constraint.
It is the unifying frame behind overfitting and underfitting.
In Simple Terms
The bias-variance tradeoff is the central tension that governs how well a model generalizes. Bias is error from a model being too simple — it…