Variational Autoencoder (VAE)

A variational autoencoder (VAE) is a probabilistic generative model, introduced by Kingma and Welling in 2013, that learns a smooth, continuous representation of data and can generate new examples from it. Like a standard autoencoder, it has an encoder and a decoder; unlike one, its encoder outputs the parameters of a probability distribution (a mean and a variance) rather than a single fixed point.

How It Works

The encoder maps each input to a distribution over the latent space, typically assumed to follow a standard Gaussian prior. A sample is drawn from that distribution and passed to the decoder, which reconstructs the input. Training optimizes the Evidence Lower Bound (ELBO), which has two parts: a reconstruction term that rewards faithful output, and a Kullback-Leibler (KL) divergence term that keeps each learned distribution close to the prior. That regularization is what makes the latent space continuous and well-organized, so nearby points decode to sensible, smoothly varying outputs.

Why Sovereign Builders Care

Because the latent space is continuous, a VAE can interpolate between examples and synthesize novel data, all while running locally on hardware you own. That makes VAEs useful for anomaly detection, data compression, and privacy-preserving synthetic data generation without sending anything to a third-party service.

A VAE is one way to structure a model's latent space, and it competes with adversarial approaches like the generative adversarial network for the generative-modeling role.