By Hari Haran
Intro
Understanding variational autoencoders (VAEs) can be a daunting task, but it doesn’t have to be! VAEs are a type of artificial neural network used to generate new data from existing data.
By learning the underlying distributions of data, VAEs are able to generate new data that is similar to the existing data.
Image Credits; Geeksforgeek
In this blog post, we’ll discuss the basics of variational autoencoders and how they work. We’ll also provide some practical examples and tips on how to get started using them. So, let’s dive in and learn about VAEs!
What is a VAE?
Variational Autoencoders (VAEs) are a type of neural network architecture that combines the capabilities of autoencoders and variational inference to learn a compressed representation of data.
In traditional autoencoders, the input data is compressed into a lower-dimensional space by an encoder, and then reconstructed by a decoder.
However, the bottleneck layer in the encoder-decoder network is usually regularised by adding constraints on the weights or biases. In contrast, VAEs aim to regularise the distribution of the latent space.
The latent space in VAEs is a compact representation of the input data, in which similar data points are closer to each other. VAEs learn to model the probability distribution of the latent space, making it possible to generate new data by sampling from this distribution.
Regularisation ensures that the distribution of the latent space is continuous and smooth, which means that small changes in the latent space lead to small changes in the generated output.
VAEs are closely linked to variational inference, which is a statistical framework that provides a way to estimate the probability distribution of latent variables in a generative model. By combining variational inference with neural networks, VAEs enable efficient and scalable probabilistic modelling of high-dimensional data.
In the context of dimensionality reduction and autoencoders, VAEs offer a significant advantage over traditional autoencoders.
While traditional autoencoders can compress data into a lower-dimensional space, they are not designed to generate new data. On the other hand, VAEs can generate new data points by sampling from the probability distribution of the latent space.
To understand VAEs more deeply, it is necessary to look at the mathematical principles behind the architecture. VAEs are built on the principles of variational inference, which uses probabilistic models to represent the latent space.
In VAEs, the goal is to maximize a lower bound on the marginal likelihood of the data, given the latent variables.
This objective is achieved by optimizing the evidence lower bound (ELBO), which is a tradeoff between the reconstruction error and the Kullback-Leibler divergence between the prior and posterior distributions of the latent space.
VAEs are a powerful tool for generative modelling and unsupervised learning, which can be used to learn compact representations of high-dimensional data and generate new data points.
With a clear understanding of how VAEs work and the benefits they offer, you can start exploring their potential applications in your own work.
What are the benefits of using a VAE?
Variational autoencoders (VAEs) offer several benefits over traditional autoencoders. One significant advantage is their ability to generate new data points from the learned data distribution.
This means that if we have a dataset of images of dogs, a VAE can learn the underlying distribution of the data and generate new, previously unseen images of dogs.
Another benefit of VAEs is their ability to learn a lower-dimensional representation of the data. By compressing high-dimensional data into a lower-dimensional space, VAEs can capture the essential features of the data, making it easier to analyze and process.
VAEs can also handle missing or incomplete data, which is often the case in real-world datasets. Because VAEs learn a distribution of the data, they can reconstruct missing data points by using the learned distribution to generate a plausible replacement.
Finally, VAEs can be used for unsupervised learning, meaning that they can learn patterns and features in the data without being given explicit labels.
This makes them useful for applications where labeled data is scarce or expensive to obtain.
VAEs offer several benefits over traditional autoencoders, making them a popular tool for machine learning researchers and practitioners.
How Do VAE works?
In a traditional autoencoder, the encoder network outputs a single value for each encoding dimension, which the decoder network then uses to attempt to recreate the original input.
The encoder in a VAE is formulated to describe a probability distribution for each latent attribute, rather than just outputting a single value.
For example, if we trained a VAE on a dataset of faces with an encoding dimension of 6, the encoder model might output a probability distribution for the size of the nose, the shape of the eyes, etc.
This approach to encoding ensures that each latent attribute is represented as a range of possible values, rather than a single point. This means that the encoder model is sometimes referred to as the recognition model, as it “recognizes” the distribution of possible values for each latent attribute.
Similarly, the decoder model in a VAE is sometimes referred to as the generative model. By constructing the encoder model to output a range of possible values, a continuous, smooth latent space representation is enforced.
Reference : Variational Autoencoders (VAEs) for Dummies — Step By Step Tutorial
This has important implications for generating new samples from the model.
This makes it possible to generate new samples by sampling from the latent space distribution and decoding the sampled values back into the original data space.
VAEs offer a powerful approach to encoding and decoding data in a probabilistic, continuous latent space representation. They have many applications in image and text generation, data compression, and more.