Autoencoders are one of the deep learning types used for unsupervised learning. There are key layers of autoencoders, which are the input layer, encoder, bottleneck hidden layer, decoder, and output.
The three layers of the autoencoder are:-
Encoder - Compresses the input data to an encoded representation which is typically much smaller than the input data.
Latent Space Representation/ Bottleneck/ Code - Compact summary of the input containing the most important features
Decoder - Decompresses the knowledge representation and reconstructs the data back from its encoded form. Then a loss function is used at the top to compare the input and output images. NOTE- It's a requirement that the dimensionality of the input and output be the same. Everything in the middle can be played with.
Autoencoders have a wide variety of usage in the real world. The following are some of the popular ones:
Transformers and Big Bird (Autoencoders is one of these components in both algorithms): Text Summarizer, Text Generator
- Image compression
- Nonlinear version of PCA
