Adversarial Autoencoder

https://arxiv.org/abs/1511.05644

• A generative models is to yield new data which subjects to a certain distribution so as to capture the distribution.
• An adversarial autoencoder (AAE) that can turn an autoencoder into a generative model.
• Matching the aggregated posterior to the prior ensures that generating from any part of prior space results in meaningful samples.

Motivation

The adversarial autoencoder matches the aggregated posterior distribution of the latent representation of the autoencoder to an arbitrary prior distribution.

The result of the training is that the encoder learns to convert the data distribution to the prior distribution, while the decoder learns a deep generative model that maps the imposed prior to the data distribution.

To speak differently, the encoder maps the data distribution into a prior distritbution in a latent space, and then the decoder returns to the original distribution again. The rationale behind is that any sample generated from the latent space subjected to the prior could be meaningful.

Method

The Adversarial training procedure is used to adjust the distritbution to certain prior, or limit the representation to certain form.

Here x is the data, and z is the code in the hidden space. The upper part is an autoencoder that build the one-to-one mapping between x and z. The lower part is the discriminator which force the distribution of z to match the prior p(z).

GAN

The discriminator guadually adjusts the distribution of generated samples to be indistinguishable from the real samples.

In AAE, the adversarial training procedure is to allow the codes generated by the encoder to match the prior distribution.

Method

• Both, the adversarial network and the autoencoder are trained jointly with SGD in two phases – the reconstruction phase and the regularization phase – executed on each mini-batch.
• In the reconstruction phase, the autoencoder updates the encoder and the decoder to minimize the reconstruction error of the inputs.
• In the regularization phase, the adversarial network first updates its discriminative network to tell apart the true samples (generated using the prior) from the generated samples (the hidden codes computed by the autoencoder). The adversarial network then updates its generator (which is also the encoder of the autoencoder) to confuse the discriminative network.

#

Unsupervised Clustering

• Using y, the indicator, can divide the data into clusters of a predefined number.
• The label information y and the style information z is disentangled.

Example

Example(Mapping)