Multi-stage hybrid text-to-image generation models

Abstract EN

Generative Adversarial Networks (GANs) have proven their outstanding potential in creating realistic images that can't differentiate between them and the real images, but text-to-image (conditional generation) still faces some challenges.

in this paper, we propose a new model called (AttnDM GAN) stands for attentional dynamic memory generative adversarial memory, which seeks to generate realistic output semantically harmonious with an input text description.

AttnDM GAN is a three-stage hybrid model of the attentional generative adversarial network (AttnGAN) and the dynamic memory generative adversarial network (DM-GAN), the 1st stage is called the initial image generation, in which low resolution 64x64 images are generated conditioned on the encoded input textual description.

the 2nd stage is the attention image generation stage that generates higher-resolution images 128 x 128, and the last stage is dynamic memory based image refinement that refines the images to 256 x 256 resolution images.

we conduct an experiment on our model the AttnDM GAN using the Caltech-UCSD birds 200 dataset and evaluate it using the frechet inception distance (FID) with a value of 19.78.

we also proposed another model called dynamic memory attention generative adversarial networks (DMAttn-GAN) which considered a variation of the AttnDM GAN model, where the second and the third stages are switched together, its FID value is 17.04.