Methods > Computer Vision > Generative Models

Pix2Pix is a conditional image-to-image translation architecture that uses a conditional GAN objective combined with a reconstruction loss. The conditional GAN objective for observed images $x$, output images $y$ and the random noise vector $z$ is:

$$ \mathcal{L}_{cGAN}\left(G, D\right) =\mathbb{E}_{x,y}\left[\log D\left(x, y\right)\right]+ \mathbb{E}_{x,z}\left[log(1 − D\left(x, G\left(x, z\right)\right)\right] $$

We augment this with a reconstruction term:

$$ \mathcal{L}_{L1}\left(G\right) = \mathbb{E}_{x,y,z}\left[||y - G\left(x, z\right)||_{1}\right] $$

and we get the final objective as:

$$ G^{*} = \arg\min_{G}\max_{D}\mathcal{L}_{cGAN}\left(G, D\right) + \lambda\mathcal{L}_{L1}\left(G\right) $$

The architectures employed for the generator and discriminator closely follow DCGAN, with a few modifications:

  • Concatenated skip connections are used to "shuttle" low-level information between the input and output, similar to a U-Net.
  • The use of a PatchGAN discriminator that only penalizes structure at the scale of patches.
Source: Image-to-Image Translation with Conditional Adversarial Networks

Latest Papers

PAPER DATE
Content-Aware GAN Compression
Yuchen LiuZhixin ShuYijun LiZhe LinFederico PerazziS. Y. Kung
2021-04-06
MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks
Jonas DenckJens GuehringAndreas MaierEva Rothgang
2021-04-03
CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN
Giorgio BarnabòGiovanni TrappoliniLorenzo LastillaCesare CampagnanoAngela FanFabio PetroniFabrizio Silvestri
2021-04-01
A Novel Adaptive Deep Network for Building Footprint Segmentation
A. ZiaeeR. DehbozorgiM. Döller
2021-02-27
Naturalizing Neuromorphic Vision Event Streams Using GANs
Dennis RobeyWesley ThioHerbert IuJason Eshraghian
2021-02-14
APS: A Large-Scale Multi-Modal Indoor Camera Positioning System
Ali GhofraniRahil Mahdian ToroghiSeyed Mojtaba Tabatabaie
2021-02-08
Segmentation of skin lesions and their attributes using Generative Adversarial Networks
| Cristian Lazo
2021-01-30
cGANs for Cartoon to Real-life Images
Pranjal Singh RajputKanya SatisSonnya DellarosaWenxuan HuangObinna Agba
2021-01-24
Synthetic Glacier SAR Image Generation from Arbitrary Masks Using Pix2Pix Algorithm
Rosanna Dietrich-SussnerAmirabbas DavariThorsten SeehausMatthias BraunVincent ChristleinAndreas MaierChristian Riess
2021-01-08
Image-based Plant Disease Diagnosis with Unsupervised Anomaly Detection Based on Reconstructability of Colors
Ryoya KatafuchiTerumasa Tokunaga
2020-11-29
Masked Linear Regression for Learning Local Receptive Fields for Facial Expression Synthesis
Nazar KhanArbish AkramArif MahmoodSania AshrafKashif Murtaza
2020-11-18
Learning Efficient GANs using Differentiable Masks and co-Attention Distillation
| Shaojie LiMingbao LinYan WangMingliang XuFeiyue HuangYongjian WuLing ShaoRongrong Ji
2020-11-17
Semantic Segmentation for Partially Occluded Apple Trees Based on Deep Learning
Zijue ChenDavid TingRhys NewburyChao Chen
2020-10-14
Pairwise-GAN: Pose-based View Synthesis through Pair-Wise Training
Xuyang ShenJo PlestedYue YaoTom Gedeon
2020-09-13
Deep Learning Based Traffic Surveillance System For Missing and Suspicious Car Detection
K. V. KadambariVishnu Vardhan Nimmalapudi
2020-07-17
3D Topology Transformation with Generative Adversarial Networks
Luca StornaiuoloNima DehmamyAlbert-László BarabásiMauro Martino
2020-07-07
HoughNet: Integrating near and long-range evidence for bottom-up object detection
| Nermin SametSamet HicsonmezEmre Akbas
2020-07-05
2D Image Relighting with Image-to-Image Translation
| Paul GaftonErick Maraz
2020-06-14
Rethinking the Truly Unsupervised Image-to-Image Translation
| Kyungjune BaekYunjey ChoiYoungjung UhJaejun YooHyunjung Shim
2020-06-11
End-to-End Change Detection for High Resolution Drone Images with GAN Architecture
Yura ZharkovskyOvadya Menadeva
2020-05-31
Medical Image Generation using Generative Adversarial Networks
Nripendra Kumar SinghKhalid Raza
2020-05-19
DeepHist: Differentiable Joint and Color Histogram Layers for Image-to-Image Translation
Mor Avi-AharonAssaf ArbelleTammy Riklin Raviv
2020-05-06
Pix2Pix-based Stain-to-Stain Translation: A Solution for Robust Stain Normalization in Histopathology Images Analysis
| Pegah SalehiAbdolah Chalechale
2020-02-03
DermGAN: Synthetic Generation of Clinical Skin Images with Pathology
Amirata GhorbaniVivek NatarajanDavid CozYu-An Liu
2019-11-20
LiDAR ICPS-net: Indoor Camera Positioning based-on Generative Adversarial Network for RGB to Point-Cloud Translation
Ali GhofraniRahil Mahdian ToroghiSeyed Mojtaba TabatabaieSeyed Maziar Tabasi
2019-11-14
Full-Scale Continuous Synthetic Sonar Data Generation with Markov Conditional Generative Adversarial Networks
Marija JegorovaAntti Ilari KarjalainenJose VazquezTimothy Hospedales
2019-10-15
Extremely Weak Supervised Image-to-Image Translation for Semantic Segmentation
| Samarth ShuklaLuc van GoolRadu Timofte
2019-09-18
Data augmentation with Symbolic-to-Real Image Translation GANs for Traffic Sign Recognition
Nour SoufiMatias Valdenegro-Toro
2019-07-17
Image to Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography
| Mohammad EslamiSolale TabarestaniShadi AlbarqouniEhsan AdeliNassir NavabMalek Adjouadi
2019-06-24
Enhanced Pix2pix Dehazing Network
Yanyun Qu Yizi Chen Jingying Huang Yuan Xie
2019-06-01
Learning to Groove with Inverse Sequence Transformations
Jon GillickAdam RobertsJesse EngelDouglas EckDavid Bamman
2019-05-14
Reversible GANs for Memory-efficient Image-to-Image Translation
| Tycho F. A. van der OuderaaDaniel E. Worrall
2019-02-07
Comparison of Patch-Based Conditional Generative Adversarial Neural Net Models with Emphasis on Model Robustness for Use in Head and Neck Cases for MR-Only planning
Peter KlagesIlyes BenslimaneSadegh RiyahiJue JiangMargie HuntJoe DeasyHarini VeeraraghavanNeelam Tyagi
2019-02-01
Generative Adversarial Networks for geometric surfaces prediction in injection molding
Pierre NagornyThomas LacombeHugues FavreliereMaurice PilletEric PairelRonan Le GoffMarlene WaliJerome LoureauxPatrice Kiener
2019-01-29
CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation
| Kishan Babu KancharaguntaShiv Ram Dubey
2019-01-11
Mix and match networks: encoder-decoder alignment for zero-pair image translation
| Yaxing WangJoost Van de WeijerLuis Herranz
2018-04-06
ObamaNet: Photo-realistic lip-sync from text
| Rithesh KumarJose SoteloKundan KumarAlexandre de BrebissonYoshua Bengio
2017-12-06
Image-to-Image Translation with Conditional Adversarial Networks
| Phillip IsolaJun-Yan ZhuTinghui ZhouAlexei A. Efros
2016-11-21

Categories