ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

1 Sep 2018  ·  Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Chen Change Loy, Yu Qiao, Xiaoou Tang ·

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge. The code is available at https://github.com/xinntao/ESRGAN .

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution BSD100 - 4x upscaling SRGAN + Residual-in-Residual Dense Block PSNR 27.85 # 8
SSIM 0.7455 # 11
Image Super-Resolution FFHQ 1024 x 1024 - 4x upscaling ESRGAN FID 72.73 # 9
MS-SSIM 0.782 # 9
PSNR 19.84 # 9
SSIM 0.353 # 9
Image Super-Resolution FFHQ 256 x 256 - 4x upscaling ESRGAN FID 166.36 # 9
MS-SSIM 0.747 # 9
PSNR 15.43 # 11
SSIM 0.267 # 11
Face Hallucination FFHQ 512 x 512 - 16x upscaling ESRGAN FID 50.901 # 2
LPIPS 0.3928 # 2
NIQE 15.383 # 4
Image Super-Resolution FFHQ 512 x 512 - 4x upscaling ESRGAN PSNR 27.134 # 6
SSIM 0.741 # 5
MS-SSIM 0.935 # 5
LLE 2.261 # 4
FED 0.1107 # 6
FID 3.503 # 2
LPIPS 0.1221 # 2
NIQE 6.984 # 2
Image Super-Resolution Manga109 - 4x upscaling SRGAN + Residual-in-Residual Dense Block PSNR 31.66 # 16
SSIM 0.9196 # 19
Image Super-Resolution Manga109 - 4x upscaling bicubic PSNR 24.89 # 37
SSIM 0.7866 # 36
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration ESRGAN Subjective score 5.353 # 14
ERQAv1.0 0.735 # 7
QRCRv1.0 0 # 21
SSIM 0.808 # 24
PSNR 27.33 # 15
FPS 1.004 # 15
1 - LPIPS 0.948 # 1
Video Super-Resolution MSU Video Upscalers: Quality Enhancement ESRGAN PSNR 27.29 # 35
SSIM 0.936 # 43
VMAF 56.69 # 8
Image Super-Resolution PIRM-test ESRGAN NIQE 2.55 # 2
Image Super-Resolution Set14 - 4x upscaling SRGAN + Residual-in-Residual Dense Block PSNR 28.99 # 15
SSIM 0.7917 # 17
Image Super-Resolution Set5 - 4x upscaling SRGAN + Residual-in-Residual Dense Block PSNR 32.73 # 2
SSIM 0.9011 # 2
Image Super-Resolution Urban100 - 4x upscaling bicubic PSNR 23.14 # 48
SSIM 0.6577 # 43
Image Super-Resolution Urban100 - 4x upscaling SRGAN + Residual-in-Residual Dense Block PSNR 27.03 # 13
SSIM 0.8153 # 10

Methods