Benchmarks
Photo
Note: waifu2x's photo models was trained on the blending dataset of kou's photo collection and ukbench.
Note: PSNR in this benchmark uses a MATLAB's rgb2ycbcr compatible function (dynamic range [16 235], not [0 255]) for converting grayscale image. I think it's not correct PSNR. But many paper used this metric.
command:
th tools/benchmark.lua -dir <dataset_dir> -model1_dir <model_dir> -method scale -filter Catrom -color y -range_bug 1 -tta <0|1> -force_cudnn 1
Datasets
BSD100: https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/ (100 test images in BSDS300)
Urban100: https://github.com/jbhuang0604/SelfExSR
2x - PSNR
Dataset/Model |
Bicubic |
vgg_7/photo |
upconv_7/photo |
upconv_7l/photo |
resnet_14l/photo |
BSD100 |
29.558 |
31.427 |
31.640 |
31.749 |
31.847 |
Urban100 |
26.852 |
30.057 |
30.477 |
30.759 |
31.016 |
2x with TTA - PSNR
Note: TTA is an ensemble technique that is supported by waifu2x. TTA method is 8x slower than non TTA method but it improves PSNR (~+0.1 on photo, ~+0.4 on art).
Dataset/Model |
Bicubic |
vgg_7/photo |
upconv_7/photo |
upconv_7l/photo |
resnet_14l/photo |
BSD100 |
29.558 |
31.474 |
31.705 |
31.812 |
31.915 |
Urban100 |
26.852 |
30.140 |
30.599 |
30.868 |
31.162 |
2x - benchmark elapsed time (sec)
Dataset/Model |
vgg_7/photo |
upconv_7/photo |
upconv_7l/photo |
resnet_14l/photo |
BSD100 |
4.057 |
2.509 |
4.947 |
6.86 |
Urban100 |
16.349 |
7.083 |
14.178 |
27.87 |
2x with TTA - benchmark elapsed time (sec)
Dataset/Model |
vgg_7/photo |
upconv_7/photo |
upconv_7l/photo |
resnet_14l/photo |
BSD100 |
36.611 |
20.219 |
42.486 |
60.38 |
Urban100 |
132.416 |
65.125 |
129.916 |
255.20 |
Art
command:
th tools/benchmark.lua -dir <dataset_dir> -model1_dir <model_dir> -method scale -filter Lanczos -color y -range_bug 1 -tta <0|1> -force_cudnn 1
Dataset
art_test: This dataset contains 85 various fan-arts. Sorry, This dataset is private.
2x - PSNR
Dataset/Model |
Bicubic |
vgg_7/art |
upconv_7/art |
upconv_7l/art |
art_test |
31.022 |
37.495 |
38.330 |
39.140 |
2x with TTA - PSNR
Dataset/Model |
Bicubic |
vgg_7/art |
upconv_7/art |
upconv_7l/art |
art_test |
31.022 |
37.777 |
38.677 |
39.510 |
2x - benchmark elapsed time (sec)
Dataset/Model |
vgg_7/art |
upconv_7/art |
upconv_7l/art |
art_test |
20.681 |
7.683 |
17.667 |
2x with TTA - benchmark elapsed time (sec)
Dataset/Model |
vgg_7/art |
upconv_7/art |
upconv_7l/art |
art_test |
174.674 |
77.716 |
163.932 |