CONF Panchard_SPIE_2021/IDIAP Perspectives and limitations of visible-thermal image pair synthesis via generative adversarial networks Panchard, Danick Marelli, François De Moura Presa, Edouard Wellig, Peter Liebling, Michael Generative Adversarial Networks image synthesis thermal imaging EXTERNAL https://publications.idiap.ch/attachments/papers/2022/Panchard_SPIE_2021.pdf PUBLIC Security + Defence, Target and Background Signatures VII, Proc. of SPIE online only 11865 1186509-1--1186509-8 2021 SPIE https://doi.org/10.1117/12.2599889 URL 10.1117/12.2599889 doi Many applications rely on thermal imagers to complement or replace visible light sensors in difficult imaging conditions. Recent advances in machine learning have opened the possibility of analyzing or enhancing images, yet these methods require large annotated databases. Training approaches that leverage data augmentation via simulated and synthetically-generated images could offer promising prospects. Here, we report on a method that uses generative adversarial nets (GANs) to synthesize images of a complementary contrast. Starting from a dual-modality dataset of co-registered visible and thermal images, we trained a GAN to generate synthetic thermal images from visible images and vice versa. Our results show that the procedure yields sharp synthesized images that might be used to augment dual-modality datasets or assist in visual interpretation, yet are also subject to the limitations imposed by contrast independence between thermal and visible images.