THESIS Colbois_THESIS_2025/IDIAP Face morphing attacks in the era of deepfakes: risks, detection & source attribution Colbois, Laurent Biometrics deepfake detection Deepfakes face morph Face Recognition Forensics image forensics Morphing Attack morphing attack detection EXTERNAL https://publications.idiap.ch/attachments/papers/2025/Colbois_THESIS_2025.pdf PUBLIC 2025 Université de Lausanne Face morphing attacks exploit vulnerabilities in automated face recognition systems through the creation and submission of fake reference passport pictures. These images blend the faces of two individuals, allowing them to share the same passport and posing a significant security threat. Initially, face morphs were generated using relatively simple image processing techniques. However, recent advancements in generative artificial intelligence (AI) have enabled the creation of particularly realistic fake images, commonly known as deepfakes. Deepfakes present an escalating challenge for society: generation techniques are rapidly advancing, diversifying, producing increasingly realistic results, and becoming more accessible to the public through widespread exposure and the availability of user-friendly commercial generators. The creation of synthetic faces or the alteration of real ones has been a primary application of deepfake technology since its inception. Consequently, the general issues posed by deepfakes are directly relevant to biometric security, particularly face morphing attacks. Deepfake technology has directly influenced the evolution of morphing attacks, leading to the emergence of novel 'deep' morphs. Traditional face morphs have been extensively studied, resulting in various methods that can detect them with reasonable accuracy. In contrast, deep morphs are a new phenomenon that requires further investigation, which is the central focus of this thesis. We examine the risks posed by modern morphing attacks by assessing their ability to deceive existing face recognition systems and discussing their potential to fool human operators who process submitted passport pictures. Additionally, we introduce a novel deep morphing attack, which exploits models designed for reconstructing faces from face recognition embeddings. This demonstrates how techniques originally designed for face reconstruction can be repurposed to generate highly effective morphing attacks with minimal effort. Overall, our findings highlight that deep morphs can deceive face recognition systems as effectively as traditional morphs, underscoring the need for updated countermeasures. To this end, we first focus on developing detection systems capable of identifying both traditional and deep morphs. We then extend our efforts to source attribution, which aims to identify the algorithm used to create a given fake media, thus going beyond mere detection. With the increasing diversity of morphing attack generators, source attribution provides valuable insights that can be integrated into investigations, serving as a partial indicator of the authorship of a fake media. We conceptualize our studies within the framework of forensic science, framing them as a search for relevant traces in AI-generated images. These traces, whether semantic (visible anomalies, such as irregular facial features) or statistical (imperceptible patterns in noise or in frequency domains), are crucial for both detecting and attributing morphing attacks. In practice, we leverage representation learning techniques to discover salient traces in a data-driven manner. We consider not only end-to-end supervised learning approaches (dominant in the field), which train deep neural networks on examples of real and fake images for binary or multiclass classification, but also other methodologies that more clearly separate the trace extraction process from the actual modeling. Specifically, we evaluate three families of feature representations: (1) handcrafted features, manually designed using expert knowledge; (2) supervised features, learned through end-to-end supervised training on labeled real and fake images; and (3) foundational features, extracted using large vision models trained exclusively on real images, making them independent of any specific attack type. We evaluate and discuss the strengths and weaknesses of these three major families of representations in terms of discriminative power (for detection and source attribution), generality, robustness, and interpretability. Overall, we find that approaches based on foundational features are particularly promising, especially in terms of generality and robustness, whereas supervised approaches tend to overspecialize to the types of deepfakes seen during training. Notably, foundational features enable the development of fully attack-agnostic detection models by providing a fine statistical description of natural images, allowing any attack to be detected as an out-of-distribution sample. Moreover, despite being developed using only real images, foundational features also provide a surprisingly well-structured space in which attacks from various generators are nicely clustered, enhancing their potential for source attribution tasks, even in open-set scenarios. Our findings demonstrate that deepfake analysis systems can greatly benefit from visual foundation models, which may surpass traditional supervised approaches. Given the rapid advancement of vision foundation models, these techniques offer a promising, forward-compatible strategy for deepfake forensic analysis, in an era where deepfake generators also keep progressing.