Fig 3 - uploaded by Susana Castillo
Content may be subject to copyright.
Left: perceived realness for each condition. Participants perceived around 65% of our high-quality face swaps and 50% of our low-quality face swaps as real. Right: Reported importance of artifacts for each condition when labelling a video as a face swap. Error bars represent the standard error of the mean (SEM).
Source publication
Videos obtained by current face swapping techniques can contain artifacts potentially detectable, yet unobtrusive to human observers. However, the perceptual differences between real and altered videos, as well as properties leading humans to classify a video as manipulated, are still unclear. Thus, to support the research on perceived realism and...
Context in source publication
Context 1
... Realism. We first analyse the assessment accuracy of both experiments as shown in Fig. 3. We find that participants believed around 65% of the high-quality face swaps and 50% of the low-quality face swaps to be real videos. As a baseline, we include the accuracy for the corresponding real stimuli, which was around 80% in both E1 and E2. Therefore, our high-quality face swaps were overall very convincing, even though they ...
Similar publications
Advances in editing tools and compression technologies have made it possible to easily manipulate videos without leaving any visual traces and then compress them using video codecs. Among the various forging operations, fine-grained manipulations such as filtering and noise addition accompany various forgery scenarios. Detecting low-level features...
Citations
... More specifically, faceswapping is a technique to create a synthetic person by combining the body and movements of one person with the face of another person (see Fig. 1). Face swaps generated by recent approaches can be created with minimal manual intervention and are nearly undetectable for human observers [33,38], with new methods even further improving their quality and generation speed [7,44]. Moreover, a perceptual study found that face swaps are able to generally convey the same emotions as the corresponding source real videos [39]. ...
... In order to create reliable avatars from face swaps, it is necessary to investigate how they are perceived. To this end, the PEFS dataset [38] contains high-quality face swaps created from recordings of free interviews and triggered emotions by a method acting protocol [13]. In contrast to previous datasets, which include distracting background elements [30,31] or well-known celebrities [17], the recordings of the PEFS dataset allow to assess the personality of people behaving naturally without clues about their personal background or career choices. ...
... Previous research has found many factors influencing the acceptance and affinity towards virtual characters and interactive agents such as distorted proportions of facial elements [19], the level of detail on the skin [43], or the animation quality and presence of motion anomalies [22]. As face swaps can be affected by artifacts like unfitting facial contours, flickering and blurriness, or unnatural expressions [35,38,39], it is possible that they are perceived less pleasant than real people. Therefore, we follow works on the personality of virtual characters [45] and ask participants about the Appeal and Eeriness of the actors in the videos. ...
In this work, we investigate facial anonymization techniques in 360° videos and assess their influence on the perceived realism, anonymization effect, and presence of participants. In comparison to traditional footage, 360° videos can convey engaging, immersive experiences that accurately represent the atmosphere of real-world locations. As the entire environment is captured simultaneously, it is necessary to anonymize the faces of bystanders in recordings of public spaces. Since this alters the video content, the perceived realism and immersion could be reduced. To understand these effects, we compare non-anonymized and anonymized 360° videos using blurring, black boxes, and face-swapping shown either on a regular screen or in a head-mounted display (HMD).
Our results indicate significant differences in the perception of the anonymization techniques. We find that face-swapping is most realistic and least disruptive, however, participants raised concerns regarding the effectiveness of the anonymization. Furthermore, we observe that presence is affected by facial anonymization in HMD condition. Overall, the results underscore the need for facial anonymization techniques that balance both photo-realism and a sense of privacy.
We present a novel framework for the evaluation of eye tracking data in portrait videos including the automatic generation of customized areas of interest (AOIs) based on facial landmarks. In contrast to previous work, our framework allows the user to flexibly create AOIs by grouping the detected landmarks. Moreover, their shape and size can be modified to better fit both the research question and the precision of the eye tracker. The framework can be used as an integrated solution to not only generate AOIs but also to evaluate viewing behavior like the overall fixation times, the similarity of scanpaths, and the number of saccades between AOIs. Other functionalities include the visualization of gaze paths and the creation of heatmaps. We demonstrate the benefits of our framework and user-defined AOI layouts via an exemplary application, i.e., the investigation of face swapping artifacts.
In this paper, we report on perceptual experiments indicating that there are distinct and quantitatively measurable differences in the way we visually perceive genuine versus face-swapped videos.
Recent progress in deep learning has made face-swapping techniques a powerful tool for creative purposes, but also a means for unethical forgeries. Currently, it remains unclear why people are misled, and which indicators they use to recognize potential manipulations. Here, we conduct three perceptual experiments focusing on a wide range of aspects: the conspicuousness of artifacts, the viewing behavior using eye tracking, the recognition accuracy for different video lengths, and the assessment of emotions.
Our experiments show that responses differ distinctly when watching manipulated as opposed to original faces, from which we derive perceptual cues to recognize face swaps. By investigating physiologically measurable signals, our findings yield valuable insights that may also be useful for advanced algorithmic detection.
