Jennifer A. Fehring, Tamara Bonaci
©SHUTTERSTOCK.COM/RROSELAVY
Deepfakes are images or videos generated using deep learning technology to change the original conditions of a piece of media. Potential uses of this technology range from satirical content, depicting public figures in comical scenarios; to generating audio to mimic a specific voice; to inserting the face of an unknowing individual into potentially embarrassing content, for example, a pornographic scene. As deepfake technologies, such as autoencoders and generative adversarial networks (GANs), become more advanced and accessible, deepfakes become easier to create and more believable. This poses some serious threats to individual and institutional safety, as scenes can be modified to alter public perception. Although technology to identify deepfakes does exist, it is essential that these methods progress as rapidly as deepfake technology so that they remain accurate. Additionally, legal consequences for deepfakes are limited, so continued advocacy for increased protection against false content is crucial.
The term deepfake refers to an image or an audio or a video signal generated by a deep learning-based technique that swaps the face, speech, or actions (further referred to as the likeness) of the original subject with the likeness of another. Popular examples of deepfakes include a 2019 video of President Obama swearing during a public service announcement, a TikTok account named deeptomcruise with more than 1.7 million followers created to specifically publish deepfake videos, and a video of Mark Zuckerberg announcing the deletion of Facebook that garnered 72 million views. The term deepfake first appeared in 2017, when a Reddit user named deepfakes began publishing content to the site depicting unknowing celebrities in pornographic movies using a machine learning algorithm they had created (https://www.bbc.co.uk/bitesize/articles/zfkwcqt). Other potentially harmful deepfake content includes hoaxes, fake news, and financial fraud. As a result, increasing attention in general media forensics has been dedicated to detecting facial manipulation in images and videos. Deepfake detection is becoming increasingly important as deepfakes become easier to create and more believable.
In years prior, creating content that believably replaced features or components of an image or video with other features was possible, but it was labor intensive, and it required special resources. Computer-generated imagery (CGI) has seen great use in media and dates back as far as the 1960s. One of the first publicly available examples of CGI used in popular media was a 49-s animation called Rendering of a Planned Highway, wherein a car was displayed traveling up a planned highway at 110 km/h, created by the Swedish Royal Institute of Technology on the BESK computer. This animation was broadcast on national television on 9 November 1961 (Du Rietz, 2016).
Since then, CGI has made appearances in many feature films, including Avatar, which was the first full-length movie made using performance capture to create photorealistic characters and worlds. Work for Avatar started in 2006, and the film was released in 2009; during this period, replacing real-life scenes with nonoriginal content was extremely labor intensive. Nine hundred people were employed to work on the film; Microsoft created a new cloud computing and digital asset management system called Gaia to permit the crew to keep track of and coordinate all stages of digital processing. A 10,000-ft2 server farm with 35,000 processor cores, and 3 PB of network area storage was used to render the data; this was considered one of the top 500 supercomputers in the world at the time.
Although a short deepfake may not require the same amount of attention and detail as a full-feature film like Avatar, technology for creating fake but believable content has progressed very rapidly in the past decade. Where once it was necessary to possess a highly specialized set of technological skills, significant financial resources, and specialized hardware and software, it is now possible for an unskilled individual to create deepfakes on devices as basic as a cellphone, using apps like FakeApp and Zao, with as little source material as a few selfies in different head positions and facial expressions. (FakeApp “is a program that lets you create masks capable of swapping faces on videos by means of the artificial intelligence developed by a Reddit user.” It can be accessed using the following link: https://www.malavida.com/en/soft/fakeapp/. Zao “is a free deepfake face-swapping app that lets you replace the face of popular characters in a video clip of a scene with your own face.” It can be accessed using the following link: https://zao.en.softonic.com/android.)
This type of accessibility will, no doubt, lead to a rise in deepfake creation and distribution, as more and more people gain access to easy and believable deepfake generators. This problem is only amplified by the rise of social media, which makes the dissemination of fake content easier than it has ever been.
Currently, there exists limited research on the believability of deepfakes, especially the believability of earlier deepfakes from the late 2010s, making it difficult to quantify how believability has increased. However, a recent studies have sought to determine how easy or difficult it is to detect deepfakes.
One 2020 study asked 60 participants to subjectively evaluate videos from a pool of 120 videos that ranged from being a clear fake to having a fakeness that was very difficult to discern (Korshunovf and Sebastien, 2020). These same videos were also evaluated for fakeness using two different state-of-the-art deepfake-detection algorithms based on the Xception (https://keras.io/api/applications/xception/) and EfficientNet (https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html) models, which were pretrained separately on Google and Celeb-DF (https://github.com/yuezunli/celeb-deepfakeforensics) deepfake databases. When presented with high-quality deepfakes, 75.5% of participants were confused and could not confidently determine whether the video was fake or real. Obviously fake deepfakes proved easy for participants to detect; however, as the deepfake quality increased, participants were more likely to incorrectly assess the video as real, even in situations where they felt certain of their decision.
It is notable that these participants were primed to be looking for fakes, and, even then, only 24.5% of participants were able to confidently identify high-quality deepfakes as fake. In this study, the two algorithms tested were not as accurate in assessing deepfakes as fake when compared to human subjects; however, the algorithms were better at identifying fake videos that humans had confidently assessed as being real.
In reality, most people are not on the lookout for fake videos and may be even more susceptible to believing fake content. In recent years, humans have become more apprehensive of photos, as photo editing has become the norm, and humans are more wary of trusting photographs. Despite this increasing distrust, overall, people do still place a great deal of trust in photographic evidence and even more trust in videos displaying known faces and voices. As deepfakes become more believable and humans continue to place trust in content that they can see, hear, and easily associate with known figures, deepfakes promise to be a great source of misinformation. This has had and will continue to have serious ramifications for individual safety and public perceptions.
The term deepfakes derives from the technique employed to create them: deep learning. Deep learning is a branch of machine learning that focuses on deep neural networks—a computational counterpart of the network of neurons used to deliver and decipher information in the human brain. A neural network in computer science uses nodes instead of neurons, and each node shares specific connections to thousands of other nodes. Typically, they are arranged in layers, and the way in which the nodes are connected determines the type of network it is and its ultimate ability to perform certain tasks. When the input or output of a neural network is an image, typically, three nodes will exist for each pixel, each containing information on the amount of color (typically, the amount of red, blue, or green colors) contained in that pixel.
An autoencoder is a special type of neural network that aims to create an output image matching the input. To avoid simply matching an input node to an output node, which would imply there has been no learning, the autoencoder compresses image information prior to reconstruction. Ultimately, this will result in loss of some finer details in the output. The recreation of an image using an autoencoder occurs in three steps: the 1) encoding step, 2) latent space step, and 3) decoding step (Fig. 1). In the encoding step, an image is taken in with many thousands of pixels, and they are compressed to a much smaller measurement space (e.g., 300 measurements), with each relating to a particular facial characteristic. This compressed version of the original image is considered an information bottleneck.
Fig 1 An autoencoder takes in an input layer, which is compressed before the data are decoded, to generate a reconstructed image.
All possible combinations of measurements of facial characteristics occupy space known as latent space. Latent space consisting of a smaller space of measurements requires a tiny fraction of the memory needed to store the original input image. From this latent space, a reconstruction is generated through the decoder. If a network can generalize a face well enough, the face from one network can pass a latent face to the decoder of another network, resulting in the replacement of the second network’s face with the features and expressions of the face from the first (Zucconi, 2019).
Another popular and effective means of creating deepfakes arises from GANs (Goodfellow et al., 2020), which consist of two neural networks, one used to generate data and the other used to distinguish fake data from real data, thereby making it adversarial in nature. In video footage of humans speaking, the subject will make a predictable range of lip, jaw, and head movements corresponding to the sounds made to generate words. A GAN takes original footage to train both models and sets up a competition between the two neural networks wherein each improves based on the output of the other. Continuous improvement to GANs has resulted in gradually more impressive feats, such as videos synthesized that realistically depict humans speaking, software that enables anyone with a webcam to generate replicas of other people’s facial expressions, and “self-re-enactment” videos rending a speaker’s facial expressions in real time (Rana and Sung, 2020).
The recent rise in deepfake quality and prevalence can be attributed to two factors: 1) publicly accessible large-scale data sets, providing source material from which deepfake algorithms can be trained, and 2) the continued advancement of deep learning techniques, such as autoencoders and GANs, that have made deepfakes easier to create through the elimination of many manual editing steps (Ajder et al., 2019). Huge amounts of publicly available footage of actors, politicians, and public figures provide ample data for these models to work from and generate high-quality deepfakes. Additional work in artificial intelligence can be used to synthesize high-quality audio, mimicking the sound of human voices, which, when paired with video results, has the potential to create highly believable clips of anyone who has published a significant amount of publicly available content. This rapidly evolving technology has led to a variety of possible photo and video manipulations.
For many reasons, fake and believable content that can be created by anyone poses a significant threat to individual and societal security and privacy. One extremely troubling fact about deepfakes is that most tend to be pornographic in nature. According to a study in 2019 that assessed the content of 14,678 deepfake videos available on the Internet, 96% of those were pornographic (Ajder et al., 2019). Of the 96% of deepfakes available depicting nonconsenting individuals in pornographic situations, 90% of those individuals were women.
The potential for harm in this scenario is obvious and can be evidenced by the struggles of one then-18-year-old woman, who was nonconsensually placed into hundreds of explicit images and videos of a porn actress (Melville, 2019). This type of content being publicly available has put her reputation, career, and personal safety at risk, and anyone with enough publicly available imagery of themselves could be put into the same situation. Deepfakes of nonconsenting individuals could also be used as a means of coercion, intimidation, or defamation. Legal action against deepfake porn has been taken in some states, such as Virginia, which has prohibited harassment via the sharing of sexual images (Brown, 2019); however, this does not protect against all fake content, and Virginia is one of very few states to enact such laws.
Additional concerns regarding the potentially negative effects of deepfakes arise in the form of fake news and public deception. Public trust in the news across the world is declining, and the continued distribution of fake news via deepfakes may potentially lead to greater public distrust. As false news is shared at an alarming rate online, a shared mentality that anything is acceptable online has become increasingly prevalent, and people have become wary enough of news online that they have started to avoid it altogether to avoid uncertainty.
However, uncertainty does not necessarily imply distrust, and many have argued that some skepticism when it comes to news may reduce susceptibility to misinformation. One study from 2020 sought to examine whether a group of individuals would express greater distrust in news shared on social media after exposure to a deepfake news clip when compared to a group not exposed to a deepfake. It was found that the deepfake group did exhibit significantly more distrust in news on social media (Vaccari and Chadwick, 2020). If the public begins to feel they cannot trust the news, it will become increasingly difficult to acquire what is considered real and meaningful information on political and social events, leading to a poorly informed public. Other harmful uses for deepfakes and their impact on society are listed in Table 1.
Table 1. Harmful uses for deepfakes and their impact at the societal, organizational, and individual levels.
While there clearly are many malicious uses for deepfakes, they do not come without some benefits. Medically, deepfakes have been recognized as an up-and-coming technology with a high therapeutic and biomedical impact. Some examples include (Jaiman, 2020) the following:
These are all in addition to the various uses that deepfakes have in media and art, such as improvements to computer-generated scenes in film, reaching a broader audience by placing different people in various scenes in video clips, creating deepfakes for humorous or satirical purposes, and various art installments (Jaiman, 2020). Although deepfakes do come with various risks, the technology already exists and is publicly available, and it offers some serious benefits to society. To fully take advantage of this rapidly evolving technology, it is essential that we learn to live safely with deepfakes.
Given the serious destructive potential of deepfakes, measures should be—and have been—taken to mitigate the risks associated with them. Significant effort has been dedicated to designing algorithms with the capability to detect deepfakes. Additionally, legal measures have been taken to protect people from the harm of deepfakes. To combat the various negative consequences, a proactive legal and computational approach to detecting and exposing deepfakes should be employed.
As previously discussed, high-quality deepfakes can easily deceive the naked eye (Korshunovf and Sebastien, 2020). As such, effort has been dedicated both to developing algorithms to detect deepfakes and to increasing the availability of datasets to researchers, so as to assist in creating these mechanisms.
The FaceForensics++ dataset, partly developed by Google, contains real and deepfake datasets to aid researchers, and it is one of the standard datasets in the development of deepfake-detection mechanisms.
Current approaches to deepfake detection can broadly be categorized as 1) deep learning-based approaches and 2) analytical approaches. Facebook recently launched a deepfake-detection challenge with prize money of US$1 million to encourage research in the field. Despite these efforts, generally, deepfake-detection methods achieve high accuracy detecting deepfakes in a single dataset but show significantly lower accuracy on deepfakes utilizing newer methods not introduced during the training phase (Shahroz et al., 2021). While existing algorithms are not perfect at detecting deepfakes, a study from April 2021 was able to achieve 93.86% accuracy at detecting deepfakes on a set of videos from which the model was not trained (Shahroz et al., 2021). With continued improvements in technology like this, it will be possible to detect deepfakes with greater confidence. It is, however, important that deepfake detectors evolve with the newly emerging methods of creating deepfakes.
One of the potentially most frightening aspects of deepfakes is that, in many U.S. states and under many conditions, there currently exist no legal consequences for the creation and distribution of deepfakes. Social media only amplifies this problem. As it stands, platforms, such as Facebook and YouTube, are not held liable for the content on their sites, including deepfakes. This issue has been of great recent interest, as currently existing legislation dictates that institutions, such as bookstores, that knowingly distribute books containing libelous content but fail to act can be held legally responsible, but distributor liability for technology platforms does not apply. To make matters worse, the creation and distribution of deepfakes is currently only legally prohibited under specific circumstances, such as nonconsensual pornography, and in certain states, including Virginia, Texas, California, and New York (Brown, 2019). Taken together, this information is extremely concerning, as there exists potential for anyone to create deepfakes of another individual without their consent and distribute deepfake content on technology platforms that do not face any legal consequences for permitting such material. Some additional measures are currently being taken by the U.S. government in the form of advancing legislation to create a task force to identify fake content and verify its origins called the Deepfake Task Force Act (Portman, 2021). Hopefully, as public awareness of this issue grows, people will become more wary of what they trust online, verify the origins of online content, and advocate for increased legal measures to protect against deepfake creation and distribution.
Deepfakes pose a serious threat to individual safety, public opinion, and political attitudes. A relatively nascent technology, deepfakes only first appeared in 2017 on the platform Reddit, and, since then, this technology has rapidly advanced, leading to more realistic deepfakes and more accessible means to create them. Through the use of publicly available algorithmic strategies, such as autoencoders and GANs, deepfake technology has progressed at staggering rates, and these technologies have been employed in various deepfake generators. With this technology, unknowing victims can be depicted in potentially harmful situations, such as pornography; politicians can be made to appear in scenes or saying things that will have serious consequences for the society at large; and fake news can be created and distributed to sway the public. Conversely, various benefits arise from deepfake technology, such as advances in art, film, and medicine. Overall, we should strive to maximize the benefits and minimize the harm posed by deepfakes by taking a proactive approach to identifying false content, verifying the origins of online information, and advocating for increased legal measures to hold individuals and platforms accountable for the creation and distribution of harmful deepfake content.
• P. D. Rietz, “Svensk datorhistoria – 1960-talet,” Tekniska museet, Stockholm, Sweden, Dec. 2016. [Online] . Available: https://www.tekniskamuseet.se/lar-dig-mer/datorhistoria/svensk-datorhistoria-1960-talet/
• P. Korshunovf and M. Sebastien, “Deepfake detection: Humans vs. machines,” 2020, arXiv:2009.03155.
• A. Zucconi. (2019). An introduction to neural networks and autoencoders. [Online] . Available: https://www.alanzucconi.com/2018/03/14/an-introduction-to-autoencoders/
• I. Goodfellow et al., “Generative adversarial networks,” Commun. ACM, vol. 63, no. 11, pp. 139–144, Nov. 2020, doi: 10.1145/3422622.
• M. S. Rana and A. H. Sung, “DeepfakeStack: A deep ensemble-based learning technique for deepfake detection,” in Proc. 7th IEEE Int. Conf. Cyber Secur. Cloud Comput. (CSCloud)/6th IEEE Int. Conf. Edge Comput. Scalable Cloud (EdgeCom), 2020, pp. 70–75, doi: 10.1109/CSCloud-EdgeCom49738.2020.00021.
• H. Ajder, G. Patrini, F. Cavalli, and L. Cullen, The State of Deepfakes: Landscape, Threats, and Impact, vol. 27. Amsterdam, The Netherlands: Deeptrace, Sep. 2019.
• K. Melville, “The insidious rise of deepfake porn videos and one woman who won’t be silenced,” ABC News Aust., Aug. 2019. [Online] . Available: https://www.abc.net.au/news/2019-08-30/deepfake-revenge-porn-noelle-martin-story-of-image-based-abuse/11437774
• N. I. Brown, “Congress wants to solve deepfakes by 2020. That should worry us,” Slate Mag., Jul. 2019. Accessed: Oct. 16, 2019. [Online] . Available: https://slate.com/technology/2019/07/congress-deepfake-regulation-230-2020.html
• C. Vaccari and A. Chadwick, “Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news,” Social Media Soc., vol. 6, no. 1, Jan./Mar. 2020, Art. no. 2056305120903408, doi: 10.1177/2056305120903408.
• A. Jaiman, “Positive use cases of synthetic media (aka deepfakes),” Medium, Aug. 2020. [Online] . Available: https://towardsdatascience.com/positive-use-cases-of-deepfakes-49f510056387#:∼:text=AI%2DGenerated%20Synthetic%20Media%2C%20aka,criminal%20forensics%2C%20and%20artistic%20expression
• S. Tariq, S. Lee, and S. Woo, “One detector to rule them all: Towards a general deepfake attack detection framework,” in Proc. Web Conf., Apr. 2021, pp. 3625–3637, doi: 10.1145/3442381.3449809.
• R. Portman. 117th CONGRESS (2021-2022). (2021, Aug. 4). S.2559 - Deepfake Task Force Act. Congress.gov. [Online] . Available: https://www.congress.gov/bill/117th-congress/senate-bill/2559?q=%7B%22search%22%3A%5B%22S.2559%22%5D%7D&s=1&r=1
Jennifer A. Fehring (j.fehring@northeastern.edu) earned her B.S. degree from Carleton College in Northfield Minnesota. She is currently a master’s degree student at Khoury College of Computer Sciences, Northeastern University, Boston, MA USA. Her research interests are primarily in privacy and security.
Tamara Bonaci (t.bonaci@northeastern.edu) earned her B.S. degree from the University of Zagreb and her M.S. and Ph.D. degrees from the University of Washington. She is an assistant teaching professor at Khoury College of Computer Sciences, Northeastern University, Boston, MA USA, and an affiliate assistant professor in the Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, USA. Her research interests include security, privacy, and the societal impact of emerging technologies, with a special interest in biomedical technologies.
Digital Object Identifier 10.1109/MPOT.2022.3229823