Experts are raising doubts that artificial intelligence was used to create a video that police are calling a “deepfake” and is at the center of an ongoing legal battle playing out in Pennsylvania.
The incident made international news last month after the mother of a high school cheerleader was accused of manipulating images and video in an effort to make it appear as if her daughter’s rivals were drinking, smoking, and posing nude.
The mother, 50-year-old Raffaela Spone, reportedly sent the doctored content from an anonymous number to three cheerleaders and their coaches.
The Bucks County District Attorney’s Office, which charged Spone with three counts of cyber harassment of a child and three counts of harassment, referenced the term deepfake when discussing the images as well as a video that depicted one of the alleged victim’s vaping.
But the accusation is raising serious questions about law enforcement’s understanding of deepfake technology, and the media’s willingness to repeat their claims without verification.
While prosecutors have not provided specifics, allegations surrounding manipulated photos in the case could be explained by widely-reported tools designed to make women appear nude in images with the aid of artificial intelligence (AI). But the alleged vaping video has spurred significant pushback from deepfake experts that spoke with the Daily Dot.
Although the alleged deepfake was initially unavailable to the public, a cellphone recording of another device playing the vaping footage was provided to NBC News last month after one of the alleged victims came forward.
Madi Hame, a cheerleader with the Victory Vipers squad who was 17-year-old at the time of the alleged harassment, denied to NBC News that the footage actually showed her vaping. Hame says she became aware of the video after it was sent, allegedly by Spone, to her cheerleading coaches.
“I thought if I said it, no one would believe me because obviously, there’s proof, there’s a video—but obviously that video was manipulated,” Hame told NBC News.
The Daily Dot reached out to Hame’s mother Jennifer, who appeared alongside Hame in the NBC News interview, to inquire about the vaping video, but did not receive a reply by press time.
Henry Ajder, a U.K.-based deepfake and synthetic media expert, was one of the first to raise questions about the vaping video after ABC News aired a similar report.
“I had my reservations about the story of a mom making deepfakes of her daughter’s cheerleading rivals,” Ajder tweeted last month. “One of the videos released by ABC doesn’t appear to be a deepfake, but they still label it as one without substantial proof or analysis.”
Other deepfake experts appear to have the same concerns. The popular deepfake creator known online by the nickname Dr. Fakenstein—who currently works as an AI artist for the comedy show Sassy Justice developed by South Park creators Trey Parker and Matt Stone—told the Daily Dot that the vaping video did not appear to be artificially-generated.
Specifically, Dr. Fakenstein noted that the video lacked certain glitches that almost always appear in deepfakes when a face is obstructed by another object.
“It’s hard to see as it’s a video of a video but, it doesn’t look like a deepfake to me,” they said. “The light is shifting like it’s natural, no glitching around where the hand covers the face, no glitching when the vape cloud goes in front of the face, which is very hard if not impossible to do without glitches.”
The Daily Dot reached out to the Hilltown Township Police Department, which investigated and later carried out the arrest against Spone, to inquire about the vaping video.
Police Chief Christopher Engelhart stated that he believed the charging documents against Spone did not definitively label the video as a deepfake.
“My understanding is it was listed as either a deepfake or similar technology,” Engelhart said.
In the criminal complaint against Spone, Hilltown Township Police officer Matthew Reiss refers to the vaping video as “the work of a program that is or is similar to ‘Deep Fakes.'”
When pressed about whether the department had entertained the possibility that the vaping video might not be a deepfake at all, Engelhart claimed that investigators had conducted a forensic analysis of the video.
“The assertion is that that victim was not vaping,” Engelhart added. “The assertion is that she [Spone] was able to manipulate that to have it appear that she was vaping.”
But experts are skeptical that even the best deepfake creators could produce such a convincing video, let alone Spone. When asked whether the 50-year-old mother had potentially elicited help from an expert deepfake creator, Engelhart stated that while it is a possibility, no evidence at this time indicates that Spone had an accomplice.
Derpfakes, another well-known deepfake artist whose creations have garnered millions of views online, also viewed the vaping video as presented by NBC News and told the Daily Dot that they couldn’t say for certain that it had been manipulated.
“I would lean towards saying it’s authentic,” Derpfakes said. “If it is a deepfake then it certainly took a certain level of visual effects skill rather than out-of-the-box deepfake software.”
During a press conference last month, Bucks County District Attorney Matt Weintraub seemed to suggest that the vaping video was based on existing footage from the victim’s social media accounts.
“The videos or the images already existed in some form and then our allegation—this is of course subject to proof at trial, she’s innocent until proven guilty—but our allegation is that Mrs. Spone took existing images from existing social media from these three victims’ existing social media accounts and manipulated them,” Weintraub said.
The statement is similar to claims made by Engelhart to the Daily Dot that Spone allegedly used legitimate video as a basis for the deepfake. Police could not answer questions on whether the alleged victim’s face was added to a video of another young female vaping or if a vaping pen and smoke was digitally added to a legitimate video of the alleged victim.
Regardless, experts are skeptical that either scenario is an accurate representation of the footage.
The prominent deepfake creator known as Shamook, whose videos have been viewed more than 21 million times on YouTube alone, argued that although the resolution of the vaping video is “very low, it’s just too clean to be a deepfake.”
“From my perspective it isn’t a deepfake,” Shamook told the Daily Dot. “The smoke from the vape would be too hard to create… I just can’t see any plausible evidence that it’s a deepfake.”
Weintraub also stated during the press conference that investigators determined that the vaping video had been manipulated after analyzing its “metadata,” a term which refers to embedded information in digital media that can reveal how and when it was last edited.
But after contacting Reiss, the officer who investigated Spone, the Daily Dot learned that police never actually obtained the original vaping video. Instead, like what was seen on NBC News, police only had access to a cellphone recording taken by Spone of the vaping video being played on a separate device. Any metadata analysis would therefore fail to include information on the source video.
In response to questions on how it could have been determined that the vaping footage was manipulated without access to the original video, Reiss argued that he could see with his “naked eye” elements that “don’t make sense.”
“The source of where Mrs. Spone came up with that video is still a question,” Reiss said. “And we hope that Mrs. Spone during the course of the preliminary hearing or trial will enlighten us as far as what her source and intent was.”
Reiss stated definitively that “artificial intelligence was used” in the photos that made several cheerleaders appear nude, which would align with current technology designed to do just that. But the existence of those allegedly altered photos, in addition to the denial from the alleged victim regarding her apparent inclusion in the vaping video, seems to be the entire basis for the belief that the footage was manipulated, even though no technical evidence thus far supports the claim.
The word “deepfake,” as mentioned in the charging documents, seems to have become a catch-all term among police, the district attorney’s office, and the media when discussing any and all digital content related to the incident.
Spone’s next court hearing is set to take place on May 14, where Reiss says more information about the investigation will be made public.
While new evidence could alter the case, the notion that police may be incorrectly referring to an authentic video as manipulated is not out of the realm of possibility.
Even more troublesome, it seems that no media outlets attempted to verify that the vaping footage was actually a deepfake before broadcasting the claim to millions of Americans.
The possible misstep by the media, and the chance that the criminal justice system could base its case on false references to deepfakes, is what worries experts the most.
“Scary though if it isn’t a deepfake,” Shamook added. “This could be the start of people being able to discredit real evidence by simply saying ‘deepfake.'”
The vaping video aside, the other allegations against Spone, if true, are still fraught and frightening. But the aftermath of the case against her, if reliant even in part on inaccurate information, could have dangerous ramifications in the world of manipulated (and non-manipulated) media.