Article Lead Image

Photo via David Goehring/Flickr Remix by Max Fleishman

What the FBI isn’t saying about its facial recognition capabilities

'There may or may not be more databases ... that the FBI may or may not be accessing.'


Lauren Walker


Posted on Jun 16, 2016   Updated on May 26, 2021, 2:34 pm CDT

He chose his words carefully.

Nearly a year ago, I spoke with Stephen Morris, assistant director of the FBI’s Criminal Justice Information Services division (CJIS), about the mounting concerns surrounding the bureau’s facial recognition database, like the number of images, their origins, and the people they contain. “The face pictures we maintain in NGI (Next Generation Identification system) are mugshot photographs [that] come from time of arrest,” Morris insisted, adding that there are “about 24 million images” in the database.

Morris’s words were mostly true; over 80 percent of the photographs in NGI are criminal. But his response obfuscated a startling reality exposed in a new Government Accountability Office (GAO) report published on Wednesday. Not only is the FBI sifting through at least 411 million additional facial photographs—contained in databases outside of NGI—to find a set of possible matches to suspected criminals, but many of those images are culled from driver’s licenses, as well as passport and visa applications.

The inclusion of civil photographs alone poses a huge concern for civil liberties advocates. “One of our biggest concerns about NGI has been the fact that it will include non-criminal as well as criminal face images,” Jennifer Lynch, a senior staff attorney with Electronic Frontier Foundation (EFF), wrote in 2014, “and you could be implicated as a criminal suspect.” Further, the GAO report found that when it comes to the system’s accuracy, the FBI conducted limited testing at best.

“There may or may not be more databases that are on the classified side that the FBI may or may not be accessing.”

When the bureau first tested NGI’s precision, according to the GAO, it inputed a suspect’s picture into a test database of 926,000 photos and asked for a list of 50 potential matches. In that scenario, the FBI determined NGI returned a match in 86 percent of cases. But NGI alone contains nearly 30 million images, according to the report, and the system’s users, such as select state and local law enforcement, are able to request candidate lists as small as two people. This means that the FBI has “not assessed the accuracy of face recognition searches … in its operational setting,” the report reads. Yet, the full system has been in place for nearly two years.

Possibly more troubling for civil liberties advocates than what the report contains is what it does not. The 411 million images held in databases populated by the State Department, Defense Department, and 16 states, according to Diana Maurer, Director for Justice and Law Enforcement Issues at GAO, only represent what the FBI showed the GAO during its assessment.

“There may or may not be more databases that are on the classified side that the FBI may or may not be accessing,” Maurer told the Daily Dot by phone. “It is always a possibility.”

In fact, in speaking to FBI CJIS privacy attorney Roxane Panarella nearly a year ago about NGI’s facial recognition database, she said that it represents just one place that agents search for matches. So how many libraries does the FBI have? “Oh, who knows,” she said, followed by some laughter. “There might be hundreds, or thousands, but there will only be some that are valuable to the FBI and some that are going to be legally allowed to be searched by us.”

The GAO report charges that the FBI dodged its legal responsibility of publicly addressing its face recognition capabilities by not publishing a System of Records Notice (SORN) until May 2016. Similarly, the FBI did not release a Privacy Impact Assessment (PIA) until May—years after the system was already in use. The FBI does not see it that way, according to Maurer, who said the FBI officials believe the PIA released in 2008 for a previous iteration of NGI suffices. Considering the time lag for public notification, the careful word selection that obscures the entire picture, and the nebulous legal framework by which the bureau considers itself to be bound, the public should also consider the following:

Much of the information contained in the GAO report rests on what the FBI told those doing the assessment. For instance, civil liberties groups worry that the FBI may be saving social media images in its system because “there are no legal or even written FBI policy restrictions in place to prevent this from occurring,” Lynch of EFF told RT in 2014. The GAO report states, “According to the FBI, the external photo databases do not contain privately obtained photos or photos from social media, and the FBI does not maintain these photos; it only searches against them.”

The GAO’s fed line about social media echoes what Morris of the FBI said in conversation nearly a year ago. “Any collection of facial images or digital imagery,” such as from social media sites or surveillance cameras, “if it is being collected, it is being collected pursuant to authorized investigative purposes,” he said. “And even then, there’s limits to how that can be used and searched.”

Lockheed Martin, the mammoth security and aerospace corporation, won the $1 billion contract nearly a decade ago to overhaul the bureau’s outdated Integrated Automated Fingerprint Identification System and overlay it with NGI, which is currently replete with finger prints, palm prints, and iris scans, in addition to facial images. But more is likely on the way.

“We did build a system that is ready for additional plug-ins for additional biometric modalities or algorithms,” Art Ibers, Vice President of Exploration and Mission Support at Lockheed Martin, told me nearly a year ago. “You could add a voice modality or a DNA modality; those are the most popular items discussed.”

While Lockheed has employees supporting NGI’s ongoing maintenance, and would be willing to build additional biometric algorithms, Ibers is confident the FBI could do it in-house. “I have a lot of respect for the FBI and their CJIS division in terms of their technical expertise,” he said, adding that if FBI were to add new biometrics, Ibers wouldn’t necessarily know about it.

“Let’s just say three, four, five years down the road there is a new biometric modality that comes into play,” Morris said last year, giving examples like “the way you walk, human scent, voice, shapes of ears,” the FBI will “get involved in deciding whether we should integrate it.”

Maurer suggests the GAO report, which came at the request of Sen. Al Franken (D-Minn.), was long overdue. But FBI spokesman Christopher Allen assured the Daily Dot last month that the Bureau is in strict adherence to privacy laws, as the Privacy Act already contains exemptions for intelligence and law enforcement agencies’ records systems.

In reaction to the GAO report’s privacy and accuracy concerns, the Justice Department said in a statement that “the FBI believes GAO staff does not fully appreciate the nature of its face recognition service as being utilized for investigative leads only and not positive identifications.”

Maurer says that there are currently no legislative mandates or congressional requests for further GAO assessment of NGI or its various capabilities. 

Share this article
*First Published: Jun 16, 2016, 5:09 pm CDT