According to confidential documents obtained by the Intercept, IBM has developed object recognition technology capable of identifying people by physical characteristics like skin color with help from the New York Police Department.
With secret access to NYPD-captured videos, IBM developed this technology to allow police to search camera footage for persons with a specific hair color, facial hair type, or skin color, the Intercept reports. The development of this technology dates back to 2012, when counterterrorism officials gained access to skin color-searching capabilities.
NYPD spokesperson Peter Donald said that this feature was only ever used “for evaluation purposes” and when a test version of the tech was available. The NYPD focused its testing on other parameters like clothing colors or characteristics in order to avoid potential allegations of racial profiling. The organization eventually decided against using IBM’s analytics in its surveillance efforts and stopped using the technology in 2016. When approached with an updated version that could identify people by ethnicity around that time, NYPD rejected the product.
Still, the developmental partnership was kept secret, and the technology was developed without public consent, which privacy and civil liberties advocates found concerning.
The potential for facial and object recognition technology to be used (or abused) by police enforcement has been a growing concern of citizens, privacy advocates, and tech company employees. Earlier this year, it came to light that law enforcement agencies had begun testing pilots with Amazon Rekognition, the company’s cloud-based facial recognition technology.
In a damning test meant to highlight the technology’s potential for racial bias, the American Civil Liberties Union used Rekognition on photos of Congress members. It misidentified 28 of the legislators, primarily Black and Latino, as criminals. Amazon said, however, that the ACLU used the technology in an unusual way and not with optimal settings in place.
Amazon workers have since protested against the technology being used by law enforcement arms such as Immigrations and Customs Enforcement.
Former IBM researcher Rick Kjeldsen told the Intercept that IBM’s technology was originally destined for self-driving cars or for use on the internet, but following 9/11, the company realized it had potential in the counterterrorism space. The technology originally focused on object recognition specifically but was later honed to identify persons by age, “head color,” gender, and skin color. Some of the IBM researchers had qualms about the technology if its accuracy improved in the future.
NYPD told the Intercept that its collaboration with IBM was about “finding a way to shorten the time to catch the bad people” after a crime. IBM did not respond to the Intercept’s questions about the collaboration or why it wasn’t made public knowledge.
Read the Intercept’s full report here.