Studies have shown that facial recognition software can return more false matches for African-Americans than for white people, a sign of what experts call “algorithmic bias.”
DETROIT — Twenty-four hours a day, video from thousands of cameras stationed around Detroit, at gas stations, restaurants, mini-marts, apartment buildings, churches and schools, streams into the Police Department’s downtown headquarters.
The surveillance program, which began in 2016, is the opposite of covert. A flashing green light marks each participating location, and the point of the popular initiative, known as Project Green Light, has been for the cameras to be noticed and help deter crime. Detroit’s mayor, Mike Duggan, received applause when he promised at his State of the City address earlier this year that expanding the network to include several hundred traffic light cameras would allow the police to “track any shooter or carjacker across the city.”
But in recent weeks, a public outcry has erupted over a less-touted tool employed in conjunction with the cameras: software that can, in a matter of seconds, suggest the identities of the anonymous people captured on video.
The facial recognition program matches the faces picked up across the city against 50 million driver’s license photographs and mug shots contained in a Michigan police database. The practice has attracted public attention recently as the department seeks approval for a formal policy governing its use from a civilian oversight board.Video from the cameras streams into the Police Department’s downtown headquarters.CreditBrittany Greeson for The New York Times
“Please, facial recognition software — that’s too far,” pleaded one resident at a recent meeting of the board.
The debate in Detroit is one of several unfolding around the country as rapid advances in facial recognition offer potentially disquieting new powers to a surveillance infrastructure that Americans have largely accepted as a fact of urban life. Immigration officials have mined driver’s license databases in at least three states, according to newly released records. The F.B.I. also routinely uses facial recognition technology to scan state driver’s license databases without the approval or knowledge of the license-holders, which a bipartisan group of lawmakers said last month raises privacy concerns.
In Detroit, whose share of black residents is larger than in any other sizable American city, it is a racial disparity in the performance of facial recognition technology that is a primary source of consternation.
“Facial recognition software proves to be less accurate at identifying people with darker pigmentation,” George Byers II, a black software engineer, told the police board last month. “We live in a major black city. That’s a problem.”
Researchers at the Massachusetts Institute of Technology reported in January that facial recognition software marketed by Amazon misidentified darker-skinned women as men 31 percent of the time. Others have shown that algorithms used in facial recognition return false matches at a higher rate for African-Americans than white people unless explicitly recalibrated for a black population — in which case their failure rate at finding positive matches for white people climbs. That study, posted in May by computer scientists at the Florida Institute of Technology and the University of Notre Dame, suggests that a single algorithm cannot be applied to both groups with equal accuracy.
Mr. Byers and other critics spoke at a public hearing called by the Detroit Board of Police Commissioners after what the board called unprecedented public interest in two facial recognition items on its agenda. One item, specific to the new traffic light cameras, was approved last week. The other, a comprehensive “acceptable use” policy for facial recognition, has yet to be put to a vote.Residents spoke in opposition to the expansion of Project Green Light, which uses facial recognition technology, during a police board meeting in Detroit in June.CreditBrittany Greeson for The New York Times
Gathered in a packed church in the Second Precinct on the city’s west side, those who expressed concerns about what is called “algorithmic bias” included Denzel McCampbell, press secretary to Representative Rashida Tlaib, the Michigan Democrat whose district includes Detroit, and Blair Anderson, a former member of the Black Panther Party who invoked the law enforcement surveillance that helped destroy the political group as a cautionary tale.
Tawana Petty, an activist with the Detroit Community Technology Project, urged fellow Detroiters to consider the city’s place in the national conversation on facial recognition. “If we allow racially biased technologies to succeed here,” she said in an interview, “there really isn’t any hope for black residents anywhere else in the United States.”
Not everyone who spoke was against the use of facial recognition.
“I’m the pastor getting the call from mothers whose son was shot or their baby got snatched up,” said Maurice Hardwick, a black pastor at a nondenominational ministry who founded a group that works with high school gang members. “People want to know two things: What happened to my child, my loved one? And who did this?”
Another Detroit resident, a white woman who walked with a cane, added: “If you’re afraid of the cameras, either you’re paranoid or you’ve got something to hide.”
Others were more concerned with a provision that would allow the police to go beyond identifying violent crime suspects with facial recognition and allow officers to try to identify anyone for whom a “reasonable suspicion” exists that they could provide information relevant to an active criminal investigation. There was also concern that the photograph of anyone who gets a Michigan state ID or driver’s license is searchable by state and local law enforcement agencies, and the F.B.I., likely without their knowledge.
Facial recognition, the Detroit police stress, has indeed helped lead to arrests. In late May, for instance, officers ran a video image through facial recognition after survivors of a shooting directed police officers to a gas station equipped with Green Light cameras where they had met with a man now charged with three counts of first-degree murder and two counts of assault. The lead generated by the software matched the description provided by the witnesses.A map at police headquarters shows the various camera locations throughout the city.CreditBrittany Greeson for The New York Times
In the absence of federal legislation regulating the technology, experts say cities and states are destined to be the first to weigh the societal risks of technology that many law enforcement officials say is critical for ensuring public safety.
As in San Francisco, which this spring became the first major city to block the police from using facial recognition, critics here have argued that facial recognition threatens civil liberties and that the pervasive racial bias in policing will inevitably extend to how it is wielded, not least because African-Americans are disproportionately represented in mug-shot databases.
When James White, an assistant police chief in charge of the Detroit Police Department’s technology, rose to respond to critics at the public hearing, he provided unexpected backup to the charge that the software comes with baked-in bias. He himself, the assistant chief said, had been misidentified as other African-American men by the facial recognition algorithm that Facebook uses to tag photos.
“On the question of false positives — that is absolutely factual, and it’s well-documented,” he said. “So that concerns me as an African-American male.”
The solution, Chief White said, is to exercise extra care. The department’s policy specifies that facial recognition will be used only to investigate violent crimes. Although the department has the ability to implement real-time screening of anyone who passes by a camera — as detailed in a recent report by the Georgetown Law Center on Privacy and Technology — there is no plan to use it, he said, except in extraordinary circumstances.
No one in Detroit, Chief White emphasized, would be arrested solely on the basis of a facial recognition match.A business participating in Project Green Light.CreditBrittany Greeson for The New York Times
“Facial recognition technology isn’t where the work stops,” he said. “It’s where the work starts.”
Civil liberties advocates say that protection isn’t enough, especially because defendants are not typically informed that facial recognition has been used in their identification. In one of the few cases to have argued that such information should be disclosed because it is potentially exonerating, a Florida appeals court ruled that a black man, Willie Allen Lynch, had no legal right to see the other matches returned by the facial recognition program that helped lead to his drug-offense conviction. Mr. Lynch had argued that he was misidentified.
A January 2018 study by two M.I.T. researchers first focused public attention on the higher misidentification rates for dark-skinned women by three leading purveyors of facial recognition algorithms. One of the co-authors, Joy Buolamwini, posted YouTube videos showing the technology misclassifying famous African-American women, like Michelle Obama, as men. The phenomenon, Ms. Buolamwini wrote in a New York Times Op-Ed, is “a reminder that artificial intelligence, often heralded for its potential to change the world, can actually reinforce bias and exclusion, even when it’s used in the most well-intended ways.”
The companies examined in the paper subsequently improved their algorithms for that particular test. But a second paper this year found that Amazon’s software had more trouble identifying the gender of female and darker-skinned faces, prompting prominent artificial-intelligence researchers to call on the company to stop selling its software to law enforcement agencies. Amazon executives have disputed the study.
It is not clear why facial recognition algorithms perform differently on different racial groups, researchers say. One reason may be that the algorithms, which learn to recognize patterns in faces by looking at large numbers of them, are not being trained on a diverse enough array of photographs.
But Kevin Bowyer, a Notre Dame computer scientist, said that was not the case for a study he recently published. Nor is it certain that skin tone is the culprit: Facial structure, hairstyles and other factors may contribute.
In Dr. Bowyer’s experiments, the recognition algorithms could achieve the same degree of accuracy for white and black Americans, but only when the algorithm was tuned to a cutoff, say, of no more than one in 10,000 false matches for the two separate groups. Given that the norm is to use the same threshold for everybody, “those programs are seeing a higher false match rate for the population of African-Americans,” Dr. Bowyer said.
A dual-threshold system would not necessarily solve the problem, he added. That would require law enforcement authorities to make a judgment about each individual’s race and apply the appropriately tweaked facial recognition software — which would in turn introduce human bias.
“Technically, it’s a very reasonable thing to say to do,” Dr. Bowyer said. “But how do you defend it, and once you put that knob out there for police to use, how do you make sure it’s not misused?”