The use of facial recognition technology in US schools is fast becoming a reality. But any protections it could provide to children could also come with many harms, especially for children of color.
In Washington State, two private schools have already adopted facial recognition technology. And in New York, the public Lockport City School District recently committed to having a facial recognition system installed by the time students return to class in September. The system is intended to detect people on a threat list, such as suspended students and sex offenders, and will scan the faces of everyone on school premises, including all children.
Governments are obligated to protect children’s lives and safety, and this is the Lockport district’s stated goal. However, facial recognition technology comes with a risk of violating people’s rights – a particularly serious risk for both children and adults of color.
Several studies have documented that facial recognition software can be less accurate for people of color and women than they are for white men, with one study finding that systems were inaccurate for up to 35 percent of darker-skinned women. Formal studies of the technology’s accuracy for children appear to be scant – which at a minimum means more research needs to be done. But the risks are real: using inaccurate facial recognition technology in schools could perpetuate racial discrimination against people who have long faced prejudice and exclusion.
For years, civil rights activists fought to establish that people of color have a right to enter or move around in buildings – including schools – precisely as white people do. Yet, with faulty facial recognition technology, children of color could soon find themselves forced to deal with a security guard or be singled out in other ways simply to go to school. The same thing could happen to teachers of color, or families of color who want to see the school play or a basketball game. Imagine a line of excited children and parents – then the potential shunting of up to a third of the Black mothers and grandmothers to one side while the white male family members sail through.
There are other potential risks to children of color if facial recognition systems are deployed throughout buildings, rather than just at the main entrance. If the software is inaccurate for such children, it could misidentify students suspected of fighting, skipping class, or otherwise breaking the rules. Depending on the quality of the image and how vigorously school officials validate the software’s results, the wrong children could be investigated or punished. And the same persistent institutional racism that leads school staff to discipline Black children disproportionately could also prompt them to believe a software’s identification over the child’s own words.
Additionally, as UNICEF, the United Nations children’s agency, has recognized, all children have rights to privacy, free expression, and association. Facial recognition can threaten each of these. Even the knowledge that someone might be watching us can make us censor ourselves. Facial recognition cameras could discourage children from being spontaneous or playful, or from associating with friends or siblings the school regards as troublemakers.
Confronted with serious concerns from parents, officials, and rights activists, the Lockport district reportedly suspended its plans earlier this month. Meanwhile, the New York state legislature is currently considering a bill that would bar schools from using facial recognition until at least 2022, while an official study is conducted.
In the meantime, ambitious companies will likely continue promoting their technology as a purported safety solution for schools. So it’s important that lawmakers create regulations to prevent software from being used in ways that violate the rights of children, including minorities, and that tech companies uphold their responsibility to respect privacy and other fundamental rights. And yes, districts should protect students’ safety – but not by sacrificing their fundamental freedoms.
That would be the wrong lesson.
