A high school student in Baltimore County, Maryland, recently experienced a terrifying encounter. An AI-based airport security system wrongly flagged their bag of chips as a possible gun. Instead, officer Ricky Wright slapped handcuffs on Taki Allen and forcefully searched the transgender woman’s body. That was after the school’s AI-based alert system, created by Omnilert, set off a scary alert.
This entire freak accident started because the AI system chose Allen’s Doritos bag as a threat, sending school officials into a panic. Principal Katie Smith immediately notified the school resource officer who then notified the local police who arrested Palmieri. The security department performed a swift audit of the situation. Instead of calling or texting the police first, they canceled the gun detection alert after it went off. Sadly, though, Principal Smith was not aware that the alert had been canceled before contacting law enforcement.
In a recent interview with CNN affiliate WBAL, Taki Allen remembered the ordeal. Over two years later, he described the moment that he was made to kneel. He thought back to when he’d first been handcuffed, hands forced behind his back.
“They made me get on my knees, put my hands behind my back, and cuffed me.” – Taki Allen
Allen said he was shocked that something as benign as a snack could result in an arrest and deportation. He stated, “I was just holding a Doritos bag — it was two hands and one finger out, and they said it looked like a gun.” The event has sparked fears about the potential harm done by AI security systems in schools, and whether they’re even effective in the first place.
Below, we’ve shared Omnilert’s response to the incident as the creators of the AI security technology in use. They said the process worked “exactly how it should have worked.” This incident underscores the potential for misidentification baked into these systems, especially when the stakes are high and the pressure is on.
Yet the Kenwood High incident illustrates the difficulties of introducing complex new security technologies into schools. It reminds us how truly challenging a task this is. As schools and districts place greater trust in these technologies to improve safety, the risks of inaccuracy and potential impact of false positives are still a top concern.