On October 24th, 16-year-old Taki Allen from Kenwood High School was handcuffed by armed police after an AI security system mistakenly alerted that he was carrying a gun; when in reality, he was holding a Doritos bag. It was said that the AI alert was sent to human reviewers who saw that no threat was present and sent this information to the principal. The principal missed this message and contacted the School Resource Officer, who was the one who called the police. This incident has caused many people to criticize the school’s procedures around the use of such technology in the school environment.
Student Taki Allen told WMAR-2 News that he had finished a bag of Doritos after football practice and put the empty bag in his pocket. Twenty mins later, armed police were on campus.
When interviewed by the news outlet, he states, “Police showed up, like eight cop cars, and then they all came out with guns pointed at me, talking about getting on the ground.”
“They made me get on my knees, put my hands behind my back, and cuff me. Then, they searched me, and they figured out I had nothing.”
The Baltimore County Police Department responded to the situation by saying that the officers on the scene “responded appropriately and proportionally based on the information provided at the time.” The department told BBC News that Taki Allen was handcuffed but not arrested, going on to say, “The incident was safely resolved after it was determined there was no threat.”
Taki Allen expressed that he doesn’t think it is “safe enough to go outside, especially eating a bag of chips or drinking something.”
In a letter to his parents, the school’s principal, Kate Smith, said that the school’s security team “quickly reviewed and canceled the initial alert after confirming there was no weapon.”
She explains that, “I contacted our school’s resource officers and reported the matter to him, and he contacted the local precinct for additional support.”
“Police officers responded to the school, searched the individuals, and quickly confirmed they were not in possession of any weapons.”
“Nobody wants this to happen to their child,“ Allen’s grandfather, Lamont Davis, explained to a news outlet.
Local politicians and many people in the community have called for further investigation into this incident, taking to Facebook to demand that more action be taken.
Omniler, the school’s weapon detection system, is an AI gun detection and emergency response technology company that “focuses on real, diverse, and representative footage rather than relying on synthetic or greenscreen-generated images.”
The system uses school cameras to identify possible weapons and send alerts to authorities.
An Omnilert spokesperson said the image that was sent to their team that Monday night closely resembled a gun. Omniler told BBC News, “We regret this incident occurred and wish to convey our concern to the student and the wider community by the events that followed.”
The company said its system first detected what appears to be a firearm, and an image of it was verified by its review team. The security firm said that its involvement with the incident ended once it was marked as resolved by the system, stating that it “operated as designed” and “the process functioned as intended: to prioritize safety and awareness through rapid human verification.”
What does this mean for safety on school campuses?
AI security systems in schools offer potential benefits like faster threat detection and proactive interventions. Events like this raise significant concerns regarding students’ privacy and safety on campus. With an over-reliance on AI, especially in the context of security on campuses, it poses questions about how to efficiently deal with false positives in order to ensure inaccurate reports don’t occur and cause disruption for student life.
The potential that AI can perpetuate and even amplify existing societal biases, leading to unfair or discriminatory outcomes, is concerning and can allow students to feel unsafe on campuses. There are many ethical and environmental concerns about the use of AI, and it is a very controversial topic, particularly in academic institutions.
This, combined with AI not being 100% accurate, allows for major errors to occur, which could be life-threatening to students. Whether we like it or not, AI is the future; however, it is too early in its development to be able to accurately handle tasks as complex as school security.
