The pandemic-era shift to remote learning sparked an acceleration of an already growing trend: education technology companies jockeying to sell schools on systems for monitoring students and filtering their internet browsing. Though school administrators will argue these tools are necessary to ensure kids’ safety online, newly released data shared with Gizmodo highlights numerous privacy concerns and unintended consequences caused by their rapid deployment. Kids and parents across the country are concerned about the new tech, with students from historically marginalized communities bearing the brunt.
The Center for Democracy and Technology (CDT) surveyed over 1,000 high school students, along with 1,000 teachers and parents of middle- and high–school-aged students to gauge their attitudes toward edtech tools. Content filtering tools that prevent users from searching for certain keywords on school devices were ubiquitous, with nearly 100% of teachers surveyed saying their schools use them in some capacity. Students and teachers say those tools are making their lives more difficult.
“Whether old or new, technologies deployed across schools have negative impacts on students, and schools are out of step in addressing rising concerns,” the report reads.
Almost three-quarters of the students surveyed by CDT said these filtering tools made it more difficult for them to complete coursework by blocking access to useful content and information. Teachers agree. Nearly half of those surveyed in the reports said they thought filtering technology left students siloed away from content that “will help them learn as a student” or “grow as a person.”
Both teachers and students say filtering originally intended to target adult content is instead being used by some school administrators to block LGBTQ+ and race-related content they deem “inappropriate.” Disciplinary action and punishment for violations aren’t experienced at the same rates for all students. The report found students who identify as LGBTQ+ and those with disabilities in individualized education programs were more likely to get in trouble due to the tools. 19% of students at schools that use filtering technology say they were even aware of students who were “outed” for LGBTQ+ as a result of the filtering. That’s up six percentage points from the 2021-2022 school year.
“Students at Title I schools, students with disabilities, and LGBTQ+ students continue to bear the brunt of irresponsible data and technology use and policies in the classroom and at home,” CDT Director of the Equity in Civic Technology Project Elizabeth Laird said in a statement. “This is alarming given that schools say they use technologies to keep all students safe and enhance their learning experience.”
AI-enabled activity monitoring software that tracks students’ online activity is similarly altering students’ day-to-day lives. Around two-thirds of teachers surveyed said they’d seen students disciplined at school as a result of AI-powered monitoring software. Shockingly, 38% of teachers said they were even aware of a student who was contacted by law enforcement as a result of the monitoring.
Teachers, students, and parents alike similarly expressed concerns over school administrators’ handling of potentially sensitive student data. More than two-thirds (73%) of parents said they were concerned, a figure up by 12 percentage points from those surveyed a year prior. Despite those concerns, just 31% of the parents say their schools solicited their input for how to responsibly use student data. Recent high-profile cases of data breaches and ransomware attacks targeting school systems may be contributed to these growing anxieties.
Groups demand guidance from Department of Education
A coalition of 19 organizations including the Autistic Self Advocacy Network, Bazelon Center for Mental Health, and the Center for Learner Equity, referenced the new report in a letter sent to The Department of Education, where they called on government officials to issue clearer guidance on ways schools and prevent discrimination against protected classes of students carried out via edtech.
“Schools, and the companies that work with them, are grappling with questions about the responsible use of AI in education,” the organization wrote in a letter shared with Gizmodo. “They would benefit from clarity on how they can fulfill their long-standing civil rights obligations alongside the expansion of AI in the classroom.”
Recent advances in generative AI technologies like ChatGPT and Google Bard are already impacting students’ daily lives. Half of the teachers surveyed by CDT said they were aware of a student who had gotten in trouble for using generative AI to complete assignments while 58% of students admitted to having used generative AI. Tech firms are racing to develop generative AI detection tools to catch those students but some advocates fear they could unintentionally penalize non native English speakers.
“The Department of Education has the authority to clarify, provide guidance, and enforce decades-old civil rights protections to the use of technology in schools,” CDT President and CEPO Alexandra Reeve Givens said. “As we approach the one-year anniversary of the White House’s Blueprint for an AI Bill of Rights, now is the time to act.”