Learning About Algorithm Auditing in Five Steps: Scaffolding How High School Youth Can Systematically and Critically Evaluate Machine Learning Applications
Project Overview
The document emphasizes the critical role of teaching high school students to evaluate machine learning applications through a structured method known as algorithm auditing. It outlines a five-step process for students to engage in this auditing of AI systems, which involves developing hypotheses, creating input sets, conducting tests, analyzing data, and reporting findings. A case study involving students auditing generative AI TikTok filters illustrates the practical application of this approach, highlighting the necessity for scaffolding and guidance throughout the learning process. Moreover, the document addresses the challenges faced in incorporating algorithm auditing into classroom activities and proposes directions for future research and implementation, underscoring the importance of preparing students to navigate and critically assess the growing presence of AI technologies in their lives. Through these efforts, the document advocates for a more informed and analytical perspective on the use of generative AI in educational contexts, ultimately aiming to enhance students' understanding and engagement with emerging AI tools.
Key Applications
Algorithm Auditing of Generative AI TikTok filters
Context: High school students engaging in after-school workshops
Implementation: Conducted a two-week workshop where students designed and audited TikTok filters using a five-step auditing process
Outcomes: Students learned to systematically evaluate AI applications, developed critical thinking skills, and connected their findings to real-world implications.
Challenges: Difficulty in scaffolding systematic input generation and data analysis; students initially struggled with the ad hoc nature of their observations.
Implementation Barriers
Practical Implementation Barrier
Challenges in accessing AI tools like TikTok's Effect House in school settings due to legal and privacy concerns. Additionally, students may face difficulties in using certain AI applications that are not easily accessible.
Proposed Solutions: Identify alternative AI applications relevant to students' lives or use community spaces for audits.
Scaffolding Barrier
Students found it challenging to systematically generate diverse input datasets for testing. Collaborative approaches for input generation can enhance student engagement.
Proposed Solutions: Explore collaborative approaches for input generation where students can collectively build datasets.
Data Analysis Barrier
Challenges in analyzing data due to the perceived repetitive nature of the task and the need for clearer guidance on statistical analysis.
Proposed Solutions: Provide clearer guidance on statistical analysis and offer tools for data visualization.
Project Team
Luis Morales-Navarro
Researcher
Yasmin B. Kafai
Researcher
Lauren Vogelstein
Researcher
Evelyn Yu
Researcher
Danaë Metaxa
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Luis Morales-Navarro, Yasmin B. Kafai, Lauren Vogelstein, Evelyn Yu, Danaë Metaxa
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai