"Would You Want an AI Tutor?" Understanding Stakeholder Perceptions of LLM-based Systems in the Classroom

AI-Powered Tutoring and Teaching Assistance

Context: Implemented in various educational settings including elementary and high school classrooms, particularly in New York City, and for teacher training in Northern Brooklyn.

Implementation: Utilizes large language models (LLMs) and AI tools to provide personalized tutoring and feedback to students. This includes Khanmigo from Khan Academy, Microsoft's AI teaching assistant, and YourWai AI teaching assistant. These tools are integrated into curricula and used in training sessions for teachers.

Outcomes: Increased access to personalized tutoring and support for students, enhanced teacher capability to integrate AI into teaching practices, and improved student engagement. Positive feedback regarding the knowledge base and accessibility of these tools.

Challenges: Concerns about over-reliance on AI, accuracy of responses, data privacy, potential biases, and lack of human interaction.

AI Reading Assessment and Tutoring

Context: Proposed for use in NYC public schools to assess and support students' reading abilities, targeting 162 schools.

Implementation: Amira, an AI reading tutor designed to assess students' reading capabilities, with a focus on privacy safeguards.

Outcomes: Intended to support literacy development for 46,000 students, aiming for increased reading proficiency.

Challenges: Proposal halted due to privacy concerns and lack of clear policies.

Assessment of Attitudes towards AI

Context: Conducted among K-12 and undergraduate students to evaluate their perceptions of AI as a learning tool.

Implementation: Includes various scales such as the Attitudes Towards Using ChatGPT (ATUC), General Attitudes towards AI Scale (GAAIS), Feedback Perceptions Questionnaire (FPQ), and Student Attitudes Toward Artificial Intelligence (SATAI) scale, which measure students' attitudes, perceptions of feedback, and cognitive, affective, and behavioral factors related to AI.

Outcomes: Provides insights into students' attitudes towards AI utility, ethical concerns, and the impact of feedback on motivation and learning.

Challenges: Potential biases in self-reported data, varying familiarity with AI tools, complexity of attitudes, subjectivity in feedback interpretation, and age-related differences in understanding assessment items.

Privacy Concerns

Concerns over the handling of student data and privacy in AI applications.

Proposed Solutions: Establishing clearer policies and safeguards to protect student information.

Accuracy and Reliability

Concerns about the accuracy and trustworthiness of AI-generated responses.

Proposed Solutions: Continuous monitoring and evaluation of AI tools to ensure their effectiveness.

Human Interaction and Perception Barrier

Concerns about reduced human contact and interaction in learning environments, as well as students' anxiety or negative feelings towards AI, affecting their willingness to engage with AI tools.

Proposed Solutions: Balancing AI use with traditional teaching methods to maintain human engagement and implementing educational programs to familiarize students with AI and address their concerns.

Technical Barrier

Limited access to technology or AI tools in some educational contexts.

Proposed Solutions: Ensure equitable access to technology and provide necessary training.

Implementation Barrier

Challenges in integrating AI tools into existing curricula.

Proposed Solutions: Develop clear frameworks and support for educators to incorporate AI into their teaching strategies.

Project Contact: Dr. Jianhua Yang

LLM Model Version: gpt-4o-mini-2024-07-18

Analysis Provider: Openai