RETUYT-INCO at BEA 2025 Shared Task: How Far Can Lightweight Models Go in AI-powered Tutor Evaluation?
Project Overview
The document highlights the involvement of the RETUYT-INCO team in the 2025 BEA shared task, which centers on evaluating AI-powered tutors. Focusing on lightweight models with fewer than 1 billion parameters, the team sought to generate teacher responses in educational dialogues while navigating resource constraints typical in the Global South. Their findings demonstrate that these resource-efficient models can perform competitively in educational contexts, suggesting a viable path for implementing generative AI in teaching and learning environments. The work underscores the potential of lightweight AI applications to address significant challenges such as high costs and privacy issues prevalent in education, thereby making advanced AI technologies more accessible and effective for diverse educational settings. Overall, the document illustrates the promising role of generative AI in enhancing educational dialogue and tutor evaluation, paving the way for innovative approaches to teaching that are both practical and sustainable.
Key Applications
Lightweight models for educational dialogue generation
Context: Educational contexts, particularly in rural areas with limited resources; target audience includes teachers and students.
Implementation: The RETUYT-INCO team experimented with open models and lightweight models under 1 billion parameters to generate teacher responses in dialogues.
Outcomes: Achieved competitive results in the BEA 2025 shared task, showing that smaller models can perform adequately even in resource-limited environments.
Challenges: Limited computational resources, privacy concerns regarding data processing, and the need for models that can run on low-end hardware.
Implementation Barriers
Resource Barrier
Limited access to computational power and high costs associated with using large models.
Proposed Solutions: Focus on developing and utilizing lightweight models that can run on low-cost GPUs or without GPU.
Privacy Barrier
Concerns about student data privacy when using cloud-based AI models.
Proposed Solutions: Implement models that can operate locally without sending data to external servers.
Performance Barrier
Smaller models may not perform as well as state-of-the-art models used in the task.
Proposed Solutions: Balance between model size and performance while exploring various lightweight model architectures.
Project Team
Santiago Góngora
Researcher
Ignacio Sastre
Researcher
Santiago Robaina
Researcher
Ignacio Remersaro
Researcher
Luis Chiruzzo
Researcher
Aiala Rosá
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Santiago Góngora, Ignacio Sastre, Santiago Robaina, Ignacio Remersaro, Luis Chiruzzo, Aiala Rosá
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai