Exploring the Relationship Between Feature Attribution Methods and Model Performance

Predictive modeling for student success and dropout risk

Context: Higher education, including both university settings and online courses, utilizing datasets from various educational platforms to monitor and predict student performance and retention.

Implementation: Employed machine learning techniques, including neural networks, deep learning classifiers, and gradient-boosted trees, to analyze student data and interaction logs. These models predict academic outcomes, including grades and dropout risks, enabling targeted interventions for at-risk students.

Outcomes: ['Identified students at risk of failure and dropout, leading to proactive educational interventions.', 'Enhanced understanding of model predictions to inform teaching strategies.', 'Improved retention rates through timely support for struggling students.']

Challenges: ['Data privacy concerns related to student information.', 'Difficulty in interpreting predictions due to the black-box nature of some models.', 'Data integration from various sources and ensuring data accuracy.']

Automatic grading and feedback on textual student responses

Context: Higher education, particularly in large courses where timely grading and feedback are essential to student learning.

Implementation: Utilized text mining and machine learning methods to automatically grade student responses and provide feedback on textual answers, enhancing the efficiency of the grading process.

Outcomes: ['Improved grading efficiency and consistency.', 'Provided timely and constructive feedback to students.', 'Reduced the grading burden on instructors.']

Challenges: ['Accuracy of grading may vary based on the complexity of responses.', 'The requirement for extensive training data and potential biases in the model.']

Technical Barrier

The black-box nature of machine learning models makes them difficult to interpret, leading to challenges in understanding how decisions are made. Additionally, there is a need for transparency in AI models to gain trust from educators and students.

Proposed Solutions: Adoption of explainable AI techniques and frameworks to enhance transparency in predictions and develop explainable AI frameworks that clarify how decisions are made.

Research Barrier

The disagreement problem between different explanation methods can undermine the trustworthiness of insights derived from models.

Proposed Solutions: Further research into the consistency of explanation methods and their reliability in educational contexts.

Data Privacy

Concerns regarding the collection and use of sensitive student data.

Proposed Solutions: Implementing robust data protection policies and obtaining informed consent from students.

Bias in AI

Risk of bias in AI models affecting grading and feedback.

Proposed Solutions: Regular audits of AI systems and incorporating diverse training datasets.

Project Contact: Dr. Jianhua Yang

LLM Model Version: gpt-4o-mini-2024-07-18

Analysis Provider: Openai