Quantifying the Effectiveness of Student Organization Activities using Natural Language Processing
Project Overview
The document explores the integration of generative AI in education through a machine learning workflow that leverages Natural Language Processing (NLP) and sentiment analysis to evaluate student-organized extracurricular activities in higher education. Utilizing the BERT large language model along with the pysentimiento toolkit, the study quantifies students' emotional responses to these activities, aiming to provide valuable insights into their effectiveness and highlight potential areas for enhancement. The findings are anticipated to assist student organizations and student affairs offices in optimizing their initiatives while contributing to the broader understanding of AI applications within educational settings. Overall, this research exemplifies how generative AI can be harnessed to analyze and improve student engagement and satisfaction in extracurricular contexts, showcasing its transformative potential in the educational landscape.
Key Applications
Machine learning workflow for sentiment analysis using BERT LLM via pysentimiento toolkit
Context: Higher education; specifically targeting student organizations and student affairs offices at universities.
Implementation: Developed a workflow to analyze sentiment from post-activity reports generated by student organizations, utilizing data preprocessing, LLM feature processing, and score aggregation.
Outcomes: Provided a quantitative Event Score for student-organized activities, enabling a better understanding of their effectiveness and areas for improvement.
Challenges: Potential biases in feedback from organizers, lack of existing metrics for evaluating activities, and resource constraints faced by student organizations.
Implementation Barriers
Technical barrier
Challenges in accurately quantifying the effectiveness of student-organized activities due to lack of established metrics and evaluation techniques. Additionally, potential biases in feedback provided by event organizers may skew the results of effectiveness assessments.
Proposed Solutions: Developing a machine learning-based workflow for data collection and analysis that incorporates both organizer and attendee feedback. Utilizing sentiment analysis to analyze open-ended comments for a more nuanced understanding.
Resource barrier
Limited time and resources available to student organizations for assessing the effectiveness of their activities.
Proposed Solutions: Automating the data collection and evaluation process through machine learning to reduce the burden on student organizations.
Project Team
Lyberius Ennio F. Taruc
Researcher
Arvin R. De La Cruz
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Lyberius Ennio F. Taruc, Arvin R. De La Cruz
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai