How Useful are Educational Questions Generated by Large Language Models?
Project Overview
The document examines the application of controllable text generation (CTG) through large language models (LLMs) in education, highlighting their ability to generate high-quality educational questions that support teachers by alleviating cognitive load and improving educational material. A human evaluation indicated that educators perceived the generated questions as valuable and effective for classroom use, particularly when aligned with Bloom's taxonomy and varying difficulty levels. The findings underscore that LLMs can produce a diverse array of relevant questions, making them a useful tool for enhancing teaching practices and promoting better learning outcomes in educational settings. Overall, the document illustrates the transformative potential of generative AI in facilitating and enriching the educational experience for both educators and students.
Key Applications
Controllable Text Generation for Question Generation
Context: Utilized in educational settings for teachers in machine learning and biology domains to generate questions from context passages relevant to their subjects.
Implementation: Employed InstructGPT to generate high-quality questions from educational context passages, using control elements such as Bloom's taxonomy to guide the generation process.
Outcomes: Generated questions were rated as high quality and useful for teaching by experienced educators, enhancing the classroom experience.
Challenges: Some generated questions exhibited grammatical issues or failed to adhere to expected pedagogical standards, leading to variability in quality and appropriateness.
Implementation Barriers
Quality Control
Generated questions sometimes contained grammatical errors or were not appropriate for educational purposes.
Proposed Solutions: Implementing filters to remove malformed questions, though this may reduce question diversity.
Assessment of Usefulness
Variability in teachers' perceptions of the usefulness of generated questions.
Proposed Solutions: Conducting broader assessments with more diverse teacher cohorts to capture a wider range of opinions.
Project Team
Sabina Elkins
Researcher
Ekaterina Kochmar
Researcher
Jackie C. K. Cheung
Researcher
Iulian Serban
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Sabina Elkins, Ekaterina Kochmar, Jackie C. K. Cheung, Iulian Serban
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai