What Would You Ask When You First Saw $a^2+b^2=c^2$? Evaluating LLM on Curiosity-Driven Questioning
Project Overview
The document explores the role of large language models (LLMs) in education, particularly their ability to generate curiosity-driven questions in subjects like physics, chemistry, and mathematics. It introduces a framework for evaluating LLMs based on their capacity to produce relevant, coherent, and diverse questions, and presents findings that smaller models can perform similarly to larger models, thereby challenging the belief that model size is the sole indicator of effectiveness. The study underscores the significance of questioning as a fundamental aspect of learning and knowledge acquisition, suggesting that LLMs can enhance educational practices by promoting inquiry-based learning. Overall, the document highlights the transformative potential of generative AI in education, advocating for its integration to foster deeper engagement and understanding among students.
Key Applications
Curiosity-Driven Question Generation (CDQG)
Context: Educational context involving physics, chemistry, and mathematics, targeting students and educators.
Implementation: LLMs were prompted to generate questions based on a dataset of statements in the mentioned subjects, simulating human curiosity.
Outcomes: LLMs displayed high capability in generating relevant and coherent questions, enhancing understanding and inquiry in educational settings.
Challenges: Diversity and depth of questions generated were sometimes limited. Smaller models showed variability in performance, particularly with erroneous statements.
Implementation Barriers
Technical
Limitations in the ability of LLMs to generate diverse and deep questions.
Proposed Solutions: Enhancing training data quality and model architecture to improve questioning capabilities.
Methodological
Existing evaluation methods may not fully capture the nuances of human-like questioning.
Proposed Solutions: Developing more comprehensive evaluation frameworks that encompass reasoning depth, creativity, and factual accuracy.
Project Team
Shashidhar Reddy Javaji
Researcher
Zining Zhu
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Shashidhar Reddy Javaji, Zining Zhu
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai