Conversate: Supporting Reflective Learning in Interview Practice Through Interactive Simulation and Dialogic Feedback
Project Overview
The document explores the transformative role of generative AI in education, particularly through the innovative application of 'Conversate', an AI-mediated interview practice system that utilizes large language models (LLMs) to foster reflective learning via interactive simulations and dialogic feedback. It emphasizes the significance of mock interviews in influencing career paths, while addressing the limitations of traditional methods. Key findings from user studies indicate that participants appreciated the adaptive follow-up questions and personalized dialogic feedback, which not only enhanced their learning experience but also alleviated feelings of judgment commonly associated with traditional interview settings. Additionally, the text discusses broader applications of generative AI in automated assessment and feedback mechanisms, underscoring its potential to enrich student learning by offering tailored support in developing critical skills such as communication and interview readiness. Overall, the findings suggest that generative AI can significantly improve educational practices by promoting individualized learning and fostering a more supportive environment for skill development.
Key Applications
AI-Assisted Interview and Assessment Tool
Context: Students preparing for job interviews and seeking to improve their communication skills, as well as higher education students aiming to enhance their academic performance through personalized feedback.
Implementation: A web-based application that utilizes generative AI to simulate job interviews, providing dynamic questions, personalized feedback on submissions, and AI-assisted annotations. It incorporates dialogic feedback to help users understand their strengths and areas for improvement.
Outcomes: Users reported enhanced realism in interview simulations, improved engagement and understanding of feedback, increased confidence in communication skills, and tailored guidance leading to better academic performance. The tool supports personalized continuous learning and helps users respond more effectively during interviews.
Challenges: The low-stakes environment may not fully prepare users for real interviews; LLMs may exhibit sycophantic behavior during feedback, and there is a dependence on the accuracy of AI assessments, with potential for misinterpretation of nuanced student responses.
Implementation Barriers
Technical and Technological Barrier
LLMs may agree with users' disagreements, leading to confusion and lack of objectivity. Additionally, there is limited access to the necessary technology and infrastructure to implement generative AI tools effectively.
Proposed Solutions: Incorporate mechanisms for LLMs to provide detailed reasoning and sufficient pushback during feedback. Invest in educational technology infrastructure and provide training for educators on how to integrate AI tools.
Emotional Barrier
Users may experience frustration during the revision process if they do not see improvements.
Proposed Solutions: Design systems that recognize user disengagement and prompt more detailed responses or adaptive feedback.
Human Factor Barrier
Resistance from educators and students to adopt AI-based tools due to concerns about reliability and job displacement.
Proposed Solutions: Providing evidence of AI effectiveness and emphasizing the complementary role of AI in enhancing rather than replacing human educators.
Project Team
Taufiq Daryanto
Researcher
Xiaohan Ding
Researcher
Lance T. Wilhelm
Researcher
Sophia Stil
Researcher
Kirk McInnis Knutsen
Researcher
Eugenia H. Rho
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Taufiq Daryanto, Xiaohan Ding, Lance T. Wilhelm, Sophia Stil, Kirk McInnis Knutsen, Eugenia H. Rho
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai