Generative Artificial Intelligence: Implications for Biomedical and Health Professions Education
Project Overview
Generative AI, especially large language models (LLMs), is transforming education in the biomedical and health professions by demonstrating impressive capabilities in medical examinations, clinical reasoning tasks, and academic assessments. While these technologies offer significant advantages, such as enhancing learning experiences and providing personalized feedback, they also present challenges, including the potential erosion of knowledge acquisition and critical thinking skills among students. To maximize the benefits of generative AI in education, experts recommend implementing strategies that promote AI literacy, including teaching competencies in AI and prompt engineering. Additionally, there is a call for educational institutions to adapt their practices to responsibly integrate AI tools, ensuring that students develop a balanced understanding of AI's role in their professional development. Overall, the effective use of generative AI has the potential to enrich educational outcomes while necessitating careful consideration of its implications for learning and critical thinking.
Key Applications
Large Language Models (LLMs) for Assessment and Question Generation
Context: Applied in various educational settings, including biomedical health professions education and higher education psychology courses, focusing on both assessment and content generation for students and faculty.
Implementation: Utilization of LLMs, such as ChatGPT, GPT-4, and Llama 3, to generate exam questions and assess performance against academic standards, including medical board exams and PhD-level assessments.
Outcomes: LLMs demonstrated performance scores equivalent to or exceeding those of human examinees in medical board exams and outperformed average student scores in psychology courses, with high detection rates by faculty.
Challenges: Issues such as hallucinations, lack of reliable citations, potential over-reliance on AI, and concerns regarding academic integrity with high rates of AI-generated submissions going undetected.
GitHub CoPilot for Programming Assistance
Context: Implemented in graduate-level programming courses, particularly in health informatics, to assist students in writing code.
Implementation: Integration of GitHub CoPilot as a coding assistant for SQL and Python tasks, facilitating coding exercises and project work.
Outcomes: Effective in supporting simple programming tasks, though less reliable for complex coding problems, with some generated solutions being correct but not the most efficient.
Challenges: Variability in the accuracy and efficiency of the solutions provided, leading to challenges in dependability for more intricate coding scenarios.
Implementation Barriers
Implementation Challenge
Generative AI can undermine critical thinking and deeper learning.
Proposed Solutions: Educators should redesign assignments to minimize over-reliance on AI and promote critical thinking.
Academic Integrity Concern
High detection rates of AI-generated work are low, leading to potential cheating.
Proposed Solutions: Institutions need to develop robust policies and tools for detecting AI use in academic submissions.
Bias and Accuracy Issues
LLMs can perpetuate biases and produce hallucinations or inaccurate information.
Proposed Solutions: Regular audits and updates to training data, along with improved evaluation frameworks.
Project Team
William Hersh
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: William Hersh
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18