Generative AI and Its Educational Implications

Intelligent Tutoring and Feedback Systems

Context: Classroom and individual tutoring, providing feedback on writing, code, and other performances.

Implementation: AI systems track student knowledge, apply contextual tutoring strategies, provide scaffolded support, analyze spoken or written language, and integrate multimodal data (e.g., speaking, writing, facial emotions). They generate text, images, movies, simulations and assessment activities, such as logically-correct computer code with syntax errors.

Outcomes: Increased student test scores (0.66 standard deviations over conventional classroom training), effectiveness comparable to expert tutors, instant feedback to students, more engaging and realistic learning experiences, personalized learning materials, ability to adapt to student needs, characterization of learner abilities, objective and standardized evaluation, and understanding and transparency.

Challenges: Rule-based or trained on specific topics; limited natural language processing; focus on higher-order thinking skills; multimodal processing; expensive to build; requires careful design of prompts; concerns about the accuracy of information; need for human oversight; requires collecting a large number of samples of student performance; potential for hallucinations; and the need for robust systems for verifying and validating AI outputs.

Automated Assessment

Context: Formative and summative assessment in digital learning environments, assessing writing, speaking, and code in educational settings.

Implementation: AI models are trained to assess writing, analyze spontaneous speech, and mine process data. Generative AI models can analyze spoken or written language and integrate multimodal data, such as speaking, writing, and facial emotions.

Outcomes: Provides instant feedback to students, removes the burden of manual grading from teachers, enables the integration of more complex performances within learning environments, and works across different languages and software code.

Challenges: Requires collecting a large number of samples of student performance, hand-scoring them, and then using machine learning techniques to train an AI model to learn to score them automatically. Confined the applicability of AI to areas where data collection is straightforward, interactions can be hand-designed, and human coders can easily characterize performance. Requires new methods for evaluating assessment characteristics (validity and reliability), potential for hallucinations, and the need for robust systems for verifying and validating AI outputs.

Personalized Content Generation

Context: Creating dynamic learning experiences, learning programming skills.

Implementation: AI systems generate text, images, movies, and simulations adapted to the learner's background knowledge and reading level. Generate contextual assessment activities and act as roleplay participants, adapting character based on student prompts.

Outcomes: More engaging, realistic learning experiences, personalized learning materials, ability to adapt to student needs, instant assessment, and deep conceptual feedback and training.

Challenges: Requires careful design of prompts, concerns about the accuracy of information, and the need for human oversight.

Technological

Data bias, design transparency, and algorithmic explainability: AI models may reflect biases in their training data, leading to unfair or inequitable outcomes. Lack of transparency regarding the design and training data of models. Difficulty understanding why a model makes a certain decision (black box problem). Potential for models to generate incorrect or misleading information (hallucinations).

Proposed Solutions: Continuous efforts in bias detection and mitigation in data and algorithms; testing and certifying systems across wide ranges of inputs; developers of educational systems need to test and certify their systems across wide ranges of inputs to assure that biases are mitigated. Development of new methods for evaluating assessment characteristics in generative AI systems; robust systems for verifying and validating AI outputs; and standards that espouse transparency and explainability around the methods.

Ethical and Societal

Introducing and maintaining standards for generative AI in education: The field will need to internally police itself with standards that espouse transparency and explainability around the methods.

Proposed Solutions: Being open about how the models were developed, tested and validated, and providing information on their intended use and limitations in their educational context. Concurrently, the field will need to continually incorporate external guidance to help steer ethics in this field, such as the European Union’s ethical guidelines on the use of AI and data in teaching and learning and education.

Societal and Educational

Challenges for educational ecosystems, new modes of communication and trust, and collaboration boundary and social norms: Educational institutions are not well positioned to evolve their curriculum quickly, and the manner in which faculty operate will need to change with regard to both the production of research and the conduct of instruction. The epistemic and social assumptions we bring to interpreting images need to be rethought due to the ability to generate photo realistic images. As the fundamental differences blur between what humans and computers are capable of, fundamental questions of attribution and provenance are raised as well.

Proposed Solutions: Curricula must change to prepare students for a rapidly evolving world. In instruction, not only must the curricula change, but also the modes of instruction must, and will change. Educational institutions will evolve to support them. Patience and generosity. There will be many perceived social transgressions and mistakes while social and professional societies evolve their understandings and practices.

Practical

Choosing the right tool for the job: Determining which model is the most effective for a particular educational experience, considering different model assumptions, learning contexts, student needs, and course objectives.

Proposed Solutions: Requires careful consideration of the learning context, the specific needs and preferences of the students, and the objectives of the course or program.