Teaching the Machine to Explain Itself using Domain Knowledge
Project Overview
The document explores the application of generative AI in education through the introduction of JOEL, a neural network framework designed for explainable AI (XAI) that aids non-technical users in grasping machine learning predictions. This framework is particularly valuable in fields that require human expertise, such as fraud detection, as it simultaneously learns decision-making tasks and their explanations, thereby improving interpretability and building trust in AI systems. By incorporating human feedback, JOEL addresses the limitations of traditional explanation methods, which are often overly technical for domain experts. Additionally, the framework employs Distant Supervision to create datasets with minimal human annotation, promoting scalability and effectiveness in educational contexts. Overall, JOEL represents a significant advancement in making AI accessible and understandable, thereby enhancing its integration into educational settings and enabling more informed decision-making.
Key Applications
JOEL (Jointly learned cOncept-based ExpLanations)
Context: Fraud detection systems involving human domain experts (fraud analysts)
Implementation: JOEL integrates human feedback into the model's learning process to provide explanations that reflect the reasoning of domain experts.
Outcomes: Improved explainability performance by approximately 13.57%, enhancing decision-making efficiency for fraud analysts.
Challenges: The challenge of ensuring that machine-generated explanations align with human experts' reasoning and addressing the cold-start problem in predictive tasks.
Implementation Barriers
Data Availability
The scarcity of labeled datasets that adequately represent the concepts needed for training, particularly in high-stakes decision-making contexts.
Proposed Solutions: Utilizing Distant Supervision to automate the annotation process based on existing knowledge from legacy expert systems.
Understanding of AI Systems
Non-technical users often struggle to comprehend technical explanations produced by AI systems, leading to mistrust.
Proposed Solutions: Developing concept-based explanations that are more aligned with human reasoning, making them easier to understand for domain experts.
Project Team
Vladimir Balayan
Researcher
Pedro Saleiro
Researcher
Catarina Belém
Researcher
Ludwig Krippahl
Researcher
Pedro Bizarro
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Vladimir Balayan, Pedro Saleiro, Catarina Belém, Ludwig Krippahl, Pedro Bizarro
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai