How GPT-3 responds to different publics on climate change and Black Lives Matter: A critical appraisal of equity in conversational AI
Project Overview
The document examines the role of generative AI, specifically GPT-3, in education, highlighting its potential and challenges in promoting equitable dialogue among diverse populations. It reveals that while marginalized groups, such as opinion and education minorities, faced less favorable interactions with GPT-3, they simultaneously demonstrated notable knowledge gains on critical topics such as climate change and the Black Lives Matter movement following their interactions. This underscores the dual nature of generative AI's impact, where it can both present barriers and facilitate significant educational outcomes. To address these inequities, the document proposes an analytical framework designed to evaluate the effectiveness of conversational AI, focusing on user experience, learning outcomes, and the diversity of response styles. The findings suggest that while generative AI holds promise for enhancing educational engagement and knowledge dissemination, careful consideration of its design and implementation is essential to ensure it benefits all users equitably.
Key Applications
GPT-3 conversational AI system
Context: Educational tools for discussing social issues like climate change and Black Lives Matter, targeting diverse user groups including opinion and education minorities.
Implementation: Conducted an algorithm auditing study using GPT-3 to engage users in dialogues about climate change and BLM.
Outcomes: Opinion and education minority groups demonstrated the largest knowledge gains and changed attitudes post-chat, despite reporting worse user experiences.
Challenges: Conversational disparities in responses led to negative user experiences for minority groups, highlighting equity issues in AI interactions.
Implementation Barriers
User Experience
Minority groups reported worse user experiences with GPT-3 compared to majority groups, which could deter future engagement.
Proposed Solutions: Suggestions included varying language styles and improving the richness of responses to enhance user satisfaction.
Project Team
Kaiping Chen
Researcher
Anqi Shao
Researcher
Jirayu Burapacheep
Researcher
Yixuan Li
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Kaiping Chen, Anqi Shao, Jirayu Burapacheep, Yixuan Li
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai