An exploratory analysis of Community-based Question-Answering Platforms and GPT-3-driven Generative AI: Is it the end of online community-based learning?
Project Overview
The document examines the role of generative AI, particularly GPT-3, in educational settings, focusing on its application in community-based question-answering platforms like Stack Overflow. It evaluates the effectiveness of ChatGPT in addressing software engineering inquiries compared to human responses. Findings reveal that while ChatGPT delivers quick, concise, and positive answers, its integration has led to a significant decrease in community engagement on these platforms, raising concerns about the diminishing human interaction crucial for collaborative learning. The study underscores the dual nature of AI's impact in education: it enhances the speed and accessibility of information but simultaneously risks undermining the collaborative and interactive aspects of learning environments, suggesting a need to balance AI use with fostering community engagement.
Key Applications
ChatGPT
Context: Community-based question-answering platforms, specifically Stack Overflow, targeted at software engineers seeking solutions to programming questions.
Implementation: The study collected questions from Stack Overflow and compared answers generated by ChatGPT with those provided by human experts. Metrics were used to analyze text length, accuracy, and sentiment.
Outcomes: ChatGPT provided answers that were shorter, more concise, and exhibited higher positive sentiment compared to human responses. The accuracy of ChatGPT’s answers was reported to be between 71% and 75%.
Challenges: Potential limitations include reduced community engagement, concerns over the accuracy of AI-generated content, and the need for revisions in the answers provided by ChatGPT.
Implementation Barriers
Engagement Barrier
Decrease in community interaction on Stack Overflow since the introduction of ChatGPT.
Proposed Solutions: Further research is needed to understand the implications of reduced community involvement and how to retain user engagement.
Quality Assurance Barrier
Concerns regarding the accuracy of AI-generated responses and the potential for misinformation.
Proposed Solutions: Implementing a review system for AI-generated content and providing clear guidelines for users on verifying AI responses.
Project Team
Mohammed Mehedi Hasan
Researcher
Mahady Hasan
Researcher
Mamun Bin Ibne Reaz
Researcher
Jannat Un Nayeem Iqra
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Mohammed Mehedi Hasan, Mahady Hasan, Mamun Bin Ibne Reaz, Jannat Un Nayeem Iqra
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai