Large Language Model for Qualitative Research -- A Systematic Mapping Study
Project Overview
The document explores the transformative role of Generative AI, particularly Large Language Models (LLMs), in the field of education, highlighting their ability to automate qualitative research processes and enhance the analysis of student feedback and survey responses. By reducing subjectivity and increasing scalability, LLMs offer significant advantages in qualitative analysis, allowing educators to gain insights more efficiently. However, the document notes challenges such as the dependency on effective prompt engineering and the potential for inaccuracies in generated responses. It underscores the importance of combining LLM capabilities with human expertise to mitigate these risks and improve overall outcomes in educational settings. Ultimately, the findings suggest that while generative AI can greatly enhance qualitative research in education, careful implementation and oversight are essential to fully realize its benefits.
Key Applications
AI for qualitative data analysis
Context: Analyzing qualitative feedback from students and patients through open-ended responses and interviews to identify key themes and patterns of satisfaction and preferences.
Implementation: AI models, including ChatGPT and open-source LLMs, are utilized to categorize open-ended survey responses and perform thematic analysis on qualitative feedback. These models are fine-tuned to enhance their ability to accurately identify and classify content, streamlining the analysis process across various educational and health contexts.
Outcomes: Significantly reduced analysis time and improved efficiency in identifying key themes and patterns, leading to enhanced understanding of user satisfaction and preferences.
Challenges: Dependence on well-structured prompts and contextual limitations, with potential risks of generating inaccurate or irrelevant outputs.
Implementation Barriers
Technical Barrier
Reliance on well-structured prompts for accurate outputs, which can lead to inefficiencies if poorly designed.
Proposed Solutions: Develop better prompt engineering techniques and provide frameworks for structured interactions.
Quality Barrier
LLMs may generate 'hallucinations' or fabricated responses that do not reflect the data accurately.
Proposed Solutions: Implement validation and filtering techniques to ensure accuracy and relevance of outputs.
Ethical Barrier
Concerns regarding inherent model biases, handling of sensitive information, and the need for ethical guidelines in data handling.
Proposed Solutions: Conduct training on diverse datasets and implement robust ethical guidelines for data handling.
Project Team
Cauã Ferreira Barros
Researcher
Bruna Borges Azevedo
Researcher
Valdemar Vicente Graciano Neto
Researcher
Mohamad Kassab
Researcher
Marcos Kalinowski
Researcher
Hugo Alexandre D. do Nascimento
Researcher
Michelle C. G. S. P. Bandeira
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Cauã Ferreira Barros, Bruna Borges Azevedo, Valdemar Vicente Graciano Neto, Mohamad Kassab, Marcos Kalinowski, Hugo Alexandre D. do Nascimento, Michelle C. G. S. P. Bandeira
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai