The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support
Project Overview
The document examines the application of small language models (SLMs) in educational settings, specifically focusing on their role in generating empathetic dialogue for individuals with PTSD. It introduces the TIDE dataset, comprising 10,000 synthetic dialogues aimed at enhancing the empathetic responses of SLMs. The findings indicate that fine-tuning these models using the dataset can improve their empathetic engagement; however, the effectiveness of these improvements varies by scenario and user demographics. The research identifies challenges such as the models' limited expressive capacity and the need for trauma-informed design principles. Overall, the study underscores the significant potential of SLMs in supporting mental health care within educational contexts while simultaneously highlighting the necessity for ethical considerations in their deployment.
Key Applications
TIDE dataset for empathetic dialogue generation
Context: Mental health support for individuals with PTSD
Implementation: Fine-tuning small language models (0.5B–5B parameters) using the TIDE dataset, which consists of synthetic dialogues reviewed for trauma sensitivity.
Outcomes: Improved empathetic response quality in specific contexts, enhancing the potential for scalable mental health support systems.
Challenges: Limited expressive capacity of small models and the need for context-aware design to ensure effective empathetic communication.
Implementation Barriers
Technical barrier
Limited expressive capacity of small language models, which may restrict their ability to generate nuanced empathetic responses. This limitation necessitates enhancements to improve performance.
Proposed Solutions: Fine-tuning models on trauma-informed datasets like TIDE to enhance their empathetic capabilities.
Ethical barrier
Need for ethical considerations in the design of empathetic AI to ensure safety and appropriateness in mental health contexts. This includes setting standards that prioritize human well-being.
Proposed Solutions: Incorporating trauma-informed principles and human-centered evaluation metrics into the development process.
Project Team
Suhas BN
Researcher
Yash Mahajan
Researcher
Dominik Mattioli
Researcher
Andrew M. Sherrill
Researcher
Rosa I. Arriaga
Researcher
Chris W. Wiese
Researcher
Saeed Abdullah
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Suhas BN, Yash Mahajan, Dominik Mattioli, Andrew M. Sherrill, Rosa I. Arriaga, Chris W. Wiese, Saeed Abdullah
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai