LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System
Project Overview
The document explores the application of generative AI, specifically through the implementation of the LLM-based Smart Reply (LSR) system utilizing ChatGPT, to bolster communication efficiency and productivity in educational settings and collaborative workplaces. The findings indicate that the LSR system significantly enhances work performance by delivering context-aware and personalized responses, thereby reducing workloads and increasing overall productivity among users. Additionally, the document addresses user experiences with the system, noting both the positive impacts and the challenges related to trust and reliance on AI-driven communication tools. It emphasizes the importance of continuous improvement in these technologies to effectively meet the needs of users in educational environments, ultimately aiming to facilitate better collaboration and learning outcomes.
Key Applications
LLM-based Smart Reply (LSR) system using ChatGPT
Context: Collaborative workplaces, targeting professionals involved in tasks requiring communication and scheduling.
Implementation: Implemented as a system integrated with Slack and Google Calendar, using a two-step response generation process.
Outcomes: Increased work performance by 5.58%, improved productivity by 40.36%, and reduced cognitive demand as measured by NASA TLX.
Challenges: Concerns regarding AI-generated responses not aligning with user intent, potential misinterpretation of context, and privacy issues.
Implementation Barriers
Trust and Privacy
Users expressed concerns over privacy and trust in AI-generated messages, especially in sensitive contexts.
Proposed Solutions: Implement secure channels for handling sensitive information and offer toggles between private and AI modes.
User Experience
Participants found the generated responses sometimes misleading or not reflective of their intended message. Additionally, the AI's inability to effectively interpret emotional nuances was noted.
Proposed Solutions: Incorporate options for message editing, a 'regenerate' feature to provide alternative responses, and implement a two-step inference mechanism using a faster model for initial responses to address response lag.
Project Team
Ashish Bastola
Researcher
Hao Wang
Researcher
Judsen Hembree
Researcher
Pooja Yadav
Researcher
Zihao Gong
Researcher
Emma Dixon
Researcher
Abolfazl Razi
Researcher
Nathan McNeese
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Ashish Bastola, Hao Wang, Judsen Hembree, Pooja Yadav, Zihao Gong, Emma Dixon, Abolfazl Razi, Nathan McNeese
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai