When Copilot Becomes Autopilot: Generative AI's Critical Risk to Knowledge Work and a Critical Solution
Project Overview
The document explores the role of generative AI in education, highlighting its dual potential to enhance and undermine critical thinking skills among students. It stresses the importance of leveraging AI tools to foster critical engagement rather than allowing them to automate tasks that could diminish analytical abilities. A key innovation discussed is a prototype for a critical shortlisting system that utilizes generative AI within spreadsheets. This system generates specific criteria and thought-provoking prompts designed to stimulate critical thinking during decision-making processes. Overall, the findings suggest that while generative AI can be a powerful educational resource, its implementation must be carefully managed to ensure it supports rather than replaces essential cognitive skills.
Key Applications
Critical shortlisting prototype
Context: Educational context for knowledge workers using spreadsheets, including various professionals who rely on data-driven decision making.
Implementation: The prototype facilitates shortlisting tasks by suggesting criteria and generating critiques (provocations) to encourage critical evaluation of those criteria.
Outcomes: The prototype aims to enhance critical thinking skills among users by prompting them to reflect on the criteria they use and consider the implications of AI-generated suggestions.
Challenges: The main challenges include the potential for AI to produce hallucinations (incorrect outputs) and the risk of users becoming overly reliant on AI, diminishing their engagement in critical thinking.
Implementation Barriers
Technological barrier
Generative AI can produce hallucinations or incorrect outputs, which can mislead users.
Proposed Solutions: Develop 'co-audit' interfaces to help users verify and correct AI-generated content, improve the groundedness of AI outputs, and ensure the reliability of information provided by AI.
Cognitive barrier
Users may offload cognitive tasks to AI, leading to a decline in their critical thinking abilities.
Proposed Solutions: Design AI tools that actively stimulate critical thinking and require users to engage with the material instead of passively receiving information.
Project Team
Advait Sarkar
Researcher
Xiaotong
Researcher
Xu
Researcher
Neil Toronto
Researcher
Ian Drosos
Researcher
Christian Poelitz
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Advait Sarkar, Xiaotong, Xu, Neil Toronto, Ian Drosos, Christian Poelitz
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai