The Anatomy of Risky Choice: Uncovering Subjective Choice Reasons with Large Language Models

Abstract: I will present a novel approach to understanding the subjective reasons underlying risky decision-making, enabled by large language models (LLM). Traditional models of risky choice often assume that decision-making relies on a single, stable reason or process, regardless of context. This simplification overlooks the variability in reasoning that may occur depending on the situation. To address these limitations, we developed an LLM-based approach that utilizes free-text reports to reveal the subjective reasons behind decisions, and we implemented a proof-of-concept in three stages. First, we extracted a comprehensive set of nearly 50 decision reasons from formal models, heuristics, and basic motivations. Second, we collected free-text retrospective verbal reports from 86 participants after each of 20 risky choices they made. Third, we employed advanced prompt engineering techniques with a state-of-the-art LLM to identify the reasons mentioned in these reports.

Our results provide strong evidence that decision reasons vary systematically across different choice problems but less across individuals. Furthermore, a simple predictive model based on the identified reasons achieves an out-of-sample accuracy of about 92%, validating the approach. Our results suggest that combining verbal reports and an LLM-based analysis with a large sample and comprehensive set of choice problems can uncover how people make risky decisions, including intricate relationships between decision reasons and types of choice problems.