Forum
What is DR@W Forum?
DR@W Forum is an interdisciplinary discussion series which focuses on theoretical and empirical research about decision making.
The usual structure of the forum is a 30 - 45 minute introduction of the topic/working paper, with ample additional time for discussion.
The audience prefers discussing work-in-progress topics as opposed to finished papers. We meet on Thursdays between 2:30 and 3:45pm during term time, with streaming via Zoom. Contact John Taylor (John.Taylor[at]wbs.ac.uk) if you would like to suggest a speaker for a future event. Notifications of upcoming DR@W Forum events along with other decision research related activities can be obtained by registering with the moderated Behaviour Spotlight email listLink opens in a new window.
Note that several talks during the 2024/25 academic year are being hosted and organised by the Economics department. This is indicated in the calendar entries. These talks will all take place in the Social Studies building. If you require further details regarding these sessions, please contact Matthew Ridley (Matthew.Ridley[at]Warwick.ac.uk) in the Economics department.
DR@W/EBER Seminar: Rafael Jimenez-Duran (Stanford)
Abstract: Large Language Models (LLMs) are said to exhibit sycophancy, a tendency to agree with users irrespective of the truth. We propose an economic framework that defines sycophancy as a preference for user approval, and develop an outcome-based sufficient statistic to detect it. Our identification strategy exploits a key architectural feature of LLMs: they are stateless, and "memory" of past interactions is constructed by summarizing conversations into short profiles appended to each new prompt. Because this memory can be controlled, toggled, and varied experimentally, we can isolate the causal path from user feedback to sycophantic behavior. We instrument the LLM's perceived cost of disagreement with a one-word variation in simulated prior user feedback. In an experiment with leading LLMs across three domains (moral judgments, factual questions, and common misconceptions) we find evidence that LLMs are sycophantic. Sycophancy is larger in subjective domains where baseline accuracy is lower and is heterogeneous across models.