Are AI Models as Irrationally Human as We Are? Inside Tomer Geva’s Seminar on LLM Cognitive Biases

We like to think of Artificial Intelligence as a purely logical engine—a "math machine" free from the messy shortcuts and irrationalities of the human brain. But as Large Language Models (LLMs) increasingly drive our financial and business decisions, a critical question emerges: Do these models inherit our psychological flaws?

On May 7th, the Gillmore Centre was thrilled to host Professor Tomer Geva from Tel Aviv University’s Coller School of Management. In a packed session at WBS, Professor Geva presented a deep dive into the "Psychology of LLMs," revealing that the digital minds we are building might be more human-like than we realized—for better and for worse.

Bridging the Gap: A Morning in the Gillmore Lab

Before the seminar, Professor Geva spent the morning in the Gillmore Lab, engaging with our research. The visit was a fantastic opportunity to find common ground between our ongoing AI and Fintech projects and his cutting-edge work on machine learning.

The discussion highlighted a shared mission: understanding how Generative AI can be leveraged to enhance decision-making while remaining aware of its inherent "personality" and pitfalls.

The Problem: Conflicting Data

Until now, research on whether LLMs exhibit cognitive biases has been a bit of a "Wild West." Some studies claimed AI was perfectly rational, while others found it as biased as a tired human. Professor Geva pointed out that these contradictions often stem from:

Small sample sizes.
Researchers "cherry-picking" specific biases to test.
A lack of control for factors like prompt length or how the question is phrased.

The Solution: A Systematic Framework

To fix this, Professor Geva’s team developed a novel systematic framework designed to strip away researcher bias. They conducted the first-ever meta-analysis in this field, measuring and controlling for various factors across a massive scale of experiments.

Key Findings from the Research:

The Mirror Effect: In many cases, LLMs jointly display cognitive biases remarkably similar to humans.
The Nuance: It’s not a 1:1 match. While LLMs fall for some traps (like certain framing effects), they actually outperform humans in others.
Recognition vs. Imitation: In a fascinating discovery, the team found that when an LLM is smart enough to recognize it is being tested for a bias, it is less likely to imitate the human response. It chooses logic over "human-likeness."

Why This Matters for the Future

The seminar concluded by addressing the methodological challenges of this new field, such as the "zero variance phenomena" (where an AI gives the exact same biased answer every time).

For us at the Gillmore Centre, these insights are vital. If we are to use LLMs for financial forecasting, credit scoring, or risk management, we must understand the "cognitive" baggage they carry. Professor Geva’s work doesn't just point out the flaws; it opens the door to cognitive bias reduction, helping us build AI that is not just faster than us, but more objective.

Contribution Highlights

New Framework: A standardized way to assess human-like behavior in AI.
Meta-Regression: The first study to control for prompt length and "recognition rates" in bias.
Bias Reduction: A roadmap for training AI to avoid common human psychological traps.

We would like to thank Professor Tomer Geva for his invaluable insights and for spending the day with our research community. Stay tuned to the Gillmore Centre blog for more updates on the intersection of AI, Psychology, and Finance.