Abstract

In this paper, we examine how the increasing use of large language models (LLMs) like ChatGPT to refine user-generated content can inadvertently alter the sentiment embedded within the text. Using a dataset of tweets about climate change, we find that LLM-rephrased tweets exhibit more neutral sentiment compared to the original tweets. Replicating a prior study on public opinions of climate change, we further demonstrate that the sentiment differences introduced by LLM rephrasing could distort established research conclusions. We propose and validate a mitigation approach to estimate the authentic human sentiment of the original tweets. This study highlights the potential threat of generative AI to the reliability of sentiment analysis and research reliant on user-generated content.

To find out more please get in touch!