Share and Follow
(The Conversation) – When a person describes an event as “probable” or “likely,” there’s usually a shared, albeit vague, consensus on its meaning. However, our recent study reveals that when AI chatbots like ChatGPT employ these terms, they aren’t evaluating the odds in the same manner humans do.
Our findings, published in NPJ Complexity, highlight that although large language model AIs are adept at conversation, they often misalign with human perceptions when expressing uncertainty. The investigation centered on words that denote estimative probability, such as “maybe,” “probably,” and “almost certain.”
In our analysis of how AI models and humans translate these words into numerical percentages, we discovered notable discrepancies. While AIs generally concur with humans on extreme terms like “impossible,” they significantly differ on ambiguous words like “maybe.” For instance, an AI might interpret “likely” as an 80% probability, whereas a human might see it as nearer to 65%.
This discrepancy could stem from the fact that humans interpret terms like “likely” and “probable” through contextual clues and personal experience. On the other hand, large language models might average conflicting uses of these words from their training data, resulting in interpretations that diverge from human expectations.
Our research also revealed that these models are influenced by gendered language and the phrasing of prompts. Changing a prompt from “he” to “she” often caused the AI’s probability estimates to become more fixed, indicating biases in its training data. Furthermore, switching from English to Chinese in prompts led to variations in probability estimates, potentially due to cultural differences in expressing and understanding uncertainty.
Why it matters
Far from being a linguistic quirk, this misalignment is a fundamental challenge for AI safety and human-AI interaction. As large language models are increasingly used in high-stakes fields like health care, government policy and scientific reporting, the way they communicate risk becomes a matter of public trust.
If an AI assistant helping a doctor, for instance, describes a side effect as “unlikely,” but the model’s internal calculation of “unlikely” is much higher than the doctor’s interpretation, the resulting decision could be flawed.
What other research is being done
Scientists have studied how humans quantify uncertainty since the 1960s, a field pioneered by CIA analysts to improve intelligence reporting. More recently, there has been an explosion in large language model literature seeking to look under the hood of neural networks to better understand their “behaviors” and linguistic patterns.
Our study adds a layer of complexity by treating the interaction between humans and artificial intelligence as a biological-like system where meaning can degrade. It moves beyond simply measuring if an AI is “smart” and instead asks if it is aligned.
Other researchers are currently exploring whether so-called chain-of-thought prompting – asking the AI to show its work – can fix these errors. However, our study found that even advanced reasoning doesn’t always bridge the gap between statistical data and verbal labels.
What’s next
A goal for future AI development is to create models that don’t just predict the next likely word but actually understand the weight of the uncertainty they are conveying. Researchers are calling for more robust consistency metrics to ensure that if a model sees a 10% chance in the data, it chooses the same word every time.
As we move toward a world where AI summarizes scientific papers and manages people’s schedules, making sure that “probably” means “probably” is a vital step in making these systems reliable partners rather than just sophisticated parrots.