The Surprising Frequency of Chatbot ‘Hallucinations’

When Google introduced a similar chatbot a few weeks later, it began spouting nonsensical facts about the James Webb telescope. The next day, Microsoft’s Bing chatbot provided all sorts of false information about the Gap, Mexican nightlife, and singer Billie Eilish. In March, ChatGPT referenced six non-existent court cases while drafting a 10-page legal brief for a lawyer to submit to a federal judge in Manhattan.

A new startup called Vectara, led by former Google employees, is now working to determine how frequently chatbots provide inaccurate information. The company’s research suggests that even in situations designed to prevent it, chatbots create false information at least 3% of the time – and as high as 27%.

This chatbot behavior, which experts call “hallucination,” can be a serious issue for anyone using this technology with court documents, medical information, or sensitive business data. Due to their ability to respond to requests in numerous ways, it is nearly impossible to definitively determine the frequency at which they “hallucinate.” According to Simon Hughes of Vectara, you would have to look at all of the world’s information to do so.

Research from Vectara revealed that these hallucination rates vary widely among leading AI companies. OpenAI’s technologies had the lowest rate at around 3%, while systems from Meta, which owns Facebook and Instagram, averaged around 5%. However, a Google system, Palm chat, had the highest rate at 27%. Vectara hopes that their publicly-shared methods will drive efforts across the industry to decrease hallucination rates.

Nonetheless, reducing the problem is challenging. These chatbots, which work on a form of AI called large language models (LLMs), operate on the basis of learning from enormous amounts of text data, and they rely on probabilities. This, in turn, can lead to inaccurate or false information being provided to users.

In light of these findings, the team from Vectara warns people to be wary of information from chatbots and the technology sold by companies like Vectara, which is offering this kind of technology for business use.

Despite efforts to minimize the problem, researchers caution that chatbot hallucination is a complex issue. Even languages models used to check the accuracy of summaries can make mistakes, because they too, operate on the basis of probabilities.

In conclusion, Vectara’s research suggests that while chatbots offer many benefits, they still need to find ways to minimize the likelihood of hallucination and minimize the spread of false information.


Unlikely Industry Player Anguilla Profits Big from A.I. Boom

Artificial intelligence’s integration into everyday life has stirred up doubts and unsettling questions for many about humanity’s path forward. But in Anguilla, a tiny Caribbean island to the east of Puerto Rico, the A.I. boom has made the country a fortune. The British territory collects a fee from every registration for internet addresses that end […]

Read More

China Surpasses U.S. in A.I. Talent: A Key Metric

China lags behind the United States in artificial intelligence that powers chatbots like ChatGPT but excels in producing scientists behind new humanoid technologies. New research reveals that China has surpassed the United States as the biggest producer of A.I. talent. The country generates almost half the world’s top A.I. researchers, compared to 18 percent from […]

Read More

Brands Brace for Impact as TikTok Faces Criticism

Amid debate in Washington over whether TikTok should be banned if its Chinese owner doesn’t sell it, one group is watching with particular interest: the many brands — particularly in the beauty, skin care, fashion, and health and wellness industries — that have used the video app to boost their sales. Youthforia, a makeup brand […]

Read More