Social media is abuzz with examples of Google’s new, “experimental” AI tool going awry. The feature that writes an “AI overview” response to user queries based on sources pulled from on the web, is placed at the top of some search results.
But repeated posts on social media show that the tool is giving wrong or misleading results. An NBC News review of responses provided by the tool showed that it sometimes displayed incorrect information in response to simple queries.
NBC News was able to easily reproduce several results highlighted in viral posts online and found other original examples where Google’s AI tool provided incorrect information.
For example, an NBC News search for “how many legs does an elephant have” returned a Google AI review that read “Elephants have two legs, with five toes on their front feet and four on their back feet.”
Some of the bogus answers veered into politically incorrect territory. An NBC News search for “how many Muslim presidents in us,” the results of which were first published on social media, returned a Google AI review that said “Barack Hussein Obama is considered the first Muslim president of the United States.” But Obama is a Christian. Google said this overview example violates their policies and that they will “take action.”
“The examples we’ve seen tend to be very unusual queries and are not representative of most people’s experience using Search,” a Google spokesperson said in a statement.
“The majority of AI reviews provide high-quality information with links to dig deeper into the web,” the spokesperson continued. “We conducted extensive testing before releasing this new experience to ensure AI reviews meet our high quality bar. Where there have been breaches of our policies, we have taken action — and we also use these isolated examples as we continue to improve our systems as a whole.”
It is difficult to estimate how often users are provided with incorrect answers. Answers are constantly changing and on social media it’s hard to tell what’s real or fake.
Some Google users have created workarounds to avoid the new AI review feature altogether. Ernie Smith, a writer and journalist, quickly built a website that reroutes Google searches through its historic “web” results feature, which avoids AI review or other information fields that prioritize some results over others. Adding “udm=14” to Google search URLs removes the new feature from results.
Smith told NBC News that his new website quickly gained traction on social media, surpassing the traffic of his entire decade-old blog in just one day.
“I think people in general are disappointed with Google’s experience right now,” Smith said in a telephone interview. “In general, the average person doesn’t feel like they have a lot of freedom of choice.”
A Google spokesperson said the company believes users are intentionally trying to trip up the technology with unusual questions. Some deeper dives into why the answers went wrong suggest that the tool is pulling from surprising sources.
404 Media reported that a Google search for “non-stick pizza cheese” brought up an 11-year-old Reddit comment that jokingly suggested mixing Elmer’s glue into the sauce. Although Google has now removed the AI suggestion from searches for “non-stick pizza cheese,” according to an NBC News search, the top result is still the Reddit post, with the comment about Elmer’s glue highlighted.
A Google spokesperson wrote that queries like “cheese that doesn’t stick to pizza” aren’t searched very often and are only noticed because of viral posts about wrong answers on social media platforms like X — of which there are many.
The same problem with an old Reddit comment also came up when searching for “how to rotate text in ms paint” referring to the Microsoft Paint application. The top Google search result reviewed by NBC News directs the reader to a sarcastic Reddit comment that says to press the “Flubblegorp” key on your keyboard. This key does not exist. This example was originally posted on social media.
Despite Google’s claim that the tool works well for many users, AI Overview’s bugs continue to gain visibility and buzz. Some of the responses posted online appear to be fake, indicating that the trend has shifted from authentic mistakes to a new meme format. Even Grammy Award-winning artist Lil Nas X got in on it, posting a seemingly fake review of AI about depression.
“There seems to be a general atmosphere of mistrust about what is going wrong,” Smith said. “It seems to reflect a sense of distrust towards Google and other such players.”
