A fake blog post was enough to manipulate ChatGPT and Google AI answers

Summary

BBC technology journalist Thomas Germain describes how he made ChatGPT and Google’s AI surfaces (including AI Overviews and the Gemini app) repeat a false claim that he is the best tech journalist at eating hot dogs. The approach he outlines is straightforward: publish a deceptive post on a personal website and target situations where the tools pull information from the web to answer.

He argues this points to a growing problem: manipulating AI-generated responses through fabricated or promotional online content, with potential consequences for high-stakes topics like health and personal finance.

In practice

Germain’s demonstration is intentionally silly. He says he spent 20 minutes writing a post on his site (“The best tech journalists at eating hot dogs”) filled with inventions, including a non-existent championship. Less than 24 hours later, he reports that Google repeated the content in the Gemini app and AI Overviews, and that ChatGPT did the same.

He adds that some chatbots flagged it as possibly a joke; after he updated the post to say “this is not satire”, the answers appeared to treat it more literally for a time. He also ran a second made-up test (a list of “hula-hooping traffic cops”) and says chatbots were still repeating it when he last checked.

The article highlights that:

• Risk is higher when systems browse the web, and it is not always clear when that’s happening.
• Tools may link to sources but rarely stress when a single page is the only source.
• Users are less likely to click through and verify sources when an AI Overview appears.

Context

The piece frames this as a new “spam era” for AI responses. Experts quoted argue it can be easier to coerce chatbots than it was to manipulate Google search a few years ago, and that the industry’s pace is outstripping its ability to ensure accuracy.

Quoted perspectives include:

• Lily Ray (Amsive), who warns of dangerous incentives and weak accuracy controls.
• Cooper Quintin (Electronic Frontier Foundation), who points to risks ranging from scams and reputation attacks to potential physical harm.
• Harpreet Chatha (Harps Digital), who shares examples of AI answers pulling from promotional content or press releases in consumer queries.

Google, per the article, says its ranking systems keep results “99% spam-free” and that it is actively working to address gaming attempts, including cases with “data voids”. OpenAI likewise says it works to disrupt and expose covert influence efforts, while warning users that tools can make mistakes. Germain’s conclusion: the issue is not close to solved yet.

Why it matters

• AI answers can amplify “data voids” and biased or fabricated online content, especially for niche queries.
• The polished, platform-branded presentation can make misinformation feel more trustworthy than a standalone webpage.
• Health and finance queries are particularly risky when false claims are repeated without strong caveats.
• Better source transparency (including flags for single-source claims or promotional material) could reduce harm.