Hackers Found a Way to Weaponize AI's Own Mistakes

There's a new phishing trick going around, and it's a genuinely strange one: attackers are exploiting the fact that AI chatbots make things up. Researchers at Palo Alto Networks' Unit 42 call it "phantom squatting" — and it works by targeting a habit large language models can't seem to shake: inventing website addresses that don't actually exist.

Ask an AI assistant enough questions about a company, and sooner or later it'll confidently hand you a URL that was never registered — a domain the model essentially imagined into being, because it sounded plausible. Normally that's just an annoying hallucination. But attackers realized these fake domains follow predictable patterns, meaning you can actually forecast which ones a model is likely to invent — and then register them before anyone else does.

To measure how bad this actually is, Unit 42 fired off over 685,000 prompts about 913 well-known brands at two major AI models. The result: roughly 250,000 unique hallucinated domains, of which more than 13,000 already lead to confirmed malicious content — mostly malware, with phishing pages making up a smaller chunk. In one case, researchers flagged a fake domain mimicking a postal service 23 days before an attacker actually registered it and built a phishing kit on top of it. In another, a hallucinated banking domain sat dormant for 51 days before someone grabbed it, cloned the real site pixel-for-pixel, faked a 4.8-star rating with "2 million users," and used it to push a malicious app.

What makes this genuinely unsettling isn't the phishing itself — it's who's clicking. As AI tools get baked into search, coding assistants, and customer support, people (and increasingly, autonomous AI agents acting on their own) are starting to just trust the links a chatbot hands them, without double-checking. Since these domains were never previously flagged as malicious, they slide right past reputation-based filters that would normally catch a shady link. No phishing email required, no malicious ad — just a model confidently pointing someone toward infrastructure a criminal happened to grab first.

Researchers are drawing a direct line to "slopsquatting," a related trick where attackers register fake software package names that AI coding assistants invent — one campaign already hid malware inside 126 npm packages with over 86,000 downloads. Phantom squatting looks like the same idea, just aimed at entire websites instead of code libraries.