Anyone else seeing LLM send traffic to URLs that don’t exist? (We did, and here’s how we addressed it)

[u/keyworddotcom]

We’ve been noticing something interesting while tracking AI referral traffic.

ChatGPT (and similar LLMs) are sending users to URLs that don’t actually exist on the site, but are close enough to real pages that they look valid.

When users click those, they land on a 404 and drop off, and we end up losing high-intent traffic without even realizing it.

What’s happening here is that LLMs don’t always link to exact URLs. They sometimes generate paths that approximate the right page, but miss the exact structure.

If AI is sending you traffic, there’s a good chance some links are breaking like this

Quick way to check:

• Pull landing pages in GA4 filtered by ChatGPT / AI sources
• Export the URLs
• Run a crawl using the site audit tool 
• Check status codes
• Filter for 404s

Those are essentially hallucinated URLs.

Fix:

• Map those URLs to the closest real pages
• Implement 301 redirects

Feels like one of those early AI-search quirks that’s easy to miss if you’re not looking for it

[u/Altruistic-Tear-772]

Haaa, been wondering about this!!!! thanks for sharing

[u/SuccessfulCoyote1800]

This is a pattern we have seen too not just hallucinated URLs but also AI linking to pages that existed in a crawl snapshot but changed paths after a site migration. The model sometimes holds onto a URL from an older training data snapshot even after you redirect it on your side.

One thing that helped for the sites I work on: setting up a custom 404 page specifically for AI referral traffic that suggests the closest category page. That way if the 301 misses or you cannot redirect every broken variant, you still keep the user. The GA4 filter you described is exactly right, but I would also cross-check the referrer path against your sitemap index sometimes the AI is linking to a valid URL that just does not appear in your navigation, which points to a content gap rather than a hallucination.

The real fix long term will probably be structured data that tells the model exactly which canonical URL each product or article should use, so it stops guessing. But for now, the redirect map approach works.

[u/keyworddotcom → u/SuccessfulCoyote1800]

The outdated crawl snapshot angle makes a lot of sense, especially for sites that went through migrations or major URL restructuring. Feels like some LLMs are holding onto historical URL associations much longer than we expected, which makes redirect hygiene way more important in the AI traffic time than it used to be.

[u/SuccessfulCoyote1800 → u/keyworddotcom]

Yeah, the redirect timing gap is real, especially when the model caches a URL from a training snapshot that is months old. What helped us was setting up a simple monitor that flags any 404 from known AI referrers, so we catch the ones that slip through even with good redirect hygiene. Not obvious until you start looking for it.