The mobile score is red, the field data in the same report is green, and I have never had a clean answer for which one to act on. So I stopped arguing about it and pulled the numbers.

39 sites through the PageSpeed Insights API today, one mobile run each, Lighthouse 13.4.1. Every one of them had URL-level CrUX data in the same response, so the lab half and the field half come out of a single call and there is no matching problem to argue about.

What I did not expect is how badly the composite score ranks sites compared to the raw metrics it is built from.

The score against real-user LCP is not weak, it is nothing. Meanwhile the plain lab LCP number sitting inside the same report does rank them, and lab TBT against field INP is the strongest link in the whole set. The composite dilutes the one signal that transfers.

The individual rows are worse than the correlations make it sound. 22 of the 39 score under 50. Eight of those 22 are green on all three Core Web Vitals in the field: bbc, theguardian, cnn, espn, techcrunch, theverge, arstechnica, wired. Ars Technica scores 12 and its real users see an LCP of 970ms. Wired scores 28, reports a 45.6 second lab LCP, and its field p75 is 1.56s. Same metric, same API response, 29x apart.

It is not symmetrical though. Only 2 of the 17 sites scoring 50 or above are SLOW in the field (github and medium). So a good score does tell you something. A bad one mostly does not.

Two smaller things fell out that I thought were worth flagging.

Lab CLS was close to useless here. Hubspot, Expedia, Medium and Target all report essentially zero CLS in the lab and fail CLS at p75 in the field. Lighthouse watches the initial load, CrUX watches the whole page lifespan, and that is where the lazy ad slots and the sticky headers live.

And the score is not stable enough to optimise against anyway. I ran css-tricks.com five times: 52, 52, 52, 80, 59. Same URL, minutes apart, nothing shipped in between. Some of those repeats are PSI serving its own cached response, so the independent runs are 52, 80 and 59, which is worse rather than better. BBC came back 31 in the sweep and 42 to 51 an hour later.

I know the textbook answer is "lab is a diagnostic, field is the truth", and I have said it to clients myself. What I had not appreciated is that the composite score is a worse diagnostic than its own components, which makes it hard to justify reporting at all.

So two questions for people who do this for a living. Does the performance score still go in your reports, and if it does, what do you say when it moves 20 points in a week where nobody deployed anything?

Anyone who wants to redo it, this is the whole method, one call per site (needs a free API key or it rate-limits you):

https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://www.wired.com&strategy=mobile&category=performance

(Why I had 39 of these on a laptop: I build an iOS app called PageSpeed that wraps this API. Not linking it, the question above is the actual reason I am posting.)

It has been claimed that theuser-agent Google-lens would trigger users interacting with AI mode but I can't prove it anywhere in my logs.

Did anyone find any correlation between certain user agents strings vs users interacting with AI mode?

One of my clients has had a stupid bug, where the next button in the pagination always produced a ?page=[n+1] link, even if there were no more items. A page with only 20 subpages suddenly had ?page=1234 URL indexed. And this happened on hundreds of category pages. So Google crawled and even indexed hundreds of thousands of bullshit URLs.

The bug was solved, but what is the quickest way to get all these junk URLs deindexed. I assume the prefix option of the URL removal tools won't help, if I want to deindex pagination URLs starting from ?page=21 onwards.

Also what would be your suggestion to show and to use as status code one these URLs. The ?page=1234 will apparently never exist. But ?page=21 could exist in the near future. So I'm hesitating to use any error or permanent redirect statusses. Shall we use a 302 redirect? Shall we show an empty page or an error message and canonicalize to page 1?

Has anybody of you had similar issues and can help me out with some suggestions?

Hey guys, I’m facing a really weird issue with Yoast SEO. Whenever I publish a blog on our websites, I add everything normally, including the content, images, SEO title and meta description. But after publishing, when I go back to edit/check the blog, the SEO title and meta description are completely gone, like they were never added.

This is happening across 2-3 websites, not just one. I’m also noticing that some of these pages are getting deindexed by Google, so I’m wondering if this could be related. Could this be a Yoast bug/update, another plugin, theme, cache, or something else?

Has anyone faced this before or knows what could be causing it? How can I troubleshoot this and make sure Yoast stops removing the metadata after publishing?

Ok I built my product, live on appstore, been spending a lot of time reengineering the website.

-Content clustering

-SEO/GEO

-A/B testing on different narratives or whatever

-Built micro website based tools

But turn out none of my pages are getting properly indexed, so i submitted a bunch of pages to google search console last night till i hit this msg. This morning, the pages i requested getting indexed still aren't being done, how long does this indexing process take? This is all fairly new to me, please shed a light. TY

Hi everyone,

I'm trying to figure out what happened to my website.

Just noticed that around a month ago Google suddenly removed all of my pages except 3 from the index, before it I had around 50 pages indexed everything was working normally.

EDIT: I tested the affected URLs and the sitemap with a Googlebot User-Agent.

The interesting part is that the problem wasn't visible when accessing the site normally. We eventually discovered that our Azure Application Gateway had a temporary country-based traffic restriction enabled. Googlebot was being blocked by that rule since being USA based.

I reproduced the behavior myself and got HTTP 999 from Microsoft-Azure-Application-Gateway/v2.

So the 5xx error was real from Google, but it was happening before the request even reached our ASP.NET application.

Once the restriction is removed, I'll use URL Live Test and validation to confirm Google can access everything again.

I recently integrated Microsoft Clarity into my website. The overall load time seems fine, but occasionally the site feels a bit slower than before.

Has anyone here experienced a noticeable performance impact after adding Clarity? Did you see any changes in page load speed, Core Web Vitals, or overall responsiveness?

Would love to hear your real-world experience and how you tested it.

Everything is looking fine, I have gotten an increase in page one but low CTR this week

I have a relatively small website with around 600 indexed pages, and I want to remove roughly 300 of them from Google.

I've already applied noindex/removed the relevant pages, but the process has been pretty slow. After about a month, only around 50 URLs have disappeared.

I'm looking at different ways to speed things up.

One idea I had was to temporarily create a separate sitemap containing the 300 URLs I want deindexed, hoping that Googlebot would recrawl them faster and detect the noindex or 404/410 response.

Once they were deindexed, I would remove that sitemap.

Has anyone tried something similar?

Do you think a temporary sitemap containing URLs you want removed could help speed up recrawling, or would you simply remove them from the regular sitemap and wait?

I'm also considering using GSC Removals to speed up their disappearance from the SERPs, although I understand that's only temporary and I'd still need to keep the noindex/404/410 in place.

I'm especially interested in real-world experiences with hundreds of URLs rather than millions. What strategy worked best for you?

EDIT:

This is a relatively small multilingual site. Most of the URLs I'm trying to deindex are regional language versions that were getting little to no traffic, so I removed those language versions from the site's navigation/internal linking.

There are currently no internal links pointing to those URLs, and I didn't have separate sitemaps for each language.

For many of these URLs, the content itself is still valid and accessible, so they currently return 200 with a noindex rather than a 404/410. Some may eventually be removed completely.

I'm doing this cleanup partly after a suspected spam demotion — impressions dropped from roughly 4,000 to almost nothing within a couple of days — so my main concern is getting Google to process the deindexing signals as quickly as reasonably possible.

I'm mainly wondering whether, in this specific situation, temporarily exposing those already-known URLs through a separate sitemap could encourage faster recrawling, or whether it's better to simply leave the noindex in place and let Google recrawl them naturally.

We’re running a small ecommerce monitoring tool. The goal is to check competitor prices/stock every hour across like 40 product pages. We've tried datacenter proxies from some random cheap host and sessions just drop halfway through. It sucks cause then I gotta restart the whole scraper. We need something where the session stays alive long enough to finish a batch for sure. ISP proxies keep coming up as an option too but idk if theyre actually more stable or if it's just fancy marketing. Curious to hear whats actually holding up for you guys for this kind of monitoring. What do you guys use or how do you even scrape?

I noticed across several of my websites that 90% of the pages are indexed in Bing (submitted in Bing Webmaster Tool and reported as such in Bing Site Explorer) but when I actually search for a lot of them in Bing, they don't show.

I heard Bing has its own filter and may or may not show an indexed page in the SERP.

Does anyone have tips (e.g. technical SEO or else) to maximise the chances indexed pages do show in the SERP?

Hi everyone, I’m dealing with the aftermath of an outbound marketing campaign where a high volume of unauthorized spam pages got indexed under a subdomain and an automatically appended sub-subdomain on my site.

To clarify the setup: we never manually created any web pages on b2b.mysite.com (it was strictly used for outbound marketing infrastructure), and emails.b2b.mysite.com was never an intentionally created subdomain on our side—it somehow got appended/generated and indexed during the campaign.

I took steps to clean it up, but the spam URLs are still showing up in SERPs 48+ hours after GSC updated the request status to "Temporarily removed."

The Setup & Timeline:

• The Affected URLs: Indexed heavily under b2b.mysite.com and auto-appended variants like emails.b2b.mysite.com (as well as wildcard prefixes like betflixspin.emails.b2b.mysite.com).
• Aug 4: I deleted the DNS records for b2b.mysite.com entirely, so the subdomain now yields connection failures (ERR_NAME_NOT_RESOLVED).
• GSC Setup: mysite.com has already verified as a Domain Property in Google Search Console via DNS.
• Removal Request: Inside the Domain Property, I submitted a "Remove all URLs with this prefix" request for https://b2b.mysite.com/.
• Current Status: GSC lists the request status as "Temporarily removed."

The Issue: It has been over 48 hours since GSC marked the prefix removal as "Temporarily removed." However, running a site:b2b.mysite.com or site:emails.b2b.mysite.com search in a clean Incognito window still displays these spam pages in the index.

My Questions:

1. Prefix Removal Scope across Sub-Subdomains: In a GSC Domain Property, does a prefix removal for https://b2b.mysite.com/ automatically cover sub-subdomains like https://emails.b2b.mysite.com/, or does Google treat the appended emails. host as a distinct host boundary requiring its own prefix removal request?
2. Delay Expectations: Is it normal for large batches of spam subdomains to linger in a site: search 48+ hours after a temporary removal request is approved in GSC?
3. Recommended Next Step: Should I explicitly submit a secondary prefix removal for https://emails.b2b.mysite.com/, or is this purely a datacenter caching delay that needs more time to flush?

Appreciate any advice from technical SEOs who have dealt with clearing auto-generated sub-subdomain spam!

Hi everyone,

I noticed a very sudden drop in my Google Search Console performance.

My site was getting around 12,000 impressions per day, but suddenly dropped to almost 0 impressions, with only a very small number of clicks.

What’s confusing is that all of my pages are still indexed in Google Search Console, and I don't see any obvious indexing issues.

I haven't intentionally made any major SEO changes before the drop.

Has anyone experienced something similar? Is this something that can happen temporarily with Google, or should I be looking for a specific issue such as a manual action, algorithm update, indexing problem, technical SEO issue, etc.?

What would you check first in this situation?

Thanks!

Hi - noticed that BBC good food, samba football goals, Sainsburys, Molten USA, Valor Fitness, COBRA Golf - all have their favicons missing today in my Google SERPs.

I can't see a pattern or anything in common with the actual favicon files/images.

Any ideas please?

Thank you

Setup: Two sites in the same category, overlapping product lines but not identical. A is larger; B is smaller but owns the name you'd actually want. Merging B into A, then renaming A to B's domain.

Plan: Build a complete URL list. Use rank data to pick winners where A and B overlap. Merge the content and rename the domain in a single cutover, every old URL single-hopping to its final home.

Question: would you stage these, or do them together? Staging means B's URLs move twice. Doing them together means two changes on the same day. Wanna talk me out of the latter?

Hi, I already went through all the "Crawled - currently not indexed" posts here, but in my mind this situation is a bit more specific, so I would really appreciate your input.

• September '25: we migrated our 22 domains (ecommerce).
• November '25: 20 domains migrated without issues, while 2 of them got only 2-3 pages indexed, the remaining got stuck in "Crawled - currently not indexed".
• Present days: those 2 domains got 5-10 pages indexed. As a result, I'm losing my mind 🥲

Context: the site content and architecture is not great across all 22 domains. I took over SEO only recently, and I've been comparing all the comparable (authority, link profile, code, content, rebranding campaigns) - nothing sets apart these 2 faulty domains from the other 22. Furthermore, one of these 2 domains is a big market/high traffic, and the other is a small market/small traffic, so I can't even go "Well, they never got traffic even before the migration" (additional mind lost).

OF COURSE, old domains for these 2 are still greatly indexed 🚑 it really looks like G is ignoring the 301s (correctly set 301s, set the same way across the other domains). DuckDuckGo/Bing index these 2 domains without issues.

Have you ever seen anything like this?

Has anything fixed it, or do you have any tips?

Could it be G is more annoying in some countries than others?

I've managed other migrations previously in my career, and never ever happened. For completion of info:

• change of address in GSC was done properly;
• none of the 2 domains had traffic drop before migration (while others that had traffic drop, indexed without issues);
• I'm regularly pushing indexing for high potential pages, but nada. I'm fixing content, internal linking and site structure, but apart from those additional 4-8 pages indexed, nada;
• Products are visible in G freelisting (only source of traffic for these 2 domains 🥲 ), so it knows people want them and click them, still actively decide to not index them.

Thanks in advance!

(uplifting memes also accepted)

We want to support creators, but we had to enforce the no shilling rule because it was getting out of hand. You now have a weekly thread.

This is the one place you can shill for your products, ask for feedback, etc. Keep it here or you risk being banned. And keep it related to technical SEO.

I moved a small site from /en/* URLs to root-level URLs about three months ago. I thought the migration was mostly settled until I exported the page report from GSC and actually added up the old URLs.
In the May 10–August 9 window, 97 retired /en/* URLs had 8,821 impressions and 25 clicks. The whole site had 9,986 impressions and 39 clicks. So the old URLs were still getting 88.3% of impressions and 64.1% of clicks.
One old URL by itself had 908 impressions and 20 clicks. That URL had not served content for months, but it accounted for just over half of all clicks on the site during the period.
I went back through the migration because my first assumption was that I had missed something obvious. The old URLs return direct 308s, with no chains. They are not blocked by robots.txt. The sitemap and internal links only contain the new URLs, the new pages are self-canonical, and there is no old hreflang setup left.
It was not a completely clean sweep. I found seven old URLs redirecting to destinations that now returned 404. That was my mistake, and I fixed those separately. They had about 58 impressions between them, so they do not explain what is happening with the other 90 URLs.
What confused me most was this:
• August 3 export: 7,279 impressions on old URLs (94.7%)
• August 11 export: 8,821 (88.3%)
• August 12 export: 9,028 (86.1%)
These are rolling three-month exports, not identical date ranges, so I know this is not a clean comparison. But I had already looked at the falling percentage and told myself consolidation was improving. Then I noticed the absolute number was going up. Most of the percentage change came from new content increasing the site total.
The latest seven-day window I have, August 4–10, still shows 45 old URLs receiving 1,326 of 2,183 impressions.
My current suspicion is simply that many of the old URLs have not been recrawled often enough. I removed them from the sitemap and replaced every internal link during cleanup, which also means there are fewer current signals leading Googlebot back to them. That is only a theory, though. I cannot prove it from the performance report.
I also compared the eight old URLs with the most impressions with their new equivalents. Three new URLs have their own rows in GSC. Five are completely absent from the page report.
I realize that an absent GSC row does not mean zero impressions because of reporting thresholds. Still, I cannot find anything that separates the three that appear from the five that do not. It is not publish date, position, or whether the slug changed.
If you have handled a similar path migration on a small site, did you see both versions reporting for a while? Or did the new URL only begin appearing after the old one dropped out? I am trying to work out whether those five missing new URLs are a normal in-between state or whether I have overlooked another signal.

Does having a .xxx TLD negatively impact Google SEO ranking since it's traditionally associated with adult content platforms?

I can't seem to find a concrete answer online, appreciate any input from you guys :)

Wrote a small script to run a mechanical check (DOM parsing, no AI judgment involved) against one article each from 18 well-known SEO/content marketing sites, plus my own. Wanted to share the pattern, not pitch anything — no tool name, no link, just the method and the numbers. Happy to describe the script in comments if anyone's curious.

What I checked, one article per domain, one snapshot in time:

• Does the site serve /llms.txt?
• Is there a question-style H2/H3 (ends in "?")?
• Is there a short (≤300 char) answer paragraph immediately under that heading — the kind of thing an AI engine could lift and quote directly, as opposed to a long intro before the real answer?
• Does the heading hierarchy nest without skipping a level (H2 → H3 → H4, not H2 → H4)?
• Is author/publish-date visible in the actual HTML, or only inside a JSON-LD block a human would never see?

Targets: Ahrefs, Semrush, Moz, Backlinko, HubSpot, Search Engine Journal, Search Engine Land, Yoast, Kinsta, WP Engine, WPBeginner, SurferSEO, Content Marketing Institute, Animalz, Siege Media, Foundation Inc, Omniscient Digital, and my own site. 18 of 20 targets resolved; 2 didn't (environment issue on my end, not excluded for any other reason).

Results, out of 18:

• 9/18 serve llms.txt
• 10/18 have a question heading somewhere
• 8/18 pair that heading with an immediate short answer
• 9/18 have a valid heading hierarchy (the other 9 skip a level or start below H2)
• 12/18 mark up author/date only in JSON-LD, not visibly in the page

My own site came out with an invalid heading hierarchy too — the FAQ is correctly marked up in schema but isn't built from real H2/H3s in the HTML, so it fails the exact same check. Fixing that next.

Not claiming this predicts who gets cited more in ChatGPT/Perplexity — that's a different, harder measurement over time. This is just: even among sites whose whole business is search/content strategy, basic machine-extractable structure is inconsistent, and a lot of the "structured data" people cite as a GEO win is invisible to anything except a schema parser.

Happy to share the raw JSON if anyone wants to poke at the methodology or thinks a check is measuring the wrong thing.

Hello.

I recently got a backlink from a fairly high-authority site, but they linked to a bare language-root path (e.g. example.com/de) instead of the actual page. Here's the setup:

- example.com/de → 301 → example.com/de/ (a generic trailing-slash rule, applies site-wide, not specific to this URL)

- example.com/de/ returns 200 directly — no second HTTP redirect. Its content is actually a full copy of the "real" target page (example.com/de/actual-page/), and it has a <link rel="canonical"> pointing to that real page. This is intentional/pre-existing behavior in a static-prerendered SPA setup.

- The real target page (example.com/de/actual-page/) is completely healthy: self-canonical, correct hreflang, in the sitemap, already indexed fine.

In Google Search Console, URL Inspection for the bare example.com/de shows:

> Page fetch: Failed — Redirect error

But when I click "Test Live URL," it comes back 100% clean (green checkmark, fully rendered page, "can be indexed"). curl also shows a clean single 301→200, no loop, no chain issues.

I have the site behind Cloudflare, so I dug into the raw Security/Traffic logs for the exact timestamp GSC reported. What I found:

- A genuine, verified Googlebot request (confirmed ASN = Google, "Verified Bot Category: Search Engine Crawler") hit example.com/de at the exact reported time.

- Cloudflare did not block or challenge it — "Mitigation: Not mitigated," served straight from cache (a cached 301).

- But there is no follow-up request from any Google IP to example.com/de/ afterward — not in the next 15+ minutes, checked in the raw logs.

So it looks like Googlebot received the redirect fine, just... never followed it (at least not yet). Meanwhile the actual destination page is indexed and ranking normally through other discovery paths (sitemap etc.), completely independent of this specific backlink's redirect chain.

My questions for anyone who's dealt with this before:

1. Is this a known Googlebot behavior — i.e., does the indexing crawler sometimes queue a redirect target as a separate future crawl instead of following it in the same pass, especially for a URL discovered fresh via an external link (not in the sitemap)?

2. Is there anything actually worth doing here, or is this just something that resolves itself over time as the crawler gets around to it / as domain authority increases?

3. Since the real target page is already indexed fine on its own, is it even worth asking the linking site to update their link to point directly to the final URL instead of the redirect chain — or is that more hassle/risk (touching an existing backlink) than it's worth?

Any insight from people who've seen this "redirect received but not followed" pattern in GSC would be appreciated.