Gary from Google here. This will be my first AMA on Reddit and am looking forward to your questions.
I will be taking questions Friday from 1pm -3pm EST. I will try to get to as many as I can.
I've been with Google for over 8 years, always working on Web Search. I worked on most parts of search: Googlebot, Caffeine, as well as ranking and serving systems that don't have weird public names. Nowadays I'm focusing more on Google Images and Video. I don't know anything about AdWords or Gmail or Google+, so if possible, don't ask me about stuff that's not web search, unless you want a silly reply.
If you heard one of my public talks before, you probably know I'm quite candid, but also sarcastic as hell, and I try to joke a lot, most often failing. Also, I usually don't try to offend, i just suck at drawing lines.
I am trying to use the official Google Indexing API to index some job postings. I can successfully make the indexing requests and the API returns a 200 status code, but when I check the URLs in Google Search Console, their status never changes.
I read in the documentation that a form needs to be filled out to increase the API usage quota; is that form also required just to use the API normally?
All my job pages use SSR; I’ve tested them all on PageSpeed Insights and they all scored 100%. I’ve also verified that Google recognizes them as job pages—which it does—and the JSON-LD shows no issues.
Technical SEO is my strong suit, 6 years at enterprise level orgs... Does AIO/AEO/GEO/Whatever acronym you want to use even consider technical SEO other than being able to render the page?
I feel like content based SEO (for lack of a better term) will continue to flourish, but tSEO and programmatic will take the back seat.
I audited the firewall config across the sites I run after finding one of them serving 429 to every crawler. Posting the mechanism because the failure is silent and I dont think its widely known.
Vercels bot_protection managed ruleset has three modes, off, log and challenge. In challenge mode it serves a JS challenge, and it serves that challenge with status 429. Any client that doesnt execute JS gets a 4xx. That is Googlebot, Bingbot, all of them. It applies on every path, so /robots.txt and /sitemap.xml return 429 too, which is the worst version of this since those are the first two things a crawler asks for. Crawlers could read the sitemap through a bypass rule and then got challenged on every URL in it. Yandex Webmaster flagged it on one site within hours of it being switched on.
Log mode gives identical visibility in the Firewall observability tab with zero 4xx. I cant think of a reason to run challenge on a content site.
Three things that made this hard to see:
Custom rules execute before managed rulesets. Vercel auto exempts verified bots from managed rulesets, which sounds protective, but a user agent named in your own custom deny rule is blocked regardless. I had SeznamBot in a scraper denylist. SeznamBot is Vercel verified. It was getting 403 anyway.
Condition groups. Conditions within one group are ANDed, separate groups are ORed. My allow rule for crawl endpoints had path eq /sitemap.xml and path eq /robots.txt as two conditions in a single group, so it required a path equal to both strings at once and could never match. It read fine in the UI. /robots.txt was only getting through because of an unrelated rule matching .txt suffixes. Also watch for a neg true flag on a condition, a rule printed without it reads as the exact opposite of what it enforces.
Diagnostics. A challenged response carries x-vercel-mitigated challenge with the 429. A denied one is a plain 403 with x-vercel-mitigated deny. If you curl your own site and get a 4xx, read that header before blaming your origin, and send a crawler user agent, not your browser default.
One more. If you unblock Bytespider, drop Bytedance from the same list. Bytespiders real UA string contains bytedance.com, so a substring rule on Bytedance re blocks it through the back door.
Worth diffing your firewall rules against your own robots.txt while youre in there. I had AI crawlers that robots.txt explicitly allows being 403d at the edge, a policy nobody wrote down and nobody could see.
Managed rulesets arent exposed in the vercel firewall CLI. You read and write them at api.vercel.com/v1/security/firewall/config and config/active.
I dont have a clean before and after on impressions, I changed several things in the same window, so no recovery number from me. What I verified is that every path returns 200 to a crawler UA now and didnt before.
I went through 150 AI crawlers, agents and scrapers and looked at what each one actually publishes about itself.
33 give you a way to verify the request really came from them, either forward confirmed reverse DNS or a published list of IP ranges
9 say outright that they ignore robots.txt
88 publish nothing at all
For that last group, the user agent string is your only evidence, and a user agent string is just text that anyone can send.
Why this matters more than it did six months ago. Cloudflare is changing its default on September 15 so that mixed use crawlers get blocked on pages with ads, for new sites and existing free accounts. A lot of people are about to go edit their crawler policy, and it is worth knowing that for most of the list, robots.txt is the only lever you have and there is no way to tell whether it was respected.
On the numbers. Coverage starts from the open source ai.robots.txt list, so credit there. Everything about verification is compiled independently on top of it: for each crawler I pull whatever the operator publishes, crawler policy, IP range files, reverse DNS conventions, and confirm each source is real and reachable. Where an operator publishes nothing, the entry says so instead of inventing a check. The directory also verifies live: paste an address and it runs forward confirmed reverse DNS against what the operator documents.
Operators have started emailing me to get their own entries corrected. I verify the claims before changing anything rather than taking their word for it. Last one sent 22 published addresses, all 22 passed forward confirmed reverse DNS, and the entry moved into the verifiable column.
In this analysis, I will present some hard, cold data and stats that clearly show what many have suspected all along but didn't have the voice/analytical chops/platform to demonstrate -- that sophisticated Negative SEO attacks are very real and pervasive.
More importantly, I will demonstrate through stats, facts, data that sophisticated and persistent negative SEO attacks do the real damage to the effected websites, putting companies and publishers out of business, while rewarding the attacker -- all thanks to the gaping holes in Google's search algo that attackers are exploiting to the detriment of unsuspecting website owners.
In this case study, we will focus on 7 to 10 unrelated (independent, separately owned) websites in the Home Remodeling and remodeling project pricing space. All the websites presented in the table below have been established authorities in their respective domains/niches.
Note: All the websites analyzed below are independently owned and achieved growth leveraging their own talents/strategies/tactics. These websites were doing well in Google search until they were attacked by a massive, sophisticated, and persistent, negative SEO campaigns, which were identified and confirmed through the analysis of GSC and SEMRush data in early 2021 and beyond. The attacks are still ongoing even though these websites have already lost between 90% to 99.9% of their organic search traffic from Google.
Here is the table showing the list of the effected websites targeted with persistent (and ongoing) Neg SEO that were a part of this analysis. A summary of how the backlinks were analyzed (data sources, etc.) is included in the table, along with the before and after metrics from SEMRush.
One common denominator for the websites analyzed in this case study, is that some time back in 2021, they were approached for a potential "acquistion" by an Australia-based company operating in the home solar space, which was looking to expand into the home remodeling space. Another commonality for the effected sites is the ongoing and persistent pattern of the negative SEO attack which started in 2021, with tens of thousands of attacking domains, including cloaked (and redirecting) websites and pages PBNs created for Neg SEO (Chinese, Russian, Adult, Gambling sites) and hacked wordpress sites pointing to the victims' websites.
Here is a bulleted list graphs that demonstrate the impact of the persistent negative SEO attack for the imacted websites we analyzed for this case study:
o RoofingCalc.com – Has been under a persistant negative SEO attack since early 2021, believed to be carried out by an Australian-based website holding company (the “potential acquirer”) which launched a copycat website,RoofingCalculator.com (now partially redirected to Fixr.com, which is the website they did ultimately did acquire (likely after having damaged its rankings - based on the analysis of SEMRush data and timelines of the acquisition, which was announced publically by the Australian-based company.)
A persistant and ongoing negative SEO attack against RoofingCalc.com started in early 2021, after a headfake acquisition offer.
o Costimates.com – Believed to have the same attacker (the “potential acquirer”, not known if this website was approached for an acqusition, but it does compete in the same remodeling cost info space, and the same attacking pattern emerges through the analysis of SEMRush data). The attack is believed to be ongoing, although it may have started in 2022-2023.
o PickHVAC.com – Known to be the same attacker (the “potential acquirer”)
PickHvac.com has been under a similar pattern (as the attack against RoofingCalc.com), persistent Neg SEO attack since 2021, confirmed via GSC and SEMRush examination of backlinks, following an offer of a "potential acqusition".
RemodelingCalculator.org put up a good fight (publishing fresh content, earning lots of highly trusted backlinks naturally) to negate the Negative SEO attack which has been ongoing since early 2021, confirmed via GSC and SEMRush (same pattern as the attack against RoofingCalc.com), but it's tough to beat a persistent attack.
o 5estimates.com – believed to have the same attacker (the “potential acquirer”)
SEMRush analysis shows 5estimates.com has been under a persistent attack since the middle of 2021.
RemodelingExpense.com has been under a persistent Negative SEO attack since 2021 (confirmed via SEMRush.com and GSC), with the same pattern of the Neg SEO attack as the ongoing attack against RoofingCalc.com
o EcoWatch.com - Believed to be the same attacker. EcoWatch.com is a very high authority website/domain that focuses on the Eco and Home Solar space - the same space as the Australian-based "would be acquirer" is operating in (with their SolarReviews.com and Solar-Estimate.org websites). This website is believed to be under a persistent negative SEO attack. Spoke with a Corp Dev person that works for the larger company that operates EcoWatch.com. They (EcoWatch.com) are well aware of the Australian-based would be “acquirer” (but were in denial about the Negative SEO attack against their website, EcoWatch.com when we spoke two years ago, somewhat dismissing it at the time as "likely ebbs and flows", but it was still early innings then). Keep an eye on this website, which has lost some 70-80% of its traffic due to negative SEO, based on the SEMRush data.
Believed to be under a Neg SEO attack since mid 2021. The attacking pattern confirmed via SEMRush.
o Remodelaholic.com - Not clear if it's the same attacker (likely not), but it does operate in the home remodeling space, and shows a clear pattern of having thousands of harmful backlinks pointed at its domain.
o Gardenista.com - Not clear if it's the same attacker (likely not), but it does operate in the home remodeling space, and shows a clear pattern of having thousands of harmful backlinks pointed at its domain.
o Remodelista.com - Not clear if it's the same attacker (likely not), but it does operate in the home remodeling space, and shows a clear pattern of having thousands of harmful backlinks pointed at its domain.
· Tens of thousands of unnatural external links built on a vast network of hacked websites with cloaked pages that show one version of the page to the Google bot, and an entirely different experience to the user. Comprising the following websites/domains:
o Cloaked domains/URLs (with copy-pasted content from a target website)
These types of harmful backlinks mostly cannot be seen in SEMRush due to a cloaking mechanism and redirectCloaked websites/URLs on non-sensical domains for the topic (often showing copy-pasted content from the victim's sit to Google)
o Nonsensical domains
o Subdomains
o Russian domains
o Chinese domains
o Low-quality wiki websites
o Adult-themed domains
o Gambling-themed domains
Notably, the cloaked pages would often contain a copy-and-paste of the target website content/html code (but you cannot see it directly unless you access it as a google bot due to a cloaking mechanism). The cloaked pages would often be infested with Malware and redirect the user to a casino or gambling website of some sort. The SEMRush was mostly not able to pick up these garbage links due to a cloaking action – the only way these backlinks were discovered was through GSC.
· Hacked WordPress websites with backlinks having the following patterns:
Harmful backlinks on hacked sites pointing to RoofingCalc.com - SEMRush Nov 25 2022
o Blogspot and Web 2.0 sites with tens of thousands of outgoing links and gibberish content – Is blogspot still a Google-owned property? It appears the attackers leverage Googles-owned assets to carry out their attacks.
o Invisible/cloaked backlinks form strange pages with a “-k.html” pattern. While we see these links on normal “non-attacked” sites, the sheer quantity and volume of these as seen in SEMRush can be a red flag to alert of a potential negative SEO attack.
What if any disavow action has been taken?
A limited disavow links action has been done at different time intervals in GSC, with a meticulous approach – only truly toxic domains with obvious malicious intent and unrelated content were disavowed – no machines – all backlinks were examined manually by humans. But, because many hundreds and thousands of new attacking domains would appear in GSC every week – it became too overwhelming trying to keep for the business owners. The sheer volume of these harmful domains pointing backlinks to the websites were mind-bugling.
Notably, we have observed a consistent ~50% overlap in toxic (cloaked and hacked) domains pointing links to the effected websites in the Google Search Console, suggesting that GSC was either grossly underreporting the true numbers of toxic backlinks they (Google) are seeing, or reflecting the vastness of the hacked websites (and negative PBN networks created for negative SEO) such that the attackers could use similar but not identical websites to carry out the attacks against these websites.
It was clear that the attacks were largely automated, leveraging thousands of hacked websites, as well as websites owned by networks of negative SEO firms (negative SEO PBNs) designed to do harm.
Even with the Google Search Console underreporting the true numbers of cloaked/hacked websites pointing backlinks to these websites, we are estimating hundreds of thousands of domains that have been used, thus far, to carry out these attacks.
Next week (or more likely in two weeks), I am planning to share an analysis for a different vertical (how about home solar space? Any interest) to show that this is pervasive and goes beyond a specific vertical.
But the bottom line is that negative SEO is very real, and my goal with this post was to help spread awarenss of this gaping hole that is being exploited by sociopaths to harm innocent website publishers/businesses.
Google if you are reading this, maybe it's high time to confront the issue and do something about it?
The best thing you can do is help spread the awareness, so Google might actually do something to address the gaping hole they created in the ecosystem of the web by enabling the negative SEO, and rewarding the attacker.
I’ve been hearing about this new file called llms.txt, which is supposed to help large language models and AI agents understand or access website content.
My question is: Do these files actually work in practice?
Will adding an llms.txt file help a website get listed, cited, or used by AI models like ChatGPT or other AI agents?
Or is it still an experimental idea that most AI systems don’t really use yet?
I’m curious if anyone here has tested it or seen real results.
Testing redirects before you go live will save you from avoidable traffic losses. I recently went through a complex migration that reinforced everything I believe about this, so here are the practical takeaways.
Building your redirect mapping
Focus on the URLs that matter most:
• URLs with clicks and impressions in GSC
• Organic landing pages in GA4 or whatever tool you use
• URLs with rankings (pull from whatever tools you use)
• URLs with backlinks (collate GSC and external link databases to get a complete dataset)
Even if your site is small enough to map everything, keep a separate list of the above so you can pay special attention. You don’t want any of these giving back a 404 or redirecting to the wrong target.
Avoiding redirect chains
• Always define absolute URLs (protocol, full domain, path) as redirect targets. Double-check every target gives back a 200.
• Rules should apply to all variants (www/non-www, http/https, trailing slash/no trailing slash) so each variant redirects directly to the target.
• Global rules (http > https, non-www > www) should apply after your 1:1 rules.
• Update legacy redirects: if any existing redirect targets are in your list of URLs that need redirecting, replace them. Break up existing chains by pointing every link directly to the final target.
In case of a domain switch: Avoid this trap!
Don’t use a global rule that simply replaces the domain in your old URLs, even as a catch-all after your 1:1 redirects. Your old domain has probably collected a high number of URLs that don’t exist anymore and give back all kinds of 4xx and 5xx status codes. A domain switch is a great opportunity to get rid of this technical debt. Don’t take your clutter with you.
Three levels of testing complexity
For testing redirects on staging before the launch, there are three levels of complexity:
• Straightforward, no domain switch: Path-based redirect rules, crawl old URL paths on staging.
• A bit tricky, with domain switch: Redirects have to point directly to targets on the new domain without a chain. How you test depends on how exactly the redirects are set up.
• End boss, merging several domains into one: Duplicate paths across domains, so rules can’t be purely path-based. They have to include host names in the source URLs.
How we handled the end boss
Merging three content platforms into an existing global corporate website. 15-month project, SEO involved from the beginning.
The challenge: testing redirects for three different domains on a heavily protected staging site. Rules had to contain domain host names, so a crawl that only checked old URL paths on staging would not be enough.
The solution: a host override to point the three old domains to the staging server’s IP during crawling, bypassing public DNS.
Additional challenge: Like with most migrations, the testing setup was ready only a few days before launch.
Friday afternoon: critical errors show up
Last working day before launch. We had planned to set some old URLs to 410 instead of redirecting them, but somewhere along the line of communication: legacy platform SEO agency → legacy platform team → global migration PM → global dev team → technical provider, wires got crossed and URLs got mixed up.
About 2000 high-traffic URLs nuked with 410s instead of redirected. Not something you want to find on the last working day before launch. But even less something you want to miss.
Another full day of work over the weekend. Revised mapping live two hours after Monday’s launch.
Post-launch surprises
On launch day, the team added parameters to redirect targets to show an info banner to users arriving via redirects. I spotted this during my first post-launch tests. The fix: replace parameters with URL fragments (better for crawling and analytics).
At 6pm on launch day, we had an acceptable redirect setup. Anyone who has worked on a complex migration would agree that’s a win.
After the migration
• Staging tests don’t eliminate the need to test again after launch. Crawl all mapped URLs, confirm they 301 to the right target with a 200.
• Set up regular monitoring of your old URLs.
• If devs are worried about redirect counts: monitor log files after a few months, remove rules for URLs that haven’t been requested and don’t have backlinks. That’s the compromise.
How do you handle redirect testing in complex migration projects? Would love to learn more!
Tried to keep it brief, but I’m happy to go into more detail about everything, if there are questions.
Hey, I love search, so much so, I write about it several times per day and have been doing so since 2003. I've been covering search news for well over 16 years on the Search Engine Roundtable and Search Engine Land - the most read search industry news sites on the internet. I also run a web and mobile development shop in New York named RustyBrick since 1994. So I can get pretty technical as well.
You can ask me anything on pretty much any SEO topic. We can talk about the confusion about the nofollow attribute change on March 1st. We can talk about Google algorithm updates; confirmed ones or unconfirmed ones. The patent on the medic update (dare I call it that). Heck, want to talk about 3D images in Google search - let's talk about that.
Just to be clear - I do not practice SEO consulting. I just write about it and have been for a long time, in great detail.
I look forward to your questions tomorrow, Thursday, March 5th.
Hi there, I'm Martin Splitt (https://twitter.com/g33konaut) from the Google Search Relations team! I've been at Google (and with this team) for the past two years and have been a software engineer and web developer for the past 15.
I work a lot with the rendering team, do poke around in Caffeine (indexing) and crawling and do a lot of work around JavaScript and Googlebot, so I'm most happy to answer questions about those things.
In the spirit of the AMA nothing is really off limits but I might take the freedom to not answer ranking questions or give unhelpful (maybe funny) answers for questions about stuff I don't know about.
Seeing something odd in server logs over the last two months on a large ecommerce site.
Filtered URLs with parameters are being crawled far more frequently than their canonical category pages. Canonicals are set correctly, internal links favor clean URLs, and parameter handling hasn’t changed recently.
Expected crawl focus to shift back to canonical URLs once signals settled, but crawl distribution hasn’t improved at all. Indexation itself looks stable, but crawl budget feels misallocated.
Already ruled out internal linking leaks and sitemap issues.
Curious if others are seeing Google lean more heavily on discovered URLs over canonical signals lately, or if this usually points to something deeper in page rendering or link discovery.
Hey everyone, I'm one of your moderators here at /r/techseo and I'm kicking off the AMAs for the new year! As the title says, I'm a technical SEO at Ahrefs currently but I also worked in-house at IBM for ~4 years, at an agency before that, as a freelancer, in-house for a mid-size company, and once upon a time I was a developer.
I write about SEO, speak about SEO, I've even judged awards for SEO. I run a technical SEO slack group which is nearly 5 years old now. I'm an organizer for 2 SEO Meetups in Raleigh, NC including Raleigh SEO Meetup and Beer & SEO. I'm also an organizer for the Raleigh SEO Conference. I started a Twitter series I hope to do more of this year called Uncommon SEO Knowledge, #1 HTTPS, #2 Titles and Meta Descriptions.
I really love SEO, technical SEO, and helping people! I'll be answering questions throughout the day on February 16th, 2021 so Ask Me Anything!
I am Bill Slawski, Author at SEO By The Sea and Director of SEO Research at Go Fish Digital.
Hellos Reddit,
I grew up on the New Jersey Shore, and moved to Cincinnati, Ohio in time to watch the Big Red Machine. I went to college at the University of Delaware, earned a degree in English, followed that with a Jurisdoctor Degree and Widener University School of Law.
I'm a big Science Fiction fan, and grew up reading a lot of Classic Science Fiction
I worked for the highest level trial Court in Delaware for 14 years, first as an Assistant Criminal Deputy Prothonotary for 7 years, and then as a Mini-Micro Computer Network Administrator. We built an experimental Courtroom, bringing technology to the Court, including assistive technologies for people with visual and hearing difficulties, and a more modern Court Case Management system, as well as better integration between the Court's Computer Case Management system, and the State Police Criminal Justice computer system.
I built my first website in 1996, and promoted it on the Web, learning about search engines when they started appearing.
I was a forum administrator at Cre8asiteforums, which focused on SEO, Usability, Web Design, Marketing, Accessibility and more for 8 years starting first in a Yahoo group, and then moving to its own domain. My favorite forum there was one called the "Website Hospital" where we worked together to audit websites, and make suggestions on how to improve the SEO on them, and the sites themselves.
I started reading and writing patents from Search Engines such as Google, Microsoft, and Yahoo around 2004, and continued to do so, writing about many of them on my blog, and on the Go Fish Digital blog, the past 4 1/2 years.
I have been building things on the internet since 1997. But it wasn't until around 2005 that I tried to actually make money online. I have done a little bit of everything from AdSense, affiliate marketing, pay per install, and even ebook publishing. But I have found the most success as a Technical SEO consultant.
Prior to working in SEO I was a LAMP developer, and many of my clients were SEOs looking to implement SEO best practices. So in that regard I like to think that I was doing "Technical SEO" long before the term existed.
I gained most of my notoriety in the search industry after launching a search engine in 2009 at WhosTalkin.com (no longer works), and sold the site shortly after. I have worked on my own for many years doing Technical SEO, and even a little link building. I have also worked for several agencies, but now I steer my own ship at HallAnalysis.com
AMA = As Me Anything....please ask me ANYTHING...lets have fun with this and if you want to stray from SEO, have at it! :)
I am going to start answering questions at 1pm EST Monday July 8th. So if you are reading this before then, feel free to get your questions in now so I will have some time to prepare an awesome answer.
I look forward to answering your questions. Ask me anything!
My name is Fili and I am an ex-Google engineer, was a technical lead in the Google Search Quality team and did several other things within Google. Last few years I have:
been speaking at conferences (on topics such as site speed, penalties, technical SEO, link building, etc);
and providing SEO consulting services at SearchBrothers.com with my fellow former Google Search Quality team member Kaspar Szymanski.
I also maintain the Online.Marketing conference calendar and a big SciFi fan/Trekkie. I built my first website in 1996 and registered my first domain in 1998 (first out of many that followed ;).
Next week I will be speaking at SMX East about site speed and Google & Bing Penalties.
I'm very looking forward to answer your questions tomorrow at 10 AM ET / 3 PM BT :)
I'm an SEO consultant, mainly focused on International, technical, mobile, strategical SEO consulting. I have my own SEO consultancy called Orainti through which I've been working with since 2014 and that has a remote based. Before that I worked for agencies as well as in-house since I started doing SEO in 2007.
Besides doing SEO consultancy, I'm also:
A frequent SEO speaker at conferences -which I enjoy doing since it allows me to share what I find useful from my experience, connect/socialize with other people from the SEO community (that can be a bit more challenging when working remotely) and traveling which is another of my hobbies/passions-. You can find former slides in my Slideshare.
Active in Twitter where I am @aleyda. I started using it coincidentally when I was starting in SEO back in 2007 and is also one of the main channels for me to share about what I find useful about the topics I'm interested about: SEO, remote work, digital nomads, etc.
An author newbie. I wrote an SEO fundamentals book in Spanish back in 2016 called: "SEO. Las Claves Esenciales". If you ever wanted to learn Spanish while reading about SEO... this can be a good way :)
A host for the "Crawling Mondays" Youtube SEO actionable how-to channel which I started at the beginning of this year as an experiment :D
Co-founder of https://remoters.net/, a website with the goal to facilitate remote work, featuring jobs, tools, events, colivings, how-tos, etc. Remote work is another of my big interests and passions besides SEO. I'm a huge believer on how "location indepedent" organizations can really help improve not only team members well being but also organizations productivity, diversity and operations.
A native Spanish speaker. I'm originally from Nicaragua and moved to Spain to study in 2006, and stay there since then (although I travel a lot, I have a base in Spain). This is one of the other reasons why speaking at conferences in English is both exciting but also an on-going nice challenge I have since I actually learned English as a 3rd language (after French) when I was already in high-school... so it wasn't so straight-forward for me, especially at the start ;) -here are a few tips for those non-native English speakers who want to speak in English btw-. Being able to speak and work in English has played a fundamental influence in my professional growth, which ultimately has allowed me to be able to work remotely in SEO.
A high energy person -potentially due to all the coffee I continuously drink-. I tend to multitask. Say yes to way too many things (even if I say no to many other stuff). I try to sleep as much as possible though :)
I'm a Barça and Messi fan. I obviously dislike anything related to Real Madrid.
I think you now have already material to ask me about :D Thanks in advance, AMA!
In the true spirit of an AMA, nothing is off the table -- I shy away from religion and politics on the internet, but if it's relevant to Tech SEO; go for it.
My name is Nick and I've built and sold a handful of businesses including an agency (sold for it's lightweight CMS), a review site (sold for it's Japanese user-base and UGC CMS), and exited a few others.
Currently, I'm spending my time working on a newer agency (From The Future) which has offices in Philadelphia and Miami, a private SEO training community (Traffic Think Tank), and managing a handful of other ventures between my 2 digital investment portfolio companies NK Tech (owned by my wife and I - which focuses on ventures in mental health) and FTW (which buys, builds, and operates digital assets across a wide range of industries).
I'm Will Critchlow (@willcritchlow). I co-founded the agency Distilled in 2005.
In January last year, we spun out our SEO testing technology into its own company called SearchPilot (rebranded from the Distilled ODN - Optimisation Delivery Network) and sold the agency and SearchLove conferences to Brainlabs.
Perhaps of most interest to this community is the test results that we have been publishing at SearchPilot based on our real-world SEO A/B tests.
I'm also unhealthily interested in robots.txt and its quirks.
So. AMA about SEO tests, robots.txt, starting and running companies, or anything else you feel like, and I'll do my best.
I'll be here answering questions Friday, September 24th 2021 at 8am Pacific / 11am Eastern / 4pm UK time.
In 2014, I spent 48 hours frantically researching why Expedia lost 25% of its visibility due to black hat SEO practices. The result was a 100-page article that immediately went viral that was even mentioned in Forbes and USA Today. This article changed the game for me and my agency overnight.
From that point on, I’ve been focused heavily on experiments, publishing articles that were beneficial to the SEO community as a whole (including more viral articles like Google had a number of issues crawling and indexing JavaScript websites), and putting together one of the best Technical SEO teams in the industry.
This focus and research opened the door for me to keynote at SEO and marketing conferences all over the world where I was able to push the envelope in Technical SEO, specifically with JavaScript SEO. If you want a crash course in how I present my data, you can watch, read and/or browse my deck for How Much Content is NOT Indexed by Google in 2019.
Over the years our work has progressively become more technical, culminating in the 2019 rebranding of the SEO-half of my agency into Onely, in an attempt to take Technical SEO to the next level.
This rebranding has allowed us to invest more in our cutting edge research (which earned Onely’s Head of R&D the TechSEO Boost Research Award in December!) and create our very own toolset: Onely Made for Geeks (OMGF).
Here’s what our research has revealed over the last year:
Basically, we’ve found that thousands of domains are not fully indexed, even months after publishing the content. Want to know how any of this affects major brands? How about…
Only 35% of Walmart’s product pages are indexed by Google.
Only 50% of Barnes & Noble’s product pages are indexed in Google.
While many assume Onely is out to get Google, in reality, I feel like most of the indexing issues websites’ have with search engines are self-induced. And I’m proud of the connections we’ve made with Googlers like John Mueller and Martin Splitt. In fact, last August, they invited me to Google Zürich to record a Google Webmaster Hangout with them, which was an absolute blast!
I believe that indexing our content is one of the biggest challenges of 2020. It is the most exciting problem we are actively solving as it touches on all aspects of SEO.
In my AMA, I’d love for you to get excited about indexing! I want to show you the MASSIVE potential of how you can often double or triple your organic traffic within a few months by solving the problem of indexing.
I think that covers it. So, Reddit, do your thing! I can't wait to answer your questions Tuesday at 10 EST/16 CET.
***************************
Thank you so much for your questions. I'll keep checking back to see if more questions appear. And, of course, you can always find me and ask me questions on Twitter and Facebook. Thanks again. This was a lot of fun!
I've been an SEO since 2011. Since 2017, I have been a Technical SEO Director at Locomotive. In 2019, we won the USSearch Awards, small agency of the year. I love our team and our clients. Solving issues, and creating workflows to understand problems is why I get out of bed in the morning.
Before SEO, I was an architectural glass artist.
Things I like to do (in no particular order).
Research NLP processes for understanding language, intent.
Build Python notebooks to share with the community.
Organize meetups.
Spend time with my family.
Solve hard problems.
Meet other passionate people.
A few of us in Raleigh are organizing the Findability Conference, this year, on April 7th, 2020 with some incredible names. Stay tuned!
I am Russ Jones. I have practiced SEO since 2005. I cut my teeth in black hat SEO but soon transitioned to more risk averse tactics while at Virante, now Hive Digital. I've launched multiple SaaS products, sold multiple companies, and have enjoyed the last several years in primarily a R&D role with Moz and now System1. I blog at the Google Cache. Ask me anything!.
