What one check covers
The checker runs the same gates Googlebot passes through before a page can be indexed, in roughly the same order, and stops collecting when 30 seconds are up. If the redirect chain alone takes that long, it reports that the site took too long to respond instead of guessing.
- 1.Follows the URL through up to 10 redirects and records every hop.
- 2.Reads the final page, or its first 900 KB when it is larger: its status code, the robots and googlebot meta tags, the X-Robots-Tag header, canonical tags and any canonical in the Link header, the title, the body classes WordPress adds, and the signs of WordPress and an SEO plugin.
- 3.Reads robots.txt for the final hostname, and for the starting hostname too when the redirect crossed hosts, and tests the URL against it as Googlebot.
- 4.Looks for the URL in your sitemaps: the ones robots.txt names, or else /sitemap.xml, /sitemap_index.xml and /wp-sitemap.xml. It reads up to six files, opening post and page sitemaps before the others.
- 5.If the page is noindex on a WordPress site, reads the homepage as well, to tell a site-wide setting from a single post’s setting.
- 6.Turns all of it into findings, each with a severity, the Search Console status it causes and, on WordPress, a hint about where the setting lives.
The starting host matters because Google has to crawl the old URL to see its redirect. A robots.txt on the old host that blocks Googlebot hides the redirect, even when the new site is fine.
How findings become a verdict
Every finding has one of four severities, shown as a label next to it. The verdict at the top is the worst severity present, with one exception: when the page could not be checked at all.
| Label | What raises it | Verdict it produces |
|---|---|---|
| Blocks indexing | A redirect loop or a chain over 10 hops; a final status of 404, 410, 401, 403, any other 4xx, or 5xx; a robots.txt rule that blocks Googlebot; noindex for robots or googlebot | Not indexable as it stands |
| Warning | No response, or a 2xx whose page then failed to load; robots.txt answering 5xx or 429 or not answering; several different canonicals; a canonical to another version or another page; a sitemap that lists a redirect, an error, a noindex page or another version of the URL; an error-page title on a 200 | Indexable, with warnings, or Could not check this URL when the page itself was not read |
| Note | A page over 900 KB read only in part; a file that is not HTML, such as a PDF; a redirect that ends on a 200; nofollow; no canonical; no sitemap or not listed in one; a WordPress archive page that is indexable | Does not change the verdict |
| OK | Each gate that passed: a 2xx status, robots.txt allowing the URL, no noindex, a self-referencing canonical, a sitemap listing | Nothing blocks indexing, when no warning or blocker is present |
Two judgments are worth knowing. A robots.txt that answers with a server error is only a warning here, but it can matter more than any blocker: Google stops crawling the whole site for the first 12 hours that robots.txt answers 5xx. And a URL that does not respond, or whose page fails to load, reads Could not check this URL, because nothing on the page was tested. Run the check again once the site answers. A robots.txt block found in the meantime still reads Not indexable as it stands.
Reading the summary and the panels
Final URL and HTTP status describe where the redirects ended. Every other finding is about that final address, which is the one Google would index.
The robots.txt line shows one result per host checked, separated by a dot when there are two. Allowed with a rule in brackets means an Allow line decided it; Allowed alone means no rule matched; Blocked names the Disallow line. To see why that line won over others, paste the file into the Robots.txt Validator.
Sitemap reads Listed in and the file name, Not found in and a number of files, or No sitemap found. Not found does not mean the URL is missing from every sitemap you have; the checker stops after six files on large sites, and the finding then says so.
Robots directives prints each robots-type meta tag as name: content, plus any X-Robots-Tag header. On a PDF or another file the header is the only place a noindex can go, and it is judged like a meta tag. Tags for other crawlers, such as bingbot, are listed but do not count toward Google’s verdict. The WordPress line names the SEO plugin it recognized: Yoast SEO, Rank Math, All in One SEO or SEOPress.
Below the summary, Findings lists every result from Blocks indexing down to OK, each with its Search Console status in quotes and, on WordPress, where to change the setting. Redirect chain repeats the hops, and Sitemap problems lists any sitemap file that could not be read. The table at the bottom of the tool maps each finding to its Search Console status; this guide does not repeat it.
Worked example: a launch with the wrong box checked
Illustrative values. A small business site was built on a staging copy, moved to https://www.example.com, and three weeks later had almost nothing in Google. The owner checked the services page.
The report
Verdict: Not indexable as it stands. HTTP status: 200. robots.txt: Allowed. Sitemap: Listed in https://www.example.com/page-sitemap.xml. Robots directives: robots: noindex, nofollow. WordPress: Looks like WordPress (Yoast SEO).
Findings: The page asks Google not to index it (Blocks indexing), Listed in the sitemap, but not indexable as listed (Warning), Links on this page are nofollow (Note). The noindex hint said the homepage was noindex too, which points to a site-wide setting.
The cause
Before: Settings → Reading → Discourage search engines from indexing this site: checked
After: Settings → Reading → Discourage search engines from indexing this site: cleared
The box was checked on staging and came along with the database. Since WordPress 5.3 it adds noindex, nofollow to every page rather than changing robots.txt, so robots.txt looked fine while every page was excluded.
After the fix
Verdict: Nothing blocks indexing. Robots directives: robots: index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1. Findings: all OK.
The page cache was purged before re-running. Without that, the old noindex would have been served for hours more.
Then, in Search Console: run URL Inspection’s live test on the homepage and the most important pages, use Request indexing on each, resubmit the sitemap in the Sitemaps report, and open Excluded by ‘noindex’ tag in the Page indexing report and click Validate fix. Expect pages to return gradually over days or weeks, not all at once.
Three more patterns worth recognizing
These come up again and again on WordPress sites. Values are illustrative.
Blocked and noindex at the same time
robots.txt: Blocked (Disallow: /tag/). Findings: Blocked by robots.txt, and The page asks Google not to index it, with the note that robots.txt stops Google from seeing the noindex.
The owner wanted tag pages out of Google and used both tools. Google cannot crawl the page, so it never reads the noindex, and tag URLs that other sites link to stay listed as Indexed, though blocked by robots.txt. Remove the Disallow line, keep the noindex, and let Google recrawl.
A sitemap left over from an old setup
Findings: The sitemap lists a different version of this URL, naming http://example.com/services/ in https://www.example.com/sitemap.xml.
A static sitemap.xml file from before the HTTPS move was still in the site root, and robots.txt pointed to it. Deleting it, pointing the Sitemap line at the SEO plugin’s index and resubmitting in Search Console cleared the warning.
An archive that is noindex on purpose
Verdict: Not indexable as it stands. Finding: The page asks Google not to index it, with the hint that this looks like a tag archive.
The SEO plugin was set to keep tag archives out of search. The verdict is accurate and nothing needs fixing. The section on noindex you should keep lists the usual cases.
Where each blocker is set in WordPress
Fix blockers in the order Google meets them: robots.txt first, because a blocked page cannot show Google anything else; then the status code; then noindex; then the canonical and sitemap warnings.
| Blocker | Where to look on WordPress |
|---|---|
| noindex on every page | Settings → Reading → Discourage search engines from indexing this site. If that is clear, a maintenance or coming-soon plugin, or a host’s staging setting, is the next suspect. |
| noindex on one post or page | Yoast SEO: the Advanced section of the editor sidebar or box, Allow search engines to show this content in search results. Rank Math: the Advanced tab, Robots Meta, No Index. SEOPress: the Advanced tab of the SEO metabox, the noindex checkbox. |
| noindex on a whole content type or archive | Yoast SEO: Settings → Content types, Categories & tags, and Advanced for author, date and media pages. Rank Math: Titles & Meta, the tab for that type. SEOPress: SEO → Titles & Metas, the Post Types, Archives and Taxonomies tabs. |
| noindex in X-Robots-Tag only | Not in the page source, so look at the server or host configuration, the CDN, or a security or headers plugin. Some hosts add it on staging and temporary domains. |
| Blocked by robots.txt | A physical robots.txt in the site root, or the virtual one edited through the SEO plugin. The Robots.txt Validator guide covers both. |
| 404 or 410 | The post is a draft, private, scheduled or trashed, all of which return 404 to visitors, or its address changed. Renamed posts get an automatic 301 from WordPress; renamed pages do not. |
| 401 or 403 | A password-protected directory, a membership plugin, or a firewall or security plugin that rejected the request. |
| 5xx | A PHP error, a plugin conflict or a host resource limit. WordPress returns 503 while it is in maintenance mode during updates. |
A 403 deserves a second look before you change anything. This checker identifies itself as RankHiveTools, and a firewall that blocks unknown tools may still let Googlebot through. If URL Inspection’s live test fetches the page without trouble, the 403 only applies to the checker.
Noindex you should keep
Not indexable as it stands is the right verdict for a good number of pages. Leave noindex in place on:
- WooCommerce cart, checkout and My account pages, which WooCommerce marks noindex itself, and order confirmation pages.
- Internal search results. Yoast SEO and Rank Math noindex them by default, and Google has long recommended keeping auto-generated search result pages out of its results.
- Thank-you, download and form confirmation pages that only make sense after an action.
- Tag, author and date archives you chose to keep out of search, for example author archives on a single-author blog.
- Login, registration and account pages from membership or LMS plugins.
- Staging and development copies, which should also sit behind a password.
Attachment pages are a special case. Since WordPress 6.4, new installs redirect them to the file itself, and Yoast SEO and Rank Math can do the same on older sites. A redirect there is better than a noindex, because the page has nothing a searcher needs.
When nothing is blocking
A clean result with the URL still missing from Google means the reason is not a setting. Search Console usually files those pages under one of two statuses, and they point to different fixes.
Discovered – currently not indexed means Google knows the URL but has not crawled it yet. On a new or small site that is often just time. On an established one, it usually means the page has few internal links, sits deep in pagination, or the site has so many low-value URLs that Google spends its crawling elsewhere. Link to the page from pages that already get traffic, and make sure it is in the sitemap.
Crawled – currently not indexed means Google read the page and decided not to index it, at least for now. Look for near-duplicates on your own site, thin pages that repeat a template with little original text, and pages whose purpose another page already covers better. Improving, merging or removing those works better than requesting indexing again.
The Sitemap line and the Note findings in this report help here: a page that is in no sitemap and that the Internal Link Checker shows with few links pointing to it is a typical Discovered case.
Confirm with Search Console before you validate
This checker shows what your server sends right now. Search Console shows what Google saw on its last visit, and validating a fix there before the live page is right wastes the validation.
- 1.Run this checker until the verdict is the one you want, purging page and CDN caches between runs.
- 2.In Search Console, open URL Inspection and paste the same URL. Under Page indexing, read Crawl allowed, Page fetch and Indexing allowed; they show what Google recorded last time.
- 3.Click Test live URL. If Indexing allowed now says Yes, Google can see your fix.
- 4.Click Request indexing. Google limits how many requests you can make each day, so save it for your key pages.
- 5.In Indexing → Pages, open the reason the page was listed under and click Validate fix. Google rechecks the listed URLs over the following days or weeks.
For a WordPress site connected to RankHive, this runs every week: it reads Search Console’s indexing data, and where the cause is something its plugin can change, such as a robots.txt rule, a canonical or a redirect, it drafts the fix for you to approve before anything is written.
What the checker cannot see
A clean result covers the gates the checker can test from outside. These are the ones it cannot.
- JavaScript. It reads the HTML your server sends. Google renders pages later, so a noindex added by script can still apply. The reverse does not work: Google may skip rendering a page whose HTML says noindex, so removing it with JavaScript does not help.
- The rest of a very large page. Past 900 KB only the start is read, which holds the head and its noindex and canonical, and a note says so. A tag placed later in the HTML is missed.
- Other crawlers. robots.txt and meta tags are judged for Googlebot only. Rules for Bingbot or AI crawlers are not part of the verdict.
- Anything behind a login, a cookie wall or a bot filter that treats this tool differently from Googlebot.
- Soft 404s beyond the obvious. It flags a 200 page whose title reads like an error, such as Page not found. An empty category or a thin page with a normal title passes.
- Quality. Whether Google wants to index a page is a judgment no outside check can make.
FAQ about the Indexability Checker
How is this different from URL Inspection in Search Console?
URL Inspection shows what Google recorded, and it is the authority. This checker shows the live causes side by side, including the exact robots.txt line, every noindex source and the sitemap that lists the URL, and it works on any public URL, including sites you cannot open in Search Console.
Nothing is blocking my page. Why is it still not indexed?
Because indexing is a choice Google makes after the technical gates are open. Pages in Discovered or Crawled – currently not indexed usually need more internal links from pages that already rank, a clearer purpose than similar pages on the site, or simply time on a new site.
Does it run JavaScript?
No. It reads the HTML the server returns. A noindex or canonical added by JavaScript can still reach Google after rendering, so compare with the rendered HTML in URL Inspection’s live test.
Why does it say Not indexable for a page I noindexed on purpose?
The verdict describes the page, not whether that is a mistake. A cart page or a tag archive you keep out of search should read Not indexable as it stands. Act on the verdict only for pages you want in Google.
Should I block a page in robots.txt or noindex it?
Noindex, if the goal is to keep it out of search results. A robots.txt block stops Google from reading the page, including its noindex, so a blocked URL can still be listed when other pages link to it. Use robots.txt to save crawling on URLs that never need to be read, such as endless filter combinations.
How long after a fix will the page be indexed?
There is no fixed time. An important page with a Request indexing often comes back within days; a large set of URLs under Validate fix can take weeks. Re-run this checker first so you know the live page is right while you wait.
What does “Discourage search engines from indexing this site” do?
It adds noindex, nofollow to every page of the site, as of WordPress 5.3 and later, and stops the core sitemap at /wp-sitemap.xml. It is meant for staging copies and should be cleared at launch. It is in Settings → Reading.
Why does the report show two robots.txt results?
Because the URL redirected to another hostname. Google needs permission on both: the old host to crawl the redirect, and the new host to crawl the page. If the old host’s robots.txt blocks Googlebot, Google never sees where the URL went.