About the Meta Robots & X-Robots-Tag Checker
Robots directives can live in several places: a generic meta robots tag, bot-specific tags such as googlebot or bingbot, and the X-Robots-Tag HTTP header that never appears in the page source. This meta robots checker fetches a public URL and gathers all of them in one view, so you can spot conflicts like a header noindex hiding behind an index meta tag.
Each directive is explained, including noindex, nofollow, noarchive, nosnippet, max-snippet, max-image-preview and unavailable_after. Snippet controls such as nosnippet and max-snippet affect how much of your text search engines may show, which can matter for AI-generated summaries too. When directives conflict, search engines generally apply the most restrictive one, so remove anything you did not intend before relying on the page for traffic.
How to use it
- Enter the page URL you want to inspect.
- Run the check to collect meta robots, bot-specific tags and X-Robots-Tag headers.
- Read the explanation for each directive and look for conflicts.
- Remove unintended noindex, nofollow or snippet limits in your CMS or server config.
Frequently asked questions
What is the difference between noindex and nofollow?
noindex asks search engines not to show the page in results. nofollow asks them not to follow the links on that page. They are independent: a page can be noindex but still pass link discovery, or indexable with nofollow links. Most pages that you want in search should have neither, which is the same as having no robots tag at all.
What does max-image-preview:large do?
It allows search engines to show a large preview of your images in results and features such as Discover. Without it, Google may use a smaller thumbnail. Many publishers add max-image-preview:large to the meta robots tag because larger images can make listings more visible. It does not change whether the page is indexed.
Which wins if meta robots and X-Robots-Tag disagree?
Search engines combine directives from all sources and generally follow the most restrictive. If the meta tag says index but the X-Robots-Tag header says noindex, the page will be treated as noindex. That is why it helps to see both together, especially when a server or CDN adds headers your CMS does not show.