Meta Robots Generator guide
Build a correct robots meta tag in seconds. Choose index or noindex, follow or nofollow, snippet and preview limits, and an expiry date, then copy the tag for your page head or the X-Robots-Tag header for PDFs and other files.
How robots rules work
A robots meta tag is a page-level instruction to search engines. It lives in the HTML head as <meta name="robots" content="...">, and the content attribute holds a comma-separated list of rules. The name can be robots, which every crawler reads, or a specific crawler such as googlebot or bingbot. If both appear, Google combines them and applies the most restrictive rule.
Without any tag, the defaults are index and follow: show the page in results and crawl its links. So the tag only earns its place when you want to restrict something (noindex, nofollow, nosnippet) or loosen preview limits (max-snippet:-1, max-image-preview:large).
The rules that matter
noindex keeps the page out of search results. nofollow tells crawlers not to follow the page's links. none is shorthand for both. nosnippet removes the text snippet and video preview. max-snippet:N caps the snippet at N characters, with 0 meaning none and -1 meaning no limit. max-image-preview accepts none, standard, or large. max-video-preview:N caps video previews at N seconds. noimageindex keeps images on the page out of image search. notranslate hides the translate link. unavailable_after takes a date, after which the page drops out of results. indexifembedded lets content be indexed when embedded in another page through an iframe, even though the page itself carries noindex.
Google's documentation lists noarchive, nocache, and nositelinkssearchbox as no longer used by Google Search. Bing still honors noarchive, which is why the generator keeps it as an option.
Worked examples
A thank-you page after a form submission. You do not want it in search results, but its links back to your site are fine. Set Indexing to noindex and leave Links on follow. Output: <meta name="robots" content="noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1" />. The preview rules do nothing on a noindexed page, but they are harmless.
A limited-time promotion ending December 31, 2026. Keep it indexed, and set unavailable_after to 2026-12-31. Google will stop showing it after that date without you having to remember to change anything. ISO 8601 dates like this are one of the formats Google accepts.
A PDF price list you want out of search. PDFs have no head section, so use the header output instead: X-Robots-Tag: noindex. Add it in your server config, for example with a header rule for *.pdf in Nginx, Apache, or your CDN, such as Cloudflare Transform Rules or a Netlify _headers file.
The robots.txt trap
The most common and most damaging mistake is combining a robots.txt Disallow with a noindex tag on the same URL. robots.txt stops Google from crawling the page, which means Google never sees the noindex. If other sites link to that URL, it can still appear in results, typically with no description. To remove a page, allow crawling, add noindex, and wait for a recrawl. Only after it has dropped out should you consider blocking it in robots.txt.
The second most common mistake is shipping a staging site's noindex to production. Many CMS platforms and frameworks have a discourage search engines setting that adds a site-wide noindex. After every launch or migration, view the source of a few key pages and search for noindex.
Preview settings: why the defaults are generous
The generator starts with max-snippet:-1, max-image-preview:large, and max-video-preview:-1. Google's guidance for Discover recommends allowing large image previews, and setting the limits explicitly removes any ambiguity about what you allow. There is rarely a reason for a normal page to restrict them.
Restrict previews only when you have a concrete reason: paywalled content where a long snippet gives away the answer, or licensed images you are not allowed to show at large size.
How we calculate: sources
Frequently asked questions
What does a meta robots tag do?
It tells search engine crawlers how to treat one page: whether to show it in results (index or noindex), whether to follow its links (follow or nofollow), and how to display snippets and previews. It goes in the page's <head>.
What is the default if I have no robots tag?
index, follow. Search engines index the page and follow its links unless told otherwise, so you only need a tag when you want to restrict something or set preview limits.
What is the difference between noindex and robots.txt?
robots.txt blocks crawling; noindex blocks indexing. If you block a page in robots.txt, Google cannot read its noindex tag, and the URL can still appear in results if other sites link to it. To remove a page from Google, allow crawling and use noindex.
How long does noindex take to work?
The page drops out after Google recrawls it, which can take days to weeks. To speed it up, request indexing of the URL in Search Console's URL Inspection tool, or use the Removals tool for an urgent temporary block (about six months).
Does Google still support noarchive?
No. Google lists noarchive as no longer used, since it retired cached page links in 2024. Bing and some other engines still honor it, so it is harmless to include.
What does max-image-preview:large do?
It lets Google show large image previews in results and Discover. Google's Discover documentation recommends it, since large images can increase clicks. The generator includes it by default.
When should I use X-Robots-Tag instead?
For files that have no HTML head, such as PDFs, images, and Word documents. Send it as an HTTP response header from your server or CDN, for example X-Robots-Tag: noindex.
Is my data uploaded?
Everything runs in your browser. Nothing you enter is uploaded to a server or stored by us.
Can I noindex just part of a page?
Not with the robots meta tag, which applies to the whole page. You can keep a section out of snippets with the data-nosnippet attribute on a span, div, or section element.
Does nofollow on the meta tag stop PageRank flowing?
It asks crawlers not to follow any link on the page. Since 2019 Google treats nofollow as a hint rather than a strict directive, so for individual links use rel="nofollow", rel="sponsored", or rel="ugc" instead.