Meta robots
A meta robots element gives indexing and following instructions inside an HTML page that has already been fetched.
How it works
Common directives include index, noindex, follow, nofollow, noarchive, and snippet controls. Support and interpretation can differ by crawler product.
How to validate it
Inspect the final rendered head and response headers, then separate successful fetching from subsequent indexing behavior.
Common failure
Blocking a page in robots.txt can prevent a crawler from seeing the noindex instruction placed inside that page.
Machine-readable editions
The same article is available as Markdown and JSON. These alternates are linked here but are not submitted through the sitemap or IndexNow.