Conditional requests
Conditional requests let a crawler ask whether a previously fetched representation has changed.
How it works
The client sends validators such as If-None-Match or If-Modified-Since; an unchanged resource can return 304 without retransmitting the body.
How to validate it
Log validator presence and response status, then verify that ETags and Last-Modified values change whenever meaningful content changes.
Common failure
A static validator attached to changing content can prevent crawlers from seeing updates even while the server returns successful responses.
Machine-readable editions
The same article is available as Markdown and JSON. These alternates are linked here but are not submitted through the sitemap or IndexNow.