Crawl Field Notes

Cache validators

ETag and Last-Modified values identify the version or modification time of a representation.

Atlas version atlas-2026-07-28-v1 · Section: HTTP behavior and efficiency

How it works

Strong ETags describe byte identity, weak ETags describe semantic equivalence, and modification dates provide a coarser time-based validator.

How to validate it

Derive validators from the actual representation or a versioned content build and test them across compressed and uncompressed variants.

Common failure

Using deployment time as Last-Modified for every page makes untouched content look freshly changed and wastes recrawl attention.

Machine-readable editions

The same article is available as Markdown and JSON. These alternates are linked here but are not submitted through the sitemap or IndexNow.

Related topics