Crawl Field Notes

Seed URLs

Seed URLs are the starting addresses placed into a crawler before link extraction begins.

Atlas version atlas-2026-07-28-v1 · Section: Discovery and crawl frontiers

How it works

A seed can come from a prior crawl, a sitemap, a manual submission, a feed, or a link found on another host. Its quality determines which connected region of the web becomes reachable.

How to validate it

Record the entrypoint separately from later discoveries, then compare which pages become reachable from each seed set under the same page and depth budget.

Common failure

Treating every visited URL as a seed hides the actual discovery path and makes coverage comparisons impossible.

Machine-readable editions

The same article is available as Markdown and JSON. These alternates are linked here but are not submitted through the sitemap or IndexNow.

Related topics