Crawl depth and reading order

Distance from the front page is not a neutral fact. It is a budget, and the pages that sit deepest spend it last.

Crawlers work a queue with finite patience: a limit on how many pages, how deep, how often. Pages one click from the home page are cheap and get fetched early. Pages five clicks down compete for whatever budget is left. On a large site, the tail is where good content quietly goes unread.

This note is the first of a short chain. The next level is one link away: continue to level two. Whether a crawler follows the chain to the end says something about how much depth it is willing to spend here.