Internal link discovery
Ordinary HTML links are the most portable discovery mechanism because they are visible without executing JavaScript.
How it works
A crawler parses anchor destinations, normalizes them, applies policy, and adds unseen URLs to its frontier. Anchor placement and graph connectivity affect how soon a page is encountered.
How to validate it
Compare the raw response HTML with the rendered DOM and record the exact source page and discovery mechanism for every new URL.
Common failure
Buttons, click handlers, and script-generated routes can look like navigation to a person while providing no crawlable URL in the server response.
Machine-readable editions
The same article is available as Markdown and JSON. These alternates are linked here but are not submitted through the sitemap or IndexNow.