Crawl Field Notes

Home Discovery Behavior Arena Knowledge Atlas Formats Structured data robots.txt Sitemaps

nofollow discovery target

The only experimental inlink to this target carries rel="nofollow".

Measurement marker: ARN-DISC-NOFOLLOW

Crawl Field Notes — How crawlers and AI agents discover and read the web. An open research field guide. Requests are logged with data-minimising telemetry; crawler identities remain unverified claims.

Privacy · Sitemap · robots.txt