# URL normalization

Canonical HTML: /atlas/url-normalization
Atlas version: atlas-2026-07-28-v1
Category: Discovery and crawl frontiers

## Definition

URL normalization maps syntactic variants to a consistent identity before deduplication and scheduling.

## How it works

Common steps include resolving relative links, removing fragments, normalizing host casing and default ports, and applying a deliberate query-parameter policy.

## How to validate it

Store both the discovered string and normalized fetch target, then test fragments, trailing slashes, parameter order, and percent-encoding independently.

## Common failure

Over-aggressive normalization can merge distinct resources, while no normalization can turn one page into an effectively infinite crawl space.

## Related topics

- [Internal link discovery](/atlas/internal-links)
- [Canonical links](/atlas/canonical-links)