Notable AI crawlers, as a table
The same facts as rows and columns. A table is structured, but unlike JSON-LD it carries no vocabulary, so a parser learns the shape, not the meaning.
| Crawler | Operator | Primary purpose | Respects robots.txt |
|---|---|---|---|
| GPTBot | OpenAI | Training & grounding | yes |
| ClaudeBot | Anthropic | Model training | yes |
| PerplexityBot | Perplexity | Live cited answers | yes |