robots.txt AI Builder
CRAWLER ACCESS CONTROL PANEL · 17 named AI bots
Dark mode
Presets:
Allow all AI
Block training, allow search
Block all AI
Custom
AI Crawler Catalog
0 / 17 ALLOWED
Sitemap URL
llms.txt URL (optional)
Disallow paths (one per line)
Applies to all bots in the User-agent: * block.
Output
0 LINES
Crawler status
0
Allowed
0
Partial
0
Blocked
Copy robots.txt
Download