A

AI Crawler Rules

JetBrains

readystack · v—

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI-search citation, or user fetch - and which lines silently do nothing. Right-click in the editor (or Tools ) → AI Crawler Rules: Check This File . Each finding appears on its line with the fix. 19 rules, offline. Runs the same engine as the VS Code extension and the CLI: npx @readystack/robots-txt-ai-crawler-audit (Node.js 18+ required). Free for the file y

Code Tools
Open in store ↗ id: 34708
Downloads
34
34
Rating
— ★
— ratings
Category
JetBrains Marketplace
Collected
2026-11-04
weekly snapshot

Similar in “Code Tools”