# Robots.txt Lint Validator Lint robots.txt syntax, flag risky rules, and test important URLs before you ship crawler directives > Canonical page: https://elysiatools.com/en/tools/robots-txt-lint-validator - **Category:** Validation - **Keywords:** robots.txt, seo, lint, crawler, allow, disallow, sitemap ## Overview Validate your robots.txt files for syntax errors, check for risky crawler directives, and test critical URLs against your rules before deploying them to production. ## Inputs - **Robots.txt Content** (textarea): User-agent: * Disallow: /admin Allow: /admin/help Sitemap: https://example.com/sitemap.xml - **Robots.txt File** (file) - **Site Origin** (text): https://example.com - **Test URLs** (textarea): / /admin /blog/post-1 https://example.com/checkout/success ## When to use - Before deploying a new robots.txt file to production to prevent accidental search engine de-indexing. - When troubleshooting why search engine crawlers are not indexing specific pages or directories. - During website migrations or redesigns to verify that staging rules are correctly updated for the live site origin. ## How it works - Paste your robots.txt content directly into the text area or upload your robots.txt file. - Enter your site origin URL and list the specific paths or URLs you want to test. - Run the validator to parse the syntax, identify structural errors, and check which test URLs are allowed or disallowed. ## Use cases - Auditing staging robots.txt files to ensure sensitive admin paths are blocked before going live. - Testing if critical landing pages or blog posts are accidentally blocked by broad wildcard rules. - Verifying syntax compliance with major search engine crawler standards. ## Frequently asked questions ### What does this tool validate in a robots.txt file? It checks for syntax errors, missing colons, invalid directives, and tests specific URLs against your rules. ### Can I test URLs with different origins? You should specify the correct Site Origin in the input field to ensure relative paths are resolved and tested accurately. ### Why is my Disallow rule flagged as an error? Common issues include missing colons after directives, incorrect wildcards, or invalid user-agent declarations. ### Does this tool support sitemap declarations? Yes, it parses and validates the format of Sitemap directives within your robots.txt file. ### Can I upload a robots.txt file directly? Yes, you can upload your robots.txt file using the file input option instead of copying and pasting. ## Related tools - [Audio Caption Sync Checker](https://elysiatools.com/en/tools/audio-caption-sync-checker): Compare SRT or WebVTT cue timing with likely speech activity and flag timing drift. - [Accessibility Checker](https://elysiatools.com/en/tools/accessibility-checker): Detect common WCAG 2.1 accessibility issues in HTML, fetched pages, or design images and return fix-ready guidance - [OCR PDF to Structured JSON Bridge](https://elysiatools.com/en/tools/ocr-pdf-to-structured-json-bridge): Extract the PDF text layer with geometry (lines by y-position, tables by column gaps, headings by font size, colon key-value pairs), then fill a user-supplied JSON Schema field by field — labels matched by normalized keys, values coerced to declared types and validated with ajv. - [PDF Logical Structure Tree WCAG Role Mapping Validator](https://elysiatools.com/en/tools/pdf-logical-structure-tree-validator): Audit a tagged PDF's logical structure tree against PDF/UA-1 (ISO 14289-1) semantics: RoleMap termination, H1-H6 heading hierarchy, Alt text on Figure/Formula, RFC 3066 language tags, illegal nesting, and structure-vs-content MCID reading order — each finding labeled with its Matterhorn Protocol checkpoint. - [Responsive Picture / srcset Art Direction Builder](https://elysiatools.com/en/tools/responsive-picture-srcset-art-direction-builder): Build complete markup with art-directed crops: per-breakpoint elements, 1x/2x density candidates or {w}-template width descriptors, optional AVIF/WebP format layers, sizes handling, CLS-safe width/height, plus HTML and JSX flavors with spec-based lint notes. - [Train/Test Split with Stratification](https://elysiatools.com/en/tools/train-test-split-with-stratification): Class-stratified train/validation/test split or stratified k-fold for CSV/JSON datasets — seeded, reproducible, with distribution reports, leakage checks, SMOTE preview and CSV export. - [Accessibility Audit Report Generator](https://elysiatools.com/en/tools/accessibility-audit-report-generator): Audit HTML or a URL for common WCAG issues and export a structured PDF report grouped by severity - [Batch Podcast Publisher Pack](https://elysiatools.com/en/tools/audio-batch-podcast-publisher-pack): Normalize a batch of podcast episodes, apply metadata and chapters, and package validated delivery files in one ZIP. ## Samples - [Txt Samples](https://elysiatools.com/en/samples/txt-samples): Sample text files in different languages and encodings - [Copyright-Free MP3 Audio Samples](https://elysiatools.com/en/samples/mp3-samples): Collection of royalty-free audio samples for testing and development purposes including nature sounds, meditation music, and ambient audio - [SVG Samples](https://elysiatools.com/en/samples/svg-samples): Scalable Vector Graphics (SVG) samples demonstrating various SVG features and techniques - [QR Code Samples](https://elysiatools.com/en/samples/qrcode-samples): Sample QR code images for testing QR code reading and generation tools ## Related content - [Web Source Reuse and Publication Audit](https://elysiatools.com/en/hubs/web-source-link-and-metadata-audits): Extract source evidence, inspect links and assets, analyze metadata and query strings, and check RSS/Atom and robots.txt before reuse, migration, or release.