Developed by Harsha - LinkedIn|Website

Robots.txt Generator

Create and customize a `robots.txt` file to manage search engine crawlers.

Generator

Set default policies and add specific rules for different user-agents.

Default Policy (for User-agent: *)

None (use specific rules below)

Allow All

Disallow All

Sitemap URL (optional)

Rule 1

User-agent

Allow paths (one per line)

Disallow paths (one per line)

Crawl-delay (in seconds)

Generated robots.txt

# Generated by DevToolBox

User-agent: *
Disallow: /admin
Disallow: /private