Robots.txt Generator
Create clean crawl rules.
Build a robots.txt file with user-agent, allow, disallow and sitemap lines.
Rules
Output
Essential browser storage only. Your text stays in your browser. Cookie Policy
Build a robots.txt file with user-agent, allow, disallow and sitemap lines.