Generate robots.txt with user-agent rules and sitemap
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Disallow: /api/
Generate sitemap.xml from a list of URLs
Generate HTML meta tags for SEO and social media
Minify or beautify HTML code to reduce file size
Encode and decode URLs and query strings
The robots.txt Generator creates properly formatted robots.txt files for your website. Configure user-agent rules, allow/disallow paths, set crawl delays, add sitemaps, and specify the canonical host. Essential for SEO management, controlling search engine crawling, and preventing indexing of sensitive pages. The visual interface makes it easy to build complex rules without memorizing the syntax.