Development
Robots.txt Generator
Generate robots.txt files for search engine crawlers with configurable rules
text· Text resultOpen tool
Quick start
Call this tool from your code in three languages.
cURL
curl -X POST 'https://api.elysiatools.com/en/api/tools/robots-txt-generator' \
-H 'Content-Type: application/json' \
-d '{"userAgent":"*","allowPaths":"/, /blog/, /products/ (comma-separated)","disallowPaths":"/admin/, /private/, /temp/, /cgi-bin/ (comma-separated)","crawlDelay":0,"sitemapUrl":"https://example.com/sitemap.xml","includeComments":true}'API reference
Send a POST request with your inputs as JSON. File parameters require a separate upload first.
Endpoint
HTTP
POST https://api.elysiatools.com/en/api/tools/robots-txt-generatorParameters
| Name | Type | Required | Description |
|---|---|---|---|
| userAgent | text | No | — |
| allowPaths | textarea | No | — |
| disallowPaths | textarea | No | — |
| crawlDelay | number | No | — |
| sitemapUrl | text | No | — |
| includeComments |
MCP integration
Add this tool to your Model Context Protocol server so AI agents can list and call it.
Server configuration
Add this block to your MCP client configuration:
mcp.json
{
"mcpServers": {
"elysiatools-robots-txt-generator": {
"name": "robots-txt-generator",
"description": "Generate robots.txt files for search engine crawlers with configurable rules",
"baseUrl": "https://api.elysiatools.com/mcp/sse?toolId=robots-txt-generator",
"command": "",
"args": [],
"env": {},
"isActive": true,
"type": "sse"
}
}
}List available tools
After connecting to the SSE endpoint, list the exposed tools:
tools/list
{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/list"
}Call this tool
Invoke the tool by its id, passing arguments built from its parameters:
tools/call
{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/call",
"params": {
"name": "robots-txt-generator",
"arguments": {
"userAgent": "*",
"allowPaths": "/, /blog/, /products/ (comma-separated)",
"disallowPaths": "/admin/, /private/, /temp/, /cgi-bin/ (comma-separated)",
"crawlDelay": 0,
"sitemapUrl": "https://example.com/sitemap.xml",
"includeComments": true
}
}
}Chain multiple tools in one session with a comma-separated toolId list, e.g. /mcp/sse?toolId=png-to-webp,jpg-to-webp,gif-to-webp (max 20).
Questions or issues? Contact [email protected]