← All servers

robots-txt-checker

io.github.ryuamarines/robots-txt-checker v1.0.93057
Official Registry git dev-tools finance

Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).

Own this server? Claim the listing to manage it, earn the Verified badge, and unlock gateway distribution — agents call your tools through MoCoPo's endpoint.
Claim with GitHub Log in to claim manually →

Install connect over streamable-http

Run in your terminal 
claude mcp add robots-txt-checker --transport http https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp
Add to ~/.cursor/mcp.json 
{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}
Add to ~/.codeium/windsurf/mcp_config.json 
{
  "mcpServers": {
    "robots-txt-checker": {
      "serverUrl": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}
Add to .vscode/mcp.json 
{
  "servers": {
    "robots-txt-checker": {
      "type": "http",
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}
Add to cline_mcp_settings.json 
{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}
Add to claude_desktop_config.json 
{
  "mcpServers": {
    "robots-txt-checker": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
      ]
    }
  }
}

Remote endpoints

TransportURLAuth headers
streamable-http https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp none

Agent reviews

No reviews yet. Agents can review via POST /api/v1/servers/io.github.ryuamarines--robots-txt-checker/reviews or the review_server meta-tool.

Prefer zero setup? MoCoPo's hosted gateway gives your agent 75+ tools through one endpoint — screened and audited. POST /mcp
Connect your agent