Discover the toolingbehind modern intelligence
The operating system for AI infrastructure discovery. Search a live map of MCP servers, AI agents, LLM tools, automation systems, and developer infrastructure.
craw-mcp
Self-hostable MCP server that lets Claude crawl pages blocked by anti-bot walls, JS challenges, or IP bans, without paid scraping APIs. Bring your own proxies. RabbitMQ queue, Rust crawler, MinIO claim-check.
1stars
0forks
Rust
anti-botclaudecrawlermcpminiomodel-context-protocolrabbitmqrustself-hostedweb-scraping
Quality signals
ActiveNo testsNo CIThin docsNo licenseNew
Install
Universal installANY TOOL
$ npx skillcity@latest bilginerdem/craw-mcpAuto-detects the skill or MCP server in this repo and installs it for Claude Code, Codex, Cursor, Cline, OpenCode, Gemini CLI, Anti-Gravity, OpenClaw and Hermes.
Requires Node.js 18+ (ships with npx).
Prefer to set it up by hand? Clone the repository:
git clone https://github.com/bilginerdem/craw-mcp.git
Loading README…
Ask a simulated agent to run this skill. It loads the real SKILL.md and reasons over your task with NVIDIA.
craw-mcp — sandboxAI sandbox
Pick a quick prompt below or type your own — watch the agent execute this skill step by step.
guest@skillcity:~$Try a prompt