coleam00/mcp-crawl4ai-ragMCPMCP Server

mcp-crawl4ai-rag

Web Crawling and RAG Capabilities for AI Agents and AI Coding Assistants

2.2kstars

572forks

Python

Quality signals

StaleNo testsNo CIThin docsNo licensePopular

Open Repository Download ZIP

AI summary

Auto-generated overview of what this skill does and when to reach for it.

MCP server that lets AI agents crawl websites, store content in a vector database, and perform RAG queries with advanced retrieval strategies like hybrid search and reranking.

Use this when

You need to scrape documentation or web pages for AI context
Your AI coding assistant lacks up-to-date knowledge of external APIs or libraries
You want to build a knowledge base from GitHub repositories or docs
You need to detect hallucinations in generated code by validating against real codebases
You want to combine vector search with keyword search for better retrieval

Don't use when

You only need to crawl a single static page without storage
Your project has strict privacy requirements and can't use cloud APIs
You're working with non-text content like videos or images

Capabilities

Crawl single pages or entire websites recursively
Store scraped content in Supabase with pgvector for semantic search
Perform RAG queries with optional source filtering for precision
Enable hybrid search combining vector and keyword matching
Extract and summarize code examples from documentation
Build and query Neo4j knowledge graphs for hallucination detection

InstallMCPRun via uvx (Python)

Universal installANY TOOL

$ npx skillcity@latest coleam00/mcp-crawl4ai-rag

Auto-detects the skill or MCP server in this repo and installs it for Claude Code, Codex, Cursor or Cline.

Requires Node.js 18+ (ships with npx).

Or install manually

uvx mcp-crawl4ai-rag

MCP Configuration

Add this to your claude_desktop_config.json

{
  "mcpServers": {
    "mcp-crawl4ai-rag": {
      "command": "uvx",
      "args": [
        "mcp-crawl4ai-rag"
      ]
    }
  }
}

Loading README…

Discover the toolingbehind modern intelligence

mcp-crawl4ai-rag