Discover the toolingbehind modern intelligence
The operating system for AI infrastructure discovery. Search a live map of MCP servers, AI agents, LLM tools, automation systems, and developer infrastructure.
litellm
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
49.3kstars
8.6kforks
Python
ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenai
Quality signals
StaleNo testsNo CIThin docsNo licensePopular
AI summary
Auto-generated overview of what this skill does and when to reach for it.
Unified Python SDK and proxy server to call 100+ AI models via OpenAI format with cost tracking, guardrails, and load balancing.
Use this when
- You need to switch LLM providers without rewriting code
- You want to track costs across multiple AI services
- You need to enforce rate limits or content filters
- You’re running multiple models in parallel for load balancing
- You want to log all AI requests for auditing
Don't use when
- You only use a single LLM provider with no plans to change
- You need real-time low-latency inference without proxy overhead
- Your app is entirely offline with no internet access
Capabilities
- Call 100+ LLM APIs using OpenAI-compatible format
- Track token usage and costs per model/provider
- Route requests to different providers based on load or cost
- Enforce guardrails like rate limits or content filters
- Log all API calls for debugging or auditing
- Support Bedrock, Azure, OpenAI, Cohere, Anthropic, and more
InstallMCP + SKILLInstall as a Claude Code skill
Universal installANY TOOL
$ npx skillcity@latest BerriAI/litellmAuto-detects the skill or MCP server in this repo and installs it for Claude Code, Codex, Cursor, Cline, OpenCode, Gemini CLI, Anti-Gravity, OpenClaw and Hermes.
Requires Node.js 18+ (ships with npx).
Or install manually
# Install into ~/.claude/skills/ mkdir -p ~/.claude/skills/litellm curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/SKILL.md -o ~/.claude/skills/litellm/SKILL.md
Loading README…
Ask a simulated agent to run this skill. It loads the real SKILL.md and reasons over your task with NVIDIA.
litellm — sandboxAI sandbox
Pick a quick prompt below or type your own — watch the agent execute this skill step by step.
guest@skillcity:~$Try a prompt