CostWise MCP SDK

Automatically reduce AI costs without changing your workflow.

The CostWise Managed Cost Policy (MCP) SDK analyzes your LLM requests, classifies task complexity, and recommends cheaper models and token limits — saving up to 90% on AI costs.

Install

pip install costwise-mcp

Quick Start

from costwise_mcp import CostPolicy

policy = CostPolicy(
    api_key="cw_your_api_key",
    backend_url="https://app.cost-wise.dev",  # your CostWise instance
)

decision = policy.optimize(
    prompt="Translate 'hello' to French",
    model="gpt-5.4",
)

print(decision.recommended_model)  # "gpt-5.4-mini" (70% cheaper)
print(decision.max_tokens)         # 256
print(decision.estimated_cost)     # $0.001154

# Call your LLM with the optimized params
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model=decision.recommended_model,
    max_tokens=decision.max_tokens,
    messages=[{"role": "user", "content": "Translate 'hello' to French"}],
)

# Report actual usage (async, non-blocking, optional)
policy.report(decision, actual_tokens=response.usage.total_tokens)

Team Policies (AI Governance)

Enforce model access and budgets per team:

from costwise_mcp import CostPolicy

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")

# Interns: restricted to budget models (tier 3), max $50/month
decision = policy.optimize(
    prompt="Write a summary",
    model="gpt-5.4",      # they asked for premium
    team="interns",        # policy enforces tier 3
)
# decision.recommended_model = "gpt-5.4-nano"
# decision.message = "Team 'interns' restricted to tier 3 models"

# Engineers: standard tier, $500/month budget
decision = policy.optimize(
    prompt="Debug this code",
    model="gpt-5.4",
    team="engineers",      # policy enforces tier 2
)
# decision.recommended_model = "gpt-5.4-mini"

# AI Research: full access, no budget limit
decision = policy.optimize(
    prompt="Analyze this architecture...",
    model="gpt-5.4",
    team="ai-team",        # tier 1 = all models allowed
)
# decision.recommended_model = "gpt-5.4" (kept for complex tasks)

Team tier levels

Configure team policies in CostWise: Settings > MCP Teams.

Error Handling

from costwise_mcp import (
    CostPolicy,
    BudgetExceededError,
    ModelBlockedError,
    TierRestrictionError,
    PolicyViolationError,
)

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")

decision = policy.optimize(
    prompt="Generate a report",
    model="gpt-5.4",
    team="interns",
)

if not decision.allowed:
    print(f"Blocked: {decision.message}")
    # "Team 'interns' monthly budget exceeded ($50.42 / $50.00)"
else:
    # Proceed with the LLM call
    print(f"Use {decision.recommended_model}, max {decision.max_tokens} tokens")

Error types

Framework Integration

The SDK works with any AI framework — it runs before the LLM call, not instead of it.

OpenAI SDK

from costwise_mcp import CostPolicy
from openai import OpenAI

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")
client = OpenAI()

decision = policy.optimize("Summarize this article", model="gpt-5.4", team="engineers")
response = client.chat.completions.create(
    model=decision.recommended_model,
    max_tokens=decision.max_tokens,
    messages=[{"role": "user", "content": prompt}],
)
policy.report(decision, actual_tokens=response.usage.total_tokens)

Anthropic SDK

from costwise_mcp import CostPolicy
import anthropic

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")
client = anthropic.Anthropic()

decision = policy.optimize(prompt, model="claude-opus-4-6", team="ai-team")
message = client.messages.create(
    model=decision.recommended_model,
    max_tokens=decision.max_tokens,
    messages=[{"role": "user", "content": prompt}],
)
policy.report(decision, output_tokens=message.usage.output_tokens)

LangChain

from costwise_mcp import CostPolicy
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")

decision = policy.optimize(prompt, model="gpt-5.4", team="engineers")
llm = ChatOpenAI(model=decision.recommended_model, max_tokens=decision.max_tokens)
result = llm.invoke([HumanMessage(content=prompt)])

LlamaIndex

from costwise_mcp import CostPolicy
from llama_index.llms.openai import OpenAI

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")

decision = policy.optimize(prompt, model="gpt-5.4", team="engineers")
llm = OpenAI(model=decision.recommended_model, max_tokens=decision.max_tokens)
response = llm.complete(prompt)

Google Gemini

from costwise_mcp import CostPolicy
import google.generativeai as genai

policy = CostPolicy(api_key="cw_...", backend_url="https://app.cost-wise.dev")

decision = policy.optimize(prompt, model="gemini-2.5-pro", team="engineers")
model = genai.GenerativeModel(decision.recommended_model)
response = model.generate_content(prompt)

Custom Configuration

from costwise_mcp import CostPolicy, PolicyConfig

policy = CostPolicy(
    api_key="cw_...",
    backend_url="https://app.cost-wise.dev",
    config=PolicyConfig(
        max_tokens_simple=128,      # override default 256
        max_tokens_medium=512,      # override default 1024
        max_tokens_complex=2048,    # override default 4096
        auto_downgrade=True,        # auto-select cheaper models for simple tasks
        blocked_models=["o1-pro"],  # local blocklist (in addition to team policy)
        telemetry_batch_size=100,   # send every 100 events (default: 50)
        telemetry_flush_interval=60, # send every 60 seconds (default: 30)
    ),
    project_id="my-chatbot",
)

Supported Models (75 models, 10 providers)

Privacy

• Prompts are never sent to the CostWise backend
• Only metadata: token counts, model name, cost, team ID
• Telemetry is optional: PolicyConfig(telemetry_enabled=False)

API Reference

CostPolicy(api_key, backend_url, config, project_id)

policy.optimize(prompt, model, task_type, max_budget, team) → Decision

policy.report(decision, actual_tokens, output_tokens, latency_ms)

Report actual usage after LLM call. Async, non-blocking.

Decision object

Get Your API Key

1. Sign in to your CostWise instance
2. Go to Settings > MCP SDK
3. Click Generate API Key
4. Copy the cw_... key
5. Set up team policies at Settings > MCP Teams