Compare Providers by Cost

Send requests to different providers through the same base URL

Toolken’s gateway detects the provider from the model name. To compare, send the same prompt using a model from each provider you want to evaluate. You do not need separate SDKs or base URLs.For OpenAI-compatible providers (OpenAI, Groq, Mistral, Deepseek, and others), use /v1/chat/completions. For Anthropic models, use /v1/messages.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.toolken.ai/v1",
    api_key="sk-...",  # your provider key, forwarded untouched
    default_headers={
        "X-Toolken-Key": "tk_live_...",
        "X-Toolken-Metadata-Feature": "provider-comparison",
    },
)

prompt = [{"role": "user", "content": "Explain backpressure in stream processing in two sentences."}]

# OpenAI
client.chat.completions.create(model="gpt-4o-mini", messages=prompt)

# Groq (OpenAI-compatible, same endpoint)
client_groq = OpenAI(
    base_url="https://gateway.toolken.ai/v1",
    api_key="gsk_...",  # your Groq key
    default_headers={
        "X-Toolken-Key": "tk_live_...",
        "X-Toolken-Metadata-Feature": "provider-comparison",
    },
)
client_groq.chat.completions.create(model="llama-3.1-8b-instant", messages=prompt)

import OpenAI from "openai";

const prompt = [{ role: "user", content: "Explain backpressure in stream processing in two sentences." }];

// OpenAI
const openaiClient = new OpenAI({
  baseURL: "https://gateway.toolken.ai/v1",
  apiKey: process.env.OPENAI_API_KEY, // forwarded untouched
  defaultHeaders: {
    "X-Toolken-Key": "tk_live_...",
    "X-Toolken-Metadata-Feature": "provider-comparison",
  },
});
await openaiClient.chat.completions.create({ model: "gpt-4o-mini", messages: prompt });

// Groq (OpenAI-compatible, same endpoint)
const groqClient = new OpenAI({
  baseURL: "https://gateway.toolken.ai/v1",
  apiKey: process.env.GROQ_API_KEY,
  defaultHeaders: {
    "X-Toolken-Key": "tk_live_...",
    "X-Toolken-Metadata-Feature": "provider-comparison",
  },
});
await groqClient.chat.completions.create({ model: "llama-3.1-8b-instant", messages: groqMessages });

PROMPT='{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Explain backpressure in stream processing in two sentences."}]}'

# OpenAI (gpt-4o-mini)
curl https://gateway.toolken.ai/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "X-Toolken-Key: tk_live_..." \
  -H "X-Toolken-Metadata-Feature: provider-comparison" \
  -H "Content-Type: application/json" \
  -d "$PROMPT"

# Groq (OpenAI-compatible, same endpoint, different model)
curl https://gateway.toolken.ai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "X-Toolken-Key: tk_live_..." \
  -H "X-Toolken-Metadata-Feature: provider-comparison" \
  -H "Content-Type: application/json" \
  -d '{"model":"llama-3.1-8b-instant","messages":[{"role":"user","content":"Explain backpressure in stream processing in two sentences."}]}'

Anthropic models use a different endpoint (/v1/messages) and a different auth header (x-api-key). You cannot compare an Anthropic model and an OpenAI model in the same SDK call, but Toolken records both and you can compare costs in the dashboard.

Reach more models with OpenRouter

OpenRouter aggregates a wide catalog of models from many providers. Set X-Toolken-Provider: openrouter and pass your OpenRouter key. You can then use any model in their catalog without signing up for each provider separately.

default_headers={
    "X-Toolken-Key": "tk_live_...",
    "X-Toolken-Provider": "openrouter",
    "X-Toolken-Metadata-Feature": "provider-comparison",
}

The dashboard attributes each request to its actual model, so cost comparisons still work.

Force a specific provider with X-Toolken-Provider

By default the gateway detects the provider from the model name. To override that detection, set X-Toolken-Provider explicitly. This is useful when two providers accept the same model name, or when you want to test the same model on different infrastructure.

curl https://gateway.toolken.ai/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" \
  -H "X-Toolken-Key: tk_live_..." \
  -H "X-Toolken-Provider: groq" \
  -H "Content-Type: application/json" \
  -d '{"model":"llama-3.1-8b-instant","messages":[{"role":"user","content":"Hello"}]}'

Getting Started

Observability

Providers & Routing

Rate Limits

Integrations

Use cases

Compare Providers by Cost

Next

Cost Per Agent

Cost by Environment

​Next

Cost Per Agent

Cost by Environment

Next