Pydantic AI Gateway: Unified Interface for Multi-Provider AI Access

Pydantic AI Gateway is a unified interface that simplifies access to multiple AI providers with a single API key. Built by the Pydantic team, it addresses one of the most challenging aspects of building production AI applications: managing multiple provider APIs, monitoring costs, ensuring reliability with failover, and maintaining observability across different services.

1. What Is Pydantic AI Gateway?

Pydantic AI Gateway acts as a proxy layer between your application and various LLM providers (OpenAI, Anthropic, Google Vertex, Groq, AWS Bedrock). Instead of managing separate API keys, billing systems, and monitoring solutions for each provider, you get a single unified interface.

graph LR
    A[Your Application] --> B[Pydantic AI Gateway]
    B --> C[OpenAI]
    B --> D[Anthropic]
    B --> E[Google Vertex]
    B --> F[Groq]
    B --> G[AWS Bedrock]

Key Philosophy:

Unlike traditional AI gateways that translate everything to a common schema, Pydantic AI Gateway uses zero translation. Requests flow through directly in each provider's native format, giving you immediate access to new model features as soon as they're released.

2. Core Features

API Key Management

Access multiple LLM providers with a single Gateway key. No need to juggle different authentication schemes or rotate keys across multiple services.

from pydantic_ai import Agent

# One key, multiple providers
agent = Agent('gateway/openai:gpt-5')
result = agent.run_sync('Where does "hello world" come from?')
print(result.output)

Cost Limits and Monitoring

Set spending limits at three levels with daily, weekly, and monthly caps:

• Project level: Organization-wide budgets
• User level: Individual team member limits
• API key level: Per-key spending controls

graph TD
    A[Cost Monitoring] --> B[Project Limits]
    A --> C[User Limits]
    A --> D[API Key Limits]
    B --> E[Daily/Weekly/Monthly Caps]
    C --> E
    D --> E
    E --> F[Real-time Tracking]
    F --> G[Block on Limit]

The gateway tracks costs in real-time and blocks requests when limits are reached, preventing unexpected bills.

BYOK and Managed Providers

Two deployment models to fit your needs:

Bring Your Own Key (BYOK):

• Use your existing provider API keys
• Gateway routes requests without markup
• Maintain direct billing relationships

Managed Providers:

• Purchase inference credits through the platform
• Single-key access to all providers
• Simplified billing (Pydantic covers card fees during beta)

Multi-Provider Support

Currently supported providers:

More providers are actively being added based on community feedback.

3. Observability and Monitoring

Built-in OpenTelemetry

Every request through the gateway can be logged to Pydantic Logfire or any OpenTelemetry-compatible backend:

graph LR
    A[AI Request] --> B[Pydantic AI Gateway]
    B --> C[Provider API]
    B --> D[OpenTelemetry]
    D --> E[Logfire]
    D --> F[Custom Backend]
    C --> G[Response]
    G --> B
    B --> A

This provides:

• Complete request/response traces
• Latency measurements
• Token usage tracking
• Error rates and types
• Cost attribution

Real-time Cost Monitoring

The gateway calculates and tracks costs for every request:

• Per-request cost breakdown
• Aggregated spending by project, user, or key
• Historical cost trends
• Budget alerts and enforcement

4. Getting Started

Quick Setup

Step 1: Create an Account

Sign in at gateway.pydantic.dev using GitHub or Google. Choose an organization name and accept the default project or create a new one.

Step 2: Add Providers

Option A - Bring Your Own Key:

1. Go to Providers page
2. Select provider (OpenAI, Anthropic, etc.)
3. Paste your API key
4. Select project to associate with

Option B - Use Built-in Providers:

1. Go to Billing page
2. Add payment method
3. Purchase $15 in credits
4. Get instant access to all providers

Step 3: Create Gateway Keys

Navigate to the Keys page:

• Admins can create project keys (no spending limits)
• Users can create personal keys (inherit project/user limits)

Step 4: Set Environment Variable

export PYDANTIC_AI_GATEWAY_API_KEY="paig_<your_key>"

Basic Usage with Pydantic AI

from pydantic_ai import Agent

# OpenAI
agent = Agent('gateway/openai:gpt-5')
result = agent.run_sync('Explain quantum computing')

# Anthropic
agent = Agent('gateway/anthropic:claude-sonnet-4-5')
result = agent.run_sync('Write a haiku about code')

# Google Vertex
agent = Agent('gateway/google-vertex:gemini-2.5-flash')
result = agent.run_sync('Summarize this article...')

Using with Other SDKs

OpenAI SDK:

import openai

client = openai.Client(
    base_url='https://gateway.pydantic.dev/proxy/chat/',
    api_key='paig_...',
)

response = client.chat.completions.create(
    model='gpt-5',
    messages=[{'role': 'user', 'content': 'Hello world'}],
)

Anthropic SDK:

import anthropic

client = anthropic.Anthropic(
    base_url='https://gateway.pydantic.dev/proxy/anthropic/',
    auth_token='paig_...',
)

response = client.messages.create(
    max_tokens=1000,
    model='claude-sonnet-4-5',
    messages=[{'role': 'user', 'content': 'Hello world'}],
)

All requests now route through the gateway automatically.

5. Benefits and Use Cases

For Individual Developers

Simplified Development:

• Test multiple models without managing multiple accounts
• Switch providers with a single string change
• Free during beta (with cost tracking)

Example:

# Easy A/B testing across providers
models = [
    'gateway/openai:gpt-5',
    'gateway/anthropic:claude-sonnet-4-5',
    'gateway/google-vertex:gemini-2.5-flash'
]

for model in models:
    agent = Agent(model)
    result = agent.run_sync('Solve this problem...')
    print(f"{model}: {result.output}")

For Teams

Centralized Management:

• Single billing relationship
• Team-wide cost limits
• User access control via invitations
• Role-based permissions (Admin/User)

Observability:

• Track which team members use which models
• Identify cost optimization opportunities
• Debug issues across the entire team

6. Advanced Features

Failover Management

The gateway can automatically route requests to backup providers when the primary is unavailable:

# Configure failover in gateway settings
# If OpenAI fails, automatically try Anthropic
primary = 'gateway/openai:gpt-5'
fallback = 'gateway/anthropic:claude-sonnet-4-5'

Cost Calculation and Blocking

The gateway calculates costs using the genai-prices database:

• Tracks input/output token counts
• Applies current provider pricing
• Enforces spending limits in real-time

Configuration options:

• Block API key if unable to calculate spend (recommended for cost control)
• Allow requests through without cost tracking (for new/unsupported models)

8. Pricing and Availability

Current Status: Free while collecting feedback

Two Models:

1. BYOK: Use your own keys, no markup
2. Managed: Buy inference through the platform, Pydantic covers card fees during beta

Enterprise Pricing: Contact the team for:

• Self-hosting licenses
• On-premises deployment
• Consulting for custom deployments
• SLA guarantees

9. Best Practices

Key Management

# Use environment variables, never hardcode
import os
from pydantic_ai import Agent

api_key = os.getenv('PYDANTIC_AI_GATEWAY_API_KEY')
agent = Agent('gateway/openai:gpt-5')

Cost Optimization

1. Set appropriate limits at all levels (project, user, key)
2. Monitor usage patterns in Logfire/OpenTelemetry
3. Choose cost-effective models for appropriate tasks:
   • Use smaller models for simple tasks
   • Reserve large models for complex reasoning

Provider Selection

# Use appropriate models for the task
tasks = {
    'simple_qa': 'gateway/google-vertex:gemini-2.5-flash',
    'complex_reasoning': 'gateway/anthropic:claude-sonnet-4-5',
    'code_generation': 'gateway/openai:gpt-5'
}

Observability Setup

# Integrate with Logfire for full visibility
import logfire
from pydantic_ai import Agent

logfire.configure()

agent = Agent('gateway/openai:gpt-5')
with logfire.span('ai_request'):
    result = agent.run_sync('Your prompt')

10. When to Use Pydantic AI Gateway

Great Fit:

• Multi-provider applications
• Teams needing cost controls
• Production apps requiring observability
• Organizations wanting vendor flexibility
• Projects requiring audit trails

Not Necessary:

• Single-provider applications with simple needs
• Prototypes with no cost concerns
• Applications with existing robust gateway solutions

Conclusion

Pydantic AI Gateway solves real problems in production AI applications: managing multiple providers, controlling costs, maintaining observability, and ensuring reliability. Its zero-translation architecture ensures you're never waiting for gateway updates to access new model features, while its unified interface dramatically simplifies development and operations.

Whether you're a solo developer experimenting with different models, a team managing costs across multiple projects, or an enterprise requiring complete control and observability, Pydantic AI Gateway provides a flexible, open-source solution that grows with your needs.

Get started at gateway.pydantic.dev.

Resources:

• Official Documentation
• Sign Up
• Pricing Database