Anthropic - Portkey Docs

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Anthropic’s Claude APIs. With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

All Models

Full support for all Claude models including Sonnet and Haiku 4-5

All Endpoints

/messages, count-tokens and more fully supported

Multi-Provider Support

Use Claude from Anthropic, Bedrock, and Vertex with native SDK support

Quick Start

Get Anthropic working in 3 steps:

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @anthropic provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "What is Portkey's AI Gateway?"}],
    max_tokens=250  # Required for Anthropic
)

print(response.choices[0].message.content)

Tip: You can also set provider="@anthropic" in Portkey() and use just model="claude-sonnet-4-5-20250929" in the request.

max_tokens is required - Always specify this parameter
System prompts - Handled differently (see System Prompts section below)
Model naming - Use full model names like claude-sonnet-4-5-20250929

Add Provider in Model Catalog

Go to Model Catalog → Add Provider
Select Anthropic
Choose existing credentials or create new by entering your Anthropic API key
Name your provider (e.g., anthropic-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Basic Usage

Chat Completions

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@anthropic/claude-sonnet-4-5-20250929",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What is Portkey's AI Gateway?" }
    ],
    max_tokens: 250  // Required
})

console.log(response.choices[0].message.content)

System Prompts

Anthropic handles system prompts differently than OpenAI. With Portkey, you can use the OpenAI-compatible format:

response = portkey.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    messages=[
        {"role": "system", "content": "You are a snarky assistant."},
        {"role": "user", "content": "How do I boil water?"}
    ],
    max_tokens=250
)

Portkey automatically formats this for Anthropic’s API.

Streaming

Streaming works the same as OpenAI:

response = portkey.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "Tell me a story"}],
    max_tokens=500,
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Catch Overloaded Error on Stream

Anthropic’s API can return an overloaded_error inside a streaming response with HTTP status 200. The error appears as an SSE event:

event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

By default, the gateway treats this as a successful (status 200) response and streams the error directly to the client, which means retry, fallback, and circuit breaker strategies do not activate (they rely on HTTP status codes). When Catch Overloaded Error on Stream is enabled on an Anthropic integration, the gateway intercepts these errors before they reach the client and converts them into HTTP 529 responses, allowing your retry and fallback strategies to trigger automatically.

This feature is only available for the Anthropic provider. Other providers (e.g., Bedrock) handle overload errors at the HTTP level, where existing retry/fallback already applies. It also only applies to streaming requests — non-streaming Anthropic requests already return HTTP 529 directly.

How it works

When enabled on an integration, the gateway:

Reads the first chunk of the Anthropic streaming response before committing it to the client
Skips any keepalive ping events
If the first meaningful event is an overloaded_error, returns an HTTP 529 response instead of the stream
If the first event is normal content, continues streaming as usual with no data loss

If no retry strategy is present and an overloaded_error is found, the request fails as a normal request with error 529. The 529 response integrates with the gateway’s existing error handling and supports all existing config combinations:

Retry: Triggers automatically when retry is configured
Fallback: Moves to the next target in a fallback strategy
Circuit breaker: Counts as a failure for circuit breaker thresholds

Performance: There is zero overhead when the setting is disabled. When enabled, only the first event is inspected before the stream is committed.

How to enable

Enable the flag on your Anthropic integration

Go to Model Catalog → Integrations → Anthropic and enable the Catch Overloaded Error on Stream flag, then create or update the integration.

Add 529 to your retry status codes

In your config, add 529 to the retry on_status_codes (or fallback on_status_codes). This supports all existing config combinations.

Attach the config to your API key

Attach the updated config to your API key so the new behavior applies to all routed requests.

Once enabled, all Anthropic streaming requests routed through the gateway are checked for overloaded errors.

Example: Fallback on overload

With a fallback config using two Anthropic integrations (both with Catch Overloaded Error on Stream enabled), if the primary returns an overloaded error during streaming, the gateway automatically retries with the backup:

{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "provider": "anthropic", "virtual_key": "anthropic-primary" },
    { "provider": "anthropic", "virtual_key": "anthropic-backup" }
  ]
}

Error response

When an overloaded error is detected, the client receives:

HTTP/1.1 529
{
  "error": {
    "message": "Overloaded",
    "type": "overloaded_error",
    "param": null,
    "code": null
  }
}

Advanced Features

Vision (Multimodal)

Portkey supports Anthropic’s vision models including claude-sonnet-4-5-20250929, claude-3-5-sonnet, claude-3-haiku, claude-3-opus, and claude-3.7-sonnet. Use the same format as OpenAI:

Anthropic only accepts base64-encoded images and does not support image URLs. Use the same base64 format to send images to both Anthropic and OpenAI models.

import base64
import httpx
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

# Fetch and encode the image
image_url = "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg"
image_data = base64.b64encode(httpx.get(image_url).content).decode("utf-8")

response = portkey.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}},
            {"type": "text", "text": "What's in this image?"}
        ]
    }],
    max_tokens=300
)

print(response.choices[0].message.content)

To prompt with PDFs, update the url field to: data:application/pdf;base64,BASE64_PDF_DATA

PDF Support

Anthropic Claude processes PDFs to extract text, analyze charts, and understand visual content. PDF support is available on:

Claude 3.7 Sonnet (claude-3-7-sonnet-20250219)
Claude 3.5 Sonnet (claude-3-5-sonnet-20241022, claude-3-5-sonnet-20240620)
Claude Sonnet 4-5 (claude-sonnet-4-5-20250929)
Claude 3.5 Haiku (claude-3-5-haiku-20241022)

Limitations:

Maximum request size: 32MB
Maximum pages per request: 100
Format: Standard PDF (no passwords/encryption)

from portkey_ai import Portkey
import base64
import httpx

portkey = Portkey(api_key="PORTKEY_API_KEY")

# Fetch and encode the PDF
pdf_url = "https://assets.anthropic.com/m/1cd9d098ac3e6467/original/Claude-3-Model-Card-October-Addendum.pdf"
pdf_data = base64.standard_b64encode(httpx.get(pdf_url).content).decode("utf-8")

response = portkey.chat.completions.create(
    model="@anthropic/claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[
        {"role": "system", "content": "You are a helpful document analysis assistant."},
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What are the key findings in this document?"},
                {"type": "file", "file": {"mime_type": "application/pdf", "file_data": pdf_data}}
            ]
        }
    ]
)

print(response.choices[0].message.content)

Extended Thinking (Reasoning Models)

Models like claude-3-7-sonnet-latest support extended thinking. Get the model’s reasoning as it processes the request.

The assistant’s thinking response is returned in the response_chunk.choices[0].delta.content_blocks array, not the response.choices[0].message.content string.

Set strict_open_ai_compliance=False to use this feature:

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    strict_open_ai_compliance=False
)

response = portkey.chat.completions.create(
    model="@anthropic/claude-3-7-sonnet-latest",
    max_tokens=3000,
    thinking={"type": "enabled", "budget_tokens": 2030},
    stream=False,
    messages=[{
        "role": "user",
        "content": [{
            "type": "text",
            "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
        }]
    }]
)

print(response)

Using /messages Route

Portkey supports Anthropic’s /messages endpoint, allowing you to use either Anthropic’s native SDK or Portkey’s SDK with full gateway features.

Using Anthropic’s Native SDK

import anthropic

client = anthropic.Anthropic(
    api_key="dummy", # we will use portkey's provider slug
    default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
    base_url="https://api.portkey.ai"
)

message = client.messages.create(
    model="@your-provider-slug/claude-sonnet-4-5-20250929",
    max_tokens=250,
    messages=[{"role": "user", "content": "Hello, Claude"}]
)

print(message.content)

Using Portkey’s SDK

cURL

curl --location 'https://api.portkey.ai/v1/messages' \
--header 'x-portkey-provider: anthropic' \
--header 'Content-Type: application/json' \
--header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \
--data-raw '{
    "model": "@your-provider-slug/claude-sonnet-4-5-20250929",
    "max_tokens": 1024,
    "stream": true,
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Chennai?"
        }
    ]
}'

You can use all Portkey features (like caching, observability, configs) with this route. Just add the x-portkey-config, x-portkey-provider, x-portkey-... headers.

Prompt Caching

Portkey works with Anthropic’s prompt caching feature to save time and money. Refer to this guide:

Prompt Caching

Learn how to enable prompt caching for Anthropic requests

Structured Outputs

Ensure that the model always follows your supplied JSON schema with Portkey’s structured outputs support.

Structured Outputs

Learn how to use Pydantic, Zod, or JSON schema for structured data from Anthropic

Web Search

Anthropic Claude models support web search as a tool, allowing the model to search the web for up-to-date information.

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    strict_open_ai_compliance=False
)

response = portkey.chat.completions.create(
    model="@anthropic/claude-haiku-4-5-20250929",
    max_tokens=1024,
    messages=[{"role": "user", "content": "What is the latest news on Poland?"}],
    tools=[{
        "type": "web_search",
        "web_search": {
            "name": "web_search_20250305",
            "max_uses": 2
        }
    }]
)

print(response.choices[0].message.content)

Set strict_open_ai_compliance to false (or use the header x-portkey-strict-open-ai-compliance: false) to receive citations in the response.

Files API

Portkey supports Anthropic’s Files API (beta), enabling you to upload, list, retrieve, and delete files through the gateway. Uploaded files can be referenced in chat completions using file_id instead of re-uploading content each request.

Files API

Upload, list, retrieve, and delete files — then use them in chat completions

Service Tier

When routing Chat Completions requests to Anthropic, Portkey automatically translates OpenAI’s service_tier parameter to Anthropic’s native speed parameter:

`service_tier`	Anthropic `speed`
`auto`	`fast`
`standard_only`	`standard`
`default`	`standard`
`fast`	`fast`
`standard`	`standard`
unknown value	omitted

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@anthropic"
)

response = portkey.chat.completions.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
    service_tier="auto"  # maps to speed: "fast" on Anthropic
)

Beta Features

Portkey supports Anthropic’s beta features through headers. Pass the beta feature name as the value:

from portkey_ai import Portkey

client = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@anthropic",
    anthropic_beta="token-efficient-tools-2025-02-19",
    strict_open_ai_compliance=False
)

Managing Anthropic Prompts

Manage all prompt templates to Anthropic in the Prompt Library. All current Anthropic models are supported, and you can easily test different prompts. Use the portkey.prompts.completions.create interface to use the prompt in an application.

Next Steps

Add Metadata

Add metadata to your Anthropic requests

Gateway Configs

Add gateway configs to your Anthropic requests

Tracing

Trace your Anthropic requests

Fallbacks

Setup fallback from OpenAI to Anthropic

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation

Documentation Index

All Models

All Endpoints

Multi-Provider Support

​Quick Start

​Add Provider in Model Catalog

Complete Setup Guide →

​Basic Usage

​Chat Completions

​System Prompts

​Streaming

​Catch Overloaded Error on Stream

​How it works

​How to enable

​Example: Fallback on overload

​Error response

​Advanced Features

​Vision (Multimodal)

​PDF Support

​Extended Thinking (Reasoning Models)

​Using /messages Route

​Using Anthropic’s Native SDK

​Using Portkey’s SDK

​Prompt Caching

Prompt Caching

​Structured Outputs

Structured Outputs

​Web Search

​Files API

Files API

​Service Tier

​Beta Features

​Managing Anthropic Prompts

​Next Steps

Add Metadata

Gateway Configs

Tracing

Fallbacks

SDK Reference

Quick Start

Add Provider in Model Catalog

Basic Usage

Chat Completions

System Prompts

Streaming

Catch Overloaded Error on Stream

How it works

How to enable

Example: Fallback on overload

Error response

Advanced Features

Vision (Multimodal)

PDF Support

Extended Thinking (Reasoning Models)

Using /messages Route

Using Anthropic’s Native SDK

Using Portkey’s SDK

Prompt Caching

Structured Outputs

Web Search

Files API

Service Tier

Beta Features

Managing Anthropic Prompts

Next Steps