LLM provider configuration

The Assistant SDK is provider-pluggable through an LLMProvider abstraction. Two implementations ship today:

MetabindAgentProvider — calls Metabind’s hosted Agent proxy at agent.metabind.ai. The proxy holds the LLM key, runs the tool loop server-side, and streams responses back as SSE. Recommended for production.
AnthropicProvider — bring-your-own-key (BYOK). The SDK calls Anthropic directly from the client. Useful for development, internal tools, or apps where the key reaches the SDK from an authenticated user-managed source.

You can also implement the public LLMProvider protocol to plug in something else.

When to pick which

Mode	Pick when
Agent proxy (`MetabindAgentProvider`)	Production. Your binary ships only a Metabind project token; the LLM key stays on the server. The proxy authenticates the project token, calls the LLM, runs the tool loop, and streams results back.
BYOK direct (`AnthropicProvider`)	Local development. Internal tools where the key is provisioned per user from a trusted source. You explicitly want a different LLM provider, region, or routing policy than the proxy offers.

The two modes are interchangeable from the rest of the SDK’s perspective — the same assistant setup and conversation state. Only the provider object changes. (On Android the provider is used directly; see the Android SDK.)

Mode 1: Agent proxy (recommended)

The Agent proxy is a Metabind-managed service at https://agent.metabind.ai. SDKs call POST https://agent.metabind.ai/{orgId}/{projectId}/chat with a Bearer-authenticated streaming request. The proxy authenticates the caller with a Metabind project token, runs the LLM call and the tool loop on the server side, and returns the result as a Server-Sent Events stream.

The client app calls agent.metabind.ai with a Metabind project token; the proxy holds the LLM key and runs the tool loop server-side, returning an SSE stream.

What you ship in the binary: a Metabind project token. What you don’t ship: any third-party LLM key.

import MetabindAssistant

let assistant = MetabindAssistant(
  serverURL: URL(string: "https://mcp.metabind.ai/my-org/oak-and-ivory")!,
  serverHeaders: ["Authorization": "Bearer \(projectToken)"],
  provider: MetabindAgentProvider(
    apiKey: projectToken,
    orgId: "my-org",
    projectId: "oak-and-ivory"
  )
)

import ai.metabind.assistant.MetabindAgentProvider
import ai.metabind.mcpappshost.LLMMessage

// Android has no MetabindAssistant wrapper — call the proxy provider directly.
val provider = MetabindAgentProvider()

provider.streamMessage(
  baseUrl = "https://agent.metabind.ai",
  apiKey = projectToken,
  orgId = "my-org",
  projectId = "oak-and-ivory",
  messages = history   // List<LLMMessage>
).collect { event -> /* TextDelta, ToolCallStart, ToolResult, Done, Error */ }

The apiKey field on MetabindAgentProvider is the Metabind project token, not an LLM provider key. The proxy uses it to authenticate the project; the LLM key is held server-side.

What the proxy does

Authenticates the request with the project token (Bearer header).
Routes to the LLM provider configured for the project.
Runs the tool-call loop server-side: when the LLM emits a tool call, the proxy invokes the project’s MCP tool, returns the result to the LLM, and continues until the LLM produces a final answer.
Streams events back to the client over SSE — message_start (includes conversationId), text_delta, tool_use, tool_result, provider_switch, message_stop.

The client never holds the LLM key, never knows which provider answered, and doesn’t run a tool loop locally — that simplifies clients and makes provider switches a server-side decision.

Server-side provider selection

The proxy is multi-provider. Each project picks one of the supported providers in MCP App Studio: Anthropic, OpenAI, or Google. The provider, model, and key are all configured server-side; the client just sees an SSE stream. To switch providers for an entire project, change the setting in MCP App Studio — no client release needed.

The Setup Agent dialog in MCP App Studio with Provider, Model, and API Key fields — the LLM key is encrypted server-side and never returned in API responses

Conversation IDs

The proxy stores conversation history server-side. The first event of every response (message_start) includes a conversationId. To continue the same conversation on a later turn (after an app restart, or across devices), echo that conversationId back on the next chat request and the proxy merges history against its stored record. The default chat surface (MetabindAssistantView) handles this for you. If you build a custom UI on top of assistant.send(...), persist the conversation ID alongside whatever else you’re storing.

Mode 2: BYOK direct (Anthropic)

BYOK direct mode bypasses the Agent proxy entirely. The client calls Anthropic with a key you provide and runs the tool loop locally. Use this for development, internal tools, or apps where the key reaches the SDK from a trusted source.

import MetabindAssistant

let assistant = MetabindAssistant(
  serverURL: URL(string: "https://mcp.metabind.ai/my-org/oak-and-ivory")!,
  serverHeaders: ["Authorization": "Bearer \(projectToken)"],
  provider: AnthropicProvider(
    apiKey: anthropicKey,
    model: "claude-sonnet-4-20250514"
  )
)

In this mode, the client runs the tool loop itself — calls the LLM, receives tool requests, calls the MCP server, returns the tool results to the LLM, and so on.

Do not ship a real Anthropic API key in a production app — anyone can extract it from the binary or DevTools. BYOK mode is appropriate for local development, internal tooling, or apps where the key is delivered to the SDK from an authenticated user-managed source. For public-facing production, prefer the Agent proxy.

Custom providers

Conform to the public LLMProvider protocol if you need to integrate something else — an internal LLM endpoint, a fine-tuned model, a different vendor. The protocol is small: a streaming method that takes messages and tool definitions and emits chunks in the SDK’s chunk format.

public protocol LLMProvider: Sendable {
  func stream(
    messages: [LLMMessage],
    tools: [LLMTool]?,
    systemPrompt: String?
  ) -> AsyncStream<LLMEvent>

  var runsToolsRemotely: Bool { get }
  func resetConversation() async
}

The exact signature varies by platform; see the per-platform headers for the concrete types.

Keys at a glance

Key	Where it lives	What it gates
Metabind project token	Your backend (mint per user / per session). Reaches the SDK via your auth flow.	Listing and calling tools on your project; authenticating to the Agent proxy.
LLM provider API key	Agent proxy mode: server-side, never in the client. BYOK direct mode: delivered to the SDK by your auth flow, ideally short-lived.	Conversational generation.

For production, mint Metabind project tokens server-side and refresh as needed. The provider’s apiKey accessor is callable so you can refresh on demand without rebuilding the assistant.

Choosing a model

The Agent proxy decides the model server-side, configured per project in MCP App Studio. For BYOK, pass the model string explicitly to AnthropicProvider. A few rough heuristics for either mode:

Default to mid-tier. Sonnet 4.6 is well-priced and capable enough for most assistant workloads.
Scale up for hard reasoning. When tool selection requires multi-step thinking or your tool set is large (50+ tools), upgrade to Opus 4.7.
Scale down for simple flows. If your assistant calls one of three tools and replies in a sentence, a smaller model saves cost without quality loss.

Test multiple models with your actual prompts before committing — model-to-prompt fit varies more than benchmarks suggest.

Per-user metering

Once requests go through the Agent proxy, Metabind’s usage tracking captures tokens per project token. If you mint per-user project tokens, the audit trail naturally segments by user. For BYOK direct mode, you’re metering against your own LLM key — use the provider’s dashboard or a backend proxy you control.

Troubleshooting

Symptom	Likely cause
All requests 401	Project token expired or the proxy can’t validate it; refresh logic broken.
SSE stream closes immediately	Proxy rejected the request — check that the project token has access to the MCP project.
Tool selection feels wrong	Sharpen tool descriptions in MCP App Studio, or upgrade to a more capable model.
BYOK Anthropic 401	Key invalid or expired; check the provider’s dashboard.

iOS SDK

Where the LLM provider plugs in for iOS.

Android SDK

Where the LLM provider plugs in for Android.

Assistant SDK overview

What the Assistant SDK is and which surface to pick.

Custom host UI

Replace the default chat UI with your own.

Getting Started

Core Concepts

Building Your MCP App

Publishing

Connecting to MCP Hosts

Assistant SDK

Operations

Sample Apps

LLM provider configuration

When to pick which

Mode 1: Agent proxy (recommended)

What the proxy does

Server-side provider selection

Conversation IDs

Mode 2: BYOK direct (Anthropic)

Custom providers

Keys at a glance

Choosing a model

Per-user metering

Troubleshooting

iOS SDK

Android SDK

Assistant SDK overview

Custom host UI

​When to pick which

​Mode 1: Agent proxy (recommended)

​What the proxy does

​Server-side provider selection

​Conversation IDs

​Mode 2: BYOK direct (Anthropic)

​Custom providers

​Keys at a glance

​Choosing a model

​Per-user metering

​Troubleshooting

​Related

iOS SDK

Android SDK

Assistant SDK overview

Custom host UI

When to pick which

Mode 1: Agent proxy (recommended)

What the proxy does

Server-side provider selection

Conversation IDs

Mode 2: BYOK direct (Anthropic)

Custom providers

Keys at a glance

Choosing a model

Per-user metering

Troubleshooting

Related