LLM provider configuration

The Assistant SDK is provider-pluggable through an LLMProvider abstraction. Two implementations ship today:

MetabindAgentProvider — calls the Agent Proxy, Metabind’s hosted agent runtime at agent.metabind.ai. The proxy runs your agent with your system prompt, on the model you choose, with your own model key encrypted server-side. It runs the tool loop on the server and streams responses back as SSE. Recommended for production.
AnthropicProvider — bring-your-own-key (BYOK). The SDK calls Anthropic directly from the client. Useful for development, internal tools, or apps where the key reaches the SDK from an authenticated user-managed source.

You can also implement the public LLMProvider protocol to plug in something else.

When to pick which

Mode	Pick when
Agent Proxy (`MetabindAgentProvider`)	Production. Your binary ships only a Metabind project token; your model key stays on the server. The proxy authenticates the project token, calls the LLM, runs the tool loop, and streams results back.
BYOK direct (`AnthropicProvider`)	Local development. Internal tools where the key is provisioned per user from a trusted source. Anthropic only.

On iOS, the two modes are interchangeable from the rest of the SDK’s perspective — the same assistant setup and conversation state. Only the provider object changes. Android supports only the Agent Proxy: its MetabindAssistant takes apiKey, orgId, and projectId and has no provider parameter. See the Android SDK.

Mode 1: Agent Proxy (recommended)

The Agent Proxy is a Metabind-managed service at https://agent.metabind.ai. SDKs call POST https://agent.metabind.ai/{orgId}/{projectId}/chat with a Bearer-authenticated streaming request. The proxy authenticates the caller with a Metabind project token, runs the LLM call and the tool loop on the server side, and returns the result as a Server-Sent Events stream. orgId and projectId must be IDs, not slugs; see Find your organization and project IDs.

The client app calls agent.metabind.ai with a Metabind project token; the proxy holds your own model key and runs the tool loop server-side, returning an SSE stream.

What you ship in the binary: a Metabind project token. What you don’t ship: any third-party LLM key.

import MetabindAI

let orgId = "<organizationId>"
let projectId = "<projectId>"

let assistant = MetabindAssistant(
  serverURL: URL(string: "https://mcp.metabind.ai/\(orgId)/projects/\(projectId)")!,
  serverHeaders: ["Authorization": "Bearer \(projectToken)"],
  provider: MetabindAgentProvider(
    apiKey: projectToken,
    orgId: orgId,
    projectId: projectId
  )
)

import ai.metabind.ai.MetabindAssistant

// Android always uses the Agent Proxy.
val assistant = MetabindAssistant(
  apiKey = projectToken,
  orgId = "<organizationId>",
  projectId = "<projectId>"
)

The apiKey argument (on MetabindAgentProvider on iOS, on MetabindAssistant on Android) is the Metabind project token, not an LLM provider key. The proxy uses it to authenticate the project; your model key is held server-side.

What the proxy does

Authenticates the request with the project token (Bearer header).
Sends your project’s server instructions and system prompt to the model on every turn. See System prompt.
Routes to the model provider configured for the project.
Runs the tool-call loop server-side: when the LLM emits a tool call, the proxy invokes the project’s MCP tool, returns the result to the LLM, and continues until the LLM produces a final answer.
Streams events back to the client over SSE — message_start (includes conversationId), text_delta, tool_use, tool_result, provider_switch, message_stop.

The client never holds the model key and doesn’t run a tool loop locally. Provider choice is a server-side decision; if the project has a failover provider and the proxy switches to it mid-conversation, the stream carries a provider_switch event.

System prompt

The system prompt is the instructions you write for your agent: its voice, rules, and goals. Write it in Metabind Studio under Project Settings → Agent → System Prompt, or set it from the terminal:

metabind agent set --system-prompt "You are the shopping assistant for Oak & Ivory. Recommend products from the catalog only."

On every model call in the tool loop, the proxy sends your project’s MCP server instructions first, then your system prompt. Use the server instructions for when to call which tool, since every MCP host receives them too, and the system prompt for how your agent behaves in your own apps. Changes take effect on the next turn, with no client release. A system prompt steers the model; it can’t force one. In Claude, ChatGPT, and other hosts, the host keeps its own system prompt, and your tool descriptions and server instructions are what guide its model.

Server-side provider selection

The proxy is multi-provider. Each project picks one of the supported providers in Metabind Studio: Anthropic, OpenAI, or Google (Gemini). You add your own model key for that provider, and Metabind encrypts it server-side; it’s never returned in API responses and never ships in your app. The provider, model, and key are all configured server-side; the client just sees an SSE stream. To switch providers for an entire project, change the setting in Metabind Studio or with metabind agent set --provider, with no client release needed.

The Setup Agent dialog in Metabind Studio with Provider, Model, and API Key fields — your model key is encrypted server-side and never returned in API responses

Conversation IDs

The first event of every response (message_start) includes a conversationId. The proxy keeps each conversation’s model context server-side for one hour after its last turn. A chat request that echoes the conversationId within that hour continues from that context; after it, the proxy has no earlier context for that ID. The proxy has no endpoint for reading a conversation’s history.

iOS: MetabindAgentProvider echoes the ID on each turn while the assistant is alive. To resume after a relaunch, save currentConversationId and pass it to MetabindAgentProvider(conversationId:) when you create the next assistant. This restores the model’s context, not the on-screen transcript: assistant.conversation.messages starts empty.
Android: the SDK has no API to read or set the conversation ID. Each MetabindAssistant starts a new conversation, and reset() starts another.

Mode 2: BYOK direct (Anthropic)

BYOK direct mode bypasses the Agent Proxy entirely. The client calls Anthropic with a key you provide and runs the tool loop locally. Use this for development, internal tools, or apps where the key reaches the SDK from a trusted source.

import MetabindAI

let assistant = MetabindAssistant(
  serverURL: URL(string: "https://mcp.metabind.ai/\(orgId)/projects/\(projectId)")!,
  serverHeaders: ["Authorization": "Bearer \(projectToken)"],
  provider: AnthropicProvider(
    apiKey: anthropicKey,
    model: "claude-sonnet-5-5"
  )
)

In this mode, the client runs the tool loop itself — calls the LLM, receives tool requests, calls the MCP server, returns the tool results to the LLM, and so on.

Do not ship a real Anthropic API key in a production app — anyone can extract it from the binary or DevTools. BYOK mode is appropriate for local development, internal tooling, or apps where the key is delivered to the SDK from an authenticated user-managed source. For public-facing production, use the Agent Proxy.

Custom providers

Conform to the public LLMProvider protocol if you need to integrate something else — an internal LLM endpoint, a fine-tuned model, a different vendor. The protocol is small: a streaming method that takes messages and tool definitions and emits chunks in the SDK’s chunk format.

public protocol LLMProvider: Sendable {
  func stream(
    messages: [LLMMessage],
    tools: [LLMTool]?,
    systemPrompt: String?
  ) -> AsyncStream<LLMEvent>

  var runsToolsRemotely: Bool { get }
  func resetConversation() async
}

The exact signature varies by platform; see the per-platform headers for the concrete types.

Keys at a glance

Key	Where it lives	What it gates
Metabind project token	Your backend (mint per user / per session). Reaches the SDK via your auth flow.	Listing and calling tools on your project; authenticating to the Agent Proxy.
Your model key	Agent Proxy mode: encrypted server-side, never in the client. BYOK direct mode: delivered to the SDK by your auth flow, ideally short-lived.	Conversational generation.

For production, mint Metabind project tokens server-side and refresh as needed. On iOS and Android, an assistant keeps the key it was created with: apiKey is a constant on MetabindAgentProvider and AnthropicProvider on iOS, and on MetabindAssistant on Android. To switch to a new key, create a new assistant with it. On Android, call close() on the old MetabindAssistant. On iOS, to keep the model’s context, pass the old provider’s currentConversationId to the new MetabindAgentProvider(conversationId:). On the web, agent.apiKey can be a function, which the SDK calls for each request; see the Web SDK.

Choosing a model

The Agent Proxy decides the model server-side, configured per project in Metabind Studio, where the model list follows the provider you pick. For BYOK, pass the model string explicitly to AnthropicProvider. A few rough heuristics for either mode:

Default to mid-tier. A mid-tier model, such as Claude Sonnet 5.5 (Studio’s default for Anthropic), is well-priced and capable enough for most agent workloads.
Scale up for hard reasoning. When tool selection requires multi-step thinking or your tool set is large (50+ tools), move up to a frontier model, such as Claude Opus 5.5.
Scale down for simple flows. If your agent calls one of three tools and replies in a sentence, a smaller model saves cost without quality loss.

Test multiple models with your actual prompts before committing — model-to-prompt fit varies more than benchmarks suggest.

Per-user metering

Once requests go through the Agent Proxy, Metabind records token usage for the project. For per-user detail, such as each user’s conversations and token counts, trace the proxy to your own Langfuse project.

Trace conversations in Langfuse

The Agent Proxy can export a trace of every turn, with prompts, completions, tool calls, and token usage, to your own Langfuse project. Traces go to Langfuse’s US cloud (https://us.cloud.langfuse.com) by default; set host to https://cloud.langfuse.com or https://eu.cloud.langfuse.com to use another Langfuse cloud region. Traces are stored in your Langfuse project, under your Langfuse retention settings. There’s no Langfuse setting in Metabind Studio or the CLI yet. Turn it on with the projects API, as settings.agent.observability.langfuse:

{
  "enabled": true,
  "publicKey": "pk-lf-...",
  "secretKey": "sk-lf-...",
  "host": "https://us.cloud.langfuse.com"
}

The secret key is encrypted server-side and never returned. Changing it requires permission to update the project’s MCP settings.

The projects API replaces settings.agent as a whole. Read the project first, add observability to its current settings.agent, and send the whole object back. provider is required, and any other field you leave out, such as the model or system prompt, is cleared. Leave out apiKey to keep the stored model key.

For BYOK direct mode, you’re metering against your own model key — use the provider’s dashboard or a backend proxy you control.

Troubleshooting

Symptom	Likely cause
All requests 401	Project token expired or the proxy can’t validate it; refresh logic broken.
SSE stream closes immediately	Proxy rejected the request — check that the project token has access to the MCP project.
Tool selection feels wrong	Sharpen tool descriptions in Metabind Studio, or upgrade to a more capable model.
BYOK Anthropic 401	Key invalid or expired; check the provider’s dashboard.

iOS SDK

Where the LLM provider plugs in for iOS.

Android SDK

Where the LLM provider plugs in for Android.

Assistant SDK overview

What the Assistant SDK is and which surface to pick.

Custom host UI

Replace the default chat UI with your own.

Getting Started

Core Concepts

Building Your MCP App

Publishing

Connecting to MCP Hosts

Assistant SDK

Operations

Sample Apps

LLM provider configuration

When to pick which

Mode 1: Agent Proxy (recommended)

What the proxy does

System prompt

Server-side provider selection

Conversation IDs

Mode 2: BYOK direct (Anthropic)

Custom providers

Keys at a glance

Choosing a model

Per-user metering

Trace conversations in Langfuse

Troubleshooting

iOS SDK

Android SDK

Assistant SDK overview

Custom host UI

​When to pick which

​Mode 1: Agent Proxy (recommended)

​What the proxy does

​System prompt

​Server-side provider selection

​Conversation IDs

​Mode 2: BYOK direct (Anthropic)

​Custom providers

​Keys at a glance

​Choosing a model

​Per-user metering

​Trace conversations in Langfuse

​Troubleshooting

​Related

iOS SDK

Android SDK

Assistant SDK overview

Custom host UI

When to pick which

Mode 1: Agent Proxy (recommended)

What the proxy does

System prompt

Server-side provider selection

Conversation IDs

Mode 2: BYOK direct (Anthropic)

Custom providers

Keys at a glance

Choosing a model

Per-user metering

Trace conversations in Langfuse

Troubleshooting

Related