Any LLM, One Trait

You have opinions about which LLM to use. We have opinions about not caring which one you pick.

The trait

Everything flows through StreamFn. It takes a model name, a conversation context, and some options. It returns a stream of events. That's it. That's the whole interface between your agent and whatever LLM is on the other end.

The harness never touches provider credentials. It never imports a provider SDK. It doesn't know if you're talking to Claude, GPT, Gemini, or a model running on your laptop. It just consumes the stream.

Nine adapters ship in the box

Each one normalizes a different provider's wire format into the same Stream<AssistantMessageEvent> protocol. SSE, NDJSON, AWS SigV4 — we handled the paperwork so you don't have to.

Anthropic SSE + thinking blocks

OpenAI SSE, multi-provider

Google Gemini SSE

Ollama NDJSON (local)

AWS Bedrock SSE + SigV4 auth

Azure OpenAI SSE

Mistral SSE

xAI SSE

Proxy SSE + delta reconstruction

Plus LocalStreamFn in the local-llm crate for fully on-device inference. No API key required. No internet required. Just vibes and a GPU.

Or bring your own

If your provider isn't on the list, implement StreamFn yourself. It's one async method. You get full control over auth, transport, and error handling.

The MessageConverter trait handles serializing harness messages into your provider's expected format, so adding a new provider is just two traits and zero forking.

// The whole contract:
async fn stream(
  &self,
  model: &str,
  context: &AgentContext,
  options: StreamOptions,
) -> Stream<AssistantMessageEvent>;

// That's it. Go home.

What you get for free

Unified event protocol Every adapter emits the same start/delta/end events. Your agent code works identically regardless of provider.

Error classification Throttled, auth failure, context overflow, network error — adapters tag errors structurally so the retry strategy knows what to do.

Structured logging All adapters use the tracing crate. Debug, warn, error — consistent observability across providers.

Swap at runtime StreamFn is a trait object. Switch providers between turns, A/B test models, fall back to local — no recompilation needed.