The model and the agent are decoupled by design.

Most AI products hardcode one model and let cost run away with usage. Switching models means a rewrite. We built the version where the agent and the model are decoupled by design — every reasoning step asks for a tier, the pool resolves the model, and changing what's behind a tier is a constructor argument. Cloud, mixed, or fully local: same agent, same prompts, same workflows.

01 · The tier abstractionThe agent asks for a tier, not a model

Three tiers sit in front of every model in the system: fast, balanced, powerful. Every call inside the agent declares which it wants via config.configurable.modelTier. The pool — a Runnable that implements the same shape as any single model — resolves the tier and invokes the underlying call. Calls without a tier get the default (balanced). The routing is set at the call site, by the step that knows its own cost/quality tradeoff. The agent code doesn't care which actual model is bound to each tier, and the prompts don't reference a model name. That's the point.

02 · Observable costTracked, not estimated

Token usage and USD cost are recorded at every call, tagged with the tier, the underlying model, the session, and a cost context label — rewrite, planning, agent_loop, grading, synthesis. USD rates are baked in for 8 model families across Anthropic and OpenAI — Opus, Sonnet, Haiku, GPT-4.1, GPT-4o, GPT-4o-mini, GPT-4.1-mini, GPT-4.1-nano. Anthropic's server-side web_search tool fee ($10 per 1,000 requests) is tracked separately on top of token cost. Streaming responses preserve usage tracking via a snapshot/delta fallback when the SDK doesn't emit usage in the final chunk. Clients see exactly what every conversation cost and where the money went.

03 · Swap the binding, not the agentModel changes are one line

Because the pool is a Runnable — the same composable interface every other piece of the framework speaks — swapping the model behind any tier is a configuration change. Want to move balanced from Sonnet to a cheaper option for a particular client? Change the binding. Want to run the powerful tier on Opus for one customer and GPT-4.1 for another? Different bindings, same agent. Want to run sensitive workloads entirely on the client's own hardware? Bind a local GGUF model — the on-device runtime implements the same Runnable interface, so the exact same agent code runs against it. No fork. No parallel codebase. No "cloud version vs. on-prem version."

04 · Native web access where it's saferCitations, not scrapers

When the agent needs to search the web, we use Anthropic's native server-side web_search tool with citation tracking — no scraping pipeline, no third-party search vendor, sources traceable back to the exact model call that surfaced them. Buyers who care about data lineage notice this. The same architecture means a deployment that can't use any cloud tool can simply not bind one — the local pool runs without it, and the agent degrades gracefully to vault-only retrieval.

05 · Why we built it this wayThe small piece that makes the promise deliverable

AI consulting is full of vendor-locked, expensive-by-default agent platforms. We built the version that lets a client start on Anthropic for speed, move workloads to OpenAI when a contract changes, and run sensitive workloads on their own hardware — all from the same product. The model pool is the small piece of infrastructure that makes that promise actually deliverable.

Tiers, per-call routing, observable cost, Runnable interface, local GGUF support, native web search. None of the pieces are exotic on their own. The work, and the value, is in stacking them so that the model becomes configuration — and the agent stops caring which one it's talking to.