Switchyard is a local decision service for AI agents and automation systems.
It uses the open-weight Laya engine to answer small, typed questions on your machine. That lets a larger model spend its context and reasoning budget on the work that needs it.
For example, an agent can ask Switchyard:
- Which team owns this incident?
- Is this log event relevant to the task?
- Which retrieved notes fit inside a 12,000-byte context budget?
- Is this input safe to pass to the next stage?
Switchyard returns a structured answer with probabilities, model metadata, and a clear fallback path. It does not generate prose, edit files, run commands, retrieve documents on its own, or replace a coding agent.
Most agent workflows contain many decisions that do not need a large general-purpose model. A tool may return 100 log lines when 10 matter. A retrieval system may return notes from several unrelated projects. A routing step may need a yes/no answer before an expensive model call starts.
Sending all of that material to a hosted model costs tokens, adds latency, and makes it harder to see why a piece of context was kept or dropped.
Switchyard handles the narrow decision locally:
retrieval or tool output
|
v
Switchyard: classify, rank, and apply a budget
|
v
smaller cited payload for the main agent
The local process keeps model weights warm, accepts typed requests, and communicates over a user-owned Unix socket. The first model download needs network access. Inference does not.
See the benchmark methodology for reproducible local transcript replay, recorded usage accounting, token-count definitions, and the limits of unlabeled results.
flowchart LR
A[Retrieved candidates] --> B[Without selection]
B --> C[All candidate text sent to model]
A --> D[Local Switchyard evaluation]
D --> E[Review selection recommendation]
E --> F[Caller applies selected payload]
F --> G[Less candidate text sent to model]
Selection must happen before the provider request to reduce its input. Adding an MCP tool alone does not intercept requests or reduce a bill. Passing the entire selection report to the model can add context instead, because that report contains original text and assessment metadata.
The saved local run from September 22, 2026 evaluated our four-item synthetic smoke fixture. It recommended keeping two items and omitting two, retaining both labeled relevant items and the single mandatory item. The complete report includes the model revision, dataset hash, pipeline fingerprint, individual measurements, and limitations.
| Measurement | Result |
|---|---|
| Original candidate text | 222 characters |
| Recommended candidate text | 128 characters |
| Recommended reduction | 94 characters / 42.3% |
| Relevant items retained | 2 of 2 |
| Mandatory items retained | 1 of 1 |
| Total benchmark wall time | 2.31 seconds |
| Measured provider tokens or dollars saved | Not measured |
This is a historical smoke result, not a holdout or a prediction of production performance. The 94-character difference becomes 23.5 estimated tokens using the report's characters-divided-by-four heuristic. No provider tokenizer or billing API measured that figure. The lexical baseline also retained both relevant items; this fixture does not establish an advantage over simple rules. The run stayed advisory and did not enable omission.
A private local run inventoried Claude and Codex session archives on this machine, then replayed a deterministic sample of 100 tool-output cases per source through the warm Switchyard daemon. Raw transcripts stay private. The public aggregate report contains hashes, usage totals, replay counts, latency, and limitations only.
| Measurement | Claude | Codex |
|---|---|---|
| Sampled cases / sessions | 100 / 100 | 100 / 100 |
Original tool-text tokens (cl100k_base) |
57,977 | 66,781 |
| Recommended tool-text tokens | 57,596 | 64,831 |
| Recommended reduction | 381 / 0.7% | 1,950 / 2.9% |
| Uncertain chunks recommended for omission | 0 | 0 |
| Request errors (kept full text) | 3 | 0 |
| p50 / p95 request latency | 4.9s / 37.6s | 3.6s / 33.5s |
| Measured provider tokens or dollars saved | Not measured | Not measured |
Historical usage recorded in those same archives (deduplicated by response ID; not invoices):
| Source | Response records | Input tokens | Output tokens | Cache notes |
|---|---|---|---|---|
| Claude | 91,693 | 2,992,480 | 65,719,188 | cache write 399,474,934; cache read 21,727,843,673 |
| Codex | 4,051 | 474,630,067 | 864,376 | cached input 464,556,672; reasoning output 227,880 |
The small recommended reductions are expected under the current safety rule: uncertain and failed predictions are retained, and only clearly irrelevant non-mandatory chunks may be recommended for omission. A 50% byte budget alone does not force large cuts when most chunks are uncertain. This replay has no human relevance labels, so it cannot enable automatic filtering and must not be read as billing savings.
These scenarios are arithmetic examples, not benchmark results. They assume 2,000 fixed prompt tokens and 8,000 candidate tokens, with no extra prompt overhead. Removing half the candidate tokens reduces total input by 40%, not 50%.
For an uncached input rate of P dollars per million tokens, saving 4,000 input tokens across 1,000 requests would avoid 4 × P dollars in input charges. Output charges, local CPU time, electricity, caching, retries, and changes in task quality need separate accounting. Switchyard adds a local inference step, so reduced input does not guarantee lower end-to-end latency.
The charts are reproducible SVG assets rather than screenshots of an interface that does not exist. Rebuild them from the saved report with:
python3 scripts/render_readme_charts.pySwitchyard is:
- a local CPU decision runtime;
- a typed API for
choice,score, andnoulquestions; - a budgeted context-selection service;
- an MCP server for Claude Code and other MCP clients;
- a Python client and middleware layer for custom agent loops;
- an optional read-only provider for curated MainFrame Vault content.
Switchyard is not:
- a coding agent or general language model;
- a document store, search engine, or vector database;
- an authorization system;
- a command executor or file editor;
- a replacement for MainFrame;
- a hosted Jev service or a claim of Jev-compatible accuracy.
Jev is TypeSafe's hosted System One model. Laya is an independent open-weight decision engine. Switchyard runs Laya locally and keeps those products separate.
Switchyard currently installs from source. Linux and Apple Silicon macOS are the supported targets. Python 3.12 is the tested version.
Install the runtime and local inference dependencies with uv:
git clone https://github.com/mayur2109/switchyard.git
cd switchyard
uv sync --extra inference
uv run switchyard init
uv run switchyard models fetch
uv run switchyard serve --backgroundmodels fetch downloads a pinned Laya revision and records SHA-256 checksums. The English and
multilingual checkpoints need roughly 1.5 GB of disk space plus runtime memory. Choose a different
private data directory with SWITCHYARD_HOME:
SWITCHYARD_HOME=/data/switchyard uv run switchyard initCheck the installation:
uv run switchyard doctor
uv run switchyard statusThe runtime reports starting, loading, ready, or degraded. A socket existing does not mean
that model loading has finished.
The CLI reads one JSON request from standard input and writes one JSON response to standard output.
printf '%s\n' '{
"state": "The payment reconciliation job is failing",
"questions": {
"owner": {
"type": "choice",
"instructions": "Which team owns this issue?",
"criteria": {
"engineering": "software and infrastructure",
"billing": "payments and reconciliation"
}
}
}
}' | uv run switchyard decideThe response contains the selected choice, probabilities, confidence, selected model, revision, routing explanation, and timing. The answer is a prediction, not permission or verified fact.
printf '%s\n' '{
"state": "The log contains a database timeout",
"questions": {
"relevant": {
"type": "noul",
"instructions": "Is this relevant to diagnosing the billing outage?"
}
}
}' | uv run switchyard decidenoul is Laya's binary decision type. choice selects among named options. score selects an
ordered score description.
switchyard select evaluates caller-supplied candidates and builds an auditable budgeted plan.
The selector:
- keeps mandatory candidates;
- ranks other candidates by relevance;
- preserves caller order for selected output;
- keeps uncertain and failed predictions as fallback;
- records IDs, citations, trust labels, scores, and reasons;
- returns estimated bytes and tokens;
- never enables omission automatically.
printf '%s\n' '{
"recipe": "candidate-relevance@1",
"task": "investigate a billing incident",
"budget_bytes": 12000,
"items": [
{
"id": "billing-policy",
"text": "Payment failures follow the reconciliation workflow.",
"citation": "vault://billing-policy",
"trust": "curated",
"mandatory": true
},
{
"id": "unrelated-note",
"text": "The office garden checklist was updated.",
"citation": "vault://unrelated-note",
"trust": "unverified"
}
]
}' | uv run switchyard selectThe response includes two payloads:
recommended_selected_itemsis the budgeted recommendation;selected_itemsis the payload actually applied by policy.
Before a reviewed policy is enabled, selected_items remains the original input. This makes the
default behavior advisory and reversible.
Current integration caveat: forward_candidates() explicitly applies
recommended_selected_items when a plan is advisory. Calling that helper therefore bypasses the
policy gate. For shadow evaluation, forward plan["selected_items"] and inspect recommendations
separately. The transcript benchmark records recommended omissions, including uncertain items
that the current budget allocator can drop; it does not establish that those omissions are safe.
The client works in synchronous and asynchronous applications:
from switchyard.client import Client
from switchyard.contracts import DecisionRequest
client = Client()
result = client.decide(
DecisionRequest(
state="The payment job is failing",
questions={
"owner": {
"type": "choice",
"instructions": "Which team owns this?",
"criteria": {
"engineering": "software",
"billing": "payments",
},
}
},
)
)
print(result["answers"])For a custom agent loop, use the middleware helpers:
from switchyard.middleware import forward_candidates, select_context
plan = select_context(
client,
"investigate a billing incident",
candidates,
budget_bytes=12000,
)
# Copies the envelope and replaces only decision_candidates.items.
# If Switchyard is unavailable, the helper returns the original candidates.
forwarded_tool_result = forward_candidates(tool_result, plan)The asynchronous versions are aselect_context and the async client methods. The middleware fails
open on runtime errors. An unavailable local decision service must not block the calling agent.
Switchyard exposes four read-only MCP tools:
decideevaluate_itemsselect_itemscapabilities
Register the local server with Claude Code:
claude mcp add --scope user switchyard -- \
"$HOME/.local/bin/switchyard" mcpThe MCP server forwards requests to the warm local runtime. It does not retrieve external context, edit files, run commands, or grant permissions.
MainFrame remains a local-first knowledge store. Switchyard does not own it.
The optional MainFrameProvider searches only these curated tiers:
Projects/ Stacks/ Patterns/ Guidelines/ Notes/
It uses argument-safe ripgrep, returns stable candidate IDs and Vault citations, and does not
write retrieval logs or modify the Vault:
from pathlib import Path
from switchyard.mainframe import MainFrameProvider
from switchyard.middleware import select_context
provider = MainFrameProvider(Path.home() / "MainFrame")
candidates = provider.search("billing workflow")
plan = select_context(client, "billing workflow", candidates, budget_bytes=12000)The core package defines a generic read-only CandidateProvider protocol, so other users can plug
in a filesystem index, database, search service, or enterprise retrieval system without adding that
system to Switchyard's core dependencies.
Switchyard never writes MainFrame notes, syncs remotes, or manages Vault credentials. Remote backup and collaboration remain explicit MainFrame operations.
The built-in recipes are:
candidate-relevance@1, which asks whether supplied passages matter to a task;log-triage@1, which asks whether supplied log events help investigate a task;route-selection@1, which chooses among caller-defined routes without executing one.
All recipes preserve stable IDs, citations, trust labels, mandatory flags, and original text. Runtime failures and uncertain predictions keep the original item.
Automatic omission is disabled until a held-out report for the exact recipe, model revision, and pipeline fingerprint passes the retention gate. The gate requires, among other checks, 99% relevant retention, complete mandatory preservation, enough labeled cases, zero errors, and measurable content reduction.
The report estimates content reduction. It does not claim provider billing savings unless the calling harness supplies before/after provider token telemetry.
Run the included smoke corpus:
uv run switchyard bench examples/evaluation/smoke.json \
--report /tmp/switchyard-smoke.json
uv run switchyard report /tmp/switchyard-smoke.jsonThe smoke corpus demonstrates the workflow but cannot enable omission. Use a reviewed anonymized holdout corpus for that decision.
Switchyard runs one warm local process per user. The process:
- verifies pinned model files;
- loads the requested English and multilingual checkpoints;
- listens on a private Unix socket;
- serializes native inference through a bounded queue;
- returns structured JSON responses;
- removes the socket during clean shutdown.
The default runtime is CPU-only. Model loading can take time on the first start. Requests have bounded input size, queue capacity, and deadlines. The current design favors predictable behavior and safe fallback over maximum throughput.
uv sync --group dev
uv run ruff check src tests
uv run pytest -q
uv buildEvery push and pull request runs linting, tests, and package builds on Linux and Apple Silicon macOS.
Switchyard is useful today as a local decision and selection service. The following work is still deliberately separate:
- automatic integration into a specific agent's next model call;
- a production-sized labeled holdout corpus;
- provider billing measurements;
- hosted multi-user deployment;
- Windows transport and service support;
- generated context summaries;
- MainFrame write access.
Switchyard runtime code is Apache-2.0. Laya is an external dependency. The installer records the upstream model revision and checksums. See LICENSE and NOTICE.