Routing to the cheapest model that might work is easy. Routing without leaking sensitive context, changing models halfway through a tool loop, or granting a weaker backend authority it should not have is a different problem.

I kept the hard decisions deterministic. Privacy rules, backend health, measured capability, tool authority, and session affinity take precedence over the learned classifier. A small local embedding model only advises on ambiguous requests; it cannot override those boundaries.

What the evidence says

In one clean five-repeat experiment, the aggressive policy sent 81.15% of requests to local inference, up from a 22.82% conservative baseline. It also completed 10 of 10 authorized write/execute tasks without transport failures or unexpected side effects.

That did not settle the larger question. Quality remained inconclusive, and the latency and remote-token gates failed. The result supports the routing architecture and its authority model, not a claim that local-first routing is automatically faster, cheaper, or production-ready.

Read the evidence

• Experiment log
• Aggressive-local policy and tool authority
• Five-repeat execution result

Incident automation is mostly an authority problem. A model can suggest a plausible fix; the harder question is whether it should be allowed to act, on what target, for how long, and with what evidence left behind.

This project follows one deliberately narrow remediation path from typed intake through assessment, an immutable proposal, human approval, atomic execution, verification, and audit. Approval grants short-lived authority to perform the exact proposed action. The model never receives open-ended access to the runtime.

What the evidence says

The current evidence includes 114 offline tests, 13 disposable-container tests, and two live integrations. In the strongest end-to-end run, the system took an owned service from HTTP 503 and an unhealthy container state to HTTP 200 and healthy, while preserving the container identity and recording each step.

It is not a production incident-response platform. One bearer token controls mutation, there is no RBAC or production target discovery, and remediation is restricted to disposable resources created by the proof of concept. The container restart is a vehicle for testing approval, telemetry, and authority boundaries, not the product claim.

Read the evidence

• Verification record
• Disposable-service recovery decision
• Security boundary and remaining risks

Once several coding agents are working in parallel, “what should run next?” stops being a prompt and becomes a control-plane question. Chat history is a poor source of truth for dependencies, decisions, stale handoffs, and work that is technically ready but unsafe to dispatch.

Agent Cockpit stores that state in the repository that owns the work. It reads versioned work orders, dependencies, decisions, handoffs, and health checks, then separates hard blockers from degraded signals. A missing approval can stop dispatch; a stale but noncritical signal can warn without freezing everything.

The CLI and synthetic demo exercise the model, including checks against the repository’s own GitHub Actions state. It does not run agents, coordinate writes across repositories, or claim production adoption. Its job is narrower: make dispatch readiness inspectable and reviewable before another agent starts changing code.

Read the design

• Quick start and live-check behavior
• Trust-state dispatch semantics
• Repository-local execution boundary
• Subagent dispatch standard

A model name is not a performance result. Runtime, quantization, hardware, concurrency, context length, and prompting scaffold all change what a deployment can actually do.

llm-dyno measures the endpoint in front of you. It uses one OpenAI-compatible interface, deterministic code-based graders, context and concurrency tests, and math/options suites designed to separate capable deployments instead of producing easy perfect scores. Prompt scaffolding is recorded as an experimental variable rather than disappearing into the result.

Three private targets have completed the full artifact pipeline and produced validated profiles and self-contained reports. Their identities and measurements stay local by design, so the public repository does not contain a model leaderboard or a comparative performance claim. The harness is still pre-release; the evidence currently supports the measurement pipeline, not a universal ranking.

Read the methodology

• Measurement and claim boundaries
• Private multi-target integration evidence
• Why the suites are discriminating
• Why grading stays deterministic

Valid JSON is only the first boundary. The shape can be correct while the requested tool is unauthorized, its arguments are unsafe, or the tool returns data the rest of the application should not trust.

This runtime puts schemas on both sides of the model/tool boundary. It uses Pydantic validation, a hard attempt budget, typed failures, and an explicit registry that validates tool arguments and results around application-owned code. Plain prose never becomes a tool call, and provider failures stop rather than entering a blind retry loop.

The evidence is intentionally modest. Hermetic tests with scripted models and deterministic tools cover malformed output, recovery budgets, authorization, and result validation. There is not yet a frozen provider-response corpus or enough provider-specific integration evidence to claim broad runtime reliability.

Read the design

• Architecture and attempt semantics
• Test coverage and evidence limits
• Explicit tool-authority decision
• Experiment status

ReAct loops are easy to demo and surprisingly easy to leave unbounded. A model can repeat a plausible action, burn retries on malformed output, or keep planning after it has stopped making progress.

This project wraps observe → decide → act → reflect in a typed state machine. Iterations, tool calls, and structured-output recovery have separate budgets. Exact repeated actions and repeated application-visible state keys stop deterministic cycles. Model, validation, tool, and budget failures return typed summaries rather than escaping as uncontrolled exceptions.

The offline vertical slice covers single-tool and multistep flows. One live local-server run discovered a model and completed an addition tool call. That is useful integration evidence, but it is one narrow path. It does not establish open-ended planning reliability or broad compatibility across providers.

Read the evidence

• Architecture and dependency boundary
• Offline and live experiment record
• Testing boundaries
• Cycle-detection decision

I built this benchmark to answer what looked like a practical question: how often could a conservative repair layer recover malformed local-model tool calls without inventing unsafe intent?

The first version treated unsafe false positives as more important than recall or F1. Then endpoint-controlled follow-up testing changed the conclusion. llama.cpp produced 27 of 27 clean non-streaming calls and 36 of 36 clean streaming/parallel calls. Ollama’s parsed chat path produced 3 of 3 and then 16 of 16 clean calls, including the model that seeded the original investigation.

The broad “local models need tool-call repair” premise did not survive. The original 117-case corpus had captured raw, unparsed completion output, so its leaderboard cannot support a general model or library comparison. I archived the project rather than keep polishing a benchmark for a problem the normal serving path had already solved.

The useful artifact is the correction: capture point and serving stack mattered more than the model-level explanation I started with.

Read the finding

• Finding and reproduction procedure
• Why the project was archived
• Precision-first methodology

This started with a concrete failure: a local model expressed a tool call as JSON inside ordinary response text, while the client expected an OpenAI-style structured tool call.

The proxy repairs only a narrow class of responses. It checks the requested name and arguments against the tools declared by the client, then abstains when intent is unclear. On the companion 117-case raw-output corpus, version 0.1.0 reached 87.5% precision and 38.2% recall on recoverable cases, with two unsafe false positives. Real OpenCode samples still failed the final task-level check even when the protocol shape improved.

Later endpoint-controlled testing explained why. Current parsed chat endpoints already returned clean tool calls, including for the seed model. The proxy is therefore relevant mainly to raw-completion consumers and older or misconfigured serving stacks. It is archived because that scope is too narrow to justify an active compatibility layer.

Read the evidence

• Experiment log and failed task verification
• Where protocol repair stops helping
• Serving-stack scope correction