↓ Skip to main content
tomrochette.com
  1. Agents/

Hybrid execution

Author
Tom Rochette
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: in progress / Readability: 3 / 5 minutes read (~915 words)
agents 48 hybrid-execution 26
Table of Contents

Small fast models handling typed decisions beside the large model: constrained decoding, validate-and-retry libraries, the decision-model wave, and the three benchmarks that measure them.

  • Anthropic structured outputs - schema-constrained decoding for Claude responses and tool inputs.
  • AnyJev - Nokia Applied Research’s training-free readout turning any instruction-tuned LLM into a calibrated decision model with rotation-averaged logits and verified early-exit bounds, with an arXiv report, third-party benchmark coverage, and a fifth-place Decision Index row, 1,242 stars.
  • Atomic Agents - the MIT Python framework assembling agents, tools, and context providers as schema-validated components on Instructor and Pydantic, the validate-and-retry family’s framework layer, 6.3k stars.
  • BAML - Boundary’s Apache-2.0 typed language for agent code, where LLM calls are functions with schema return types and the error-correcting parser from its structured-output era, 9.4k stars.
  • CLM - the Apache-2.0 contrastive state-action encoder (a Qwen3-8B embedding model plus a 75 MB head) behind the wave’s first architectural dissent, whose Jev-parity claims both independent boards answer with near-zero scores, 2.9k stars.
  • CUA-S1 - Cua’s open-weights 706k-parameter System One checkpoint that scores form actions without generating text, the verifiable counterpart to Jev’s closed contract.
  • Decider - Mapika’s Apache-2.0 one-pass family from 2B to 35B-A3B on Qwen3.5 decoders, JevBench’s first open number one and the category’s clearest sealed-protocol collapse record.
  • Decision Index - the second independent scoreboard for Jev-class models, 110,201 public requests over 42 benchmarks plus private tests, where seven systems sit above Jev (eighth) and the boards disagree by tens of points.
  • Guardrails - the Apache-2.0 validator-hub framework composing Input and Output Guards from 65 installable risk validators around 100-plus LiteLLM-reachable providers, with validate-and-reask structured generation, now under Harvey after its September acquisition.
  • H2O-Lightning-4B - H2O.ai’s Apache-2.0 Qwen3.5-4B decision fine-tune with vision, the first open-weights system measured above hosted Jev on an independent composite.
  • Instructor - Pydantic in, validated objects out, with a re-ask when validation fails.
  • Jeeves - PostHog’s 9B reasoning decision model that thinks before it answers typed questions, beating Jev’s published numbers on its own tests at ten times the latency.
  • Jeff - the AutoJev-fork fine-tune family (0.8B and 2B Qwen3.5, Gemma 4 E2B) speaking Jev’s request format at 22 ms locally, with the frankest self-run benchmark table in the wave.
  • Jev - TypeSafe’s System One model that skips text generation entirely, typed decisions with calibrated confidence at 70-500ms, early access since September 2026.
  • JevBench - Benchmark Heaven’s MIT scoreboard for Jev-class decision models, scored on intelligence, calibration, speed, and cost, whose boards keep getting rebuilt under the wave (Jev first through v1.4.2, the unranked reference on the October v1.6.1 redesign, with H2O-Lightning-4B now the open leader above it).
  • Jevlike - the community’s one-day reverse-engineering of the Jev contract, an MIT option-attention starter, dormant since launch day, its head lifted by CUA-S1.
  • Kev - Jared Palmer’s Apache-2.0 decision-model family (0.8B to a full-weights 27B, versioned together as Kev 1.0) speaking the Jev API locally, with the wave’s strongest pre-registered eval discipline.
  • Laya - the Apache-2.0 open-weights decision-model family answering typed questions in a single pass, the open rival to Jev’s contract with its limits stated on its own model card.
  • NanoJev - the 0.6B game-task replica with the most complete pipeline and the least verification, every benchmark unreplicated.
  • Nimble - Bespoke Labs’ one-day open Jev with the category’s only human-labeled head-to-head (Jev wins by 1.2 macro points) and an unlicensed repo.
  • Ollaya - the Apache-2.0 local runtime serving twenty-one open decision-model families behind a wire-identical Jev API, with parity checks against each author’s own code.
  • OpenAI Structured Outputs - schema-guaranteed responses via constrained decoding.
  • Outlines - logit-masked generation following types, schemas, regexes, or grammars.
  • SemIf - frozen open models reading typed option probabilities straight from the logits, with a client-side WebGPU demo, called OpenJev until 2026-09-18.
  • Strands Decider 2B - the Strands Agents team’s Apache-2.0 2B decision model with the wave’s best-published training discipline (seed tables, a soup, a calibration contract) and the weakest third-party reading so far, 585 stars.
  • tool-eval-bench - the MIT auditor for tool-calling quality on self-hosted serving stacks, 69 deterministic multi-turn scenarios with full traces plus a decision-model track over the same /v1/systemone surface.
  • Valen - the Apache-2.0 open-weights Jev-family decision model with vision (0.8B/2B/4B, ONNX and GGUF exports, training code and benchmarks published), three weeks old with a 2-point HN footprint.
  • Von - the Apache-2.0 395M ModernBERT encoder answering typed questions on CPUs through Python and TypeScript SDKs and a Docker image, the wave’s smallest credible generalist, near the bottom of the accuracy tables by third-party reading.
  • Winnow - EldanRing’s Apache-2.0 Gemma 4 family (E2B to 12B, GGUF, vision) matching Jev answer-for-answer on public subsets, the community checkpoint that three independent readers rank at the top.

Its members are compared on shared rows in the Hybrid Execution Feature Matrix.

Changes
#

  • 2026-08-24 - Added Anthropic structured outputs.
  • 2026-08-24 - Added Instructor.
  • 2026-08-24 - Added OpenAI Structured Outputs.
  • 2026-08-24 - Added Outlines.
  • 2026-09-18 - Added Jev.
  • 2026-09-20 - Added CUA-S1.
  • 2026-09-21 - Added Jevlike.
  • 2026-09-21 - Added Kev.
  • 2026-09-21 - Added Laya.
  • 2026-09-21 - Added NanoJev.
  • 2026-09-21 - Added Nimble.
  • 2026-09-21 - Added SemIf.
  • 2026-09-22 - Added JevBench.
  • 2026-09-29 - Added Jeff.
  • 2026-10-06 - Added Jeeves.
  • 2026-10-06 - Added Ollaya.
  • 2026-10-07 - Added Decision Index.
  • 2026-10-07 - Added AnyJev.
  • 2026-10-07 - Added Atomic Agents.
  • 2026-10-08 - Added tool-eval-bench.
  • 2026-10-11 - Added CLM.
  • 2026-10-11 - Added Decider.
  • 2026-10-11 - Added H2O-Lightning-4B.
  • 2026-10-11 - Added Von.
  • 2026-10-11 - Added Winnow.
  • 2026-10-11 - Added BAML.
  • 2026-10-12 - Added Strands Decider 2B.
  • 2026-10-13 - Added Valen.
  • 2026-10-13 - Added Guardrails.