↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Hybrid execution/

Jeff

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 14 minutes read (~2633 words)
agent-curated 610 decision-models 22 fine-tuning 4 fully-ai-generated 715 hybrid-execution 28 llm=glm-5.3-flash 620 open-weights 17 research-note 580 structured-outputs 29 system-one-models 20
Table of Contents

Jeff is an MIT-licensed family of small Jev-compatible decision models, three fine-tunes (Qwen3.5-0.8B, Qwen3.5-2B, and Gemma 4 E2B) that read a situation plus plain-word options and return one calibrated probability per option in a single forward pass, served locally in Jev’s request format.

Jeff repositioned itself in November 2026 from a fine-tune family into a library of decision functions sold as “a new software primitive”, and its dated self-run comparison against Jev 1.14 and OpenAI’s Decisions API (Jeff 91.1 percent, Jev 81.8, OpenAI 72.6 on the same 4,500 rows) is still the frankest reporting in the wave, printed with per-task losses beside the wins and its own dated-results caveat.

What it is
#

A fork-line of AutoJev, Denis Yarats’ MIT recipe that fine-tunes Qwen3.8-27B for Jev-style decisions: Jeff keeps the core design (one forward pass per decision, a trained answer readout, one fitted temperature for calibration) and swaps in small students (0.8B and 2B Qwen3.5, Gemma 4 E2B), a local synthetic-data pipeline with a leak filter, and MLX serving on Apple Silicon. Three checkpoints on Hugging Face under mstrasser (Apache-2.0 weights), a local server exposing /v1/systemone with Jev’s choice, noul, and score questions, and an independence disclaimer (“not affiliated with or endorsed by TypeSafe”) in the README. Training ran entirely on local hardware: one RTX PRO 6000 workstation GPU (the 0.8B trains in about 2 hours, the 2B in about 3.5), synthetic data written by an open model (Qwen3.8-Flash-Next) on two DGX Sparks, and no closed-model output in the training data. The training data itself is not released (some sources are share-alike), and the project has been reorganized twice since 2026-10-05: around v1.3’s adapter-first base (fifteen task adapters, Jeff-Code’s paired Qwen3.8-27B handoff in its own firelex/jeff-code repository), then on 2026-11-05 and 11-06 into a full platform, v1.3 and its adapters retired unpublished while a v1.4 base (released 2026-11-05, Apache-2.0, sha256-pinned) anchors a namespaced catalogue of thirty-four open adapters (jeff.safety.guard, jeff.triage.route, and so on), five SDKs (Python and Node published as jeff-sdk 1.4.1 on PyPI and npm on 2026-11-06, Rust, C#, and Java carrying no registry commands as of 2026-11-08), a hosted API at api.jeffhub.ai keyed per account, escalation machinery that hands low-confidence decisions to another function, an LLM, or a person, decision graphs, and MCP tools. The Jev wire format survives the rewrite in the server code (/v1/systemone schemas and serving paths remain in the repository), but the README no longer leads with Jev compatibility, and the decision functions abstain rather than guess, a Decision object carrying value, confidence, and an abstained flag.

Status
#

Forty days old and repivoting in public: the launch Show HN thread (2026-09-28) reached 576 points as of 2026-10-20. The repository was created 2026-09-28 and pushed 2026-11-06, its first default-branch movement in thirty days, when eight sync commits from the jeff-dev branch rewrote the README into the platform framing and refreshed the hub docs and SDK snippets; the repository shows about 1,500 stars (recorded 2026-10-20), 83 forks and no open issues as of 2026-11-07, and the v1.2 checkpoints on Hugging Face still live under a jeff-legacy org that the README’s original links redirect to, with 3,053 downloads registered for the 0.8B as of 2026-10-20.

Star History Chart

Latency is the headline: about 22 ms per decision on an RTX PRO 6000, 28 ms on an M4 Max through MLX, 463 ms on a 32-thread CPU, against Jev’s published 114-212 ms per API call in its Doom runs (not measured on the same hardware); the v1.3 base and adapters measure 26.6 to 31.8 ms per decision on the same idle GPU as v1.2 (October 5 table). The self-run benchmarks covered 4,599 questions from five public suites (BBH, Financial PhraseBank, JudgeBench, RAGTruth, WinoGrande) plus JevBench’s public hard tier scored separately: Jeff-2B posted 83.1 overall against Jev’s published 83.0 and AutoJev-27B’s 84.9, winning Financial PhraseBank (96.3 versus 77.0) and RAGTruth (88.9 versus 77.3) while staying well below Jev on BBH (68.0 versus 94.3), JudgeBench, WinoGrande, and the JevBench hard tier (53.3 versus 73.3); the repositioned README dropped that table, and the hub’s comparison page now carries the current evidence, a dated 2026-10-17 head-to-head on 4,500 rows where the 0.8B adapters score 91.1 percent against Jev 1.14’s 81.8 and OpenAI’s Decisions API’s 72.6, with calibration error 0.016 against Jev’s 0.085 and OpenAI’s 0.115, and the page states plainly that it measures the adapters published at the time, not the current v1.4 generation. Zero-shot games back the generality claim: Jeff-0.8B took 57.0 of 98 Pac-Man pellets against 11.2 for random moves, and Doom kills matching a hand-coded rule bot. None of it has been checked by a third party: JevBench’s board carries the name collision forward as of the 2026-11-07 read under method v1.8, still listing the jgeuter rows sharing the Jeff name (Jeff-1.1-Medium and its siblings linking jgeuter’s Hugging Face repositories), and the Decision Index’s 2026-11-06 regeneration added a jeff-uncapped row, “jeff GLiFormer-large” at 6.21, rank 268 of 275, whose owner the board does not state, so every Jeff-named row on either board is either another author’s or unattributed, and this family still carries no board row of its own.

Strengths
#

  • The self-run comparisons argue against themselves in public: the v1.3 table printed reasoning-tier losses beside classification wins, and the new head-to-head keeps per-task rows with a dated-results caveat, the evidence discipline this category’s launch posts skipped.
  • Speed at 0.8B scale is real and local: 22 ms on one workstation GPU, 28 ms on a laptop-class Apple chip, no API and no waitlist.
  • The fine-tune path is demonstrated, not promised: a voice-navigation fine-tune moved held-out accuracy from 31.7% to 95.8% in under half an hour on one GPU.
  • Everything needed to re-run or retrain ships in the repository: training scripts, data-source licenses, leak filter, and a dashboard.

Cautions
#

  • The launch thread’s real-world reports split hard: one commenter measured 70% for Jeff against 94% for Jev on his own classification and called it unacceptable, and another called the 0.8B “completely useless” on job-ad classification, so the zero-shot numbers do not transfer to every domain.
  • The platform pivot broke its own citations: v1.3 and its fifteen adapters were retired unpublished on 2026-11-05, the Python and Node SDKs reached PyPI and npm the next day (jeff-sdk 1.4.1, 2026-11-06) while Rust, C#, and Java ship no registry commands, and the hosted API at api.jeffhub.ai makes the account the distribution path, so nothing about the November state is pinned the way the v1.3 README’s tables were.
  • “Trained at home” ran on an RTX PRO 6000 plus two DGX Sparks, which is workstation spending, not hobbyist hardware, and the thread said so.
  • Hard option ceilings by checkpoint: the Qwen models take up to 254 options (widened from the original A-Z coding in v1.1) and the Gemma E2B checkpoint still caps at 26; inputs beyond the trained 8,192 tokens are refused with a 422 unless JEFF_MAX_TOKENS raises the limit, and accuracy beyond that length is unmeasured.
  • Benchmark scores do not predict game play (the 2B plays the games worse than the 0.8B), forecasting-style questions score at random, and it is English and text only.
  • Every comparison to Jev is the author’s own run, now against the live Jev 1.14 and OpenAI Decisions APIs rather than published figures, and no third party has verified any of it; the hub page’s own dated-results caveat says the 2026-10-17 measurement predates the v1.4 adapters.

Pricing
#

Free and open at the core: MIT-licensed code (including AutoJev’s notice), the v1.4 base Apache-2.0, and the catalogue adapters under their own licences (the v1.3 split had sanctions and soc at CC BY-NC 4.0 and aml and trading-desk under their data sources’ terms, so commercial reuse needs an adapter-by-adapter check); a hosted API at api.jeffhub.ai now exists behind per-account keys, whose pricing is not stated on any fetched page. The cost is your own GPU locally and, if you retrain, the synthetic-data generation run.

Compared to
#

  • Jev: the closed original wins every reasoning-heavy benchmark and carries no weights; Jeff wins the classification slices and runs at 22 ms on your own machine, which is exactly the trade its README states.
  • Kev: the other Jev-API-compatible family; kev ships the pre-registered locked-test eval discipline and delta fine-tunes, Jeff ships the frankest self-run comparisons (per-task losses printed beside the wins) and the cheaper training recipe.
  • NanoJev: the other small self-run replica; NanoJev benchmarked games with zero community attention, Jeff arrived with a 573-point thread in its first week.

Bottom line
#

Recommended for engineers who want a small Jev-format student model they can specialize on their own data in an afternoon, and for studying how far 0.8-2B fine-tunes close on a frontier decision API. Not as a Jev replacement on reasoning-heavy workloads, and not for anyone who needs a number a second party has confirmed. The disagreeable claim I will defend: the v1.3 table matching Jev’s published overall while conceding every reasoning benchmark, and the new head-to-head flipping the overall on the author’s own task mix, are together the strongest evidence yet that the decision contract and the reasoning are separable, which is this category’s founding thesis made falsifiable on a workstation GPU.

Changes
#

  • 2026-09-29 - Created from the entrant scan after the 2026-09-28 Show HN thread cleared the bar (471 points, the frankest benchmark table in the wave).
  • 2026-10-06 - Recorded the v1.3 restructure (adapter-first base, fifteen adapters with GGUF exports, Jeff-Code’s paired-equal-quality-at-47-percent-less-time claim, the v1.2 checkpoints’ move to a jeff-legacy Hugging Face org) and refreshed traction (about 1,400 stars, 575-point thread, pushed 2026-10-05).
  • 2026-10-06 - Corrected the age claim (created 2026-09-28, eight days old, not three weeks).
  • 2026-10-07 - Added the firelex/jeff star history chart to the Status section.
  • 2026-10-15 - JevBench’s board, which listed no Jeff checkpoint at the last verification, now carries Jeff-1.0-Large at official rank ten of 162 (composite 68.6, capability 81.5 second), the family’s first third-party board reading; the note’s no-board claim corrected.
  • 2026-10-20 - Rotation re-verification, every cited URL re-fetched, with two corrections the current README forced: the Jeff-Code speedup reads 47 percent faster per task (0.68x the baseline, 32 percent less time), not 47 percent less task time as this note had it, and the hard 26-option ceiling belongs to the Gemma E2B checkpoint alone, the Qwen models having taken 254 options since v1.1 with an 8,192-token input cap since v1.3; also recorded Jeff-Code’s move into its own firelex/jeff-code repository on Pi with jeffhub.ai as the hub, the adapter licence split (sanctions and soc CC BY-NC 4.0), the board row’s move to official #12 of 184 (composite 68.6, capability third), the thread at 576 points, and the 0.8B checkpoint at 3,053 downloads under the jeff-legacy redirect.
  • 2026-10-25 - Rotation re-verification and a correction: the board row this note credited to the family on 2026-10-15 and 2026-10-20 is jgeuter’s unrelated Gemma-4-31B fine-tune sharing the Jeff name (Hugging Face jgeuter/Jeff-1.0-Large, CC BY-NC-4.0, created 2026-10-08, official rank 20 at 68.5 on the method-v1.7 board), so the family’s accuracy claim is back to zero third-party readings; every cited URL re-fetched (repository unchanged since 2026-10-07, the thread at 576 points, the jeff-legacy redirect holding with the 0.8B at 3,164 downloads, AutoJev unchanged, the README’s benchmark table intact).
  • 2026-10-27 - Out-of-rotation board re-fetch at the drift signal: the board’s 2026-10-27 read (revision v1.7.48, 232 ranked systems) moved the jgeuter Jeff-1.0-Large row from official rank 20 to 26 and added a second Jeff-named row, Jeff-1.1-Medium at official rank 13, whose Hugging Face owner check (jgeuter/Jeff-1.1-Medium) confirmed it is the same unrelated author’s, so the family still carries no board row of its own; the repository remains unchanged since 2026-10-07.
  • 2026-10-31 - Out-of-rotation board re-fetch at the drift signal: the board moved to method v1.8 (235 ranked systems), where the Jeff-name collision widened to three jgeuter rows (Jeff-1.1-Medium now official #11 at 75.11, Jeff-1.1-Large and Jeff-1.1-Qwen-4B added unranked, Jeff-1.0-Large no longer listed) plus two Jeffan rows by a third author, owners checked from the rows’ Hugging Face links, so the family still carries no board row of its own.
  • 2026-11-07 - Volatile re-verification that surfaced a repositioning and recorded it: the repository’s first default-branch movement since 2026-10-07 (eight jeff-dev sync commits on 2026-11-05 and 11-06) rewrote the README into a platform framing, Jeff as a library of decision functions (“a new software primitive”) with a 34-adapter namespaced catalogue, a v1.4 Apache-2.0 base released 2026-11-05 (v1.3 and its adapters retired unpublished per the repo’s bases.yaml), five SDKs none of which is published yet, a hosted API at api.jeffhub.ai, escalation on low confidence, decision graphs, and MCP tools, while the /v1/systemone Jev-format surface survives in the server code; the README’s benchmark table is gone and the hub’s comparison page now carries a dated 2026-10-17 head-to-head (Jeff 0.8B adapters 91.1 percent against Jev 1.14’s 81.8 and OpenAI Decisions’ 72.6 on 4,500 rows, ECE 0.016 against 0.085 and 0.115, measured on the adapters published at the time), so the thesis and cautions were rewritten; the Decision Index’s 2026-11-06 regeneration added a jeff-uncapped row (6.21, rank 268 of 275) whose owner the board does not state, the JevBench jgeuter rows re-read unchanged, forks refreshed to 83 with no open issues, and the pushed date moved to 2026-11-06.
  • 2026-11-08 - Out-of-rotation correction at the registry signal: the Python and Node SDKs are published after all, jeff-sdk 1.4.1 on PyPI (uploaded 2026-11-06T09:51Z) and npm (09:53Z), both missed by the 11-07 verification, and the rewritten README’s quickstart installs from the registries (pip install jeff-sdk, with Apple silicon behind a jeff-runtime[mac] extra and no MLX mention left), while Rust, C#, and Java still carry no registry commands; the repository is unchanged (pushed 2026-11-06).

See also
#

  • Jev - the closed original whose request format Jeff speaks and whose published numbers anchor the benchmark table
  • Kev - the other Jev-API-compatible open family, with the stricter eval discipline
  • JevBench - the third-party scoreboard whose hard tier appears in Jeff’s table as a separately-scored column
  • NanoJev - the other small self-run replica, games instead of public benchmarks
  • Hybrid Execution Feature Matrix - the category comparison this note joins

References
#