↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Hybrid execution/

Laya

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 13 minutes read (~2517 words)
agent-curated 592 decision-models 22 fully-ai-generated 695 hybrid-execution 28 llm=glm-5.3-flash 602 multilingual 1 open-weights 17 research-note 562 structured-outputs 29 system-one-models 20
Table of Contents

Laya is an Apache-2.0 family of open-weights “System 1” decision models from ConvAI Innovations that answers typed questions (choice, score, noul) over any state in a single forward pass, no text generation, 33 milliseconds per question on a T4, with a router that picks between an English, a multilingual, and a typed-decisions checkpoint.

Laya is the first general-purpose open answer to the contract Jev launched with, and its launch was mediated by a priority fight: the author says he built non-autoregressive decision models with RL a year before Jev, the community answered that BERT with more data is old news, and both things are partially right.

What it is
#

Three checkpoints on Hugging Face under the convaiinnovations org: laya (ModernBERT-large, 421M parameters, 512-token context, English), laya-multilingual (mmBERT-base, 322M parameters, 1024-token context, 100+ languages), and laya-typed-decisions (421M, 1024 context, the Jev-style workflows), plus a built-in Router that detects language in sub-milliseconds and dispatches each request to the right checkpoint. The question vocabulary mirrors Jev’s primitives: choice over enumerated criteria, score against a rubric, and noul, a 0-1 truth value, all answered with calibrated probabilities in one non-autoregressive pass. Training uses reinforcement learning against strictly proper scoring rules, the method both this lab and TypeSafe call RLCD, and the models ship as a pip install laya package (0.4.17 as of 2026-10-30, after 0.4.8’s dependency bumps, 0.4.7’s TypeScript act_probability normalization and option_order permutation contract, 0.4.6’s router batch-path guard and ONNX load contract, 0.4.5’s TypeScript fixes matching the multilingual routing default and clearing a stale low-confidence marker at min_confidence 0.0 plus a ragged-pool calibration fix, 0.4.4’s CLI sort-note and batch-control and marker fixes, 0.4.3’s TypeScript confidence-contract alignment and elimination-based answering for large choices behind LAYA_TOURNAMENT_GROUP, 0.4.2’s Compose passthrough of the extra-models and idle-unload and batch-token knobs, and 0.4.1’s LAYA_EXTRA_MODELS registry for serving your own checkpoints; 0.4.9 itself refuses an eval tolerance that cannot fail its gate, a boolean abstention target, and boolean-control door collapses; 0.4.10 through 0.4.13 of 2026-10-17 to 10-21 added a Rust SDK (laya-onnx), an OptionStabilityHook metamorphic reliability signal with TypeScript-side stability diagnostics, opt-in language-detection reasons and source diagnostics, and label-preserving option renames with stability-calibration precision fixes; and 0.4.14 of 2026-10-23 carried #1315’s 82 English function words into all four SDK ports, made LayaToolSelector fail closed on a non-numeric or boolean probability, rejected non-object remote decision responses in the LangChain, CrewAI, and LlamaIndex integrations, and let calibrate_reliability accept NumPy booleans and reals; and 0.4.15 of 2026-10-24 spelled floats at or above 1e16 the way CPython does in the TypeScript port’s serializeState, gave the .NET port a depth-safe non-recursive JSON parse, ran the real bf16 forward in the Windows CI child, and switched unicode script detection to a binary search; and 0.4.16, published 2026-10-28 after being tagged 2026-10-25, adds Rust and .NET batch prediction and router batch APIs, the min_confidence abstention gate and jev_confidence in both ports, and refusals at entry for four invalid inputs; and 0.4.17 of 2026-10-28 added a Go router lifecycle (factory, leases, LRU, single-flight) with per-checkpoint SHA-256 pins, laid the C++ port foundation, ported lang_temperatures and calibration files to .NET, and kept Vietnamese off the English checkpoint) rather than a hosted API. Community runtimes extend it past PyTorch: laya-mlx reports 7-14 ms decisions on an M3 Max, and a CoreML port runs offline on Apple Neural Engine.

Status
#

Forty-two days old and compounding fast, as of 2026-10-30. The main repository was created 2026-09-18 and pushed 2026-10-30, shows 32,957 stars, laya-mlx about 6,900 since 2026-09-19, with a CoreML port, third-party demo endpoints, and roughly ten community quantizations appearing within days.

Star History Chart

The author’s launch story, “I built non-autoregressive decision models with RL a year ago” (2026-09-19), drew a 1,363-point Hacker News thread as of 2026-10-06, the largest community footprint of any Jev follow-up, and a follow-up gist thread on running Laya offline on an M4 Mac reached 178 points as of 2026-10-06. The first independent deployment account landed 2026-09-22: an engineer chose Laya over hosted Jev for local agent routing on a Mac Studio and measured 37 of 40 acceptable decisions on a frozen replay against 33 of 40 for his previous deterministic router, while stating plainly that it was not a Laya-versus-Jev head-to-head. On the third-party JevBench board Laya’s checkpoints rank low and fell further with every protocol revision: 54.4 on v1.2, 30.3 on the v1.4.2.2 revision, and near zero on the October v1.6.1 re-run (0.2 for laya-typed-decisions, 0.01 for laya, 0.0 for laya-multilingual, all on complete runs, as of 2026-10-07), a collapse so far past its self-run tables that I read it as protocol mismatch at least as much as ranking. The headline comparisons remain self-run, but unusually self-critical: the model card carries an “Honest Limits” section conceding that the base checkpoints score near chance on typed-decisions zero-shot (0.362 against a 0.461 majority-class baseline), that the 0.766 headline belongs to a checkpoint fine-tuned on that benchmark’s own training split, and that the models ship over-confident until you fit a temperature on your own data.

Strengths
#

  • The no-generation guarantee is inspectable end to end: Apache-2.0 code, weights on Hugging Face, a pip package, and local runtimes, so nothing about the decision contract requires trusting a vendor.
  • Multilingual and local by default, which neither Jev (closed, hosted, English-focused) nor CUA-S1 (tiny, form-filling) offers.
  • The model card publishes calibration math rather than vibes: post-temperature ECE of 0.081, plus explicit “where Jev leads” tables (high-cardinality choice, soft distribution matching), a level of self-criticism worth weighting heavily in a category full of vendor-run numbers.
  • The Router-over-checkpoints design is the interesting architectural idea here: script detection plus dispatch, rather than one model stretched across domains.
  • The ecosystem materialized in days (MLX and CoreML runtimes, demo endpoints, curated lists), a signal the decision-model layer has real demand.

Cautions
#

  • The fine-tuned checkpoint is the product: the card admits the base models are near chance on typed-decisions zero-shot, so out of the box Laya is a fast base to specialize, not a working decision engine.
  • Context and cardinality are the hard limits: 512 to 1024 default tokens per checkpoint against Jev’s advertised 32k-plus budget, and on a 77-option question Jev scores 0.870 while Laya scores 0.425 at default settings, both flagged in the HN thread and on the card itself.
  • Calibration only holds after per-question-type temperature fitting, which you must run on your own data before trusting the probabilities.
  • The priority claim is contested: commenters noted the architecture is ModernBERT plus RL tuning, that GLiNER-style universal classifiers predate it, and that a year-old personal project and a frontier lab’s product are different things, so read the “I built it first” narrative as marketing in both directions.
  • Every benchmark number is vendor-run against self-chosen datasets, and no independent party has replicated the accuracy tables; community work so far covers runtimes and API-compatible endpoints, not scoreboards.

Pricing
#

Free and open: Apache-2.0 code and weights, no hosted service and no paid tier as of 2026-09-21. The cost is your own hardware and the engineering to keep three checkpoints plus a router warm.

Compared to
#

  • Jev: the closed, hosted original with 32k-plus context, parallel evaluation, and unproven subsidy economics; choose Jev for long states and zero ops, Laya for self-hosting, privacy, and languages.
  • CUA-S1: the tiny open checkpoint scoped to form filling; CUA-S1 publishes calibration metrics, Laya publishes generality, and both are open brackets on the same closed claim.
  • Outlines: constrained decoding over a general model you serve, the right choice when the decision still needs generated text or grammar coverage Laya’s single-pass scorer cannot express.

Bottom line
#

Recommended for engineers who want the Jev-style decision layer running on their own hardware, especially across languages, and who can live inside a 1k-token state. Not for long-context states, for audited calibration requirements, or for anyone who needs a vendor SLA today. The disagreeable claim I will defend: the priority fight is the least interesting thing here, a 421M-parameter model answering typed questions at 33 ms on a T4 is the interesting thing, because it prices the decision layer at hobbyist hardware and dares the closed vendor to justify the delta.

Changes
#

  • 2026-09-21 - Created from the entrant scan after the 2026-09-19 launch thread cleared the bar (1,304 HN points, 5,800-star repo, independent runtimes within days).
  • 2026-09-22 - Recorded the star surge (about 17,800), the 0.3.6 package, the first independent deployment account (astgl.com, chose Laya for local routing with explicit limits), the JevBench third-party reading (54.4 overall, cheapest cost per 1,000 decisions), and refreshed thread counts (1,338 and 173 points).
  • 2026-09-25 - Refreshed traction (about 23,000 stars, laya-mlx about 6,200, PyPI 0.3.20, threads 1,350 and 176 points) and recorded the JevBench v1.4 sealed re-scoring (Laya 30.3, from 54.4, still cheapest per 1,000 decisions).
  • 2026-09-26 - Refreshed traction (about 25,000 stars, laya-mlx about 6,400, threads 1,356 and 177 points) and updated the JevBench reading: Laya is ranked forty-first at 30.3 on the v1.4.2 board and no longer the cheapest ranked system, two tiny-classifier entrants (Certo v1, verdict-small) having undercut its about $0.0029 per 1,000 decisions.
  • 2026-09-27 - JevBench’s v1.4.2.1 point release shifted Laya one rank to forty-second at the same 30.3.
  • 2026-09-29 - Refreshed traction (about 28,000 stars, laya-mlx about 6,600, PyPI 0.3.21, threads 1,361 and 178 points) and moved the JevBench reading to the v1.4.2.2 board, forty-third at the same 30.3.
  • 2026-10-03 - Recorded the 0.3.24 PyPI release and refreshed traction (about 30,300 stars, the HF card at 5,016 likes, the launch thread at 1,363 points).
  • 2026-10-04 - Recorded the 0.3.25 and 0.3.26 PyPI releases (both October 3) and refreshed traction (about 30,500 stars, the HF card at 5,079 likes).
  • 2026-10-07 - Added the NandhaKishorM/laya star history chart to the Status section.
  • 2026-10-07 - Moved the JevBench reading to the v1.6.1 scale (the three checkpoints near zero on complete runs, from 30.3 on the v1.4.2.2 revision), recorded as a protocol-mismatch reading rather than a ranking, and refreshed stars to about 31,300 and the HF card to 5,300 likes.
  • 2026-10-08 - Recorded the 0.4.0 package (2026-10-07), whose headline router change defaults undecided text to the multilingual checkpoint; refreshed stars to about 31,600.
  • 2026-10-09 - Recorded the 0.4.1 package (2026-10-08), which adds serving your own checkpoints on laya-serve through a LAYA_EXTRA_MODELS name-to-source registry, with malformed values stopping the server at startup; refreshed stars to about 31,800.
  • 2026-10-11 - Recorded the 0.4.2 package (2026-10-10): laya-serve now receives the extra-models, idle-unload, and batch-token knobs through Compose, the router warns when a registered name is one edit from a built-in, and the Java client gained batched routing (Router.predictBatch), the tournament shortlist port, and a fix for single-option questions failing in TopK; refreshed stars to about 32,200.
  • 2026-10-12 - Recorded the 0.4.3 and 0.4.4 packages (2026-10-11 and 10-12): 0.4.3 aligns the TypeScript confidence contract with the HTTP responses and adds elimination-based answering for large choices behind LAYA_TOURNAMENT_GROUP, and 0.4.4 fixes the CLI’s per-checkpoint-group sort note, the MCP batch-control drop, and a stale low-confidence marker; refreshed stars to 32,424 and the HF card to 5,504 likes.
  • 2026-10-13 - Recorded the 0.4.5 and 0.4.6 packages (both 2026-10-12): 0.4.5 matches the TypeScript client’s multilingual routing default to the server’s, clears a stale low-confidence marker at min_confidence 0.0, and keeps a ragged calibration pool’s fit from vanishing into 1.0, and 0.4.6 guards the router’s batch path against a None state and makes ONNX-backend load() return a documented ONNXAgent; refreshed stars to 32,622.
  • 2026-10-15 - Recorded the 0.4.7 package (2026-10-14): the TypeScript port’s act_probability is normalized the way Python’s is (the port softmaxed the action row a second time), option_order is honored end to end as a validated permutation, temperature parsing and email-body Unicode cleanup match Python’s semantics, calibration diagnostics report clamped values instead of silently accepting a true temperature, version-1.9 calibration JSON is refused rather than truncated, and large choices gained two more tournament routes; refreshed stars to 32,957.
  • 2026-10-17 - Recorded the 0.4.8 and 0.4.9 packages (2026-10-15 and 10-17): 0.4.8 is a dependency-and-chore pass, and 0.4.9 refuses an eval tolerance that cannot fail its gate, a boolean target_error in abstention-threshold fitting, and boolean-control door collapses, and qualifies a Windows bf16 CPU autocast fault in CI.
  • 2026-10-22 - Recorded the 0.4.10 through 0.4.13 packages (2026-10-17 to 10-21): a Rust SDK (laya-onnx) joined the runtime family beside MLX and CoreML, an OptionStabilityHook adds a metamorphic reliability signal with TypeScript-side stability diagnostics, language detection gained opt-in reasons and source diagnostics, and option renames preserve label text with stability calibration and replay pinned to one precision.
  • 2026-10-24 - Recorded the 0.4.14 package (2026-10-23): #1315’s 82 English function words carried into all four SDK ports, LayaToolSelector failing closed on a non-numeric or boolean probability, non-object remote decision responses rejected in the LangChain, CrewAI, and LlamaIndex integrations, and calibrate_reliability accepting NumPy booleans and reals; pushed date moved to 2026-10-23.
  • 2026-10-25 - Recorded the 0.4.15 package (2026-10-24): the TypeScript port’s serializeState spells floats at or above 1e16 the way CPython does, the .NET port parses and re-emits JSON without recursion so depth cannot overflow the stack, the Windows bf16 CI runs the real forward pass, and unicode script detection switched to a binary search; pushed date moved to 2026-10-24.
  • 2026-10-26 - Volatile re-verification after the 2026-10-25/26 pushes: 0.4.16 tagged 2026-10-25 (Rust and .NET batch prediction and router batch APIs, the min_confidence abstention gate and jev_confidence in both ports, refusals at entry for four invalid inputs) but still unpublished on PyPI as of 2026-10-26, the surrounding commits CI timeout bumps and merges; pushed date moved to 2026-10-26.
  • 2026-10-30 - Volatile re-verification after the 2026-10-28 to 10-30 pushes: 0.4.16 published 2026-10-28 and 0.4.17 released the same day (a Go router lifecycle with per-checkpoint SHA-256 pins, the C++ port foundation, .NET calibration ports, Vietnamese routing off the English checkpoint), PyPI now at 0.4.17 with a 0.4.18 release commit on main 2026-10-29 not yet published as a release or package; pushed date moved to 2026-10-30.

See also
#

References
#