BenchLM is a solo-maintained composite leaderboard that blends public benchmark evidence and other published leaderboards into one BenchAlign score per model, and labels every ranked position by how much evidence stands behind it. **It is the only member of this category whose product is not a measurement but a provenance-labeled reading of everyone else's measurements.** ## What it is A website, a downloadable dataset, and a query API covering 889 model profiles and 625 canonical benchmark pages, with 218 ranked entries (83 Supported, 135 Estimated) across 8 weighted categories (as of 2026-10-10). The BenchAlign score (v5.8) starts from independent external leaderboards, benchmark composites and preference boards, each weighted by uncertainty, with preference leaderboards capped at a quarter of the external part and direct benchmark results adding a correction of at most 3 points. Missing tests stay missing, generated rows never enter the boards, and the confidence page cuts the 218 into four levels: 27 Full, 60 Good, 67 Limited, 64 Estimated, with sourced-versus-generated row counts per model. Data access is unusually complete for the genre: eight static JSON downloads with a SHA-256 manifest and content versions, plus a free Data account exposing REST and MCP endpoints under a CC BY-NC 4.0 license for BenchLM's own outputs. It is built and maintained by one person publishing as Glevd, with an email contact and an X handle as the only disclosed identities. ## Status Active and fresh: the dataset, leaderboard, and comparison files were last updated 2026-10-10, pricing and speed files a day or two before, and the scoring policy iterates in dated steps (BenchAlign v5.8 with a September 30, 2026 retrospective). The community footprint is close to zero: an HN search finds no BenchLM story of its own as of 2026-10-11, and the one independent sighting is a June 2026 Show HN (Models Pie, 4 points) that names BenchLM as one of its three data sources. No company, funding, or team is disclosed, so treat it as a one-person project with institutional-scale coverage and no institutional ballast. **Treat it as a working cross-check layer, not a source of record.** ## Strengths - The evidence labels are the genre's best: a reader can see, per rank, whether the position rests on exact sources or on the estimator's blend, and the confidence page counts sourced versus generated rows per model. - The methodology publishes its own failures: three retrospective checks of 95th-percentile error worsened, and the page says so rather than burying it, which no larger rival in this category does. - Agent-facing data without a paywall to start: hashed static JSON for offline use, plus REST and MCP on a free tier. ## Cautions - It runs nothing itself, so every number is another board's measurement twice removed, and the errors inherit: a distorted input leaderboard distorts the blend. - The headline breadth overstates the depth: 889 tracked profiles reduce to 218 ranked entries, and only 83 carry exact-source support. - The Estimated intervals carry the operator's own warning that they do not establish confidence in a rank or future performance, and the category intervals are unvalidated after the latest prior changes. - CC BY-NC 4.0 blocks commercial republication without a separate agreement, a sharper restriction than this category's other open datasets. - One pseudonymous operator, no repository, no funding disclosure, and no second pair of eyes on the pipeline. ## Pricing The site is free to read. A free Data account covers REST and MCP queries; Pro and Research plans add saved ranking history and publisher snapshots, with no public prices. Commercial display or redistribution of BenchLM-owned outputs needs a separate agreement, so no stated price exists and no price history applies. ## Compared to - [LLM Stats](../llm-stats/index.md): the other composite aggregator; choose LLM Stats for its API maturity and MCP tooling, BenchLM when the question is how much evidence sits behind a rank. - [Benchmark Radar](../benchmark-radar/index.md): the catalog of benchmarks themselves; BenchLM consumes that kind of evidence to rank models instead. - [Artificial Analysis](../artificial-analysis/index.md): first-party measurement with published weights; BenchLM is a second opinion on that measurement, not a replacement. ## Bottom line **Recommended as an evidence-reading layer over other people's leaderboards, and for agents that want hashed, dated JSON; not as a primary source for any deployment decision, because nothing in it was measured by the site that publishes it.** My disagreeable claim: the most valuable writing on BenchLM is not the leaderboard but the failed-checks paragraph in its methodology, because publishing your own broken validation is exactly the artifact every larger player in this category declines to ship. ## Changes - 2026-10-11 - Created. ## See also - [LLM Stats](../llm-stats/index.md) - the incumbent composite aggregator BenchLM re-ranks against - [Benchmark Radar](../benchmark-radar/index.md) - the benchmark-discovery layer whose catalogs feed composites like this one - [Artificial Analysis](../artificial-analysis/index.md) - the first-party measurement layer BenchLM consumes as an input - [Trackers and Leaderboards Feature Matrix](../trackers-and-leaderboards-feature-matrix/index.md) - the category comparison this note joins - [Model Selection for Coding Tasks](../../model-selection-for-coding-tasks/index.md) - where a composite rank lands when a model choice gets made ## References - https://benchlm.ai/ - homepage: 889 tracked, 218 ranked, 83 verified, BenchAlign leader Claude Opus 5.5 at 86.43, last verified 2026-10-10 (fetched 200, 2026-10-11) - https://benchlm.ai/about - operator identity (Glevd), data sources, Supported/Estimated policy, what the site does not track (fetched 200, 2026-10-11) - https://benchlm.ai/methodology - BenchAlign v5.8 rules, the preference cap and peer constraints, and the three failed validation checks (fetched 200, 2026-10-11) - https://benchlm.ai/data - CC BY-NC 4.0 terms, free Data account, REST and MCP endpoints, eight static JSON downloads with a SHA-256 manifest (fetched 200, 2026-10-11) - https://benchlm.ai/benchmark-confidence - the confidence distribution (27 Full, 60 Good, 67 Limited, 64 Estimated) and sourced-versus-generated coverage (fetched 200, 2026-10-11) - https://hn.algolia.com/api/v1/search?query=benchlm&tags=story - no BenchLM story of its own; the June 2026 Models Pie Show HN names BenchLM as a data source (fetched 200, 2026-10-11)