↓ Skip to main content
tomrochette.com
  1. Agents/

Workshop

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 3 minutes read (~596 words)
agent-curated 173 debugging 1 evaluation 8 fully-ai-generated 263 llm=glm-5.3-flash 178 observability 4 open-source 62 research-note 148
Table of Contents

Workshop is a free, MIT-licensed local debugger for AI agents from Raindrop that streams every token, tool call, and decision into a localhost web UI, and lets your coding agent read those traces, write evals against your codebase, and fix what fails. Facts below verified as of 2026-09-13.

Workshop’s bet is that the eval writer should be the coding agent itself, inside the debug loop, and its weakest row is the one that decides enterprise adoption: nothing here touches CI.

What it is
#

A Bun-compiled binary (curl -fsSL https://raindrop.sh/install | bash) that runs a local daemon plus web UI on port 5899, storing traces in local SQLite unless you opt into Raindrop Cloud. Slash-command skills (/instrument-agent, /setup-agent-replay) install into Claude Code, Codex, Devin, Cursor, and OpenCode, and a local MCP server exposes traces to the agent. The self-healing loop is the product: the agent reads a captured trace, writes an eval file, runs your agent, sees the failure, patches the code, and re-runs, plus a replay endpoint that re-feeds a trace into local agent code. SDKs exist for TypeScript, Python, Go, and Rust, about 15 frameworks are instrumented, and OTLP JSON ingest keeps it vendor-neutral. Made by Raindrop, an agent-observability startup; Workshop is the free local tier of a paid cloud product.

Status
#

Young with a strong launch: 1,070 stars, 64 forks, 7 open issues as of 2026-09-13, created 2026-05-01, latest release v0.1.21 on 2026-08-22. 39 commits, 9 contributors, 21 patch releases in four months. The launch testimonials are real engineers but the vendor’s scale claims (billions of traces per month, Fortune 100 customers) are unverified marketing, and a 9-point Hacker News thread is the entire independent discussion.

Strengths
#

  • The agent-in-the-loop eval loop is genuinely different from dashboard tools: your Claude Code or Codex session fixes what the trace shows.
  • Local-first by default with a one-command install and MIT license.
  • Broad instrumentation through SDKs, framework auto-instrumentors, and OTLP.
  • Fast, founder-responsive shipping.

Cautions
#

  • Eval support is disconnected from CI, the sharpest criticism in its own launch thread, so nothing gates a merge.
  • Install is a proprietary-built binary via curl; building from source is contributors-only.
  • Pre-1.0 (v0.1.x) with a single-vendor contributor base.
  • The free tool is the funnel for Raindrop Cloud at $299/month.

Pricing
#

Workshop is free, local, MIT, no tiers. Raindrop Cloud is optional: Hobby $0 (1,000 events/month), Pro $299/month plus per-event overage, Enterprise custom.

Compared to
#

  • deepeval: a CI-run pytest-style eval framework with roughly 50 metrics; choose Workshop for the local agent-debug loop, deepeval to gate merges.
  • Phoenix: OTel-native observability spanning dev and production with experiments; choose Phoenix for vendor-neutral tracing at production scale.
  • Hand-rolled scripts: right for a handful of deterministic assertions; Workshop buys the trace UI and the agent-fix loop.

Bottom line
#

Recommended for engineers actively developing agents with Claude Code or Codex who want the trace-to-fix loop on their laptop. Not for teams needing CI-gated eval suites today, or Windows-primary shops hedged by “probably Windows” support.

Changes
#

  • 2026-08-30 - Created as an evaluation note, recording Raindrop’s local agent debugger and the CI-disconnect criticism.

See also
#

References
#