Holdout Systems · v0 · 2026-07-20

Know what an AI agent will actually do — before it touches your systems

Everyone selling you an agent has a demo. Nobody hands you an audit. Holdout builds a working copy of your system from data you already have, lets the agent do the job there, and then checks every record it touched against what should have happened — with points off for damage. It keeps re-running after you deploy, so a quiet model update can't silently break what worked last month.

Feature 1 · Env inference

A working copy of your system, built from logs and exports you already have — and proven faithful by replaying a day of real traffic against it.

See the reconstruction results →
Feature 2 · Agent testing

Agents work queues of real records — real tickets, real invoices, real CRM messes — and are scored item by item, with damage subtracted.

See what the agents really did →
For the detail-minded: How the scoring works — plus the five practice tasks we built the harness on.