Proves · private beta

Stop shipping agent changes on vibes.

Verdictz tests a change to your agents (a model, a prompt, a skill, a whole harness) on the same tasks, side by side, and hands you a one-page verdict on success and cost. With an interval you can defend, and the checks that say why you can trust it.

Verify a change Read a sample report

TestsClaude CodeCodexAGENTS.mdskillsmodelsHarbor suites

verdictz.recurz.ai/verdicts/v_8Q2K/reportExample data

rust-backend v3 · Claude Code · Opus 5.5 · confirmation on 100 fresh tasks

Does a test-first AGENTS.md help on Rust backend tasks?

Kept+9.0points of pass rate

The test-first AGENTS.md raised the pass rate from 48% to 57% on tasks it had never seen. The 95% interval runs from +3 to +15 points, wholly above zero.

Measure	Empty setup	Test-first	Change, 95% interval	Reading
Pass rate	48%	57%	+9.0 pts, +3 to +15	Kept
Cost per solved task	$1.42	$1.31	−8%, −17% to +2%	Inconclusive
Time per task	6m 40s	7m 05s	+6%, −3% to +15%	Inconclusive

21 flipped to pass12 flipped to fail67 tiesbaseline: pool of 4 runs, reusedspent $79 of an $83 quote

Why you can trust it

6 of 6 pass

Plan frozen before the first attemptSample size, tasks and the bar were hashed (6c89…fafc) before any spend.

Fresh tasks decided itNone of the 100 tasks was seen while the change was being screened.

Grader checked firstIt rejected 34 of 34 deliberately broken solutions.

Validity scan13 attempts flagged, spread across both arms. The verdict holds without them.

Second reviewerThe grader agrees with you as often as another person does.

Read once, at the planned sizeNo peeking. Stopping early was allowed only by the plan’s own rule.

New in Verdictz

Eight ways a verdict got harder to argue with

Shipped October 2026

01New

01 · Side by side

See why a task flipped, not just that it did.

Pick any task that changed and Verdictz opens both arms’ attempts on one step axis, with the tests passing over time and the final patches diffed. Then you call which was better, and that call feeds the grader check.

recurz verdict trace v_8Q2K ledger-order-rounding

Test-first vs empty › ledger-order-rounding › repeat 2

The candidate ran the tests first and fixed the right function

failed → passed

empty setup · src/ledger.rs

- amount.round()

+ (amount * 100.0).round() / 100.0

test-first · src/ledger.rs

- amount.round()

+ amount.round_dp_with_strategy(2,

+ MidpointNearestEven)

02 · Validity scans

Every attempt is read for cheating and broken runs.

A model reads each attempt for edited tests, time limits, sandbox errors and early quits, and quotes the step it found. Then the verdict is computed again without the flagged attempts, so you can see whether they change the answer.

Flags stay flags until a person confirms them. Nothing is dropped silently.

recurz eval scan v_8Q2K

The gain holds without the 13 flagged attempts

Still kept

Scanner	Empty setup	Test-first
Edited a test or protected file	1	1
Hit the time limit	3	3
Sandbox or tool error	2	1
Gave up before the tests passed	1	1
Flagged attempts	7	6

All attempts Without flagged

Tests editedempty setup · settle-rounding · step 22

- assert_eq!(settle(2.5), 2.0);

+ assert_eq!(settle(2.5), 3.0);

03 · Failure flow

Know which failures the change fixed.

Each baseline attempt is paired with the same task and repeat in the candidate arm, so the table reads as a true before and after. Failure modes come from your review queue; suggested ones stay dashed until you accept them.

recurz verdict show v_8Q2K --failures

Of the baseline’s 31 wrong fixes, 17 passed with the change

105 paired attempts per arm on the 35-task screen. Darker cells hold more attempts.

Empty setup ended as	Passed	Wrong fix	Stopped early	Timed out	Total
Passed	39	4	2	1	46
Wrong fix	17	10	3	1	31
Stopped early	1	6	9	2	18
Timed out	1	2	2	5	10
With test-first	58	22	16	9	105

Modesfixed the wrong functionrounded the display onlysuggested: stopped at first green test

04 · Second reviewer

Hold the grader to a human standard, not a perfect one.

Give a label set a second reviewer on an overlapping slice. If two careful people disagree 15% of the time, a grader can’t be held to better, and you’ll see both kinds of error it makes: failures it missed and good work it failed.

recurz eval grader list rust-backend@v3

The grader agrees with you about as often as a second person does

You vs a second reviewer · 40 attempts

	They pass	They fail
You pass	17	3
You fail	3	17

85% agree 72 to 94%

You vs the grader · 150 attempts

	Grader pass	Grader fail
You pass	62	9
You fail	12	67

86% agree 80 to 91% · 12 missed failures · 9 good fixes failed

05 · Leaderboards

A leaderboard that admits a tie.

Every setup on an eval version, with its pass rate interval, cost per task, and the range of places it could land given that interval. Share it publicly, or keep it to your team.

recurz eval compare rust-backend

Test-first leads, but could be second

rust-backend v3 · Opus 5.5 · 35 tasks, 3 repeats each

Example data

Setup	Rate	Could place	Cost / task
Test-first AGENTS.md	71%	1st–2nd	$1.18
rust-review skill	66%	1st–3rd	$1.53
Read-search hints	60%	2nd–4th	$1.21
Memory on	57%	3rd–5th	$1.40
Empty setup (baseline)	52%	4th–5th	$1.09

Axis 40% to 85%. Hollow is the baseline.rust-review costs 30% more for no clear gain

What a verdict costs

Kept on fresh tasks, for about $83.

Every study is quoted before it starts, with the chance it has of finding a real gain and the chance it calls a dud a win. We measured both for every design, on our own recorded attempts, and made the best one the default.

Confirmation design	New spend	Calls a dud kept	Keeps a real +15 point gain
13 live tasks, 3 runs vs a pool of 4	$32	3.8%	48%	quick screen
35 tasks, 3 runs vs a pool of 4	$87	3.2%	81%
100 fresh tasks, 1 run vs a pool of 4	$83	3.0%	94%	Default
60 fresh tasks, 3 runs vs a pool of 4	$149	2.9%	95%
100 fresh tasks, 1 run each (the old way)	$166	2.1%	75%

Simulated with the engine’s own statistics: 2,000 confirmations per row, drawn from 1,540 recorded attempts on Rust backend tasks. Spend is provider cost at list price for the new attempts; the pool of 4 baseline runs is paid once and reused until the model changes. Kept means the 95% interval sits above zero on tasks the change never saw.

Before you pay

Price and power on one screen

Set the smallest change worth acting on. The quote says what the study can and can’t see.

While it runs

Stops early, by its own rule

Only at looks the plan froze, and only if the simulator shows the rule keeps false wins under 5%.

Every model

Only tasks that can move

Live task sets skip what a model always or never solves, so you pay for the tasks that discriminate.

Bring your own tasks

Any Harbor suite, or your own merged PRs.

Import a public suite or turn your team’s fixed bugs into tasks. Each one has to prove it can tell right from wrong before it can decide anything.

1 · BringA suite or a PR listA folder, a tarball, Hugging Face, the Harbor registry, or a list of your merged pull requests.
2 · CheckFour checks per taskIt builds. The reference fix passes. The untouched code fails. It isn’t a duplicate.
3 · ApproveA draft versionTasks that pass land in a draft you approve on the Tasks page. The answers stay hidden from the agent.
4 · StudyPick it in New studyVersions are frozen, so a verdict on v3 always means the same 35 tasks.

From the CLI

Everything on the page is one command away.

The same verdicts from recurz, the one CLI for every Recurz product. Gate a deploy on --require kept.

# bring a Harbor suite into a draft version
recurz eval import hf:org/suite --format harbor --into rust-backend

# price and power before you pay
recurz eval simulate rust-backend@v3 --model opus-5.5 --design pool-one

# run it, stopping early when the answer is clear
recurz eval run rust-backend --baseline empty --candidate test-first --stop-early

# fail CI unless the change is kept
recurz verdict show v_8Q2K --require kept

# the one-page report, as a file anyone can open
recurz verdict export v_8Q2K --format html -o report.html

What buyers ask first

Can we use it now?

Yes, in private beta. Task sets, studies, fresh-task confirmation, reports, scans and leaderboards run today. You bring the change; we run the first verification with you.

What does a verdict cost?

It is priced from the number of tasks, the arms and the model’s list price, quoted before it starts, with a cap on every attempt. A confirmation on 100 fresh tasks against a reused baseline pool is about $83 at list prices.

What if the answer is inconclusive?

Then we say so. Inconclusive means the interval crosses zero, which is not the same as no effect. The report says how big a change the study could have seen.

Do you need our code?

Not to start: public suites and our own task sets work on day one. To test on your own work, tasks come from your merged pull requests, and you approve the list before anything runs.

05 · Proves

Did the change help?

Two products answer it. Each one is useful alone, and each hands its work to the next verb in the loop.

VerdictzStatistically sound evals: did the change actually help?You are here
ProofzCatches sandbox escapes and signs a receipt for every run.Coming soon

Private beta

Ship agent changes with proof.

Get started Email the founders See the API

Available now: paired verdicts with 95% intervals, fresh-task confirmation, one-page reports, side by side, validity scans, second reviewers, leaderboards with rank ranges, Harbor import and the recurz CLI. Coming: verdicts from your team’s live sessions, and on pull requests.

Every number on this page is example data unless it says otherwise.