CI specimen / deterministic replay

Model change assay

Pair two version-pinned runs. Expose the uncertainty. Decide whether the evidence permits a merge. This Pages build replays embedded data; it does not call the installed local service.

ASSAY NO. A-0012 100 paired samples

01 protocol

Prepare the run

Choose a controlled fixture, comparison pair, and allowed regression before starting the assay.

The default tool-selection comparison is generated by the Python reference implementation. Alternate controls are labeled, deterministic interaction fixtures.

Evaluation suite

Baseline

Candidate

Versioned scorer

Allowed regression 0.00

strict0.20 lenient

Dataset pinloading

Required scorer pinloading

Candidate scorer pinloading

02 process

Evidence pipeline

Awaiting a run. No verdict has been issued.

1Pindataset + scorer
2Replaypaired samples
3Scoresame contract
4Infer95% paired CI
5Gatemerge verdict

inoculateseparateread

Baseline mean —

model-v1

Candidate mean —

prompt-v2

Paired delta —

——

95% confidence interval

Merge decision NOT RUN

Run the protocol to produce a verdict.

03 uncertainty

Score and confidence evolution

baseline candidate 95% CI

Start or step the protocol to develop the trace.

The interval uses the implementation’s paired normal approximation. It is not a claim of universal statistical validity.

04 specimens

Paired sample plate

Each well is one matched example. Select a well to inspect its input, expected output, prediction, and exact score transition.