Validate and Operate the Service

The capstone is ready for evaluation when it can explain both usable and unusable results. A baseline success is necessary, but it does not test whether the service distinguishes insufficient evidence, poor observed quality, missing provenance, and interrupted execution. The scenario suite makes those differences visible through the same execution path.

Treat the released capture as a reproducible simulator artifact. The evidence fixture records the Python outputs used by the book. Your live run has its own identifier and environment record. Browser and Python outcomes should be interpreted under the same contract, without assuming that their different random generators produce identical histograms.

Work through the outcome matrix

Use the browser lab, or run each named scenario locally with the command structure from Build a Reproducible Workload Service.

These are scenario expectations for the declared simulation, not promises about every conceivable parameter override. Record both execution and scientific status. A failed validator is not equivalent to measuring a low score, and a completed process is not equivalent to establishing the quality claim.

Worked example: trace a decision back to evidence

The captured baseline stops at 5,000 shots per basis. Its score is 0.9561, lower bound 0.917177, and half-width 0.038923. The lower bound clears 0.9 and the width is below 0.04. Its provenance and history checks also pass.

The budget capture stops at 1,000 shots per basis with score 0.9545. Its interval is approximately [0.867465, 1.041535], too broad to meet the precision requirement. The high point estimate does not justify acceptance. The correct outcome is a completed, valid, inconclusive experiment.

The degraded capture stops at 200 shots per basis because its upper bound, 0.807116, is already below the threshold. Rejection is a supported conclusion about the observed-score target. It is not an execution crash.

The invalid-provenance capture has baseline-like numerical evidence but fails the model-identity requirement. This case tests whether the client actually respects validation rather than displaying a green result whenever the plotted interval passes.

Verify behavior as well as numbers

Run the project's tests from its installed environment:

python -m pytest tests/test_lab.py

The suite checks analytic probabilities, scenario behavior, interrupted continuation, cancellation, evidence mutations, checkpoint legality, and local API boundaries. Its QEC checks address the separate memory example. Read failures in terms of the contract they protect; a test name alone is not an explanation of correctness.

For an interrupted run, compare the saved batch prefix before and after resume. For drift recovery, inspect the new identifier, context, seed, and recoveryOf relationship. Do not merge those samples into one stationary estimate. For imported evidence, recompute validation rather than trusting a cached Boolean.

Statistical coverage requires its own reasoning. The approved finite-look construction protects the declared trajectory under its assumptions. A coverage simulation can test selected parameter settings and expose regressions, but its finite results do not prove the rule for correlated hardware measurements or changing operating regimes. The uncertainty chapter states the boundary of the guarantee.

Exercise and worked answer

A proposed release produces three observations: the baseline remains accepted; the invalid-provenance scenario is also shown as accepted; and the browser's baseline counts differ from Python's while both records validate against their own compatible execution provenance. Which observation blocks release?

Worked answer: The invalid-provenance acceptance is a release-blocking error because the client has bypassed the final contract. Different counts from different samplers are not themselves a failure; compare their declared models, validated evidence, and appropriate statistical behavior. A supported exact replay under one pinned engine is a different test.

Correct the acceptance path, rerun the affected invalidity and integration checks, and regenerate the release evidence when code or contract artifacts change. A completed teaching release should include the working local project, browser exercise, captured results, and an honest account of its limits. Public multiuser deployment, real-device calibration, and operational service guarantees remain additional engineering work beyond this local simulator.