A quantum service has several opportunities to produce a convincing wrong answer. A basis rotation can be incorrect while the output remains a well-formed histogram. A valid sampler can feed a stopping rule that ignores repeated inspection. A correct calculation can be attached to an incompatible engine record. A useful test strategy follows these boundaries instead of treating one successful example as proof of the entire stack.
The lab separates model tests, statistical checks, execution tests, and evidence validation. Each category needs an expected result that does not simply repeat the implementation under test. This chapter explains what those checks establish and how to retain that meaning when the software changes.
Compare independent representations
The model test obtains probabilities from the constructed Aer density-matrix circuit and compares them with an analytic expression after applying the readout channel. It exercises XX, YY, and ZZ at multiple noise settings. The analytic result is a useful oracle because its derivation does not depend on sampling the same circuit again.
Those probability checks do not establish every property of a sampled run. Separate tests exercise scenario outcomes, checkpoint handling, resume, and cancellation. For example, the interrupted scenario should preserve its completed batch prefix when resumed. Its final batch sequence can be compared with uninterrupted execution under the same pinned implementation and configuration.
For statistical arithmetic, use independently constructed count fixtures and known radii. Monte Carlo can examine coverage over a parameter grid, but has its own uncertainty and cannot prove the mathematical guarantee for every case. The finite-look proof remains in Shots, Uncertainty, and Stopping Rules.
Worked example: test the validator, not only the checksum
Suppose a release accidentally accepts a history entry whose score is missing. Changing an exported document without updating its digest will trigger an integrity mismatch, even if the history validator is broken. That test would demonstrate the checksum path while leaving the structural failure undiscovered.
A targeted validator test must also consider a consistently resealed malformed document. It should reject missing fields, nonnumeric values, negative counts, and disagreements between cumulative checkpoints and incremental totals. Resealing does not make malformed evidence valid. This separation is useful whenever the threat model includes a buggy producer that can generate its own checksums.
Similarly, an additional batch after an accepted or rejected decision must fail semantic validation. The issue is the procedure represented by the record, even if every count is nonnegative and every field has the correct type. The recorded seven-look contract cannot be silently changed by appending more observations.
From the installed local project directory, the existing test module is executable:
python -m pytest tests/test_lab.pyThe lab supplies the project and its pinned dependency specification. A test result belongs to the code and environment actually tested; retain that association when publishing a capture.
Define compatibility before upgrading
Three compatibility questions deserve separate answers. Can the new software parse the old record? Does it interpret the same fields with the same meaning? Can it continue the old execution without changing its sampler or statistical procedure? A yes to the first does not imply the others.
The lab's evidence format is shared by the browser and Python implementations, while their samplers have different engine identities. A record may be inspectable across implementations without being resumable across them. Resume requires compatible execution provenance. A new engine can start a new run and be compared under an appropriate statistical design.
Schema evolution also needs an explicit policy. Adding an optional field can still break a consumer that rejects unknown properties. Changing a threshold while keeping the same method identifier is more serious: old and new decisions would appear to refer to one contract. Release the changed artifact under a distinguishable identity and preserve access to the earlier interpretation.
An actual provider integration would additionally test native instructions, result ordering, and failure behavior against that provider. The local simulator tests do not certify those external interfaces.
Exercise and worked answer
A candidate release parses an old evidence file successfully. It changes the default stopping threshold and permits resuming that old job with a different random generator, while retaining the old method and engine labels. Which tests should block release?
Worked answer: Contract identity tests should reject the undisclosed threshold change. Resume compatibility tests should reject the sampler substitution. Parsing success establishes neither property. The corrected release must identify its changed contract and engine, start a separate experiment where required, and compare results using the declared tolerance and uncertainty design. Exact histogram equality is appropriate for a supported deterministic replay; it is not a universal expectation between independent engines.