Scientific Correctness and Reproducible Claims

A quantum result is not correct because it came from a quantum processor. It is correct only relative to a claim, a method, a baseline, and an uncertainty model.

This chapter separates three ideas that are often conflated:

Execution success: the job ran and returned data.
Statistical validity: the estimate has quantified uncertainty under the sampling model.
Scientific usefulness: the result supports a meaningful claim better than relevant alternatives.

54.1 Claim hierarchy

Diagram loads as you read

54.1 Claim hierarchy · Figure 1

View diagram source

flowchart TB
    Data[Raw data] --> Estimate[Estimate]
    Estimate --> Statistical[Statistical claim]
    Statistical --> Physical[Physical claim]
    Physical --> Application[Application claim]
    Application --> Business[Business claim]

Each layer adds assumptions. Do not let a valid lower-layer claim imply an invalid higher-layer claim.

54.2 Evidence package

A reproducible claim needs an evidence package.

Illustrative listing · yaml

evidence_package:
  claim_id: claim-qchem-2026-04-19-001
  claim_text: "mitigated estimator achieves lower error than classical approximation on benchmark set B"
  workload:
    problem_family: electronic_structure
    instance_set: benchmark-set-B-v3
  execution:
    backend: qpu-alpha
    runtime_contract_id: runtime-contract-123
    shots: 50000
  artifacts:
    raw_measurements: s3://...
    compiled_circuits: s3://...
    calibration_snapshot: s3://...
    mitigation_config: s3://...
    analysis_notebook_or_pipeline: git://...
  statistics:
    estimator: recorded
    confidence_method: bootstrap | analytic | bayesian
    multiple_testing_policy: recorded
  baselines:
    classical_baseline: recorded
    simulator_baseline: recorded
    previous_qpu_baseline: recorded

54.3 Reproducibility loop

Amazon Braket Hybrid Jobs document job inputs, source modules, device selection, output artifacts, and checkpoints. These patterns are useful beyond Braket: long-running hybrid experiments need checkpointed state and recoverable outputs. [R99]

Diagram loads as you read

54.3 Reproducibility loop · Figure 2

View diagram source

sequenceDiagram
    participant Dev as Researcher
    participant Repo as Code repository
    participant Runtime as Runtime/job service
    participant Store as Artifact store
    participant Review as Review board

    Dev->>Repo: Commit analysis and workload code
    Dev->>Runtime: Submit versioned job
    Runtime->>Store: Write inputs, checkpoints, raw outputs
    Runtime-->>Dev: Return result metadata
    Dev->>Review: Submit evidence package
    Review->>Store: Verify artifacts
    Review-->>Dev: Accept, revise, or reject claim

54.4 Baseline discipline

A quantum experiment without baselines is not an experiment. It is a measurement event.

Diagram loads as you read

54.4 Baseline discipline · Figure 3

View diagram source

flowchart LR
    Workload[Workload] --> QPU[QPU result]
    Workload --> Simulator[Simulator result]
    Workload --> Classical[Classical baseline]
    Workload --> Prior[Prior quantum run]
    QPU --> Compare[Comparison]
    Simulator --> Compare
    Classical --> Compare
    Prior --> Compare
    Compare --> Claim[Claim strength]

Baselines should be selected before the run. Otherwise the experiment becomes a search for favorable comparisons.

54.5 Statistical controls

Control	Purpose
pre-registration	prevent moving the goalpost after data arrives
holdout instances	detect overfitting to benchmark set
repeated runs	separate device drift from method improvement
confidence intervals	prevent single-point result worship
multiple-test correction	avoid cherry-picking from many circuits
raw-data retention	allow independent reanalysis

Diagram loads as you read

54.5 Statistical controls · Figure 4

View diagram source

flowchart TB
    Claim[Claim] --> PreReg[Pre-registration]
    Claim --> Holdout[Holdout set]
    Claim --> Repeat[Repeated runs]
    Claim --> CI[Confidence intervals]
    Claim --> Multiple[Multiple-test policy]
    Claim --> Raw[Raw-data retention]

54.6 Telemetry for science

OpenTelemetry semantic conventions show the value of common names for telemetry attributes across systems. Quantum platforms should adopt the same principle: standardize experiment identifiers, target identifiers, circuit hashes, artifact hashes, and result metadata. [R102]

Diagram loads as you read

54.6 Telemetry for science · Figure 5

View diagram source

flowchart LR
    Trace[Experiment trace] --> Span1[Compilation span]
    Trace --> Span2[Queue span]
    Trace --> Span3[Execution span]
    Trace --> Span4[Mitigation span]
    Trace --> Span5[Analysis span]
    Span1 --> Attr[Common attributes]
    Span2 --> Attr
    Span3 --> Attr
    Span4 --> Attr
    Span5 --> Attr

54.7 Review rubric

Illustrative listing · yaml

claim_review:
  reject_if:
    - no raw artifacts
    - no target snapshot
    - no baseline
    - no uncertainty model
    - no mitigation disclosure
  downgrade_to_exploratory_if:
    - benchmark selected after seeing data
    - device drift not analyzed
    - repeated runs missing
    - result sensitive to one outlier circuit
  accept_if:
    - claim scope matches evidence
    - uncertainty is quantified
    - baselines are relevant
    - artifacts are independently replayable

54.8 Claim labels

Use explicit labels to prevent overclaiming.

Diagram loads as you read

54.8 Claim labels · Figure 6

View diagram source

stateDiagram-v2
    [*] --> Exploratory
    Exploratory --> Reproduced: repeated internally
    Reproduced --> Validated: independent pipeline or team
    Validated --> Published: review approved
    Published --> Deprecated: superseded or invalidated
    Reproduced --> Exploratory: drift/regression
    Validated --> Reproduced: external concern

Label	Meaning
exploratory	useful for learning, not decision-grade
reproduced	repeated under similar conditions
validated	checked by independent reviewer or pipeline
decision-grade	approved for a defined operational decision
deprecated	no longer valid under current evidence

54.9 Correctness anti-patterns

Diagram loads as you read

54.9 Correctness anti-patterns · Figure 7

View diagram source

flowchart TD
    Anti[Correctness anti-patterns] --> ShotFishing[Shot fishing]
    Anti --> BenchmarkShopping[Benchmark shopping]
    Anti --> HiddenMitigation[Hidden mitigation]
    Anti --> NoBaseline[No baseline]
    Anti --> DriftBlind[Ignoring drift]
    Anti --> BusinessLeap[Jumping from physics result to business claim]

The strongest quantum teams will be known not only for the results they publish, but for the weak claims they refuse to make.