Experiment Data Engineering and Reproducibility

A quantum result without provenance is not a result. It is a number detached from the physical conditions that produced it.

The data engineering problem is harder than it first appears. A run is not only a circuit and a count table. It is a bundle containing input intent, source code, parameter values, compiler decisions, target model, calibration snapshot, execution timing, shot data, mitigation configuration, post-processing code, uncertainty model, and human interpretation.

The platform should make reproducibility the default path.

13.1 The experiment bundle

Every production quantum run should produce an experiment bundle.

flowchart TB
    Bundle[Experiment bundle] --> Intent[Problem intent]
    Bundle --> Source[Source program]
    Bundle --> Circuit[Circuit / IR]
    Bundle --> Compile[Compiler artifact]
    Bundle --> Calibration[Calibration snapshot]
    Bundle --> Execution[Execution metadata]
    Bundle --> Shots[Shot data]
    Bundle --> Mitigation[Mitigation settings]
    Bundle --> Post[Post-processing code]
    Bundle --> Report[Trust report]

A minimal bundle schema should include:

experiment_id: exp_2026_04_18_001
project: catalyst-screening
owner: research-platform@example.com
intent:
  problem_class: hamiltonian-estimation
  success_metric: energy_estimate_with_ci
source:
  repository: git@example.com:team/quantum-apps.git
  commit: 6d1f...
  environment_hash: sha256:...
program:
  source_language: qiskit
  circuit_hash: sha256:...
  parameters_hash: sha256:...
compiler:
  compiler_name: qiskit-transpiler
  compiler_version: 2.x
  target_profile_hash: sha256:...
execution:
  backend: provider:device
  shots: 20000
  submitted_at: 2026-04-18T19:00:00Z
  calibration_snapshot: cal_...
results:
  raw_counts_uri: s3://...
  mitigated_result_uri: s3://...
  trust_report_uri: s3://...

This is deliberately similar to modern ML experiment tracking, but quantum needs stronger coupling to hardware state.

13.2 Provenance graph

Linear logs are insufficient. Quantum provenance is a graph.

flowchart LR
    Problem[Problem specification] --> Program[Source program]
    Program --> Circuit[Logical circuit]
    Circuit --> Transpile[Transpiled circuit]
    Target[Target profile] --> Transpile
    Calibration[Calibration snapshot] --> Transpile
    Transpile --> Executable[Executable workload]
    Executable --> Job[Job]
    Job --> Raw[Raw results]
    Calibration --> Raw
    Raw --> Mitigated[Mitigated estimate]
    Mitigation[Mitigation config] --> Mitigated
    Mitigated --> Report[Trust report]

The same source program may produce different executable workloads under different calibration snapshots. The same executable may produce different statistical estimates under different shot counts. The same raw result may produce different final estimates under different mitigation settings. The provenance graph makes those distinctions explicit.

13.3 Data classes

Not all quantum data has the same value or sensitivity.

The platform should avoid a single retention policy for all artifacts. Raw shot data can be large, while trust reports are compact and valuable.

flowchart TD
    Data[Quantum data] --> Raw[Raw shot data]
    Data --> Derived[Derived estimates]
    Data --> Meta[Metadata]
    Data --> Telemetry[Operational telemetry]
    Raw --> RetainRaw[Retain by project policy]
    Derived --> RetainDerived[Retain for reproducibility]
    Meta --> RetainMeta[Retain with access control]
    Telemetry --> Aggregate[Aggregate for operations]

13.4 Immutable run records

Run records should be immutable. Corrections should create new versions rather than overwriting old results.

stateDiagram-v2
    [*] --> Created
    Created --> Submitted
    Submitted --> Executing
    Executing --> Succeeded
    Executing --> Failed
    Submitted --> Cancelled
    Succeeded --> Interpreted
    Failed --> Analyzed
    Interpreted --> Superseded
    Analyzed --> Superseded
    Superseded --> [*]

An immutable record allows two important behaviors:

1. Operators can debug what happened without losing the original evidence.
2. Researchers can publish revised interpretations without rewriting history.

A useful version chain looks like this:

flowchart LR
    RawV1[Raw result v1] --> MitV1[Mitigation v1]
    RawV1 --> MitV2[Mitigation v2]
    MitV1 --> ReportV1[Trust report v1]
    MitV2 --> ReportV2[Trust report v2]
    ReportV1 -. superseded by .-> ReportV2

13.5 Shot-level versus aggregate storage

Shot-level records preserve ordering and associated per-shot metadata. An aggregate histogram preserves joint outcomes within each measured bitstring, but usually loses shot order and temporal correlations. For a fixed measurement setting and an appropriate sampling model, counts can support observable estimation and count-based reanalysis, including readout mitigation. Neither format recovers analog readout signals that were never retained.

flowchart TB
    Shots[Shot stream] --> Preserve{Preserve individual shots?}
    Preserve -- yes --> ShotLake[Shot-level lake]
    Preserve -- no --> Counts[Aggregate counts]
    ShotLake --> Reprocess[Reprocess with new methods]
    Counts --> Estimate[Reanalyze methods supported by counts and metadata]
    Estimate --> NewEstimate
    Reprocess --> NewEstimate[New estimate]

Decision rule:

The default should depend on result value, not habit.

13.6 Metadata that operators actually need

Good metadata answers operational questions quickly.

flowchart LR
    Query[Operational question] --> Metadata[Metadata index]
    Metadata --> Artifacts[Artifact store]
    Metadata --> Telemetry[Telemetry store]
    Metadata --> Audit[Audit log]
    Artifacts --> Answer[Answer with evidence]
    Telemetry --> Answer
    Audit --> Answer

Metadata should be indexed. A pile of JSON next to result files is better than nothing, but it is not an operational data system.

13.7 Dataset design for quantum results

A useful result store separates identity, artifacts, metrics, and lineage.

erDiagram
    EXPERIMENT ||--o{ RUN : has
    RUN ||--o{ ARTIFACT : produces
    RUN ||--o{ METRIC : reports
    RUN ||--o{ EVENT : emits
    RUN }o--|| COMPILATION : uses
    RUN }o--|| CALIBRATION_SNAPSHOT : references
    ARTIFACT }o--|| STORAGE_OBJECT : stored_as

Minimal tables or collections:

This model supports both research and operations. Researchers ask, “which method worked?” Operators ask, “which platform state caused failures?” Both need the same lineage.

13.8 Reproducibility levels

Reproducibility is not binary.

flowchart TB
    L0[L0: claim only] --> L1[L1: result artifact]
    L1 --> L2[L2: source and parameters]
    L2 --> L3[L3: compiler and calibration provenance]
    L3 --> L4[L4: raw data and post-processing]
    L4 --> L5[L5: replayable or statistically comparable workflow]

Most internal exploratory work can target L3. Published or business-critical results should target L4 or L5.

13.9 Result promotion pipeline

Do not let every notebook output become an official result. Promote results through gates.

flowchart LR
    Exploratory[Exploratory run] --> Candidate[Candidate result]
    Candidate --> Validate[Validation suite]
    Validate --> Review[Human review]
    Review --> Promote{Promote?}
    Promote -- no --> Archive[Archive as exploratory]
    Promote -- yes --> Official[Official result]
    Official --> Publish[Publish or use downstream]

Promotion criteria:

• calibration state is acceptable,
• benchmark context is known,
• uncertainty is reported,
• source and artifacts are versioned,
• mitigation settings are documented,
• result survives independent review,
• alternative explanations have been checked.

A quantum platform should make unofficial outputs easy to generate and official outputs hard to fake.

13.10 Reproducibility checklist

Before a result is trusted, require:

flowchart TD
    Result[Candidate result] --> Checklist[Reproducibility checklist]
    Checklist --> Complete{Complete?}
    Complete -- no --> Missing[Return missing evidence]
    Missing --> Checklist
    Complete -- yes --> Trusted[Eligible for trusted use]

The final artifact of a quantum run should not be a screenshot, a notebook cell, or a count dictionary. It should be a governed, versioned, inspectable evidence package.

Additional technical sources: [R267], [R298].