Experiment Warehouses and Quantum Data Products

Quantum organizations eventually drown in artifacts: circuits, transpilation outputs, pulse schedules, calibration snapshots, queue metadata, shot files, mitigation settings, optimizer traces, simulator runs, notebooks, plots, and claims. A filesystem full of timestamped folders is not an experiment system.

An experiment warehouse turns quantum operations into queryable institutional memory. It should answer: what did we run, why did we run it, under which physical conditions, what changed, what did it cost, and which claims depended on it?

61.1 Warehouse architecture

flowchart LR
    Runtime[Quantum runtimes] --> Ingest[Artifact ingestion]
    Sim[Simulators] --> Ingest
    CI[CI pipelines] --> Ingest
    Notebooks[Exploration notebooks] --> Ingest
    Ingest --> Lake[Immutable artifact lake]
    Ingest --> Warehouse[Queryable warehouse]
    Warehouse --> Catalog[Experiment catalog]
    Warehouse --> BI[Cost and quality dashboards]
    Lake --> Evidence[Evidence packages]

The artifact lake preserves raw material. The warehouse supports analysis. The catalog provides human navigation.

61.2 Core data model

erDiagram
    EXPERIMENT ||--o{ RUN : contains
    RUN ||--o{ ARTIFACT : produces
    RUN ||--o{ METRIC : reports
    RUN }o--|| TARGET_SNAPSHOT : uses
    RUN }o--|| SOURCE_VERSION : built_from
    RUN }o--o{ CLAIM : supports
    CLAIM ||--o{ REVIEW : receives

    EXPERIMENT {
        string experiment_id
        string owner
        string objective
        string status
    }
    RUN {
        string run_id
        string experiment_id
        string run_class
        datetime submitted_at
        string target_id
    }
    TARGET_SNAPSHOT {
        string target_snapshot_id
        string backend
        string calibration_hash
        string capability_profile
    }

Every run should be joinable to source, target, artifacts, metrics, and claims.

61.3 Events, traces, and metrics

Use observability patterns from the rest of infrastructure. OpenTelemetry semantic conventions provide a general structure for traces, metrics, logs, resources, and common attributes [R102]. Quantum platforms should extend these ideas with quantum-specific attributes.

flowchart TB
    Span[Trace span] --> Attributes[Attributes]
    Attributes --> Workload[workload.type]
    Attributes --> Target[target.id]
    Attributes --> Shots[shots]
    Attributes --> Circuit[circuit.hash]
    Attributes --> Mitigation[mitigation.policy]
    Attributes --> Calibration[calibration.hash]

Recommended quantum attributes:

61.4 Artifact immutability

Mutable results are not evidence.

flowchart LR
    Artifact[Artifact] --> Hash[Content hash]
    Hash --> Store[Immutable store]
    Store --> Manifest[Manifest reference]
    Manifest --> Claim[Claim package]

Treat these as immutable:

• source snapshots
• canonical circuits
• compiled circuits
• target profiles
• raw shot data
• mitigation inputs and outputs
• optimizer traces
• review decisions

Derived dashboards can change. Evidence inputs should not.

61.5 Quantum data products

A data product is a curated, documented, versioned dataset with an owner and consumers.

flowchart TB
    Raw[Raw runs] --> Curate[Curated dataset]
    Curate --> Docs[Data contract]
    Docs --> Consumers[Consumers]
    Consumers --> Feedback[Quality feedback]
    Feedback --> Curate

Examples:

61.6 Lineage queries

The warehouse should answer lineage questions quickly.

flowchart LR
    Claim[Claim] --> Run[Run]
    Run --> Circuit[Circuit]
    Run --> Target[Target snapshot]
    Run --> Source[Source version]
    Run --> Data[Raw data]
    Run --> Review[Review]

Useful queries include:

• Which claims used a target snapshot later marked degraded?
• Which algorithm versions improved cost per accepted result?
• Which compiler change increased two-qubit count on a target family?
• Which mitigation policy reduced variance but introduced bias?
• Which notebook generated a promoted workflow?

61.7 Retention policy

Not all artifacts deserve the same retention.

flowchart TB
    Artifact[Artifact] --> Class{Class}
    Class -- exploratory --> Short[Short retention]
    Class -- decision --> Long[Long retention]
    Class -- regulated --> Hold[Legal / compliance hold]
    Class -- benchmark --> Medium[Medium retention]

Decision-grade evidence should outlive the systems that produced it. Exploratory scratch data can expire after its learning value is exhausted.

61.8 Warehouse quality checks

flowchart LR
    Ingest[Ingest] --> Validate[Schema validation]
    Validate --> Hash[Hash verification]
    Hash --> Link[Lineage completeness]
    Link --> Quality[Quality scoring]
    Quality --> Publish[Publish]

Warehouse checks should block publication when:

• required fields are missing
• source hash is absent
• target snapshot is missing
• run status is ambiguous
• raw data and summary disagree
• evidence package references mutable paths

61.9 Access control

Data access must separate operational transparency from sensitive details.

flowchart TB
    User[User] --> Role{Role}
    Role --> Researcher[Researcher view]
    Role --> Operator[Operator view]
    Role --> Auditor[Auditor view]
    Role --> Customer[Customer view]
    Researcher --> Redact[Policy redaction]
    Operator --> Redact
    Auditor --> Redact
    Customer --> Redact

Sensitive fields may include customer identifiers, proprietary algorithms, vendor contract terms, export-controlled technical detail, and raw hardware diagnostics.

61.10 Operating rule

The warehouse is not a passive archive. It is the control surface for organizational learning.

flowchart LR
    Runs[Runs] --> Memory[Institutional memory]
    Memory --> Better[Better decisions]
    Better --> Runs

If a team cannot query its past experiments, it cannot systematically improve its future experiments.

Additional technical sources: [R267].