Quantum Data Products and Benchmark Catalogs

The long-term asset of a quantum platform is its curated evidence, not only its devices. The operator's job is to turn that thesis into a managed system: a contract, a workflow, a set of measurable controls, and a review loop. Current vendor and standards material is useful, but it should be treated as input to an operating model rather than a substitute for one [R130][R131].

Operating model

The operational pattern is consistent across this chapter: define the contract, validate early, execute under a bounded policy, capture evidence, and feed the result back into the platform.

flowchart LR
    Intent[Intent] --> Contract[Contract]
    Contract --> Validate[Validate]
    Validate --> Execute[Execute]
    Execute --> Evidence[Evidence]
    Evidence --> Review[Review]
    Review --> Improve[Improve]
    Improve --> Contract

Core principles

The first principle is versioned benchmark contracts. A quantum system becomes difficult to operate when this is informal. Make it explicit in manifests, API schemas, dashboards, and review gates.

The second principle is result cards. Hardware time, review time, and scientific attention are scarce. The platform should reject bad work early and explain how to fix it.

The third principle is failed-run retention. It should be represented as a first-class object rather than hidden in scripts or notebooks.

The remaining principles are statistical contracts, owner-defined freshness, and claim-aware access. These are the mechanisms that make the system improvable rather than merely usable.

mindmap
  root((Quantum Data Products and Benchmark Catalogs))
    versioned benchmark contracts
    result cards
    failedrun retention
    statistical contracts
    ownerdefined freshness
    claimaware access

Lifecycle

A mature implementation should have lifecycle states. A draft object is cheap to change. A reviewed object can be used by a team. A production object can support decisions. A deprecated object remains visible but should not be used for new claims.

stateDiagram-v2
    [*] --> Draft
    Draft --> Reviewed: technical review
    Reviewed --> Production: release gate
    Production --> Suspended: incident or policy failure
    Suspended --> Reviewed: fix validated
    Production --> Deprecated: replacement available
    Deprecated --> Retired
    Retired --> [*]

Failure modes

The major failure modes are cherry-picked benchmarks, untraceable dashboards, stale baselines, lost raw shots, unreviewed comparison claims, and schema forks. Each has a different owner and a different corrective action. Avoid generic labels such as “quantum failed.” They erase the distinction between physics, software, policy, and interpretation.

flowchart TB
    Failure[Failure detected] --> Classify{Classify}
    Classify --> Physics[Physics or backend]
    Classify --> Software[Software or runtime]
    Classify --> Data[Data or evidence]
    Classify --> Policy[Policy or governance]
    Classify --> Claim[Claim or interpretation]
    Physics --> Action[Corrective action]
    Software --> Action
    Data --> Action
    Policy --> Action
    Claim --> Action

Control surface

The control surface should be smaller than the implementation. Users need stable inputs and predictable outputs. Operators need deeper controls. Reviewers need evidence. Executives need portfolio-level signals. Do not force all personas into the same interface.

classDiagram
    class UserContract {
      purpose
      inputs
      limits
      outputs
    }
    class OperatorControls {
      policy
      routing
      quarantine
      rollback
    }
    class EvidenceBundle {
      provenance
      raw_data
      analysis
      reviewer_state
    }
    class DecisionView {
      cost
      risk
      maturity
      claim_status
    }
    UserContract --> EvidenceBundle
    OperatorControls --> EvidenceBundle
    EvidenceBundle --> DecisionView

Metrics

Metrics should separate system health from scientific value. A platform can be healthy while an experiment is inconclusive. A benchmark can improve while user experience degrades. Keep these dimensions separate.

flowchart LR
    Metrics[Metrics] --> Health[System health]
    Metrics --> Quality[Scientific quality]
    Metrics --> Cost[Cost and capacity]
    Metrics --> UX[Developer experience]
    Metrics --> Governance[Governance]
    Health --> Dashboard[Review dashboard]
    Quality --> Dashboard
    Cost --> Dashboard
    UX --> Dashboard
    Governance --> Dashboard

Review cadence

The review cadence should match risk. Low-risk exploratory work can use automated checks. Production claims require human review. External claims require independent challenge. Regulated or high-stakes use requires audit-grade evidence.

flowchart TB
    Work[Work item] --> Risk{Risk class}
    Risk -- exploratory --> Auto[Automated checks]
    Risk -- internal decision --> Peer[Peer review]
    Risk -- external claim --> Board[Claim review board]
    Risk -- regulated --> Audit[Audit trail and approval]
    Auto --> Archive[Archive evidence]
    Peer --> Archive
    Board --> Archive
    Audit --> Archive

Operator checklist

• Convert informal practice into a versioned contract.
• Reject invalid work before it reaches scarce hardware.
• Preserve enough evidence to explain results later.
• Separate system-health metrics from scientific-quality metrics.
• Assign owners to every failure class.
• Review claims more strictly than exploratory runs.