Vendor Evaluation Scorecards — Operating Quantum Computers

This appendix provides practical scorecards for evaluating quantum hardware, cloud access, software tooling, and operational maturity.

F.1 Vendor scorecard

Category	Weight	Evidence	Score
hardware quality	25	fidelity, topology, stability, uptime	0-5
tooling and APIs	15	SDKs, primitives, IR support, documentation	0-5
operations	20	calibration reports, incident process, queue policy	0-5
data and provenance	15	raw shots, metadata, export, retention	0-5
security and governance	10	access control, tenant isolation, compliance	0-5
economics	10	cost per accepted result, reservation model	0-5
roadmap credibility	5	delivered milestones, transparent limits	0-5

Diagram loads as you read

F.1 Vendor scorecard · Figure 1

View diagram source

flowchart TB
    Score[Vendor score] --> Hardware[Hardware]
    Score --> Tooling[Tooling]
    Score --> Ops[Operations]
    Score --> Data[Data]
    Score --> Security[Security]
    Score --> Economics[Economics]
    Score --> Roadmap[Roadmap]

F.2 Benchmark worksheet

Illustrative listing · yaml

vendor_benchmark_score:
  vendor: string
  backend: string
  benchmark_id: string
  date: date
  hardware:
    topology_score: int
    stability_score: int
    performance_score: int
  operations:
    queue_score: int
    incident_score: int
    metadata_score: int
  economics:
    cost_per_successful_result: number
    reservation_cost: number
  decision:
    recommendation: scale | pilot | hold | exit
    rationale: string

Diagram loads as you read

F.2 Benchmark worksheet · Figure 2

View diagram source

flowchart LR
    Benchmark[Benchmark] --> Scores[Scores]
    Scores --> Weighted[Weighted total]
    Weighted --> Decision{Decision}
    Decision -- scale --> Scale[Scale]
    Decision -- pilot --> Pilot[Pilot]
    Decision -- hold --> Hold[Hold]
    Decision -- exit --> Exit[Exit]

F.3 Red flags

Red flags should block procurement until resolved:

no access to raw results,
no calibration provenance,
benchmark-only tuning with no disclosure,
vague queue policy,
no incident process,
roadmap claims without delivered milestones,
security review gaps,
contract terms that prevent reproducibility.

Diagram loads as you read

F.3 Red flags · Figure 3

View diagram source

flowchart TD
    RedFlag[Red flag] --> Block[Block decision]
    Block --> Remediate[Remediation plan]
    Remediate --> Evidence[Evidence review]
    Evidence --> Clear{Resolved?}
    Clear -- yes --> Continue[Continue evaluation]
    Clear -- no --> Exit[Exit]