Runbook Templates — Operating Quantum Computers

This appendix collects operational runbook templates. They are intentionally concrete. Adapt names, thresholds, and escalation paths to the hardware and provider environment.

A.1 QPU quality regression

Diagram loads as you read

A.1 QPU quality regression · Figure 1

View diagram source

flowchart TB
    Alert[Quality regression alert] --> Scope[Scope affected backends and workloads]
    Scope --> Freeze{Freeze intake?}
    Freeze -- yes --> Stop[Pause affected job classes]
    Freeze -- no --> Watch[Continue with warning]
    Stop --> Compare[Compare calibration, compiler, facility, workload changes]
    Watch --> Compare
    Compare --> Cause{Likely cause}
    Cause --> Cal[Calibration]
    Cause --> Comp[Compiler]
    Cause --> Fac[Facility]
    Cause --> Work[Workload mix]
    Cal --> Action[Mitigate and run benchmark gate]
    Comp --> Action
    Fac --> Action
    Work --> Action
    Action --> Release{Release?}

Illustrative listing · yaml

runbook_qpu_quality_regression:
  trigger:
    - golden_circuit_failure
    - benchmark_drop_above_threshold
    - user_trust_report_warning_rate_spike
  immediate_actions:
    - identify affected backend_versions
    - freeze high_criticality submissions if needed
    - preserve current calibration and compiler artifacts
  diagnostics:
    - compare latest calibration deltas
    - compare compiler release history
    - inspect facility telemetry window
    - inspect workload mix and mappings
  recovery:
    - rerun targeted calibration
    - roll back compiler or target model if needed
    - run benchmark release gate
    - notify affected users

A.2 Calibration publication failure

Diagram loads as you read

A.2 Calibration publication failure · Figure 2

View diagram source

sequenceDiagram
    participant Cal as Calibration
    participant Bench as Benchmark gate
    participant Sched as Scheduler
    participant Ops as Operator
    Cal->>Bench: candidate calibration
    Bench-->>Cal: failed benchmark
    Cal->>Ops: alert
    Ops->>Sched: keep previous active model
    Ops->>Cal: request targeted recalibration
    Cal->>Bench: new candidate
    Bench-->>Sched: promote only after pass

Checklist:

keep previous active model unless it is unsafe,
tag failed candidate as rejected,
preserve failed benchmark artifacts,
identify whether failure is local or global,
block compiler promotion tied to failed model,
update incident log if user workload was affected.

A.3 Provider outage

Diagram loads as you read

A.3 Provider outage · Figure 3

View diagram source

flowchart LR
    Outage[Provider outage] --> Detect[Detect API/job failures]
    Detect --> Classify[Classify: submit, queue, execute, retrieve]
    Classify --> Route{Alternative target?}
    Route -- yes --> Revalidate[Revalidate target compatibility]
    Route -- no --> Hold[Hold and notify]
    Revalidate --> Execute[Execute alternate]
    Hold --> Resume[Resume when provider recovers]

Provider outage record:

Illustrative listing · yaml

provider_outage:
  provider: string
  detected_at: timestamp
  affected_operations:
    - submit
    - status_poll
    - result_retrieval
  affected_workloads: []
  user_visible_impact: string
  workaround: alternate_provider|simulator|reservation_delay|none
  resolved_at: timestamp|null

A.4 Suspicious result

A suspicious result is not necessarily wrong. It is a result whose evidence does not support its claimed interpretation.

Diagram loads as you read

A.4 Suspicious result · Figure 4

View diagram source

flowchart TB
    Result[Suspicious result] --> Freeze[Freeze publication/use]
    Freeze --> Reproduce[Attempt reproduction]
    Reproduce --> Provenance[Inspect provenance]
    Provenance --> Baseline[Compare simulator/baseline]
    Baseline --> Hardware[Check hardware context]
    Hardware --> Decision{Trust?}
    Decision -- yes --> Release[Release with notes]
    Decision -- no --> Retract[Retract or mark invalid]

Questions:

Was the source artifact exactly identified?
Was the target model version recorded?
Was calibration in a warning state?
Did mitigation exceed its validated domain?
Did shot count support the claimed confidence?
Did a compiler or SDK version change?
Did bit ordering or observable mapping change?

A.5 Cost spike

Diagram loads as you read

A.5 Cost spike · Figure 5

View diagram source

flowchart LR
    Spike[Cost spike] --> Attribute[Attribute by project/user/backend]
    Attribute --> Cause{Cause}
    Cause --> Volume[More jobs]
    Cause --> Shots[More shots]
    Cause --> Queue[Reservations]
    Cause --> Retry[Retries/failures]
    Cause --> Sim[Simulator/HPC usage]
    Volume --> Policy[Policy action]
    Shots --> Policy
    Queue --> Policy
    Retry --> Policy
    Sim --> Policy

Actions:

Cause	Response
unexpected job volume	tighten quota or require review
excessive shots	require precision justification
reservation waste	improve reservation planning and cancellation policy
retries	fix failure cause before allowing more retries
simulator cost	add budget alerts and right-size execution

A.6 Security incident involving quantum credentials

Diagram loads as you read

A.6 Security incident involving quantum credentials · Figure 6

View diagram source

flowchart TB
    Incident[Credential incident] --> Revoke[Revoke token/key]
    Revoke --> Scope[Scope affected jobs and data]
    Scope --> Rotate[Rotate dependent secrets]
    Rotate --> Audit[Audit access logs]
    Audit --> Notify[Notify stakeholders]
    Notify --> Harden[Harden controls]

Minimum response:

revoke compromised credentials,
stop active jobs if integrity is uncertain,
identify submitted jobs under the credential,
inspect artifact access,
rotate related service credentials,
review least-privilege boundaries,
update runbook and training.

A.7 Incident severity guide

Severity	Example	Response
SEV-1	incorrect trusted result delivered to production decision	immediate freeze, executive notification, full postmortem
SEV-2	backend-wide quality regression affecting published reports	pause affected workloads, user notification
SEV-3	provider outage delaying non-critical jobs	notify users, reroute when possible
SEV-4	isolated failed job with clear retry path	automated repair guidance

Diagram loads as you read

A.7 Incident severity guide · Figure 7

View diagram source

flowchart LR
    Impact[Impact] --> Severity[Severity]
    Severity --> Response[Response speed]
    Severity --> Communication[Communication]
    Severity --> Postmortem[Postmortem depth]

A.8 Postmortem template

Illustrative listing · yaml

postmortem:
  incident_id: string
  severity: SEV-1|SEV-2|SEV-3|SEV-4
  summary: string
  timeline:
    - time: timestamp
      event: string
  impact:
    users: []
    workloads: []
    results_at_risk: []
  root_cause: string
  contributing_factors: []
  what_worked: []
  what_failed: []
  corrective_actions:
    - owner: string
      action: string
      due: date
  evidence_links: []

The postmortem should focus on evidence integrity, not blame.