Metrics Programs and Executive Dashboards

Metrics can improve quantum operations or distort them. A dashboard that optimizes gate fidelity alone may encourage fragile local improvements while system-level workload quality worsens. A dashboard that optimizes QPU utilization alone may encourage low-value jobs. The metric program must reflect the operating purpose.

This chapter adapts measurement-program concepts from NIST security measurement guidance [R166] to quantum platform operations.

flowchart LR
    Objective[Operating objective] --> Question[Measurement question]
    Question --> Metric[Metric]
    Metric --> Data[Data source]
    Data --> Review[Review cadence]
    Review --> Action[Management action]

Measurement principles

A useful metric has an owner, an interpretation rule, and an action path. Without those, it is decoration.

mindmap
  root((Metric quality))
    Decision linked
    Context bound
      target
      epoch
      workload
    Actionable
      owner
      threshold
      runbook
    Resistant to gaming
    Reviewed over time

Metric families

Quantum platforms need multiple metric families because no single number represents health.

flowchart TB
    Dashboard[Executive dashboard] --> Physics[Physics health]
    Dashboard --> Workload[Workload success]
    Dashboard --> Cost[Cost and capacity]
    Dashboard --> Evidence[Evidence quality]
    Dashboard --> Security[Security and governance]
    Dashboard --> Learning[Learning velocity]

Leading and lagging indicators

Leading indicators warn before failure. Lagging indicators tell whether the system already failed. An executive dashboard needs both.

flowchart LR
    Leading[Leading indicators] --> Intervention[Intervention]
    Intervention --> Lagging[Lagging outcomes]
    Lagging --> Review[Review policy]
    Review --> Leading

Examples:

Workload-normalized metrics

Raw hardware metrics should be normalized by workload class. A backend that looks poor on one metric may be excellent for a shallow, connectivity-friendly workload.

flowchart TD
    HardwareMetric[Hardware metric] --> Normalize[Normalize by workload class]
    WorkloadShape[Depth, width, topology, shots] --> Normalize
    Normalize --> Score[Workload suitability score]
    Score --> Route[Routing decision]

A workload suitability score can combine target support, historical success, queue state, cost, and evidence eligibility. It should be explainable, not merely predictive.

Executive view

Executives need fewer metrics and stronger interpretation. The dashboard should show trend, decision threshold, owner, and current action.

flowchart LR
    Metric[Metric] --> Threshold[Threshold]
    Threshold --> Status[Status]
    Status --> Owner[Owner]
    Owner --> Action[Current action]

Example dashboard rows:

Anti-patterns

flowchart TD
    Bad[Metric anti-pattern] --> Vanity[Vanity fidelity]
    Bad --> Utilization[Utilization worship]
    Bad --> Average[Averages hide target classes]
    Bad --> Unowned[No owner]
    Bad --> Static[Metric never retired]

The most dangerous quantum metric is the one that looks precise while hiding assumptions. If a dashboard does not say which target, which epoch, which workload class, and which evidence level, it is probably too vague for management action.

Operating rule

Every quantum metric should map to a decision, a threshold, an owner, and an action. If it cannot, remove it from the executive dashboard.