Third-Party Verification Audits and Benchmark Witnesses

Quantum platforms need reviewers that are not the same people who built the benchmark, chose the target, tuned the mitigation, and wrote the claim. Third-party verification does not make a result true, but it reduces the risk of selective reporting, hidden assumptions, and unreviewed drift.

Diagram loads as you read

Third-Party Verification Audits and Benchmark Witnesses · Figure 1

View diagram source

flowchart LR
    Claim[Quantum claim] --> Evidence[Evidence package]
    Evidence --> Witness[Independent witness]
    Witness --> Review[Verification review]
    Review --> Outcome[Accept, qualify, or reject]

Verification roles

Role	Function
benchmark owner	defines benchmark intent and scoring
platform operator	runs workload and preserves evidence
provider	supplies backend and operational facts
witness	observes or independently verifies execution
auditor	checks process and controls
scientific reviewer	checks inference and claim strength
customer representative	validates business relevance

Diagram loads as you read

Verification roles · Figure 2

View diagram source

mindmap
  root((Verification ecosystem))
    Benchmark owner
    Platform operator
    Provider
    Witness
    Auditor
    Scientific reviewer
    Customer representative

Witnessable events

Diagram loads as you read

Witnessable events · Figure 3

View diagram source

flowchart TB
    Witness[Witnessable events] --> Spec[Spec freeze]
    Witness --> Target[Target snapshot]
    Witness --> Compile[Compile profile]
    Witness --> Execution[Execution start and end]
    Witness --> Calibration[Calibration epoch]
    Witness --> Raw[Raw result capture]
    Witness --> Claim[Claim review]

A witness does not need to see everything. The point is to observe the events that would otherwise be easiest to manipulate after the fact.

Benchmark witness protocol

Diagram loads as you read

Benchmark witness protocol · Figure 4

View diagram source

sequenceDiagram
    participant Owner
    participant Operator
    participant Witness
    participant Evidence
    Owner->>Evidence: Freeze benchmark specification
    Witness->>Evidence: Sign spec hash
    Operator->>Evidence: Attach target snapshot
    Operator->>Evidence: Attach compile and runtime profile
    Witness->>Evidence: Sign execution window
    Operator->>Evidence: Attach raw and aggregate results
    Witness->>Evidence: Sign completeness statement

The witness should sign hashes of artifacts, not necessarily copy every artifact. This supports confidential review while preserving integrity.

Audit dimensions

Diagram loads as you read

Audit dimensions · Figure 5

View diagram source

flowchart LR
    Audit[Audit] --> Process[Process]
    Audit --> Evidence[Evidence]
    Audit --> Statistics[Statistics]
    Audit --> Security[Security]
    Audit --> Claims[Claims]

Dimension	Questions
process	Was the review path followed?
evidence	Are artifacts complete and immutable?
statistics	Are uncertainty, selection, and failures disclosed?
security	Were credentials, access, and custody controlled?
claims	Does the wording match the evidence?

Anti-gaming controls

Diagram loads as you read

Anti-gaming controls · Figure 6

View diagram source

flowchart TB
    Gaming[Benchmark gaming] --> Cherry[Cherry-picking runs]
    Gaming --> Target[Target selection bias]
    Gaming --> Mitigation[Undisclosed mitigation]
    Gaming --> Drift[Hidden calibration drift]
    Gaming --> Posthoc[Post-hoc metric changes]
    Cherry --> Controls[Controls]
    Target --> Controls
    Mitigation --> Controls
    Drift --> Controls
    Posthoc --> Controls

Controls include pre-registered benchmarks, locked scoring code, target selection rules, failure reporting, witness signatures, independent reruns, and public limitation statements.

Audit trail

Diagram loads as you read

Audit trail · Figure 7

View diagram source

flowchart LR
    Spec[Spec hash] --> Target[Target hash]
    Target --> Compile[Compiler hash]
    Compile --> Runtime[Runtime profile]
    Runtime --> Results[Result hashes]
    Results --> Review[Review decision]

A good audit trail is boring: hashes, timestamps, identities, target identifiers, calibration epochs, runtime parameters, access logs, and reviewer decisions.

Verification outcomes

Diagram loads as you read

Verification outcomes · Figure 8

View diagram source

stateDiagram-v2
    [*] --> Submitted
    Submitted --> Verified
    Submitted --> Qualified
    Submitted --> Rejected
    Qualified --> Revised
    Revised --> Submitted
    Rejected --> Retired

Outcome	Meaning
verified	evidence supports the stated claim
qualified	evidence supports a narrower claim
rejected	evidence does not support the claim
deferred	missing artifacts prevent decision
retired	claim is obsolete or superseded

Operational rule

Do not make external quantum-performance claims without independent evidence review proportional to the importance of the claim.

Diagram loads as you read

Operational rule · Figure 9

View diagram source

flowchart LR
    Importance[Claim importance] --> ReviewDepth[Review depth]
    ReviewDepth --> Witness[Witness or audit]
    Witness --> Claim[External claim]