Synthetic scenario / Not current CEX evidence

Test Case Fail Demo

Back to Run Records

Synthetic teaching scenario · counterfactual steps

Decision replay

A failure, its fix and a passing test answer different questions. Explore the decision boundary without changing real records.

Required case failed · No-Go

Fix the defect and perform a targeted rerun. The original failure remains in the ledger.

synthetic-failure

Decision impact

How does one required-case failure affect the decision?

No-Go

Run Result

Fail3 Cases · 2 Pass · 1 Fail

Product Status

At RiskP0 risk not accepted

Release Decision

Do not releaseRemain No-Go until fixed and rerun

Linked Defect

DEF-MDQ-004High · Open

Synthetic run record

WebSocket ordering checks

Run Failed
Run ID
DEMO-RUN-FAIL-001
Repo
demo/cex-market-data-quality-lab
Tested version
demo-sha-not-current-evidence
Execution time
2026/08/18 17:20 (Taiwan time, demo)
TC-MDQ-016Accept sequentially increasing market-data updates
P00.18sPass
TC-MDQ-017Reject stale-sequence updates
P00.21sFail
TC-MDQ-018Restore the latest snapshot after reconnecting
P10.44sPass
×TC-MDQ-017 failure detailsOpen by default so material risk is not hidden

Expected

Reject the stale-sequence updateAfter sequence 104, sequence 103 must not overwrite current data.

Actual

The stale update was acceptedSequence 103 overwrote 104, returning the order book to stale state.

Impact

Users may see stale market data

Risk

High · Release blocking

Reproduction

Failed 3 of 3 times

Linked defect

DEF-MDQ-004 · Stale update overwrites current state

Status
Open
Severity
High
Owner
William (Demo)
Next
Fix → targeted rerun → release re-assessment

Sanitized execution output

Execution Log

Exit code 1

Only logs directly supporting the failure decision are shown; secrets, internal paths, and unrelated output were removed.

Command
python -m pytest tests/test_order_book.py::test_rejects_stale_update -q
Source
DEMO-RUN-FAIL-001 · sanitized excerpt
  1. INFOcase=TC-MDQ-017 start current_sequence=104
  2. ASSERTexpected=reject actual=accepted incoming_sequence=103
  3. ERRORstale update replaced current order-book state
  4. INFOresult=FAIL defect=DEF-MDQ-004
View raw log excerpt
FAILED tests/test_order_book.py::test_rejects_stale_update
AssertionError: stale sequence 103 was accepted after sequence 104
Expected: update rejected and current state preserved
Actual: stale update replaced current state
1 failed, 2 passed in 0.83s

Reading rules

  1. Test Case FailRecords the actual failure of one check.
  2. Run FailThis Run contains at least one case that did not meet acceptance criteria.
  3. Release No-GoThis case is a required P0 and its risk is not accepted; not every failure automatically produces the same release decision.