ReferenceOperations
Incident response
Incidents are how the platform reports what it must never fix silently. Each has a kind, a severity and two flags:
blocks_trading (closes the trading gate: no BUY, SELL or TRANSFER) and blocks_issuance (no issue, burn or
refill for the asset). Open incidents are under Admin → Reconciliation; an open incident with the same source is
reused, so repeated detection does not flood operators. Resolving needs a note and is audited.
General procedure
- Acknowledge in the console (owner = you). Do not resolve until the cause is understood.
- Contain: if money or keys may be at risk and the incident does not already block trading, use Admin → Emergency (pause one asset or all). Freeze affected customers if their accounts are involved.
- Investigate with the evidence the incident links: the reconciliation run (node and ledger read), the chain movements (Dara Scan shows each movement and whether a platform intent explains it), the audit log.
- Correct only through the platform's own flows (issue/burn/refill with the human signers, clawback, cash entries with four-eyes). Never edit balances in the database.
- Resolve with a note naming the cause and the correction. Then resume: resolve the incident, then resume each paused asset (two deliberate steps).
Kinds
| Kind | Severity / blocks | Means | Response |
|---|---|---|---|
UNEXPLAINED_TOKEN_MOVEMENT | CRITICAL, trading + issuance | One of our assets moved with no matching intent (no MEMO_HASH of ours) | Key-compromise playbook below |
UNEXPECTED_AUTHORITY_CHANGE | CRITICAL, trading | A signer, threshold or flag of one of our accounts changed without an intent | Key-compromise playbook; verify the account layout (ceremony.Verify runs at every start) |
UNKNOWN_HOLDER | CRITICAL, trading + issuance | An authorized trustline of our asset on an account we did not create | AUTH key or issuer compromise: revoke the trustline (freeze path), claw back, rotate AUTH |
SUPPLY_OUTSIDE_TRUSTLINES | CRITICAL | Our asset in a claimable balance, liquidity pool or contract balance | Identify how it got there (it needs an authorized holder's key); claw back where possible |
UNEXPECTED_OPERATION | CRITICAL | A DEX offer or other operation on our asset by our accounts that no intent asked for | Treat as key misuse |
SAC_DEPLOYED | WARNING | Someone deployed the asset's Stellar Asset Contract (anyone may) | No funds can move through it without authorized holders; note it, watch SUPPLY_OUTSIDE_TRUSTLINES |
INDEXER_GAP | CRITICAL | The indexer cannot read a ledger it needs (outside the node's history) | Check the Galexie lake (deploy/datalake/; refill a gap with galexie scan-and-fill) and the backend's BACKFILL_* settings; STELLAR_RPC_RUNBOOK.md |
CHANNEL_SEQUENCE_FOREIGN | CRITICAL | Someone else used a channel account's sequence; the channel is quarantined | Channel key compromise: rotate by replacing channels (KEY_MANAGEMENT.md) |
CHAIN_OPERATION_UNRESOLVED | CRITICAL | An intent's outcome could not be proven (RECONCILIATION_REQUIRED); its holds stay | Find each attempt's hash on-chain; if one applied, its effect is real: post it through the handler path with engineering; if none can ever apply (maxTime passed on every node), fail it |
AUTHORIZATION_CHANGE_FAILED | CRITICAL | A freeze or unfreeze did not apply on-chain | Retry from the console; check AUTH key and issuer flags |
RESERVE_SHORTFALL | CRITICAL, issuance + BUY for the asset | Supply above unexpired backing | New approved reserve snapshot, or burn from COLD |
EMERGENCY_PAUSE | CRITICAL | A Platform Admin paused trading | Resolve when the reason is gone, then resume assets |
| reconciliation checks | per check | One kind per failing check | RECONCILIATION_RUNBOOK.md |
Key-compromise playbook
- The incident already stops trading. Pause everything (Emergency) to stop issuance too.
- Identify the key: the movement's source account and the signers that could have signed it (
AUTHORITY_MATRIX.md§2). - Rotate it through the human signers (
KEY_MANAGEMENT.md); for a channel, replace the channels. - Contain the tokens: freeze the receiving accounts if they are ours, claw back what was moved, re-issue to the
rightful holders under normal four-eyes and signatures (
CLAWBACK_POLICY.md). - Run reconciliation; it must pass before trading resumes. Write the post-incident report.
Communication
Customer-facing status comes from the gate (/v1/status): the wallet shows "trading paused" automatically. Any
statement beyond that (cause, impact, timeline) is the operator's decision and must not include customer data.