# A protocol stops working (/invest/incidents/protocol-outage)

A protocol can stop working without losing any money. Its program may fail on every call, its operators may pause it, an account may disappear, or all its cash may be lent out. This page explains what the vault does in each case.

The examples start from the [example vault](/invest/incidents#the-example-vault-used-on-these-pages): Alice holds 4,000 USDC of Senior, June holds 1,000 USDC of Junior, 3,000 USDC is invested in one **leg** (one investment position of the vault) and 2,000 USDC is idle cash. Each share is worth 1 USDC at the start.

## Terms used on this page [#terms-used-on-this-page]

- **Fresh**: a leg is fresh when its value was refreshed recently. "Recently" means within the vault's maximum valuation age, which each vault sets in Solana slots and the program caps at 150. A slot is one Solana block, about 400 milliseconds, so 150 slots is roughly one minute.
- **Stale**: a leg that is not fresh.
- **Suspension**: a guardian action that sets a leg aside so the vault can keep running without it.
- **Low value**: the cautious end of a leg's reported value range, roughly what it would fetch if sold now. Withdrawals are priced at it.

## Why an outage stops the vault [#why-an-outage-stops-the-vault]

Every deposit, withdrawal funding and allocation is priced from the leg values. Before any of them, the program checks that every leg is fresh. If one leg cannot be valued, nobody knows what a share is worth, and the program refuses to guess. The action fails with `StaleValuation`.

This is deliberate. Pricing shares from an old number would let someone who knows the protocol is in trouble trade against everyone else.

## Example 1: the protocol fails every call [#example-1-the-protocol-fails-every-call]

Alice has two withdrawal requests: one for 500 Senior shares that is already funded, so 500 USDC waits for her in the vault's payout account, and one that is not yet funded. Then the protocol's program starts failing every call.

| Action | Result |
| --- | --- |
| `refresh_leg` | Fails with the protocol's own error. |
| Bob, a new investor, deposits 100 USDC into Junior | `StaleValuation` |
| Funding Alice's second request | `StaleValuation` |
| The executor allocates 10 USDC to the leg | `StaleValuation` |
| Alice claims her first request, already funded | **Succeeds**: 500 USDC |

*Table: once the leg is stale, every action that prices shares stops, but money already set aside for a withdrawal is still paid.*

The takeaway: money already reserved for you in the payout account does not depend on any protocol. A funded request can always be claimed.

The same happens when a protocol account the adapter needs is closed (the transaction fails with `AccountOwnedByWrongProgram`) or when the protocol uses up all the computing budget a Solana transaction is allowed. In each case the failed transaction changes nothing, and the vault stops once the leg goes stale.

## Which real protocols stop the valuation [#which-real-protocols-stop-the-valuation]

Not every adapter needs the protocol's program to value a leg. Some read the protocol's accounts directly, and those keep working while the program is down.

| Adapter | Valuation calls the protocol's program | During an outage of that program |
| --- | --- | --- |
| Kamino | Yes | The leg goes stale and the vault stops. |
| Jupiter Lend | Yes | The leg goes stale and the vault stops. |
| OnRe and the ONyc token leg | Yes (OnRe's `get_nav`) | The leg goes stale and the vault stops. |
| Huma | No | The leg is still valued. Only money going into Huma fails. |
| Hastra | No | The leg is still valued. Exits that sell the token keep working. |
| Solstice | No | The leg is still valued. Exits that sell the token keep working. |

*Table: which adapters need the protocol's program to value a leg, as tests on copies of the real protocols show; adapters that read accounts directly keep the vault running through an outage.*

Keeping a leg valued while its program is down is a deliberate choice. Akashi's off-chain monitor warns the guardian in that case, so a person decides whether the value can still be trusted.

## Example 2: the protocol pauses itself [#example-2-the-protocol-pauses-itself]

The protocol's operators pause it. The Huma, Hastra and Solstice adapters then report the `PROTOCOL_PAUSED` flag, and Kamino's emergency mode makes its adapter fail. A report with a blocking flag is refused with `AdapterFlagged`, so the leg goes stale as in Example 1.

## The fix: suspend the leg [#the-fix-suspend-the-leg]

**What it is.** The guardian or the admin calls `suspend_leg`. The leg's mode becomes `Suspended`.

**Why it exists.** An outage is not a loss. The protocol may be fine and come back in an hour. Marking the leg down (see [A hack the protocol has not reported](/invest/incidents/hack-discovered)) would book a loss that may not be real, and would stop Senior withdrawals if the leg is large. A suspension keeps the vault running without booking anything.

**How it works.**

- The leg keeps the last value range it reported. The vault's state does not change and no loss is booked.
- The leg no longer has to be fresh, so the vault's other actions work again.
- Withdrawals still price the leg at the low value of its last range, and the gap to its book value is taken from Junior first.
- No new money can go into the leg (`LegSuspended`), but the executor can still take money out if the protocol answers.
- `unsuspend_leg` returns the leg to the mode it had before. It must be refreshed in the same transaction, because it is no longer exempt from the freshness check.

```mermaid
sequenceDiagram
    participant G as Guardian wallet
    participant V as akashi_vault program
    participant U as Holder
    Note over V: Leg 0 is stale: deposits and fundings fail with StaleValuation
    G->>V: suspend_leg(leg 0)
    V->>V: mode = Suspended, nothing booked
    U->>V: deposit / fund_withdrawal
    V->>V: leg 0 is skipped by the freshness check
    V-->>U: succeeds
    Note over V: Later, the protocol is back
    G->>V: refresh_leg(leg 0) + unsuspend_leg(leg 0)
    V->>V: mode = previous mode, value from the new report
```

*Figure: a suspension takes a silent leg out of the freshness check so the rest of the vault keeps working.*

### Example 3: a suspension keeps the vault running [#example-3-a-suspension-keeps-the-vault-running]

After the outage in Example 1, the guardian suspends the leg. The leg stays at 3,000, Junior at 1,000 and Senior at 4,000, and the vault stays Active. Alice withdraws 500 Senior shares and receives 500 USDC. In the tests on the real protocols, Bob's Junior deposit, which failed while the protocol was paused, now succeeds too.

### Example 4: Junior pays the cautious gap [#example-4-junior-pays-the-cautious-gap]

Suppose Junior holds 2,000 USDC instead of 1,000, and the leg last reported a book value of 3,000 with a 2% haircut, which means a low value 2% lower: 3,000 × 0.98 = 2,940. After the suspension:

- Senior leaves at full value: Alice receives 1,000 USDC for 1,000 shares.
- The 3,000 − 2,940 = 60 USDC gap comes out of Junior, so a Junior share is priced at (2,000 − 60) ÷ 2,000 = 0.97 USDC. 500 Junior shares pay 500 × 0.97 = 485 USDC.

### Example 5: the protocol comes back a little lower [#example-5-the-protocol-comes-back-a-little-lower]

The protocol returns, and the leg now reports 2,997. The guardian refreshes and lifts the suspension in one transaction. The 3,000 − 2,997 = 3 USDC drop is booked as an ordinary loss: Junior falls from 1,000 to 997. The leg is `Active` again. If it stops answering again, the vault stops again.

### Who can suspend [#who-can-suspend]

- A stranger who calls `suspend_leg` or `unsuspend_leg` gets `NotGuardian`.
- Only an `Active` or `CloseOnly` leg can be suspended (`CloseOnly` means money can be taken out but not added), and only a `Suspended` leg can be lifted. Anything else fails with `InvalidLegMode`.
- A suspended leg that turns out to be hacked can be marked the usual way. The mark replaces the suspension.

## Example 6: the guardian marks a silent leg instead [#example-6-the-guardian-marks-a-silent-leg-instead]

A silent leg sits next to a healthy one. Before anyone acts, even an exit from the healthy leg fails with `StaleValuation`. The guardian marks the silent leg at 0, which books it as a loss and moves the vault to Recovery, the state in which new deposits and investments stop while an incident is handled. The impaired leg no longer has to be fresh, so the executor can take 100 USDC out of the healthy leg, and Alice can withdraw 1,000 Senior shares at full value because Junior covered the loss.

Use a mark when the money is probably gone, and a suspension when the protocol is probably just down.

## Example 7: the protocol comes back after a mark [#example-7-the-protocol-comes-back-after-a-mark]

The guardian marked the 3,000 USDC leg at 0. Junior lost its whole 1,000 and Senior lost the remaining 2,000, so Senior fell to 2,000 with an impairment (a loss tracked for repayment) of 2,000. Then the protocol comes back whole.

1. The leg reports 3,000 again, but it stays capped at its mark of 0. A report alone does not undo a mark.
2. Any cash the executor takes out is recovery and repays Senior first.
3. The admin queues a `RaiseMark` change with `queue_change`. It waits for the governance delay, which is 1 day by default and set by the curator; executing it early fails with `TimelockNotElapsed`.
4. A partial raise to 1,000 books 1,000 of recovery, all to Senior: Senior goes from 2,000 to 3,000 and its impairment from 2,000 to 1,000.
5. A raise to the leg's full position books the rest. Senior is restored first, then Junior. The leg leaves `Impaired` and becomes `CloseOnly`.
6. The admin calls `resume`. The vault is Active with the leg still open. The executor can take money out of it, but cannot add more (`LegCloseOnly`) until governance re-opens it with an `UpdateLeg` change.

The guardian cannot raise a mark directly. Calling `mark_leg_impaired` with a higher number fails with `LimitNotLower`.

## Example 8: back before anyone acts [#example-8-back-before-anyone-acts]

If the protocol comes back before the guardian does anything, nothing is booked. The vault was stopped for a while, then the next refresh succeeds and everything continues. Alice withdraws 100 Senior shares and receives 100 USDC.

## Example 9: the protocol works but nobody can withdraw [#example-9-the-protocol-works-but-nobody-can-withdraw]

All the protocol's cash is lent out, Huma's instant-withdrawal allowance is used up, or the market pool an exit sells into is empty. The adapter still values the leg, and adds the `WITHDRAW_DISABLED` flag. This flag does not block the vault: only the executor's exit fails.

- The vault keeps running and pays withdrawals from idle cash. Alice withdraws 2,000 Senior shares and receives 2,000 USDC, all of the vault's idle cash.
- The next request waits: funding it fails with `InsufficientIdle` until cash returns.

The takeaway: a liquidity crunch delays withdrawals but does not change share prices or book a loss.

## Advanced details [#advanced-details]

- Suspended and impaired legs are exempt from the freshness check. The `Vault` account records them in its suspended and impaired leg masks (one bit per leg).
- `suspend_leg` stores the leg's previous mode in the `Leg` account. `set_leg_close_only` on a suspended leg changes that stored mode instead, so the leg becomes `CloseOnly` when the suspension is lifted.
- Suspended legs do not hold Recovery up: `resume` checks only impaired legs and Senior's impairment.
- The blocking flags are `PROTOCOL_PAUSED`, `DEBT_PRESENT` and `PRICE_UNCERTAIN`. `WITHDRAW_DISABLED` and `MARKET_STRESS` do not block valuation. In the report each flag is one bit of a number, with the values 1 (`PROTOCOL_PAUSED`), 2 (`WITHDRAW_DISABLED`), 4 (`DEBT_PRESENT`), 8 (`PRICE_UNCERTAIN`) and 16 (`MARKET_STRESS`).