Scenario
Your approver is on a plane
One night, minute by minute. The people and the incident are made up. Everything the product does in it is real, down to the words on the screen.
The org has break-glass turned on, with the default thirty-minute wait. The package in
question, payments-prod-read, requires one approval and grants access for four
hours. It has one named approver, who is somewhere over the Atlantic.
02:07 — the page
The payments API starts returning 500s. The on-call engineer is paged, opens the dashboard,
and requests payments-prod-read with a one-line comment: "Paged for INC-4821,
need to look at the queue." On the Worker's next ten-second pass, the approval email
leaves for the approver's inbox, where it will sit, unread, until the plane lands.
02:08 to 02:37 — nothing
This is the part every on-call engineer knows. The request is correct, the control is working, and the queue does not move. They do what they can without the role, which is not much. There is no standing admin to fall back on in this org, on purpose, and nobody is going to wake the CTO to click a button.
02:37 — the glass
Thirty minutes after it was raised, the request changes on the engineer's own Approvals tab. The card turns red, gains a Break glass tag and the words still unanswered, and offers exactly one button. There is no checkbox: you cannot bulk-approve your own bypasses.
The button opens a confirmation, and the confirmation is written to be read rather than clicked through:
Nobody has answered this request, so you are about to approve it yourself. That is allowed here — and it is recorded as exactly what it is.
- Access is granted straight away, and still expires on this package’s normal schedule.
- Your organization’s admins and this package’s approvers are emailed immediately.
- It is written to the audit log and listed in the Break-glass report, permanently.
Why do you need this now?
The Continue button stays disabled until the reason is at least twenty characters long, because a required field with no floor gets answered with "x". The engineer writes one real sentence. Continue leads to a second, shorter screen:
Last check. You are approving your own request for payments-prod-read. The moment you confirm, the emails go out and the entry appears in the report — there is no way to un-send them.
Under it is the reason again, headed Your reason, as it will be sent, so the engineer reads their own sentence the way the approver will. Back returns to the first screen with the text intact.
02:38 — Yes, break the glass
The card disappears from the list. The decision is recorded as an ordinary approval with a break-glass marker, and from here the product does what it does for every approved request. Within seconds the Worker adds the engineer to the package's groups and schedules the removal for four hours out. On its own ten-second pass it emails the approver on the plane and every admin in the org. This is the whole of what they receive:
Subject: Break-glass used: payments-prod-read (Example Co)
oncall@example.com approved their own request for the access
package 'payments-prod-read' in Example Co.
This is a break-glass approval. The request had been waiting 31
minutes with no decision from an approver, so your organization's
settings allowed the requester to proceed without one.
Their stated reason:
INC-4821: payments API returning 500s since 02:04. Need read on the
payments queue to find the message that is poisoning it.
On the original request they wrote:
Paged for INC-4821, need to look at the queue
The access is time-bound like any other grant and will be removed
automatically about 4 hours after it was given.
If this was not legitimate, remove the access now from the dashboard
— early removal is always available and is never blocked.
Every break-glass is listed under Reports → Break-glass usage.Three things are deliberately missing from it. There is no Approve button, because there is nothing left to approve. There is no "click to confirm this was fine", because a notice that needs an answer to be complete is a notice that fails when nobody answers — which is the situation it was sent in. And there is no softening. It says what happened and what to do if it should not have.
02:40 — an admin reads it
One of the org's admins is also on call tonight and sees the mail arrive. They read the reason, recognize the incident, and go back to the incident channel. Had they not recognized it, the fix was one click away and nothing could have stood in front of it.
03:22 — resolved
A malformed message is found and moved aside; the API recovers. The engineer goes back to bed. Nobody thinks about the access again, which is normally how access outlives the night it was needed for.
06:38 — the access removes itself
Four hours after it was granted, the Worker removes the engineer from the package's groups, the same way it would have removed an approved grant. Nobody is awake. Nobody needs to be.
09:15 — the plane lands
The approver turns their phone on to two emails: the approval request from 02:07, and the break-glass notice from 02:38. The request no longer needs them. They open Reports → Break-glass usage and find one row: the engineer, the package, 31 minutes waited, access ended, the removal time, and the reason, in full.
That is the entire aftermath. No password to rotate, no group membership to remember to undo, no standing admin to explain at the next audit. There is an email and a report row, and the access has already gone.
The same night, without it
Run the night again without break-glass and the engineer's options at 02:37 are the familiar ones: somebody with standing admin does it for them, the shared emergency credential comes out of the vault, or the incident waits for a plane. The first two work. Both leave something behind that somebody has to notice and undo, and on the morning after an incident, nobody does — which we went through option by option in break-glass vs. standing admin vs. a shared root password.
Break-glass is not free, and it should not be on for everyone. What it costs you is written down too.