Most tabletop exercises are designed not to find anything

The exercise runs for two hours. A scenario is read out, people describe what they would do, somebody takes notes, everyone agrees it was valuable, and an after-action report is circulated the following week. Nothing in the organization changes. The exercise is repeated annually because a policy or an insurer requires it, and it is treated as a thing that is passed.

An exercise that everyone passes has measured nothing. That is not a criticism of the people in the room; it is a property of how the exercise was designed. If the scenario admits an answer that the current plan handles, the room will find it, because that is what a room of competent people does under mild social pressure. The value is entirely in whether the design makes it possible to fail.

The scenario is not the scarce input¶

A common reason exercises stay shallow is the belief that designing one is expensive. It is not. CISA publishes over a hundred tabletop exercise packages (opens in a new tab) at no cost, covering ransomware, insider threat, phishing and industrial control system compromise, with sector-specific versions for water systems, healthcare, local government, ports and elections. Each one ships template objectives, a scenario, discussion questions, slides, feedback forms and an after-action report template.

So the scenario is free, and a firm charging you primarily for scenario authorship is charging for the cheapest part. What is scarce is a facilitator willing to push on the point where the room becomes uncomfortable, and an organization willing to let the exercise reach a decision it has been avoiding.

The two questions¶

In our experience nearly all the value in a first exercise comes from two questions, and neither is technical. Both reliably stall a room that believed it had a plan.

The first is who decides to stop production. In a manufacturing or utility setting, the containment action that actually limits damage is usually to disconnect or halt something that is making money or delivering a service. That decision is not the security function's to make. Ask, in the room, who has the authority, and then ask what happens if that person is on a plane, which is the version of the question that finds the real answer. We have watched a plant discover that the answer was nobody, and that everyone had assumed it was somebody else. That discovery is worth more than the rest of the exercise combined, and it cost one question.

The second is who decides whether to pay. It is uncomfortable, executives often prefer to keep it hypothetical, and keeping it hypothetical is exactly how it becomes a decision made at three in the morning by whoever is awake. The useful form is not what would we do, which invites a principled answer nobody is bound by. It is: who signs, what is the threshold, who must be told first, and does the insurer or counsel have to approve. Those have answers, and finding out they do not exist is the point.

Who is in the room¶

An exercise attended only by the security function tests the group that already knows the plan. The people who determine whether a response works are the ones who get called: operations, the plant manager, legal, finance, communications, and whoever owns the customer relationships. NIST's incident response guidance (opens in a new tab) puts preparation ahead of detection for reasons that are largely organizational rather than technical, and this is the clearest case of it.

Include the vendor question too, because in an OT environment the honest answer to how do we rebuild that controller is frequently that an integrator does it, and the exercise should establish whether that integrator answers the phone at the weekend and what the contract says about it.

What comes out of it¶

The deliverable is not the after-action report. It is a short list (five or six items is typical and realistic) of decisions that have no owner, with a name and a date attached to each. Two of them will usually be the questions above. One will be a piece of evidence nobody could produce, which is where an exercise turns into an engineering change: if the room cannot answer which accounts touched that share, the finding is a logging gap and it belongs in a backlog rather than in a report.

The honest limit is that a tabletop is a discussion, and people are better at describing what they would do than at doing it. It will not tell you whether your backups restore, whether your detection fires, or whether the phone tree works. Those need a real test, and a tabletop that claims to have validated them is overclaiming. What it does establish, cheaply and in one morning, is whether the decisions have owners, and that is the thing most often missing.

The test¶

At the end, ask what changed. If the answer is a report, run it again with a harder scenario and the operations people in the room. If the answer is that four decisions now have names against them and one logging gap is in somebody's backlog, it worked, and next year's should be harder still.