In a hypothetical packaging-cell recovery test, the supervisory application opens and the controller connection appears healthy. Then a simulated equipment fault produces no operator alarm. Has the function recovered?
For an operational technology (OT) recovery test, answer that question through acceptance criteria established before work begins. Choose one production function, specify its minimum operating capability and set a recovery-time objective. Assess technical restoration, operational validation and the production-release decision separately.
The worksheet proposed here is an editorial planning aid. It organizes scope, dependencies, observations and acceptance decisions around one function. Its usefulness depends on the site’s technical and operating teams choosing meaningful checks and interpreting the results within the conditions tested.
Set the boundary and recovery objective
Begin with the function that operations needs back. In the packaging-cell example, the scope is the supervisory control and data acquisition (SCADA) function for cell 3 following a server failure.
For this hypothetical test, the minimum capability includes displaying selected equipment states, presenting specified alarms and recording the associated events. The site sets a six-hour objective from recovery authorization to readiness for a production-release decision. Those functions and that deadline are illustrative; each site must establish its own operational needs.
The test uses an isolated replacement server and a controller simulator. Its plan identifies the interfaces represented by the simulator, the equipment excluded from the test and the resources available at the start. Live equipment behavior and product quality remain outside its scope.
Put those boundaries on the worksheet alongside the objective. They govern every later acceptance statement. An offline result establishes performance under the test conditions; actual production release requires the remaining site checks.
The plan should also identify the coordinator, participating disciplines, permitted equipment interactions and stop conditions. Use replacement of a failed server as the initiating event for this example. A cyber-incident exercise needs separately defined assumptions and acceptance criteria developed with the incident-response team.
Assemble the package needed to recover
Scott Taylor’s practitioner account describes restoration delays despite available backups and emphasizes examining the conditions required to complete recovery. It provides a reason to investigate local dependencies, without establishing how frequently those problems occur across industry (Automation.com, October 2, 2026).
For cell 3, turn “the backup is available” into a reviewable inventory. The recovery package should identify the intended configuration, its supporting files and the tools required to restore it. Relevant entries might include application projects, installation media, compatible software versions, licenses and replacement hardware.
Aleksei Cherepov’s account concerns a legacy Biturox SCADA research unit and a three-year-old backup. It describes recovery work involving old archives, installation media and compatibility problems. That context makes it useful as a dependency example, without turning it into a general migration procedure (Automation.com, September 22, 2026).
Have the designated recovery personnel demonstrate that they can retrieve the package and use the required tools under the scenario’s assumptions. Document access arrangements without placing passwords or other secrets in the worksheet.
External assistance belongs in the inventory too. Name the support contact, the expected contribution and when it becomes necessary. Distinguish assistance actually obtained during the test from availability that the team merely assumed.
For the packaging cell, a useful preparation question is specific: could another qualified controls engineer recover the selected function using this package and its instructions?
Separate three acceptance decisions
Anna Ribeiro’s reporting examines the work between restoring technology and resuming production, including operational dependencies, validation and responsibility for restart decisions. These observations support making the decisions explicit; the three milestones below are this article’s proposed organizing framework (Industrial Cyber, September 1, 2026).
Technical restoration establishes that the recovered platform is available for functional testing. For cell 3, the controls engineer checks the installed application and configuration identifiers, successful startup and basic data exchange with the simulator. The acceptance record identifies exactly which checks passed.
This milestone concerns the restored components and their technical connections. The operational meaning of the data is assessed next.
Operational validation establishes whether the required operating functions behave as expected. In the example, operations and controls compare displayed states with the simulator’s known states, trigger the specified test faults and examine the resulting alarms and event records.
A functioning connection satisfies one technical check. Displaying the correct equipment state and presenting the required alarm satisfy different operating checks. Each needs an expected result, an observation and a reviewer.
At this stage, the missing alarm prevents acceptance of the supervisory function. The team has demonstrated some capability, but it has not met the minimum operating requirement.
Production release is the site’s authorization to resume production. Name the role with that authority and the approvals required by site procedures. Technical and operational reviewers supply their findings; the release authority evaluates them with the other required checks.
In the offline example, the team rehearses this decision. Its record identifies the live validation still required before an actual release can be considered.
Put the acceptance decision in the worksheet
A useful worksheet should let another qualified reviewer reconstruct the decision. Include the production function, scenario, recovery objective, configuration identifiers, dependencies, acceptance checks, elapsed times and corrective actions.
The following completed entry shows how to document a failed operating check. Every detail is hypothetical.
| Field | Completed example |
|---|---|
| Function and test setup | Cell 3 SCADA fault annunciation; isolated server connected to a controller simulator |
| Expected result | Changing the simulated cell-fault state from healthy to fault produces the specified operator alarm with the correct cell identification and an event-record entry |
| Observed result | At 2 hours 20 minutes after recovery authorization, the status indication changes, but the required alarm and event entry are absent |
| Evidence retained | Test-step record, simulator-state capture, operator-display capture and event-log extract |
| Reviewer | Operations representative, supported by the controls engineer |
| Acceptance decision | Failed: withhold operational acceptance because a required alarm function has not been demonstrated |
| Corrective action and closure evidence | Controls engineer to resolve the discrepancy, document the correction and repeat the affected checks; retain the successful results and reviewer’s acceptance |
This entry ties the decision to an observable result. “Alarm problem” would leave the next reviewer to determine what failed, how it was tested and why it mattered.
Continuing the hypothetical example, the controls engineer finds that an alarm configuration file was omitted from the recovery package. The correction includes the approved file, an updated package inventory and repeated technical checks affected by the change. The team then repeats the fault and clearing checks and captures the results.
Operational acceptance follows only when the specified functions pass. The record preserves the original failure as well as the successful retest.
Run the test and measure recovery time
Before starting the clock, settle the starting condition. For cell 3, recovery authorization is the common reference event. The worksheet states whether replacement hardware, installation media and support personnel are already available.
During execution, an observer records active work, waiting time and unexpected assistance. If an engineer supplies an undocumented workaround, capture what prompted it, who authorized it and which checks followed. Review its suitability before adding it to the recovery instructions.
Suppose the initial technical checkpoint passes at two hours, the alarm discrepancy appears at two hours 20 minutes, and operational validation—including the correction and affected retests—finishes at five hours. The rehearsed release decision is completed at six hours.
All times are cumulative from the same starting event. Retain separate timestamps for technical checks repeated after the correction; the two-hour checkpoint does not describe the final configuration by itself.
Compare the result with both parts of the objective: elapsed time and minimum operating capability. Report the offline test’s six-hour result alongside the remaining live checks. It does not establish the full time required to resume production.
Preparation also needs visibility. If hardware procurement or server staging occurred before the clock started, list that work as excluded and assign a separate measurement where necessary.
Close gaps with evidence and a retest
Review each discrepancy against the original acceptance criteria. A useful corrective action names the affected function, the work required, an accountable owner, a due date and the evidence needed for closure.
For cell 3, “update the backup” is too vague. A better action is to add the approved alarm configuration to the recovery package, update its inventory and verify that the designated team can restore the function from those materials.
A subsequent targeted test should start from that revised package. This tests the documented recovery method, rather than relying on a server that was repaired during the first exercise.
Changes to equipment, software or operating requirements should prompt a review of the worksheet’s assumptions. The technical and operating owners can then determine which checks need repeating and whether the scope remains appropriate.
For the next planning meeting, choose one production function and bring its controls, maintenance and operations owners together. Agree on the required capability, the recovery objective and the acceptance authority before scheduling the test.
Leave the review with a precise account of what the team demonstrated, the conditions under which it did so and the work still needed before production can resume.
Sources consulted
- Scott Taylor. “A Backup Is Not a Recovery Plan: Measuring What It Really Takes To Restore OT After an Incident.” Automation.com, October 2, 2026.
- Anna Ribeiro. “From IT restoration to production restart: Why manufacturers continue to struggle with operational side of cyber recovery.” Industrial Cyber, September 1, 2026.
- Aleksei Cherepov. “When the Server Dies and the Backup Is Three Years Old: Reviving a Legacy Biturox SCADA System.” Automation.com, September 22, 2026.