Can This Cell Restart? How to Plan a PLC and SCADA Recovery Drill

The SCADA screen is back. Before restarting the cell, the shift supervisor needs an answer: what has the recovery team proved?

Build a PLC and SCADA recovery drill around one failure scenario, written acceptance criteria and a defined timing objective. Use the results to show which functions work, how long recovery took and whether designated operations, safety and quality personnel have the evidence needed to authorize production.

In an interview with Anna Ribeiro, AMDT CEO Stefan Jesse emphasized that restored systems can leave operations unable to restart because the intended configuration, supporting dependencies and responsibilities remain unresolved. (Industrial Cyber, September 1, 2026)

The framework below is a proposed exercise design. Adapt its checks and approval roles to the cell and the site’s procedures.

Choose one failure and draw the boundary

Consider a hypothetical batch-mixing cell whose SCADA server has failed. For this exercise, assume the PLC and cell network remain available and the equipment is in an approved stopped state. The task is to restore supervision and demonstrate readiness to resume the selected production function.

Specify what that function includes: current process displays, authorized commands, the correct recipe and the operating sequence. Map the services needed to support it, including recipe storage or authentication where applicable.

For a controller-failure scenario, build the boundary around the replacement PLC and the functions it must recover. Check the role of remote I/O, drives, operator interfaces and adjacent equipment.

Keep one assumptions sheet for the whole exercise. It should name available equipment, simulated services, permitted assistance and exclusions. If the spare arrives preconfigured or a shared service remains outside the test, make that visible when reporting the result.

Agree on acceptance and timing before preparing the test

Give the team three decisions to support.

Configuration acceptance: Does the restored system match the approved baseline? Specify the project revisions, settings and comparison method. Resolve uncertain revisions through the site’s change process; obtain approval for a justified deviation or defer the affected test.

Functional acceptance: What must the system do, and what observation will count as a pass? For the hypothetical mixing cell, a recipe check might require the selected recipe identifier, revision and transferred setpoints to match the test specification. “Recipe screen opens” would address only one part of that test.

Production release: Which operating, safety and quality checks must be completed, and who has authority to accept them? Include applicable first-piece, batch or process checks from existing site procedures. Keep authorization to conduct controlled production tests separate from authorization to resume normal production.

Consider communication behavior within functional acceptance. Dominic Iadonisi and Joe Weiss describe a switch-replacement incident in which network disruption affected turbine controls and operator displays. Their account supports including application-specific communication checks in the plan. (Control Global, July 27, 2026)

Next, decide whether the drill establishes a baseline or tests an agreed recovery target. Scott Taylor distinguishes the business recovery objective from the duration demonstrated in a controlled exercise. (Automation.com, October 2, 2026)

Have operations set the target from the cell’s operating needs, with engineering and maintenance confirming the resources assumed. Fix the start event and endpoint before execution: for example, authorization to begin restoration through completed production-release approval.

A test that excludes detection, containment or procurement cannot establish an outage-to-restart objective that includes them. Report the narrower interval separately. When no target exists, label the result as a baseline and use it to inform planning.

For the hypothetical mixing cell, suppose the team chooses a three-hour drill target with replacement hardware already available. These figures illustrate the exercise design; they are not an industry benchmark.

Build a package the recovery team can use

Choose the configuration to restore with controls and operations together. Assemble the relevant controller projects, HMI or SCADA applications, device settings, recipe revisions and network documentation around that baseline.

Check the supporting installation media, engineering tools, runtime versions, firmware, licenses and hardware. Taylor’s practitioner account describes restoration obstacles involving compatibility, access, support and personnel despite the availability of backup files. (Automation.com, October 2, 2026)

Assign each package component to a named owner who can confirm its revision and availability under the scenario. Use established security procedures for recovery access; keep passwords and license secrets out of the exercise worksheet.

Aleksei Cherepov’s firsthand account of a legacy SCADA recovery describes incomplete documentation, dependence on old installation media and compatibility difficulties after server hardware failure. The installation was a research unit. Treat its experience as a prompt to check the local package, while verifying any migration path against the applicable vendor documentation. (Automation.com, September 22, 2026)

For the mixing-cell example, include the recipe service’s connection settings and the means of checking recipe currency. Establish how the restored server will reach the test service and how that arrangement differs from production.

Give each exercise stage its own result

Tabletop coordination tests decisions and handoffs. Walk through who authorizes restoration, arranges access, obtains equipment, resolves configuration questions and approves progression. Name the exercise lead and stop authority.

For a cyber incident scenario, ask the incident-response team to establish criteria for accepting backup copies and replacement systems, and for authorizing reconnection. Derive the restoration sequence from the application’s dependencies and have the responsible engineers review it.

Bench restoration tests the package on designated equipment or an approved isolated environment. Keep it isolated from live outputs and production control networks unless a separately approved test arrangement requires a connection. Mark simulated signals and substituted services in the test records.

In the hypothetical mixing-cell drill, suppose the application starts but recipe retrieval fails because a connection setting was omitted from the package. The startup check passes; the recipe check fails. Log the missing setting as a finding, correct the package through the appropriate process and repeat the affected checks.

A successful bench retest demonstrates the functions represented by that setup. Leave checks involving actual process equipment explicitly untested.

Authorized production validation establishes evidence under the site’s maintenance, change-control and operating procedures. Plan the test window, equipment state, stop conditions and rollback arrangements. Have qualified personnel perform applicable safety-function checks using approved procedures.

Only the designated approvers can authorize normal production after reviewing the required results. If this stage cannot be performed, report bench restoration as demonstrated and production release as untested.

Connect the worksheet to actual acceptance decisions

Use a compact worksheet as an index to the evidence. The entries below are prompts; replace them with specific revisions, named people and references to completed test records. Use the rows relevant to the chosen scenario.

Each referenced test record should contain the exercise stage, predefined criterion, expected result, actual observation, pass/fail/not-tested status, evidence location and assessor. This supplies the decision detail without turning the worksheet into an unreadable table.

For the example’s recipe test, retain the failed bench result alongside the successful retest. Add a separate production-validation record when that check occurs on the actual system.

Classify outstanding findings by their effect on release. Mandatory checks must be completed; other issues need an explicit disposition from the appropriate authority, with any restrictions stated. An empty findings cell should never stand in for a release decision.

Compare the result with the target and close the gaps

During execution, capture component-availability checkpoints, completion of functional checks and production-release approval. Separate hands-on work from waiting for equipment, access, specialist support or decisions.

Show concurrent tasks on one timeline. Do not add overlapping task durations to calculate overall elapsed time. Keep tabletop and bench rehearsal times separate from a later production-validation run.

Suppose a later end-to-end run for the hypothetical mixing cell takes three hours and twenty minutes, measured against the same start event, endpoint and resource assumptions as its three-hour target. All release checks pass and the authorized personnel approve production. Technical acceptance is complete, but the timing objective was missed by twenty minutes.

Report both outcomes. Review the delay and assign corrective work rather than changing the clock boundaries afterward. If the run used extra specialist assistance, disclose it; a different shift’s readiness needs evidence under its expected staffing.

For every failed or incomplete check, assign an owner, a corrective action and the evidence needed for closure. Repeat affected tests after correction, assess whether broader validation is necessary and update the restoration package. Set review triggers for relevant equipment, software and process changes.

End with a statement the plant can substantiate: which configuration was restored, which functions were demonstrated, the elapsed time against the stated target, unresolved restrictions and who authorized production release. That is the useful deliverable from the drill.

Sources consulted

1. The cited passages were checked against the original publications during this rewrite.
2. Scott Taylor, Automation.com, October 2, 2026. “A Backup Is Not a Recovery Plan: Measuring What It Really Takes To Restore OT After an Incident.” Practitioner discussion of restoration prerequisites, measured exercises and recovery targets.
3. Aleksei Cherepov, Automation.com, September 22, 2026. “When the Server Dies and the Backup Is Three Years Old: Reviving a Legacy Biturox SCADA System.” Firsthand account involving a research installation and server hardware failure. Product-specific compatibility and migration claims were not independently validated or adopted as instructions.
4. Anna Ribeiro, Industrial Cyber, September 1, 2026. “From IT restoration to production restart: Why manufacturers continue to struggle with operational side of cyber recovery.” Original interviews on operational recovery. The article attributes the relevant observations to AMDT CEO Stefan Jesse; interview contributors include vendor representatives.
5. Dominic Iadonisi and Joe Weiss, Control Global, July 27, 2026. “When digital communications became safety-critical: How a network switch failure affected control system logic and operator situational awareness.” Authors’ incident account informing communication-validation questions. The underlying NRC inspection report was not reviewed, and the incident findings remain attributed to the authors.