Give the Water Plant a Network It Can Live With

Water treatment facility network and control cabinets at sunset
In this Field Note
  1. Start with the consequence of losing communications
  2. Build a permitted-flow list people can read
  3. Make vendor access a controlled visit
  4. Stage changes around the plant
  5. Include the remote site and the spare
  6. Leave the utility in control

At a water utility, a network fault arrives as an operational question. Is the tank reading current? Can the operator reach the remote site? Is the vendor connected to the right system? What continues to work if this link disappears?

Those questions deserve better answers than “the firewall looks fine.”

In municipal water communications work, I care about whether the operator can understand the system and recover it. A small district needs useful boundaries, reliable communications, and ownership of the finished network. It should not need to become a large IT department to operate what it has bought.

Start with the consequence of losing communications

Sit down with the operator and controls specialist before changing the network. For each important site or service, establish what communications carry, how current the information must be, and what the process does when the connection is lost.

A delayed historical upload, a stale display, and the loss of a control path are different events. Document who determines the safe operating response. Network reachability alone does not tell you the state of the process, and a networking contractor should not invent that answer.

Build the initial inventory from existing records, supported configuration views, vendor information, and approved observations. NIST’s OT security guidance explains the special reliability and safety constraints involved and the risks of active discovery against sensitive equipment. An assessment should not begin by pointing a general-purpose scanner at unfamiliar controllers.

Keep process-control decisions and intrusive testing with the authorized people. If there is evidence of an active compromise or an unsafe process condition, use the utility’s incident and operational response procedures rather than treating the event as routine network cleanup.

Build a permitted-flow list people can read

Before choosing VLAN numbers or firewall appliances, list the communications that are needed. The following is an illustrative planning worksheet, not a ready-made policy for a particular plant:

Source and destination Question to settle Evidence or owner
Operator workstation to control-system service Which systems and functions are required during normal and degraded operation? Operator and controls-vendor confirmation.
Remote site to the central system What traffic, timing, and return path are required? What happens when it stops? Controls documentation and approved observation.
Vendor access gateway to support target Which target, account, access window, and approved maintenance task? Named authorizer and vendor scope.
Network management to infrastructure Who may administer, monitor, and collect configurations? Utility account ownership and management policy.
Office or guest devices toward operational systems Is any access actually required, and through which controlled service? Explicit business justification; no inherited general access.

Add exact addresses or device groups, protocols, ports, direction, owner, and justification in the utility’s controlled version. Include less-visible dependencies such as time, name resolution, licensing, and approved update services. “It needs the internet” is a question to investigate, not a useful final rule.

Separate office, guest, management, and operational traffic according to those requirements. A VLAN does not, by itself, decide which routed traffic can cross the boundary. Verify the routing and enforcement point, including any alternative paths that bypass the intended firewall.

Make vendor access a controlled visit

A support vendor may need remote access without needing permanent, unrestricted reach across the district. Establish an identifiable account, an approved access path, a defined destination, and a way to end the session or revoke access.

The joint agencies’ primary OT mitigations prioritize removing unnecessary public exposure, replacing default passwords, protecting necessary remote access, and separating IT from OT. These are practical review items, not a claim that a particular network is certified or compliant.

Where the remote-access system supports it, use strong multifactor authentication, preferably phishing-resistant methods. Older operational equipment may need protection at an appropriate gateway or controlled support environment rather than changes it cannot support itself.

Test the whole visit: authorized access begins, the required task is possible, unrelated destinations remain unavailable, the session is recorded appropriately, and access can be closed without disturbing the process. An old shared account that nobody knows how to disable is an ownership problem as well as a security problem.

Store credentials in the approved vault. Give the utility control of the accounts and recovery methods, and document how a change in vendor or personnel is handled.

Stage changes around the plant

Translate the approved flow list into a design, then validate it away from live operations where practical. Rehearse device configuration, management access, and rollback using representative equipment or a suitable model. A lab can expose a rule error; it cannot establish every timing requirement of the installed control system.

For the operational change, agree on the window, authority, sequence, observation, stop conditions, and rollback with the operator and controls team. Make one bounded change at a time. The operator should verify the relevant process indication or approved transaction; an engineer’s ping test is supporting evidence, not operational acceptance.

Verify prohibited paths too, using safe checks the team has approved. Retain a short acceptance record describing what was demonstrated and what remains an assumption. The goal is a supportable boundary, not a large set of rules that nobody is willing to touch.

Include the remote site and the spare

A weak power supply or failed backhaul can remove visibility just as effectively as a policy mistake. Include radio condition, power, interface errors, access, and recovery dependencies in the network plan. Treat missing telemetry as missing evidence until the responsible people establish what has happened.

Use dependency-aware monitoring so one failed transport does not bury the operator in a collection of unrelated device alarms. Keep a clear distinction between loss of network observation and an alarm produced by the operational system itself.

Do not automate process-equipment resets in response to missing pings. Even a communications-device power cycle needs an understood purpose, authorization, and known effect on the site.

Prepare a replacement for critical network equipment with compatible software, protected configuration files, needed licenses or certificates, labeled connections, and a tested recovery procedure. Check that authorized staff can obtain those materials when the normal network is unavailable.

Leave the utility in control

The useful handoff is concrete: accounts the utility owns, an understandable design, the permitted-flow list, protected backups, a demonstrated restore, and named responsibilities. Optional monitoring and occasional engineering support can then be chosen for their value, not because the original installer kept the only map.

For a first internal step, pick one remote site and one vendor-access path. Establish their owners, dependencies, and recovery methods. If those answers are uncertain, they define a sensible scope for outside help.

Oaktown Tech’s water utility work covers network assessment, design, implementation, and handoff. Controls programming, formal certification, and specialist incident investigation need their own agreed scope and expertise. The network should give the operator reliable information and manageable recovery, then let them get on with running the plant.

Share this article

LinkedInFacebookXEmail

Field discussion

What would you test next?

Ask a question, challenge an assumption, or share a lesson from the field. Keep customer names, live addresses, credentials, and sensitive network details out of public comments. Charles reviews every comment before it appears. See the privacy notice for how comment information is handled.

Join the discussion

Your comment will appear after Charles reviews it. Required fields are marked .

A pseudonym is fine.

Working through a similar problem?

Describe the network, the difficulty, and the outcome you need. Charles reviews the brief and helps establish the right scope.