Arizona outage highlights need for strong network design, monitoring capability in critical communications

Last week, much of northern Arizona was unable to access the Internet, call 911, use cellular devices, pay with credit cards or get cash from ATMs for about 15 hours, according to multiple news reports. Why? A single conduit with multiple fiber lines was cut.

Of course, a communications line being cut is nothing new. Almost everyone with experience in communications has at least one war story about such a problem, whether it was caused by Mother Nature, an unintentional mistake by the infamous “Backhoe Bob” or an intentional act by vandals (the suspected issue in Arizona).

What makes the Arizona fiber cut unique is the extent of the outage. Modern network design and IP-routing capabilities are supposed to prevent this sort of thing from happening. When a line is cut, some outages in the immediate area are expected, but most traffic automatically should be rerouted to an alternative path and reach its destination. There may be degraded network performance because of the lost capacity caused by the disruption, but information still can be transmitted.

Media reports indicate that CenturyLink—the carrier that owns the affected fiber conduit—was planning to deploy a conduit of backup fiber lines to serve northern Arizona by the end of this year. Until then, the vandalized conduit is the only one to serve northern Arizona, a spokesman for the carrier said in one story.

In addition, it took some time for CenturyLink to find the damage, because workers had to inspect the conduit “mile by mile” before they discovered the cut, according to numerous reports.

Thankfully, it does not appear that any seriously negative consequences resulted from the Arizona outage. Most media reports focused on annoyances and inconveniences. Some people had to delay purchases because credit cards were not being accepted and cash was not available from ATMs. Students had to resort to other measures to conduct research to send completed assignments to teachers.

But it could have been worse—much worse.

Imagine what could have happened if a significant emergency event had occurred during this outage period. With 911 service unavailable in some areas, just notifying authorities of the problem could have been a challenge. Meanwhile, the outage would have made it much more difficult to coordinate the response, especially if it called for a significant evacuation of people whose primary sources of communications were down.

And, without the prerequisite connectivity, many cloud-based applications are worthless.

This episode should serve as a reminder of the necessity for characteristics in mission-critical and business-critical networks that cannot afford downtime.

Redundant paths—preferably via disparate technologies, such as fiber and high-speed wireless links—are a must from key nodes in the networks, and network designers have to take the time to ensure that diverse paths don’t eventually lead to the same conduit without an alternative route, which leads to potential single point of failure. All too often, network operators pay for backhaul links from separate providers to ensure redundancy but later discover that traffic from both links are carried in the same physical conduit, which undermines the purpose.

While this approach may not be economically feasible at all points in the network, key network sites—those that guarantee coverage in an area—should be physically hardened and have ample power backup, in case commercial power is unavailable.

Meanwhile, when outages happen, it is important to have systems established that allow the technical staff to identify the problem quickly—probably via sensor technology—and provide them with the resources necessary to resolve the issues as soon as possible.

These are the standards that FirstNet and other critical-infrastructure entities must meet, along with heightened cybersecurity for both the network and the data that is transmitted on it. For a commercial carrier like CenturyLink, a 15-hour outage is bad day at the office. If a similar outage occurs on FirstNet, there’s a very good chance that lives will be lost.