An Avoidable Accident at a Water Treatment Plant

Accidental Keystroke Leads to Water Crisis

“To err is human.” A true statement that is often followed by a domino effect of major consequences. The Orange Water and Sewer Authority (OWASA) in Chapel Hill, NC recently experienced first-hand how a slight mistake can lead to a “worst-case scenario”. The municipal water treatment plant was affected, as well as 80,000 residents in North Carolina’s Orange County area.

A recent article detailed how a distracted water treatment plant operator unintentionally hit a button. This signaled a fluoride feed pump to increase the levels of fluoride in the water. What is normally an 8-12% operating speed for the pump was increased to 80% for approximately three hours.

Authorities discovered that the plant operator noticed disparate fluoride levels at one point and went to review the data to make adjustments. That operator then became distracted and forgot to check the issue, then making a computer keystroke that unintentionally triggered the inciting event.

The Consequences of Out-of-Date Methods 

While OWASA officials established “that no water with increased fluoride levels made it into the drinking water for OWASA customers,” this breach did trigger an emergency water interconnect from the city of Durham. The following day, a water main funneling the water from Durham had a break. This forced a Do Not Drink/Do Not Use order for southern Orange County residents and businesses for more than 24 hours.

The Health Department responded by closing all hotels and restaurants served by OWASA. They even postponed and relocated the UNC men’s basketball game against Notre Dame. In all, nearly 80,000 people were affected by this accidental error that quickly ballooned.

Could This Error Have Been Avoided?

Once each of the subsequent issues was managed and under control, investigators began their search for the how to best prevent a problem like this from happening in the future.

The water main break was the result of outdated and old infrastructure. Simply put, the water treatment plant was not up to standards for the increased water supply. Additionally, water levels were checked manually. This is no longer necessary. Modern technology, such as Cloud-Based SCADA Monitoring Systems, is widely available. SCADA provides real-time, accurate readings of water levels and other relevant data. The last issue investigators uncovered was that training procedures for employees were not as compliant or practical as necessary.

All of this added up to a huge accidental error. Fortunately, the consequences were kept to a minimum. But the whole situation could have been easily avoidable with updated technology and practices.

Web & Cloud-Based SCADA Systems

The most important takeaway from this event is the need for better monitoring and alert systems. Water treatment is vital to public safety. Effectively monitoring these practices allows for accurate data collection, as well as a safety net for when a disruption occurs to regulated output.

If OWASA had updated their equipment and made use of a cloud-based SCADA monitoring system, the appropriate operators and departments would have been notified. The SCADA system would have alerted decision makers about the high fluoride levels in the water, allowing them to review real-time data about the situation. Additionally, it would have alerted operators that the emergency water main specs were out-of-date and too short for interconnecting Durham’s water.

A cloud-based SCADA system does not rely on the physical presence (or aptitude) of operators. Instead, it records and transmits data in real-time, making it accessible to the proper departments from anywhere that has an Internet connection.

Accidents happen. But learning from mistakes and taking steps to eliminate repeated problems is the best outcome for scenarios primed to have a domino effect. Humans make errors, but humans can also learn from their mistakes.

How SCADA Systems Help Water Treatment and Wastewater Plants

The beauty of modern plumbing is that when we turn on our faucet, we can reasonably expect freshwater to come pouring out. There are few places left in the country that are untouched by the advancements and regulations of public water. So when the Washington Post published an article in 2016 about researchers finding unsafe levels of industrial chemicals in the drinking water of 6 million Americans, wastewater treatment plants and water management services took notice.

The report cited in the Post article found that “194 of 4,864 water supplies across nearly three dozen states had detectable levels of the chemicals.” Of those water supplies, 66 services had at least one sample that exceeded the EPA’s recommended safety limit for two types of chemicals. That ratio might not seem like a high number, but 66 water services affect six million Americans, so it’s clearly not a small issue.

Water treatment plants and water usage facilities can do their part by making sure their monitoring equipment is up-to-date and as accurate as possible. While SCADA (supervisory control and data acquisition) systems are more commonplace in modern operations, an updated version exists and is proving to a more reliable and better solution: a cloud-based SCADA system.

A cloud-based SCADA system allows water management plants to not only monitor levels of specific chemicals and toxins but to have precise records accessible from anywhere. No longer are digital read-outs only available at a fixed point on the SCADA unit. Instead, any manager or operator who needs data can access it from their own satellite- or WIFI-enabled device.

In the contaminated water study, the EPA sought to mitigate the ramifications until stricter guidelines could be drawn up. When it comes to healthy drinking water, Americans don’t want to waste time in the bureaucratic process of defining regulations.

Unfortunately, Congress mandates that before the EPA imposes new limitations on the nation’s water utilities, it has to prove that there is a meaningful opportunity to improve public health. It is a long, arduous process that takes years; officials have not successfully regulated any new contaminant in two decades because the process is complicated and contentious.

Another benefit of a cloud-based SCADA system is that data collected in real-time from the contaminated areas can be studied, compared, and shared with researchers in a faster, more efficient, digital manner. By comparing the data points, researchers can have the most accurate knowledge from which to draw, and that hopefully can lead to quicker results and faster action.

Because of our industrial advancements, the environment is changing faster than we can understand. However, because of our technological advancements, we can use the digital tools available, like a cloud-based SCADA solution, to monitor, record, and support research for improvements.

Remote Monitoring of Distribution System Operations at Pressure Reducing Valve Vaults

The City of Barrie, Ontario water operations services approximately 147,000 customers. The water operations consist of a single low-lift pumping station that supplies a surface water treatment plant as well as a network of several groundwater wells, in-ground storage facilities, booster stations, and elevated storage tanks. As with most water systems, the focus has been on gathering monitoring and alarm data from these major operations points. However, the water is pumped through miles of water distribution piping, often with little or no consistent monitoring of the overall system health.

Over the last few years, the City of Barrie has been adding monitoring to several locations in the system using the cloud-based SCADA system from High Tide Technologies. This system enables the water operations team to get a better idea of the overall condition of the entire distribution system. The most advantageous place to measure pressures and flows has been at pressure reducing valve chambers. The reason for this is that the pressure reducing valve is located on the border of two pressure zones. By placing a pressure transmitter on each side of the valve the operations personnel can monitor the pressures in each zone. These chambers are also usually roomy enough to accommodate some extra equipment.

The installations are quite simple. Insertion flowmeters can be installed into some brands of pressure reducing valves. These flow meters require no changes to existing piping and will provide flow data suitable for analysis of system health. Many times these chambers will not have power. If that is the case, power generators are available that will use the water flowing through the piping, along with the pressure drop, to generate electric power. This is not a significant amount of electric power, but enough to run instrumentation and the High Tide Technologies remote terminal unit (RTU). The picture below shows the inside of the PRV chamber with the High Tide Technologies RTU.

The City of Barrie has nine pressure reducing valve chambers using High Tide Technologies RTUs. Eight of the chambers have power supplied to them. One uses the power generator described above. For more than three years, the power generator, the High Tide Telemetry unit, and the instruments have been providing data and alarms to the Barrie operators. This information allows them to better understand their system, ensure that it is working properly and it allows them to respond to unusual occurrences quickly.

Dave Truax, Supervisor of Ground Water Supply for the City of Barrie, says, “High Tide telemetry units allow us to collect data and to alarm our system in locations where we would otherwise have no monitoring capabilities. Traditional SCADA is expensive to install and to maintain, but High Tide units provide us with many of the benefits of SCADA at a much lower cost. The units have proven themselves to be reliable and we have improved our system performance thanks to the information supplied by High Tide’s system. One of the biggest advantages is their ease of installation – it is virtually plug & play. Just about anyone can install them without the need for extensive training.”

Because the High Tide Technologies system is a “cloud-based” system, the operations staff have access via a secure portal from any Internet-connected device. Through this portal, they can look at a snapshot of system status or drill down and look at trends for pressures and flow rates. Some examples are shown below.

The advantages of this system are extensive. The operators receive alarms when pressures or flow rates are out of range that will alert them to possible leaks. They can look at trends to see if the system is functioning correctly and if the PRVs are set appropriately for that particular time of year. All of this without having to physically visit the valve vaults especially during those cold winter months.

Monitoring and Controlling Bypass Pumps with Cloud-Based SCADA

Milcrofton Utility District, located in Eastern Williamson County, Tennessee, provides clean drinking water to over 6,700 customers. The utility district maintains 232 miles of water mains, 13 water storage tanks, and 10 water pumping stations.

Dramatic population growth in this area necessitates creative solutions to keep up with demand. The Avalon Water Booster Station pumps to a water storage tank that services approximately 400 homes. Growth and irrigation use during the summer months left the existing pumps at this station undersized. Milcrofton Utility District determined the existing skid-mounted water booster pumps needed to be replaced. During installation of these new pumps, Milcrofton would need to utilize a bypass pump to provide continuous water service to the Avalon area.

During a 30-day construction period, the bypass pump would require a design condition provided by the Utility (500 gpm at 323′ TDH). This bypass pump would also require a telemetry system to ensure the Avalon Water Storage Tank would fill and drain appropriately. Adding this system to the bypass pump would reduce the need for on-site supervision and allow remote control, freeing up resources to work on the new installation. WASCON, Inc., representing

High Tide Technologies, was asked by Milcrofton Utility District to provide a proposal for the project. After reviewing the project conditions, a 6×4 diesel driven standard centrifugal pump was selected. This pump was fitted with a LOFA control panel ideal for pairing with the High Tide Technologies (HTT-1100) telemetry system.

High Tide Technologies, based in Nashville, TN, enables users to create a complete SCADA solution utilizing field units, satellite, cellular, or Ethernet communications as well as the Internet to monitor and provide automatic control. High Tide also offers a web-based software called TelemetryVIEW, allowing the user to view data and initiate manual controls from any Internet-connected computer or mobile device.

For the Milcrofton project, HTT-1100 units were installed at the Avalon tank and on the bypass pump at the Avalon booster station. Set points for when the pump would turn on and off were determined and programmed into High Tide’s TelemetryVIEW website. The HTT-1100 unit at the tank also provided water elevation data. David Pine, Application Engineer with WASCON stated, “I was most impressed by the ease of working with HTT and how quickly the site could be customized to meet the customer’s needs.” Milcrofton Utility District had a solution in place that both avoided service interruption and decreased the need for on-site supervision.

 

Monitoring and Controlling Bypass Pumps with Cloud-Based SCADA

 

A week into the project, a local resident requested that operation of the bypass pump be limited during night hours. David Mundie, President of High Tide, was notified of the request. After a simple programming modification and installation of a relay unit at the HTT-1100 on-site, the bypass pump would no longer operate between 11 P.M. and 7 A.M. without manual override. Mundie says, “Traditional SCADA requires a lot of infrastructure to function, which would not be available at a temporary site. Because of the ease of installation and the nature of our various communication options, cloud-based SCADA enabled the customer to keep automatic controls functioning even with their system in a bypass mode.”

Milcrofton Utility District also had access to TelemetryVIEW, which allowed them to view the status, run-times, suction/discharge pressure, tank elevation, and control the bypass pump from any Internet-connected computer or mobile device. According to Mike Jones, General Manager of Milcrofton Utility District, “High Tide Technologies is user-friendly, has customizable features, and a very fast field response over our existing radio system. The experience of a fast and knowledgeable service partner, combined with High Tide Technologies, was a perfect combination.”

Cloud-based SCADA provided Milcrofton Utility District with a simple solution during the 30-day construction period. Continuous water service and reduced manpower enabled Milcrofton to better serve the residents of Williamson County during a time of rising demand.