AMSA Year 2000 Survey – Background

The Association of Metropolitan Sewerage Agencies (AMSA) is a dynamic coalition of over 200 of the nation’s publicly-owned wastewater treatment agencies. AMSA members collectively serve the majority of the sewered population in the United States, and treat and reclaim more than 18 billion gallons of wastewater each day. Over the past 28 years, AMSA has maintained a close working relationship with both Congress and the U.S. Environmental Protection Agency in the development of environmental legislation and policymaking.

Locally, AMSA member agencies play a major role in their communities, often spearheading watershed management efforts, promoting industrial/household pollution prevention and water conservation, and developing urban stormwater management programs. AMSA members are true environmental practitioners who work daily to ensure the safety and quality of our nation's water supply.

AMSA conducted a survey of its members to assess the extent to which wastewater agencies have evaluated the Year 2000 (Y2K) problem, the estimated costs to remedy the problem, the status of implementing solutions, the impacts of potential system failures, and whether plans are in place should systems fail. Seventy-six of AMSA’s 206 agencies responded to the June 10, 1998 survey, and 43 responded to the October 2, 1998 follow-up survey.

Computer Use and Level of Automation

Computers, microchips, electronic data logging/analysis, and remote monitoring/control systems are widely used and are critical components in the overall functions of the nation’s public wastewater treatment agencies. These systems contribute to varying levels of automation in the industry. While many of the functions within wastewater agencies can be automated or computerized -- such as administrative functions (i.e., billing, payroll, finances, etc.), process control operations, or laboratory functions -- all these functions can be performed manually, and a significant portion of the industry is not fully automated.

Respondents to AMSA’s survey indicated an average level of automation of 54 percent. For example, some agencies use automated billing systems, while treatment plants may operate manually. Other agencies have fully automated administrative operations, process operations, and industrial compliance programs, but may not have automated data processing in their laboratory.

The survey examines the level of implementation of Supervisory Control and Data Acquisition (SCADA) systems within the AMSA membership. SCADA systems can allow operators to remotely collect operational data, and control operations of pump stations or treatment plant processes from a single location. Among the survey respondents, 88 percent currently implement some form of SCADA system, and nearly 100 percent of respondents indicated future plans to use SCADA systems. It should be noted that although a wastewater treatment agency may use SCADA in some of its processes, this does not necessarily mean that the entire treatment process is automated. For instance, a SCADA system may be used to monitor and collect data from remote pumping stations; however, the SCADA may not monitor treatment plant processes.

Nearly 100% of the agencies responding to the recent AMSA survey indicated that computers were used in process control, laboratory, industrial compliance, billing systems, and for other administrative purposes, such as finances, inventory, and maintenance management. A complete listing of responses on the use of computers/microchips in agency functions includes:

• Administrative: billing, accounts payable, payroll, human resources, purchasing, telephone systems, assessments, procurement, contract management, capital investment programs, general ledger, office automation, pensions

• Maintenance: system and plant maintenance management, inventory

• Operations: process control, embedded programmable logic controls, SCADA, electronic pressure recorders, generators, collection system monitoring, flow monitoring, mobile equipment, meter reading and routing

• Laboratory: laboratory analysis, calibration, reporting

• Industrial Waste: permitting, industrial compliance determinations, sampling

• Engineering: project tracking, geographic information systems, computer-aided drafting (CAD)

• Reporting: National Pollutant Discharge Elimination System (NPDES) reporting and monitoring

• Other: interactive voice response, Internet, energy management, telephones, security, radio, elevators, fire alarms.

Assessment and Action

A vast majority of AMSA survey respondents (90 percent) have developed a plan to assess and address the Y2K problem. Many of these assessments are very formal processes which are either initiated under a comprehensive local government assessment or as part of the agency’s overall planning processes (it should be noted that 50 percent of the AMSA membership are agencies which operate under the jurisdiction of a local city or county government, while another 50 percent of AMSA members operate as regional districts). A little more than half of the agencies are addressing (or intending to address) the problem in-house, while the remainder are using consultants or a combination of in-house staff and consultants.

Costs

The costs to address the Y2K problem vary widely for survey respondents. Forty-five percent of the wastewater agencies which reported estimated costs indicated that the cost to address the Y2K problem was relatively minimal, ranging from 0 to $100,000, while 15 percent reported estimated costs in excess of $1,000,000, with the two highest reported values being $15,000,000. Most of the agencies reporting expenditures in excess of $1,000,000 were relatively large systems, however 17 percent of these were agencies serving populations less than 250,000. In general, most agencies reported total estimated costs to fix the Y2K problem between 0 to 2 percent of annual operation costs. Four agencies reported estimated costs to fix the Y2K problem over 10 percent of annual operation costs. The average annual budget for an agency serving one million people is approximately $125,000,000.

Progress in Implementing Solutions

Implementation of solutions to the Y2K problem varies widely, though most responding agencies have made some progress. Approximately 95 percent have begun to implement solutions to the Y2K problem, while 26 percent are complete or nearly complete. To address the Y2K problem, many agencies are systematically checking and upgrading systems which are not Y2K compliant.

Figure 1 illustrates a timeline of responding AMSA POTW Y2K efforts. As illustrated in the graph, a majority of the agencies will have completed the awareness, inventory, and assessment phases of Y2K conversion by January 1, 1999. Responding agencies are poised to focus Y2K efforts on repair, testing, contingency planning, and implementation in 1999 and most have plans to implement Y2K ready systems by January 1, 2000.

Figure 2, which illustrates the current status of responding agencies in accordance with six defined phases of Y2K remediation, also highlights that Y2K repair, testing, contingency planning, and implementation in will be high priorities for wastewater agencies in 1999.

Figure 3 illustrates the status of assessment/correction of Y2K problems associated with embedded microchips. Embedded microchips present a special challenge to the Y2K issue as they are pervasive in a range of systems and equipment including: meter readers, programmable logic controllers, security systems, elevators, alarms, etc. Responding agencies are testing these systems and receiving assurances from vendors that these systems are Y2K compliant. As the graph illustrates, a majority of the responding agencies are still in the assessment phase for each type of system, while many have completed embedded chip assessments, and are currently implementing remediation efforts, especially for mission critical systems such as plant process and remote process operations.

Impacts of Year 2000 Failure

Though most agencies believe they will be Y2K compliant in 1999, AMSA’s survey requested that agencies project the resulting impact, should a Y2K failure occur in any critical systems. A breakdown by agency function follows:

Administration - Computers are used throughout the administrative functions of a wastewater treatment agency. Billing, payroll, human resources, and many other functions depend on accurate computerized record-keeping and reporting. Potential failures in billing systems are the most troublesome to agencies responding to the survey. Should systems fail in the event of a Y2K problem, nearly all agencies indicated that delays in billing would result in serious cash flow interruptions. These interruptions in cash flow are unlikely to directly affect operations, as many agencies have cash reserves on hand, or may be able to negotiate with vendors to extend bill due dates, however, such a failure is likely to have major impact on the administrative functions of the agency. Some agencies reported that they have backup contingencies should there be a failure in automated billing.

Process Control - All responding agencies with automated process controls have the ability to switch to manual operations almost immediately or within hours in the event of a Year 2000 failure. Approximately 15 percent of AMSA agencies reported potential treatment plant problems and possible compliance issues as a result of switching to manual mode. Potential additional costs would be incurred with the addition of staff or the payment of overtime.

One of the biggest concerns in this situation is that collection system and plant operational data would not be immediately accessible for the operators, and whether this would lead to sewage backups, overflows, or compliance problems. However, most agencies reported that switching to manual mode would pose no, or very minor, problems as many automated operations run in parallel with "manual" instrumentation and control. For instance, a wastewater treatment plant may use programmable logic controllers (PLCs) within its treatment operations to control valves or pump operations based on flow or pressure readings. In normal operations, the data from these controllers would be relayed to an operator’s computer control screen, and the PLCs would automatically activate valves or pumps accordingly. Should one or more PLCs malfunction, an operator would no longer receive data via the computer control screen and would have to "manually" read flow meters or pressure gauges. The operator also could not rely on the PLCs to automatically activate appropriate valves or pumps, and would thus have to "manually" adjust these controls.

One potential catastrophic failure issue which was noted and which is beyond the control of the wastewater agency is the occurrence of a major regional electrical power failure. There are a wide-range of capabilities in terms of operating treatment plants in the absence of a electrical power.

During a recent meeting with the electric power industry trade groups, power industry officials indicated a "cautious optimism" regarding their industry’s ability to meet the Y2K challenge by January 1, 2000. The groups referenced a recently released report titled, "Preparing the Electric Power Systems of North America for Transition to the Year 2000 - A Status Report and Work Plan" which was submitted to the Department of Energy on September 17, 1998 (available at http://www.nerc.com. In the report, officials indicated that any power outages that occur due to Y2K issues are likely to be localized and short-term (i.e. hours or days), and that the chance of a widespread power grid failure is basically zero. Localized and short-term power outages will not produce widespread treatment plant disruption as most facilities have dual power feeds from differing electrical sub-stations as backup, and/or have reserve capacity within the treatment and collection system to store flows until power is restored.

In the extreme case of a regional, long-term electrical power failure, 37 percent of the responding agencies indicated that all treatment plants under their control could operate indefinitely throughout the power outage. Most of these facilities would use diesel, methane, or natural gas powered generators, though fuel availability would clearly be an issue. Some plants could operate generators with methane produced from on-site solids digestion processes. Another 28 percent of the agencies indicated that their plants could operate at partial capacity, or that some, but not all of the plants under their control could operate fully. Remote pumping stations may also be affected by a regional, long-term power failure. While, 70 percent of responding agencies would be able to operate their remote pumping stations in the case of a long-term electrical power failure using diesel generators, other agencies would have to rely on mobile generators, or in-line and off-line storage capacity to contain any flows that could not be pumped to the treatment plant.

Laboratory - Should laboratory systems fail, the issue would be whether agency laboratories could adequately and accurately analyze sample results, and report compliance problems adequately to regulatory and public health agencies. Some agencies can operate in manual mode, while others indicated that out-sourcing of lab functions could be implemented.

Industrial Compliance Programs - Should industrial compliance systems fail, the wastewater treatment agency would not be able to adequately monitor industrial customer compliance, which could result in undetected high strength discharges leading to treatment plant upsets, delays in issuing permits, and noncompliance with federal regulatory pretreatment requirements.

Plan of Action

Nearly 55 percent of the agencies have plan of action should all or a portion of their computer systems fail as a result of the Y2K problem. As noted in Figure 1, all respondents plan to address the issue of contingency plans by January 1, 2000. Contingency plans will discuss issues concerning: 1) how manual operation should be initiated in the case of system failures; 2) chemical and fuel supply needs; 3) coordination with other local entities; 4) manpower needs; and, 5) correction of system failures.

Conclusion

Based on the results of the AMSA survey and follow-up discussions with wastewater treatment agency staff, it can be concluded that the large segment of wastewater industry represented by AMSA’s membership will respond effectively to the challenges presented by the Y2K problem. However, a significant portion of work will need to be completed prior to January 1, 2000. Remediation, testing, implementation, and contingency planning should be high priorities in 1999.

While treatment plants have become more and more automated over the past ten years, many treatment plants still operate fully manually, and even automated plants can be reverted back to manual mode in a matter of a few minutes or hours depending upon the complexity of the system and manpower availability. Some problems are bound to occur, and may involve either or both internal system problems or external factors that are beyond the control of a public wastewater treatment agency; however, careful program management and proper contingency planning should minimize the impacts of these problems on public health and the environment.