New York Power Authority has five years of experience using the HydroX diagnostic expert system on one unit at its 912-MW St. Lawrence project. The system has provided valuable on-line operational data from the turbine and generator, and the utility plans to install the system on the remaining 15 units at this plant.
Current technological advances in condition monitoring involve using an increasing number of complex systems to diagnose the operating status and condition of hydroelectric generators. Advanced systems routinely employed include equipment needed to monitor bearing vibration, air gap, partial discharge (PD), and flux. Proper interpretation of the information gathered using these systems can lower operating and maintenance expenses and reduce unscheduled outages and catastrophic failures. However, the volume of available data from these systems and the interpretation necessary to evaluate the complex waveforms and spectrums can overwhelm plant personnel and resources. As a result, sophisticated software and algorithms often are necessary to correlate and interpret this data in a way that will establish the overall condition of the generator and drive train.
HydroX™ (for Hydro Expert) is a knowledge-based expert system for on-line monitoring of hydroelectric generators. Iris Power and GE Energy developed this system over five years of working with the New York Power Authority (NYPA). After a further two years of prototype evaluation on a 55-MVa generator at NYPA's 912-MW St. Lawrence project, the system has been commercially deployed.
NYPA's operational experience with this system to date has been positive. Problems have been minimal because the system was installed on a newly refurbished unit. It has provided data on minor irregularities that are within normal operating parameters. NYPA plans to install the system on the remaining 15 units at St. Lawrence.
Identifying NYPA's needs
NYPA began an upgrade and life extension project of its hydroelectric fleet in the late 1990s. As part of the scope for this work, NYPA identified several key technologies necessary to more completely monitor a large hydro generator. In addition to the normal process data (such as temperature, machine load, etc.), specialized monitors NYPA considered to be critical to the diagnostic expert system included air gap, bearing vibration, stator PD, and core vibration. On-line monitors were available for some needs, but their cost or complexity made them prohibitive for inclusion in an expert-based monitoring system.
Over time, competition in the marketplace led to development of several monitors that are suitable for this type of generator monitoring.1 In the case of PD monitoring, where no solution existed, NYPA and Iris Power collaborated on a research and development effort that resulted in the development of HydroTrac™, an on-line PD monitor for hydro generators, in 2001.2
Developing a diagnostic expert system
EPRI developed one of the first diagnostic expert systems for on-line turbine-generator monitoring, called GEMS, in the 1980s.3 A later attempt by GE to create a commercial system based on this research prototype failed. The technical success of GEMS spawned follow-up work by EPRI and others in the area of hydro generator monitoring.4 In fact, many of the machine behavior models developed for GEMS were very relevant to the development of HydroX.
Over the past ten years, it has become possible to develop distributed client-server applications. This innovation enables the use of distributed architecture, with one computer being a database server, one being a communication server, one running the expert system engine, etc.
In the late 1990s, NYPA initiated a research project to interview design, operations, and maintenance personnel to document a rule set for an expert system. This was a multi-year effort using experts from original equipment manufacturers, industry, academia, and utility engineering and operations staff. The knowledge base developed as a result of this project was later used to create HydroX.
Only in the past few years have practical PC-based tools been available for development and commercial deployment of expert system-based plant monitoring systems. System1®, a software platform offered by GE Energy, is such a tool. System1 is a distributed software product based on a SQL server database and contains components for data collection from remote systems via OPC (an open-source protocol that runs on Microsoft Windows), a production rule engine for processing user-defined rules, and a design tool for developing and testing rules and developing custom user interfaces. HydroX is based on System1.
Utilizing the knowledge base developed earlier with NYPA, Iris created a modular set of HydroX Rulepaks in System1. Encoding each major sensor group (operating data, mechanical vibration, PD, air gap) in its own Rulepak facilitated the customization of HydroX for the available machine sensor data. If a particular monitor, such as PD, is not available, the rules dealing with those inputs can be removed, leaving the rest of the system functional.
One particular challenge in any expert system is dealing with uncertainty in the data analysis. System1 has built-in mechanisms for indicating the severity of a problem. In HydroX, this capability was extended to utilize a MYCIN-like uncertainty scheme (a method designed to deal with uncertainty in expert systems) to combine facts from various sensor inputs into diagnosis with a certainty factor. As sensor readings vary further from expected values, or multiple indications of a problem become apparent, the certainty in the diagnosis of a fault condition increases.
Where possible, the prediction of "expected" value for sensors is made based on mathematical models of machine parameters that are tuned for the specific unit. These predicted values are compared to the actual measured values (see Figure 1), and deviations are analyzed by the rules to compute a diagnosis. For example, the predictions of thrust bearing pad temperatures are made based on the thrust bearing oil temperature and megawatt load on the machine. This basic equation is then customized to account for heating/cooling time constants of the machine with load and the actual readings obtained at full load for each sensor.
For many sensors, the alarm thresholds may be significantly different depending on the mode of the machine. HydroX has rules to determine the machine mode and, where necessary, different thresholds and even executable rules dependent on this mode. The specific modes HydroX recognizes are: standstill, mechanical run up/run down, rated speed de-energized field, field energized but unsynchronized, synchronized unloaded, load transient, and loaded thermally stable. An example of this behavior would be air gap measurements, where significantly different nominal air gaps can be expected depending on the machine state. Figure 2 shows a trend plot of air gap during start-up of a unit at NYPA's St. Lawrence project.
HydroX uses this information to set mode-specific thresholds for alarms. The machine mode also is used in several instances to calculate and alarm on the trend of a sensor's values. For example, the trend of nominal air gap during field flashing can indicate a specific type of problem that just trending air gap at nominal machine load would not detect.
Current industry trends are to move to more automated plants, with less on-site expertise and fewer operations staff. As described above, HydroX can calculate and trend key features and synthesize summary indications from complex data sets from monitors such as vibration, air gap, and PD. Using these intermediate indicators, along with diagnostic rules, an expert system like HydroX can filter and focus attention to abnormal values and provide diagnosis of specific faults as well as possible remedies. In addition, trending of such parameters over years can indicate long-term degradation that may otherwise go undetected until damage limits are breached.
An ideal time to install the sensors/monitors necessary to support a system like HydroX is during a plant refurbishment/upgrade. At the St. Lawrence project, NYPA is performing a plant life extension project on 16 units. This project began in 2004. During each unit's upgrade, additional sensors were installed to support the expert system, and interfaces were created to the plant control and monitoring systems. Using the data acquisition portion of HydroX, data was collected from these systems over time on several units, making it possible to identify machine-specific behavior and characteristics. The generalized rule set developed was customized through "tuning" algorithms. These algorithms were developed to account for specific generator behaviors due to subtle differences in manufacture or external factors such as seasonal changes in ambient conditions.
The first unit equipped at St. Lawrence was Unit 18, in 2005. During the time this unit was off line for refurbishment, NYPA personnel installed six PD monitors, 24 stator core temperature monitors, four air gap sensors, two radial/tangential stator frame accelerometers, two generator guide bearing X/Y displacement monitors, one thrust bearing axial displacement monitor, one keyphasor (which indicates the position of the rotor shaft relative to the stator), two turbine guide bearing X/Y displacement monitors, and one head cover accelerometer.
Refurbishment work is complete on 12 of the 16 units at the plant, and the thirteenth is currently undergoing refurbishment. In the future, the identical sensor set will be installed for each unit. Once completed, all 16 units will be monitored using HydroX.
Two dedicated PCs run the HydroX components: the data acquisition system, SQL serve database, diagnostic rule engine, and user interface. These computers were installed on a separate local area network (LAN) and interfaced to the other necessary plant systems, including: the Generator Control System, to obtain conventional unit sensor data; HydroTrac for PD data; and a Bently Nevada 3500 rack for air gap and vibration data. The interfaces to external systems were accomplished using an OPC data interface.5
Experience to date
The prototype HydroX has been monitoring Unit 18 for five years. One difficulty with this approach is that because the units are coming on line after a major overhaul, the number of faults is minimal. In addition, many of the long-term trending rules for conditions such as PD can take years to calculate and are just now providing useful values.
One significant problem related to the tuning of the rules. The models and algorithms used to provide predicted sensor values require substantial tuning for various constants, which can only be done once the unit is in service. It is not practical for a field service engineer to be on site for possibly weeks to collect data for the various machine states needed to tune the rules. For this reason, a set of "auto-tuning" rules were written. These rules track data during initial unit operation and automatically calculate and enter the specific constants needed for the various predicted sensor values. The rules use linear regression to determine the dependency of two independent variables on a given sensor input. This dependency is usually calculated during start-up.
A similar problem was found with the setting of alarm limits for measured values. A multitude of custom values must be set for HydroX to calculate malfunction certainties properly. These values are usually known by plant personnel and used for basic alarming of critical parameters. In many cases, these values can be based on given machine standards. HydroX addresses this issue by incorporating an automatic tuning system for alarm limits.
A final lesson from this experience concerns the reliability of the system. Some sensors and computer components have failed since the original installation of the system in 2005. In addition, plant network security has been a source of problems. As network security becomes ever more stringent, this forces frequent upgrades of software hardware and protocols, all of which may require reconfiguration of the various systems in HydroX.
Over time, the HydroX system will be installed on all 16 units at St. Lawrence.
- Lyles, John F., T. Earl Goodeve, and Gregory C. Stone, "Using Diagnostic Technology for Identifying Generator Maintenance Needs," Hydro Review, Volume 7, No. 4, June 1993, pages 58-67.
- Lloyd, Blake A., S.R. Campbell, and Gregory C. Stone, "Continuous On-line Partial Discharge Monitoring of Generator Stator Windings," IEEE Transactions on Energy Conversion, Volume 14, No. 4, December 1999, pages 1131-1138.
- Klempner, G.S., A. Kornfeld, and Blake A. Lloyd, "The Generator Expert Monitoring System (GEMS) Experience with the GEMS Prototype," EPRI Utility Motor and Generator Predictive Maintenance Workshop, EPRI, Palo Alto, Calif., December 1991.
- Roehl, A., and Blake A. Lloyd, "A Developing Standard for Integrating Hydroelectric Monitoring Systems," EPRI Motor and Generator Conference, EPRI, Palo Alto, Calif., November 1995.
Evens Jourdain, an engineer with the New York Power Authority, provided logistics and knowledge of the units at the 912-MW St. Lawrence project for installation of the HydroX system. Blake Lloyd, vice president of automation with Iris Power, performed engineering work for installation of HydroX. Peter Lewis, mechanical development engineer with Iris Power, developed the HydroX rulepaks.
This article has been evaluated and edited in accordance with reviews conducted by two or more professionals who have relevant expertise. These peer reviewers judge manuscripts for technical accuracy, usefulness, and overall importance within the hydroelectric industry.