Maintaining aging transformers, ensuring access to spares, and determining remaining life are three critical issues hydro project owners and operators face. In this discussion, five project owners share the work they are performing and technologies they are using to diagnose problems and prevent failures.
Transformer maintenance is a challenge for every hydro project owner. In addition to preventing and predicting failures, owners must ensure the availability of spares. Owners also must deal with issues related to oil leaks, such as reclaiming oil, filtering to avoid contaminated oil, and performing condition monitoring with oil samples.
Five hydro project owners with recent transformer maintenance, repair, and replacement experience share their insights and lessons learned in the areas of: addressing key concerns with transformers, applying new testing and monitoring technologies, and maintaining aging equipment.
Representatives of the owners are:
– Rene J. Bray, P.Eng., performance and maintenance section head for the high-voltage, direct current (HVDC) division of Manitoba Hydro. Rene is responsible for all maintenance of transformers in the HVDC stations, including replacements and upgrades. Over the past seven years, he has been directly responsible for replacing 18 transformers and repairing three;
– Bruce L. Broweleit, electrical engineer with Grant County Public Utility District (PUD) No. 2. Bruce is Grant County PUD’s subject matter expert on hydro transformer maintenance. He has been involved in testing, inspection, and ongoing maintenance of the transformers for more than 16 years. Bruce is project manager for the work to supply five replacement generator step-up transformers each at the 957-MW Priest Rapids and 1,038-MW Wanapum projects;
– Louis Caracciolo, electrical test and evaluation technician for the U.S. Army Corps of Engineers. Lou assists with the evaluation of a majority of the power generation equipment at the 1,812.8-MW Dalles and 2,160-MW John Day projects. He performs testing and condition monitoring for 33 generator step-up transformers and three station service transformers;
– James E. Guenther, hydro maintenance coordinator for the Lower Colorado River Authority (LCRA). Jim is responsible for planning and coordinating maintenance and capital projects for LCRA’s six hydro projects. He oversees transformer maintenance and determines corrective actions. He has been involved in the procurement and replacement of eight transformers; and
– Jill R. Smith, electrical engineer with the U.S. Department of the Interior’s Bureau of Reclamation. Jill provides technical assistance to Reclamation’s power plants regarding transformer testing, diagnostics, and overall health assessment. She also researches and evaluates new transformer testing equipment for use at Reclamation’s hydro facilities.
Hydro Review: What are your key concerns with respect to transformers? How are you addressing these concerns?
Lou Caracciolo: Our main transformer concerns are: their age (most are 40 to 50 years old); prevention/ prediction of failures or predicting “end of life;” addressing oil leaks and providing containment; and having accurate condition assessments.
To address these concerns, the Corps uses an aggressive test and evaluation program. We perform routine and non-scheduled maintenance when necessary, which can involve changing out high-voltage bushings, repairing gaskets, refurbishing heat exchangers, repairing leaks, and reclaiming oil. We also construct/maintain oil containment barriers in combination with an oil/water separator system.
Finally, we use the HydroAMP asset management tool, which was created by a team representing the Federal Columbia River Power System. This Excel spreadsheet provides a snapshot of all the major power train equipment within the system. The spreadsheet is an integral part in developing a near- and long-term investment plan for the equipment. HydroAMP’s transformer condition assessment is based on several rating factors: equipment age, insulating oil analysis, power factor/exciting current results, operation and maintenance history, and a data quality indicator that measures accuracy of the previous information.
Jim Guenther: Our main concern is adequacy/availability of spares. LCRA does not have spare generator step-up transformers. The long delivery time for a replacement transformer is a concern. Many of our old transformers have bushing styles, pass-through terminal strips, and tap changer parts that are no longer available. LCRA is a member of the Doble community. Doble Engineering offers diagnostic instruments and services, as well as a library of statistically significant apparatus test results. The Doble bulletin board and used transformers advertised on the Internet by equipment traders are backup options to obtain a used transformer if we had a sudden failure.
Another concern is that there are no manufacturers building new transformers in the U.S. We have experienced quality concerns with vendors building transformers in third-world countries. We also are seeing an across-the-board issue with vendors not meeting or barely meeting the temperature rise specifications. This possibly is tied to the increased cost of copper and steel, which seems to have vendors honing their design margins closely to keep material costs down.
A third concern for us is predicting failure. We use Doble’s bank of test history for comparison of our various test results. On the older transformers, we have begun using insulating paper analysis to help determine remaining life.
In 1999, LCRA Hydro began a ten-year program to replace ten of our 12 generator step-up transformers, which had been in service 40 to 50 years. As of June 2007, we had replaced eight.
Rene Bray: Our primary concerns relate to age of the transformers and when the rate of failure will increase beyond our ability to have an adequate number of spares. We are also concerned about predicting failure via testing and determining the remaining life of the transformer.
We have one spare for each different type of unit at each site, to allow for quick replacement in the event of a failure. For example, a unit was drained in May 2007 for internal inspection. Based on dissolved gas analysis (DGA) results, we found the source of the problem and replaced the transformer in eight days. Five of these days were for moving the old unit off the pad and the new unit on the pad, including two days to reach the remote location.
For the future, based on the age and delivery times, we are looking at keeping two units of each at site.
Jill Smith: Our primary concerns related to transformers include:
– Predicting failures;
– Preventing or properly containing catastrophic failures and the best methods to apply fire prevention/suppression;
– Transformer life extension, along with providing proper electrical and mechanical maintenance of our aging transformer fleet;
– Proper assessment of corrosive sulfur issues and actions taken to mitigate or pacify the problem;
– Logistics and challenges of procuring new replacement transformers; and
– Proper oil processing/filling/transportation techniques to avoid contaminated oil.
We perform “traditional” condition monitoring to assess the health of our transformer fleet. This includes:
– DGA (by sending regularly scheduled oil samples to a DGA laboratory; we also perform on-site DGA using newer technology for transformers with suspected or known issues or units in an emergency condition);
– Oil samples sent to a lab for DGA analysis also will be evaluated for interfacial tension (IFT), acid number, Furans, and dielectric strength, as needed;
– On-line DGA monitoring of transformers with suspected or known issues;
– On-line transformer oil processing (when necessary);
– Dielectric power factor tests;
– Excitation current tests;
– Leakage reactance tests;
– Frequency response analysis; and
– Regular mechanical maintenance.
Corrosive sulfur is such a new issue, with no industry standard solutions. Until a solution is found, we will monitor transformers known to have corrosive sulfur more closely.
Many hydro project owners are replacing aging transformers. For example, Grant County Public Utility District is installing new generator step-up transformers for the five units at its 957-MW Priest Rapids project.
Bruce Broweleit: Our most important transformer concern is reliability. A forced outage can be costly, both in terms of lost revenue from power sales and from collateral effects. These collateral effects are sometimes difficult to quantify and predict because they can involve agreements with agencies external to Grant County PUD. Having a viable spare transformer on hand only partially alleviates the consequences of a failure, primarily by shortening the time to restore lost generation. However, it still takes a finite amount of time to swap transformers, during which time undesirable effects will require attention and remedy.
As transformers approach the end of their life, squeezing the last possible megawatt out of them and then replacing them in an orderly fashion is desirable. Failure prediction becomes the basis of how Grant County PUD makes decisions regarding replacement. Part of the equation involves the cost of maintenance, especially when extraordinary maintenance is required. Also involved is the seemingly ever-increasing lead-time required for replacements.
As transformers approach the end of their life, squeezing the last possible megawatt out of them and then replacing them in an orderly fashion is desirable.
An additional concern is the environmental consequences of a violent failure. Our generator step-up transformers are situated on dams spanning one of the largest rivers in North America and are in the focus of several fish, water quality, and land use special interest groups and regulatory agencies. Allowing sizeable quantities of insulating oil to contaminate the river could have awful consequences to both the PUD and the environment. To avoid this problem, we have provided containment, but preventing failures is just as important. We place emphasis on regular maintenance, condition monitoring, and more sophisticated protective devices on our new transformers. We have resisted the use of biodegradable oils, primarily because oil-dissolved gas analysis does not yet have an established historical database to develop limits for predicting incipient failures. In addition, there are concerns about long-term stability and sludging.
Hydro Review: Are you applying new testing and monitoring technologies? What technologies are you using?
Caracciolo: Yes. We want to have the best means with which to resolve our concerns. Technologies we use include: on-line fault gas analyzers, sweep frequency response analysis (SFRA), portable gas analyzers (photo-acoustic spectroscopy), Furan analysis, cooling monitors, and thermography. These tools have allowed us to diagnose problems and prevent failures on several occasions, as well as facilitating predictive maintenance.
Guenther: Maintaining availability prompted us to go to new technologies. We need to identify a problem while there is still a possibility of repair. We also need to monitor gradual degradation so we can plan replacements well in advance of the long delivery times for new transformers.
We perform power factor tip up, partial discharge, thermography, DGA oil testing, and SFRA. Thermography has allowed us to see/repair overheating loose connections, load tap changer contact problems, plugged coolers, and overloading due to multiple transformers not sharing load equally. Oil testing has indicated tap changer contact problems and the presence of corrosive sulfur. Insulating paper analysis has been a good indicator of remaining life in the insulation.
Bray: We are mainly sticking with the standard tests for transformer diagnosis – ratio, resistance, power factor and capacitance, and DGA. The only new monitoring we are doing is SFRA, to baseline all units to help predict future concerns.
Smith: We have implemented frequency response analysis at many of our facilities. This is a useful tool to detect physical internal mechanical changes in transformers. We have used this technology to confirm the integrity of many transformers. We also have implemented online DGA at a few of our facilities. This has proven to be a useful tool to more closely monitor transformers with suspected or known problems.
Use of these new technologies has been prompted by an aging transformer fleet, lengthy time durations to replace transformers, shortages of transformer core steel, and the desire to avoid oil spills into waterways.
We are evaluating incorporating technological upgrades for new transformers, such as installing additional on-line condition monitoring technology or improved methods of monitoring transformer vitals, such as embedded temperature detectors. Technological upgrades for older transformers typically are reserved for transformers with a known problem that requires close monitoring, further investigation, or additional monitoring protocols. We also have looked into new types of insulating oils for select transformer locations.
Broweleit: In the mid-1990s, one generator step-up transformer at Priest Rapids exhibited a continuing “controlled” increase in dissolved-gas-in-oil levels that was evidenced in the mid-1990s. Due to concerns that this could turn into a major fault, we installed a Syprotec HYDRAN® 201R Model i intelligent fault monitor (now supplied by GE Energy) in 1996 to allow ongoing assessment of transformer condition. Later, this transformer was removed from service for repairs, and the monitor was moved to another transformer. Grant County PUD purchased a second Syprotec monitor in 1997, so that the two generator step-up transformers of the most concern could be monitored.
The Syprotec monitors showed that online monitoring of oil-dissolved gases had value because of the dynamic nature of gas generation with respect to load and temperature. Grant County PUD personnel felt that an incipient fault would be much more likely to be detected in the early stages by using the monitors. At the time, our practice was to take one sample per year for laboratory DGA. We have since doubled the frequency of lab DGAs, but that still leaves 363 days a year when transformer condition is uncertain. A fault could easily develop during the six-month window between samples and go undetected until protective relaying is activated, resulting in a forced outage.
The Syprotec monitors had some shortcomings. First, they could not withstand full vacuum and so must be removed prior to vacuum-fill operations. Second, the gas-permeable membrane used to separate the dissolved gas from oil had a limited and fairly short lifetime. Finally, the monitor was only sensitive to four gases: hydrogen (H2), carbon monoxide (CO), acetylene (C2H2), and ethylene (C2H4), in decreasing order of sensitivity. The data is presented to the user as a composite value, so there was no way to determine the actual parts per million (ppm) of each gas without taking a lab sample for DGA. This limited the value of the monitor to that of an early warning device to trigger additional diagnostic actions. (To be fair, that is how it was marketed.)
After a couple years of experience with the Syprotec monitors, Grant County PUD wanted an online monitor that provided results similar to lab DGA tests. In 2001, personnel performed a market survey and determined that Serveron Corporation had the most cost-effective equipment with eight-gas monitoring technology. A TrueGas™ Oil-phase monitor was purchased and in- stalled in September 2002 on one of the gassing Priest Rapids generator step-up transformers. A second TrueGas monitor was installed on a second gassing Priest Rapids transformer in December 2003 (this transformer had previously suffered a fire from high-voltage bushing failure and been refurbished). Both Syprotec monitors were then retired.
These TrueGas monitors were kept in place until the transformers were replaced. The monitors were then moved to other, old generator step-up transformers at Priest Rapids. One monitor has been moved twice as the transformers continue to be replaced. The new generator step-up transformers come equipped with Serveron’s newer TM-8 monitor. The TM-8s are more sensitive than the TrueGas monitors and incorporate other desirable technological advances.
The TrueGas monitors have helped us appreciate how dynamic conditions are inside the generator step-up transformers. They also provide more assurance that an incipient fault will be caught in the early stages. The eight-gas technology allows dynamic diagnosis of conditions inside the transformer, enabling better decision making regarding their operation. In some cases, load limits have been placed on the transformers because it was clear that certain conditions exacerbated the gassing.
Both the TrueGas and TM-8 monitors have been useful in providing quality assurance/ quality control feedback to the laboratories that test our DGA samples. In one case, this information prompted the lab to change its procedures and recalibrate its equipment. However, there are ongoing significant differences with certain gases between the lab and monitor results that have prompted discussion/debate. Some of the differences seem explainable (i.e., why oxygen and nitrogen concentrations vary so much with lab samples verses the monitor). But occasionally, there are other differences that seem directly related to calibration of equipment. The question is, whose?
Performing online dissolved gas analysis of transformers with suspected or known problems allows hydro project owners to assess the health of the transformer fleet and identify equipment that needs attention.
The Serveron monitors’ performance seems fairly sensitive to calibration, and they require routine replacement of about $450 a year in consumables. Grant County PUD is surveying the market for monitors to be specified and supplied with the future replacement transformers for the Wanapum facility that might not have these shortcomings.
Our new generator step-up transformers also are equipped with a Neoptix T/Guard+ fiber optic temperature monitoring system. This system replaces the conventional simulated hot spot temperature sensors traditionally supplied with transformers with fiber-optic sensing probes embedded in the core/ coil assembly that measures the temperatures directly. The cost of these systems has dropped dramatically in recent years while manufacturing survivability has greatly increased, making this very cost-effective technology. We have 21 probes installed on each transformer, only eight of which are actively monitored. Out of four generator step-up transformers supplied to date, only two probes have been in question. The cooling fans are controlled by the T/Guard+ system, and the eight monitored signals are logged by our plant control system for trending, annunciation/relaying, and long-term analysis. Benefits include more accurate temperature measurement, which will assist in maximizing loading and reducing loss of insulation life from overheating.
In 2004, we added two pieces of test equipment to our diagnostic tool inventory, a leakage reactance (LR) interface and an SFRA. Both are manufactured by Doble. The LR works in conjunction with Doble’s M4000 insulation analyzer (power factor test set), and the SFRA is a stand-alone device. Both instruments are intended to provide information regarding changes to the internal structure of transformers, but they are based on different principles and present the data in different formats. While there is some similarity in the physical properties measured, there are also differences. Thus, the two technologies are complementary. Presently, the LR and SFRA are used for baselines on new transformers before commissioning (these tests also are required at the factory) and for extraordinary diagnosis. For routine annual maintenance, we only do insulation power factor, Megger, transformer turns ratio (TTR), and direct current (DC) winding resistance tests.
Hydro Review: What is your number one transformer maintenance issue? What steps are you taking to address it?
Caracciolo: Our primary concern is stopping oil leaks. We continue to aggressively either curtail or repair all leaks.
Guenther: We are concerned that corrosive sulfur in the oil of our new transformers could cause failures in a few years. In the past year, we have tested all our transformer oil to the new ASTM International standard for corrosive sulfur. Five of our “new” (two-year-old) transformers indicated elevated levels of corrosive sulfur.
Our company has taken an active part in the session at the Doble conference on this fairly new issue. We are monitoring where the whole industry is going with this new issue and are taking somewhat of a wait-and-see stance.
Bray: We are concerned about tap changer and diverter maintenance. We are systematically overhauling all the tap changers over the next four years.
Smith: A maintenance issue at many of our facilities is performing timely regular electrical and mechanical maintenance and ensuring each transformer receives individual, specialized, or specific tests. Depending on the location, this may be due to a number of factors, including the loss of expertise due to an aging workforce.
Maintaining aging transformers, ensuring access to spares, and determining remaining life are three critical issues hydro project owners and operators face.
We employ a small centralized group of technical experts who analyze and assist our field offices in the event of questionable and/or poor test results or if a failure occurs. This group examines the criticality of the transformer and its known condition (based on available test data and transformer history) to determine if additional or specialized testing/maintenance is necessary. If the field office cannot perform testing, the centralized technical personnel may travel to the site and perform the routine or any specialized testing necessary in an attempt to maintain the original testing schedule (outage time and funding permitting).
Broweleit: The item that has caused the most trouble is the high-voltage bushings. We have had several failures, and a few have been catastrophic, resulting in collateral damage and forced outages. We perform maintenance on the transformers annually. During this maintenance, a number of different tests are performed. Insulation power factor testing (which also measures capacitance) has provided the only indication of incipient problems with the bushings, but the “warnings” have not been consistent. We have found that failures can occur even though the test data does not look that alarming.
At Priest Rapids, where we had violent failures, in 2001 we replaced the high- and low-voltage bushings on all the transformers. We began replacing the transformers themselves in 2005. We have replaced a few individual bushings on the Wanapum transformers that power factor test results have indicated are deteriorated. These bushings were replaced with like-kind spare bushings on-hand, not modern units. We intend to begin replacing these transformers in 2010. The new transformers at both plants will be supplied with new, modern bushings, which should eliminate this problem as a reliability/maintenance issue for many years.
There are companies that offer on-line monitors that could help address concerns about bushing condition and provide warning of incipient faults. They have an advantage over the maintenance methods we use in that they ascertain the condition of the bushings in real time and under live operating conditions. Unfortunately, this technology cannot be applied to our equipment the way it is presently configured. The on-line monitors use the bushing power factor tap for connection, but we use the tap to provide signals for relaying and synchronization. The last time this technology was evaluated (about five years ago), none of the providers allowed the signal to “pass-through” their connections to the tap. Thus, installing the monitor would also require installing line potential transformers and significant re-wiring to provide the relaying and synchronization functions. This would make bushing monitoring prohibitively expensive for the perceived benefit.
Another problem, believe it or not, is spiders. The transformers and appurtenances make a very attractive habitat for the common garden spider and a couple other types. With a steady diet of indigenous insects, they grow huge (almost 2 inches in diameter) and cover the radiators with their webs. Besides the personnel hazard, we are concerned about potential reduced cooling efficiency and how to clean the radiators without damage. We would like to use insecticides, but the location prevents use of most commercially available products because if the overspray reached the river it might possibly kill fish. We are evaluating a product called Ecotrol EC, manufactured by EcoSMART Technologies. This product is made from ingredients that meet the requirements of the USDA National Organic Program.
Mr. Bray may be reached at Manitoba Hydro, HVDC Division, 1146 Waverley Street, Bay 3, Winnipeg, Manitoba R3T 0P4 Canada; (1) 204-474-3851; E-mail:firstname.lastname@example.org. Mr. Broweleit may be reached at Grant County Public Utility District No. 2, 15655 Wanapum Village Lane SW, Beverly, WA 99321; (1) 509-754-5088, extension 3104; E-mail: email@example.com. Mr. Caracciolo may be reached at U.S. Army Corps of Engineers, The Dalles Dam, P.O. Box 564, The Dalles, OR 97058; (1) 541-298-7588; E-mail: lou.caracciolo@ usace.army.mil. Mr. Guenther may be reached at Lower Colorado River Authority, 3700 Lake Austin Boulevard, Austin, TX 78703; (1) 512-473-3200; E-mail: firstname.lastname@example.org. Ms. Smith may be reached at Bureau of Reclamation, U.S. Department of the Interior, Hydroelectric Research and Technical Services, Mail Code 86-68450, P.O. Box 25007, Denver, CO 80225; (1) 303-445-2307; E-mail: email@example.com.