Electricite de France is building five centralized monitoring and control centers to provide remote monitoring of more than 300 of its 450 hydroelectric plants. This setup will allow the utility to reduce the unavailability of turbines and optimize production.
By Jerome Boudon, Laurent Bacon and Cedric Bellec
|This article has been evaluated and edited in accordance with reviews conducted by two or more professionals who have relevant expertise. These peer reviewers judge manuscripts for technical accuracy, usefulness, and overall importance within the hydroelectric industry.|
Hydropower is a strategic energy source with many advantages: It is renewable, well- integrated in the environment, essential to efficiently adjust the electricity distribution network to market needs, and very profitable. Electricite de France owns more than 450 hydroelectric plants that have total capacity of about 20 GW. Hydropower is an energy source for the future that EDF must reinvest in for several reasons:
— To cope with new challenges in an increasingly demanding energy market with constantly growing energy demand, particularly during hours of peak demand;
— The production/consumption balance is becoming strained and therefore increases the value of good positioning in the market; and
— Competitive bidding for concession renewals could lead to the loss of a portion of the assets EDF currently operates.
|Many of the 450 Electricite de France hydroelectric plants, located at dams throughout the country, are being connected to the new centralized monitoring and control centers.|
This new industrial and social model is being achieved through evolution of EDF's operating methods and maintenance. New operating methods involve having all of the utility's large hydro plants remotely supervised from centralized monitoring centers, to assist local operators in their daily jobs. New maintenance methods involve replacing corrective maintenance with preventive and predictive maintenance, setting up standard maintenance programs and work procedures, and better optimizing maintenance operations and outages.
The target vision for EDF is to:
— Modernize and standardize all its power facilities, including renovating the plants (instrumentation and control, distributed control systems, regulators and protection), with standard solutions defined according to segmentation of the turbines into three different types based on power production;
— Develop industrial partnerships that involve a subcontracting policy adapted to fleet segmentation, improving flexibility and reinforcing local integration; and
— Renovate the information technology (IT) system, which involves improving maintenance efficiency (via enterprise asset management, enterprise content management, and scheduling tools), establishing a centralized monitoring center to enable remote monitoring and operation, and setting up a robust telecommunication network to support the new IT system.
The resulting Renouveau project, which began in 2007, aims to retrofit the supervisory control and data acquisition (SCADA) systems for more than 300 of the utility's 450 hydroelectric plants that are considered medium or large (the remainder are small and are taken into account by another project that will connect the plants to the monitoring center in a less expensive fashion), as well as create five centralized monitoring and control centers (CMCC) to perform advanced supervisory and control operations and guide preventive maintenance.
The CMCCs are located in each of the five regional hydraulic generation business units: UP CENTRE, UP EST, UP ALPES, UP MEDITERRANEE and UP SUD-OUEST (see Figure 1).
The IT solution chosen after a one-year proof-of-concept period, called e-Exploitation, combines proprietary and third-party software. It is made of two sub-systems:
— PLEX for "local" usage (inside the hydro plants); and
— PREX for "regional" usage (inside the CMCCs).
The system is built to last for the next 20 to 30 years. It must be able to be upgraded during that period and manage 800,000 pieces of real-time information and store that data over 10 years at least. This work is part of a company-wide project dealing with organization, regulations and resources within EDF.
To validate the CMCC concept and organization, the first one was installed in December 2010 in Toulouse (southwest of France), and seven hydro plants were connected to it. During the next 1.5 years, many interesting events were analyzed, and this work helped EDF decide on a general deployment in all the regions for more than 300 hydro plants.
About the IDC
To provide information and services to the CMCCs and to globally manage information exchange between these entities, a single centralized data center was built. It uses new IT technologies in a new domain: cloud computing for industrial systems. In essence, the term cloud computing refers to a large number of computers connected through a real-time communication network.
EDF's Industrial Data Center (IDC) for hydraulic plants is located in Grenoble. It is divided into three independent rooms. Rooms 1 and 2 are dedicated to CMCCs, and Room 3 is dedicated to development and integration environments and some other computing systems for hydro. The IDC uses the EDF network infrastructure, which includes secure virtual LANs to communicate between the data centers and the computing equipment in the power plants.
Everything was designed to provide high availability thanks to a high level of redundancy and independency (see Figure 2). The HVAC, power supplies, servers, network and security are independent in each room and redundant to the other production room.
Rooms 1 and 2 are in an active-active execution mode. This means all the systems are running in a load balancing mode on separate servers, but each room is dimensioned to accept all the load in case one room fails.
The IDC provides several valuable benefits:
— High system availability thanks to redundancy and separation between software and hardware, as well as virtualization, which allows migration of the application from the early stage of design to maintenance operation through integration;
— High scalability because the new application can be dispatched on a new server in minutes;
— Maintenance is performed differently because a virtual system can be cloned to be studied and fixed without altering its operation;
— Centralization allows centralized maintenance so the team has to go to only one location, providing fast access to hardware for correction;
— Expertise can be centralized in the hands of senior industrial IT experts; and
— Information exchange is controlled on a server-to-server basis and centralized to provide better SCADA security.
This system has flexibility for adding, modifying or removing services, as well as moving the location of the CMCCs in the case of a storm that makes the telecommunication network unavailable. With this setup, the monitoring system for one region can be started in the CMCC of another region.
Feedback about the project
The UP SUD-OUEST CMCC was commissioned in December 2010 and is located more than 500 km from the IDC. Feedback from the operators confirms the qualities of the architecture. Operators think servers are locally set up and do not notice any network latency.
Furthermore, the other four CMCCs are already in operation in the IDC, allowing technical exchange between business units and simplifying future infrastructure commissioning.
The CMCC is not only used for operating virtual systems (70% of the virtual hosts) but also for testing and qualifications systems (20%). The remaining 10% are dedicated to general usage: DNS, authentication server, office IT server.
The IDC is currently hosting 300 virtual servers, and the target is 500 in order to have a clone of all systems that are rolled out in the power plant, for testing and maintenance purpose. One development and three integration and qualification environments are already up and running, providing programming and testing environments to the engineering team.
Benefits and concerns of the solution
Return on investment, availability and reliability of the server's centralization are the main "pros" of the architecture.
However, this distributed architecture relies on a long-distance network. This network has to be reliable and provide reasonable latency according to the involved distances and real-time requirements.
Most of the cost is related to setting up all the facilities of the data center (walls, fire alarms, redundant power management and HVAC).
The IDC's innovation is all about using cloud computing techniques for industrial systems. The scalability of the solution allows maintenance and security-oriented design. Functional segregation (virtualization, centralization, firewalling, VLAN) provides faster and better diagnosis and easier upgrades and testing. Potential future upgrades to the system include:
— The CMCC is applicable to other industrial usage at EDF Group, such as hosting an optimization algorithm;
— The CMCC is qualified to broadcast operation alarms, meaning the current alarms architecture could be upgraded from a phone line to internet protocol infrastructure; and
— Virtualization for its maintenance and scalability qualities can be designed to be used in smaller systems, such as systems within power plants. One server can execute several "virtual" servers.
EDF also needs to decide how to implement a disaster and recovery strategy. One option might be to create a new room delocalized from Grenoble, the location of the IDC, as a disaster and recovery solution. If the building containing the IDC burns, for example, all the CMCCs will be stopped.
Jerome Boudon manages the development of computer systems, Laurent Bacon is project manager and Cedric Bellec is industrial network and security manager with Electricite de France.