NPP Control Computer Aging Management Strategy, Implementation and Results

1
NPP Control Computer Aging Management Strategy,
Implementation and Results
Technical Meeting on Integration of Analog and
Digital IC Systems in Hybrid Main Control Rooms

• Rick Hohendorf
• Computers and Control Design Dept.
• OPG

2
Agenda

• Basis
• Initiation of the Strategy
• Experience in Applying the Strategy
• Results
• Looking Ahead

3
Basis

• Control Computers (DCCs) are vital to the
  operation of OPGs nuclear power plants
• More than 30 yrs old with no OEM support
• Large, complex systems representing a substantial
  investment in engineering
• A DCC channel at Darlington has 2300 inputs
  cabled from the field and 430 control outputs
  cabled to field equipment
• Control program software must be precise

4
Strategy Initiation

• Pickering A recognized the inherent value of
  proven control software to run the reactor
• Porting this software to a new platform would
  have been a large technical risk
• The initial strategy was to acquire as many
  spares as possible this was only a stop-gap
• In the early 1990s, a Strategic Lifetime
  Assessment recommended replacement of the IBM
  1800s by a hardware emulator

5
Strategy Initiation

• An ES-1800 hardware emulator had been produced
  for the U.S. military
• Exhaustive testing confirmed that the ES-1800
  faithfully executed the control S/W
• The dual channel DCC replacement was successfully
  installed on Units 1 and 4
• There was minimal impact on field cables or on
  the operator interface

6
Results

• ES-1800s have run without any emulator-
  attributable faults since in-service
• System health improved from red to white
• Replacements did not occur on Units 2 and 3
  because of the Safe Store decision
• The need to replace the IBM 1800s on Units 1 and
  4 has been validated by difficulties encountered
  in maintaining the Unit 2 and 3 DCCs for
  de-fuelling purposes

7
Strategy Initiation

• A second aspect of the strategy was also
  initiated on Pickering A replace obsolete
  devices without affecting control programs
• De-couple operator interface changes from the
  control programs
• Initially, the Pickering A control room operator
  could only peek and poke at control program
  memory contents via error-prone hexidecimal input
  and output devices

8
Strategy Initiation

• The PACE project in the mid-1980s cut the wires
  to the hexidecimal I/O devices and grafted on a
  network of 5 microprocessors
• One microprocessor on each DCC channel replicated
  the actions of the hexidecimal devices, rapidly
  accessing memory contents
• The remaining 3 microprocessors implemented a
  typical CRT-based operator interface using the
  data acquired

9
Results

• The essential characteristic of the PACE upgrade
  is that substantial improvements were achieved
  without compromising the proven performance of
  the control programs
• Now that the microcomputer network is a quarter
  century old, a second operator interface upgrade
  has been initiated. It will use modern technology
  but again will not disturb the DCC hardware and
  software

10
Applying the Strategy

• For the DCCs at Pickering B, the 1st element of
  the strategy was applied by upgrading a number of
  obsolete devices including
• core memory boards
• watchdog timer
• printer
• moving arm disk
• power supply
• fuel handling interface

11
Results

• With the upgraded devices installed, the system
  health of the DCCs at Pickering B will improve
  from yellow to white this year as soon as soon as
  two software changes are installed (this
  installation has already been completed on some
  Units)
• The reliability and maintainability improvements
  have been made without compromising the control
  program integrity

12
Applying the Strategy

• When making control computer aging management
  strategic decisions, technical risk is far from
  the only factor considered
• When the four Canadian nuclear Utilities got
  together to decide on a joint replacement for the
  Varian DCCs used in many CANDU plants, a set of
  24 criteria was applied
• A number of alternatives were investigated
  including distributed control systems (DCSs)

13
Applying the Strategy

• Development and Implementation Cost
• Operation / Maintenance
• Risk Due to System Design Changes
• Risk Due to Software Changes
• Risk Due to Project Complexity
• System Reliability
• Long-term Support Availability
• Vendor Attitude
• Human Factors Engineering
• Licensibility
• Maintenance Re-training and Documentation
• Operator Re-training and Documentation

14
Applying the Strategy

• Obsolescence Risk
• System Documentation
• Space / Cabling
• Software Qualification
• Hardware Qualification
• Simulator Impact
• Testing / Verification Scope
• Implementability
• Performance and Stability
• of Users (User Base)
• Maintainability
• Expandability

15
Applying the Strategy

• When the four Utilities applied the criteria, the
  hardware emulator approach was the unanimous
  choice and with a 5 to 1 margin
• A Varian emulator existed and was in use at
  several off-shore CANDU stations, but the company
  was about to exit the business
• A COG Joint Project was formed and in the end all
  8 CANDU utilities plus AECL pooled resources to
  buy the manufacturing rights

16
Results

• In re-manufacturing the Varian emulator, the
  opportunity was taken to upgrade some of the
  technology. For example, an unreliable power
  supply was replaced by a dual-redundant and
  hot-swappable device
• The prototype passed acceptance testing earlier
  this year. Initially 150 will be built.
• A long-term support agreement will guarantee
  availability for the next 25 years

17
Applying the Strategy

• At Darlington, the same 2-pronged strategy was
  applied. First a number of weak devices were
  re-designed and replaced including power
  supplies, memory and disk drives
• Since no hardware emulator was available for the
  Darlington DCCs, a company was commissioned to
  produce one
• The latest system health report shows DCC heath
  tracking better in direct correlation to the
  installation of the improved devices

18
Results
19
Looking Ahead

• This aging management strategy has proven to be
  uniformly successful by dealing with obsolete
  hardware devices without compromising the proven
  control programs
• With Intellectual Property rights, a Utility can
  ensure future availability consistent with
  Station extended lifetimes

20
Looking Ahead

• Computer-based control systems have increasingly
  short life cycles
• Conversely, nuclear power plant life-times are
  becoming increasingly longer
• A Utility may be faced with replacing the brain
  of the generating station 4 or 5 times during its
  life-time
• Increasingly shorter outages mean that control
  system replacements cant be done during normal
  outages

21
Looking Ahead

• If not planned properly from the outset, dealing
  with control system aging could have a major
  negative impact on future plant capability
  factors and costs
• Also, if the main control room operator interface
  is too tightly coupled with the reactor control
  system, each control system replacement will also
  mean re-training plant operators and re-doing
  operating and abnormal response procedures.

22
Looking Ahead

• Techniques do exist for building in future
  flexibility both at the system design level and
  at the software engineering level
• Establishing a reactor control system aging
  management strategy should be part of initial
  planning for all nuclear power plant life
  extension or new build projects