System Disturbance and Blackout Analysis

System Disturbance and Blackout Analysis: Identifying Trends in System BehaviorRobert W. Cummings21mpe03-cummings-opener-3247050The electric industry can learn many lessons from the analysis of both system disturbances and blackouts. It is extremely important to analyze both to identify trends in system behavior that are initiating, causal, contributory, or merely coincidental. Those trends can help prioritize efforts to eliminate, or at least reduce, the probability of recurrence of such events.For purposes of this discussion, the analyses are limited to system events on the bulk power system (BPS), not on the distribution system. Customer outages caused by weather events, such as storms, are not addressed. However, at times, the behavior and performance of distribution systems or the loads can factor into any system event. The article will also discuss such occurrences.The major difference between a system disturbance and a blackout is the loss of load; system disturbances do not necessarily result in load loss, but blackouts do. For example, a major system disturbance in 2007 resulted in the tripping of nine generators totaling 4,457 MW across three states and three North American Electric Reliability Corporation (NERC) regions, but only 37 MW of local load was lost. Although this was a serious disturbance (it was the largest generation loss in the Eastern Interconnection other than in the major blackouts in 1965 and 2003), the event was not widely reported, and it was largely out of the public eye.It is also incredibly important to cast a very wide net when preserving and collecting data. Initial data preservation requests must be done on an interconnection-wide basis, with data collection requests being able to focus on the area of highest impact. In the 2007 disturbance, most of the generation was lost in Indiana and Illinois, but four generators tripped or rapidly reduced output (174 MW total) in New York City as the result of turbine controls being sensitive to system frequency deviation (a minimum frequency of 59.863 Hz was recorded) caused by the tripping of the other generators. Those same generators had tripped in the 2003 blackout with no apparent reason discovered at that time (mystery solved).Another very important aspect of event analysis is to constantly be on the alert for repeated occurrences of problems, even in small disturbances. Early detection of trends of minor problems can prevent them from becoming contributory to larger system disturbances. The sensitivity of the New York City generators to frequency deviations cited previously is a good example.Importance of Attention to DetailSo, what is important in analyzing an event? Everything!Some basic tenets of those analyses are the following:

Know the operating condition of the system immediately prior to the disturbance throughout the interconnection. There are four major interconnections in the United States and Canada: Eastern, Western, the Electric Reliability Council of Texas (Texas), and Québec. All four are operated at a nominal 60 Hz, but they are not synchronized with each other, being connected only through dc ties. Often, system events have impacts throughout the interconnection.

If the disturbance trips significant amounts of resources and/or load across a large portion of the interconnection, power flow and dynamics analyses are needed to verify the sequence of events during cascading and transient interactions between impacted system elements.

Establish the duration of the event from inception to when “system normal” conditions are reestablished.

Establish what system elements were tripped or were operated during the event.

Establish an accurate sequence of events, typically requiring accuracy down to the millisecond level using high-speed data measurements from phasor measurement units (PMUs), digital fault recorders, digital relay data, and, when inverter-based resources (IBRs) are involved, point-on-wave data.

Determine the exact reason for the triggering of each relay or remedial action scheme on the transmission system. There is no such thing as a “sympathy trip.”

Determine the exact reason for tripping each resource—it may be electrical, mechanical, or controls that caused the trip, not necessarily a relay.

Look for other concurrent events across the Interconnection—other units or lines tripping may be the result of misoperation or miscoordination of protection systems. This tenet also includes looking for events that happen even seconds later, which may be caused by perturbations created by changes in frequency, voltage, and/or current.

Look for signs of interarea oscillatory behavior. Natural modes of such low-frequency oscillations have typically been less than 1.0 Hz. However, with the rapid deployment of IBRs, interarea oscillations may no longer be dominated by the physical characteristics of synchronous machines. The reason is that IBRs can oscillate at any frequency because of their computer control systems.

It should be noted that any single-line-to-ground fault in a substation left uncleared will eventually become a multiphase fault. That constantly observed fact is important in the world of IBRs because we have little to no experience with their behavior during protracted faults.

Four Types of Events in a DisturbanceThe following four basic types of events occur in a system disturbance:

Initiating event: An event that kicks off a system disturbance such as a lightning strike, a line or generator tripping, or a failure of a transmission system element. The initiating event is not necessarily the root cause of the disturbance.

Causal event: This event is often described as the “root cause” of a disturbance, but it may be well into the sequence of events and may require other contributory factors to set the stage for it to happen.

Contributory event: This is any event that contributes to the severity of the disturbance or may be a precursor or prerequisite for the causal event to really create a problem.

Coincidental event: This includes any event that occurred during the sequence of events of the disturbance but cannot be directly tied to a cause from the rest of the event. One needs to be careful about declaring an event as only coincidental; the relationship to another event in the sequence of events may be difficult to immediately identify.

Importance of Determining Actionable Causes for EventsThe analysis of an event should always be aimed at determining “actionable” root causes. Sometimes, too deep of an analysis may result in a root cause that is not correctable. For instance, one could argue that a root cause in the 2003 blackout was greed—someone wanted to save money on tree trimming, and they did not adequately trim trees in their rights-of-way. The blackout was initiated by lines sagging into the trees.Protection systems are notoriously blamed when they misoperate (fail to operate when they should or operate unnecessarily). Wherever protection systems are concerned, correct wiring should be verified, and relay, digital fault recorder, or PMU data should be used to help determine what the relay actually “saw.”Remember too that sometimes system elements can and do fail unexpectedly. Protection systems are often described as failing. In today’s age of digital relays, equipment failures themselves are rarer. Often a protection system “failure” can be traced back to a bad setting, a miscalculation of the setting, or a misapplication of the protection feature itself, not to an equipment failure.Question EverythingThere are several phrases used in explaining system events that need to be questioned, corrected, or become part of a recommended corrective action. The following are examples of phrases that should be questioned:

The protection system performed as designed: The system may have performed as designed, but the design may be flawed, and the trip may have not been desirable or expected.

The unit or circuit had a sympathetic trip: The protection systems have no emotions and are not drinking buddies. Each protection system operates for a very specific reason; each trip must be explained in full.

That part of the system is always in alarm or slightly overloaded: Neither of these explanations is acceptable; this phrase it indicates complacency with an off-nominal condition. During the 2003 blackout, a strain bus was the limiting component on a 138-kV circuit. The line’s conductor was rated for much more current, but because the rating and the associated alarm were based on the strain bus rating, the constant alarm on overload was ignored by system operators.

The generators tripped because all lines were lost from the plant: Although this is sometimes true, often the reason the last line(s) tripped is that the units went unstable. This cause was the case with a 2007 system disturbance where two large generators were isolated to one transmission line. The initial thought was that the units tripped because the last line tripped. In fact, when you do a time-synchronized analysis between the relay action and the high-side breakers opening, the digital fault recorder records for the last line to trip show a classical curve of the units going out of step before the relay actions.

The unit tripped on reverse power: This case is tricky because there is undoubtedly proof of the reverse power relay having operated, but that cannot be viewed as the cause of the unit tripping. It is important to remember that many plants depend on the reverse power relays to open the unit breakers following any sort of boiler or turbine trip. This occurrence is very noticeable on nuclear units when the reactor scrams for other than electric system problems. The generator is allowed to coast down, extracting any latent energy in the steam cycle. Consequently, after several seconds following the scram, the unit breakers are opened by the reverse power relays. Yes, those relays did operate, but the cause of the unit’s demise was not reverse power.

Reverse power relay potential problems: During a 2015 system disturbance, two generators tripped offline at different times because of exciter design differences between the units—one unit had an exciter powered by the auxiliary power system; the other had on-shaft self-excitation. Both units experienced a prolonged voltage degradation to each unit’s auxiliary equipment service buses. One unit’s breakers tripped immediately because of the loss of the auxiliary power supply to the generator exciter, initiating a loss of excitation trip that immediately opened the high-side breakers. On the other unit, the low voltage caused a loss of auxiliary equipment service buses that scrammed the reactor, but the shutdown mode was waiting for the generator to coast down and eventually let the reverse power relays trip the high-side breakers. That unit had electromechanical reverse power relays, with the typical 20 s (cumulative) of reverse power flow being necessary to cause a trip. Meanwhile, back at the unit shaft, initially, the exciter kept on trying to excite the unit, causing the production of some power from the generator. Then, it ceased to excite and caused the power flow into and out of the unit to cross zero approximately 50 times (exhibited in the unit’s reverse power alarms). The electromechanical reverse power relay had to reach a total of about 20 s (cumulatively) of reverse power into the generator before it would activate. The breakers were finally opened about 52 s after the reactor had scrammed and stopped supplying the turbine with steam.

Feeding reactive power into both ends of a tie line: Often a transmission operator can be fooled into thinking that their control area is holding up the entire system by supplying large amounts of reactive power to his neighbors while its neighbors believe that their system is supplying large amounts of reactive power to the other area…both are right, and both are wrong. When a transmission circuit is loaded significantly above its surge impedance loading, real power may be flowing in one direction across the line, while reactive power is flowing in from both ends. Such was the case in several lines involved in the 2003 Northeast blackout, which were loaded to three to four times the surge impedance loading of the lines. Under those conditions, a voltage gradient appears from both ends of the line, approaching zero volt somewhere in the middle. Examination of fault recordings from several lines bore this out, with each end tripping on directional distance relays for opposite quadrants of the relay impedance target diagrams.

The Changing Face of Event Analyses for a Changing SystemYes, Alice, we are still going to have a transmission system in the future to serve large load centers, such as New York City and Los Angeles…there is simply not enough physical space for sufficient local solar and wind resources to serve those load centers.Event analysis as a science has significantly changed since the 1965 Northeast blackout. Back then, there were no digital relays, oscillography was far more primitive (on paper recordings), the timing in a sequence of events was rarely measured down to milliseconds, and the industry was just getting its feet wet in digital simulations, which required the largest mainframe computers to perform even basic power flow and dynamic analyses.Over the years, the electric system continues to evolve as renewable resources, such as solar and wind power, rapidly increase, and battery energy storage systems are added to the mix. All of those resources are coupled to the system by power electronics (inverters) and create new challenges to the engineering and operation of the electric system. When IBRs supplant retiring fossil-fueled generation (rotating machines), there is an inherent reduction of system inertia and synchronizing torque. Reduced fault current may require redesign and modification of transmission protection systems, and the distributed nature of the new resources makes the well-understood flows from central generating stations to load centers insufficient. In addition, the new resources often require new transmission lines to collect and distribute their output.The very nature of the load composition and behavior is also changing as well. Traditionally, loads were largely resistive with some synchronous motors. However, models for those loads are now obsolete for accurate power flow and system dynamic simulations. The new load models must be capable of representing chargers for everything from phones to electric vehicles, LED lighting, variable-speed drive motors, microgrids, and the impacts of rooftop solar panels. The combination of embedded resources on the distribution system has made the load far less predictable and somewhat schizophrenic.As the system resources and loads have changed, so too must the forensic analysis techniques for system disturbances. The high penetration of IBRs and load changes relies increasingly less on the physics of rotating machines and progressively more on the control systems for all of the power electronics coupling the loads and resources to the system. Today’s power system engineer had better be well-versed in controls engineering, particularly involving the controls of IBRs and electronically coupled loads.The following discussion presents a sample of system disturbances over the last 20 years, highlighting what lessons were learned and how analyses have had to change.2003 Northeast BlackoutThings had improved somewhat by the time of the 2003 Northeast blackout, with significant advances having been made in computer-based power flow and transient stability analyses. The NERC Reliability Coordinators were new functions, and their state estimation systems were still under development. There were several high-speed digital fault recorders and several digital relays in service as well as some sequence of events recorders, but none of them was synchronized to a time standard (such as a GPS synchronizing signal). That made it a monumental task to place the thousands of events during the disturbance (lines and generators tripping, relay actions, and so on) into a reasoned, detailed sequence of events and verify the causes of each of the trips. For the large part, operator logs, voice recordings, and data from 4-s scan rates from supervisory control and data acquisition (SCADA) systems were the best available. But even for those data, the time stamps were unreliable; time stamps were often added to the data when the computers “logged in” the data at the control centers.Several man-months were spent on “daisy chaining” and correcting time stamps for the sequence of events. Fortunately, GPS-synchronized digital disturbance recorders had recently been installed on the Michigan–Ontario transmission interface, giving the engineers a time anchor for future disturbances (see Figure 1).cummings01-3247050Events were often observed in the current or voltage domains at two nearby locations of digital fault recorders, allowing a time offset to be calculated between the two locations. That process was then repeated for a third and fourth location (and so on), allowing a crude but effective sequence of events to be created. This process was repeated for several parallel paths and was, eventually, verified by power flow and transient stability simulations.Forensic power flow analysis was still an inexact science. For normal power flow solutions, one knows the real and reactive loads, generation, and shunt reactive devices (reactors and capacitors) at each bus. The power flow program then solves for voltages, phase angles, and line flows on the system elements. In a forensic analysis, voltages, loads, and line flows may be, at least, measured and recorded during the event, and the power flow program solves for the voltages and phase angles. The engineer must then iteratively adjust the real and reactive loads until the solved voltages and flows match more closely to the measured values. This process must focus on one set of relatively synchronized readings and must be done repeatedly to create a picture of the disturbance as it progresses. In the 2003 disturbance, forensic power flow analysis was possible for the hour before 4:10 Eastern Daylight Time (EDT) while several lines were tripping in the sequence of events.Some of the key takeaways from that 2003 disturbance include the following:

The need for reliability coordinators who have an interconnection-wide view of the behavior of the system in real time.

During the pre-4:10 EDT time frame of the disturbance, several 138-kV lines were tripped in northeast Ohio as they sequentially loaded up after previous trips and sagged into trees. Figure 2 shows the simulated impacts of that 138-kV cascade on the loading of the critical Sammis–Star 345-kV transmission line. The Sammis–Star 345-kV line was tripped by exceeding its “relay loadability” limit, a system protection setting where the line loading was thought to have exceeded its ability to carry power.

The need for a real-time measurement of phase angle between different zones of the system to enable reliability coordinators to detect system stress and, possibly, take corrective action. That would have been exhibited (see Figure 3) as a growing angular separation between the Cleveland area and the core of the Eastern Interconnection. Analysis showed that had about 2,000 MW of load been shed before the disturbance went “dynamic” at 4:10 EDT, the cascade could have been halted. Had that growing phase angle separation been known at the time, there was almost a whole hour from the beginning of the disturbance until the time that the event went “dynamic.”

The need to examine the entire extra high voltage transmission system to determine if there were any directional distance relay settings that could artificially limit the ability of transmission circuits to temporarily carry heavier flows (this is known as relay loadability) The purpose of this review susceptibility to premature tripping is to allow time for system operators to correct the overloads and avoid system cascades. It has often been argued that the relay loadability of a 230-kV line near Niagara Falls was the root cause of the 1965 blackout.

The need for time-synchronized high-speed measurements continuously recorded to enable trend analysis and the possible relation of events that are sometimes hundreds of miles apart. In the initial analysis, tripping of about 400 MW of generation in Maine was thought to be the result of the New England “island” rebalancing its load and generation. However, a detailed analysis showed that the generation that tripped did so as the result of a remedial action scheme being inadvertently triggered by a line tripping in central Michigan. The related detailed timeline also indicated that loss of that generation had caused the separation of New England from the rest of the Eastern Interconnection. The good news is that New England survived the disturbance as an island, whereas the New York system to which New England was connected blacked out.

The need to identify the causes of each line and generation trip is important. One unanswered question in 2003 was why several gas turbine generators in New York City tripped or severely reduced their output. That question would remain unanswered until another event in 2007 in which several generators tripping in Indiana and Illinois caused a frequency perturbation that was seen across the Interconnection. Those New York generators were impacted once again because their turbine controls were susceptible to frequency deviations.

An understanding of the operation of reverse power relays in disconnecting generators. Several large generating units remained connected to the grid as the frequency in the Michigan “island” deteriorated to near 24 Hz despite their prime movers (boilers and a nuclear reactor) tripping earlier during the disturbance. The reverse power relays that should have tripped them had no reverse power available from the system, and they finally tripped when the voltage caused the relays to “drop out.”

cummings02-3247050cummings03-3247050Westwing Disturbance: June 2004The 2004 Westwing disturbance in Arizona highlighted the potential problems caused by a single point of failure within a transmission protection system. A single-line-to-ground fault on the 230-kV system failed to be cleared by the protection system at Westwing substation because of the failure of an auxiliary relay to trip a 230-kV breaker. The result was a protracted fault that lasted for almost 38.9 s! That fault was finally cleared by the tripping by overreaching backup relaying functions of a nearby directional distance protection system. The voltage, current, and frequency perturbations caused by the fault resulted in the tripping of about 4,610 MW of generation, some as far away as Alberta, Canada, and the loss of about 995 MW of load. The frequency in the Western Interconnection fell to 59.54 Hz.This disturbance caused by the failure of a single auxiliary relay eventually caused the electric industry in North America to examine their protection systems for potential single points of failure. The duration of this fault also highlighted the truism that a single-line-to-ground fault left uncleared will eventually become a multiphase fault. But the event also set the stage for event analysis being wary of protracted faults and their potential to cause cascading of the electric system, which was further emphasized by several other disturbances discussed in this article.Western Interconnection Forced Oscillation Event: September 2005One of the vexing problems developing for a system is the detection of forced oscillations caused by misbehaving control systems. In this event, the control system on a small (30-MVA) generator excited a 0.395-Hz oscillation on the California–Oregon Interties. Since the oscillation was close to the north–south natural system frequency (0.403 Hz), the resultant oscillation was observable throughout the Western Interconnection. The California Independent System Operator (CAISO) Reliability Coordinator modal analysis software detected the low damping on the North–South Mode B. Such forced oscillations can become more problematic as control systems of the increasing number of IBRs come into play.Eastern Interconnection Frequency Event: August 2007This disturbance was of a different nature—virtually no load was lost—but it warranted careful analysis because about 4,260 MW of generation was tripped in the Eastern Interconnection, causing the frequency to dip to 59.864 Hz. The event was initiated by six single-line-to-ground faults on a 765-kV line equipped with single-pole switching and related to fast-valving action by the generators associated with that line.The event solved the mysterious gas turbine trips in New York City mentioned in the 2003 Blackout analysis that had controls susceptible to frequency deviations. The event also highlighted the susceptibility of some generation to trip due to a unit digital control system function known as the power–load unbalance (PLU) function on three generators. That function is used to prevent turbine overspeed instead of a mechanical overspeed protection system. The PLU logic senses turbine mechanical input in terms of steam flow and pressure and generator current output to quickly determine if the unit has suffered a complete loss of outlet paths, also termed 100% load rejection. The PLU function typically triggers a rapid closing of the steam valves, reducing the mechanical power input to zero, thereby preventing further acceleration of the unit. The valves are subsequently reopened if the PLU condition clears. Three units in Indiana and Illinois (totaling about 1,670 MW) experienced PLU operations, contributing to the severity of the generation loss for the Eastern Interconnection.Upper Midwest System Separation: September 2007In this system disturbance, a very wide variety of system conditions was exhibited, rivaling the 2003 Northeast blackout in complexity. Initially, North Dakota, Minnesota, Manitoba, and Saskatchewan separated from the Eastern Interconnection with overfrequency in the resultant “island.” Saskatchewan then separated from Manitoba and North Dakota because of overfrequency conditions at the Boundary Dam generating station. New overfrequency protection systems had been recently installed at those generators, but the relays were left at factory settings (60.5–61.0 Hz), tripping 711 MW on seven generators, which were all well within the 102% overfrequency continuous capability of the machines. Although the machines were designed for their potential exposure to overfrequency in that part of the system, the underfrequency relay settings were not adjusted for those potential conditions. Undesirable tripping of the Boundary Dam generation because of these settings was deemed to be causal in the separation. As a result of the separation, SaskPower lost about 900 MW of load and generation because of underfrequency protection.Resynchronization of Saskatchewan to the Eastern Interconnection was hampered by high voltages and large standing angles across breakers on open-ended tie lines between Saskatchewan and Manitoba. Those conditions caused the misoperations of some synchronization protection schemes.South Florida Disturbance: February 2008The South Florida disturbance was initiated by a single-line-to-ground fault that was not immediately cleared because the protection systems were turned off by a technician during troubleshooting. As with all single-line-to-ground faults left uncleared, this one evolved into a three-phase fault that was finally cleared by remote protection systems. The disturbance included tripping of 25 transmission circuits, about 4,000 MW of generation tripping, and a loss of about 2,300 MW of load due to underfrequency load shedding (UFLS) relays.Figure 4 shows the area (in blue) where the underfrequency of 59.82 Hz caused the UFLS system to trip about 2,300 MW of load across the Florida Reliability Coordinating Council (FRCC) system. The low frequency in northern Florida did not exceed the UFLS trip points and tripped no load.cummings04-3247050Some important aspects and results of the South Florida disturbance are the following:

The system impacts for loss of load were limited with the FRCC footprint.

The majority of load lost was restored within 2 h.

No major equipment damage was reported.

The two nuclear units that tripped did so as designed.

There were no thermal overloads of the transmission system.

The South Florida disturbance marked the first use of PMUs across the entire Eastern Interconnection to show the wave-like behavior of the frequency, voltage, and current perturbations traveling across the interconnection. Figure 5 shows the temporal displacement and attenuation of the oscillations at different locations throughout the interconnection. An analysis of the oscillations at distant locations across the interconnection also highlights the wide range of potential impacts.cummings05-3247050

The frequency perturbation is most pronounced in magnitude in the Florida Power & Light (FPL) system. Also, being closest to the actual disturbance, the perturbation occurred in FPL first.

There are 1,000-MW swings across some 500-kV lines in the Tennessee Valley Authority (TVA).

There are 30-kV swings on the 500-kV system in Ontario, east of Toronto.

There are 12-kV swings on the 230-kV system in Toronto.

There are 600-MW swings on the 345-kV transmission lines in western Massachusetts.

Arizona–Southern California Outage: September 2011This system disturbance, which resulted in the loss of more than 7,800 MW of load and almost 7,000 MW of generation, was initiated by the tripping of a 500-kV line due to a series capacitor switching error. Over an 11-min period, the underlying transmission systems between Arizona and San Diego cascaded and eventually blacked out San Diego.Some notable observations were as follows:

This was the first use of time-synchronized SCADA data and PMU data together to provide a detailed sequence of events. Once those data were delivered to the analysis team, it only took a few hours to perform a task that had taken months to perform for the 2003 Northeast blackout.

When the 500-kV line tripped, the standing phase angle difference and voltage differential across the open 500-kV breaker were too large to reclose. Redispatch of both the Arizona and San Diego systems would have been necessary, but time ran out.

There was an insufficient amount of recorded data available about the performance of the generation and its reasons for tripping in the San Diego/Comisión Federal de Electricidad (CFE) area to draw any conclusive cause for the UFLS effort not being successful. However, once islanded, despite shedding 75% of its system load, there were too many generation trips happening too rapidly to rebalance the area (see Figure 6). Generation trips were speculated to have been caused by low voltage and rapidly declining frequency—the very things that the UFLS was intended to backstop against. Figure 6 illustrates the battle to regain load/resource balance in the San Diego island. Load continued to be tripped by UFLS, but generation tripped even faster because of depressed voltages and frequency, causing the ultimate collapse of the island.

Some remedial action schemes were activated along the California–Mexico border, undesirably tripping generation by misinterpreting system conditions (tripping correctly for what they were seeing, but not as intended).

cummings06-32470502015 Washington, D.C., Disturbance: 7 April 2015In this disturbance, the Washington, D.C., area experienced a severe, prolonged voltage sag caused by a protracted 58-s fault on the 230-kV system in eastern Maryland. The event was initiated by the failure of a 230-kV lightning arrester and protection system failure to isolate that fault. About 530 MW of load was lost because of customer’s loads automatically switching to backup power sources (such as the White House) and customer protection systems separating from the grid because of the low voltage. Almost 2,000 MW of generation was tripped during the event, including two nuclear units at Calvert Cliffs Nuclear Power Plant in eastern Maryland.The protection system failure was due to failure to operate of auxiliary relay trip circuits on redundant protection systems; one system failed because of a loose connection; the other appeared to be an intermittent discontinuity in the circuit. Those protection systems employed earlier “teenaged” digital relays that required auxiliary relay circuits to handle the currents necessary to trip breakers.The Washington, D.C., disturbance also uncovered the reverse power relay potential problems discussed earlier.Inverter Base System Disturbances: Moving Into the Grid of the FutureBeginning in 2016, the North American electric system began experiencing system disturbances caused by tripping of IBRs. For a significant period, the industry had believed the majority of IBRs to be limited to rooftop solar systems. However, the solar industry had moved well beyond the rooftops and had developed substantially sized solar plants (from 200 to 500 MW) in the desert Southwest. Those plants consisted of several IEEE Standard 1547-2004-compliant inverters installed en masse to create large solar facilities. However, the voltage and frequency ride-through requirements of that standard were insufficient to provide the needs of the BPS, where momentary cessation could not be tolerated as it could on the distribution system in small photovoltaic (PV) facilities.Forensic event analysis was now facing the relative unknown realm of power electronics applied at utility scale (1 MW and higher). Power engineers had to learn different control modes of IBR operations: Continuous operation mode: Actively inject current into the grid

Momentary cessation mode

Momentarily cease injecting active current into the grid but remain electrically connected
Triggered by abnormal system voltages (<0.9 or >1.1 per unit)

•Trip mode (cease to energize)

Ceased injecting current and will delay returning to service (typically in about 5 min)
May also mechanically disconnect from the grid.

Accurately representing IBRs poses an increasing challenge in event modeling because very little of these IBR dynamic behaviors are modeled in conventional power flow and dynamic simulations.

The grid operators also were faced with a world where SCADA scan rates are almost useless and even PMUs are too slow to capture the millisecond-level dynamic behavior of the IBRs. Similarly, the interarea oscillations that had been involved in several system disturbances are no longer limited to the oscillatory behavior of rotating machines (synchronous machines). Moreover, the interactions between inverters and other power electronics and controls are no longer confined to the sub-1-Hz frequencies dictated by the physics of the machines. IBRs are, after all, computer-controlled devices that can react in milliseconds to system conditions. If not well coordinated, the IBR response speed may destabilize the overall power grid; if used carefully, that speed can be used to tremendous advantage for system stability.

With the growing number of IBR facilities, there is an ever-increasing chance of control interaction and forced oscillations. If the frequency of the resultant oscillations is close to the resonant frequency (i.e., the natural oscillatory modes of the interconnection), the forced oscillation created by a 200-MW plant in Florida can be observed across the entire interconnection.

Also of concern is that the PMUs themselves, which have been invaluable in observing system oscillations and overall performance, cannot accurately measure perturbations far outside of their primary designed bandwidth.

NERC has published several technical reference documents, recommended practices, and issued industry guidance on the best practices for incorporation of IBRs into the BPS.

Several IBR-Related System Disturbances

There are several system disturbances in the recent past where significant amounts of IBRs and other resources were lost (“tripped” or went into momentary cessation) when the IBR performance was undesirable for common system events (faults and switching). These include, but are not limited to the following:

Blue Cut Fire disturbance (2016)

Canyon 2 Fire disturbance (2018)

Palmdale Roost and Angeles Forest disturbances(2019)

San Fernando disturbance (2020)

Odessa disturbances (2021)

California 2021 disturbances (2021)

Victorville, 24 June 2021

Tumbleweed, 4 July 2021

Windhub, 28 June 2021

Lytle Creek Fire, 25 August 2021

Texas Panhandle Wind Event (2022)

A closer look at some of these disturbances can be instructional about the potential problems the industry is facing in event analysis. In all cases, there is very little, if any, point-on-wave high-speed data recordings to adequately analyze specific IBR behavior.

Blue Cut Fire Disturbance: 16 August 2016The Blue Cut Fire disturbance kicked off a rash of significant system disturbances caused by IBRs during what are considered to be normal transmission system operating conditions. That disturbance occurred during a forest fire in California that caused multiple line faults. The Blue Cut Fire itself caused 13 500-kV line faults and two 287-kV line faults. The most egregious was the fault at 11:45:06 Pacific Daylight Time.Several distinct observations of this event are listed here:

A 500-kV line-to-line fault cleared normally in 2.5 cycles (41.7 ms).

The PV resources impacted amounted to 1,178 MW (per SCADA 4-s scan-rate data). This was later backcalculated to be a 2,500-MW momentary loss using the Kundur equations from known system inertia at the time of the event (not seen by SCADA or even by PMUs).

There were 26 different solar developments involved in an area with a radius of more than 200 miles.

All IBR plants were connected at 500 kV or 230 kV (the only sizable transmission lines in that area).

Plants consisted of equipment from 10 different inverter manufacturers. None of the protection relays or breakers operated at the 26 solar sites; all action was taken by onboard inverter controls.

Following this event, a detailed report was issued by NERC in June 2017, highlighting the use of “momentary cessation” by the inverters (cessation to energize). NERC also issued an industry recommendation on 20 June 2017 to remedy the use of momentary cessation on IBRs wherever possible and to limit instantaneous tripping for frequency perturbations.Canyon 2 Fire Disturbance: 9 October 2017The California fires continued to wreak havoc with IBRs in the electric system. This event included the following:

A normally cleared 220-kV phase-to-phase fault occurred followed by a normally cleared 500-kV phase-to-phase fault.

SCADA recorded a 900-MW resource, later backcalculated to a 1,500-MW loss from interconnection inertia.

The disturbance reflected no frequency-related IBR tripping but showed a continued use of momentary cessation and signs of transient-overvoltage-related tripping of IBRs.

Following this event, a detailed disturbance report was published by NERC in February 2018 with a second industry advisory issued in May 2018.Palmdale Roost Disturbance: 11 May 2018This event involved

A fault on a short 500-kV line—cleared in 3.6 cycles

approximately 900 MW of BPS-connected PV involvement, again with load-embedded PV resource tripping. Similar load changes to those in Figure 4 indicate rooftop solar involvement.

Angeles Forest Disturbance: 10 April 2018This event involved the following:

A 500-kV “bolted” line-to-line fault occurred—cleared in 2.6 cycles.

The line is mixed 500 kV underground/overhead.

There were lopsided RLC line parameters—unbalanced line shunt reactors were used to compensate.

There was ∼1,100 MW in BPS-connected PV (SCADA recorded) during the evening PV ramp (see Figure 5).

There was evidence of a jump in the CAISO net load (see Figure 7) coincident with the disturbance—an indicator of distribution rooftop PV losses.

One natural gas turbine at a 2-on-1 combined-cycle power plant tripped offline because of the fault event on low gas pressure while loaded at 125 MW. That resulted in reduced output of the associated steam turbine from 150 MW to 75 MW over the course of 19 min.

cummings07-3247050Western Electricity Coordinating Council Base Case Review: IBRsA joint NERC–Western Electricity Coordinating Council (WECC) report from August 2020, titled “WECC Base Case Review: Inverter-Based Resources,” highlights the problems of accurately modeling the IBR fleet. Figure 8 illustrates the differences between actual system measurements and IBR model performance.cummings08-3247050It is obvious from those illustrations that the current models do not remotely reflect the realities of inverter performance.Observations and RecommendationsTo continue to monitor and improve the reliability of a BPS with the growing IBR penetration, there are several things that should be pursued in addition to the three-pronged approach being pursued by NERC: adoption of NERC Reliability Guidelines, improvements to the U.S. Federal Energy Regulatory Commission Generator Interconnection Procedures and Agreements, and enhancements to NERC Reliability Standards.

The electric power industry should adopt IEEE Standard 2800-2022, “IEEE Standard for Interconnection and Interoperability of Inverter-Based Resources (IBRs) Interconnecting With Associated Transmission Electric Power Systems.”

Better high-speed monitoring devices that can detect interarea oscillations across North American Interconnections are needed for the detection of forced oscillations and their sources.

Many more point-on-wave high-speed data recorders are needed across North America, especially near larger (100+ MW) IBR plants.

Much better IBR plant-level models are needed for the analysis of system events.

Remember, the new “smart grid” of tomorrow is only as smart as we envision it and make it happen today. What we do is not rocket science—it is more important!For Further Reading“Major event analysis reports,” North Amer. Elect. Rel. Corp., Atlanta, GA, USA. [Online] . Available: https://www.nerc.com/pa/rrm/ea/Pages/Major-Event-Reports.aspx“Event analysis, reliability assessment, and performance analysis,” North Amer. Elect. Rel. Corp., Atlanta, GA, USA. [Online] . Available: https://www.nerc.com/pa/RAPA/Pages/default.aspx“WECC base case review: Inverter-based resources,” North Amer. Elect. Rel. Corp., Atlanta, GA, USA. [Online] . Available: https://nerc.com/comm/PC/Pages/Inverter-Based-Resource-Performance-Task-Force.aspxBiographyRobert W. Cummings is a president and founder of Red Yucca Power Consulting, LLC, Albuquerque, NM 87111, USA.Digital Object Identifier 10.1109/MPE.2023.3247050Date of current version: 19 April 20231540-7977/23©2023IEEECoverSKM Power ToursMastheadDigsilentHow to Keep the Lights onPreparing for the FutureNuscaleBlackoutsUnexpected ConsequencesSystem Disturbance and Blackout AnalysisThe Utility Operational Response to the 14 August 2003 BlackoutReal-Time Grid ManagementChallenges in Operator TrainingFrom “Animal Crackers” to Winter Storm UriAre We Prepared Against Blackouts During the Energy Transition?Asking QuestionsIn MemoriamPES MeetingsReducing BlackoutsAd IndexAspen