Setting the Standard


NERC’s new cyber security rules may minimize cost of compliance, but they leave utilities guessing on how to identify risks.

Fortnightly Magazine - January 2008

Liam Baker, vice president for regulatory affairs at US Power Generating, questions whether his company’s power plants and control systems in New York and Massachusetts must comply with the electric industry’s new mandatory standards for cyber security.

Baker voiced his doubts in written comments he filed in October with the Federal Energy Regulatory Commission (FERC). It all starts with ISO New England (ISO-NE), which runs the market dispatch for the power plants that US Gen owns through its indirect subsidiary, Boston Generating, LLC.

“ISO-NE’s dispatch philosophy,” Baker explains, “is the system can always withstand the sudden loss of the largest generator in the region.” And New England’s capacity market,” he adds, with its built-in reserve margin, “further ensures that, even on the hottest peak-load days, there is sufficient operable capacity to securely cover load with multiple large generators out of service.”

Thus, Baker questions how any single power plant, grid facility or related computer control system can qualify as absolutely “critical,” a finding required for cyber standards to apply. Baker concedes that some facilities remain essential, such as special black-start units, or other “must-run” plants that supply reactive power or some other unique local reliability need. Beyond that, however, he suggests Boston Gen or virtually any other generator in New England “could logically assume” none of its individual plants are “critical” to the bulk electric system.

“Without additional guidance,” he adds, there is no way of knowing if this is an appropriate assumption.” (See, Comments of US Power Generating Co., pp. 3-5, FERC Docket No. RM06-22, filed Oct. 9, 2007.)

Baker has pinpointed the “N-1” theory that underpins the electric reliability standards created and enforced by the North American Electric Reliability Corp. (NERC). By building in multiple levels of redundancy to guard against all adverse first-level contingencies, the industry in effect ignores the likelihood of any single failure. The remedy lies not in preventing breakdowns, but in designing the regional bulk-power grid so the system will survive them. In effect, no asset is ever allowed to become “critical” in the first place.

By contrast, however, the new cyber standards require proof of criticality—that a given failure could wreak havoc. That brings enforcement into the realm of probability theory, where the risk analysis may turn on a roll of the dice.

Doubts in Congress

On Capitol Hill, the House Committee on Homeland Security, led by chairman Bennie Thompson (D, Miss.), asserts that NERC’s decision to make cyber security contingent on the criticality of physical and cyber assets, represents “a conceptual mistake.”

Thompson’s House committee held a hearing in mid-October to hear testimony from computer security experts after news had leaked that government engineers staged an experimental cyber attack that succeeded in taking over computer control systems for a power plant located at the DOE’s Idaho National Laboratory. (See, “Aurora Attack”)

Moreover, Thompson joined with two other House Subcommittees (Emerging Threats, Cybersecurity, Science and Technology; plus Transportation Security and Infrastructure Protection) to file formal written comments with FERC demanding a rethinking. The congressmen fault the idea that cyber security rules should focus only on risks that threaten a complete breakdown of a regional grid.

At the October hearing, star witness Joseph M. Weiss, managing partner for Applied Control Solutions, claimed Congress and FERC created a potential “conflict of interest” by relegating electric system cyber security to a group (NERC) still funded by the industry. As a member of NERC’s drafting team for the cyber standards, and as a professional engineer and certified information security manager (CISM), Weiss echoed Liam Baker’s analysis at US Gen:

“It so happens,” said Weiss in his testimony, “that many of the largest electric utilities have determined in their risk assessments that they have no—zero—critical generation assets … nuclear included … [because] their systems have been designed to withstand ‘N-1’ contingencies.”

That ignores the threat of simultaneous multiple contingencies, Weiss said. “What if a Trojan Horse planted in numerous generation control systems should awaken at the appointed hour,” he asked, “and simultaneously trip a whole collection of plants in a region? “Very possible scenarios such as this are being discounted out of hand by people in positions of authority who really do not understand cyber security.”

Taking a cue from Weiss, the House committee called for standards that would safeguard national defense and infrastructure, plus the welfare of citizens and communities.

“Every critical infrastructure in the country,” the committee reasoned, “is dependent upon the bulk power system: chemical plants, banks, refineries, hospitals, water systems, and military installations.”

Yet Congress well understands that economic and societal considerations fall outside the scope of authority that it granted to FERC and NERC (as the nation’s designated ERO — Electric Reliability Organization), in EPAct Sec. 1211 (which amended Federal Power Act sec. 215). The issue received a thorough airing out last year, in the run-up to FERC’s Order 693 (March 2007), approving the first of NERC’s proposed mandatory reliability standards. At that time the American Public Power Association accused FERC of focusing too much on how “newsworthy” a blackout might be, especially if it took out electric service in New York City or Washington, D.C. (See, Commission Watch, “The Rush to Reliability,” Feb. 2007.)

The NERC standards define the term “critical cyber assets” to include only those programmable electronic devices and communication networks essential to the reliable operation of other facilities, known as “critical assets,” which if destroyed, degraded or forced off line, would affect reliability or operations for bulk power system. Critical cyber assets also must feature connectivity through an Internet Protocol routing, or else dial-up Internet accessibility, as explained in NERC’s proposal and in FERC’s initial evaluation in its recently issued notice of proposed rulemaking (NOPR). (See, Docket RM06-22, 120 FERC ¶61,077, July 20, 2007 (NOPR Decision). See also FERC Docket No. RM06-16, filed Aug. 28, 2006 (NERC Proposal).)

Concerned over NERC’s narrow view, the House committee proposed instead that the electric industry should adopt cyber security standards already developed by the National Institute for Standards and Technology (NIST). After all, federal and quasi-federal entities such as Bonneville Power and the Tennessee Valley Authority already must comply with the NIST rules. (See NIST SP 800-53, at

The industry objects that the NIST standards were developed for government information systems, and so might prove too costly for the private sector, or not mesh well with industrial control systems (ICS). Two recent developments suggest otherwise, however.

First, a recent report from the MITRE Corporation found NIST SP 800-53 could comport satisfactorily with NERC CIP cyber standards. (See, final.doc.)

Second, on July 13, four days before FERC issued its NOPR, NIST released an updated augmented version of its SP 800-53 cyber standards, designed for application to industrial control systems (ICS). The public comment period on the augmented NIST standards closed August 31.

As NIST senior research scientist Stuart Katzke explained in comments filed with FERC on October 5, “we have received no ‘show-stopping’ comments concerning the effectiveness of the NIST standards.” Katzke said the updated ICS version would become final by the end of calendar 2007.

The Risk Profile

Arizona Public Service leans toward the expansive view that the NERC cyber rules require a look at infrastructure and socio-economic vulnerabilities.

In written comments filed at FERC October 5 by Pauline Foley, senior regulatory attorney for Pinnacle West Capital Corp. (the holding company), APS proposed a multi-step risk-based process for determining which physical and cyber assets should qualify as critical. According to APS, the risk assessment should consider such factors as the number of customers affected by a loss of generating capacity, and whether an outage would affect “major customers,” or “other critical infrastructure.” (See “Identifying Critical Cyber Assets: The APS Model”)

Some small co-ops and municipal utilities anticipate trouble in judging how asset failures might affect the entire grid. FERC appears sympathetic. It suggests a “voluntary exchange of information” on cyber security, plus a formal procedure for NERC or its various regional reliability organizations (RROs) to provide external oversight of third-party decisions that identify critical assets, in order to assure a “wide-area view.” (See NOPR, ¶¶s 112, 113.) The majority of the industry disagrees, however, with some suggesting RROs would find the task overwhelming, as it would require particular knowledge of the capabilities of thousands of individual assets, plus minute details concerning software types, vendors, and vintages of equipment.

Meanwhile, NERC declines to include explicit guidance within the standards themselves to explain how to conduct a risk-based assessment to identify critical physical or cyber assets. “If ‘how’ language were included,” says NERC, “it would de facto become the only acceptable method. This would … potentially introduce common vulnerabilities … [and] could lead to other problems such as separate specific requirements for each manufacturer … as well as each model within a manufacturer’s product line.” (See, Comments of NERC, FERC Docket No. RM06-22, pp. 13-14, filed Oct. 5, 2007.)

NERC would prefer to draft a separate set of informal how-to “guidelines,” contained outside the standards proper. That would keep the guidelines flexible and allow updates outside the cumbersome standard-setting process, which requires FERC’s OK.

In fact, NERC already has offered some guidance in its “Frequently Asked Questions,” a 30-page document posted on its website:

QUESTION: Why aren’t all Cyber Assets associated with the Bulk Electric system required to be secured and protected under the Cyber Security Standards?

ANSWER: The implementation … is limited … by focusing on Critical Assets … that are essential to the operation of the bulk electric system … .

Yet consider another Q&A paragraph, appearing further down on the very same page:

QUESTION: Does redundancy of the Critical Asset or a Critical Cyber Asset change the criticality of these assets?

ANSWER: No, in NERC’s Cyber Security Standards, redundancy does not affect the criticality of any asset.

The DOE’s Western Area Power Administration may have solved the problem. WAPA claims it has isolated its SCADA and AGC systems so as to have “very little interaction” with the outside world.

“There is no access to the Internet, no email, and no connectivity to the corporate network.”

Cost of Compliance

FERC pegs the annual industry-wide cost of compliance at $24.7 million, but MidAmerican Energy counters that as a single company, its costs likely will equal a “substantial fraction” of FERC’s total estimate.

Reliant Senior Counsel Gretchen Scott notes that compliance costs for critical black-start generating units could well exceed revenues. As she explains in her company’s written comments, “Reliant is aware that numerous companies in the industry are evaluating whether they can continue to provide black start service when to do so would be at a loss.”

NERC handles compliance cost in a trio of concepts—“reasonable business judgment,” “technical feasibility,” and “acceptance of risk”—that all mean essentially the same thing. That is, an industry participant can decline to fix a cyber risk if it would prove too costly or create new risks for operations or reliability. Such ideas, however, have riled regulators and software experts.

In December 2006, in its preliminary assessment, the FERC staff warned that “for interconnected control systems of various entities, an acceptance of a cyber risk by one entity marks an acceptance of the risk for all connected entities. The staff report added that any party accepting risk would become “the weak link in the chain.”

That warning led FERC in its July 2007 NOPR decision to instruct NERC to remove all traces of the term “acceptance of risk” from the final standards. FERC ordered the same fate for “reasonable business judgment.”

However, NERC now has conceded that FERC “makes a strong case” for killing the reference to “reasonable business judgment.” Utilities and NERC now say they do not object to removing language referring to “risk acceptance” or “reasonable business judgment”—as long as the re-write occurs through NERC’s stakeholder process, and retain a degree of flexibility.

A key question remains, however. Can companies claim technical infeasibility simply because costs are burdensome, as NERC insists? Or must utilities bend to FERC’s will, as expressed in the NOPR, and make costly upgrades or replacements of legacy control systems?

NERC proposes to change the term “technical feasibility” to “exception for reliability.” That would reflect the overriding industry sentiment that, in comparison to control centers and modern corporate IT systems, many legacy control systems used for field assets (substations, gen plants) employ UNIX programs or pre-Internet, pre-desktop applications that, quite simply, cannot be patched without threatening operations and jeopardizing reliability.

National Grid Attorney Joel deJesus explained the problem in his company’s written comments to FERC: “Most equipment covered by the CIP standards, such as relays and remote terminal units, utilizes proprietary embedded software that has absolutely no common heritage with the typical desktop PC environment.

“Such proprietary and closed environments,” he continued, “often do not include desirable security features (authentication, filtering, virus scanning, logging, etc.) and simply cannot be upgraded to include such feature without extensive vendor co-operation and/or wholesale equipment replacement.”

Software engineers voice dismay at such claims. Consider the Instrumentation Systems and Automation Society, now at work on a comprehensive set of cyber security standards (the ISA99 project). The project’s five-man leadership team (four of whom were members of NERC’s standards drafting team) filed written comments at FERC:

“It is not acceptable, in our view, to identify unacceptable risks, and then leave them because the existing equipment cannot be appropriately hardened.”

Entergy offers an interesting solution: It proposes that NERC’s cyber security standards in the near term should apply only to control or data centers, which more likely employ more up-to-date software applications, to allow time for the standards drafting team to better define risk assessment methods for field assets. (Entergy also recommends using the NIST standards.)

Back at National Grid, attorney deJesus sees a hidden silver lining. He notes that the older proprietary software found in legacy utility control systems marks “a positive attribute from a cyber security point of view, in that the ubiquitous malware targeted at the typical desktop PC environment will not function.”

“While it is true,” he says, “that ‘security via obscurity’ is not a sufficient defense, such obscurity is a valuable addition layer.”