Published on Fortnightly (http://www.fortnightly.com)

Home > Printer-friendly > Studying Apples and Oranges


 

RTO cost/benefit studies are difficult to reconcile.

The premise behind the Federal Energy Regulatory Commission's (FERC) push for regional transmission organizations (RTOs)-that they will provide positive economic benefits to society- increasingly is being challenged.

Over the course of the past year, at least five studies examining the costs and benefits of RTO formation have been published, with more certainly to follow.1 Not surprisingly (given the differences in approach, methodology, assumptions, and geographic scope), the results have added fuel to the ongoing national debate about the economic benefits that may or may not result from RTO formation.

While there are significant differences between the five RTO cost/benefit studies summarized in this article, they all share two principal characteristics. First, each concludes that creating RTOs will provide substantial benefits to society. However, only the RTO West study provides an attempt to detail the actual costs and, therefore, net benefits of RTO implementation. Second, each study also finds that while there are overall benefits from RTO formation, there will be winners and losers as a result of this process. These findings have raised further concerns, particularly among regulators, and are leading the debate in the direction of more detailed analyses of state-level customer class impacts.

The variety of differences in the time period, scope, and assumptions make comparing the various studies difficult. Even so, comparing the apples-and-oranges studies does yield some useful insights.

This article compares the methodologies, results, and critiques of the five published cost/benefit studies. The comparison is by no means exhaustive, focusing instead on the major issues and characteristics of the various studies. In addition, based on our experience with similar quantitative modeling assignments, we discuss what we consider to be among the most difficult issues in quantifying RTO costs and benefits, and also offer some thoughts on the next steps in the evolution of cost/benefit studies.

Analyzing the Past: The Mirant and NYISO Studies

The study that Energy and Environmental Analysis, Inc. conducted for Mirant attempts to quantify only one source of the potential benefits associated with a single Northeast market, and does not address any of the costs associated with establishing or operating a single Northeast RTO. 2 The study developed an estimate of potential efficiency benefits by comparing historical market price and power flow data to "revised" prices and flows that presumed more efficient dispatch of units in PJM, New York Independent System Operator (NYISO), and ISO-Northeast (ISO-NE) during periods when transmission was not constrained.

In conducting the analysis, the authors sought to identify and measure the benefits of eliminating uneconomic power flows from high-price areas to low-price areas. The study first identified periods when transmission was unconstrained between June and December 2000, and then determined instances of uneconomic power flows between regions. During unconstrained transmission periods when prices failed to correlate or converge, and when energy flowed from a higher-price region to a lower-price region, such flows were considered uneconomic. 3 This was described largely as the result of inefficient regional coordination and a lack of standardized procedures and market design.

Assuming that centralized dispatch of the Northeast markets would eliminate these uneconomic flows when transmission capacity was available, the Mirant study estimated the associated annualized benefits by deriving "revised" prices and flows using statistical analyses and comparing them to historical data. The study estimated total potential annual benefits of approximately $440 million. While the results showed positive benefits for all three regions, NYISO enjoyed the greatest benefit ($256 million) and PJM the least ($76 million). ()

The NYISO asked consultants LECG to review the Mirant study. The resulting NYISO study took exception to a number of the Mirant study's assumptions, as well as certain fundamental aspects of their methodology, including:

The NYISO study first attempted to replicate the approach and methodology used in the Mirant study in order to verify the results. Using the Mirant approach and methodology, the NYISO study estimated approximately $140 million less than the original Mirant study in aggregated benefits.

The NYISO study then altered its replication analysis by comparing real-time prices and flows in New York and PJM (as opposed to day-ahead prices and real-time flows). Though the combined benefit to New York and PJM increased approximately 17 percent from the replication analysis (from $189 million to $222 million), the shift in benefits from New York to PJM was dramatic.

Next, the NYISO study conducted a separate analysis using data from October 11, 2000 to August 31, 2001, a time period that the authors believed more accurately reflected current market operating conditions. The study also used sensitivities that varied system operating conditions, including increased transmission-constrained curtailments, reduced transmission interface limits, and steeper sloping supply curves. 5 These sensitivities indicated a range of annualized benefits between $275 and $440 million. However, the distribution of these benefits significantly favored PJM, with NYISO expected to experience real costs from Northeast RTO formation.

Modeling the Future: The PJM, FERC, and RTO West Studies

The PJM study also compared three separate Northeast markets to a single Northeast market. 6 The results were reported in terms of changes to energy savings to load, generation production costs, and generator revenues. By splitting the results into these categories, the PJM study began to look at the distribution of benefits between market participants, in addition to the geographic distribution of benefits examined in previous studies.

The base case results indicated an overall annual benefit of $262 million from forming a single Northeast RTO. However, on a regional basis, NYISO would experiences a $22 million loss as a result of the formation of a single Northeast RTO, while both PJM and ISO-NE would experience positive benefits. () More specifically, the PJM study shows that the winners, in terms of generators and load, include New York consumers as well as generators in PJM and New England. The losers include consumers in PJM and New England and generators in New York.

PJM also analyzed the benefit of improving coordination between the three Northeast system operators absent a single Northeast RTO, but assuming a joint and common market between PJM and the Midwest. Interestingly, the coordinated market sensitivity from this analysis produced benefits for consumers in all three markets, though the benefits in New England were essentially negligible. On the other hand, generators in both PJM and New York both lost net revenue, particularly in PJM.

The FERC study, performed by ICF Consulting, examined the potential nation-wide benefits of RTOs and related policy issues. ICF employed a proprietary production cost model to establish a baseline of production costs, and then estimated relative benefits (as measured by production cost savings) of several policy scenarios over a 20-year period. The policy scenarios analyzed included:

The FERC study estimated that the RTO Policy scenario would result in substantially more value over 20 years than the Transmission Only scenario-approximately $34 billion of additional production costs savings on a net present value (NPV) basis. While the Transmission Only scenario estimated cumulative NPV savings from the base case of $6.2 billion, the RTO Policy scenario resulted in a cumulative NPV savings estimate of $40.9 billion. The Demand Response scenario estimated cumulative NPV savings of $60 billion, or $19 billion more than the RTO Policy and Transmission Only scenario.

Interestingly, two sensitivities performed around the RTO Policy scenario to examine the impact of larger vs. smaller RTOs did not have a substantial impact on the savings estimate. The estimated savings were +$500 million and -$700 million for the larger RTO sensitivity and smaller RTO sensitivity, respectively (or $41.4 billion and $40.2 billion).

Meanwhile, the primary aim of the RTO West study7 conducted by Tabors Caramanis & Associates was to assess the benefits of establishing RTO West in the RTO West and WSCC regions. () The results indicated benefits from RTO West implementation in all western regions, except for California. In every region there were load savings, with the greatest savings in the RTO West region (including British Columbia). On the other hand, in every region generators were expected to see a decline in their net revenues, particularly in California and the RTO West region. However, the total savings to load was greater than the net lost revenue to generators.

A number of sensitivities were run to assess the impact of changes in physical conditions, such as low water/high gas prices and new generation in Montana. The results indicated that the two main benefit drivers in the analysis were the elimination of pancaked transmission loss charges and the increase in reserve sharing. The elimination of pancaked transmission loss charges had a significant impact on reducing congestion charges, while increased reserve sharing dramatically lowered production costs.

Far From a Simple Exercise

Not surprisingly, most studies have been critiqued by one or more interested parties. What follows are highlights of some of the principal areas of debate regarding assumptions and/or methodology.

Several studies were criticized (or criticized one another) for the selection of an inappropriate study period. For example, the Mirant and NYISO studies both relied on a partial-year of historical data (from which benefits were extrapolated), but each questioned the suitability of the period the other had selected. The PJM study, on the other hand, was criticized for estimating RTO benefits using a market simulation of a past calendar year (2000) that did not account for generation and transmission enhancement planned or underway. 8

Not surprisingly, most studies received criticisms regarding assumptions and/or methodology from one or more interested parties.

Several studies were criticized for the selection of the study period over which annual benefits were estimated. At one end of the spectrum, the Mirant, NYISO, and PJM studies were criticized for selecting short time frames that arguably were not suitable for extrapolating annualized data. At the other end, FERC's study was criticized for forecasting too far into the future, and calibrating its forecast with year 2000 data. Critics argued that calibrating the base case to the year 2000 resulted in an overestimation of the benefits from RTO formation given the unusually high wholesale market prices seen during that period.9

In addition, several parties commented on the way that transmission capacity and constraints were treated in the studies. The FERC study was criticized for its use of a model that did not use a detailed representation of the transmission system for what is essentially a transmission study. 10 The Mirant and NYISO studies also questioned each others' assumptions regarding the amount of available transmission capacity among the Northeast ISOs in order to estimate the benefits of more economic dispatch. While the Mirant study revealed only a modest impact (16 percent decrease in benefits) when transmission capacity was constrained, the NYISO study found the results to be considerably more sensitive to more stringent assumptions about transmission transfer capability.11

Furthermore, several study critiques focused on specific assumptions regarding generation resources. Some argued that the modeling of hydro resources in the RTO West study status quo case created an unrealistically high reliance on thermal units for reserves, and thus overstated the annual generation cost savings due to formation of RTO West by $150 million. 12 Similarly, the Mirant and NYISO studies debated the slope of supply curves in the Northeast and the impact that varying such an assumption can have on an estimation of short-term benefits associated with economic dispatch. The FERC study was criticized for presuming that formation of RTOs would cause generators to move toward best practices and thus improved efficiency and availability.13

Considerations Going Forward

Many critiques of the studies published to date highlight the inherent complexity and the judgment required in undertaking an RTO economic analysis. It is quite clear that a number of issues must be well thought through. The following discussion of issues that must be carefully considered in conducting any such analysis expands on some of those previously mentioned, and offers further insights from our own experiences in working with market simulation and other models on similar assignments. This list is by no means exhaustive, but presents several issues that will substantially impact the conduct and results of an RTO cost/benefit analysis:

While choosing an appropriate model is a required step in any such analysis, the choice of the modeling tool to be used in an RTO cost/benefit study must be suitable for answering the question at hand. The use of market simulation models that can accurately and dynamically represent the operation and limitations of the transmission system (like GE MAPS) will be increasingly important.

One of the principal tasks in any comparative study is the need to establish a credible base case. For an RTO cost/benefit study, the modeling of the status quo can be more challenging than the modeling of changes in wholesale markets attributable to RTO scenario(s). For example, the highly fluid nature of the current regulatory and market environment poses significant issues for modeling the status quo. The impact of FERC's Standard Market Design initiative and evolving RTO developments directly influence not only market modeling, but also the shelf life of the modeling results.

Many of the existing cost/benefit analyses focus largely on the impact of RTO decisions on certain (typically multi-state) geographic areas. However, regulators and other groups increasingly are pushing the debate down to the level of assessing the impact of RTO formation on specific customer classes on a state-by-state level. Some regulators have expressed a desire to know the extent and timing of the impact of RTO costs and benefits on different groups of ratepayers, based on how changes in wholesale market price flow through to retail tariffs. Providing this level of analytic detail requires modeling the complex interface in each state between wholesale and retail prices, necessitating a sophisticated understanding of state level tariff structures and policies.

Net Winners and Losers

Unfortunately, the result of more detailed analyses to determine specific winners and losers may further galvanize RTO opposition, even if there is eventual consensus on the overall net benefit of RTO formation, as the issues take on a more parochial tone. There is certainly a need to understand better the actual costs as well as benefits of RTO formation on a macro level, and more effort should be focused in this area to determine whether net benefits can be credibly established. However, no matter how great the potential net benefits, there still are likely to be both winners and losers.

The fact that certain groups would lose, but lose less than others will gain, should not prevent the achievement of overall net benefits to society. However, one should not expect the "losers" to support the structural change without assurances of appropriate transition arrangements. Once the industry becomes comfortable with the likelihood of net benefits from RTO formation, assuming they can be credibly demonstrated, the debate needs to evolve from identifying winners and losers. Instead,in order to move RTO development forward, the focus needs to be on developing the mechanisms to equitably distribute net benefits, at least over some transitional period.

An expanded version of this article is available on PA Consulting's Web site, www.paconsulting.com/energy.

  1. While the five studies described all quantify the benefits of RTO formation, only one looks at the potential costs. However, for the purpose of this article we will refer collectively to these analyses as cost/benefit studies. Additional studies are expected for PJM-MISO-SPP, Entergy (Louisiana and Arkansas) and by the Southeastern Association of Regulatory Utility Commissioners (SEARUC).
  2. Affidavit of E. Harry Vidas and Bruce B. Henning, FERC Docket No. RT01-99-000, October 9, 2001, p.3.
  3. Id. at pp. 6, 9.
  4. "The lack of LMP pricing in NEPOOL makes any estimation of supply curves, or assessment of gains from trade, based on posted NEPOOL prices of doubtful value." Id. at p. 6.
  5. In the transmission-constrained curtailments case, transfer limits were set equal to actual flows in transmission-constrained hours. In the reduced interface limits case, the transfer limit was set at the higher of actual flows or the actual average flows during constrained hours. Steeper sloping supply curve estimates were applied in both the NYISO and PJM markets.
  6. PJM used the GE Multi-Area Production Simulation (GE MAPS) model to simulate hourly locational marginal prices in the three individual northeast markets assuming separate unit commitment and dispatch functions. The RTO West study also used GE MAPS to model the RTO West and WSCC regions.
  7. Tabors Caramanis & Associates, "RTO West Benefit/Cost Study: Final Report Presented to RTO West Filing Utilities," March 11, 2002.
  8. "PJM Moves to Pull Out of Northeast RTO" Generation Week, June 16, 2002.
  9. Comments of the Large Public Power Council on the Economic Assessment of RTO Policy Prepared by ICF Consulting for the Federal Energy Regulatory Commission, Docket No. RM01-12-000, April 9, 2002, pp. 12-15, and Comments of the Public Service Commission of South Carolina, Docket No. RM01-12-000, April 9, 2002 (LPPC Comments), p. 12.
  10. Comments of the Large Public Power Council on the Economic Assessment of RTO Policy Prepared by ICF Consulting for the Federal Energy Regulatory Commission, Docket No. RM01-12-000, April 9, 2002, p. 6.
  11. Supplemental Affidavit of E. Harry Vidas and Bruce B. Henning, FERC Docket No. RT01-99-000, October 26, 2001, p. 9. See also Andrew Hartshorn and Scott Harvey, "Assessing the Short-Run Benefits From a Combined Northeast Market," LECG LLC, October 23, 2001, pp. 6, 42-44.
  12. Linc Wolverton et al., "Corrections to the Benefits/Costs Study for RTO West," April 19, 2002, p.1 in Attachment B of Tabors Caramanis & Associates, "Response to the RTO West Benefit Cost Study Critique Dated April 19, 2002," June 4, 2002.
  13. Comments of the Large Public Power Council on the Economic Assessment of RTO Policy Prepared by ICF Consulting for the Federal Energy Regulatory Commission, Docket No. RM01-12-000, April 9, 2002, p. 15.

Articles found on this page are available to Internet subscribers only. For more information about obtaining a username and password, please call our Customer Service Department at 1-800-368-5001.

Source URL: http://www.fortnightly.com/fortnightly/2002/09-0/studying-apples-and-oranges