In the administered North American electricity markets, a high level of secrecy concerning bids, bidders, and computations is currently the norm. The decision to maintain such secrecy has little discussion and the impacts of secrecy on prices and efficiency have never been comprehensively studied. One of the very few surveys of transparency in this area, a CRA report prepared in 2007, concludes that “[f]ew, if any, of the markets had evaluated information disclosure explicitly for its effects on competition or market efficiency.”1 In practice, the issue of transparency has been left to Adam Smith’s “invisible hands.” Recent statistical analysis from the Texas independent system operator indicates that the benefits from additional transparency may be considerable.
In 1776, in his book, The Wealth of Nations, Adam Smith made an offhand reference to “an invisible hand.” Since few of us have ever read the book in its entirety, it’s useful to observe by his words that Adam Smith wasn’t nearly as naïve as legislators and federal regulators who have done away with checks and balances over the past 16 years with such catastrophic consequences:
He generally, indeed, neither intends to promote the public interest, nor knows how much he is promoting it. By preferring the support of domestic to that of foreign industry, he intends only his own security; and by directing that industry in such a manner as its produce may be of the greatest value, he intends only his own gain, and he is in this, as in many other cases, led by an invisible hand to promote an end which was no part of his intention. Nor is it always the worse for the society that it was no part of it. By pursuing his own interest he frequently promotes that of the society more effectually than when he really intends to promote it. I have never known much good done by those who affected to trade for the public good. It’s an affectation, indeed, not very common among merchants, and very few words need be employed in dissuading them from it.2
Interestingly, the only mention by Smith of an invisible hand occurs in this passage warning the reader against those who claim that their activities are for the public good—almost the exact opposite of the usual interpretation.
Adam Smith was not a strong supporter of secrecy in business, correctly fearing that such arrangements tended to raise prices above their natural level:
But though the market price of every particular commodity is in this manner continually gravitating, if one may say so, towards the natural price, yet sometimes particular accidents, sometimes natural causes, and sometimes particular regulations of police, may, in many commodities, keep up the market price, for a long time together, a good deal above the natural price. When by an increase in the effectual demand, the market want of general price of some particular commodity happens to rise a good deal above the natural price of those who employ high profits, their stocks in supplying that market are generally careful to conceal this change. If it was commonly known, their great profit would tempt so many new rivals to employ their stocks in the same way, that, the effectual demand being fully supplied, the market price would soon be reduced to the natural price, and perhaps for some time even below it. If the market is at a great distance from the residence of those who supply it, they may sometimes be able to keep the secret for several years together, and may so long enjoy their extraordinary profits without any new rivals.3
Adam Smith touches on the theme of secrecy a number of times, for in general he opposed business combinations that enforce such rules as detriments to competition.4
Since the mid-1990s, transparency in North America’s electricity markets has decreased dramatically. While the traditional pre-filed contracts at FERC, retail rate cases, and open outcry markets were hardly perfect, the transition of 50 percent of the wholesale electricity markets in the United States and Canada to highly opaque, administered markets has reduced our ability to understand wholesale prices. Secrecy in administered markets ranges from a high level in MISO and PJM where the bids, bidders, and price resolution are secret, to ERCOT where bids and bidders are made public only after two months.5
In general, the high level of secrecy has been adopted from the example in California. This is ironic since California has suffered from the secrecy that allowed such market schemes as Ricochet, Death Star, Load Shift, and Get Shorty.6 All of these would have been impossible without the shield provided by the rules in place at the California Independent System Operator.
Surprisingly little discussion has taken place concerning the lack of transparency at RTOs. FERC has adopted a slightly inconsistent policy of allowing the ISOs to define their own levels of secrecy—generally without public discussion or justification—while retaining traditional transparency rules for utilities in FERC’s Form 1s and energy trading in the commission’s Electric Quarterly Reports. FERC’s 2008 final order in RM07-19-000 and AD07-7-000 provide offhand guidance concerning transparency at the RTOs:
Our proposal to reduce the lag time for release of offer and bid data to three months was supported by most commenters. Some commenters requested a shorter lag time or immediate release. Others proposed the release of additional information, such as system lambda… Our proposal cuts the current lag time for most RTOs and ISOs in half. Because this is a substantial change, RTOs and ISOs should become accustomed to the new release time and observe its effects before committing to an even shorter time. However, as we proposed in the NOPR, we permit the RTOs and ISOs to propose a shorter time, with accompanying justification, or a longer time of four months if they can demonstrate a collusion concern. Alternatively, they may propose an alternative mechanism if release of a report were otherwise to occur in the same season as reflected in the data. These options provide the flexibility requested by commenters…
We assume the data to be released would consist not only of physical offers and bids but demand and virtual offer and bids as well. However, if RTOs and ISOs object to such inclusion, they may address it in their compliance filings. Likewise, if they desire to release additional data such as system lambda, they may propose it in their filings…
We adopt the NOPR proposal to retain the masking of identities. The objection that sophisticated market participants may be able to infer identities of those submitting offers and bids does not resolve confidentiality concerns; if anything, it argues for more protection, rather than less. We decline to establish a time period for the eventual unmasking of identities, but invite RTOs and ISOs to propose a period when such unmasking might be permitted, if they believe it to be desirable.7
Given the lack of debate concerning transparency within RTOs, it’s not surprising that the argument for keeping bids secret generally involves the theory that public bidding will aid conspiracies to set pricing. The flaw in this argument is self-evident. Conspirators are free to provide their information to each other. They aren’t likely to avoid a price-fixing scheme simply because the RTOs do not supply the data. Schemes like Project Stanley in Alberta didn’t rely upon the ISO’s Web site; instead the conspirators coordinated their activities using the telephone.
Two years ago, ERCOT changed its transparency rules in response to a settlement at the Texas PUC. This is the first opportunity to observe whether changing transparency rules actually affects bidding. While the obvious common sense answer is so clear that it almost seems superfluous to address it, the fact is that market participants generally have argued against reducing the level of secrecy on the basis that they, themselves, could take advantage of the additional transparency to raise prices. Luckily, the facts back common sense rather than legal rhetoric.
In an efficient market, prices converge to marginal cost since bids higher than marginal cost aren’t able to change the equilibrium price. Bids higher than marginal cost will reduce the probability of sale, however, so any inefficient bids will reduce the bidders’ potential profits. The real world is short on efficient, competitive wholesale electricity markets. Real world markets often display a degree of concentration that makes perfect competition difficult to achieve. In Texas, for example, one market participant dominates the Dallas zone. This participant, all things being equal, will receive prices above marginal cost, since the marginal revenue line crosses the marginal cost curve at a smaller quantity than that observed in perfect competition (see Figure 1).
The only check on the ability of market participants to set prices higher than those that would take place in perfect competition is the presence of other competitors. In a world in which bid data never was released, market participants would be able to judge their degree of market power only by experimentation. While the demand curve and the marginal revenue curve reflect the response of competitors, the exercise of market power wouldn’t be obvious to competitors without another round of experimentation with their own bids.
The lack of competitive information would reward the exercise of market power since the experimentation process by its competitors necessarily takes time. In the extreme example above, any market participant with market power could count on a substantial period of higher prices, while its competitors tried alternative bidding strategies and finally derived their competitor’s market price.
ERCOT publishes bids in 60 days. Bids in the other U.S. RTOs are released after 180 days. Within this time period, market participants with market power have an incentive to raise prices above marginal cost since any market response will be delayed by the time for other market participants to feel out their new bids. In Texas, Docket 31972 addressed this specific issue. In its August 23, 2006 decision, the Texas PUC found:
In balancing the concerns of the commenters on both sides of this issue, the commission has determined that it would be appropriate to change the disclosure requirement on a gradual basis. This will enable both the commission and the market participants to become accustomed to the new disclosure procedure and make any necessary changes to their operations. The implementation schedule for disclosure is also being tied to the schedule for increases to the offer cap, thereby further emphasizing the commission’s decision that these two issues are interrelated. Under the revised disclosure schedule contained in the rule, effective March 1, 2007, most of the required disaggregated information will be disclosed 90 days after the day for which the information was accumulated. This is one-half of the current disclosure timeframe of 180 days, but much longer than the 48- hour to 30-day time periods contained in the proposed rule. On the same date, the offer cap contained in the rule will increase from $1,000 per MWh to $1,500 per MWh. Effective March 1, 2008, the disclosure of disaggregated information will take place 60 days after the date the information was accumulated. This corresponds to the date that the offer cap is increased to $2,250 per MWh. Finally, two months after the market begins operation under a nodal market design (approximately March 1, 2009), the disclosure period is reduced to 30 days while the offer cap is raised to $3,000 per MWh.8
This order was litigated extensively and eventually was replaced with a 60-day delay on the release of bidding data:
However, the commission is also sympathetic to the concerns expressed by Constellation that the time period before disclosure should be long enough to avoid encouraging collusion or other market manipulative activities. Except for intervals when an event trigger is reached, the commission agrees that, for most of the information subject to the rule, disclosure after 30 days may not be necessary. Therefore, while the commission cannot agree with the 90-day delay as proposed by Constellation, the commission determines that the appropriate delay for disclosure of individual offer curves, except when the event trigger is implemented, should be 60 days. The commission finds that this delay in disclosure will not cause a loss of public confidence because much of the time prices in the ERCOT-administered markets are not subject to the type of price spikes that could create an impression of market power abuses or other market failures. In some cases, however, prices may spike to higher than usual levels and cause public concern and the need for more public information. To address such events, the proposed amendment includes an event trigger that would require the public release of entity-specific information on a much quicker timeframe. The proposed amendment requires that, when the trigger is exceeded, the portion of every market participant’s offer curve that is equal to or exceeds the trigger level will be disclosed seven days after the day for which the information is submitted. The commission finds that the disclosure of this limited type of entity-specific information is sufficient to retain public confidence in the ERCOT markets while minimizing early disclosure of entity-specific information.9
Implementation of the order took place with market data for September 22, 2007.10 For the first time, a situation existed in which there was a statistically testable hypothesis. Using available data concerning bidding behavior on an hourly basis both before and after the change in disclosure delay, regression analysis can test whether additional transparency does reduce bids, and indirectly, prices.
This analysis uses a set of ERCOT bidding rules and market conditions to determine bidding behavior. It doesn’t attempt to model ERCOT’s pricing algorithm since the algorithm is considerably more complex, and less transparent, than the bidding data it uses as part of its calculations (see Figure 2).
First, the analysis calculates two measures of bidding behavior in the ERCOT balancing energy services market. “Maximum bid” represents the highest bid during the hour and “average bid” represents the average bid during the hour. The simplicity of these two measures constitutes their primary value. There are an infinite number of possible measures that could be designed to characterize the bid curves. Opening the analysis to each one of these would eliminate the significance of the statistical results, since each alternative potentially would have a high t statistic. The best course is to choose and test the simplest hypothesis to avoid biasing the statistical estimates.
Following the same argument, the independent variables also are simple. The first two independent variables are natural gas prices and ERCOT load. These two variables are standard choices for independent variables in wholesale electricity markets and have been used in many studies. The analysis adds three other independent variables:
• Shame Cap: For years, ERCOT published bids over a specified price. The price level has changed over time to its current level of 100 times natural gas prices.
• Reporting Delay: The number of days until bid data is revealed.
• Price Cap: The maximum bid accepted by ERCOT’s computer algorithm.
The models tested are:
• Max Bid = A + B x Gas Price + C x Load + D x Shame Cap + E x Reporting Delay + F x Price Cap.
• Mean Bid = A + B x Gas Price + C x Load + D x Shame Cap + E x Reporting Delay + F x Price Cap.
The data used are the most recent 22,280 hourly observations available on ERCOT’s web site. Since the error terms are highly correlated, the analysis uses STATA’s “Robust” regression algorithm to avoid any bias in the statistical coefficients.11 All pricing data is deflated to eliminate the impact of inflation on the data set. No other adjustments or alternative specifications are modeled.
The maximum bid model is significant at the 99.9 percent level. The maximum bid in each hour is reduced by $0.95 for each day of delay in reporting the bidding data. This result confirms that additional transparency will lead to competing bids at the high end of the market (see Figure 3A).
The regression results for average bids also are significant at 99.9 percent. The impact of a one-day reduction in reporting delay of bids is $0.053/MWh (see Figure 3B). The confidence interval around this value is 7/10ths of one cent.
The conclusion is straightforward. When the Texas PUC reduced the bid delay from 180 days to 60 days, it reduced the average bid submitted to ERCOT by $6.32/MWh. Interestingly, the impact of a single day change in bid reporting is relatively large compared to changes in the price cap or in the “shame cap,” an indication that competitive dynamics may well be a better enforcer of market efficiency than arbitrary bidding rules. Certainly, this reprises Adam Smith’s belief that true market discipline comes from competition, not the rules of the trade association.
It’s interesting to test the alternative hypothesis. In May 2008, a paper the author presented at the American Public Power Association caught the interest of the Wall Street Journal. Energy reporter Rebecca Smith, whose reporting is well known and respected in the industry, conducted her own investigation.12 As part of her story, she commented on the “hockey stick” bids filed by one of the market participants. Hockey stick bids are those that have normal economic prices at lower levels and then a massive “stick” where the last few megawatts are priced from ten to hundred times the going price.
FERC prohibits hockey stick bidding:
First, bids that vary with unit output in a way that is unrelated to the known performance characteristics of the unit are prohibited. An example of this bidding practice is the so-called “hockey stick” bid where the last megawatts bid from a unit are bid at an excessively high price relative to the bid(s) on the other capacity from the unit. A variant of this pattern could be a single unit in a portfolio that is bid at an excessively high level compared to the remainder of the portfolio, without any apparent performance or input cost basis.
A second category of prohibited bids are those that vary over time in a manner that appears unrelated to change in the unit’s performance or to changes in the supply environment that would induce additional risk or other adverse shifts in the cost basis. An example of this is a bid that appears to change only in response to increased demand or reduced reserve margins, particularly if the timing of the bid is related to public announcements of system conditions or to timing of outages in a participant‘s portfolio.
Should public utility market participants engage in any of the prohibited behavior discussed above, their rates will be subject to increased scrutiny by the commission and potential refunds. This could result in further conditions or restrictions on their market-based rate authority, including prospective revocation of market-based rate authority.13
In Texas, hockey stick bids are discouraged, but there’s no outright prohibition. Logically, Ms. Smith’s WSJ story should have had little impact on the market. To the degree it did, it should have had approximately the same level of impact as the “shame cap.” Yet the regression results indicate a very different impact. Using a dummy variable to measure the impact on high bids on and after July 17, 2008 indicates that Ms. Smith’s article caused a $71/MWh reduction on high bids in ERCOT (see Figure 3C).
Adam Smith believed that competition, not regulation, is the best defense against market problems. The analysis above illustrates that this is the case in ERCOT and, almost certainly, in other administered wholesale electricity markets where high levels of secrecy are the norm. In addition, the analysis proves that publishing a story in one of the nation’s most respected business newspapers has a significant impact—but only if data from the market can be accessed soon enough for the media to use it in their stories.
1. Analysis of Data Release Practices in Centrally-Dispatched Electricity Markets, CRA, July 25, 2007, p. iii.
2. The Wealth of Nations, Book IV, Chapter 11, pp. 484-485.
3. Ibid., Book 1, Chapter 7, pp. 67-68.
4. Adam Smith discusses trade organizations that restrict competition by the enforcement of regulations in a variety of contexts. For example, in Book 1, Chapter 10, Part 1, he says, “The majority of a corporation can enact a bye-law with proper penalties, which will limit the competition more effectually and more durably than any voluntary combination whatever. The pretence that corporations are necessary for the better government of the trade, is without any foundation. The real and effectual discipline which is exercised over workmen is not that of his corporation, but that of his customers.”
5. At PJM, bids from different markets are reported in a muddled format that makes it impossible to ascertain which bids have been submitted to which market. MISO reports only bids that are actually taken. In most administered markets, bids are revealed after a substantial delay and without revealing the identities of the bidders.
6. A FERC investigation in early 2002 revealed internal Enron memos referring to numerous trading schemes—including “Death Star,” “Get Shorty,” etc.—that traders used to manipulate California energy markets.
7. Wholesale Competition in Regions with Organized Electric Markets, Oct. 17, 2008, p. 222.
8. Order Adopting Amendment to §25.502, new §25.504 and new §25.505, as approved at the Aug. 10, 2006, open meeting, Aug. 23, 2006, pp. 28 and 29.
9. Order Adopting Amendment to §25.505, as approved at the Aug. 16, 2007, open meeting, Aug. 16, 2007, p. 18.
10. Whitepaper related to posting changes for PUCT Projects 31972 & 33490, Nov. 12, 2007, page 2.
11. Linear Regression is a standard tool in economics. Unfortunately, the mathematics assumes that the error term for each hour is statistically independent of the previous hour. This is clearly not the case in the real world. STATA’s “Robust” algorithm corrects for the correlation of the error terms and produces unbiased statistical parameters. It should be noted that the use of the “Robust” option does not, and should not, affect the point estimates of the parameters.
12. “Deregulation Jolts Texas Electric Bills,” Rebecca Smith, Wall Street Journal, July 17, 2008.
13. Order Establishing Prospective Mitigation and Monitoring Plan for the California Wholesale Electric Markets and Establishing an Investigation of Public Utility Rates in Wholesale Western Energy Markets, Apr. 26, 2001, p. 19.