Frontiers of Efficiency

Deck: 

What conservation potential assessments tell us about ‘achievable’ efficiency.

Fortnightly Magazine - April 2011

Studies of energy conservation potential—known as energy efficiency potential assessments, or simply CPAs (conservation potential assessments)—can help identify and quantify gains that might come from energy conservation, namely: how much energy efficiency may be available, in which particular market segments, and when and at what cost. Such studies are essential to integrated resource planning. They serve as a cornerstone to designing effective, energy-efficiency programs. Increasingly, they are used to justify efficiency in energy policy, and serve as a basis for setting energy efficiency performance standards (EEPS). Their importance is hard to exaggerate—though not impossible—depending on how they are conducted and how their results are used.

Beginnings

The roots of the CPA method can be traced to the 1978 National Energy Conservation Policy Act (NECPA). The Act, primarily a response to the energy crisis following OPEC’s 1973 oil embargo, addressed domestic concerns about energy reliability and prices, along with international concerns about economic and national security. The Act considered conservation—in the sense of using less non-renewable natural resources and increasing end-use efficiency—as one of several approaches for addressing both sets of concerns.

An important development emerging from the Act was the need for research determining how far conservation could go in addressing these concerns over the long run. Much of this research took place at national laboratories, through studies focusing on understanding and measuring energy consumption at the end-use level, and identifying loss sources and how they might be mitigated. Recognizing—and defining—conservation as a resource with a “potential” that could be expressed in terms of a conventional supply curve, was perhaps the main conceptual breakthrough arising from the research.

This new view of conservation—plus a method for measuring its potential—were formalized in pioneering works by Allan Meier and others in the early 1980s (Meier 1982, 1983).1 What followed was a comprehensive CPA for the first regional power plan, published in 1983 by the Northwest Power and Conservation Council. However, it wasn’t until the introduction of integrated resource planning (IRP) as a new ratemaking standard (in the Energy Policy Act of 1992) that estimating conservation supply curves became a widespread practice.

Classes of Conservation Potential

Primarily, a CPA quantifies gaps between the current energy consumption and technically feasible lower levels. The analysis generally begins by identifying all end uses potentially offering large energy savings, plus the measures necessary to achieve those savings. This analysis yields a “technical” potential: the expected technically feasible conservation levels that might be realized over time, under specific engineering assumptions about performance and applicability of various conservation measures. From a planning point of view, technical potential can be characterized as the conservation opportunities that are available immediately through retrofits in existing structures, or else emerging over time, either via new construction or when current equipment stocks are gradually replaced.

Technical potential is expressed using a conservation “supply curve.” This curve ranks various energy-efficiency measures in order of increasing costs, and shows how large the reserve of conservation resource is for each measure. As Meier (1982) explained, a supply-curve approach provides a consistent accounting framework for addressing conservation measures and predicting their impacts and costs. It also allows conservation options to be readily compared with new energy supply alternatives, as both can be uniformly expressed as the marginal cost of obtaining an energy unit—a critical and necessary step in IRP.

From here, successively smaller portions of this potential are designated as A) “economic,” B) “achievable,” and C) “program” potential. These potential levels can then be differentiated according to specific constraints (or barriers) preventing realization of the potential at each level.

Economic potential can be defined as the subset of technical potential expected to be cost-effective, according to one or another of various jurisdiction-specific criteria, often the total resource cost (TRC) test. In general, a measure is cost-effective if its per-unit cost of conserved energy is lower than the cost of the energy it saves. A measure’s cost of conserved energy varies, depending on the initial cost, quantity, and expected life of the resultant energy savings, relative to conventional technologies.

The economic potential likely to be realized depends on the many factors that help determine just how willing (and able) are consumers to embrace conservation. These factors include the energy prices, the cost of adopting conservation measures, the availability of adequate information, and a host of other barriers. Achievable potential represents conservation levels likely to be realized once these factors have been accounted for.

Compared to technical and economic potential, achievable potential is often described in less precise terms, with its value reported as a range of estimates, underscoring uncertainty not only in how it is defined, but also in methods used in quantifying it. Since levels of achievable potential depend on efforts necessary to capture the potential, the concept of “program” potential is often used to evaluate conservation quantities that may be achieved using varying investment levels in incentives and marketing efforts by conservation program administrators.

Estimating Conservation Potential

Methods for estimating technical potential fall into three broad categories. In the first category are integrated national or regional macroeconomic models, such as the U.S. Department of Energy’s National Energy Modeling System (NEMS) and Consolidated Impacts Modeling System (CIMS). In these models, energy efficiency typically is treated in terms of economic decisions involved in production, conversion, and consumption of energy products.

The second category consists of models with roots stretching back to the end-use forecasting models of the late 1970s. In these models, energy efficiency potential is determined through generating a baseline forecast at the end-use level, and comparing its results to a second forecast, incorporating marginal impacts of a broad range of efficiency measures.

Finally, accounting models estimate potential for individual measures based on their expected savings and projected market saturation. End-use forecasting and accounting methods belong to a “bottom-up” approach, in that they begin with individual measures, assess their impacts, and then aggregate results up to end-use, market segment, and sector levels.

Regardless of the approach used, technical potential assessment tends to be complex. It requires compiling and carefully examining a large number of technologies, their costs, potential impacts and how they interact with energy systems and each other. This requires an enormous amount of data and sophisticated computations.

Economic potential is—at least conceptually—much easier to estimate. At the basic level, it requires calculating a benefit-to-cost ratio for each measure with a technical potential, and screening the list of measures for cost-effectiveness. The TRC or its variants are the most widely used criteria with a decision rule: if the conserved energy cost is less than the cost of energy it displaces, the measure is prudent. The cumulative potential for measures meeting the criterion is considered economic. In most studies supporting utility planning initiatives, hourly analysis is conducted to capture the full system-wide capacity and energy benefits for individual measures. This step makes assessment significantly more complex.

Estimating achievable potential is more difficult simply because it involves making assumptions about notoriously illusive consumer behaviors and decision-making. Achievable potential emphasizes consumer motivations to adopt conservation measures or barriers preventing them from doing so. These questions not only involve the actual economic return, but individual consumer perceptions of the cost and potential benefits of conservation.

Methods of estimating achievable potential either rely on experience or are derived from behavioral models of technology diffusion—models that predict whether a conservation measure is adopted according to simple assumptions about the consumer’s “payback acceptance.” However, evidence from existing literature suggests neither approach is entirely satisfactory. In several studies, achievable potential has been expressed as a range of values, derived from variants of a conventional technology diffusion model simulating consumers decisions to invest in energy efficiency according to expected returns and how they are influenced by available incentives.

Experience-based approaches either rely on other CPA results, or else look to the achievements of past conservation efforts (made locally or in other geographic areas). Either approach can be problematic. The past might not effectively predict the future because consumers participating in early conservation program stages might be driven by different motivations compared to later participants—known as the “early-adopter” phenomenon. We can never know with certainty whether a market has been saturated, and benchmarking against other programs requires making inferences across geographical areas, such as utility service territories. This can be problematic because utilities may have unique service areas, and their experiences may not be transferable.

How Much Can We Conserve?

Since 2000, a swelling number of CPAs for electricity and natural gas have been produced in the United States, appearing in a wave that seems to crest every five years or so. They differ in their orientation, geographic scope, time horizon, and method. Largely, they are either policy-oriented attempts by non-governmental organizations (NGOs) or advocacy groups to advance policy on energy efficiency policy, or planning-oriented studies by utilities that support activities related to resource planning and program design. The geographic scope of these studies may be limited to a single utility service area, or may cover an entire state, region, or country, depending, it would seem, on their orientation. In most studies, potential estimates are broken down by market segment, but at times only aggregate figures are reported. Twenty years seems to be the norm for a planning horizon, but shorter-term perspectives of 5 to 10 years aren’t uncommon.

Research undertaken for this article identified nearly 100 electric CPAs completed since 2000, covering 37 states and three Canadian provinces.2 (Actual numbers are probably higher, but not all studies were publicly available or readily accessible.) These studies also included several “meta” analyses, summarizing and comparing other study results, including an impressive effort by Carla Frisch (Frisch 2008)3 and several reports by the American Council for an Energy Efficient Economy (ACEEE),4 Georgia Institute of Technology, and others.

Not all the main classes of conservation potential are reported—or analyzed—in all studies. Nearly all planning-oriented studies begin with technical potential, and report all classes of potential by sector.

In fact, some studies supporting utility IRPs left out economic potential because both the calculation of avoided costs—a crucial ingredient in estimating economic potential—and also the optimal qualities of conservation, were products of, rather than inputs into, the modeling process. With a few exceptions, policy-oriented studies tended to skip technical and economic potential altogether, and only reported achievable potential. These studies offered 72 estimates of achievable potential, most frequently expressed as a “maximum.” Fifty-five estimates of technical and economic potential were available. A few studies with a regional scope provided estimates of potential in multiple states.

On average, technical potential is estimated at 27 percent of forecast consumption, a common measure for assessing relative magnitudes of conservation potential, across the 37 states. About 72 percent of these (21 percent of forecast consumption) is expected to be economic, on average. These studies also show achievable potential of 17 per percent, on average (Figure 1). In many cases, achievable potential is large enough to offset projected load growth entirely, and, in several cases, lower loads to below their base-year levels.

The range of estimates seems to vary by a study’s scope. Studies at the national and state levels show the highest potentials in all classes, although there were only four national studies, and only one reported economic potential (Figure 2).

The data suggest a weak correlation between technical potential and lengths of planning horizons—which is reasonable, given that much technical potential consists of discretionary retrofit opportunities presumably available immediately and, hence in theory, independent of time. Achievable potential levels also appear to be uncorrelated with planning horizons—and that is surprising, given that the amount of achievable conservation depends on effort levels and, crucially, time. Studies with shorter time horizons should estimate a more limited efficiency potential than those over longer timeframes. This result was so for a few studies specifically considering variable planning horizons and reporting significantly higher achievable potential levels for longer periods.

The average state-level values mask large variations in estimates for all three classes of potential, and particularly for achievable potential. Estimates range between 12 percent and 43 percent for technical, and between 11 and 36 percent for economic potential. The range widens to between 3 percent and 31 percent for achievable potential.

The spread of these estimates points to significant uncertainties involved in estimating conservation potential, particularly achievable potential. This variability is reflected in relationships between average values for each class of potential and their variance. Coefficient of variation (CV), the ratio of the standard deviation to the mean, is a useful statistic for comparing the degree of variation from one data series to another, even if their means differ drastically from each other.5 Estimates for technical potential indicate a CV of 21 percent, rising to 32 percent for economic, and more than doubling to 43 percent for achievable potential.

The variation in estimates of technical potential can largely be explained by differences in technical assumptions, geography (climate), and characteristics of energy markets where studies are performed. Economic potential also is expected to vary across markets because of differences in avoided costs, the costs of deploying conservation measures, and economic assumptions, such as discount rates, which are usually mandated by local regulation. Achievable potential isn’t independent of technical potential. However, the data show a large variability in achievable potential even when it is normalized to technical potential and variations in technical potential explain less than 50 percent of variations in achievable potential. The remaining variability in achievable potential is more difficult to explain, mostly because methods for deriving estimates aren’t always spelled out, at least not satisfactorily.

Methodology or Ideology

Stark differences in potential estimates have prompted attempts to explain them, including a recent article in The Electricity Journal. That article described a study that investigated whether study sponsors (or performers) can influence conclusions about efficiency potential. Specifically, the study asked whether NGOs and advocacy groups, whose mission is to advance energy efficiency, tend to find higher potential than utilities more interested in selling electricity than conserving it. Based on sector-level data from 23 CPAs, with a focus on southern states, the study found NGO-sponsored studies had higher achievable potentials, while utilities tended to find the highest technical and economic potentials and lowest achievable savings. “Despite the pattern,” the study concluded, “differences between these estimates by sponsor type aren’t statistically different from zero;” thus, sponsorship didn’t matter.6

Results from the larger sample of studies summarized here suggest a different conclusion. Much observed variability in data can be explained if results are grouped by study objectives and orientation (Figure 3). The data indicate, at the aggregate level, that policy-oriented studies show achievable potentials of 25.5 percent on average—an estimate nearly twice as large as the 13.4 percent reported in utility-sponsored studies. The difference is also statistically significant. Moreover, statistical tests suggest that results not only differ on average, but seem to be derived from fundamentally different populations.

These differences, however, don’t necessarily signal bias. Regardless of methods used, derivation of long-term potential requires making a large number of assumptions about technology, economics, and consumer behaviors, and how these factors interact. Certainly, an element of judgment is involved in any research of the scope and complexity as a potential study. Researchers will, at the very least, decide data sources to use, how to prioritize information, and choose among alternative assumptions. There is, however, no evidence in these studies that data might have been manipulated to favor any particular interest.

Higher rates of achievable potential in policy-oriented studies may derive from fundamental differences in approach and the critical technical, market, and economic assumptions made. For example, many policy-oriented studies include in their analyses effects of codes and standards as policy instruments for achieving conservation. Studies also vary in terms of mixes of measures analyzed. Policy-oriented studies generally rely on national and regional sources for market data, while most planning-oriented studies include substantially more specific, local data forming the basis for estimating potentials—one reason they tend to be more expensive.

From an economic point of view, policy-oriented studies typically use lower societal discount rates to derive estimates of the conserved energy costs, while planning-oriented studies use significantly higher-weighted average capital costs for the sponsoring utilities. Policy-oriented studies also place greater emphasis on indirect, non-energy benefits (the so-called “co-benefits”), such as local income and employment effects, environmental benefits, and consumer energy bill savings.

Extreme values distort data distribution and adversely affect the reliability of results. The wide range of estimates for achievable potential includes several such values. These values occur at both ends of the distribution values for achievable potential. Extraordinary claims require extraordinary evidence, yet these extreme values tend to derive from studies that neither explain the methodology particularly well, nor make a credible case for the reliability of data used.

Comparing results across these studies would have been much easier if all three classes of potential were analyzed and reported consistently. Policy-oriented studies, however, rarely report technical and economic potential. As a rule, they go directly to achievable potential, and, in many cases, do so without providing convincing details on methods as to how they were derived. This opacity of the methodology makes direct comparisons difficult. Policy-oriented studies are similar in purpose, but glaring differences emerge among them in methodology, scope, and analytic rigor.

The Guide for Conducting Energy Efficiency Potential Studies, a component of the National Action Plan for Energy Efficiency,7 recommends that policy makers forego estimation of technical and economic potential, and focus instead only on achievable potential. Presumably, that’s because “technical and economic potential studies are of limited value for planning purposes.” This prescription is surprising. While rough estimates of achievable potential might be sufficient for setting performance standards (at least in the short run), detailed analyses of technical and economic potential supply curves are essential to utility IRP processes. But this might be beside the point, since it isn’t clear how one derives achievable potential without first carefully examining what is technically feasible and economically justified.

There is also the observation in the Guide that in most studies, conservation measures are screened for failing cost effectiveness, thus technical potential ends up being “virtually the same as the economic potential.” The accumulation of data undermines this observation. The evidence from the studies reviewed for this article show that estimates of economic potential are, on average, 72 percent of technical potential, and that only a few studies mention any qualitative screening of measures before estimating technical potential.

Policy Objective or Planning Imperative

Estimating conservation potential differs from forecasting. As the experience of the last three decades has shown, energy efficiency must be acquired. To achieve the potential, consumers must be encouraged through education and incentives, delivered by well-conceived programs. Understanding the achievable potential may be important, but the determination has to begin with an understanding of what is technically feasible and economic. Certainly, estimating technical and economic potential is a complex and expensive undertaking. It is, however, not just a step toward achievable potential, as the Guide suggests. Rather, these estimates provide the context and points of reference crucial to answering policy questions regarding achievable potential for establishing standards.

As for establishing performance standards, the right question for policy makers to ask ought to be not what is achievable, but what portion of technical and economic potential might be reasonably achievable. A large amount of reliable data on actual performance of conservation programs is available from multitudes of evaluation reports and best-practice studies. These studies are sufficiently informative if establishing performance standards is the only objective in an assessment.

Supply curves for energy efficiency don’t remain static. New end uses emerge regularly. And as more efficient technologies become available as well, new technical opportunities will arise for conservation. Over time, as energy supply costs increase, energy suppliers will be willing to invest in higher-cost conservation measures and, as energy prices rise, consumers will find conservation more attractive than purchasing more power. These changing consumer preferences, when coupled with shifts in macroeconomic conditions and new policies (such as adoption of more stringent energy codes and standards, or imposition of a carbon tax), will improve the prospects for economic and achievable potential. In their initial study of conservation supply curves, Meier, et. al. (1983) asked if eventually we will exhaust the apparently large supply of conservation. The answer was, and still remains, “probably not.”8

In the early 1980s, the Bonneville Power Administration joined with Pacific Power & Light Co., one of Oregon’s two major investor-owned electric utilities, to implement the Hood River Conservation Project. A main goal of the project, proposed by the Natural Resources Defense Council, was to test the upper limits of achievable conservation. The project achieved a maximum market acceptance of 85 percent. Ever since, regional planners have assumed 85 percent of economic potential can be achieved in the long run. While the true figure may be debatable, the approach has decidedly worked: it shifted the debate from what is achievable to the more relevant questions of what is technically feasible and economic.

Endnotes: 1. Meier, Alan (1982), Supply Curves of Supplied Energy, LBL-14686, Lawrence Berkeley National Laboratory, Berkeley, Calif.

2. See http://www.Fortnightly.com/exclusive.cfm?o_id=619 for a complete list of conservation potential studies cited in this article.

3. Frisch, Carla, Electric Utility Demand-Side Management: Defining and Evaluating Achievable Potential, Duke University, 2008.

4. See for example: Bech, Fredrik, et al., Powering the South: A Clean & Affordable Energy Plan for the Southern United States, Renewable Energy Policy Project, Washington, D.C., 2002; Steven, et al., The Technical, Economic and Achievable Potential for Energy-Efficiency in the U.S. – A Meta-Analysis of Recent Studies, proceedings, ACEEE Summer Study on Energy Efficiency in Buildings, 2004; Energy Center of Wisconsin and the American Council for an Energy Efficient Economy (ACEEE), A Review and Analysis of Existing Studies of Energy efficiency Resource Potential in the Midwest, 2009; Chandler, Sharon and Marilyn Brown, Meta-Review of Efficiency Potential Studies and Their Implications for the South, Georgia Institute of Technology, August 2009; Brown, Marilyn, et al., Energy Efficiency in the South, Georgia Institute of Technology, April 12, 2010; and Northeast Energy Efficiency Partnership (NEEP), From Potential to Action, How New England Can Save Energy, Cut Costs and Create a Brighter Future with Energy Efficiency, An Analysis of the Region’s Economically Achievable Electric Efficiency Potential, prepared by Optimal Energy, 2010.

5. This ratio, called the coefficient of variation (CV), represents the ratio of the standard deviation to the mean, and it is a useful statistic for comparing the degree of variation from one data series to another, even if the means drastically differ from each other.

6. Chandler, Jess (2010), “A Preliminary Look at Electric Efficiency Potential,” The Electricity Journal, Vol. 23, Issue 1, January-February 2010.

7. U.S. Environmental Protection Agency, National Action Plan for Energy Efficiency, Guide for Conducting Energy Efficiency Potential Studies, November 2007.

8. Meier, Alan, Janice Wright and A.H. Rosenfeld (1983), Supplying Energy Through Greater Efficiency: The Potential for Conservation in California’s Residential Sector, University of California Press, Berkeley, Calif.