In 2009, unconventional shale gas emerged as the dominant driver in North American natural gas markets. Rapid increases in shale gas production and shale-driven upward revisions to the U.S....
Gas-Market Forecasts: Betting on Bad Numbers
Why predictions from the Energy Information Administration may contain systematic errors.
and comprehensible, our decomposition analysis is conducted for three pairs of variables in the natural-gas market involving prices, domestic flows, and imports. The two prices are the average wellhead price and prices paid for natural gas by electricity producers. The flow variables include dry natural-gas production and consumption by electricity producers. The later was selected because the electricity sector comprises the most dynamic, market-sensitive component of natural-gas consumption along with industrial sector use. Imports include those from Canada and imports of LNG.
Prices. The EIA forecasts natural-gas prices in constant dollars. To establish a consistent basis for comparison, these constant price forecasts are inflated by the corresponding forecasts for the price deflator for gross domestic product (GDP). Once the forecasts are sorted, the prices are converted back to 2006 dollars using the latest GDP price deflator.
The forecast evaluation metrics for the one- through four-year-ahead forecasts from 1998 to 2006 appear in Table 1. On average, the one-year-ahead average percentage forecast error for the wellhead natural-gas price is 16 percent with an absolute error of $1/Mcf. These errors steadily rise and reach more than 45 percent with the four-year-ahead forecast and $2.60/Mcf.
The RMSE (root mean squared error), which penalizes large errors more severely than the average percentage error (see “Appendix,” for full explanation) , is almost 35 percent for the one-year-ahead forecast. Like the average percentage error, it too rises with the forecast horizon, reaching more than 57 percent with the four-year-ahead forecasts.
The decomposition of the MSE (mean squared error) for the one-year-ahead wellhead natural-gas price forecast errors indicates that 54.7 percent of the errors can be attributed to systematic bias. This bias crests to almost 88 percent for the three-year-ahead forecasts. While random disturbances are substantial for the one-year-ahead forecast, the large proportion attributed to bias is noteworthy. A plot of the actual time series for wellhead natural-gas prices and the four different forecasts appears in Figure 1 and illustrates the tendency of the EIA price forecasts to systematically under-predict actual prices. The results for electric generator’s natural-gas costs are very similar to those for wellhead natural-gas prices.
Market Flows. Table 2 shows the forecast errors for natural-gas consumption by electricity generators and for dry natural-gas production. The forecast errors are much smaller than those associated with the forecast errors for prices, which is a common phenomenon. Price forecasting often is more difficult than forecasting demand and production series, which often contain a sizeable trend component or signal. Nevertheless, the forecast errors for these two key natural-gas market flows are substantial.
The EIA forecasts for natural-gas consumption in electricity generation consistently are below actual observations of gas use in this sector (see the average percentage errors in Table 2) . This is somewhat counter-intuitive because given that EIA under-estimates prices paid for natural gas by electric generators, it would seem that lower prices would imply higher, not lower, natural-gas consumption, all other things held equal. One of the big changes affecting the electricity sector’s use of fuels has been the sulfur-dioxide emissions-trading program. That program has exerted a dramatic