In 2009, unconventional shale gas emerged as the dominant driver in North American natural gas markets. Rapid increases in shale gas production and shale-driven upward revisions to the U.S....
Gas-Market Forecasts: Betting on Bad Numbers
Why predictions from the Energy Information Administration may contain systematic errors.
effect on the opportunities for fuel substitution in power generation, as shown by Considine and Larson 6. Whether the NEMS correctly models the role of permits in power-sector fuel demand and fuel switching could be an important question.
The absolute error for the one-year-ahead forecast for electric generators natural-gas consumption is more than 900 billion cubic feet, which is more than 15 percent of consumption in this sector. In addition, the RMSEs are around 20 percent, nearly four times the errors found in econometric forecasting models of energy demand. 7 Like prices, the error decomposition analysis for natural-gas consumption by electric generators reveals a substantial bias across all four forecast horizons.
The forecast errors for dry natural-gas production reveal further problems. As the average percentage errors indicate, EIA consistently over-predicts dry natural-gas production. The absolute errors are quite sizeable in relation to marginal supplies of gas, specifically imports of LNG. For example, the one-year-ahead forecast error for production is 590 billion cubic feet, which is about equal to LNG imports in 2006. The two- through four-year-ahead forecast errors exceed one trillion cubic feet.
The mean squared error decomposition for natural-gas production also reveals sizeable bias, especially for the three- and four-year forecasts. Unlike prices and consumption forecast errors, the model component of the errors is more than 40 percent for the one- and two-year forecasts. This fact suggests that the model itself is generating systematic errors for the near-term forecast horizon. The time path of each forecast depicted in Figure 2 illustrates that even though EIA has been scaling back its projections of natural-gas production, the model still portrays an upward track for production albeit from a lower base during each forecast year.
Imports. Another important factor influencing natural-gas markets is imports. The largest external source of natural gas into the United States is Canada, although EIA expects imports of LNG to become significant in the future. Among the forecast errors examined in this study, those associated with EIA’s projection of imports from Canada are the lowest. Similar to the other forecast errors, however, the forecasts contain either bias or systematic errors arising from the model.
The projections of LNG imports are not as accurate as those for Canadian imports. The RMSEs are quite large and, while the bias components are relatively small, the proportion of the forecast errors associated with the model remains substantial, especially for the first and third year-ahead forecasts. This finding could be associated with the rather idiosyncratic nature of the LNG import forecasts.
To understand what is happening in the LNG forecast error decomposition, a scatter plot of the actual versus predicted LNG imports appears in Figure 3. A perfect forecast in which the predictions are equal to the actual observations is plotted on the solid line. A “good” forecasting model should generate forecasts close to the line of perfect forecasts and randomly scattered around it. As Figure 3 illustrates, there are several very large over-predictions of LNG imports. The small number of these very large errors most likely accounts for the erratic swings in the