Public Utilities Reports

PUR Guide 2012 Fully Updated Version

Available NOW!
PUR Guide

This comprehensive self-study certification course is designed to teach the novice or pro everything they need to understand and succeed in every phase of the public utilities business.

Order Now

Model Risk Management: How to Avoid an Earnings Surprise

Fortnightly Magazine - December 2004

is a perfect representation of the market, others may not see it that way. Auditors will require proof that the model is perfect before they approve the accounts. Aggrieved shareholders could sue the company claiming the imperfect model distorted earnings and hence share price. Not only does the model have to be perfect, it has to be seen as perfect.

But anyone who has traded in illiquid markets knows there is no such thing as a perfect model. An element of "black art" or common sense is applied to all valuations. Model risk management is where these assumptions and the model's logic are documented and tested so that the auditor and the aggrieved shareholder's legal team can clearly see that the company has been reasonable and conservative with regard to calculating MTM earnings through internal models.

Learning From History

The 2001/2002 crash of many energy traders' stock prices, the many restatements of earnings, and the subsequent lawsuits provide good examples for how not to do MTM, as well as some shining examples of probity.

Examples abound from the days of energy marketing in the late 1990s and early 2000s, when valuations were less rigorously audited. A marketer would do a deal and mark it to market immediately to create instant "earnings" of $10 million, $50 million, or even $100 million (yes, really); in some instances the counterparty on the other side of the deal reported a large profit too, if that's what their model said. This tendency was exacerbated by the traders' performance-related pay, with the "gamers" creating high unrealized "earnings" and collecting an employee performance bonus of cold hard cash before the music stopped, so to speak.

Companies have, and had, two approaches to this problem. First, they apply operational risk management and effort to ensure the model is a reasonable representation of real life (i.e., the output values are as close as possible to actual market values). This is an ongoing process. As more price data is observed, models can be modified to incorporate these observations.

For example, after a few years of observing out-of-the-money option values, a volatility "smile" can be incorporated into the valuation process to capture the observed market value above the theoretical value. The first guiding principle is a simple check: At the time of the deal the most likely MTM value is slightly negative, a reflection of the bid-offer spread.

Second, traders have well-developed procedures to track and document how the model works, what changes have been done to it, and how these changes have altered the output values. This documentation improves transparency of operation but, more importantly, is a fundamental part of the auditing process and any legal defense. A lack of documentation and procedure plays right into the hands of plaintiffs and creates a big risk for an auditor.

Model Risk Management

Model risk management, therefore, is the process of applying operational risk management to modelling and valuation models to demonstrate that the models are robust and the assumptions reasonable and explicit. Good model risk management requires the same discipline and attention that