Public Utilities Reports

PUR Guide 2012 Fully Updated Version

Available NOW!
PUR Guide

This comprehensive self-study certification course is designed to teach the novice or pro everything they need to understand and succeed in every phase of the public utilities business.

Order Now

Model Risk Management: How to Avoid an Earnings Surprise

Fortnightly Magazine - December 2004

companies give to market risk management.

First, companies need a policy to dictate the minimum acceptable standards for all models. Models that allocate capital and directly impact earnings require the highest standards, but almost all models in an organization require a minimum standard since, by definition, the models help with business decisions. The policy should describe the conventions of model development, ownership and management of models, model structure conventions, acceptance testing, the sign-off process, and change management. The same policy standards and gravitas that are applied to a company's VAR calculation should be applied to all valuation process models.

Second, each model should be thoroughly documented, ideally during the building process. Documentation should describe the model's objective, the algorithms, and the inputs and their respective units. Documentation also should provide a user manual. The intention is that anyone provided with this documentation can use the model, a key part of personnel risk management.

Third, the model should be tested to ensure it functions as required and as defined in the documentation. By this stage the model should have been tidied up to remove unnecessary code and to rationalize its operation and features to make it more company-standardized. The model should be accepted as fit for use only after successful testing. Then the model should be frozen, preventing any changes from inadvertently altering the model's output. User passwords and password protection of code are two common tools employed.

Fourth, each model should include as an integral part of the code a written commentary describing its version number and date, the developer/manager, and any changes or bugs found since acceptance.

Fifth, given the serious financial consequences of a flawed model, key models should be reviewed or audited by external parties. There are two broad types of review (and acceptance testing): white box and black box. White box testing is line-by-line checking to ensure that the model uses the defined algorithms as intended, and that the outputs are correct. Black box testing is a benchmarking/calibration process, not looking so much at the formulae but how the model output compares to what is expected. Given the complexity of many portfolio and risk models, black box testing often is the easier process. It also limits the potential for losing intellectual property in the process.

External review is a plug for specialist consulting services. Many companies don't recognize that external review is also a fixed cost. Since auditors have been trampled by not spotting earnings manipulation, and they now spend a lot of time making sure they understand exactly what they are auditing. The client pays for this time, either directly through the auditor or by more proactively managing the audit through specialist model review and sign off.

Corporate Action

The Financial Accounting Standards Board's statement 133 and its international counterpart, International Accounting Standards statement 39, have pushed the energy industry down the MTM route, which creates significant opportunities for earnings swings and distortions. To avoid this distortion or the appearance of distortion, companies need to apply model risk management where a model creates a value that goes