The FRB, OCC & FDIC requires validation of computer risk models

The basis for the validation is described as follows

 

Computer models are conceptual depictions of the various relationships between events and values in the real world, and are used by banking institutions to estimate risk, analyze business strategies, and estimate values of financial instruments and acquisitions.  However, even the most complex of these models are still only approximations of real events and require the use of certain generalizations, making model development an extremely complex process.  If proper attention is not paid during model development, the most minor of irregularities can lead to major errors in a model’s predictions and recommendations.  Furthermore, given that the internal logic of most models is usually very abstract and somewhat limited, it requires considerable business judgment and proficiency to apply model results correctly. 

 

Thus, it is critical that any model used by an organization be properly validated to ensure that these judgments are made using the most accurate information possible.  This model validation not only increases the reliability of a model but also promotes improvements and a better understanding of its strengths and weaknesses among both model users and bank management.

 

The best way to avoid computer model risk is to implement a sound model validation structure that includes a strong validation policy and appropriate independent review. To properly establish the validity of any model, it is important to review the three basic components of the model: the information input component, which includes the assumptions and data put into the model; the processing component, which includes the theoretical model that transforms the information inputs into estimates using often complex computer programs; and the reporting component, which translates the estimates calculated by the computer code into practical business information.  Errors in any of these three components might cause the model’s output information to be worthless or deceptive. Therefore, it is necessary to examine all three of these model components before concluding the validity of a model.

 

In order to adequately assess these model components, there are three general procedures that need to be used.  First, an independent review of the logical and conceptual soundness of the model is needed.  Second, a comparison must be made of the model against a similar “benchmark” model yielding like results.  Finally, a comparison of the model’s predictions against actual real-world events is required to verify its accuracy.