Model Behavior

• TBA - Writer
• April 2012
• 

Over the past decade, financial institutions enhanced their enterprise risk management practices in response to corporate governance considerations and regulatory drivers, including the Sarbanes-Oxley Act, COSO ERM recommendations from the Treadway Commission and supervisory efforts from the Basel Committee on Banking Supervision, among others.

Despite these enhancements, the financial crisis of 2008-09 highlighted certain gaps in the ERM practices at financial institutions, including insurance companies. One notable area of exposure involved the use of complex financial models in business and risk management.

For example, a misunderstanding of the strengths and weaknesses of Value at Risk models may have led leadership at multiple financial institutions to become overconfident and overly reliant on models in decision-making. In light of the heightened use of models, both regulators and management teams have increasingly focused on the importance of model risk management at the enterprise level.

Understanding Model Risk

Model risk arises when errors in the design, inputs, assumptions, implementation, modification or control of models expose companies to financial loss, poor decision-making or reputational damage. Misuse of models and misunderstanding their purpose are also significant sources of model risk.

To fully understand and control model risk, it is important to be aware of the intersection of business processes with the model life cycle. (See chart, page 27.) The model life cycle reflects the ongoing process by which models are developed, implemented, used and changed within a business process. Business processes--such as product pricing, asset valuation, and asset/liability management--take external and internal data (including key assumptions) and feed it into models, which transform the inputs into information used by management to make decisions. Organizations traditionally focus on the business process risks, but overlook risks posed at different stages of the model life cycle. The underappreciated model-specific risks can severely impact business process objectives.

Model Risk Management

MRM includes the identification of key business processes (such as pricing and valuation) that rely on models and the implementation of internal controls to identify, measure and control model risk across the model life cycle.

Effective MRM should incorporate three lines of defense:

1. Controls exerted by the business and corporate functions that own, use and manage models.

2. Enterprise risk functions and committees responsible for enterprise model governance, including the model definitions, inventories, policies and guidelines, as well as independent model validation and monitoring.

3. Internal audit, which independently assesses the design and effectiveness of controls and policies owned by the first two lines of defense.

In developing these frameworks, financial institutions have several existing regulatory guidelines at their disposal. Banks are generally further along in implementing MRM due, in part to the issuance of OCC 2000-16 and Basel II requirements.

OCC 2011-12/SR 11-7 was recently issued by the Federal Reserve Board and the Office of the Comptroller of the Currency.

It includes enhancements to the OCC 2000-16 guidance, incorporates expectations associated with the Basel requirements, and recommends that banks manage model risk like other risks by covering:

-- Model development, implementation and use.

-- Model validation.

-- Governance, policies and controls, including senior management and internal audit oversight.

For the insurance industry, Solvency II requirements have helped define MRM expectations.

Additionally, the International Association of Insurance Supervisors has outlined MRM practices for models used for Own Risk and Solvency Assessment and regulatory capital calculations.

Both Solvency II and IAIS guidance cover key MRM concepts, including independent model validation, statistical quality tests, calibration tests, documentation, use tests and governance.

Model Risk in the Industry

While banks were early adopters of model risk, insurers are paying increased attention--thanks to the credit crisis and their increasing reliance on models in key processes, including:

-- Strategic and other business planning and forecasting.

-- Product development, underwriting and pricing.

-- Client advice and asset management.

-- Investment decision support and monitoring.

-- Valuation of assets and liabilities.

-- Asset and liability management.

-- Risk management--catastrophe, market, credit, operational, liquidity.

-- Stress testing and scenario analysis.

-- Capital calculations--statutory, economic.

The growing significance of options and guarantees in insurance products and the evolution of financial reporting and solvency regulation are likely to increase the reliance on models by large insurers. The following governance, competitive and regulatory drivers also highlight the need for MRM:

-- The desire of management and the board to understand how models are used for pricing, capital allocation and other strategic decisions.

-- The prospect of rating agencies assessing insurers' economic capital modeling practices (including methodology, assumptions, data quality, process, testing and validation) within ERM frameworks.

-- U.S. insurers that are designated as bank holding companies, classified as systemically important financial institutions under the Dodd-Frank Act, or that own thrifts will be subject to supervision by the Federal Reserve Board, the OCC and the Federal Deposit Insurance Corp.

-- As global insurers implement Solvency II in the European Union, they must meet specific internal model approval and governance requirements.

The potential impacts of model risk, and the evolving regulatory and competitive landscapes, compel insurance companies to establish a comprehensive MRM framework.

Large, global firms can leverage Federal Reserve Board/OCC guidance and Solvency II requirements in designing their frameworks, but building them can be expensive and time-consuming.

Therefore, it is critical that senior management support MRM and position it as a critical risk management activity, not a compliance exercise.

Taking the First Steps

To be effective and sustainable, MRM frameworks must:

-- Accurately reflect company size and complexity.

-- Focus on complex models posing material risks.

-- Encompass the model life cycle in terms of controls and related requirements.

-- Align to ERM frameworks, reflecting business strategy, risk appetite and organizational roles (especially the various "lines of defense").

-- Meet the requirements of multiple regulatory regimes, such as Solvency II.

Insurers seeking effective MRM should establish a task force of senior executives representing business and control functions to manage MRM development and implementation.

The task force should focus on:

-- Defining what constitutes a model, developing enterprise model inventories (including those provided by vendors) and cataloging model owners, users, purposes, etc.

-- Determining risk assessment criteria for model classification.

-- Analyzing gaps between current model controls and regulatory guidance, such as from the Federal Reserve Board/OCC or Solvency II.

-- Identifying change management needs and an implementation plan to address gaps.

-- Communicating plans to stakeholders, including model owners, risk committees, senior management and the board.

Wherever insurers are on the MRM continuum, they must take steps now to confirm that their policies and processes are sufficient to their needs.

The more reliant insurers are on models to support ERM and business decisions, the more robust their programs must be.

Similarly, the sooner the planning and assessment begins, the sooner and more likely insurers will be able to develop the MRM capabilities they need to prosper in the more highly regulated, post-crisis environment.

--------------------------

Model Risk Management in Action

As one of the largest diversified insurance and financial services companies in the United States, Nationwide Insurance provides a full range of services including property/casualty insurance, life insurance, annuities, mutual funds and pensions. Following the 2008 credit crisis, the company's senior leadership recognized the need for a more formal risk management and governance structure surrounding key models.

"Although we had strong risk management procedures, our overall approach to managing model risk was not consistent across the businesses, and expectations for model owners were not clear," said Kai Monahan, senior vice president and chief audit executive.

The company's Enterprise Risk Management group led an effort to obtain broad stakeholder buy-in and the support of all major business units. With assistance from Ernst & Young, Nationwide began benchmarking the current MRM approach against leading industry practices.

This effort led to a straw-man MRM approach that helped kick-start the design process for Nationwide's new MRM framework.

"By leveraging the existing governance structure, including our Enterprise Risk Council and Risk & Capital Modeling Committee, we achieved consensus from a broad cross-section of the enterprise regarding the need for more robust and consistent model risk management," Monahan said. "We also created buy-in for the key tenets of our approach."

Nationwide's internal audit team played an important advisory role, leaving the detailed design to the ERM team. Practicality and simplicity guided the design. "The team was leery of building a governance structure that would collapse under its own weight," Monahan explained. "Our MRM program had to enable better decision-making in the business, not become a bureaucratic barrier to it."

Other building blocks of Nationwide's model governance approach included:

-- Framework overview: Clearly defined scope, objectives, model definitions, ownership and responsibilities.

-- Model inventory: Including risk assessments and prioritization criteria (such as materiality) for models in use across the business.

-- Validation structure: Tiered approach for validation based on risk.

-- Validation criteria: Clearly defined standards and expectations related to documentation, data quality and other factors.

In building the initial model inventory, Nationwide's Risk & Capital Modeling Committee members conducted detailed discussions with model owners to understand the models' purpose and relative risk levels. The model inventory will undergo periodic updates as models change and new models emerge.

Although still early in the implementation phase, several benefits are evident. One, there is increased awareness and understanding of the models used to support significant business decisions. The models themselves are more transparent. Development of a model inventory, prioritization of models based on materiality and overall risk, and more robust validation processes have identified important areas for improvement.

"Today, we have clear and valuable insight into our model development practices and model usage, insight that we previously lacked," Monahan said.

Nationwide expects to evolve its MRM structure for greater efficiency and effectiveness without compromising the initial design simplicity. The internal audit team will keep close tabs on early validation efforts and assess the effectiveness of the overall model governance structure when the time is right, according to Monahan.

"We have already learned a great deal about the effort and time required for validation of models. That is why simplicity and efficiency are important," he said. "However, we also recognize the strategic importance and overall value of our MRM program."

By Kai Monahan, David Meyer,Jim Embersit and Gagan Agarwala

(Contributors: David Meyer is a partner, Jim Embersit is executive director and Gagan Agarwala is an executive with Ernst & Young LLP; and Kai Monahan is senior vice president and chief audit executive for Nationwide Insurance. They may be reached at monahak1@nationwide.com, david.meyer@ey.com, jim.embersit@ey.com and gagan.agarwala@ey.com.)