* Risk models overstate danger of many loan books
* Banks conservative despite regulatory criticisms
* Asset weighting relies on uncertain calculations
By Owen Sanderson
LONDON, May 30 (IFR) - Banks are struggling to predict default rates of loans on their own books, according to research from Barclays, with many overestimating the probability that borrowers won't repay, and treating assets as riskier than they actually are.
Regulators and others have claimed that banks use their default predictions to make assets appear less risky than they actually are. But the opposite seems to be true - real performance seems to have beaten bank assumptions. Barclays found that actual default rates were more than 50% lower than bank predicted default rates.
This matters because banks use their predictions to come up with risk weighted asset figures, a key metric used to assess bank health, and even in the contracts for contingent capital bonds.
More sophisticated banks which employ either of the Internal Rating Based approaches allowed under Basel II use probability of default (PD) for each portfolio as an input to their own risk models, which determine how much capital they allocate against specific assets.
This approach has been criticised by members of the Basel Committee, alongside other regulatory figures, such as Sheila Bair, former head of the FDIC. The criticisms usually argue that banks manipulate their models to make their capital requirements lower. Other approaches - standardised risk weighting using credit ratings, standard risk buckets, or simple leverage ratios - cannot be manipulated in the same way.
The BIS published a study in January looking at risk-weighting in the trading books of banks, where it made 15 banks risk weight an identical sample portfolio. Banks reported capital requirements varying from EUR13.4m to EUR34.1m for the same portfolio.
Banks including BBVA and Deutsche Bank have included "RWA optimisation" - meaning tweaking risk models - in their capital raising plans.
The numbers from Barclays, however, should reassure critics of the internal ratings based approach, since they show that although banks are erring, it is mostly towards being too cautious, at least in banks which report the numbers at all.
Barclays built its dataset using the detailed "Pillar 3" disclosures, which Basel II banks must provide once a year. But banks including BBVA, BNP Paribas, Credit Agricole, KBC, Santander, Societe Generale and UniCredit simply did not provide relevant data.
Others, including Deutsche Bank, Commerzbank and UBS, did not compare estimated with actual probability of default, though did disclose other key metrics - loss given default (LGD) estimates and expected loss (EL).
Barclays argues that these cannot be used to assess whether banks are good predictors of asset performance. LGD figures are supposed to assume a downturn, so should be consistently worse than actual results, while EL is a loss expected on the assets owned today, ignoring any provisions already taken.
The analysts also raise issues with the PD data, some of which is calculated through the business cycle and some of which is calculated at a single point in time.
But despite the data limitations, they state that "over the past few years, the PDs and the LGDs have consistently been lower (i.e. more conservative) than expected."
Barclays distinguishes between average errors and absolute errors in predicting the probability of default. Average error allows an undershoot one year to offset overshooting in other years.
On this basis, some banks look extremely good. Swedbank, according to Barclays, had average error of 0% over 6 years of data, while Danske managed 4%. But these figures rise to 79% and 37% respectively once this offsetting is excluded.
Using average error across the whole sample gives a figure of -35% - actual PD is 35% lower than banks predict. But absolute error is 54% across all the banks, so Barclays concludes: "It's clear that accurate forecasting is hard, and this adds another layer of uncertainty to the resulting RWAs."
With better data, this could help investors figure out how accurate bank assessments of their own risk weighted assets are, which is crucial to assessing how risky a bank is. Some investments, such as contingent capital, have triggers built in which are based directly on reported RWAs.
But Barclays cautions: "On the face of it, this could give some insight into relative confidence in the resulting RWAs on a bank by bank basis, but that is almost certainly a step too far given the qualifications noted." (Reporting By Owen Sanderson, editing by Julian Baker, Alex Chambers)