* RBS legal bills set to pile up tmsnrt.rs/2dttowg
* Deutsche woes spark fears over RBS DoJ settlement
* RBS faces new claim of up to $2.5 bln from small businesses
By Andrew MacAskill and Lawrence White
LONDON, Oct 10 (Reuters) - Royal Bank of Scotland may have to pay out as much as $27 billion, roughly the market value of the bank, in misconduct fines and lawsuits over the next few years, lawyers and analysts said.
That bill represents the upper end of estimates to settle a range of claims related to RBS's alleged misconduct before and during the financial crisis, including mis-selling mortgage backed securities (MBS) in the United States.
Investor concern over RBS's outstanding legal and compliance woes increased after news last month that the DOJ is seeking up to $14 billion from Deutsche Bank for its role in the mis-selling of MBS in the run up to the financial crisis.
"The concern is that it could be another Deutsche Bank-style situation where the fines that come in are higher than the market expects," said Laith Khalaf, an analyst at Hargreaves Lansdown, Britain's largest retail stockbroker.
"Litigation is a real Sword of Damocles hanging over the bank at the moment and until that is out of the way it is very difficult to see a reason to invest in RBS."
An RBS spokeswoman declined to comment on the potential size of the legal bill.
If RBS wins the court cases and the fines are at the lower end of analyst estimates, the total would be around $5.5 billion, most of which it has set aside to cover those damages.
However, analysts say the bank could have to pay the U.S. Department of Justice as much as 9 billion pounds ($11.19 billion) in the next few months. Even the lowest estimate of 2 billion pounds ($2.49 billion) would make it largest fine in the bank's history.
In addition, analysts estimate RBS will have to pay between $3.5 to $5 billion to settle a similar case with the U.S. Federal Housing Finance Agency, after it sold $32 billion of MBS before the financial crisis.
A total settlement bill at the high end of estimates would eat into RBS's core capital levels, currently 14.5 percent.
A JPMorgan analyst said last month that the bank will have to pay $9.8 billion in U.S. fines and that every additional $1 billion will cut the bank's capital by 0.34 percentage points.
Although at a healthy level, the analysts are keeping a close eye on capital levels following a disappointing performance in stress tests ordered by the European Banking Authority earlier this year.
The "litigation, investigations and reviews" section of RBS's half-year results published at the end of July ran to 20 pages, or 11,457 words, and outlined 38 separate issues the bank is dealing with.
In the latest legal headache, RBS customers are seeking as much as 2 billion pounds in damages for mis-selling Enterprise Finance Guarantee loans designed to help small companies.
The lawsuit comes after the bank admitted last year that it misled some small businesses in default or severe distress about the amount they had to repay on the government-backed loans. Some of the small business owners say they lost their businesses because of the way the bank behaved.
"The scheme was heavily mis-sold. That is not in dispute. However, victims should be able to claim damages for the losses they have suffered," said Indyren Yagambrun from M&M solicitors, who is bringing the case.
"Within the compensation scheme RBS has implemented, that wasn't included. All the scheme provided was a minimum redress, but crucially, it did not deal with the consequential losses."
RBS still faces about 13 other civil litigation cases for alleged mis-selling of asset-backed securities, according to a source at the bank.
The bank also faces a potential 6 billion pound bill for allegedly misleading shareholders before the bank's 2008 bailout in a trial to start next year.
The bank also faces a separate legal case involving over 100 small businesses claiming about 1 billion pounds. They say the bank deliberately pushed firms on their books into bankruptcy to pick up their assets cheaply between 2008 and 2013.
The BBC reported on Monday that an RBS executive had asked staff to search the books of existing RBS customers to see if any companies could be put into restructuring, or their interest rates bumped up, in what the executive called a "dash for cash."
The documents show that the bank gave higher bonuses to staff based on fees collected for "restructuring" business customers' debts, the BBC said.
RBS said in a statement it had let some small business customers down in the past, but denied it deliberately caused them to fail.
"A number of our customers did not receive the level of service they should have done or, importantly, that they would receive now," the statement said.
However, "we have seen nothing to support the allegations that the bank artificially distressed otherwise viable SME businesses or deliberately caused them to fail." ($1 = 0.8042 pounds)
Editing by Sinead Cruise and Anna Willard