The committee is also pushing for Parliament, and not the government, be responsible for deciding what the increase in the licence fee should be, while the National Audit Office should examine the BBC's requests for increases.
The committee's argument is that the BBC is being asked to bear additional costs that previously had been borne from general taxation, such as being required to help the elderly and disabled pay for digital switchover.
If the BBC was not asked to bear these costs, then increases in the licence fee could be kept at the rate of inflation and closer to its current level of £126.50. On April 1, it will go up to to £131.50, which is an above-inflation increase of 4.2%. At present, the BBC is proposing for increases at the retail price index rate plus at least 2.3%.
At this level, in eight year's time the licence fee will hit £180 and cost the public £4.25bn.
Lord Fowler, chairman of the committee, said: "There is an overwhelming case for licence fee increases to be properly scrutinised by Parliament with the assistance of the National Audit Office. There is no justification for the present position, which is in effect a deal between the government and the BBC."
In addition, the House of Lords committee said it was backing the competition authorities' attempts to break up exclusive sports rights, namely the BSkyB monopoly on the live Premier League matches.
However, it does not see the European Commission's decision on splitting the broadcast rights into six packages, with no one broadcaster owning more than five, as going far enough to create a competitive market.
If you have an opinion on this or any other issue raised on Brand Republic, join the debate in the .