The withdrawal of the £260m offer from David and Frederick Barclay worsens the financial situation for Lord Black's firm, which missed a $7.4m (£3.98m) debt payment on Monday.
Lord Black had accepted a £260m offer for Hollinger Inc from the Barclay twins, but Hollinger International sued to block the sale.
Control of Hollinger Inc would have given the Barclay brothers majority voting control over Hollinger International, which is the parent company of the Telegraph Group.
In Delaware last week a court blocked Lord Black from selling Hollinger Inc to the twins, who already own The Scotsman and The Business newspapers and the Ritz Hotel.
The Barclay brothers could still enter the race by tabling an offer in the auction in which private equity firms 3i, Candover and Apax are bidding against Express Newspapers and former owners of the Telegraph, the Berry family, who are thought to be bidding alongside the Daily Mail & General Trust and Cinven.
The Barclays' exit from the race poses a serious financial problem as with them gone so is their commitment to buy Hollinger Inc debt, which led to Monday's missed payment.
If the Barclay brothers do re-enter the race, speculation is that they will make a direct approach to the Hollinger International board.
A source told Reuters: "They're much more likely to make a full-scale bid for Hollinger International... They'll just make an offer to the board."
This morning Daily Express owner Richard Desmond sold off his porn publishing empire, which includes Asian Babes and Readers' Wives for £20m to Remnant Media in an effort to bolster his chances in the race for the Telegraph Group.
If you have an opinion on this or any other issue raised on Brand Republic, join the debate in the .