Shareholders are threatening legal action unless the board of Hollinger, which also owns the Chicago Sun-Times, the Jerusalem Post and the Sydney Morning Herald, responds to demands for an investigation into the payments.
The charge is being led by US fund manager Tweedy Browne, which holds 18% of Hollinger.
The calls for the inquiry comes amid global attention on the issue of so called "fat cat" payments to senior company executives. Earlier this week, shareholders rejected a multimillion-pound deal for GlaxoSmithKline chief executive Jean-Pierre Garnier.
The calls come ahead of what is expected to a rancorous annual general meeting in New York later today.
According to reports, the payments were made as part of non-compete deals signed when Hollinger sold off most of its Canadian newspaper business, including The Globe & Mail to CanWest Global Communications Corp.
The investors are arguing that the money paid over several years both directly and through Ravelston Corporation, controlled by Lord Black, should have gone to the company rather than to four directors including Lord Black.
Tweedy Browne is demanding for the money to be repaid, according to a letter that has been filed with the Securities & Exchange Commission in New York.
In the Financial Times, Christopher Browne, managing director at Tweedy Browne, said he made the call after failing to reach an agreement with Lord Black.
"It's time for this company to come into the 21st century and act like a good corporate citizen. We'll sue to compel [the board] to do this," he said.
If you have an opinion on this or any other issue raised on Brand Republic, join the debate in the .