The three are said to be unhappy that the former chief executive of Ofcom could soon be heading up a rival broadcaster and will argue that there would be a conflict of interest.
Yesterday, it emerged that Allen could depart ITV as early as next week with a £10m pay-off.
The ITV was said to have entered into negotiations with Allen over the timing of his departure, putting pressure on the board to find a replacement quickly.
If ITV believed Carter is the best man for the job, it would face the ire of its rivals, who are believed to be building a case of conflict of interest, arguing that as head of Ofcom Carter had access to detailed information about their businesses, which would be highly useful to rival ITV.
There are other big names that have been mentioned in the race, with many seeing Dawn Airey, Sky's programming boss, as a very serious candidate. Prior to Sky, Airey ran Five and was instrumental in bringing in the quality US programming, which the station now has a reputation for with shows such as 'CSI', 'House' and 'Grey's Anatomy'.
Other candidates include: former BBC director general Greg Dyke; Channel 4 chief executive Andy Duncan; and David Elstein, former chief executive of Five and current chairman of Sparrowhawk Media.
If you have an opinion on this or any other issue raised on Brand Republic, join the debate in the .