They've written to the Prime Minister after he said he didn't want to be involved ahead of the general election - if the Green Party isn't included.
Premier's Political Editor Martyn Eden says an 'empty seat' situation where a debate is held without the Conservatives is very unlikely.
"That is just a ploy at the moment. They know Cameron wouldn't let that happen. He would either go in or make sure that the whole broadcasts failed," he said.
"He won't tolerate an empty chair situation so I think it's an arm twisting exercise on their part."
The Labour, Liberal Democrat and Ukip leaders have sent letters to the Prime Minister saying it would be "unacceptable" if he failed to show up.
It said the debates in 2010 "were watched by more than 20 million people and enthusiastically endorsed by all those who took part, including yourself".
The letter continued: "In recent days, you have announced that you are unwilling to take part in debates as proposed by the main broadcasters for the 2015 General Election," the missives added.
"I believe it would be a major setback to our democratic processes if these debates were not repeated in 2015 because of one politician's unwillingness to participate.
"I hope you will agree that the decision as to who should take part in the televised debates should not be in the hands of any party leader, each of whom inevitably has their own political interests to defend. It must be a decision independently and objectively arrived at."
Downing Street said its position on the debates "hasn't changed", and that any multi party debate must include the Green party.
Premier's Political Editor Martyn Eden said: "He's been persuaded that debates are not his best suit and therefore he's not urgent to go on them.
"He wants the Greens on and I don't think it's just a benevolent gesture towards the Green party, I think it's because he sees the Greens as a threat to Labour and the Lib Dems.
"I think he wants them there as a counter balance to UKIP who will clearly take votes off the Tories but probably the other two parties as well."