The Liberal Democrats decision to change their mind over the controversial so-called "bedroom tax" is "staggering hypocrisy" according to Labour.
But Lib Dem Leader Nick Clegg says the "single room subsidy" must be reformed to stop people who can't find a smaller home to live in being penalised financially.
He also says it should be scrapped for the disabled.
Lib Dem MP for Eastleigh Mike Thornton, who's a Christian has been defending the u-turn on Premier's News Hour, he said: "if you admit that you havn't got something right in the past people leap on the fact that you've got something wrong , rather like the Philistines did. Rather than look at what you're doing to improve things, like a good Christian should. And I think what we're looking at is saying, 'yeah we didn't get it completely right"
The changes to housing benefit were introduced last year in England, Scotland and Wales but were met with fierce criticism from many campaign groups and faith leaders.
Lib Dem Chief Secretary to the Treasury Danny Alexander said his party had to 'take stock' and look at changing the rules.
Currently council tenants who get their rent paid for by the government face a cut in their housing benefit if they live in a house with more bedrooms than they need.
If they have one spare room they will see a cut of 14% in their benefit and if they have two or more they face a drop of 25%.
Now the Liberal Democrats say nobody should face the penalty if there are no smaller properties available in their area and that disabled people should be excluded.
That's something the Church of Scotland has been calling for.
Convener of the Church and Society Council Revd Sally Foster-Fulton said last year: "We call for the politicians responsible to rethink, revise and review this legislation which is penalising some of the most at risk members of society through no fault of their own.
"This benefit cut unfairly targets some of our most vulnerable and asks people already struggling to shoulder an even bigger burden on our behalf.
"It is a flawed policy whose implementation has already had devastating results for disabled people."
Mr Alexander said his party would start campaigning for change immediately and if it couldn't convince its Conservative coalition partners it would made a manifesto pledge ahead of the next General Election.
The Lib Dems had always fully supported the change until this point and the Labour party said it was guilty of 'hypocrisy'.
Hear more from Lib Dem MP for Eastleigh Mike Thornton: