Writing in the Daily Mail he says Christians should vote to leave the EU in next month's referendum because it's "bringing about division rather than unity".
He uses the Bible passages of Moses leading the people of Israel out of captivity in Egypt as a reason to leave.
He writes: "There should be no squeamishness about unity, peace and harmony. Many countries contribute to brotherly fellowship and international peace making without surrendering their democratic controls.
"In the case of relationships within the EU, many of us feel that the current structures and arrangements are now causing the very division, conflict and unhappiness that they were created to cure."
The Church of England has not taken an official position on the referendum.
Lord Carey says it "seems natural" that religious leaders should vote to remain in the union.
But, he says, the EU has failed in its aim to unify the continent.
"To follow the analogy of marriage and divorce there comes a time when such harm is taking place within the marriage that there is no choice but to end it.
"But of course, the relationship with the EU is not a marriage. It is a treaty which can be replaced by a better treaty."
Uncontrolled immigration is where "danger lies", he says.
"It is clear that neither local or national government can provide basic infrastructure, schooling and healthcare for our current growing population."