The 63-year-old peer revealed he has stood down from his role as a Presbyterian elder in Belfast and will be moving to England, as he criticised the "direction of things" within the Church.
Last week, Irish Presbyterians voted to relax ties with the Church of Scotland - which voted last month to draft new legislation allowing ministers to conduct same-sex weddings.
He told the BBC: "It is no longer possible for me to defend the position of the Presbyterian Church.
"It is no longer possible for me to feel that it is an appropriate expression of my faith and so I have decided - after 30 years as an elder and a lifetime as a member of the Presbyterian Church - to resign from the eldership and membership of the Presbyterian Church.
"That is a very difficult and painful decision, but it is one that has come about over a period of time watching the direction of things."
The Liberal Democrat peer also cited a decision by the Presbyterian Church of Ireland to exit the World Council of Churches in 1980 after objections to grants from the body's Programme to Combat Racism to armed liberation movements in Southern Africa.
Meeting in Belfast last week, Irish Presbyterian also decided to adopt a controversial policy which will see people in same-sex relationships refused full church membership.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.