He was due to take part along with others standing in the South West of England on Wednesday but cathedral bosses have claimed his appearance could pose a risk to public order.
Benjamin hit the headlines when he joked about the rape of Labour MP Jess Phillips - something he refused to apologise for.
Recent public appearances have seen protests against him.
A spokesperson for Exeter Cathedral said: "Under the rules of the Electoral Commission, we may exclude candidates from a non-selective hustings for a number of reasons, including concerns about public order.
"In this case, the cathedral believes that the presence of one particular candidate may cause a risk to public order, given a number of incidents over the last few weeks."
The cathedral has invited UKIP to find a replacement.
UKIP says the decision is "outrageous" and an affront to democracy.
A full list on candidates standing in the South West elections can be found here.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.