Revd Graham Nelson attended the rally in Preston and told Mr Johnson he was "not welcome in Preston" whilst he told fellow out campaigner Michael Gove he should be ashamed of himself.
Then "a couple of men in suits" pushed and shoved the vicar with one heard to say "get him out of here", the Lancashire Evening Post reports.
"I didn't go to the Flag Market intending to heckle them, but it just got the better of me," Revd Nelson told the newspaper.
"To be fair, when I shouted those things I expected there would be a reaction from someone in the audience, some of whom were clearly Brexit supporters.
"But the men in suits looked like they could have been campaign officials and I heard one of them shout 'Get him out of here.
"That's fine, I'm not taking this any further. It's all part of the big debate. Passions can run high at times.
"In fact I'm laughing it off. I haven't had so much fun since I heckled some speakers in Glasgow during the Scottish independence referendum last year."
Vote leave has been contacted for a response.
The referendum on whether the UK should remain a member of the EU will take place on June 23rd.