The group has made the recommendation to a parliamentary inquiry into how to reduce the size of the Upper House.
Currently the House of Lords has more than 850 members, making it the second-largest legislative body in the world.
Two archbishops - Most Rev Justin Welby and John Sentamu - and 24 bishops are members of the House of Lords and can vote on legislation.
But the NSS says that Church of England leaders should not automatically get a vote in the house.
Stephen Evans, the campaigns director of the National Secular Society, said: "There is no reasonable justification for allowing Church of England bishops to act as ex-officio legislators.
"The Bench of Bishops is an anomaly in a modern, liberal democracy and if a need to reduce the size of the chamber has been identified the Bishops' Bench is an obvious place to start."
The society has campaigned for the removal of the Bishops' Bench before; following a 2012 campaign, the Government proposed reducing the number of Church of England bishops from 26 to 12 but these reforms were later dropped.
In 2010, the society conducted a poll in which 70 per cent of Christian respondents believed that it was wrong that 'some Church of England bishops are given an automatic seat in the House of Lords'.
A spokesperson for the Church of England said: "The number of bishops in the House of Lords has been capped for over a century and a half – the only grouping in the House to which those conditions apply. Bishops also retire at the age of 70.
"The bishops in the Lords have long argued that the size of the House needs to be addressed."