Witnesses said the first blast inside the Jolo cathedral in the provincial capital sent churchgoers, some of them wounded, to stampede out of the main door.
Army troops and police posted outside were rushing in when the second bomb went off about a minute later near the main entrance, causing more deaths and injuries.
The military is checking a report that the second explosive device may have been attached to a parked motorcycle.
The initial explosion scattered the wooden pews inside the main hall and blasted out windows, and the second bomb hurled human remains and debris across a town square fronting the Cathedral of Our Lady of Mount Carmel, witnesses said.
Police said at least 20 people died and 81 were wounded, correcting an earlier toll due to double counting. The fatalities were 15 civilians and five troops, and the wounded were 14 troops, two police and 65 civilians.
Troops in armoured carriers sealed off the main road leading to the church while vehicles transported the dead and wounded to the town hospital. Some casualties were evacuated by air to nearby Zamboanga city.
Defence secretary Delfin Lorenzana said: "I have directed our troops to heighten their alert level, secure all places of worships and public places at once, and initiate proactive security measures to thwart hostile plans."
The office of President Rodrigo Duterte said: "We will pursue to the ends of the earth the ruthless perpetrators behind this dastardly crime until every killer is brought to justice and put behind bars. The law will give them no mercy."
The statement added: "The enemies of the state boldly challenged the government's capability to secure the safety of citizens in that region. The (armed forces of the Philippines) will rise to the challenge and crush these godless criminals."
Jolo island has long been troubled by the presence of Abu Sayyaf militants, who are blacklisted by the US and the Philippines as a terrorist organisation because of years of bombings, kidnappings and beheadings.
A Catholic bishop, Benjamin de Jesus, was gunned down by suspected militants outside the cathedral in 1997.
No one has immediately claimed the latest attack. It came nearly a week after minority Muslims in the predominantly Roman Catholic nation endorsed a new autonomous region in the southern Philippines in hopes of ending nearly five decades of a separatist rebellion that has left 150,000 people dead.
Although most of the Muslim areas approved the autonomy deal, voters in Sulu province, where Jolo is located, rejected it. The province is home to a rival rebel faction that is opposed to the deal as well as smaller militant cells that are not part of any peace process.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
