Gilbert Deya, 65, who has run an evangelical church in Peckham, south-east London, has been wanted by Kenyan authorities since 2004 over claims he stole five children.
Deya, who fought a long legal battle to remain in the UK, denies any wrongdoing. However, two judges in London ruled he had "no arguable grounds" to seek a judicial review in his latest bid to block extradition to stand trial.
Lord Justice Gross, sitting with Sir Kenneth Parker, said it was "little short of scandalous" that his case had taken so long to resolve and it was "essential" that such extradition cases were firmly "gripped" by the Home Secretary.
He added: "The court will be more than willing to play its part."
The pastor was initially arrested in the UK in December 2006 following the issuing of an international arrest warrant by the chief magistrate's court in Nairobi, Kenya.
He is alleged to have said he could deliver "miracle babies" to post-menopausal or infertile women in Britain.
Once the women were convinced they were pregnant they travelled to Kenya with his wife Mary where they "gave birth" in back-street clinics before being given a baby.
The infants are alleged to have been taken from their real parents.
More than 50 families in Nairobi have said their children were stolen.
The then Home Secretary Jacqui Smith ordered his extradition in 2008, while Theresa May did so again after further investigations in 2011.
