The Archbishops were joined by Chief Rabbi Ephriam Mirvis, Sheikh Ezzat Khalifa, Sheikh Mohammad al-Hilli and Sheikh Qari Asim.
At the vigil, Archbishop Welby said that those who were killed were at the forefronts of everyone's mind as London looks towards the future.
Archbishop Welby said: "We have all of us come together because it is a moment of sad reflection but also a moment of determination for our nation together.
"In standing here, we represent the three Abrahamic faith communities, all equally committed to a peaceful future."
The Archbishop added that in the story of death of resurrection of Jesus, we find a God who is with us in sorrow and a God whose love conquers "the despair and destruction that the events of Wednesday speak of".
Cardinal Nichols said that, since Wednesday, Londoners have shown that they will never be cowed by terrorism.
"No person and no event will drive a wedge between us, together we will prevail," said the Archbishop of Westminster.
On Wednesday, Khalid Masood ploughed a hired car across Westminster Bridge and before fatally stabbing PC Keith Palmer.
His victims on the bridge American Kurt Cochran who was on the last day of a trip celebrating his 25th wedding anniversary with his wife, and Aysha Frade, who worked in administration at independent sixth-form school DLD College London, in Westminster.
Last night 75 year old Leslie Rhodes from Clapham, south London died in hospital.