The Church has released the findings of an independent review into its handling of the abuse.
The review, titled 'An Abuse of Faith' was launched after Ball was sentenced in 2015 to 32 months' in prison after admitting the abuse of 18 children between 1977 and 1992.
He was released from prison earlier this year having served half his sentence.
The review by Dame Moira Gibb found the Church failed to respond appropriately while the Archbishop of Canterbury at the time, Lord Carey, didn't listen to survivors of abuse.
In her foreword, Dame Moira said: "This report considers the serious sexual wrongdoing of Peter Ball, a bishop of the Church of England who abused many boys and men over a period of twenty years or more.
"That is shocking in itself but is compounded by the failure of the Church to respond appropriately to his misconduct, again over a period of many years.
"Ball's priority was to protect and promote himself and he maligned the abused. The Church colluded with that rather than seeking to help those he had harmed, or assuring itself of the safety of others."
She also told Premier: "He was appearing outwardly as a good and holy man while actively harming others. He abused the faith the people were rightly entitled to have in him as a bishop, and more importantly he abused the faith of those who sought spiritual guidance from him and instead found hurt, deceit and manipulation."
Responding to the review, Justin Welby said: "Abuse of Faith makes harrowing reading: the Church colluded and concealed rather than seeking to help those who were brave enough to come forward.
"This is inexcusable and shocking behaviour and although Dame Moira notes that most of the events took place many years ago, and does not think that the Church now would conduct itself in the ways described we can never be complacent, we must learn lessons."
Lead Safeguarding Bishop Bishop Peter Hancock said there was absolutely no excuse for the church's response and it's brought humility to the church.
He added that the church failed the survivors and called the content of the reports 'disturbing.'
He told Premier: "We did too little too late and what we sought to do was lamentable.
"There were huge mistakes and errors made across the Church. So it is a time as a church we have to be very clear and say those that were vulnerable, those who came to us for help, were not given the help they should have had."
Lord Carey has also apologised for his actions. In a statement, he said: "'An Abuse of Faith' makes deeply uncomfortable reading.
"I welcome the recommendations of the report and I accept the criticisms made of me."
The report has eleven recommendations for the Church including getting the right support in place for survivors, the leadership of bishops, strengthening guidance, reviewing the Archbishops' Lists and the effectiveness of disciplinary measures with regards to safeguarding related cases.