Most Revd Justin Welby was speaking at Muen Church in Shanghai, as part of his ten-day visit to the country.
Mr Welby also acknowledged how hard Chinese people work and the pressures they experience, and that they shouldn't evangelise through aggression or manipulation, but by reflecting God's character despite the difficulties of life.
He said: "One of the things that strikes me about China is how hard people work and the pressure they are under. Pressure includes the pressure from within us to succeed.
"We witness every day by what kind of people we are individually and as a community.
"In the New Testament the Greek word for witness and the word for martyr (as in someone who dies for their faith) are the same, 'martyroi'.
"As today in many places, for the church of the first centuries, to be a witness to Jesus Christ very often meant death.
"Peter was very clear that the heart of witness lay not in aggressive shouting at people or any other form of manipulation or disruption, but in lives that were lived so clearly that people would ask why the Christian lived in such a way and that the Christian would 'always be ready to give an explanation for the hope that is within you', but with gentleness and grace.
"Witness always includes words, but it starts with actions because it is those that people around us understand most clearly."
China has partly or fully demolished hundreds of churches in the last few years, including one church where believers formed a human shield around it, in an attempt to save it.
It's illegal to worship outside of a state-sanctioned church in China, forcing tens of millions of Christians to worship underground or in secret places.
Despite this, Christianity is growing rapidly in China, although scholars disagree on how fast it is growing, and how many Christians will eventually populate the country.
Andrew Boyd from the anti-persecution charity Release International told Premier's News Hour it was a brave speech.
He said: "What he's urging Christians in China to do is to recognise that they're called to be witnesses.
"He's drawing attention to the fact that the word in the Greek is the same as a the word for martyr, in other words, he's saying that there is a cost involved in being a witnessing Christian in China."
You can read the whole of Most Revd Welby's sermon here.
Premier's Des Busteed speaking to Andrew Boyd: