Most Revd Justin Welby said people are entitled to fear the impact of uncontrolled migration on their communities.
In an interview with Parliament's The House magazine, he said that the scale of the migrant crisis meant such anxieties were entirely reasonable.
"Fear is a valid emotion at a time of such colossal crisis. This is one of the greatest movements of people in human history. Just enormous. And to be anxious about that is very reasonable," he said.
"There is a tendency to say 'those people are racist', which is just outrageous, absolutely outrageous."
The Archbishop said it was essential that the "genuine fear" that people felt was listened to and resources put in place to address their concerns.
"In fragile communities particularly - and I've worked in many areas with very fragile communities over my time as a clergyman - there is a genuine fear: what happens about housing? What happens about jobs? What happens about access to health services?" he said.
Archbishop Welby said the British people had shown in the past they were able to deal "brilliantly" with the challenges of the situation in the Middle East.
"It is simply a question of the scale on which we are prepared to act, in a way that spreads the load so it can be managed," he said.
"Fear is justified, I wouldn't want to criticise that for a moment, but so is hope wholly justified, because we have the capacity.
"We're those kind of people, we always have been."
Lord Andrew Green is a Christian and the director of the thinktank Migration Watch, which says current levels of immigration are too high.
The peer said immigration was good for Britain and had been for many years, however it needs to be controlled.
He told Premier's News Hour: "If we go on as we are at present levels, we're adding every year the population of a city the size of Liverpool. That's half a million people.
"The people at the receiving end are the people whose housing and jobs are affected, and they're not the ones living in leafy suburbs.
"They are the ones who don't have a lot of money, they can't just move out of an area because they don't like the change, and they are concerned about it.
"I admire the Archbishop for his courage in saying this, and for the measured way in which he expressed it.
"I think that what he said was a wonderful breath of fresh air."
Listen to Premier's Alex Williams speaking to Lord Green on the News Hour:
