The White House said it could not confirm if the video was genuine but it did say it believed Kenji Goto was dead.
US president Barack Obama said it was a "heinous murder".
Mr Goto, 47, was captured by IS militants in Syria in October. It's believed he was there trying to secure the release of a friend.
Obama said the journalist "courageously sought to convey the plight of the Syrian people to the outside world".
Prime Minister David Cameron said: "I condemn what appears to be the despicable and appalling murder of Kenji Goto. It is a reminder that Isil [IS] is the embodiment of evil, with no regard for human life."
The death ended days of negotiations to try and save the Christian. Islamic State had demanded the release of a would be suicide bomber from Jordan. Jordanian officials said they would release the terrorist but only if they had proof a captured pilot was alive.
Japanese Prime Minister Shinzo Abe reacted with horror: "I feel indignation over this immoral and heinous act of terrorism.
"When I think of the grief of his family, I am left speechless. The government has been doing its utmost in responding to win his release and we are filled with deep regret."
Mr Goto's mother, Junko Ishido, said: "Kenji has died and my heart is broken. Facing such a tragic death, I'm just speechless."
The journalist had spoken about his Christian faith in the past.
Speaking to a magazine in Japan he said: "I have seen horrible places and have risked my life, but I know that somehow God will always save me."
But he added he would never deliberately do anything dangerous, "Do not put the Lord your God to the test," he said.