The diplomat was suspected of possessing, but not producing or disseminating, child pornography including images of pre-pubescent children, a US source said.
The Vatican declined to identify the priest, but said he was back in Vatican City and that Vatican prosecutors had launched their own probe.
If the accusations pan out, the case would be a major embarrassment for the Vatican and Pope Francis, who has pledged "zero tolerance" for sexual abuse.
The diplomat would be the second from the Vatican's diplomatic corps to face possible criminal charges for such crimes during Francis's papacy.
Any trial in the Vatican would come as Francis's own financial czar, Cardinal George Pell, is on trial in his native Australia for alleged historical sex abuse cases.
The US State Department said it had asked the Vatican to lift the official's diplomatic immunity on August 21, but the request was denied three days later. For the State Department to make such a request, its lawyers would have needed to be convinced that there was reasonable cause for criminal prosecution.
The Vatican said the State Department notified it on August 21 of a "possible violation of laws relating to child pornography images" by one of its diplomats in Washington.
A US official said the priest was a senior member of the embassy staff. The Vatican year-book lists three counsellors who work under the nuncio, or ambassador.
The Vatican said recalling the priest was consistent with diplomatic practice of sovereign states.
In declining to identify him, the Vatican said the case was subject to confidentiality while still under investigation.
The Vatican has recalled envoys before - including its former ambassador to the Dominican Republic, who was recalled in 2013 after being accused of sexually abusing young boys in the Caribbean nation.
The Vatican justified its decision to remove Monsignor Jozef Wesolowski from Dominican jurisdiction by submitting him first to a canonical court at the Vatican, and then putting him on trial in the Vatican's criminal court, which has jurisdiction over the Holy See's diplomatic corps.
Wesolowski was defrocked by the church court, but died before the criminal trial started.
Dominican prosecutors initially baulked at the recall, and they never filed charges because of his immunity.
The Vatican does not have extradition treaties.
After he was defrocked, Wesolowski lost his diplomatic immunity and the Vatican said he could be tried by other courts, but it refused to provide Dominican authorities with information about his whereabouts or how he had pleaded to the charges.
