The Outlook Trust, a group that works with retired people, was speaking after Age UK also found those same older people are also facing a "double whammy" of not having their care needs met either.
Age UK found that if you are an older person in this position, you are almost two-thirds more likely to have been lonely for 'much of the time' during the past week compared to those who receive some care and support.
Its report also found that there are over a million older people in this country who need care but do not receive it from any source.
Robin Rolls, from the Outlook Trust, told Premier: "I think we do this thing with loneliness - it's not in my backyard, it's not me, it won't happen to me.
"And of course we're just at this point in our nation's history over the next few years where we're going to find that there are more older people than younger people for the first time ever."
"There are some real choices for the government to make about what they do about older people, as the nation grows older more and more people are going to need extra care - and are they going to put in the resources to help the terrible tragedy of these old people being lonely, or are they just going to let it go?"
Mr Rolls added that being lonely can have an enormous impact on somebody's quality of life: "I think it has a really strong impact... look at the Garden of Eden, it's not good for human kind to be alone.
"And we know that other research shows that other research shows that loneliness is linked to mental illness, it's linked to depression and in some cases suicidal thoughts and attempts."
Listen to Premier's Hannah Tooley speak to Robin Rolls here: