A freedom of information request found the majority of authorities do not have systems in place to measure the scale of the problem, other than collecting statistics on the number of children already receiving help from social services.
The charity is urging the government to make councils properly record all neglect and children who could be at risk.
It says this would make it easier to intervene faster and bring help to young people before the neglect develops.
Action for Children chief executive Sir Tony Hawkhead said: "The tragedy is that due to a lack of gathering the right information, children whose lives could be improved are needlessly put at further risk.
"This is unacceptable when we know more can be done - we cannot allow the suffering of any child. Neglect can be stopped in its tracks.
"Families need help as early as possible so they can make positive changes in their lives, transforming their and their children's stories by being the best parents they can be."
Some local authorities responded to the request saying they were able to build a picture of who may be affected by collating information on risk factors such as parental mental health, parent learning disability, substance misuse and domestic violence.
Hear from Head of Policy and Research at Action for Children Emma Smale here: