Ann Dowker, an Oxford university researcher, said: "It is not the case that a large number of children are simply bad at maths and that nothing can be done about it."
Dr Dowker found children who had difficulties in maths often struggled only with one or two aspects. But by being labelled, or labelling themselves, as "bad at maths" they could end up finding all maths hard.
Common difficulties were memorising mathematical facts such as multiplication tables, solving problems expressed in words, and calculations involving different stages, she said.
Dr Dowker said short bursts of individual teaching can make a big difference - if they are targeted at specific misunderstandings.
"Individualised work with children who are falling behind in arithmetic has a significant impact on their performance. The amount of time does not need to be very large to be effective."
A pilot project by the National Numeracy Strategy, being run in 25 authorities, provides 30 minutes of maths teaching a week for children who need extra individual attention.
The strategy's programme is still being evaluated, but Dr Dowker said that help of that sort can have a significant impact.
Her research shows that putting children into a low-ability set or stream can exacerbate the problem by failing to stretch their mathematical skills.
She found that although many children experienced some difficulties with maths, it was quite common to have a marked differences in ability between different aspects of maths.
She said: "Some children do apparently difficult things before apparently easy things."
Some children may have difficulty remembering their multiplication tables, but be able to do word problems. Others may be able to remember facts, but lack strategies for working out unfamiliar sums.
Her report comes amid concern from employers and government about a general fall in maths skills among adults.
In England, four times as many adults have problems with numeracy than with literacy and most have been experiencing difficulties since they were seven years old.
What works for children with mathematical difficulties: www.dfes.gov.ukresearch