Primary teachers who do not understand maths are holding back pupils' progress in the subject and failing to stretch the brightest children, Estyn said this week.
Following school visits this year, inspectors concluded that it was underconfidence and a lack of deep mathematical understanding, rather than dislike of the subject, that hampered some teachers' ability to pass on numeracy skills.
But in a report published this week, inspectors said that despite failings such as poor marking standards, maths teaching had improved overall. Nearly all children now enjoy the chance to solve problems for themselves, they said.
But standards at key stage 1 have remained static since 2003. Inspectors said this was partly caused by pupils being routinely marked down by staff who lacked confidence in their ability to teach maths.
"Many staff view awarding a level 2 as a `safe option' as they do not feel confident enough to award the higher level," the report said.
This lack of confidence also led to an over-reliance on textbooks and worksheets at the expense of the needs of individual children, it said.
"Where schools slavishly follow published schemes, standards are lower," inspectors concluded.
The brightest pupils, in particular, were demotivated by work that was simply "more of the same type of sums" rather than being a fresh challenge, the report said.
In response to the report, Dr Bill Maxwell, the chief inspector of schools for Wales, said maths needed to be challenging if standards were to rise.
"In the best schools, staff challenge pupils effectively," he said. "Schools need to focus on offering more opportunities for pupils to use and apply maths in their daily work, including improving the level of challenge for the more able to develop their thinking and problem-solving skills."
The report also said teachers needed to mark work in greater detail to help pupils understand how they could improve.
"Marking occasionally consists merely of ticks and crosses . This kind of marking reinforces failure and is often a waste of teachers' and pupils' time," the report said.
Schools with high expectations achieved the best results, Estyn said. And visiting high-performing schools was a good way improve teaching, although few teachers did this, the inspectors found.
Estyn recommended that local authorities should give teachers more training in maths and on making their assessments more accurate and consistent. Schools were also asked to deepen teachers' understanding of the subject so they could push pupils to do better.
But staff were praised for the early identification of pupils who underachieved in maths. The activity-led foundation phase, in particular, was credited with improving concentration and enthusiasm for maths, particularly among boys.
Trainee teachers in Wales are no longer obliged to pass key skills tests that measure their ability in maths and English. But nearly all teachers and support staff with responsibility for three to seven-year-olds have been trained in mathematical development in the foundation phase.
In nearly all schools, the youngest pupils learnt maths through practical activities, which inspectors deemed very effective.
Cross-curricular use of maths was also on the increase, with pupils measuring the height of sunflowers in science and counting in Welsh. But Estyn said schools should only use numeracy in lessons where it was appropriate and would help deepen their understanding.