One in five primary schools in England claims it has no pupils who are gifted or talented, according to government officials.
But teachers are acting illegally if they do not identify bright children or those who excel in sports or the arts, civil servants have said.
Tim Dracup, head of gifted and talented policy at the Whitehall Department for Children, Schools and Families, said too many teachers viewed gifted and talented (Gamp;T) provision as elitist or their pupils as not clever enough.
Those considered to be gifted or talented perform in the top 5-10 per cent of every school, so all primaries and secondaries should have identified these pupils and be providing extension activities to stretch them.
The most recent statistics show that about 820,000 pupils in England have been identified as gifted and talented - some 10 per cent of the school population. But the one in five estimate suggests that about 3,400 primaries are not running extra activities for their brightest pupils.
This is not the first time the Westminster Government has raised concerns about teachers' lack of involvement in Gamp;T provision. The former schools minister, Lord Adonis, wrote to all schools last year to remind them that identification of Gamp;T pupils was a legal duty.
Secondaries have traditionally been better at noticing bright pupils, as national policy has been geared towards older children and has only been extended to include pupils from ages four to 19 since 2007.
"I hope I don't need to persuade teachers how important the census is in shaping national policy," Mr Dracup said. "It seems many have philosophical issues with the label `gifted and talented', but the census is statutory and if they are not filling it in, then they are acting illegally.
"The guidance doesn't give anyone the opportunity to say, `There's nobody we can identify here,' because it's relative to their group of children. We want all schools to put down a marker and give extra challenge and support."
Mr Dracup wants schools to make sure they particularly identify more poor pupils as gifted and talented, especially those who are in care. "It's easy to spot the high attainers, but much more difficult to spot those who are gifted and talented but underperforming, especially if they are masking their ability," he said.
Denise Yates, chief executive of the National Association for Gifted Children, said in her experience many schools "shy away" from identifying the gifted and talented.
Major changes to provision for Gamp;T children will come into force next year. The Centre for British Teachers education trust will no longer have responsibility for a national learners' academy, website and register, and schools will be asked to run more activities themselves.