Fewer children are being entered for extension papers in English, maths and science, taken by 11-year-olds who are up to three years ahead of their classmates.
The Qualifications and Curriculum Authority, the quango which runs the tests, admits the pressure to meet literacy targets has led to a fall in the results of the country's brightest children.
It is so concerned about the drop in entries that it plans to review how the most able children can be stretched by primary tests.
However, more children are passing the maths test despite a steady decline in the number of entrants.
Martin Tibbetts, chair of the National Association for the Teaching of English, believes the pressure on primary teachers to reach literacy targets has led to an increased focus on children just below the average at the expense of the most able.
Only 59 children in the country achieved level 6??? in the extension English paper sat by more than 21,600 pupils last summer. The English paper has seen the largest fall in entries. Three years ago nearly 34,500 children sat the test compared to 21,600 last year.
Mr Tibbetts, headteacher of Cheslyn Hay county primary in Walsall, said:
"Many teachers feel forced to concentrate on children who are borderline level 4 candidates who could boost league-table performance. Sadly many bright kids are being left to swim on their own and heaven help you if you have special needs. The average ability range has become the new battleground."
A QCA spokesman said: "The English variations are likely to be a combination of both statistical anomalies and teachers concentrating their efforts on borderline level 4 pupils.
"QCA is concerned about the drop in entries and is reviewing the current extension papers in the longer term to find more appropriate ways of extending the most able at the end of key stage 2."