Pupils from Reception and Year 1 are being taught in pupil referral units (PRUs), The TES can reveal.
Campaigners have attacked the practice of sending five and six-year-olds to the establishments, which are normally used as educational corrective centres for seriously troubled teenagers.
But a total of 35 pupils aged five, and 55 children aged six were in PRUs at the beginning of this year, according to the annual school census.
The statistics, taken during January 2011, show a total of 715 children of primary age being educated in the establishments, plus a further five children aged under five.
The units are often used as a temporary "fix" for children who have been expelled from school before they can permanently move on elsewhere. Pupils at risk of exclusion are also sent to PRUs for specialist support that it is hoped will see them return to mainstream schools.
Enver Solomon, policy director at the Children's Society, said PRUs were inappropriate for pupils with special educational needs (SEN).
"Children with particular needs should get appropriate support, and it would be inappropriate for them to be left in a PRU if that's not the case," he said.
"Children with difficulties should be in an appropriate setting. PRUs are not designed to be places where needs can be assessed, and they should only be used as interim support."
Experts say infants often end up in PRUs when mainstream schools are no longer able to cope with their severe behavioural problems but a suitable special school is not available.
Eleonora Christodoulides, from parent-support service the Advisory Centre for Education, said: "There is no evidence a PRU can meet these children's needs any better.
"To be objective, assessment of SEN should also be done in their normal educational environment, a school, rather than in a PRU where they are getting virtually one-to-one support."
Barbara Knowles, executive director of the Social, Emotional and Behavioural Difficulties Association, said the figures could show evidence of a "punitive regime" in schools.
"It could also show a lack of special schools in a particular area, which is why these children are being sent to PRUs instead - that's the worst scenario," she said.
"Teachers could also be sending children to PRUs as a short-term measure so their needs can be assessed and then they can return to school. These statistics show the importance of early intervention and identification of behavioural and learning problems, and the importance of Sure Start centres."
Karen Summerbell, head of Broom Cottages PRU in County Durham, which regularly has very young attendees, said pupils of this age are often sent to PRUs because they have special educational needs - frequently emotional and behavioural - and their mainstream school does not feel it can support them while the assessment process is taking place.
At any one time, there are usually two pupils aged four or five at the unit. According to Mrs Summerbell they usually arrive distressed.
"Schools often don't feel they can sustain the child beyond the support they have already put in," she said. "After arriving, most children are quickly more settled, particularly if their behaviour has been about distress.
"This means they come on quickly. But there will always be a stigma attached to them because of their attendance at a PRU."
Jacky Mackenzie, secretary of the National Organisation for Pupil Referral Units, agreed children were being sent to PRUs because teachers found it extremely difficult to support traumatised children in Reception and Year 1.
"These pupils can show aggressive behaviour such as kicking, spitting and biting, and this can be very destabilising for other children," she said.
"Sometimes this behaviour is because the child has been a victim of physical or sexual assault, and this is their response and reaction."
The census shows that in 2001 there were 308 PRUs. By 2011 this had risen to 427.
The number of children sent to PRUs has fallen from 14,280 in 2001 to 14,050 in 2011. Of these, 10,905 are white, 885 are mixed race, 500 are Asian and 995 are black.
Half of the five to 10-year-olds in PRUs are eligible for free school meals.