The absentees, who are not enrolled in any school, are mainly from Years 10 and 11.
Chief inspector Mike Tomlinson said: "We have been looking at the numbers that should be in schools against the numbers recorded, and there is a gap of about 10,000.
"Some local education authorities don't know where pupils are, and in some they write them off their books. Some of them have just opted out of the school system."
He added that the situation could get worse if pupils become more willing to opt out of the system.
The Office for Standards in Education said the figures were calculated by looking at drop-offs as cohorts move between school years. But little is known about the pupils who are missing from the system.
Chris Keates, deputy general secretary of the NASUWT, said: "This is very worrying. Any child who slips out of the system is a cause for concern, which is why most organisations are supporting the unique pupil identifier number. We don't know what circumstances these children are in. It is an enormous number and that should worry everyone."
But John Bangs of the National Union of Teachers said high pupil turnover in some areas could be to blame. "It may well be that there is a lag in the records," he said.
A Department for Education and Skills spokesman said: "There are strict rules on when schools can delete pupils from their admissions register. From June 2002 there will be a new process of transferring pupil records electronically when children move from one school to another. Where a school does not know the destination school in advance, pupil records will be held on a central database."