An average of four inspectors a year have been banned from inspecting schools for unprofessional behaviour or for writing misleading reports.
A total of 43 registered inspectors have been deemed unfit to inspect since the Office for Standards in Education began visiting schools in 1993.
David Bell, the chief schools inspector, told academics at London university's Institute of Education that the high number who had been removed from Ofsted's list showed that inspectors were watched closely.
"Inspectors arguably receive as much scrutiny as the schools themselves," he said.
After his lecture Mr Bell said he believed that disqualifications of inspectors had taken place at a steady rate during Ofsted's first decade.
In some cases inspectors had been barred following complaints by schools, he said, while in others, Ofsted had noticed irregularities in several of their reports.
"It would not just be for one report," he said. "It would have to be a very serious offence or a number of misleading reports."
Mr Bell stressed that more than 90 per cent of schools felt their inspections had been fair and accurate. The number of inspectors disqualified appears high when compared to figures for teachers banned from working in schools by the General Teaching Council for England. The council barred only three teachers last year, the first year in which it heard complaints.
The National Association of Educational Inspectors, Advisors and Consultants, said it felt that inspectors were under significantly more scrutiny than teachers during inspections.
John Chowcatt, the association's general secretary, said: "There is a very detailed code of conduct for inspectors which schools know about, and there are also clearly laid-out procedures for schools who want to complain.
"We have been arguing for some time that the balance is unfair. Schools do not have the same requirement to be courteous to inspectors that inspectors have to teachers."
Carol Fitz-Gibbon, a professor of education at Durham university and a leading critic of Ofsted, said the education watchdog was under insufficient scrutiny, no matter how many inspectors it had disqualified.
"I have no doubt that before the next election the Government will want to be able to produce good reports so it can claim that something has worked," she said. "So what I want to know is, when will we see a Royal Commission investigation of Ofsted?"
In his speech to the Institute of Education, Dr Bell attacked academic criticism of Ofsted, suggesting that much of it had been biased, ill-informed and "below the belt".
"The 'Ofsted, Schmofsted' approach that characterises the critiques doesn't take us any further," Mr Bell said.