It said that while leaders of ISI teams were "generally effective", other members often varied in quality and experience.
Tony Hubbard, director of ISI, said: "One of the biggest challenges that we have is getting people to make consistent judgments firmly based about standards."
He said most inspectors were also teachers or heads. "If you're using people straight out of school who only inspect once a year you're going to gain some things and have some problems to overcome," he said.
Those team members have a strong understanding of how the school system operates he said, but can have trouble with elements such as report-writing. Training courses are being organised to tackle this. Mr Hubbard said: "We're tightening up on appraisal."
Inspectors also reported difficulties when team members had to inspect subjects in which they lacked expertise. Mr Hubbard said that, under new arrangements, inspectors only report in full on one subject, though they may also cover other subjects in curtailed form.
ISI inspects about 1,200 independent schools in England and Wales. Around 10 per cent of its inspections are monitored by OFSTED.