Its report said schools, local authorities and the Government shared the blame for failing to protect children.
Ministers have responded by promising fines and up to five years in jail for employers who neglect to vet staff.
Ofsted's damning verdict came in a report commissioned by the Government in January, after a teacher on the sex offenders register was allowed to return to school.
This week two sex offenders were cleared by the Department for Education and Skills to work in schools, despite advice to consider blacklisting them. The advice came from Sir Roger Singleton, formerly head of Barnardo's, who was appointed in January by Ruth Kelly, then education secretary, to advise on such decisions.
A DfES spokesman said his recommendations had been followed. Twenty-two sex offenders had now been blacklisted.
Ofsted said local authorities and schools were failing to keep records of checks.
Only three out of 58 schools inspected were able to provide evidence they had carried out checks. Some ignored recommendations not to employ staff and there were variations in the checking of governors and volunteers, inspectors found.
Lax regulation allowed agencies to avoid vetting supply teachers unless specifically asked.
The report sparked a fresh round of hand-wringing despite Ofsted's acknowledgement that it was "highly likely" checks had been done. Maurice Smith, chief inspector, said: "Schools and local authorities must keep up-to-date and reliable records.
"Currently, everybody thinks that somebody else is doing this or it is somebody else's job when, in fact, hardly anybody has secure evidence that any of the simplest tasks have been completed at all, or when.
"I want to walk into any school next term and, if I ask for a list of staff, and secure evidence that their identity, qualifications and criminal records have been checked, the school will be in a position to show me."
The Government promised to tighten rules on vetting staff in schools. A DfES spokesman said the Safeguarding vulnerable groups Bill would ensure employers faced up to five years in jail, or a pound;5,000 fine, if they failed to vet staff properly.
Schools will have to review their records, which will be checked by Ofsted.
Criminal record checks on overseas applicants will be mandatory.
Alan Johnson, the Education Secretary, said: "More needs to be done to ensure proper records are maintained."
John Dunford, general secretary of Association of School and College Leaders, said: "Schools pay a heavy premium for supply teachers. The minimum service they should expect is that agencies carry out full checks on staff."
Safeguarding children: an evaluation of procedures for checking staff appointed by schools is available at www.ofsted.gov.uk
OFSTED GUIDE TO VETTING
What Ofsted says schools should do:
* Establish and secure the identity of staff, or seek assurances that others have done so
* Verify the authenticity of qualifications - establish which staff require checking against List 99 and the Criminal Records Bureau, and ensure this is done
* Maintain up-to-date records, showing these three tasks have been carried out, and when they were done
* Take advantage of the online training available from the National College for School Leadership for headteachers and governors, and be held to account for not doing so