Amanda Spielman has warned that “things can go wrong” when school systems become too reliant on data.
Ofsted’s chief inspector was speaking at the launch of a critical new report into school accountability carried out by a heavyweight education commission.
The commission led by the NAHT heads' union have called for an overhaul of the way English schools are held to account warning that the current system is doing more harm than good.
It warned that comparing pupil performance between schools in different settings was unreliable and yet this was how school effectiveness is being judged.
Speaking at the launch of the report Ms Spielman said: “There is only so much that can be done with data.
"It is worrying when any system becomes over-reliant on data. We have seen the ways that things can go wrong.
"There isn't a holy grail – we can see that balance is required.
"I have been very clear that we should be focused on what is happening in schools and what is being taught, what the substance of education is and how it is happening."
Ofsted is planning to launch a new inspection framework that will place more emphasis on a school’s curriculum and how it is delivered and less on the results a school achieves.
However, the NAHT has called on Ofsted to put its plans on hold amid concerns about how extra workload the new framework might create.
The new commission report raises concerns about the use of data in the current school accountability system.
It calls for schools to be judged on three-year average data and that Ofsted should judge a school’s effectiveness based on how it performs against similar schools.
The report also urges the DfE not to create any new performance targets and to use an Ofsted judgement of "requires improvement" as the threshold at which schools receive extra support.