Authors of high-profile research who claimed to have uncovered the most effective type of head have pledged to release “further details” of their methods and data, following growing academic scepticism and government encouragement.

The decision by the management researchers follows a meeting with the Department for Education’s chief scientific adviser Tim Leunig. It marks a sudden change of heart for the authors of “The one type of leader who can turn around a failing school”.

Earlier this week, they told TES that they had no plans to reveal their methodology, despite increasing academic doubts surrounding their conclusions.

Their study identified five types of school leader in England, using labels such as “surgeon” and “soldier”, which have entered education lexicon since its findings were published in a Harvard Business Review (HBR) article in October.

The article was hailed as “phenomenal” by the BBC’s Newsnight.

Since then, a series of academics have raised questions about an apparent lack of transparency over the methodology, and have called for details to be released.

But this week Alex Hill, one of the article’s four authors, told TES that any concerns were misplaced and said they could merely reflect different approaches used in education research, and his own field of management and leadership research.

Asked about whether he would be producing an academic paper outlining his methodology, he said: “That takes a huge amount of time and effort, and, to be perfectly honest, we are doing a lot of other things and so it’s not a motivation for us.”

But later the same day, following a meeting at the DfE with Dr Leunig, Ben Laker - another of the researchers - tweeted that they were now working with him to “publish further details on our data and method and host a [DfE] seminar”.

‘It is hard to understand what they did with all the data they say they collected’

Dr Leunig told TES: “We said more people in the schools sector would find their results convincing if they understood them better.”

Becky Allen, director of Education Datalab, said that she had spoken to dozens of academic colleagues who had raised questions about the research.

“Without full academic papers, it is hard to understand what they did with all the data they say they collected,” she said.

“And we simply cannot know how generalisable these findings are without knowing how they selected the schools to include in the study.”

Dr Hill told TES: “At the moment, the reason we have not published our method is simply that our motivation was simply to help practice. HBR, we felt, was a good route to do that, and that journal does not talk about method.”

He said that the authors shared 20 pages of data with HBR before publication, and that the article was peer-reviewed.

Christian Bokhove, a maths educationalist from the University of Southampton, questioned how neatly the different types of heads were clustered together in a chart in the article demonstrating a relationship with what happened in their schools.

He said such a clear finding was very unusual in the social sciences. “Even if they used clustering techniques correctly, it’s quite speculative to do that sort of stuff, and that worried me a little bit,” he said.

Asked about questions over whether their categorisation was subjective, Dr Hill said two researchers had collected the data, and the other two had tested it to try to ensure objectivity. He added that the typology of heads - the different categories - arose “completely organically” as the research progressed.

“The clusters are a lot clearer than we expected, and we were quite shocked,” he said. “The one thing I found particularly shocking was the clear relationship between the subject the teachers had studied and taught and how they behaved as leaders. It was almost too clear, so we looked at it again, but that is actually how they were,” said Dr Hill.

The paper said the researchers interviewed 411 leaders, as well as people who worked for them, and recorded what they did over a seven-year period.

Dr Allen said that, from her experience working on studies that extracted data from, or involved large-scale surveys with, schools, she would be “surprised that they overcame the ethical and technical challenges in doing this without significant research funding in place”.

Dr Hill said there was no specific grant for the work, but it was funded from general government money for research on management.

He said that the researchers carried out multiple face-to-face interviews with all 411 leaders, across 160 academies, over a seven-year period. They also had remote access to data coming directly from schools in the study - secondaries that became academies after being put in special measures.

Trusts not involved

When Dr Hill was informed that some big academy trusts - which would account for a large proportion of those schools - had told TES their academies were not involved, he said he was unable to comment other than saying that the aim of the study was not to name and shame.

Dr Allen also questioned how representative the study was. Her own research suggests that heads with a maths or English background were “significantly over-represented” in the study, while those from science backgrounds were under-represented.

Dr Hill said he did not know if the study was representative, and argued that all research was limited. He admitted that because this study was limited to academies, and under-performing schools, it could mean that it was “skewed”.

However, he stressed that it was a large study, looking at impact over a long period of time, and that people working in schools had told him that the findings were “spot on”.

Ron Iphofen, a consultant who has written about ethical decision-making in social research, said research had to clear a high bar if it was to inform public policy.

He said he would want to know how the different categories of leaders were generated, and how the researchers ensured that the data from different schools was comparable.

“If they cannot tell us these sort of things, you have the issue of research integrity,” he said.

What the study found

Few pieces of education research have attracted as much publicity as the Harvard Business Review article written by Alex Hill, Liz Mellon, Ben Laker and Jules Goddard.

Their findings were eye-catching: only one type of school leader is truly effective, and they are paid the least, and receive little official recognition.

The research outlined five types of leaders:

* Surgeons, who boost exam results by removing poor-performing students and focusing on Year 11 pupils, but see results drop dramatically in the one or two years after they leave. Usually from a PE or religious studies background.

* Soldiers, who cut support staff and non-essential activities, but see exam results remain static, while costs “bounce back” after they leave. Usually from IT or chemistry.

* Accountants, who focus successfully on improving school revenue, but exam results remain static. Usually from maths backgrounds.

* Philosophers, who like debating and sharing ideas, but make little difference to exam results or finances. Usually English or language teachers.

* Architects, who re-design their school for the long-term, where performance starts to get better in their third year, and continues to improve after they leave. Usually studied history or economics, and had a career in industry.

@geomr