For many policy advisers, working in No 10 would be the pinnacle of their career, a confirmation that they had finally made it. But for James O'Shaughnessy, it was merely a "stint" before he got down to his real mission: improving schools.
No sooner had he quit his role as head of David Cameron's policy unit at the end of last year than he was back at his alma mater, Wellington College, helping it to take over the running of more academies.
And just last month he penned a controversial report for the right-wing thinktank Policy Exchange, calling for the Department for Education to be given greater powers to intervene in poorly performing schools.
Speaking to TES, Mr O'Shaughnessy says extra powers are vital if the government is to respond to what he predicts will be a big rise in the number of schools deemed substandard by Ofsted. Education secretary Michael Gove has not shied away from controversy in promoting academies, but Mr O'Shaughnessy wants him to go further: any school not classed as "good" by Ofsted should be immediately converted to academy status, he says. A failure to face up to this long tail of underperformance could prove fatal for the government, he believes.
"You can't be in a situation where you have thousands of schools that have been told they need help, but there aren't enough people to give them help," he says. "That is a proper political problem, because it looks as if you've diagnosed the illness but you don't have enough doctors to sort it out. If you're not seen to be sorting it out, then it can become a competence issue.
"My understanding is they are reorganising the Department and they are putting more resources into this area," he adds. "I just don't think anyone has got their heads round how many schools will be told they have to sort themselves out."
With so many thinktanks currently operating in Westminster, it might be tempting to dismiss Mr O'Shaughnessy. But with his connections in No 10 and in the Department for Education (he has already discussed his report with Mr Gove), it is safe to assume that his views carry more sway than most.
As well as calling for immediate academy conversion, his paper recommends that any school that does not improve within three years should be forced to join a successful chain. Should the school receive a third disappointing Ofsted rating, the academy would be placed in the hands of an organisation - which could be a profit-making business - on the basis of payment by results.
Introducing more private business to state education is one of the great fears inspired in opponents on the Left by Mr Gove's reforms. But to see Mr O'Shaughnessy as a typical swivel-eyed free marketeer, eager to let corporations loose on England's education system, would be wide of the mark, he insists.
"My starting point is, I genuinely think we have a problem with this chronic seam of underperformance, so the question is, what do we do about it?" he says. "I am definitely not an ideologue for getting for-profits in for their own sake.
"But if state solutions don't take us all the way, and if we think there are good private sector providers out there, oughtn't we find a way to use their capacity and apply it to the problem at hand? The sensible answer is, yes."
Much of Mr O'Shaughnessy's CV reads like so many of those working in Westminster, be they politicians or special advisers: private school, Oxbridge, thinktank, before taking the step to the "ivory tower" of policymaking in Downing Street.
But unlike many other advisers who ensconce themselves in their offices, Mr O'Shaughnessy says he wants to be involved with what is happening on the ground. He has joined Anthony Seldon, Wellington College's master, in his crusade to sponsor underperforming state schools, taking them under the wing of the Wellington College brand. They are looking for schools to sponsor, with Mr O'Shaughnessy hosting a conference at Wellington at the end of this month on how independence can improve schools.
Mr O'Shaughnessy is so confident in the power of academy chains that he believes they will one day follow in the footsteps of the country's top private schools, such as Wellington, and open branches abroad. Schools, he says, could become a vital export in the years to come.
While working with academy chains might be a culture change from his high-octane life in No 10, school reform was always closest to Mr O'Shaughnessy's heart.
"I worked for Cameron for four years: two and a half years in opposition and a year and a half in government," he says. "It was an amazing experience, with the election campaign, the coalition negotiations; all of that was extraordinary.
"But it was always my intention to do a stint and leave and get involved with the thing I care most about, and that's schools."
1989-94: Attended Wellington College, Berkshire.
1995-98: Studied philosophy, politics and economics at St Hugh's College, Oxford.
1998-99: Teacher at Mathieson Music School, Kolkata, India.
2001-03: Special adviser to Iain Duncan Smith, then leader of the Conservative Party.
2004-07: Deputy director at thinktank Policy Exchange.
2007-10: Conservative Party director of policy and research; lead adviser to David Cameron, then party leader.
2010-11: Special adviser to Prime Minister David Cameron and director of policy.
2012: Head of group strategy at Wellington College.