This plan has been years in the making. As early as 1990, he wrote in the Sunday Times: “When people see pictures of the Princess of Wales taking her sons to nursery school, they ‘Ooh’ and ‘Ahh’. Why don’t they get outraged? I’m not saying it’s wrong for her sons, but why aren’t nursery schools available for everyone? We have a tremendous problem of deference in this country. It leads to educational apartheid.”

And this week, he said: “There is too much deference to social position and not enough deference to intelligence.”

Mr Walden, who has announced he will not stand again as Conservative MP for Buckingham at the age of 55, is not afraid of attacking royalty if its behaviour “demeans the national intelligence”, and has long criticised the “apartheid” public and private education system in Britain.

These criticisms may seem misguided, if not bizarre, coming from a Conservative MP representing the most “true Blue” county in the land, but they are simply the frank beliefs of a man who has “come out” politically, and said goodbye to a place powered by what he calls “illusion”.

“I want to irritate everyone in the business to the maximum degree so they ask themselves this simple question: Unlike any other country in Europe, why do the governing classes in Britain - by that I mean TV editors, Conservative ministers and businessmen - have nothing whatever to do with the state sector?”

Mr Walden, whose resignation as higher education minister in 1987 was just as abrupt as this week’s surprise announcement, wants to see a school shake-up which would bring Britain more in line with France, where the best schools belong to the state and the private sector is reserved for specialist and religious schools.

In the first phase of the reorganisation, he envisages the former direct-grant schools being coaxed back into the state sector. Manchester Grammar School is keen to come back, and 74 of the 120 former direct-grant schools have told him they are interested in changing their status from independent to maintained.

The plan would cost money, says Mr Walden, and he does not want to see the resurrection of the grammar schoolsecondary modern system. These schools would remain selective, but they would be open to everyone, not just the rich.

The central piece of philosophy behind the plan is that state schools are never going to be good as long as the people who run the country avoid them.

Unfortunately, apart from Alan Howarth - like Walden, a former higher education minister - none of his Conservative colleagues was interested in the idea.

Mr Walden dismisses the Prime Minister as a man “lacking in imagination”, and says: “My side don’t know what I’m talking about. They think I want a class war. If you question the private schools, there is a defensive reflex as if you want to burn them down.”

What he’s talking about is selection by aptitude, and not by money. And he put this to the Labour leader at a recent meeting. Labour is prepared to sit back and allow independent schools to educate the rich, so why doesn’t it contemplate bringing the private schools under the state wing and allow selection by aptitude, he asks.

Tony Blair was interested enough when he wrote explaining his ideas to invite him to a meeting. What followed, says Mr Walden, was off the record but “a grown-up” conversation about education.

The meeting and the word “grown-up” are important because, in telling Charles Strickland, chairman of the Buckingham Conservative Association, of his resignation, he condemned political life as a system which “precludes a grown-up dialogue with the public”. He also wrote that he found it hard to fix his mind into “a Left-Right compress”. The meeting itself was unusual, he says, because an opposition leader did not normally have private meetings with Conservative MPs.

He believes it is significant that the last Labour statement on education said nothing about independent schools.

The problem is the “culture of resentment” towards the independents among the liberal Left. Labour was not always against selection - the anti-selection policy is a relatively recent one, he says.

But Mr Walden is not planning to switch parties, even if he has also denounced the nursery voucher scheme as “ridiculous”, and no more than a ploy for using taxes to give money to people like him.

He claims to have no immediate plans other than spending more time with his family - his wife, Sarah, is a successful art historian - and reading lots of novels in preparation for his chairmanship of the Booker Prize committee.

But he does promise we’ll be hearing more.