Take the national qualification framework. It is everywhere in post-16 education and training. Life in colleges is classified in five levels.

Students and staff can be ranked according to the qualification they have and the qualification they aspire to. Colleges can be assessed by their performance in a few short numbers - the number of level 2 or 3 qualifications. And now, with the skills strategy, money can be targeted on particular types of learning. When an idea is so widespread, the easiest thing is to accept it. But what if flawed data results in the wrong policy ?

The beauty of the qualification framework is that it tidies up the messy business of post-16 education and training - 16,000 qualifications can be classified. The complexity of individual achievement and potential can be summarised in a number between 1 and 5. This simplicity makes the qualification level a useful tool for researchers. The highest qualification level obtained by an individual is a proxy for their skill level.

The strength of the qualification framework is the work that has gone into producing it. A vast business-education complex has spent hundreds of millions of pounds over 20 years classifying qualifications.

Education and training works on progression and hierarchy. Professions, exam boards and colleges invent their own levels to fill the gaps where the national framework does not reach. Before the framework existed, there were ranking schemes from City and Guilds, BTEC and others. The national framework adapted these schemes and developed them. This helped secure acceptance but is the main source of its weakness.

For all the research underpinning the framework, some of its rankings are set by politics. GCSEs are classed at level 2 and A-levels at level 3 because that was what the government of the day decreed. This established a set of equivalences, which may or may not be true. University subjects are always at level 4 or 5, whatever the real level of difficulty.

Graduate-only professions ensured that their qualifications were level 5.

Statistics inevitably involve value judgments, but there is nothing inevitable about the way in which admission to the framework is controlled.

While old qualifications have their level by right, new qualifications are only admitted if they pass numerous tests. The inclusion of A-levels in the framework requires the defence of standards. As a result, the barriers to entry are so high that qualifications are only submitted for inclusion where there is a clear gain - either substantial public funding or clear recognition within the education system.

We are left with a framework biased in certain directions - towards initial education, full-time study and courses that are all about progression. The framework is a perfect tool for measuring the progression of people from school to university to their first job.

What the framework does not do well is measure the education and training that goes on in the rest of adult life. Evening and weekend learning only leads to national framework qualifications where this helps secure funding.

The most valuable qualifications paid for by adults - anything from taxi-driving to network engineering - are invariably offered outside the framework.

Most learning at work is short, specific and uncertified. This may suit business but it does not suit government. If employers will not voluntarily enter staff for the right qualifications, the skills strategy will find ways to change their minds.

A package of incentives and publicity is on its way, backed up by the threat of compulsion. If voluntarism does not work, the Government may use legislation to persuade employers to use the framework. Colleges will be pushed into dropping non-target courses and to concentrate on level 2 and 3.

This is policy with an inexorable logic - but at what cost? Qualifications have a vital role in society. They certify the competence of individuals.

They assure customers that companies have qualified staff. Where the right qualification does not exist, someone invents it.

What is odd about the current skills consensus is the belief that qualifications only matter if they fit the national framework.

Researchers like measuring things. The education and training system is gearing up to produce the right kind of data - achievements in qualifications that are best taken full-time and when you are young.

Julian Gravatt is finance director of The City Lit, London