The scale of the challenge posed by the proposed literacy and numeracy qualifications for secondary pupils is revealed in new research published by the Scottish Qualifications Authority.
Its "progress report" lays bare the need to develop a clear rationale for the new tests: there are even splits over which fundamental skills should be tested.
"The relationship between literacy and numeracy and other qualifications, specifically English and mathematics, is not clear to stakeholders," states the report baldly.
Within literacy, the SQA report finds a huge variation of views on the definition of a "text". Some argue that a pupil's skill in writing formal standard English prose should be the benchmark for testing literacy; others say that such a test would "cause difficulties for those who regularly communicate using non-standard styles and technologies" and that the literacy test should be designed to fit them.
Employers say they want evidence of the ability to fill in forms and take notes, skills in grammar, punctuation and spelling, and the ability to understand, implement and give instructions.
The report also reveals deep schisms within the teaching profession over whether pupils should be allowed to use calculators in the numeracy test.
Employers are also split on the issue, but stress they want employees to be able to read and understand charts and tables and do basic arithmetical calculation.
Eddie Mullan, chair of the Scottish Mathematical Council, said the report reflected many teachers' concerns about the vagueness of the guidance issued to date. He felt that a split between teachers and employers on the purpose of numeracy testing was inevitable since teachers could not know where a pupil would eventually be employed.
Sue Ellis, an expert in literacy at Strathclyde University, said the report reflected many of the problems inherent in literacy assessment and urged the SQA to keep the assessment simple.
"There can be a tendency to look for a linear set of literacy `skills', but it is more realistic to see literacy learners as working to a `broad horizon' of competencies," Ms Ellis said.
"The problems often lie in orchestrating the different bits of information - remembering spelling while also thinking about the content, the handwriting, the layout. Bringing all the independent `bits' of knowledge together can be hard for some young people, but that is what they must be able to do."
Meanwhile, Michael Russell, the Education Secretary, announced this week the launch of a literacy action plan in the autumn, building in part on the findings of the Literacy Commission set up by the Labour Party.
"Education policy and Curriculum for Excellence have a hugely important role, but if we are to make progress we must look beyond nurseries and schools to see what else we can do to make a difference," he told a literacy conference in Glasgow.
The CfE management board would be providing advice later this month on the literacy and numeracy qualifications, said Mr Russell.
Their discussions will inevitably include the "progress report" published by the SQA on the new tests - tests which many teachers believe are not necessary but which have been espoused by politicians in all parties.
The SQA report acknowledges that the early guidance (experiences and outcomes) for literacy and numeracy in the Curriculum for Excellence programme "provides a sound basis from which qualifications can be developed". But it warns that "additional work is required to develop a rationale for the qualifications in these two areas".
The report adds: "Progression pathways into, within, and leading from qualifications at SCQF (Scottish Credit and Qualifications Framework) levels 3, 4 and 5 are not yet clear for all learners who may use these qualifications".
It says there is "significant concern" about how skills will be differentiated across these levels, and it accepts that schools are looking for "significant support" in how to go about it.
Some of the SQA research predates the decision by Mr Russell to change the format of the qualification from its original incarnation of a portfolio showing how well pupils are performing in literacy and numeracy in all subjects to a unit within maths and English.
The report's findings appear to justify that decision, describing the "scale and management of the gathering, storing and assessment" of portfolio evidence as "the most frequently-raised concern" by staff.