In a report issued yesterday (Thursday), Audit Scotland concludes that, while the agreement has been of benefit to teachers and led to other improvements, the lack of performance measures means it has been unable to judge the wider impact.
In a warning to the Scottish Executive, the report casts doubt over whether all parts of the agreement can be sustained beyond its first implementation phase, which ends this year.
Peter Peacock, Education Minister, commented: "With such a major programme of change, you should always expect some teething problems but, overall, as this report shows, there can be real satisfaction that implementing the agreement is going well."
But Audit Scotland suggests that the Government's drive to cut the size of the public sector and the uncertainties of council funding "will create a more challenging environment for local authorities to sustain the positive momentum identified in this report".
The report points out that authorities such as Highland are already arguing that tight spending limits will make it hard for them to achieve the final reduction in class contact time by August this year. That requires a further hour less for primary teachers in front of a class.
Audit Scotland also notes that areas of the agreement such as continuing professional development have so far enjoyed protection through specific grants, and authorities will have to start planning now for when that will no longer be the case.
Robert Black, Auditor General for Scotland, said: "Performance management arrangements need to be strengthened to demonstrate that the agreement has delivered value for money and is improving education in Scotland.
"The agreement is strong in detailing what needs to be done and by when, but it is less clear about how the cost and impact of the changes should be assessed."
Audit Scotland wants to see a string of performance measures to check if the agreement is working effectively. These should cover recruitment and retention, staff morale, workload and skill mix, the quality of leadership, better classroom practice and the impact of all these improvements on pupils' attainment.
The report also urges the executive to spread the good news about best practice so the various elements of the agreement can be fully reinforced.
Audit Scotland's investigation paid tribute to positive benefits such as industrial harmony in schools, better teacher recruitment, the teacher induction scheme, improvement in the quality and variety of CPD, and more support for non-teaching tasks.
The downside includes the greater burden on headteachers because of reductions in class contact time, the failure to meet targets for support staff, limited uptake of the chartered teacher scheme and the upheaval in secondary schools caused by new career and management structures.
Audit Scotland makes clear it has not assessed the impact of the agreement on the quality of teaching and attainment, or the changes in negotiating arrangements for teachers' terms and conditions. These are being left to HMIE which will report on its evaluation later this year.
After that, Audit Scotland and the inspectorate will consider what more needs to be done to check on long-term progress, including the effectiveness of the various approaches taken by education authorities.
Audit Scotland commissioned independent surveys of 507 heads and deputes, 1,411 teachers and 2,582 support staff.
Report and reaction 4-5 Leader 22