The AAP survey, on which the new Scottish Survey of Achievement will be based, first chooses schools that are representative of the country as a whole. They will reflect factors such as free meal entitlement, the urban-rural balance, large and small schools. Every survey rechecks that its sample reflects the national profile.
Each school is asked to select one of the four year groups used in P3, P5, P7 and S2. This lessens the “school effect” which could influence the results. “It is better to have fewer schools and more year groups than more schools and fewer year groups,” one of the officials in charge said.
With the year group selected, lists of pupils are randomly drawn based on date of birth. Fears that this could mean a group of like pupils being chosen - low attainers or high flyers - are said to be ill-founded since the pupils are chosen to be representative of the year group across Scotland as a whole not in a particular school.
Schools are asked to volunteer and the Executive usually has a respectable response because schools know information is not going to identify or rank them. They also want to see how they compare with national standards.
Written tests are administered by the school and practical assessments form a sub-sample since it is not necessary to test every pupil on every item in every school to obtain a picture of how the system as a whole is performing.
Teachers nominated by their authority are deployed as field officers to carry out practical assessments and mark some of the tests on the written papers that are not coded. The most recent science exercise involved 160 field officers. Teachers see it as a valuable contribution to their professional development and are keen to take part.
The AAP is a joint endeavour between the Scottish Executive, Learning and Teaching Scotland (which advises on what is to be tested) and the Scottish Qualifications Authority.
The major technical change required is to increase the AAP sample of pupils being tested, probably to double the existing 5 per cent of each year group. This is felt not to be large enough since the new survey is also intended to provide results in sufficient detail for each authority.
That is not likely to be a problem in secondary schools - the figure of 150 schools involved at S2 is felt to provide a realistic sample. But the 450 schools involved in the primary sample would be too small.
Officials are also considering how to handle the SSA survey in small authorities. One possibility is to sample by pupil, not school; otherwise the three secondaries in an area like Clackmannanshire would be hit every year.
