Study patterns and avoid exam debacle
THE Deloitte amp; Touche report confirms that the Scottish Qualifications Authority acted without paying due attention to systems strategy, and without adequate input from those who under-stand information management. Higher Still is a complex systems project, not a data management or an organisational change project.
Our main research interest is organisational computing, specifically "socio-technical" systems. We seek to explain interactions ocurring when technology is placed in organisational settings.
Hence politics, morale, existing work practices and preferences are acknowledged as influencing systems design. Some researchers even call software design "congealed politics", and wise project planners will endeavour to accommodate such factors.
Looking at the SQA from a socio-technical perspective, we ask questions such as: Who agreed the objectives of the Higher Still initiative?
Were these objectives debated and whose voices were heard?
How were the roles and remits of those to be involved defined?
Who decided what infrastructure was required, and who prepared the call for tender?
Who decided what kinds of data would be collected, and who prescribed the protocols for this?
What were the competences of the steering group and the supervisory board?
What part in the process did these bodies (the steering group and board) actually play?
Who decided how the project would be aligned with the practices of schools and the workplace as well as with higher and further education?
Deloitte amp; Touche have touched on these questions, but their remit is to make palliative recommendations. Ours, in contrast, is to place the SQA story in the context of other projects where social and organisational factors are ignored, and see if there are common patterns.
Higher Still does indeed provide examples of several classic "dysfunctional" patterns, of which three are "formal facade", "shoot the messenger" and "too much, too late".
For example, we have seen several instances where apparent formality in strategic decision processes was used solely to satisfy external bodies concerned with inspection and monitoring, without having any significant impact upon the strategic outcome.
This "formal facade" is familiar in many public sector systems projects where "steering committees" of the great and the good are assembled, regardless of competence, in-depth knowledge, and track record on similar projects.
"Too much, too late" refers to failures in data design that led to the recording of too many attributes, in rigidly-designed fields that caused systems failure, and delays that led to the appointment of untrained staff, too late. This resulted in missing and inaccurate awards, and in printouts wich fail to make a pupil's status transparent to an admissions officer or employers.
"Shoot the messenger" led to hours wasted by teaching staff continually resubmitting data to a system seemingly unable to process them, compounded by the fact that local school administrative systems were incompatible with each other or the SQA database.
While we have found many such patterns in our own information strategy research, we have also found that investigating patterns can provide a valuable basis for reflection and learning. If an adequate project blueprint had been available early on, and exposed to skilled academic advisers, some of the waste and distress might not have happened.
Patterns can be used positively as well as negatively. One that might be now applied to the SQA is "divide and conquer". Think of the database as an entity serving several groups (parents, pupils, employers, admissions officers) and apply the principles of subsidiarity and segmentation so that datasets reflect what is needed for any specific purpose - given a Higher in a domain, the employer can assume that literacy, numeracy and other core skills are part of the deal. Locally gathered data can be held there and accessed as required if compatible systems are used across schools.
This year's SQA certificate contained a mass of confusing data. What research was done into what users - pupils, parents, employers, universities - wanted? Most probably want a clear, separate statement of final grades at Standard grade, Intermediate 2 and Higher. The other data simply clog the record.
A second useful pattern would be the "strategy team mix" - the way attention is given to those closely involved in the system strategy process to ensure an appropriate mix of knowledge about working practices, authority to make things happen, and interest in the project itself, as well as in seeing it successfully completed. With the SQA, we can question whether any of these facets was considered.
Furthermore, ensure that the steering or supervisory board has people who know about systems in all their complexity, and who understand organisation change and the role of systems in that. We have learned about the background of many of those involved only after their resignations.
From a "patterns" perspective, the SQA story can be salvaged as a rich resource for social learning. But the story is not over and indeed the shambles may be repeated next summer. It is not too late to learn the lessons. The question is whether those in charge are prepared to undertake that learning and to do something about it.
Dr Elisabeth Davenport is professor of information management at the school of computing of Napier University. Dr Keith Horton is lecturer in the school of computing, and he has acted as systems strategy consultant to many public and private-sector organisations.