How could leading funders, a major university, project research teams, education authorities, headteachers and child protection officials have allowed thousands of questionnaire surveys into Scottish schools, without any support for pupils who were in danger or distress? That is perhaps the biggest question raised by projects such as the Edinburgh Study of Youth Transitions, and Crime and Communities that Care (CTC). Both have been part-funded by the Scottish Executive, and the transitions study has also been funded by the Economic and Social Research Council.

Both are reputable projects whose laudable aims and dedicated staff are not in question. Both want to help young people avoid offending and antisocial behaviour. Yet, as revealed in The TES Scotland last week, they gave many thousands of questionnaires to school pupils on sensitive topics, including crimes committed against children, without basic supports such as contact numbers of helpful agencies - let alone agreed policies for helping pupils whose answers proved disturbing.

The Edinburgh transitions study has not even intervened when pupils say they have made a serious attempt to end their lives in the past year, or that an adult has “really hurt” them. The transitions study probably holds a unique cumulative record of distressed children from the age of 11: they are now about 18. It is itself very distressing if no action has been taken to protect them. The CTC does not intervene, even with pupils who say adults at home try to hurt them or each other. In one CTC neighbourhood alone, 100 children gave this response.

Answers to our opening question are salutary, if unpalatable. First, youth crime topics lend themselves to rash enthusiasms. Second, for most parties, the welfare and safety of individual children was not the priority: something else came first. Third, child protection officials either failed to acquaint themselves properly with the projects, or took a narrow, rigid view of what “child protection” was about, or both.

Focusing on problems young people cause to others is always popular, politically and publicly: antisocial behaviour, youth crime and delinquency, drug misuse, teenage pregnancy, truancy. Research projects appear urgent and worthwhile to parents, teachers, funders and most authorities, so that consent is sometimes given without considered thought.

When the emphasis is on problems children cause, then the problems that adults cause children and the welfare of those children fall (however unintentionally) into second place. Yet, because of known correlations between child abuse, neglect and problem behaviours, those are precisely the surveys where it is most important to be aware that some respondents may need urgent support and protection.

The influence of right-wing individualistic American thought about delinquency and its influences, detectable in both these project questionnaires, is also likely to have made the surveys less threatening to adults who were considering giving permission to go ahead. It appears more important for questionnaires to ask whether parents supervise kids’

homework than whether they mistreat kids at home. But which really has the greater influence on problem behaviours?

Total confidentiality and anonymity became the overriding goal for both projects, pushing everything else into second place, for two reasons. The priority was not individual welfare but accurate information about patterns of behaviour among young people generally, who had to be persuaded to tell the truth about their dodgy behaviour. And the main worry of headteachers, parents and councillors when considering co-operation was not the welfare of any distressed pupils. It was publication of anything that gave their school or neighbourhood a bad name.

It is unclear what negotiations the CTC had with local authority child protection officials. The Edinburgh transitions study, however, arranged not to ask direct questions on child abuse or neglect, since the researchers would have had to report the matter and the children would lose their anonymity. But instead of meeting child protection guidelines, this sidesteps them. Not asking directly doesn’t mean abuse is absent.

Why then did Edinburgh protection officials agree to this compromise? And why did they fail to perceive that it is not just direct responses about abuse which are child protection issues? These include responses about self-harm and suicide attempts, early drink and drug use, early sexual experience, prolonged sadness or absence of friends. They involve questions about crimes against children, such as assault with a weapon. Any experienced child protection worker could have detected, from certain clusters of responses, the minority who gave cause for concern and needed speedy offers of help.

Since abuse and neglect have significant influence on offending and problem behaviours, the study was also missing much essential data, and some questions could only be “half-asked”. For instance, parental forms of discipline listed in the transitions study could not include physical punishment.

I believe that, belatedly, the two projects must talk urgently with national and local child protection agencies about their data, and about getting help to young people whose responses cry out a need for it. What, for instance, will be done now for those teenagers who told the transitions study that their first sexual experience came as young as eight or 11?

Of course, there are genuine dilemmas. We need information about young people’s behaviour en masse, and we need them to feel safe enough to answer truthfully. A blunt, defensive reaction by child protection officials that all children giving certain responses must now be reported will simply shut most of them up. All sides need to compromise by exploring alternatives such as contact sheets and helplines; a means for researchers to direct distressed young people discreetly to professional help; the constructive use of guidance staff; and the active involvement of protective agencies and children’s advocacy agencies at design and analysis stages of research projects.

Those who fund these studies need to include ethical demands for child protection safeguards. So, surely, must all Scottish education authorities and headteachers, before they allow any research project to unleash surveys - however well-intentioned - on their pupils.

Dr Sarah Nelson is a researcher and writer on child protection, child sexual abuse and the effects of abuse on adult survivors.