The class is rapt. Every pupil is engaged, their attention fully focused on the colourful interactive display being projected at the front of the room.

This is the first time the teacher has shown them the new literacy app that everyone has been raving about, and, so far, it seems to be doing the trick.

But will the same activity still hold these pupils’ focus the following week? How about the week after that? The children might be paying attention now, but whether their increased levels of focus will translate into noticeable gains in their reading ability further down the line is a different matter.

And if, ultimately, their attainment doesn’t improve, who is to blame for that? Is it the fault of the app developer? Or perhaps the problem lies with the schools and how the programme was implemented.

A new guidance report from the Education Endowment Foundation, released today, suggests that the latter could be a significant factor in the poor reputation and performance of edtech in schools. It’s trendy to simply say that edtech doesn’t work, and, after reviewing the evidence from multiple studies, the report authors found plenty of cases where “technology has fallen short”. But there were also enough examples of potential impact to suggest that edtech can work, and that part of the reason it doesn’t lies with schools as much as with tech developers.

The findings are drawn from an EEF-commissioned review of meta-analyses published since 2012 and an earlier review conducted by Durham University, which looked at meta-analyses published between 1990 and 2012, as well as the results of individual EEF-funded projects with technology inputs that were published before December 2018.

The report focuses primarily on “applications of technology that aim to improve learning directly” (so it does not look at technology used for administration purposes or to track pupil data, for example), and takes a broad definition of what this technology includes; the evidence reviewed covers a wide range of products and initiatives, from hardware and software to online games and tutoring programmes.

This breadth proved challenging, admits Eleanor Stringer, head of programmes at the EEF and one of the report’s authors. “The research in this area is hard to draw strong conclusions from because of the variety of ways in which technology is used and the huge variety in the outcomes,” she says.

As with much of the research around the impact of screen time on teens, lumping everything together as “tech” is problematic.

For example, trying to teach science through a virtual-reality headset is very different to training kids on times tables using an app on their phone; likewise, utilising video tutorials for homework differs from using a visualiser to model best practice answers. Yet we can still get some useful insights from the evidence base, argues Stringer. “What we think is that the evidence shows that technology can be effective, but there’s a big italic emphasis on the word ‘can’,” she explains.

So how can we ensure that tech has the potential to have an impact on learning?

Stringer says edtech developers have a big role to play in making products more in tune with best practice in teaching. “If you’re developing a phonics or spelling app, you shouldn’t be starting from what you know about how to code an app and how to make it snazzy - you should be starting from what we know about how kids learn literacy and how this will be helpful,” she says.

Alison Clark-Wilson, principal research lead of the EDUCATE Project at UCL Institute of Education, agrees that edtech developers have a responsibility to ensure that the products they are marketing to schools really deliver on teaching and learning.

“We should always be cautious and not be afraid to ask the edtech developers about how they know that their technology does what it has been designed to do,” Clark-Wilson says. “We all need to know what is going on beneath the edtech bonnet - there are too many shiny Lamborghinis powered by lawnmower engines out there and it is hard for teachers, lecturers, learners and parents to know which questions to ask to reveal this.”

The way in which products are marketed to schools can also be problematic, Stringer argues. This is something that sets edtech apart from other resources and equipment, such as textbooks.

“The developers seem to sell tech more directly to schools. They make it seem more snazzy and exciting and pitch ideas to schools that might not be right for them,” she says.

This can lead to teachers being “oversold” on what technology can do and how easy it will be to use.

So, better products built with more education input is our starting point. Cleaning up marketing is the next move. But school leaders and teachers have a big role to play, too.

According to Stringer, realising the potential of technology depends on two things: whether the tech is being used in an “evidence-based way to improve teaching and learning” and whether it is being supported by “good implementation”, following a process of thoughtful planning and review, as laid out in the EEF’s guidance report Putting Evidence to Work: a school’s guide to implementation.

In theory, this shouldn’t be a problem, Stringer points out. After all, school leaders are already aware of the need to put pedagogy at the heart of teaching and to ensure that any new approaches are evidence-informed and well-implemented.

But, she adds, when it comes to edtech, these lessons can sometimes be forgotten. “I think the lessons apply to most other areas of educational intervention and approaches as well, but we just think that it’s worth hammering them home in relation to the use of technology in the classroom, because we think they’re often overlooked,” she says.

The report is not blaming leaders and teachers for the failings of technology, but it does set out actions that school leaders can take to make sure that their schools get the most out of edtech - and that they are spending their limited budgets wisely.

“Our aim is to give schools better evidence and a better framework for thinking through their decisions about introducing or using a new technology,” Stringer says.

The report sets out clear recommendations for successfully implementing edtech and lists three key areas in which there is promising evidence of the benefits: explanations and modelling; pupil practice; and assessment and feedback (see box, opposite).

But whichever area of pedagogy a school chooses to focus on, the starting point in considering the technology solutions available should always be the same.

1. Define a learning need first

“Technology is much more likely to improve learning if it is introduced in response to an identified need,” the report states.

Without defining a clear need, schools run the risk of edtech becoming “a solution in search of a problem”.

This is a statement that rings true for Claire Lotriet, assistant headteacher teaching and learning at Henwick Primary School in south-east London, and a former Tes edtech columnist. She recalls implementing one new edtech tool in her classroom, only to realise she was using it to solve a non-existent issue. “I felt like it was a solution to a problem I didn’t really have,” she says. “It was more of a faff and really there was no issue to solve in the first place.

“I’d say edtech can certainly fall prey to that at times. Just because it might look good or seem like a great idea, in reality it might not improve a situation or make it more manageable, and can even make it worse.”

School leaders can avoid repeating Lotriet’s mistake, the report advises, by first clearly defining the problem to be solved. This is a process that can be linked to wider school planning, such as departmental development proposals or a pupil-premium strategy.

2. Be clear on the rationale for how tech will improve learning

Only once leaders have identified and clearly defined the learning need should they consider why technology might provide an effective solution to this issue and begin looking at the available options.

To help with this stage, the authors of the report suggest that school leaders should ask themselves three key questions:

• How tightly does the technology link to the problem that you have identified? For example, does the reading programme you are considering focus on the aspect of reading (decoding, fluency, comprehension) your children are struggling most with?
• How will it change teaching? For example, will teachers find it easier to explain, model, assess progress or provide feedback?
• How will it improve learning? For example, will children work more efficiently? Will the technology help them to learn for longer, more productively?

Answering these questions will help leaders to make sure that their choices are driven by pedagogy, and that they are not simply investing in tech for the sake of it.

James Siddle, headteacher at St Margaret’s CE Primary School in Lincolnshire, and head of Kyra Research School in the county, was a member of the advisory board for the EEF report. He believes that posing these questions is a particularly important step.

“I was recently working with a group of teachers and headteachers and some of the heads said that they were hoping to invest in a new ICT suite and did I think it was a good idea? For me, that was the wrong start point,” he says.

“Rather than thinking about ‘how can we build up our digital infrastructure?’, people [should] say ‘what do we need it for? What do we want to achieve? What can digital technology offer that something else might not offer?’ I think that’s a very important consideration.”

To help answer these questions, school leaders will need to turn to the evidence around specific tools and interventions.

Rose Luckin, professor of learner-centred design at UCL Institute of Education, says that this is crucial. “We should not put our faith in any educational technology for which there is no evidence that it has an impact on learning and/or teaching that is positive and constructive,” she says. “It is extremely important that all educators demand impact evidence before engaging in the purchase of any technology for their classrooms.”

3. Consider whether technology will supplement, enhance or replace existing teaching

Paying close attention to the evidence is important for the next stage of the process, too. Once schools have defined the learning need and identified a promising tech solution, the report advises that there are further decisions to make about how the technology will be used and if it will “supplement, enhance or replace” what teachers already do.

This will involve determining not just whether the technology will have an impact, but whether it will have “a greater positive impact than the alternative”.

However, Lee Parkinson - a teacher and the founder of the Mr P ICT website, which provides CPD to help teachers use technology more effectively - believes that teachers should always avoid thinking of edtech as a “replacement”.

“It isn’t about looking to replace what we are already doing,” he says. “For example, when I receive messages from teachers on Twitter asking if I know an app that can help children measure length, my answer is: ‘Yes, a ruler’. We need to look at what we’re already doing and seek ways in which technology can take us beyond that, to enhance and transform, not just substitute.”

The evidence to date does indeed suggest that technology is most effective when it is used to supplement or enhance regular teaching, rather than replace it, the report states. But this relies on it being carefully integrated into lessons by staff who have been trained to use it and to support children to use it effectively (for more on that, see below).

4. Prepare for implementation

Once the problem, potential solution and rationale are clear, it is time for school leaders to consider exactly how they will make the new technology solution work in practice.

This involves determining whether the school has the capacity to implement the approach effectively, including with “appropriate training and support for teachers”. Lack of training, Parkinson believes, is one of the biggest “obstacles” to technology being used well in schools.

“The mistake schools make is they buy technology, let’s say a class set of iPads, wheel them into a classroom and expect teachers to simply get going with it,” he says.

Instead, he continues, if school leaders want technology to have an impact, they need to “start with their staff” and make time and space for adequate training.

Steve Higgins, a professor of education at Durham University, agrees with Parkinson; he feels that training is a part of the technology puzzle that schools often miss.

“As a general rule of thumb, we spend more on the technology and the software than we do on thinking about people’s immediate training needs and then what I think of as their professional development needs,” he says. This, he argues, is a mistake.

There are two sets of skills that teachers need if they are going to be able to use technology effectively.

“Initially, you need to be able to use the technology fluently at a skills level - like an interactive whiteboard or a new interactive TV screen for presentation - but then once you’ve learned the skills of using that fluently, you then need to work out what it’s good at in the classroom,” Higgins says.

“So there’s a period of time where it needs some development work in school to work out how it complements what you’re already skilful at doing, and it’s that bit, I think, that it’s hard to find the time to justify.”

The EEF report similarly stresses the importance of training teachers in how to use technology properly, and includes this as part of the final step in its guidance.

Some of the other questions that schools and teachers should consider as part of this stage in the process include: is the right equipment available? What initial support will be required to introduce pupils to the technology being used? How will delivery of the approach be monitored to ensure that it is used as intended? Is there an initial and ongoing financial cost? Is this both affordable and justifiable?

Cost, of course, will be a huge factor for many schools, and is a major reason why more guidance is needed in this area in the first place - especially as school budgets are dwindling across the country. The financial implications alone make it worth taking the time to ensure that technology is only introduced when there is clear evidence that it can be implemented effectively and that it is better than the “analogue” alternative.

“These things cost a lot of money,” Siddle says. “They’re a big investment for schools, so getting it right is really important. And where’s the trade-off there? Are we doing something that we can achieve somewhere else for a lower cost?”

What should not be enforced, say the majority of researchers, is a general ban on any technology in the classroom.

“Technology is now embedded in the wider world and it would be rather odd if it wasn’t used in schools, because there would be a big disconnect between pupils’ and teachers’ experience of technology inside and outside school,” Higgins says.

Rather, what this report aims to stress is that tech can and should be part of teaching, but only if the product, the reasoning and the implementation are all right. And some of that comes down to the attitude of schools to force the necessary changes.

“The idea that there’s ever going to be a killer app or a killer device that’s going to transform all of teaching and learning, I just don’t think that’s the case. Teaching and learning is too complex,” says Higgins. “So it’s those choices about when and how to use [technology] in terms of having really clear learning objectives in relation to where pupils are, in terms of the curriculum that you’re teaching: that’s the kind of thing that teachers need to think through.”

Helen Amass is deputy commissioning editor at Tes

This article originally appeared in the 29 March 2019 issue under the headline “Don’t tech it for granted”