AI and Scotland’s assessment reform

Artificial intelligence could have huge implications for qualifications, assessment and more, say Richard Holme and Rick Grammatica
10th December 2023, 10:30am

Share

AI and Scotland’s assessment reform

https://www.tes.com/magazine/analysis/general/artificial-intelligence-scotland-assessment-reform
Digital library

As memories of the Covid-19 pandemic slowly fade, the latest era-defining issue in education is artificial intelligence. Everyone from government leaders to social media influencers are weighing in with opinions, and AI technology is changing fast - it can all feel a bit bewildering at times.

In Scotland, the independent report from the International Council of Education Advisers, commissioned by the Scottish government, advised that educators be prepared for “a shift in the education paradigm and to ask fundamental questions” and warned that we must not be “beguiled by the excitement that AI is currently generating”. 

Artificial Intelligence has been part of our everyday lives for decades, powering search engines, streaming services and smart speakers - most of which teachers will have been using without even realising.

Seeing AI as a digital librarian

The big change, at the end of 2022, was the release of ChatGPT, allowing anyone with the internet to generate human-like content. Rapid developments have kept AI in our newsfeeds and fuel fears of advanced systems (known as artificial general AI, or AGI) that can replace human workers or even endanger humanity. 

ChatGPT and other generative AI tools use large language models (LLMs) trained on massive amounts of data. This information is then stored in an interconnected neural network, similar to the human brain. When we “speak” to the AI - known as “prompting” - it refers to this network to predict the best answer.

One way to understand this is to imagine an LLM as a digital librarian, where you can ask anything related to its catalogue. For example, ask it to generate a sentence in the style of Shakespeare and it accesses its vast “memory” of Shakespearean literature to craft a sentence that mimics the Bard.

For a Higher English teacher, this may simultaneously spark excitement at the possibilities and fuel fear of potential misuse.

The application of AI for assessment is particularly relevant to teachers and educators in Scotland. The recent Hayward review, notably, called for carefully planned and resourced reform of senior-phase exams.

This would include the adoption of a Scottish Diploma of Achievement (SDA) to incorporate project-based learning - and an emphasis on coursework, where fears around generative AI will likely be raised.

Another recommendation was to retain external exams, albeit with a review to reduce the likelihood of question prediction and reliance on rehearsed answers.

AI has the potential to contribute to the evolution of traditional assessment approaches, as students can use it to answer questions and write essays. Suggested solutions have included more controlled, invigilated exams.

Or alternatively, as Matt Glanville, head of assessment principles and practice for the International Baccalaureate, has argued, we must “adapt and transform our educational programmes and assessment practices so that students can use these new AI tools ethically and effectively”.

The benefits of AI in education

A major advantage of AI is that students could have formative and summative assessment tasks tailored for any level or topic. Personalised AI chatbots can use a dialogue-based approach, encouraging learners to verbalise their thinking. This provides rich data, helping teachers assess each student’s learning process, not just their output.

AI can also help teachers manage the learning journey, providing appropriately spaced repetition based on each student’s progress.

As chatbots become more advanced they could be used as one-to-one tutors, adapting to the specific needs and level of each student, offering tailored feedback and guiding learners through problem-solving processes.

This could reduce situations in which learners feel lost in the learning experience or get the answers from friends, family members or the internet. AI tutors are particularly useful for learners during independent study - such as the project-learning element of the proposed SDA - and in situations in which a teacher isn’t readily available, such as in large classes, at home or in remote communities, including rural areas of Scotland.

A final benefit is the ability to curate authentic, contextualised assessment opportunities. AI can create customisable and realistic virtual environments, giving students opportunities to practise in a realistic and psychologically safe way. AI simulations or avatars can react in a lifelike way and adapt to responses.

For example, students could practise speaking languages with an AI avatar in a natural and safe way. And of course, many young people have already experienced AI in virtual environments such as video games. Clearly, AI may help revolutionise assessment in ways that weren’t realistically possible before.

Can chatbots hallucinate?

It is important, though, to acknowledge some of the challenges. These include possible bias as AI is “trained” on data, so those deciding on what to select, and reject, will have ultimate power over this knowledge base - although the same could be said of the architects of existing curricula and assessments.

Another concern is that chatbots can “hallucinate” - highlighting wrong information or harmful content. There’s also the question of protecting students’ data. 

Although AI will affect us in all walks of life, from an education perspective we need to bring the focus back to the learner. Scottish education - see policy guidance from the General Teaching Council for Scotland, Education Scotland and Curriculum for Excellence - all put the learner at the centre. So, perhaps we should be considering learners’ views of AI and LLMs.

Ultimately, whatever we do, the pupils of today are going to have to live with the impact of AI in their everyday lives.

Richard Holme is reader and lecturer in education studies at the University of Dundee. Rick Grammatica is a teacher, trainer and learning designer who operates the AI Learning Designer website, which supports educators in the use of AI

You need a Tes subscription to read this article

Subscribe now to read this article and get other subscriber-only content:

  • Unlimited access to all Tes magazine content
  • Exclusive subscriber-only stories
  • Award-winning email newsletters

Already a subscriber? Log in

You need a subscription to read this article

Subscribe now to read this article and get other subscriber-only content, including:

  • Unlimited access to all Tes magazine content
  • Exclusive subscriber-only stories
  • Award-winning email newsletters

topics in this article

Recent
Most read
Most shared