Why the International Baccalaureate grading system is flawed

Students in England will be receiving their A-level grades shortly, and I hope the process is less fraught than what has happened with the International Baccalaureate. I am reasonably confident it will be, as, although this year has made assessment of students very difficult, the A- level boards seem to have chosen a model that is reasonable and is trusting in the professionalism of schools, teachers and administrators. Unfortunately, this is something that the IB has failed to do. 

In the IB system, there are many angry educators upset about the results awarded to their students and their centres. Undoubtedly, there will also be many students and parents distressed that they have received grades much lower than they were expecting and will be rejected from the university they were to attend. I can only empathise with an outstanding student who was waiting to go to the London School of Economics and predicted 40-plus points to find that she has received 6 rather than 7 grades and now will have her future severely affected - in her eyes arbitrarily ripped away.

Although, at my centre in China, we have not been too severely impacted, this failure is not surprising as the process IB chose to use was deeply flawed. I am writing not only because I am angry and wish to inform other interested parties, but also because I contacted the International Baccalaureate Organisation on many occasions about the problems with the process and never received an acceptable response. So let me outline the issues with the process.

Using the internal assessments (IAs) as an important assessment tool was always going to be problematic for the following reasons:

First, the Covid-19 lockdown affected different countries differently. In China, we went into lockdown in February, which meant that my students have not had anything other than online support since December. However, other countries didn’t go into lockdown until just before the Easter holidays, and so they had a normal or almost normal IA programme. How was the IBO going to account for this difference? I received no adequate response other than that it couldn’t allow for the specific problems of individual students. It is now apparent that the IBO made no adjustments for this problem. However, this wasn’t a problem of individual, specific students but a policy decision made by the IBO. A decision that IBO knew would favour some regions of the world and some ethnic groups over others. An awful decision.

The effect of relying excessively on internal assessment is hard to determine; at my centre, visual arts was severely affected. What would one expect if students had no access to the art studio or equipment for the whole of the second semester of Year 13, when they would typically have both developed their skill level and been most productive? Fortunately, at my centre, most of the science subjects finished their experimental work before we went into lockdown and the emphasis we put on the IAs paid off - we were lucky. Nevertheless, I know of other schools that have had many students fail to get into university because of problems with science grades. Problems are not necessarily restricted to underperformance with the internal assessment

Unfortunately, some schools, especially those that went into lockdown early, have the second problem that undermines the value of IA. Some wealthy parents of privileged students will have hired personal tutors - can we be sure that the work presented is the students’ work rather than reflecting the help given by the private tutor? Unfortunately, with schools in lockdown, with staff unable to meet with students in the usual way, this is likely to have been a significant and unquantifiable problem in some centres.

The next problem is, are the IAs a useful assessment tool? Do they accurately reflect student performance? From the data I have available to me, I can demonstrate that, in many subjects, the IAs have historically been the worst indicator of students’ final grades and that teachers’ predicted grades have been a better indicator. I’ve been teaching in the IB system since 1994, and most IB teachers know how inconsistent the moderation of internal assessment by the IBO is. Nonetheless, the IBO assures us that it has excellent statistical tools at its disposal and this is not a problem. Why, then, has the IBO failed to communicate the algorithm it is using openly? 

The most popular thread on the forum that IBO has created for IB Diploma teachers is: “Can anybody explain to us the insane algorithm for the final grades?”

Even though the IBO has been less than forthright about its use of statistics and the algorithm it is using to produce grades, it has released some information about how Further Maths grades have been created. 

From the IB we get: “This session Further Mathematics grades are being calculated based on each student’s other subject grades. This means that if the other subject grades change as a result of EURs [re-marking by the IBO], the grade awarded for Further Mathematics could also change.”

I had assumed the IB would be using the HL Maths Internal Assessment (as this seemed at least sane), but no, if your Further Maths students were poor in the Language B or Literature class, this would be factored into the Further Maths grade. It is possible that when the window for EUR closes, your Further Maths students’ grades may go up -- presumably based on how well students did in Sesotho Literature or any other subject in other countries.

The IBO also warns if you appeal the result, the IB will ask the computer if another grade should be awarded (again based on the changing fortunes of students studying physical education or any other subject). If the computer tells them to lower the grade, they will.

It is ironic if your Further Maths student was a “risk-taker” (a learner characteristic the IB claims to value) and chose to take two first languages rather than a first and a second language and thus received a lower grade. The IBO will doubly punish them for this foolhardiness. First, by giving them a lower language score and then by taking points away from Further Maths. It is doubly ironic that the IBO has a compulsory course for students called “Theory of Knowledge”. Yet it fails to understand that correlation does not equal causation. Because there is a correlation between IB Further Maths grades and IB Diploma grades, you cannot merely extrapolate in the other direction by assuming a causal relationship. The idea that an algorithm might have biases and values baked into the assumptions the algorithm was based on is presumably well beyond the understanding of IB administrators.

This shambles wasn’t necessary; I’m sure that many in the IB system are looking enviously at the process Cambridge has put in place. Teachers give predicted grades, rank the students and gather evidence. If Cambridge sees the grades are not consistent with the historical performance, they can ask for proof and, if necessary, moderate using the ranking. Certainly not perfect, but better, more ethical, more internationally minded and more transparent than the IBO solution.

Why am I writing this? Indeed, I feel disrespected, I feel students have been hurt, I feel the IBO has behaved badly. But the reason is this debacle could have been avoided if the IBO had communicated with its educators and not been so driven by its hubris. Perhaps the IBO will learn from this experience, but I’m not very optimistic.

John Humphreys
Centre principal, Dipont Education, China