The A-level exams were cancelled in March in response to the spread of COVID-19. Subject grades in the UK were originally awarded according to a teacher-assessed ranking of students at each centre and the expected grade distribution of the centre (e.g. school). The expected centre grade distribution has been based on the past performance of the centre, adjusted for the GCSE results of the current intake.
The process has met the criteria set by parliament, but bemused and angered many students who had not attained their predicted grades and found that they have been subject to a mysterious ‘algorithm’ which was informed by the performance of other students who happened to have previously attended their school and differed if more than five students had taken the subject at their centre and again if more than 15 students had taken the subject.
The tension between achieving the required top-down distribution from bottom-up inputs, led to such hostility, that the whole system was abandoned in favour of teacher predictions alone.
The Government’s response for England and Wales had been that the objective of the exercise has been met: grades in 2020 match the distribution in previous years, universities can allocate their places as before, the system can move forward, and students can get on with their lives.
As a parent, I thought that the system had been cruel. As a commercial real estate researcher, I found unlikely parallels to modelling portfolio performance.
Like the distribution of A-level grades, despite the wide variation, the performance across individual portfolios follows a predictable and fairly narrow distribution. In property investment terms, the power of diversification removes the idiosyncratic property level variations and leaves (mostly) market return (it is hard to diversify the specific property risk entirely when buying such large assets).
Top-down property modelling (forecasting market return) therefore tends to ignore specific property influences and, quite rightly, market participants often comment that individual property returns can differ significantly from the forecast market average.
Ofqual were charged with creating a similar distribution of A-level grades to previous years by distributing the grades to students via an algorithm. This is similar to asking top-down property forecasters to predict individual property performance around the market average. In my experience, this exercise can end in acrimony, with objections mirroring those of students receiving their 2020 A-level grades, unless the drivers of model and the data to calibrate the model are fully agreed.
