Well designed accountability measures can help students and schools grow
In 2013, the then Education Secretary Michael Gove announced he was reforming GCSEs to equip more young people with the knowledge and skills needed to succeed in their educational and work journey.
His plans for a successful transition included replacing the existing schools accountability measure.
Out went the 5ACEM scale which counted the number of pupils achieving grade C or better in five GCSEs, including maths and English.
In its place stood a combination of two new measures - Progress 8 and Attainment 8 - to assess the quality of secondary school teaching.
They came into force in 2016, marking the most important change to the schools accountability system since performance tables were first published in 1992.
But what was so wrong with 5ACEM that it needed to be completely ditched instead of simply adjusted?
Well, the main fault with 5ACEM was that because it measured the numbers getting grade C and above, schools had the perverse incentive of focusing on those pupils on the grade C/D borderline.
Pushing them over the boundary to get a grade C counted towards league table positions whereas helping someone move from an E to a D or a B to an A had no impact on that.
There were a couple of other perverse incentives linked to 5ACEM.
Firstly, it was said to encourage schools to choose exams that were seen as easy to pass rather than what were best for the students.
Secondly, because only five GCSEs (including maths and English) counted towards league table purposes, some schools narrowed their focus to just five subjects instead of the broader curriculum.
Another concern was that as it did not show how much students had improved against expectations, teachers’ hard work helping them produce better performances went unrecognised.
That meant good schools serving more disadvantaged communities were disproportionately hit by negative inspections.
At the same time, poorly performing schools which benefited from high-attaining intakes or were focused on getting lots of pupils over the C/D grade boundary fell under the inspectors’ radar.
The bottom line was that 5ACEM said more about the types of pupils taught in different schools than the effectiveness of education provided by those schools.
Progress 8 was designed to address those issues.
Before we get into that, here’s a quick explainer of how Progress 8 and Attainment 8 work.
Attainment 8 measures a student’s attainment across eight KS4 subjects - maths and English, three EBacc subjects and three other qualifications. The figure is the sum of the student’s results in those eight subjects.
Right, now the calculation part.
The Progress 8 score is the student’s Attainment 8 score number minus their Estimated Attainment 8 score (created at KS2), divided by 10.
A negative number signifies decline, a positive one shows progress.
A school’s Progress 8 is the mean average of all its students.
Failing to hit a threshold target on Progress 8 triggers an Ofsted inspection.
Since its introduction, educationalists have broadly seen Progress 8 as a fairer measure and having a positive effect as it incentivises teachers to help all students improve no matter where they are on the achievement spectrum.
Proof of this is shown in one study which found that once 5ACEM was removed, students outside the C/D borderline group seemed to make more relative progress compared to those at the borderline.
Clearly that is a positive for Progress 8 but it has not been all plain sailing.
It quickly emerged that a very small number of low achievers can distort a school’s Progress 8 results. This generally affects schools in the most disadvantaged areas.
It led to allegations of off-rolling by schools, where lower-performing pupils were removed from the register before sitting GCSEs so they did not impact on league table positions.
As a result, two sets of league tables are now published with one adjusted for the statistical impact of the small number of negative outliers.
The overall issue of off-rolling pupils is being looked at separately by the DfE.
Some also criticise it saying that because it only uses SATs to differentiate between pupils it is biased against schools serving more disadvantaged children.
The counter-argument is that adjusting for characteristics such as gender, ethnicity and free school meals would not necessarily improve measures of school effectiveness.
But, the main criticism of Progress 8 is that it incentivises Ebacc subjects at the expense of the non-Ebacc, especially the creative and vocational.
Although creative and vocational subjects are in the Progress 8 third grouping from which students pick their GCSEs, the way attainment is measured, it is weighted in favour of the traditional academic subjects of the Ebacc.
Not only that, quite a few vocational qualifications that could have gone in the third group have been defunded and don’t exist anymore. That means more traditional GCSEs are being taken instead.
All this has led to concerns that post-16 students are inadequately prepared for further and higher education and the workplace.
In our next blog we will be looking at whether Progress 8 can be adjusted to address these concerns and if so, how?
Read More: Levers of change: How policymakers can shape the system
Greater Manchester's MBacc: What digital skills education may look like
