Analysis helps improve the student experience
At AQA we are always working to improve what we do.
No-one wants students to leave an exam hall feeling they weren’t given the opportunity to shine.
Which is why we work hard to ensure papers are as well constructed as possible.
Key to understanding the design quality of our maths GCSE exam papers is looking at how well each question is answered.
Our assessment designers broadly know the degree of difficulty of each item and therefore what proportion of students should be able to answer them.
The more straightforward questions should be returning higher marks while the most complex ones, which are designed to separate the best students from the rest, will have fewer successful answers.
If, in our post exam analysis we see questions where students have not performed in line with our expectations, we need to take a more nuanced look at how that item has been written and looks on the page.
How we design our maths GCSEs
When we set about improving our maths GCSE papers, we examined each question’s language, visual design, the context it was set in and where it sat within the paper.
Clear concise questions and clean crisp visuals are vital
Language was a key part of the redesign. Wording must never hinder understanding.
At the forefront of our thinking was that we are testing students’ maths not their literacy.
An irrelevant word or sentence could hamper someone’s best performance.
For example, in our specifications, students should understand the word ‘integer’, so that needs only be tested once. Anyone who doesn’t know that word will be penalised several times if it is loaded into other items.
We stripped out everything that was redundant and repetitive so the questions are simply phrased and diagrams clearer.
Removing redundant or repetitive words allows students to focus on showing their best maths
Our research also revealed that simple visual adjustments can reduce the cognitive load on students.
Taking away everything that is not needed or distracts the eye allows them to focus on producing their best work.
As a result of that research, we created more white space around items and diagrams so there is more space to make notes and we also made all the working out lines fainter.
It may not seem like a big change but it will have helped the students.
Do all maths exam questions need a context?
This decluttering was applied to the use of context in questions.
Putting maths questions into a context can aid understanding as long as it is clear and not culturally dependent.
We could use sapphires and carats as a way of testing someone’s arithmetic. But that could be very confusing if they did not know much about gem stones or how they are measured.
Money, however, is a great way to contextualise maths because it is cross-cultural.
Some questions benefit from being set in a context but that has to be appropriate
Our designers now know that if an item does not require context it should be removed.
If context is being used, they ensure it is useful and appropriate.
How we order questions improves student experience
A few years ago, teachers started raising concerns that Multiple Choice Questions (MCQs) used at the start of papers were too difficult.
We had thought MCQs, which have to contain plausible but incorrect options to be valid exam items, were a good opener.
But after looking into it we decided putting that type of question in front of anxious students at the start of a high stakes test was not the best approach.
They could become disheartened and disengaged if they got wrong answers early on, which is not what we wanted.
We also found that using them early in a paper disrupted the smooth ramping up of difficulty through the paper.
So, now we look at the placement of each item and no longer use MCQs at the start of an exam.
Our redesign work paid off and everyone can benefit
We are really pleased with the outcome of our changes and how students performed in the papers.
We will, of course, continue to refine the maths papers, while still producing consistent and reliable assessments.
Not only that, the lessons we’ve learned are being shared across AQA so students of all subjects can benefit.
Read more:
What comes first, the question or the mark scheme?
How does a question paper get made?
Four key ingredients for a high quality assessment
