For teachers creating formative tests for use in the classroom and awarding organisations producing national exams for thousands of students, the principles of good assessment are the same. We have tried to distill good assessment into the four key ingredients for creating high-quality academic assessments.
Be clear what is being assessed
The first and most important thing is to be as clear as possible about exactly what is being assessed. Does the assessment want to find out what a student knows or is it trying to explore a deeper level of understanding? Or perhaps the assessment focuses on the skills students have developed, for example the ability to construct persuasive arguments or analyse data effectively. Maybe a number of things are being assessed at the same time.
It’s surprisingly easy to lose focus right off the bat by not pinning down exactly what is actually being assesses. One way that exam boards do this is in terms of ‘assessment objectives’ – the two, three or four key things that any given qualification is all about. All questions should clearly trace back to one (or more) of these.
Use the right type of ‘item’
Once the focus is clear, the next thing to consider is the best way to get there. In the UK we use a variety of ‘item types’, ranging from multiple choice questions to open-ended coursework briefs.
There are rules of thumb connecting item types to areas of focus: open-ended essay questions that invite a wide range of responses are effective in assessing the ability to discuss and evaluate; more constrained ‘Describe’ or ‘Explain’ type questions are used to assess knowledge and understanding; and questions set in real-world contexts are often used to assess application.
In short, the item type must create the best possible opportunity for students to demonstrate whatever is being assessed.
Avoid ambiguity in the question
With a ‘what’ and a ‘how’ established, the next consideration is the specific task being set. In particular, the language used in the question should be as clear as possible, containing all (and preferably only) the information needed to complete the task.
The aim of the question is to get students to do exactly, and only, what is intended for them to do. But language is a slippery customer and ambiguity is always lurking around the corner. The word ‘not’ is a regular offender, with questions easily misinterpreted because students expect questions to be phrased in the positive and often a question still makes perfect sense if the ‘not’ is overlooked. For example, ‘Which of the following is not a cause of global warming?’ could be easily misread as ‘Which of the following is a cause of global warming?’ - it makes sense as a question but it is asking an entirely different thing.
If there is a possibility that a word, phrase or whole sentence could be interpreted in an alternative way, it probably will be. Students are up against it in the heat and stress of an exam, so it’s crucial that each question has a single, clear meaning.
Spare a thought for the markers
Finally, don’t forget about the mark scheme. While the question tells the student what to do, the mark scheme is the tool by which markers determine how well students did it. This brings us full circle, back to the focus of what is being assessed.
A mark scheme must be fully aligned with the question – only giving credit for exactly what the question prompts students to do. Mismatch between a mark scheme and a question is always a danger, particularly around choice of ‘command word’. For example, imagine a question that asked students to ‘Describe…’ but a mark scheme that focused on explanation.
A mark scheme must also be applicable to the full range of possible responses to a question. Simply stating a model answer provides little, if any guidance to markers. In particular, it’s important to define the grey areas that exist between marks so that all markers can make consistent judgments and that a student gets the same (or similar) mark regardless who is marking their work. Mark schemes must be designed such that they work consistently across the full achievement range, like a ruler measuring shapes of different sizes.
There are, of course, other important considerations when creating an assessment, such as the level of demand targeted, how much material to sample across a test, consistency with past assessments and ensuring questions are equally accessible to different groups of students. However, no assessment will get off the ground without careful treatment of these four key areas.
