In the last couple of years I have become more interested in the use of multiple choice questions. I had never really used them with any regularity and considered them a rather limited American idiosyncrasy. My first foray into exploring them, and their use, was sparked by Dylan Wiliam’s use of ‘hinge point questions‘. As I encountered this approach it was clear that Wiliam’s focus was on the usefulness of the approach for formative assessment. Indeed, there is much to commend the immediacy of multiple choice questions in giving the teacher useful feedback. That being said, my teacher intuition has made me hesitant about the effectiveness of multiple choice as a method of instruction in my classroom.
I started to dig into some research before I committed to using this method for our new KS3 English curriculum redesign. Having read Paul Bambrick-Santoyo’s excellent book, ‘Leverage Leadership‘, I was more intrigued by this method. It makes a compelling case for the ongoing assessment opportunities provided by this straight-forward strategy (of course, this method is very relevant to American SAT tests etc.). In particular, he advocates combining multiple choice questioning with more traditional essay based responses (he doesn’t really address the matter of short answers as an alternative to multiple choice questions).
I then read some excellent blogs by fellow English teachers employing multiple choice questions as a real focal point of their pedagogy:
Joe Kirby has written with his usual eloquence and drive on the usefulness of multiple choice questions here.
Each English teacher presents a sound case for the efficacy of this particular questioning method. So much so, they are investing their time and energy to create an array of such questions at their heart of their KS3 curriculum. I wasn’t ready to commit to this method yet, so I kept digging for evidence. Perhaps I was searching out confirmation of my bias against the strategy? Very possibly. Still, I stumbled on some compelling research that outlined some serious reservations with the approach which did confirm my educated intuition having used multiple choice only sparingly myself.
The most interesting research I found was undertaken by Kang, McDermott and Roediger (2007) – see here. They found evidence that short answer tests proved more effective than multiple choice questions in exploiting the ‘testing effect‘ – the well established evidence that testing can enhance learning and memorisation, not just simply assess it. The reason for this was that short answers proved more difficult, by not having alternate answers available, therefore students had to work harder by having to retrieve their answer from memory. This alternative method of short questions, arguably easier to devise, but certainly something I find easier, appeared to trump multiple choice.
Another negative factor posed by the research of Roediger and Marsh (2005) – see here – reveals that the incorrect answers in the multiple choice model can easily be mistaken by students and create a ‘false memory‘, whereat they remember the wrong answer over the correct answer. Robert Bjork, in his research on multiple choice tests – see here – does challenge the findings of Roediger and friends, with the important proviso that the questions must be well constructed and the incorrect answers must be plausible. Yet, if the answers are plausible then the potential ‘false memory‘ effect looms into view! So far, so muddy.
I then had the pleasure of reading this excellent article by Barbara Bleiman – see here – on the English key stage three curriculum. It sagely questions what knowledge and understanding we should be looking to foster in our students. It really got me thinking about what type of knowledge we are seeking. It make me consider whether conventional questions – like short questions and longer, essay-style questions – are more effective in securing the complex knowledge of the great literary texts being explored. It was interesting food for thought. I was losing faith in A) Use regularly.
Where does all this research get us? It proves that multiple choice questions are tricky things to create, and utilise effectively, and that there isn’t definitive evidence that they are any better than more typical short question types. The best answer would be to undertake my own research, but I won’t take that opportunity, given my hesitation at the cost, given my limited time. I would be intrigued by an research gathered by Joe Kirby and Phil Stock in their schools. Of course, their findings may prove that this method of instruction works in their school context and for their students. Finding research papers should prompt us into action and consider our practice, but sometimes we should plough on and find the best evidence in our own unique circumstances and that of similar schools.
So my answer to the use of multiple choice questions in my English lessons:
C) No. At least not yet.
Other interesting related reading:
Daisy Christodoulou has written with insight on the positive impact of multiple choice questions here.
Christina Milos has written about the disadvantages of multiple choice questions here.
Harry Fletcher-Wood has charted a brilliantly systematic approach to multiple choice as ‘hinge-point’ feedback here.