Multiple Choice Questions: A) Use regularly B) Don’t use

In Feedback & Questioning, Teaching & Learning by Alex Quigley12 Comments

20140314-191941.jpg

In the last couple of years I have become more interested in the use of multiple choice questions. I had never really used them with any regularity and considered them a rather limited American idiosyncrasy. My first foray into exploring them, and their use, was sparked by Dylan Wiliam’s use of ‘hinge point questions‘. As I encountered this approach it was clear that Wiliam’s focus was on the usefulness of the approach for formative assessment. Indeed, there is much to commend the immediacy of multiple choice questions in giving the teacher useful feedback. That being said, my teacher intuition has made me hesitant about the effectiveness of multiple choice as a method of instruction in my classroom.

I started to dig into some research before I committed to using this method for our new KS3 English curriculum redesign. Having read Paul Bambrick-Santoyo’s excellent book, ‘Leverage Leadership‘, I was more intrigued by this method. It makes a compelling case for the ongoing assessment opportunities provided by this straight-forward strategy (of course, this method is very relevant to American SAT tests etc.). In particular, he advocates combining multiple choice questioning with more traditional essay based responses (he doesn’t really address the matter of short answers as an alternative to multiple choice questions).

I then read some excellent blogs by fellow English teachers employing multiple choice questions as a real focal point of their pedagogy:

Joe Kirby has written with his usual eloquence and drive on the usefulness of multiple choice questions here.

Phil Stock has written an equally persuasive and useful post about his focus upon multiple choice questions here and here.

Each English teacher presents a sound case for the efficacy of this particular questioning method. So much so, they are investing their time and energy to create an array of such questions at their heart of their KS3 curriculum. I wasn’t ready to commit to this method yet, so I kept digging for evidence. Perhaps I was searching out confirmation of my bias against the strategy? Very possibly. Still, I stumbled on some compelling research that outlined some serious reservations with the approach which did confirm my educated intuition having used multiple choice only sparingly myself.

The most interesting research I found was undertaken by Kang, McDermott and Roediger (2007) – see here. They found evidence that short answer tests proved more effective than multiple choice questions in exploiting the ‘testing effect‘ – the well established evidence that testing can enhance learning and memorisation, not just simply assess it. The reason for this was that short answers proved more difficult, by not having alternate answers available, therefore students had to work harder by having to retrieve their answer from memory. This alternative method of short questions, arguably easier to devise, but certainly something I find easier, appeared to trump multiple choice.

Another negative factor posed by the research of Roediger and Marsh (2005) – see here – reveals that the incorrect answers in the multiple choice model can easily be mistaken by students and create a ‘false memory‘, whereat they remember the wrong answer over the correct answer. Robert Bjork, in his research on multiple choice tests – see here – does challenge the findings of Roediger and friends, with the important proviso that the questions must be well constructed and the incorrect answers must be plausible. Yet, if the answers are plausible then the potential ‘false memory‘ effect looms into view! So far, so muddy.

I then had the pleasure of reading this excellent article by Barbara Bleiman – see here – on the English key stage three curriculum. It sagely questions what knowledge and understanding we should be looking to foster in our students. It really got me thinking about what type of knowledge we are seeking. It make me consider whether conventional questions – like short questions and longer, essay-style questions – are more effective in securing the complex knowledge of the great literary texts being explored. It was interesting food for thought. I was losing faith in A) Use regularly.

Where does all this research get us? It proves that multiple choice questions are tricky things to create, and utilise effectively, and that there isn’t definitive evidence that they are any better than more typical short question types. The best answer would be to undertake my own research, but I won’t take that opportunity, given my hesitation at the cost, given my limited time. I would be intrigued by an research gathered by Joe Kirby and Phil Stock in their schools. Of course, their findings may prove that this method of instruction works in their school context and for their students. Finding research papers should prompt us into action and consider our practice, but sometimes we should plough on and find the best evidence in our own unique circumstances and that of similar schools.

So my answer to the use of multiple choice questions in my English lessons:

C) No. At least not yet.

 

 

 

Other interesting related reading:

Daisy Christodoulou has written with insight on the positive impact of multiple choice questions here.

Christina Milos has written about the disadvantages of multiple choice questions here.

Harry Fletcher-Wood has charted a brilliantly systematic approach to multiple choice as ‘hinge-point’ feedback here.

20140314-191957.jpg

Comments

  1. Pingback: Why use multiple-choice questions? | Pragmatic Education

  2. Is the pay off with using multiple choice the fact that it can be marked and feedback given very quickly as the system can be automated? With marking and feedback such a considerable part of our workload, is that a satisfactory sacrifice?

    1. Author

      By automated, do you mean Socrative etc? If so, yes, after doing all the work sorting the questions, then ensuring you have all the kit for all the students (which I don’t), then I can understand some time being saved…perhaps. Instead, I get students to give short answers, often orally, for instantaneous feedback. or written for more considered, detailed answers, and check those when I’m marking books.

  3. Because I teach English I tend to give short answer/essay style tests and quizzes. Putting one’s thoughts into writing and conveying a point effectively is a skill I want to help my students develop, and I don’t think multiple choice questions will enhance those skills. My students are not the best when it comes to writing, so I have them constantly practicing and developing different writing techniques during warm-up exercises at the beginning of class, as well as on tests and quizzes.

  4. Pingback: How to design multiple-choice questions | Pragmatic Education

  5. Pingback: Multiple Choice Questions: A) Use regularly B) Don’t use | HuntingEnglish | Learning Curve

  6. Pingback: Standin’ in line marking time…. | Distant Ramblings on the Horizon

  7. Pingback: How we learn – a change of approach to teaching academic PE | drowningintheshallow

  8. Pingback: socrative – composing Qs | Learning and Teaching at Abbot Beyne

  9. Hi Alex,
    Im trying to put together an RCT in our on settings around this. there is a really good follow up paper from 2014 you might want to have a look at,

    McDermott, Kathleen B., Pooja K. Agarwal, Laura D’Antonio, Henry L. Roediger III, and Mark A. McDaniel. “Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes.” Journal of Experimental Psychology: Applied 20, no. 1 (2014): 3.

    I’d love to have a chat with you about this some time. Im only in Halifax,

    Jo

    1. Author

      Thanks Jo – I’l take a look – I’m not sure if I have read it or not. I think some of my thinking has developed in this area. Would be very happy to chat. Alex

Leave a Reply