The Problem with Research Evidence in Education – Part 2

In Debates and Polemics, Evidence in Education by Alex Quigley14 Comments

20140626-190708-68828223.jpg

Yesterday I attended a CEBE (Coalition for Evidence-Based Education) event about the role of research evidence in education. The use of evidence is squarely in the sights of the Department for Education, hence they hosted this ideas fuelled event.

The question the event raised was: Should Government have a role in choreographing an evidence system in education? If you have any views please share them in the comment section.

I wrote a blog stating my views on ‘The Problem with Research Evidence in Education’ before the session – see here. A lot of the themes of the session chimed with my post.

The event discussed some really interesting ideas related to the ‘evidence ecosystem‘ in education. Ideas and debate ranged from the role of government; what we can learn from other professions; a prospective role for a College of Teaching (there was a really promising model of a College of Policing – shared by the excellent Rachel Tuffin, Director, Knowledge, Research and Education at the College); national and local structures to create a ‘pipeline‘ for research; the role of Teaching Schools and researchers; different types of research evidence – from large scale RCTs and small scale action research etc.

One if the most interesting statements of the day came from Jonathan Shepard, Professor of Oral & Maxillofacial Surgery; Director, Violence and Society Research Group, Cardiff University. From his position as practising surgeon and lead researcher, he made the following statement (I apologise if a word or two is out of place):

“Education academics should continue to practice in the classroom.”

This is an interesting notion. Is academia too divorced from practice? Does this divide ultimately need broaching if we are to ever have a truly research-led profession? If so, how? Can we shine a light on the current good practice that sees Higher Education institutions work well in unison with schools?

He stated it would be absurd if a Professor of Obstetrics was not practising as an obstetrician. His view (and mine), is that research must be firmly rooted in the problems of the front line. Undoubtedly, working in the classroom, and having direct engagement with the classroom, teachers and students, has crucial credibility for anyone communicating research evidence to teachers.

Prof Jonathan Shepard made a second statement that raises all sorts of interesting questions about research evidence and the gap between academia and the classroom. He stated:

“Professors of Education should have their offices in comprehensive schools.”

What an interesting suggestion. I’m inclined to agree. How about you?

Comments

  1. This is an interesting notion. A similar argument could be made with respect to Managers at all levels; but especially Senior Managers. Perspectives change as your daily experiences change. The only way to remain in touch with the everyday reality of those at the sharp end (who, ultimately, are the means to deliver whatever it is the Manager has a responsibility to deliver) is to be a part of that reality. SLT (and Head Teachers) should do as much teaching as they can manage. In the same way, Managers everywhere should be closely involved with the day to day realities of work. As a Senior Manager, I am still heavily involved with delivery (about 60% of my time). Without this I would not be able to keep in touch with the issues facing my group on a day to day basis….experiencing it is very different from hearing or reading about it. I guess the same thing applies to educational researchers (distance results in not being aware of the real issues, and time is spent solving problems that aren’t important?).

    1. Author

      Quite. I agree on the SLT point. There is an interesting shift in many schools towards ‘executive’ style leaders, which will be interesting to see play out.

  2. Over the past 10 years or so, I have found myself working with a variety of Profs. Those that have left Professorship to re-enter education, to lead schools and institutes and engage with us as sector professionals are worth every penny.
    They share the same professional disquiet as we do that those that lead DfE and Ofsted seem completely out of step with the education and research community. What is most disappointing about the soundbyte of government (read more of that here – http://goo.gl/2qjqIb – is that it bases its ‘sense’ on self-referential enquiry. Cobblers.
    Elizabeth Truss has identified that UK schools use fewer text books than higher performing countries, and uses that evidence to suggest we should go back to text books. Which ones?
    When both Shanghai and Singapore seek to leave the PISA rat race, what is the point of our joining it?
    Truss’s statement can be read here, and it makes me weep. http://goo.gl/gAwFYL

  3. It sounds like a really interesting meeting. I think that there is clearly a need for greater research literacy within schools and I quite agree that the gap between educational research and the classroom needs to be closed. I’m not convinced that the DfE and Ofsted are ‘out of step’ as your previous commenter suggests: I think that your meeting forms part of a pattern of real engagement with the profession.
    At the heart of this matter, however, is the overlapping range of ways in which teachers can engage in research. You mention this in your previous post. Teachers need to engage with high quality, effective research. This does not necessarily mean that they conduct the research themselves (I think that you’re quite right about the dubious value of individual action research projects), but it does demand that there is a pipeline of research that is clearly relevant to teachers. Whether or not educational researchers need to be based in schools, I’m not sure – it’s an interesting idea. What’s certainly a great start is to have ‘front-line’ teachers like you involved in well-designed, targeted research that is co-ordinated by professors of education. I’ve blogged about some of these issues myself at http://wp.me/p4GggC-c

  4. Pingback: Education Panorama (July ’14) by @TeacherToolkit | @TeacherToolkit

  5. Hi Alex,

    I have three points.

    1. Shouldn’t the title read “The problem with teachers’ attitude to research?” The dominant part of your post seems to deal with the attitude of teachers, which you characterise as a rather defensive set of prejudices. If they get themselves into a position in which effective practice is equated with a (right wing) political perspective of which they disapprove, then teachers have got themselves into a very bad place, which will only end in tears. And if you have a problem communicating the results of valid research (let us assume Assessment for Learning, for instance) then the problem might well be with the teachers, or at least with the environment in which teachers work, as with the researchers. Researchers should not constrain the sort of research they do to what is going to produce results that are necessarily palatable to teachers. Maybe the researchers need to tell the teachers to change their practice completely.

    I am not talking here about the 99% of educational research which is completely irrelevant to classroom practice but the 1% which effectively addresses the requirement of teaching and learning but which may not necessarily come up with the “right” answer.

    2. The problem, it seems to me, is research, in any field, addresses process, while teachers think about their job largely in terms of personality. This happens because experience in the classroom shows that it is personality that motivates and process (e.g. individualised feedback) is such a lot of hard work that it is almost impossible to do with the numbers of students that modern education throws at you. So the best strategy is to project a strong personality and talk loftily about “independent learning” as if it were a deliberate strategy.

    Take the medical analogy – you might research the effectiveness of a certain course of drugs or a surgical procedure, both of which assumes that the process being researched is not going to be completely different every time it is used, depending on the doctor. But if you do some educational research which says “give more feedback” and you hand that advice out to 100 teachers, you will get 100 completely different sorts of practice.

    I note that Elizabeth Truss’ argument on textbooks makes James Wilding weep. I think her argument is very strong and find it very bizarre that the process of teaching people to read and write effectively should be dismissed as a “rat race”. But as I suggest on my blog at http://edtechnow.net/2014/06/30/textbooks/, I think the word “textbook” may have led to some misinterpretation of Truss’ argument. What she is talking about is the professional production of learning resources which are a) going to be digital not paper-based and b) going to be activity and process driven, not expositive.

    If you have a digitally mediated learning design, comprising a series of activities with adaptive pathways through those activities, you then have something that can can encapsulate the results of research theory, can be deployed with a reasonable degree of consistency, and (because it will automatically harvest learning outcome data) the success of which can be automatically monitored. Then you will have something akin to a medical procedure that can provide the implementation half of the research loop.

    I am not, of course, talking about writing the teacher and the teacher’s personality out of the script. That is essential and will still be infinitely variable. But the variability and effectiveness of the teacher will be something over and above the learning design, not something on which the implementation of the design is wholly dependent. That will allow research to be conducted, not on the teacher (still very difficult) but on the learning design (aka digital textbook for those with a taste for old-fashioned analogies).

    The mis-match between research and classroom is down to a mis-match between process-driven technology and personality-driven craft. The belong in different worlds. And although personality matters (mainly because role-modelling matters) – it should not be controlling element but rather a piece on the pedagogical chessboard.

    3. All you will achieve by bringing the researcher into the classroom is to put the researcher into the same straight-jacket in which the teacher is currently bound. Which is why “action research” and its variants amount to little more than methods of emasculating the disruptive potential of the true researcher. I wouldn’t put my money on your research hub model working.

    Crispin.

    1. Author

      Hello Crispin,

      In terms of number 1. Yes – it is more about teacher attitudes, but my headline was aiming to spark interest rather than connote accuracy.

      2. I’m not sure of your argument pertaining to research here. I haven’t listened hard enough to Truss in terms of characterising her argument. Speaking to an audience of textbook (digital or analogue) producers about the power of textbooks doesn’t strike me as a subtle debate about the intricacies of the learning design! I think your ideal of a flexible digital design being something akin to a medical intervention then discounts all of those personal emotions and biases of individual teachers that you recognise. You appear to go down the route that digital textbooks have a positive standardising effect if I am not wrong. The problem with such textbooks, however notionally flexible, is that they never quite meet the needs of unique students in unique school contexts; the curriculum is always in flux; teachers are humans and they veer from the script; any technical, technology competence takes time and money in training terms, never mind just the purchasing – such liberal funding no longer exists; and, finally, I have never been presented with a textbook which is as good as something I could create myself (and I value the autonomy and control of doing it myself).

      3. I don’t think my hub model is as you describe. One, I don’t advocate ‘action research’ – there is a whole blog in that one. I don’t advocate teachers all become researchers; nor researchers become teachers, nor some unsatisfactory half-way house. Our hub is about getting the academics with the best understanding of learning, for example Prof Rob Coe, from Durham, and mediate with leaders in schools, to support teachers in understanding the best research and interventions available. Then support them in using this research-led approach to inform school improvement. It is quite specific. It is about being excellent consumers of research, not producers. I would hope that if you put the ‘true researchers’ with top notch ‘true teachers’ within a robust leadership framework, then good things should happen. I say should – maybe it won’t. Being in an RCT with robust independent evaluation will surely tell us! If it doesn’t work we have added to the knowledge in the world and we will learn some valuable lessons I’m sure.

      1. Alex,

        Thanks very much for the considered reply.

        1. Yes of course – a catchy title is everything!

        2. When I speak of Truss’ argument, I am just going by the article in the Telegraph at http://www.telegraph.co.uk/education/secondaryeducation/10925490/Britain-needs-a-revolution-in-the-classroom.html. I am not sure the nature of the audience is significant. If I were a politician, I’d certainly choose a sympathetic audience every time if I could. Though I would not say that the sort of textbooks that are currently published are sufficient. Part of the reason for this, I think, is the nature of the market, in which teachers tend to buy textbooks that have been endorsed by the exam board, and publishers pay the exam boards for that endorsement (or belong to the same group of companies). The market is effectively corrupt and certainly uncompetitive. So the fact that I support aspects of Truss’ message does not mean that I am an unreserved supporter of her audience.

        I think you hone in on the key question which is the balance and relationship between what is fixed and what is variable. Yes, I certainly recognise the variability of teaching, both at the macro level (different children, teachers, objectives) and micro level (the need for variable response to the contingencies of the classroom). But that doesn’t seem to me to deny the existence of fixed principles of pedagogy. A civil engineer might build a different bridge for every site that he works on – but that does not alter the fact that there are certain principles of civil engineering that remain constant and which inform the design of all those bridges, and that there are likely also to be various tools of the trade and processes that also remain fixed. And this common part of the professional toolkit is what research is all about. If nothing is fixed, if everything is infinitely variable, then there is nothing to research and nothing to be said about pedagogy.

        In any debate, I think it is easy to caricature the opposing point of view. I am not in favour of using textbooks to force teachers to teach in a particular way – if that is what you mean by a standardising effect. Teachers have always used textbooks as a pick-and-mix, veering away from the text to address contingencies that arise. And so they should. What I am arguing for, in the model of a disaggregated collection of learning activities that can be sequenced and re-sequenced to suit the needs of the teacher, is set of learning resources that the teacher will want to use, not be constrained to use (except, perhaps, by their Head of Department), because they improve learning outcomes. And that improvement in learning outcomes will be evident in the research data, after all the other variables have been averaged out, not because the learning resource was forced on the teacher but just because the learning resource was given to the teacher and used by choice.

        You say that you have never seen a textbook which is as good as something you could create yourself. I can quite believe it. But doesn’t that strike you as strange? Your expertise is in managing the classroom – you are only an amateur at the design of learning resources. Doesn’t the fact that you can do it better than the professionals say something about the market for learning resources? Most professions start very generalised and become increasingly specialised, with greater division of labour, as the quality of expertise grows. By that measure, teaching is still at a very early, craft stage of its development.

        When we move from paper-based to digital resources (and I interpret “digital resources” in terms of activity-based software, not PowerPoints), then the teacher’s deficit of expertise in comparison with the professional software house becomes all the more stark.

        I have referred elsewhere to the initiatives in the 1970s around Nuffield and SMP Maths – attempts to produce quality learning resources. Athough they did pretty well, I don’t think that either attempt was entirely successful because I think they became self-study schemes, which somewhat emasculated the teacher and the potential for social interaction and whole-class teaching. But in my view, that was in significant part because of the paper-based medium in which they were produced. I think that digital software provides a huge opportunity for resources that will support classroom teaching and not undermine it.

        And digital resources will not only be more social but they will be more flexible in the way that they are used. They can be paramaterised and allow individual teachers or communities of practice to author specialised content for particular software tools. Given sequencing tools, different activities can be combined into different learning designs. This is a vision to empower teachers, not emasculate them.

        3. I apologise – I misunderstood the significance of your hub. And I certainly support the work that Professor Coe is doing. Though it still strikes me that, even if you manage to introduce research-based practice at your school, you will have an uphill battle spreading it outwards to others, merely by sharing practice and training. And I suspect that good learning resources, the tools of the trade, will be needed to support that progress in a way that will encourage viral replication and genuine disruption.

        In a Twitter conversation with Lord Lucas last year, Dylan Wiliam suggested that the fatal flaw with RCTs would be the expense of collecting enough data to compensate for the many variables (intake, teaching style etc) involved in teaching. In this regard, a further huge benefit of data-driven ed-tech is that it will collect data automatically, principally for the purpose of informing the teaching process, but available also for the purposes of optimising and researching provision. I suspect that only ed-tech will provide data on the scale that is required to research what works in education on a sustainable basis.

        But I have to admit that there has been some success in London, on a model which you could perhaps argue was similar to your hub, so maybe your chances are better than I was suggesting. Either way, it will be very interesting to see how you get on. Ultimately, I don’t think it’s an either/or.

        Good luck,
        Crispin.

  6. Yes. Completely agree. This is one of those ideas that no seems so blindingly obvious I can’t think why it isn’t the case.

  7. I fully support more homework in school so that students will get higher test scores; nonetheless, i’ve always hated homework.

Leave a Reply