The news about education this week has been filled with the reemergence of Grammar Schools and the rightful concern about the stresses and strains of school budgets. Flat cash is proving more like ‘flat broke’ for too many schools. Yesterday, there was one less heralded chink of light and it emerged from an unexpected source: Ofsted.
Yes, you heard me right. Ofsted are proving to be undertaking a quiet revolution. Under the aegis of the new Chief Inspector, Amanda Spielman, Ofsted have been undertaking research studies to evaluate the reliability of their judgments.
Released yesterday, the research, entitled plainly ‘Do two inspectors inspecting the same school make consistent decisions?’ explored how reliable their short inspections were proving. It was a small study of 24 judgements, with 22 of those judgements being agreed. Of course, when the repercussions of a bad inspection judgment can prove so damaging, just under 1 in 10 unreliable judgements across inspectors appears too much. It is, of course, but having this degree of inter-rater reliability between inspectors is actually really good for such a complex judgement.
We can tentatively suggests measures to improve the training and reliability of inspectors is improving, or that shrinking the sheer number of inspectors and centralizing training is more likely to increase reliable judgements. I have been critical of Ofsted many times, but it is right to share when they do deserve some praise.
We should rightly guard our optimism. There is a paradox that being reliable may come down to having too limited a inspection model. Perhaps a narrow focus on data is a reason for this good degree of reliability? Perhaps anchoring each inspection in a previously ‘Good’ school is likely to offer quite consistent judgements?
And still, questions about the validity of what Ofsted inspectors look at, and look for, on their inspections need to be asked. Likely even more important that the internal reliability of Ofsted judgements, we should be looking at the validity of inspection. That is to say, is the inspectorate succeeding in measuring what it is intended to measure and is there any improvement in schools as a result of the inspection. Also, what are the unintended consequences of inspection judgments? Issues still abound with Ofsted and its place in the education system.
Perhaps most positively though, in the commentary by the Chief Inspector, Amanda Spielman. She recognized the issue of how reliability can prove a double-edged sword:
“For one thing, we would not want to over-simplify inspection in the pursuit of consistency. A tick-box approach, for instance, might lead to improved reliability but would be a mechanistic approach to inspection that would almost certainly undermine its validity.” (Full Commentary)
The approach clearly being spearheaded by Spielman is a willingness to evaluate the workings of Ofsted. Already, there is a commitment to further research and self-evaluation:
“We are beginning to shape up what this research programme should look like.”
Such scrutiny should be applauded. A commitment to evaluation should be the foundation stone of our entire school system. In a week where the dubious policy of Grammar Schools has been heralded as a star-spangled solution for all of our social mobility ills (sans evidence), we should be thankful from smaller, less headline-grabbing, but no less significant announcements.
With cautious optimism, let’s celebrate the quiet effectiveness of the new Chief Inspector in stark relief compared to the over-confident bombast of some of her predecessors. We are teachers: we live in hope, with no little idealism. We don’t need to get caught up in grading to recognise signs of improvement.