A major challenge for school leaders and teachers interested in evidence-based practice (EBP)  is to be constantly seeking out EBP 's limitations and weaknesses.  To do this I recommend that headteachers and school research leads play close attention to the work of Professor Trisha Greenhalgh, who recently authored an article entitled Of Lamp Posts, Keys, and Fabled Drunkards: A Perspectival Tale of 4 Guidelines (Greenhalgh (2018).  In this article  Professor Greenhalgh describes her own experience as a patient arising from a high impact cycling accident and how evidence-based guidelines were misused in her treatment.

The use and abuse of guidelines

Without going into the details of Professor Greenhalgh's accident - which involved coming off a bicycle at 20 mph, hitting the road surface resulting in multiple fractures -  there were a number of occasions where according to Professor Greenhalgh's account  guidelines were either misapplied or not used at all during her treatment..

• a guideline that existed and was relevant but which was not used 
• a guideline that was not relevant but which was used 
• a guideline that was relevant but was misremembered and misapplied by commentators claiming to be giving evidence based advice 
• a guideline that did not exist but which was quoted by adherents of EBM as if it had existed (and which was also misremembered and misapplied).

Professor Greenhalgh subsequently identifies three reasons why this misuse of guidelines can happen.

First, we are hard-wired to classify. So when a doctor comes across a patient the tendency to classify them as part of a group.  Once that is done, this leads to the patient being treated on the basis 'guidelines' which are designed to meet the needs of the 'average patient' not the individual.

Second …. bounded rationality-that is, the idea that because real‐world decisions often involve numerous options, outcomes, and contextual factors, we unconsciously simplify the problem to make it possible to cope with cognitively and manage practically.  Indeed, the inexorable pressures of modern clinical work often require us to use such "fast and frugal" reasoning p(6).  

Third, there is an over‐valuing of rationality (doing the thing right-as in following rules and guidelines) over reason (doing the right thing-as in making the right moral choice for this patient at this time, given these contingencies). p(6)

What are the implications for senior school leaders and school research champions?

• Do 'average' pupils, 'average' classes or 'average' schools exist or are they all unique with their own special requirements
• It is essential to keep up to date with the latest research and guidance provided by the Education Endowment Foundations, as otherwise you may miss out on something that could have real benefits for your pupils.
• However, just because the EEF have produced a new set of guidance, or added something to the Teaching and Learning Toolkit, or published something promising research findings - this does not make them a priority for your school.  As they maybe other matters or issues which are far more relevant to your pupil's needs
• In these very early days of Research Schools and relatively inexperienced school research leads there are very real risks that colleagues may get things wrong - misremembered or misapplied.  So it is really important when someone says ' the research says' that the response is 'Ok, what claim are you making and what is the warrant for your claim?' Wallace and Wray (2016) and Booth, Colob, et al. (2016)
• What structures have been put in place to help identify the misapplication or misuse of research evidence or guidelines. What processes are in place to help address the consequences of things going 'wrong'? 
• There is a distinction between clinical judgment and organisational judgment - clinical judgment refers to decisions made about individual patients, whereas organisational judgment is applied at scale, across the organisation.   As such, evidence-based practice may be more useful when applied to the school as a whole, rather than trying to apply it to decisions about individual pupils.
• The evidence about a particular problem is never set in stone and there is an ongoing need for conversations to continue to unpick the nature of the problem, so the appropriate actions can be taken.

To conclude

Professor Greenhalgh cites Sir John Grimley Evans who in 1995:

There is a fear that in the absence of evidence clearly applicable to the case in the hand a clinician might be forced by guidelines to make use of evidence which is only  doubtfully  relevant,  generated  perhaps  in  a different grouping of patients in another country at some other time and using a similar but not identical treatment. This is evidence biased medicine; it is to use evidence in the manner of the fabled drunkard who searched under the street lamp for his door key because that is where the light was, even though he had dropped the key somewhere else. (page 451)

Reference

Booth, W., Colob, G., Williams, J., Bizup, J. and Fitzgerald, W. (2016). The Craft of Research (Fourth Edition). Chicago. The University Of Chicago Press.
Greenhalgh, T. (2018). Of Lamp Posts, Keys, and Fabled Drunkards: A Perspectival Tale of 4 Guidelines. Journal of Evaluation in Clinical Practice. 0. 0.
Wallace, M. and Wray, A. (2016). Critical Reading and Writing for Postgraduates (Third Edition). London. Sage.