New : Teachers and performance related pay - what's the evidence?

Last week Schools Week published a piece by Lee Miller – Deputy Chief Executive of the Thinking Schools Academy Trust (TSAT) titled Performance related pay will solve teacher retention crisis which describes a new teacher pay structure for the TSAT – which includes

• From September 2018 a minimum starting salary of £25,000 (£2,000 above the national average)
• All progression at the upper pay levels is based solely on performance (excellent teaching)
• Teachers at the top of the scale – who exceed performance targets receive a 3% non-consolidated bonus

Having read the article – my first response was to ask: What research evidence is there to support the pay structure which has been put forward?

Pietro Marenco (@PMarencoHR and scienceatwork.com) very kindly tweeted an extract from (Weibel et al., 2009) who undertook an meta-analysis of the effect of pay for performance on performance and who state:

Our meta-analysis clearly demonstrates that the task type moderates the effect of pay for performance on performance. Pay for performance has a strong, positive effect on performance in the case of noninteresting tasks. Pay for performance, however, tends to have a negative effect on performance in the case of interesting tasks. The vignette study reveals (a) why pay for performance sometimes undermines performance and (b) how pay for per- formance produces hidden costs, which also need to be accounted for. 

Pay for performance causes a cognitive shift, that is, it strengthens extrinsic motivation for behavior (causes a price effect) and at the same time weakens intrinsic motivation for behavior (causes a crowding-out effect). Depending on the strength of these two opposing effects, pay for performance either hurts or promotes personal efforts: The more intrinsic motivation was there at the beginning, the more of it can be destroyed. 

Hidden costs arise even if the price effect is stronger than the crowding-effect. The loss of intrinsically motivated behavior has always to be compensated by external rewards.  P18

Which tends to suggest that in the context of teaching – a complex task if ever there was one (Shulman, 2004) - performance related may have a negative effect on performance and may ‘crowd-out’ intrinsic motivation (and associated discretionary effort.

If we turn now to research evidence which relates specifically to the education sector, we can look at the (EEF, 2017) summary on performance pay which states:

The results of rigorous evaluations, such as those with experimental trials or with well-controlled groups, suggest that the average impact of performance pay schemes has been just above zero. Some approaches appear to show more promise, such as bonuses or enhanced pay to attract teachers to challenging schools, or loss aversion, where the award has to be paid back if student results fall below a certain level. 

And which goes onto to state:

The evidence is not conclusive. Although there has been extensive research into performance pay, much of this is either from correlational studies linking national pay levels with general national attainment or from naturally occurring experiments. More recent randomized trials have had mixed results. Overall, it is hard to make causal claims about the efficacy of performance pay on the basis of the existing evidence

In addition, a systematic review by (Bajorek and Bevan, 2015)  of performance-related-pay in the UK public sector who found.

…. some evidence that PRP schemes can be effective across the three domains of the public sector for which there was evidence available (health, education and the civil service), but findings within and between the sectors are mixed, with scheme effectiveness often dependent on scheme design and organisational context 

Of particular interest to those who manage the teaching workforce is that there may well be - gender and age differences in the response to PRP

(Leigh, 2012)) and (Jones, 2013) report evidence to indicate that male teachers respond more positively to and support PRP schemes than their female counterparts, and Jones (2013) also highlighted that women are more likely to reduce their hours under PRP than men. Evidence also suggests that teachers with more experience display negative reactions to PRP in comparison to early career teachers (Jones, 2013; Leigh, 2013), although it is  PRP in the UK public sector  unclear whether this results from hostility to changes in the system, or previous negative experiences to PRP.(p91)

Finally, we could examine the work of (Lynch et al., 2016) on teacher retention who found

Factors that are significantly associated with intent to stay in the profession could be labelled as ‘protective factors’, worthy of attention among school leaders and policymakers. 
Unsurprisingly, we found that by far the strongest predictor is ‘job satisfaction’. Among the other factors, the strongest predictors are: 
• being proud to work at the school 
• having adequate resources 
• being well supported and valued by school management 
• having an effective governing body 
• appropriate pay for level of responsibility. P17

(Since I posted this blog @Jonathan_Haslam of the Institute of Effective Education has pointed me in the direction of this IEE Best Evidence in Brief on performance pay)

In summary

On the one hand, it would appear that the research suggests that:

• Performance related pay is not suited to complex tasks such as teaching 
• Performance related pay may reduce intrinsic motivation
• The impact of performance related on pupils’ results is just above zero
• Male teachers tend to respond more positively to PRP than their female counterparts
• The introduction of PRP may lead to female teachers reducing the number of hours taught
• More experienced teachers are more likely to display negative reactions to PRP compared to early career teachers 
• Job satisfaction is by far the biggest predictor of teachers’ intention to stay in the profession

On the other hand

• There is a problem of recruitment and retention
• The research evidence could be viewed as being inconclusive
• Pay does appear to influence teacher retention rates
• The senior team of a MAT clearly believe that they have the expertise to make it work
• There would appear to be stakeholder support from the relevant teaching union

So where does this leave us?

Evidence-based practice involves the conscientious, explicit and judicious use of the best available evidence – research, organisational data, practitioner expertise and stakeholder  in order to make decisions which will hopefully lead to a favourable outcome.

Given the nature of the evidence – the decision as to whether to proceed with PRP within a MAT is a judgment call.  That said, in making that judgement one hopes that the decision-makers have undertaken an appropriate process of due diligence and have asked the following questions.

• Is the issue an important problem for which a remedy is sought and that can be locally implemented?
• How substantial are the desirable anticipated effects?
• How substantial are the undesirable anticipated effects?
• How robust and secure are the different sources - research, practitioner expertise, stakeholder views and school data - of evidence?   
• Does the balance of the desirable and undesirable effects favour the proposal or a realistic alternative?
• How large are the resource requirements – attention, time, money, professional learning?
• What impact does the decision have on educational equity?  Will it help close gaps in attainment?
• Are there important ethical issues which need to be taken into account? 
• Are there key stakeholders – teachers, parents, trustees, who would not accept the distribution of the benefits, harms and costs?
• Would the intervention adversely affect the autonomy of teacher, department, school or MAT?
• Are there important barriers that are likely to limit the feasibility of implementing the intervention (option) or require consideration when implementing it?
• Is the intervention or strategy sustainable?

And finally

How strong is the recommendation ranging from a strong recommendation – where benefits clearly outweigh costs or vice versa, with consistent supporting evidence from research evidence, practitioner expertise, school data and stakeholders – without major limitations.  Or is it a weak recommendation with uncertainty in estimates of benefits and costs Some supporting evidence from research evidence practitioner expertise – though with major limitations.

References

BAJOREK, Z. M. & BEVAN, S. M. 2015. Performance-related-pay in the UK public sector: A review of the recent evidence on effectiveness and value for money. Journal of Organizational Effectiveness: People and Performance, 2, 94-109.
EEF 2017. Teaching and Learning Toolkit : Performance Pay. London: Education Endowment Foundation.
JONES, M. D. 2013. Teacher behavior under performance pay incentives. Economics of Education Review, 37, 148-164.
LEIGH, A. 2012. The economics and politics of teacher merit pay. CESifo Economic Studies, 59, 1-33.
LYNCH, S., WORTH, J., BAMFORD, S. & WESPIESER, K. 2016. Engaging Teachers: NFER Analysis of Teacher Retention. Slough: NFER.
SHULMAN, L. 2004. The Wisdom of Practice-Collected Essays of Lee Shulman: Vol 1, ??????, Jossey-Bass.
WEIBEL, A., ROST, K. & OSTERLOH, M. 2009. Pay for performance in the public sector—Benefits and (hidden) costs. Journal of Public Administration Research and Theory, 20, 387-412.

Guest post by Sarah Selezynov - Japanese lesson study: a question of culture

Given the publication of the EEF's evaluation report on Lesson Study, it seems sensible to take a slightly broader perspective on Lesson Study, so this week's post is by Sarah Selezynov - Programme Leader - Bespoke Leadership Programmes; IOE - Learning & Leadership; UCL Institute of Education. Sarah has extensive knowledge of Lesson Study and is currently organising two lesson study events in December - with Professor Akihiko Takahashi - to explore the Japanese approach to problem solving in mathematics, lesson study as a tool to improve teaching and learning, and the role of the koshi.

(see http://www.ioe-rdnetwork.com/uploads/2/1/6/3/21631832/takahashi_events_flyer2.pdf for more details)

As a school leader who is interested in Japanese lesson study (JLS), you are probably reading the debate on this blog with interest – Should I or shouldn’t I?  Will it make a difference to my pupils in my school?  How can I be sure that the time and effort my school invests in this will pay dividends for pupil learning?  And you are right to be cautious.

And yet, I qualify this warning by saying that I believe that JLS has great potential for teachers and pupils.  JLS aligns with the wider research base on effective teacher professional development: it focuses on learning and not performance, begins with an end goal, engages teachers in and with research over an extended time frame, in collaborative groups.  And our research has shown strong evidence of improved teacher practice and student learning (Godfrey et al, forthcoming). 

So why the warning?  Because borrowing an education policy from another country and expecting it to simply work here as it does there, doesn’t really work – it rarely has.  Pasi Sahlberg (who is not against global borrowing per se) describes how a ‘network of interrelated factors – educational, political and cultural - …function differently in different situations’ (2011: 6) meaning that we cannot be sure that any one educational approach will function in the same way when it is translated from one country to the next.

So what do we need to consider when attempting to use JLS as an approach to teacher professional development in Britain?  First and foremost, we need to understand the cultural differences between Japan and Great Britain and how this might affect teachers’ responses to JLS. 

Hofstede (2010) categorises cultural differences along five dimensions, using a 0-100 ranking.  And on three of these rankings, Great Britain has a very different score to Japan:

1. Uncertainty avoidance (Gap: Japan 11, Great Britain 68.5)

‘The extent to which the members of a culture feel threatened by ambiguous or unknown situations' (2010: 191).

The lengthy, meticulous and detailed planning of JLS, including exploring known evidence through kyozai kenkyu, and the significant time spent predicting student responses, are all attempts to avoid any unanticipated events in the research lesson.  British teachers are very likely to be much less averse to taking risks in lessons and to therefore plan in less detail and not see the need for kyozai kenkyu.  

Uncertainty avoidance cultures also feel a greater need for protocols and rules, which may explain the formal and rigid processes of JLS.  It is highly likely that English teachers would not see the need for this level of formality and would want to deviate from LS protocols.  

Uncertainty avoidance leads to a greater tendency to believe in and revere expertise: hence the valued role of the koshi or ‘expert other’ in JLS.  British teachers engaged in LS are likely to value practice expertise as much as academic expertise, and less likely to see the need for a koshi. 

2. Individualism versus collectivism (Gap: Japan 36, Great Britain 3)

Individualistic societies are those where 'ties between individuals are loose: everyone is expected to look after him- or herself and his or her immediate family' (2010: 92).  In collectivist societies, 'people …. are integrated into strong, cohesive in-groups, which throughout people's lifetime continue to protect them in exchange for unquestioning loyalty' (2010: 92).

Great Britain is a highly individualistic society: occupational mobility is higher, teachers are managed as individuals and feedback on performance is given directly.   This difference is reflected in the English performance management system for teachers, the hiring and firing based on performance judgements, and performance feedback being given directly to the teacher after a lesson observation.  Japan is a collectivist society: occupational mobility is lower, teachers are managed collectively and it would not be productive to the group to give direct feedback to an individual.   JLS has evolved as a way of giving feedback on performance through the group, with the lesson plan as a collaborative product.  We might predict that teachers in Great Britain would shy away from the live observation element of JLS, fearing a judgement on their individual professional performance, which may affect job security.   Other collaborative aspects may also be challenging to implement in Great Britain, such as committing extended amounts of time to collaborative lesson planning process and working towards a whole school shared research theme.

3. Long term orientation (Gap: Japan 3, Great Britain 40.5)

‘The fostering of virtues oriented toward future rewards - in particular perseverance' (2010: 239). 

The importance of perseverance and effort is clearly seen in JLS, where a research theme will be pursued by a school for two or three years.  This contrasts with the British short-termist attitude which is likely to influence policy decisions about the time teachers are asked to commit to investigating a research theme. 

 In summary, some key cultural differences between Great Britain and Japan are likely to mean teachers struggle with several distinguishing features of lesson study as a research process, namely:

• Focusing on a shared research theme over a longer period of time;
• Spending time on collaborative lesson planning, including exploring relevant material around their research theme;
• Being observed by colleagues as they gather evidence in the research lesson;
• Seeking outside expertise to develop and enhance their research ideas.

In our work with schools, we have managed to support teachers to engage with models of JLS that feature all of the above elements and these teachers and leaders have spoken highly of lesson study.  However, we have also encountered schools who say they are doing ‘lesson study’ but do not work on a shared research theme, nor plan collaboratively, nor act as silent observers in the lesson observation, and do not look to outside expertise to enhance their learning.  

What does this mean for you as a school leader?  If you are already using JLS, make sure that teachers are not just paying lip service to its features but adhering strictly to the features that distinguish lesson study as a research process.  If you are seeking to introduce JLS, anticipate the above cultural resistance.  Make sure teachers really understand why JLS is designed in the way it is and what you will lose if you leave out any of its critical research features.

References

Godfrey, D., Seleznyov, S., Wollaston, N., and Barrera-Pedamonte, F. (forthcoming). Target oriented lesson study (TOLS) Combining lesson study with an integrated impact evaluation model.
Hofstede, G., Hofstede, G.J. and Minkov, M., 2010. Cultures and organizations: Software of the mind (Vol. 3). London: McGraw-Hill.

Sahlberg, P., 2011. Finnish lessons. Teachers College Press.

The school research lead, improvement research and implementation science

This week saw the welcome announcement of the appointment of Dr Becky Allen as the director of the UCLIOE’s Centre for Education Improvement Science.  On appointment, Dr Allen wishes to help develop “a firmer scientific basis for education policy and practice” and drawing on methods such as laboratory experiments and classroom observation.

Now regular readers of this blog will know that I have often expressed a concern over how educational researchers often misuse terms associated with evidence-based practice.  So, given this new initiative in improvement science it seems sensible look at a definition of improvement science/research and to do this, I’ll use the work of (LeMahieu et al., 2017)

Improvement Research : a definition (LeMahieu et al., 2017)

Improvement research is … about making social systems work better. Improvement research closely inspects what is already in place in social organizations – how people, roles, materials, norms and processes interact. It looks for places where performance is less than desired and brings tools of empirical inquiry to bear and to produce new knowledge about how to remediate the undesirable performance. Put simply, improvement research is not principally about developing more “new parts” such as add-on programs, innovative instructional artifacts or technology; rather, it about making the many different parts that comprise an educational organization mesh better to produce quality outcomes more reliably, day in and day out, for every child and across the diverse contexts in which they are educated.

Examples of Improvement Research/Science

1. Networked Improvement Communities;
2. Design-Based Implementation Research;
3. Deliverology;
4. Implementation Science;
5. Lean for Education;
6. Six Sigma;
7. Positive Deviance

As such, (LeMahieu et al., 2017) state that All seven of the approaches  ……. share a strong “common core”. All are in a fundamental sense “scientific” in their orientation. All involve explicating hypotheses about change and testing these improvement hypotheses against empirical evidence. Each subsumes a specific set of inquiry methods and each aspires transparency through the application of carefully articulated and commonly understood methods – allowing others to examine, critique and even replicate these inquiry processes and improvement learning. In the best of cases, these improvement approaches are genuinely scientific undertakings

In other words, improvement research is a form of ‘disciplined inquiry’ (Cronbach and Suppes, 1969)

What Improvement Science Is Not?

However,  as (LeMahieu et al., 2017) note a major distinguishing feature of  improvement research, is what it does not attempt to do.  Improvement research is not about creating new theories or research and development.  Nor is about seeking to evaluate existing teacher strategies, interventions of field-based trials.   Rather improvement science is about doing more of what works, stopping what doesn’t and making sure everything is joined up in ways which bring about improvements in a particular setting

Given this stance, then statements about the Centre for Education Improvement Science (CEISbeing about ‘laboratory experiments and classroom observations’ seem a little incongruent with the existing work in the field.

My confusion about the work of the CEIS is further compounded by mention in Schools Week where it describes Improvement Science London, which is also based at UCL, improvement science involves the recognition of “the gap between what we know and what we put into practice” and using the “practical application of scientific knowledge” to identify what needs to be done differently.   However, that could probably more accurately be described as ‘implementation science’ (a subset of improvement science admittedly).  So, let’s delve into a little more detail about what is meant by the ‘implementation science.

What is implementation science?

(Barwick, 2017) defines Implementation science (as) the scientific study of methods that support the adoption of evidence based interventions into a particular setting (e.g., health, mental health, community, education, global development).  Implementation methods take the form of strategies and processes that are designed to facilitate the uptake, use, and ultimately the sustainability – or what I like to call the ‘evolvability’ – of empirically-supported interventions, services, and policies into a practice setting (Palinkas & Soydan, 2012 ; Proctor et al., 2009); referred to herein as evidence-based practices (EBPs).

Barwick goes onto state that Implementation focuses on taking interventions that have been found to be effective using methodologically rigorous designs (e.g., randomized controlled trials, quasi-experimental designs, hybrid designs) under real-world conditions, and integrating them into practice settings (not only in the health sector) using deliberate strategies and processes (Powell et al., 2012 ; Proctor et al., 2009; Cabassa, 2016).  Hybrid designs have emerged relatively recently to help us explore implementation effectiveness alongside intervention effectiveness to different degrees (Curran et al,  2012).

As a consequence – implementation science sits on the right hand side of the following figure (taken from (Barwick, 2017))

So where does this leave us?

Well on the one hand, I am really excited that educational researchers are beginning to pay attention being done in field such as improvement and implementation science.  On the other hand, I’m a bit disappointed that we are likely to make the same mistakes as we have with evidence-based practice, and not fully understand the terms we have borrowed. 

Finally – this post may be completely wrong as I have relied on press releases and press reports to capture the views of the major protagonists – as such I may be relying on ‘fake news.’

References

BARWICK, M. 2017. Fundamental Considerations for the Implementation of Evidence in Practice. MelanieBarwickJourneysInImplementation [Online]. Available from: https://melaniebarwick.wordpress.com/ [Accessed 15 November 2017].

LEMAHIEU, P., BRYK, A., GRUNOW, A. & GOMEZ, L. 2017. Working to improve: seven approaches to improvement science in education. Quality Assurance in Education, 25, 2-4.

The effectiveness of lesson study has been called into question, following a £543,000 study at 181 schools, and 12,200 pupils made no difference to Y6 pupils attainment in reading and mathematics

The effectiveness of lesson study has been called into question, following a £543,000 EFF study involving 181 schools and 12,200, pupils found it made no difference to Y6 pupils attainment in reading and mathematics.

Lesson Study is a CPD approach originating in Japan that has become more popular in England in recent years and is a collaborative approach to professional learning.  Simply put, lesson study is a joint practice development approach to teacher learning, in which teachers collaboratively plan a lesson, observe it being taught and then discuss what they have learnt about teaching and learning

The project found no evidence that a particular version of Lesson Study improves maths and reading attainment at KS2.  However, there is evidence that some control schools implemented similar approaches to Lesson Study, such as teacher observation. As such the trial might, therefore, underestimate the impact of Lesson Study when introduced in schools with no similar activity. 

So does this EFF report sound the ‘death-knell’ for Lesson Study in England.  David Weston, Chief Executive of the Teacher Development Trust states in the TDT blog

There are some possible options.

1. If we decided to ignore the above and assume that the pedagogical content was effective, then either:

a. Lesson Study is an ineffective mechanism in all cases, or

b. it was an ineffective mechanism in this particular case

2. If we were determined to conclude that Lesson Study is always effective (which is also not plausible), then we would conclude:

a. This implementation is flawed, or

b. This pedagogical content is definitely bad.

My suggestion would be that none of the above conclusions are supported, in my view, by any reasonable reading of this study and the wider evidence base. We also need to question the extent to which we can draw any strong conclusions from a study where so many in the control group appeared to be engaging in similar practice

However, a report on peer lesson observation published by the EEF at the same time indicated that peer observation led to no overall improvement in combined maths and English GCSE scores for pupils of the teachers involved.  This would suggest the concerns about that the control group in the Lesson Study evaluation were enjoying improvements in pupil outcomes, and offsetting the impact of Lesson Study are possibly not warranted.

So what are school leaders and research leads to do.  First, if you are thinking about implementing Lesson Study it would be worth remembering there is more than one variety of Lesson Study.  In particular, I would recommend that you have a look at the work of Sarah Selezynov of the UCLIOE who identifies seven components of Japanese Lesson Study as this will allow you to  make comparisons between for want of a better phrase ' the original and cheap imports'.  

Second, and this is more generic advice, it's worth turning to the work of ( Miller et al., 2004) state when critically examining whether to implement change or changes, which appear to be fashionable,  school leaders and school research leads, could usefully ask themselves the following questions.

• What evidence is there that the new approach can provide productive results. Are arguments based on solid evidence from lots of schools followed over time?

• Has the approach worked in schools similar to our own that face similar challenges?

• Is the approach relevant to the priorities and strategies relevant to our school?

• Is the advice specific enough to be implemented? Do we have enough information about implementation challenges and how to meet them?

• Is the advice practical for our school given our capabilities and resources?

• Can we reasonably assess the costs and prospective rewards (Amended from (Miller et al., 2004) pp 14-15

If the answers to these questions suggest positive outcomes, it may well be that school may have identified a change which has ‘legs’.  

And finally, if there is one lesson to come out of this discussion, it is that school leaders need to actively engage in evidence-based school leadership.  Failure to do so, will lead to resources being misused, time being wasted, workloads increasing and pupils not making the progress they deserve.

Reference

MILLER, D., HARTWICK, J. & LE BRETON-MILLER, I. 2004. How to detect a management fad—and distinguish it from a classic. Business Horizons, 47, 7-16

School Research Lead - From evidence to implementation

During this Thursday's #UKEdReschat there was a lively discussion hosted by @StuartKime which focussed on the implementation of research and the ingredients associated by with successful implementation

However, for me this cycle of implementation misses out a fundamental step - how to bring together all  the evidence and then make a decision about how to proceed. For as (Alonso-Coello et al., 2016) 

Often the process that decision-makers used, the criteria that they consider and the evidence that they used to reach their judgments are unclear.  They may omit important criteria, give undue weight to some criteria, or not use the best available evidence.  Systematic and transparent systems for decision-making can help to ensure that all important criteria are considered and that the best available research evidence informs decisions.  P1

Adopting some form of 'evidence' framework can have a number of benefits for school leaders. (Alonso-Coello et al., 2016) subsequently identify a number of such benefits, which I have adapted for the use in a school-setting.

• You and your fellow decision-makers, will have an improved understanding of the advantages and disadvantages of the various actions being proposed
• Helping ensure that you include all important criteria in the decision-making process
• Providing you with a concise summary of the all best available evidence – be it research evidence, school data, stakeholder views and practitioner expertise
• Helping colleagues will be in a better position to understand the decisions by senior leadership teams and the evidence supporting those decisions

As mentioned in a previous post  evidence-based school leaders make explicit the criteria they use to make a decision.  In the context of your school, these criteria may well change depending on what domain and sub-domain of school leadership and management you are concerned with (Neeleman, 2017).  The criteria for making decisions about teaching and learning – may well be different to the criteria you apply to making financial decisions.  In addition, you may want to take into account whether criteria are adjusted for different parts of the organisation.  In the context of your school, the criteria being applied at say level of the Board of MAT Trust, may well be different to how the criteria are applied at Key Stage 1 in a primary school.    Using (Alonso-Coello et al., 2016) as a starting point, let’s look at some of the criteria that could be applied to decision-making (see Figure 1)

 Figure 1 Evidence to Decision Template

Element	Criteria
Priority of the problem	Is the issue an important problem for which a remedy is sought and that can be locally implemented?
Benefits	How substantial are the desirable anticipated effects?
Costs	How substantial are the undesirable anticipated effects?
Certainty of the evidence	How robust and secure are the different sources  - research, practitioner expertise, stakeholder views and school data - of evidence?
Balance	Does the balance of the desirable and undesirable effects favour the intervention or the comparator?
Resource use	How large are the resource requirements – attention, time, money, professional learning? Does the balance of costs and benefits favour the intervention or the comparator?
Equity	What impact does the decision have on educational equity?  Will it help close gaps in attainment? Are there important ethical issues which need to be taken into account?
Acceptability	Are there key stakeholders – teachers, parents, trustees, who would not accept the distribution of the benefits, harms and costs? Would the intervention adversely affect the autonomy of teacher, department, school or MAT?
Feasible	Are there important barriers that are likely to limit the feasibility of implementing the intervention (option) or require consideration when implementing it? Is the intervention or strategy sustainable?
Additional comments and recommendation

Furthermore, application of the above framework needs to be seen in the context of the strength or otherwise of the evidence - be it research, practitioner expertise, school data or stakeholder views.  However, that is another discussion which I will explore in a future post.

And finally

Having reflected on the EEF school improvement cycle - it seems to me insufficient attention is being paid to how you turn evidence into a decision and the processes necessary to support evidence-based decision-making - and this represents a fundamental flaw in the five stage process put forward.  Evidence-based decision making is so much more sophisticated than a simple model of priorities, external research, implementation, evaluation and mobilisation - and involves critical appraisal of multiple sources of evidence, aggregation of that evidence, and subsequent integration of the evidence into the decision making process.


References

ALONSO-COELLO, P., OXMAN, A. D., MOBERG, J., BRIGNARDELLO-PETERSEN, R., AKL, E. A., DAVOLI, M., TREWEEK, S., MUSTAFA, R. A., VANDVIK, P. O., MEERPOHL, J., GUYATT, G. H. & SCHÜNEMANN, H. J. 2016. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ, 353.

NEELEMAN, A.-M. 2017. Grasping the scope of school autonomy: a classification scheme for school policy practice  Belmas. Stratford on Avon, England