Facts:

The Logical Chain found that few schools evaluated the impact of CPD on teaching and learning.
Ofsted, 2006

Most evaluations seem to draw on the teacher, which can be superficial and lead to bias results.
Harris et al, 2006

The majority of CPD evaluations only look at participants’ reactions. Mostly widely used tool a survey and a questionnaire.

Research findings seem to suggest that schools need greater support and training in order to evaluate the impact of CPD.

Research findings suggest that schools need more support and training in evaluating the impact of CPD.

Only 24% of schools evaluate changes in pupil attitudes — making it the least frequently evaluated aspect.

Evidence suggests many schools still regard INSET days as the main form of teacher CPD.

Organisational change, value for money and changes in teacher behaviour were less likely to be evaluated.
Goodall et al, 2005

Effective teacher CPD improves teaching and learning and has one of the largest impact on student outcomes (Hargreaves, 1994 and Craft, 2000). This means that getting it right is crucial. But, when it comes to CPD, how do we know we are getting it right? It’s a topic I regularly raise in my initial consultation meetings with schools and other educational institutions and it’s often the case that the answer I get tallies with research that states that CPD evaluation is often a neglected step and that many school leaders struggle to carry out any sophisticated, in-depth analysis (Porritt, 2005 and Goodall et al., 2005). It is understandable as there is a reported lack of knowledge and experience needed to carry out such evaluation (Guskey, 2000 and Goodall et al., 2005). Furthermore a report by the Department for Education and Skills on ‘Evaluating the Impact of Continuing Professional Development’ (2005) found that only 24% of schools evaluate changes in pupil attitudes and a fewer than 10% of evaluation taking place rarely influenced the planning of any future CPD.

This worries me. It worries me in terms of ensuring that you get value for money from your training. It worries me in terms of planning using outcomes linked to CPD and, as a former school leader myself, it worries me in terms of ensuring the best education possible for learners. This is something I feel passionately about and why I believe seeking advice and guidance on how to measure CPD effectiveness is imperative. With that in mind, what follows are a few suggestions that can help to structure your thinking when considering how to evaluate your CPD.

The Purpose of evaluating the impact of CPD

One must consider from the outset not only the desired outcomes of CPD, but how these will be measured. What evidence will be obtained to determine and demonstrate that a positive difference is being made? Evaluation serves two purposes. Firstly, to identify whether the programme provides positive outcomes for a school (summative) and secondly, to identify how the programme itself can be further improved (formative). For me, building a long-term CPD programme is vital. It’s only when this is done that you can you accurately assess the gains, as well the next steps you can take.

Where to begin

Thomas Guskey’s five levels of professional development offer a template when thinking about CPD evaluation. These are as follows:

Level 1: Participants reaction

Will the information be useful? Did the material make sense? Was the leader knowledgeable and helpful?

Level 2: Participants learning

Did the learner obtain new knowledge and skills?

Level 3: Organisation, support and change

What was the impact on the organisation? What support was provided to initiate change(s).

Level 4: Participants use of the new knowledge and skills

How does the participant apply new knowledge and skills. How is this assessed?

Level 5: Student outcomes

What is the impact on learners? Achievement, confidence, attendance, behaviour, self-esteem.

Ask yourself whether the type of data you already use can fully answer the questions linked to each of the levels. If not, what changes can be made to make sure that they do? When consulting with schools, I’ve found this to be a useful activity to focus attention on a school’s current CPD programme and how evaluation can go towards improvement. For instance, research shows that the majority of CPD evaluations take place at Level 1 (participant reactions), straight or soon after the CPD programme has taken place. Although obtaining participants reactions is very important, only relying on Level 1 means that the evaluation is often brief, subjective and difficult to interpret. It is important to consider when planning your professional development how each level of evaluation can be put into practice, acted upon and the evidenced. Each level should build on what has come before.

Changing what evaluation means

Evaluation can often be a frightening prospect, however it should never be avoided due to fear of obtaining evidence that might show undesirable outcomes. Evaluations that focus on how teachers’ practice and embed new knowledge and learning from a professional development programme are invaluable for determining impact as well as the time and money spent. Furthermore, CPD evaluation should not be seen as a tick-box exercise for governors, inspectors or other external stakeholders. Investing in this process is about really wanting to improve pupil learning and the quality of teaching in your school.

Evaluating the Impact of CPD in your school

DosDon’ts
When planning CPD, determine from the outset how you intend to evaluate its impact.Do not just add evaluations practices at the end of your CPD programme or as an add-on. Determine from the start what children will learn differently as a result of the CPD activity.
Focus on measuring the difference it can make to teacher practice and student outcomes, rather than just the CPD activity itself.participants perceptions — this could lead to bias and very subjective results.
When using Guskey’s five levels framework for evaluating professional development try starting with Level 5 first and working backwards.Avoid focusing just on the professional development programme, the material and the training itself.
Use a range of quantitative and qualitative data — questionnaires, interviews, focus group meetings, observations, feedback sheets, reflection logs etc. Consider carefully the nature of questions, rigorous baseline.Avoid making CPD evaluations burdensome. With the right training, a practical and collaborative approach with the use of rigorous tools this can become quite straightforward.
Involve all participants in the process of the evaluations from the start. CPD evaluations are not the only job of the senior team.Do not begin any form of evaluation until you are clear on:
• The level of questions you will address (at each of Guskey’s levels).
• How the information will be gathered.
• What is measured.
• How this information then be will be used.

Despite its challenging nature, the long-term commitment and critical planning needed for evaluating the impact of CPD is critical if maximum gains for students are to be achieved — measuring the impact of training is an integral step on the journey towards ensuring the best training and the best outcomes.

Veema: signoff

Costa Constantinou (BA, MA, PGCE)
Director of Educational Services

References

  • Craft, A. (2000). “Continuing Professional Development: A practical guide for teachers and schools”. London: Routledge Falmer.
  • Edmonds, S. and Lee, B. (2001). “Teacher Feelings About Continuing Professional Development”. Education Journal, 61, 28–29.
  • Goodall, J et al. (2005). “Evaluating the Impact of Continuing Professional Development (CPD)” Department for Education and Skills.
  • Guskey, T.R. (2000). “Evaluating Professional Development”. Thousand Oaks, Ca.: Corwin Press.
  • Hargreaves, A. (1994). “Changing Teachers: Changing Times”. Toronto: OISE Press.
  • Harris, A et al. (2006). “What Difference Does It Make? Evaluating The Impact of Continuing Professional Development In Schools”. Scottish Educational Review, University of Glasgow, Volume 37.
  • Ofsted (2006). “The Logical Chain”.
  • Porritt, V (2005). “London’s Learning, developing the leadership of CPD”. Department of Education and Skills.