Evaluation

The F.P.I.E. process (pronounced ‘eff pye’) underpins The Change Consultancy’s approach to change projects. F.P.I.E. draws heavily on our collective experience; in particular, it taps into project management and business psychology practice. As a consequence we can offer substantial experience in consulting on change projects from start to finish. We now look at evaluation.
Evaluation

Stage Step
# 4 Evaluation (E) # 9 Evaluate the Selected Option for Change

“If the plan does not work, change the plan, not the goal” Anonymous

A drinks company, famous for its cider, had commissioned a development centre project from the proclaimed leader in its field. Two years later – having designed the development centre, trained its assessors, and run several centres – the Director of Personnel handed me the materials and asked me to evaluate them. I did.

In short, the product was very good, up to a point. As assessment for development materials they were fit for purpose. It was the feedback of outcomes that gave me cause for concern. Those being assessed received feedback based on a report that looked rather clinical. Gaps in capability were exposed, but there was little in the way of development guidance.

Effective development centres stand or fall on the quality of feedback. A large manufacturer with worldwide development programmes refers to its assessors as coaches, which nicely captures the essence of the role.

My one sentence summation to the Director of Personnel was, “Your development centre is a great assessment centre.” My written report contained a short list of recommended improvements to convert it into a development centre.

This case study sticks in my mind because it clearly fitted the category of evaluation. Although I had not been involved in the previous eight steps I agreed with the change goal, hence my recommendation to modify the chosen option in line with the original change goal.

Evaluation against the original change goals is such an important step. Not doing so denies the opportunity for corporate learning. Few would disagree with corporate memory being refreshed on a regular basis. Yet evaluation often does not take place, because the people involved are immediately moved on to their next change project.

The Change Consultancy believes that evaluation should be built into every organisation change project. We can support this with digital media, which meets one of our goals, that is, to provide simple-to-use change techniques for clients.
Take the Evaluation Matrix for example. It provides:

  • Bridges between diagnosis to implementation, or implementation to evaluation
  • Comparison of options with measures of performance
  • Measures of performance drawn from objectives

Evaluation Matrix: Worked Example
To increase the efficiency of the packing machine from 75% to 80% by 1st April. The total budget available is £20,000%.

Options
Measures of Performance #1 #2 #3
75% to 80% 76% 79% 83%
By 1st April Early March End March Mid May
Within £20,000 £10, 000 £20,500 £33,500

If there are no precise, quantitative measures, you can use:

  • Ranking (Best, Middle, Worst)
  • Weighting (Hi / Med / Lo)
  • Scales (1 to 10)

Worked Example: Less Precise Measures

Options
Measures of Performance #1
Improvement Groups
#2
Information On Waste
#3
Production Layout
#4
Machine Centre
Reduce Overall Costs
(High-Medium-Low and/or estimate)
Low Low Medium
£250k
High
£1.2m
Improved Communication
(Scale: 1 [Low] to 10 [High]
7+
High
6
Med/High
2
Low
2
Low
Increased Productivity
(Units per Shift)
Some Effect
Incentives May Be Required
Difficult to Estimate Some Effect
Layout Must Decrease Bottlenecks
Med/High
Labour Reductions

Here are a further three variations offered by the evaluation matrix:

  • Often reveals the need for more data for the remaining option(s)
  • You can choose more than one option, provided their combined benefits & costs can be assessed
  • Measures of performance can be differentially weighted
    (more important vs less important)

In Interactive #2 I wrote about the experiences of a famous-brand frozen food company. Six months after the assessment centres were completed the change team conducted an evaluation.

The change team were pleased that they had taken this route, because the transition of the successful staff into the training established for them was seamless. Indeed a lesson was learned; the only appointees who were struggling to make the transition were those originally labelled ‘marginal’.

Six months before, based on my experience of other assessment centre projects, I had advised that the ‘marginals’ might end up presenting the change team with a difficult decision. Hence, I suggested that reassigning the ‘marginals’ or letting the ‘marginals’ go, if they wished, might be best for all concerned. Six months later, the change team acknowledged this was a major learning point, not only in these special circumstances, but also with broader organisation issues in mind.

Finally, The Change Consultancy has a pragmatic outlook on organisation change. When we come across examples of good practice, we like to share them with Interactive readers. Outlined below is the ‘plus/delta’ used by a well-known USA paper products manufacturer. ‘Plus’ means ‘went well/keep’ and ‘Delta’, the symbol for change, means ‘didn’t work as advertised/change’.

I watched it being applied by a shift manager after leading a new selection system training session with team leaders. He walked up to flipchart and drew a vertical line down the middle, then a horizontal line near the top. This was followed by the symbols you can see below. He then called for ‘plus’ points from the group, followed by ‘delta’ points, which had dates and actions, with names, appended. The whole process took ten minutes.

+
  • ……………..
  • ……………..
  • ……………..
  • ……………..
  • ……………..
  • ……………..

It was a powerful demonstration that evaluation can be conducted quickly, while fresh in the minds of participants. It is a practice that works particularly well in team development situations. Ten minutes, that’s all it takes!

Stage # 4 : Evaluating Change Toolbox
Steps # 9 : Evaluate the Selected Option for Change
Hard Techniques Evaluation Matrix
Soft Techniques Plus/Delta

Discussion: Is evaluation, as described, sacrificed to the god of expediency? The word ‘blame’ is not mentioned by us. Nonetheless, how prevalent is blame, when evaluation does occur? In your experience, how could the evaluation process be improved?

Interactive builds The Change Consultancy’s library of articles about organisation change. It is edited by Tom Lindsay who is also a major contributor. Our heading of Interactive is deliberate, because The Change Consultancy is keen to engage with its audience. Each article contains Discussion points. If you wish to comment, then we cordially invite you to get in touch with Tom at tom@thechangeconsultancy.com

Tom Lindsay is a graduate Chartered Psychologist and MBA specialising in business psychology. In the 1990s he was a part-time Course Tutor of Planning & Managing Change for the North East Region of the Open University’s Business School. Prior to this he held personnel positions in blue chips and the NHS. For the past twenty eight years he has been a consultant concentrating on the people aspects of change. He is a director of The Change Consultancy.