Patrick Gallen, Partner, People and Change Consulting, Grant Thornton Ireland
Most firms record the number of training courses they offer their employees, the number of learners who complete them and the direct costs of these programmes. They tend to capture learner satisfaction with formal training as well through surveys or evaluation sheets that learners complete afterward.
However, fewer companies move beyond these basic efficiency measures to assess learning and development effectiveness, including the degree to which employees apply what they learn on the job, and the business outcomes that follow.
Only a small minority conduct a return on investment (ROI) analysis to assess whether the benefits of a training initiative outweigh the costs. This has become even more important during the pandemic, as organisations struggle with re-skilling their staff, coupled with the challenge of a finite amount of resources to deliver the greatest impact for their organisation and its people.
In my opinion the starting point for all measurement and reporting strategies should be to answer this question: Why do we want to measure?
In the 1950s, Don Kirkpatrick built on the work of Raymond Katzell to suggest four levels of training evaluation. In the 1970s, Jack Phillips altered Kirkpatrick’s 4th level slightly, adding a 5th level for ROI analysis. The 5 levels include:
Reaction – Level 1 is the ‘entry level’ measurement where participants complete questionnaires or evaluations after completing the training, often using Likert scales, with questions on content, relevance and quality of delivery.
Learning – Level 2 assesses the amount of learning that took place and might include tests after (or before and after) the training.
Application – In Level 3, organisations assess the degree to which learners apply what they have learned on the job. Staff may like a course and gain knowledge and skills from it, but if they don’t use what they learn, then the training will have limited value for the organisation in most cases.
Results and Impact – Evaluators at the Level 4 stage aim to assess whether the training had the intended results, such as an increase in sales or employee engagement. Kirkpatrick would argue that if Level 3 shows that salespeople make more calls/appointments following the training, for example, and then sales go up, you can reasonably conclude that the training contributed to more sales. Phillips would insist that you need to go further to determine the degree to which training led to the increase in sales.
ROI – A level 5 analysis tells you whether the benefits of a training programme justify its costs. This is a detailed and expensive method of evaluation to arrive at a benefit-cost ratio, by including not just the cost of the training programme, but all the other costs, including the opportunity cost of the time of those attending.
As organisations tend to mature in their use of measures, analytics and reporting with time and practice, learning proves no different. If the name of a measure starts with a number, it is almost certainly an efficiency measure.
As organisations advance on the maturity continuum, they need to balance learning efficiency measures with learning effectiveness measures. Learning Managers need to develop targets and goals with the CEO and the senior management team, so that efficiency and effectiveness measures apply to all training programmes; and outcomes apply to the more strategic ones.
This should then be encapsulated within a more holistic approach to Learning and Development measurement, where the learning measurement strategy within the organisation helps to focus learning efforts on outcomes that meet business leaders’ needs and expectations.