Important Learning Metrics and How to Capture Them

Imagine that you create a learning module or course. You have a goal that you want to achieve to begin with, you go to great effort to plan how you will best deliver it and you end up with a great final product! Nice!

Imagine it is now 6 months down the track…

Has your module been completed as required?

Was the solution valuable for the learner?

Has the course changed the behaviour of your learner as desired?

Have you met your goal?

These are very important questions, though surprisingly are not always explored post the release of a learning solution. Creating a learning solution without testing its impact is like giving a person glasses for poor eyesight and not checking in to see if it has actually improved their vision. It doesn’t make much sense. You don’t want to spend time and money on a learning solution that doesn’t provide a good return (Mindtools, 2018).

We have discovered a useful framework that you can use to measure your learning solutions. This blog will explore each stage of the framework and provide practical tips on how to implement it in your organisation. 

The framework is Kirkpatrick’s 4-Level Training Evaluation Model.

The four levels of evaluation are:

1. Reaction

2. Learning

3. Behaviour

4. Results

Level One: Reaction

This level of evaluation focuses on how your learner reacts to your learning solution. Testing your learner’s reaction is a great way to gain insight into the efficiency of the delivery method of your course.

Did they enjoy completing the learning?

Did they feel that the learning was valuable?

Did all your learners attempt to access the learning solution?

Was the learning easy for them to comprehend and navigate?

How many of your learners completed the full learning solution?

Practical tips:

1. Track completion rates: Using your Learning Management System (LMS) check for completion rates (Watson, 2017). Are learners accessing the learning solution and completing the whole module? If learners are opening the module though are often not completing it, you could investigate why this is the case.

2. Have your learners complete the module in a testing room: Observe them to see if they are finding it easy to navigate and are maintaining focus. You can have a group discussion at the end of the testing to find out what people thought.

3. Distribute a satisfaction survey: This can be provided within the learning module or can be sent to learners once they have completed it.



Level Two: Learning

This level focuses on what your learner has learnt from completing the solution.

Did their knowledge increase because of the learning solution? (Mindtools, 2018).

Were the learning objectives obtained by the learner?

Practical tips:

1. Test/Assessment: Provide your learner with a test or assessment at the end of the learning solution. This will enable you to observe if the learner obtained the required knowledge.

2. Talk to the learner: Grab a coffee with your learner and have a chat about what they learnt. You can discover a lot through conversation and you may also receive other useful feedback at the same time.



Level Three: Behaviour

This is a very important level and evaluates the change in your learner’s behaviour, post completion of the learning solution. If you are implementing a learning solution you will want to see behaviour change. The last thing you want is for your learning solution to be completed and the learner to finish it, not make any necessary changes to behaviour and eventually forget the content.

Is the learner behaving differently because of the learning?

Practical tips:

1. Gather evidence: Have learners provide evidence of how they are changing their behaviour post the training. You can also obtain this from their supervisor (Watson, 2017).

2. Self-assessment/360-Degree assessment: Get the learner to assess how their own behaviour has changed since the learning. To gain a greater perspective, have the learner’s supervisor, direct reports and peers complete a review of them to gather information on any potential behaviour changes (Watson, 2017).

It is important to consider that behaviour change can happen (or not happen) due to many reasons that may be separate from the learning solution. If a learner does not change their behaviour post a course, do not assume that it is because the learning solution wasn’t effective. Explore if there are environmental factors impacting their behaviour change such as their supervisor not allowing them to change or the fact that they personally have no desire to change themselves and therefore choose not to.



Level Four Results

This level is all about the tangible and observable outcomes achieved by the learner completing the solution.

Are they doing better on their KPI’s?

Are business goals being met more effectively?

Are goals and targets being met as a result of the learning?

Practical tips:

1. Measure goals/targets: Is there a measurable goal linked to the behaviour change required from the learning? For example, if a learner was completing a ‘Communicating Well with Customers’ course, you could check to see if customer complaints aimed at them had reduced post the course. If a learner was completing an ‘Accurate Report Writing’ course you could check to see if errors in their reports had reduced.

2. Savings: Figure out the savings involved by the results achieved from the learning. For example, if the learner is now making less errors on reports, their supervisor is spending less time checking the reports and therefore how many hours of the supervisor’s time is saved as a result?



How do you record metrics at your organisation? Do you explore the return on investment when you implement learning solutions? We would love to hear your thoughts. We hope that this added value to your craft and we look forward to discovering new and effective ways to measure metrics into the future.

                                                         References

Mindtools. (2018). Kirkpatrick’s Four-Level Training Evaluation Model. Retrieved from https://www.mindtools.com/pages/article/kirkpatrick.htm

Watson. (2017). 6 e-Learning Metrics that Matter. Retrieved from http://www.gofluent.com/blog/6-elearning-metrics-that-matter/