This blog article was written prior to LEO Learning becoming part of GP Strategies.
Truly effective learning measurement is about what you do once you have the data and insights at your fingertips.
Once the data is flowing, L&D teams need to be able to leverage those insights in powerful, useful ways. Whether that’s to demonstrate business impact or to refine the delivery of digital learning programs, the insights gleaned from learning measurement must be leveraged in a systematic way that supports a culture of continuous improvement.
We know that sometimes organizations struggle to understand how to leverage learning measurement insights effectively. Below are simple steps that turn one-off insights into a continuous improvement process:
Leveraging Data Insights: Our Step-by-Step Process
You can apply this process in different ways depending on the maturity of your measurement strategy and tools available.
Getting Started: Describe What Happened
At the most basic level, use your learning data to describe the level of learner engagement and what happened as a result of a learning program. This could be using data from your LMS to find out how many learners completed the learning and identifying to what extent they applied their learning back in the workplace, using feedback and evaluation from managers or on-the-job assessment tools.
Don’t dismiss ‘happy sheets’—with the right questions they can provide very valuable information. And remember, the Kirkpatrick four-level framework is reasonably robust, and there are a myriad of techniques to collect qualitative and quantitative data that don’t involve a data analytics tool. Check out Brinkerhoff, for example.
Develop Easy-to-Understand Reports
Create focused, easily-digestible reports that tell the story of what you discovered in an engaging way. Focus on what’s been discovered and the next steps i.e. how you will use these insights to tweak approaches
“Good” Data Insights in Practice
Taking a sales training program as your focus, identify the impact the learning has had on on-the-job performance using a small data set, such as LMS data, manager feedback, and sales figures pre- and post-learning completion. Share your findings and posit how you can improve outcomes with refinements to learning design and delivery. Apply those changes to your next learning cohort.
Use Learning Data to Describe What Might Happen
Deepen your level of insight by moving from describing what has happened to what might happen. Develop a set of hypotheses during the learning design phase which can be thoroughly tested with learning data.
Broaden your approach to gathering insights by including data from a range of learning activities and look at different sources of performance and business data. This broader approach enables not just the ability to test what might happen but also enables your team to look at data to identify unexpected or unforeseen insights and patterns.
At this level, you’re looking to identify ways to make systematic improvements to learning design, and apply and test these insights in learning delivered to other learner groups in the business. This includes combining insights on learning impact with business data to understand the role learning plays on employee performance.
“Better” Data Insights in Practice
Continuing the sales training example above, ‘better’ here would be testing and then understanding which parts of the learning program had the most benefit and applying this approach to other related programs, territories, or job roles in the sales function. You would then examine the impact these changes had on these new test cases to further validate the approach or understand the difference in effectiveness.
Use Insights to Predict Performance
At the highest level of maturity, you should be looking to predict the impact a learning program will have on learner performance, having extensively modeled and tested different approaches to design and delivery across a range of learning activities and learner groups.
This insight should be used to engage in dialog with senior stakeholders on how learning can impact business performance by developing staff capability at a highly-targeted level.
Zone in on Data-Driven Learning Design and Delivery
While ‘Better’ is expanding your approach to gathering data insights by adding more data sources, at “Best,” L&D teams should be looking to zone in and take a more incisive approach to drilling into data. Once you’ve got to grips with the data that’s available to you, you can start identifying exactly which data sources can be used to refine and inform your approach to learning design and delivery.
That could mean looking at data to understand how to deliver more personalized learning based on a learner’s competence or confidence level, or using data to understand collective weaknesses across a specific function and designing learning to bridge that gap.
At this level, your data insights should be informing the design and delivery of learning and feeding into a culture of continuous improvement across the organization.
“Best” Data Insights in Practice
Continuing our sales training example, at this level, data insights would be used to understand areas of weakness in sales executive performance and suggest improvements not just to training but on accompanying business processes that may be impacting performance.
Learning data could be used to understand the factors that influence the impact of learning based on specific sets of criteria. This insight is then used to deliver highly personalized learning geared to individual preference and competency levels, driving down learning time and delivering efficiency savings to the business.
About the Authors
