Forums
Metrics - Measure Training Effectiveness
Training & Organizational Development
Metrics - Measure Training Effectiveness
A forum for exchanging ideas about skills training, leadership training, management training, compliance training, e-learning, as well as organizational development and effectiveness.
While I've been a trainer for several years, the companies I've worked for have not implemented metrics to measure training effectiveness.
I'm now with a new company who would like to get metrics i
1
Cat:Topic ForumsForum:ForumId58
Cat:Topic ForumsForum:ForumId58Discussion:DiscussionId34199
Forums » Topic Forums » Training & Organizational Development » Metrics - Measure Training Effectiveness
Forums
»
Topic Forums
»
Training & Organizational Development
»
Metrics - Measure Training Effectiveness
|
Metrics - Measure Training Effectiveness
posted at 1/28/2008 1:38 AM EST
|
Posts: 1
First: 1/28/2008 Last: 1/28/2008 |
I've been doing a lot of research on evaluation lately, and I've decided that nearly every article, theory, approach out there is just a rehash of Kirkpatrick's work. Even the '5th level' offered by Phillips is less a new level of measurement than it is a way to pull the data from the other levels together into a detailed report for a client. To my way of thinking, my time is best spent looking for ways to adapt Kirkpatrick's work to my training designs than in trying to develop something new.
These are just my personal opinions and musings, and are not intended to judge how others are measuring their work. k |
|
|
Metrics - Measure Training Effectiveness
posted at 1/28/2008 10:02 AM EST
|
|
Posts: 544
First: 9/27/2004 Last: 9/13/2011 |
I agree with ksucidlo, I have spent the last few years looking for alternate training evaluation models. Everyone I found is either a more detailed focus on one of the four "levels" or it is a model that has elements of Kirkpatrick embedded in it. Elwood Holton has an excellent model he revised in 2005 that falls into that category, but it is probably more suitable for research than practice.
The main advantage of Kirkpatrick is that people are familiar with it, but again, I would make sure you had some other types of metrics (cost, attendance, need, business goals, etc.) to provide some context. |
|
|
Metrics - Measure Training Effectiveness
posted at 2/27/2008 6:35 AM EST
|
|
Posts: 1
First: 2/27/2008 Last: 2/27/2008 |
While I agree that Kirkpatrick is the gold standard for measuring training effectiveness, I have a slightly different issue: how do we persuasively measure the effectiveness of our learning *strategy*? I think that this is a crucial issue that seems often overlooked - measuring the effectiveness of a particular course or program is important but intrinsically tactical. If we want to be viewed as strategic, don't we need to be able to speak to that conceptual level?
I can think of several ways to approach the issue, but they all seem to be subject to a lot of extraneous "noise", external factors that also impact the measure. Does anyone else have any thoughts? |
|
|
Metrics - Measure Training Effectiveness
posted at 2/27/2008 7:22 PM EST
|
|
Posts: 108
First: 4/15/2007 Last: 8/17/2009 |
KBascom, you raise an interesting and important question. Firstly, the problem of extraneous noise is a problem with measuring at Level 4 of the Kirkpatrick model as well, so in this regard is not peculiar to measuring the success of a strategy. So, the usual methods of isolating external factors, such as trend line and control groups, are applicable here.
My second observation is that strategy has no concrete impacts on an organization, except through the tactics employed to enact the strategy. So, to measure the impact of the strategy, we may need to measure the impact of the tactics. And in large part, are not the tactics used in the learning strategy the specific programs employed in the organization? Some of these programs will be the traditional training program, such as a classroom based leadership program and an e-learning module on harassment prevention. With these, Kirkpatricks model applies, and ipso facto, the results cascade up to reflecting on the impact of the strategy. Other parts of the strategy may not be enacted through a training program as such, but through an organizational initiative. It could be to improve the learning capability of the organization (strategy) through hiring more professional instructional designers (tactic) or upgrading the e-learning infrastructure (tactic). Once again, the success of the strategy can evaluated through evaluating the impact of the tactics. And this can be done using traditional evaluation methods (ROI, benefit-cost ratio, NPV, etc). Here, once again, the external factors can be isolated using trend lines, control groups, etc. The model also contains a number of KPIs (and an assessment tool) for determining the success of the strategy. This is one view of the training world. However, it does illustrate how the impact of a learning strategy can be measured. I may be way off the mark in addressing your question. I dont like to speak in generalities, so Ive tried to give a couple of examples of what I am talking about. If you mean something different, please tell us about one or two of your strategies and we can discuss in more concrete terms. Les Allan Author: From Training to Enhanced Workplace Performance Business Performance Pty Ltd http://www.businessperform.com |
|
|
Metrics - Measure Training Effectiveness
posted at 2/28/2008 9:08 AM EST
|
|
Posts: 544
First: 9/27/2004 Last: 9/13/2011 |
By learning strategy, do you mean how well your delivery methods/system fit with learners' needs? Or do you mean how well learning fits with other organizational goals? Or something else entirely?
|
|
|
Metrics - Measure Training Effectiveness
posted at 5/13/2008 8:16 AM EDT
|
|
Posts: 1
First: 5/13/2008 Last: 5/13/2008 |
The dreaded Levels! Here are our recent experiences that had it not been for some customer nudging )we're the training vendor) we probably never would have done to these lengths to measure.
Not only are Levels 4 and 5 (Jack Phillips) important measures of and L&D initiative, they are achievable if time and resources are applied internally and from external vendors to ensure they are tracked. We know this because as a vendor, we were driven by a Top 10 fortune 500 client to "prove it" or leave. Although the tracking was a bit consuming, we achieved a 167% first year ROI ($26 million closed on a $397k total investment). Leading to a 269% 3-year ROI! We knew our solutions were powerful, but certainly not to these sort of numbers! |
|
|
Metrics - Measure Training Effectiveness
posted at 5/30/2008 1:06 AM EDT
|
|
Posts: 4
First: 5/30/2008 Last: 5/30/2008 |
Hi Frisquee,
By using Donald Kirkpatrick Model is good enough to evaluate the training effectiveness. Iesallan, I'm a great fan of of the Kirkpatrick also. Lately, I have attend a course; which we are using the Evaluating the impact as the stage 1 to decide the Training Programme; then following by the Stage 2 exam the application to the job; review the learning evaluation and lastly we focus on the reaction. It's works. |
|
|
Metrics - Measure Training Effectiveness
posted at 6/4/2008 7:05 AM EDT
|
|
Posts: 148
First: 9/30/2004 Last: 12/12/2008 |
Go to www.ispi.org to learn more about human performance technology and how to evaluate the gaps in performance.
ISPI was founded by Geary Rummler who was one of the first people to promote the idea of performance engineering. In order to evaluate, you must first identify what need to be changed, and how you will try to effect change. Best, vivlin |
|
|
Metrics - Measure Training Effectiveness
posted at 6/26/2008 6:46 AM EDT
|
|
Posts: 2
First: 6/26/2008 Last: 6/26/2008 |
Some great information here! I agree with all the comments. I do have one criticism that comes with 18 years exeperience. Who is the recipient of your measures? If it's management, they probably don't know anything about any of this.
First, find out what they measure and value. Find a way to link your measurements to theirs. If they want to know "butts in seats", you better measure it even though I detest it as a learning professional. Second, educate them slowly on learning effectiveness measurements. I've seen too many organizations spend too much time measuring only to find out no one is even using the info. You become isolated from the operation and less valued even with the best intentions. |
Forums » Topic Forums » Training & Organizational Development » Metrics - Measure Training Effectiveness
