Proving 900 Percent ROI

 here are five areas you can measure to determine the ROI of e-learning, says David Posner, IT project manager and learning measurement specialist at Cisco Systems:

    1. Efficiency: Training helps you do your job faster. For example, a call center representative takes a customer-service training course where he learns to handle irate customers more efficiently, and as a result he spends less time per call and can handle more calls per day.

    2. Effectiveness: Training helps you do your job better. A recruiter may spend the same amount of time interviewing applicants after taking a behavioral interviewing class, but because of the training now she makes better hiring decisions.

    3. Cost avoidance: The money you arent spending on travel, lodging, and time away from the job for classroom training. You cant call this cost savings because after the first year its no longer part of your budget, Kelly says.

    4. Customer satisfaction: how the training impacts the trainees relationship with customers. This is related to effectiveness but its not the same, Posner says. Its the overall impact training has on the customers perception of you.

    5. Business revolution: Youve learned something that radically changes the way you work. For example, the sales staff learns a completely new approach to team selling that changes the way they sell and the way they are rewarded. This one is rare and hard to measure, Posner says.

    Its most important to measure the first two, to be sure training is causing an improvement in quality as well as quantity. Time savings alone is useless if quality is negatively impacted, Posner says.

    There are no absolute formulas for measuring e-learnings value, but by combining hard data and self evaluations, you can get a reflection of the overall impact training has on the business.

    Both types of data are equally flawed for different reasons, he says. Quantitative data, such as hours of training completed or increased sales, doesnt take into account other factors that can impact behavior. For example, if a sales rep takes a course on closing deals, then sells $800,000 over his quota the following quarter, is the increase due to training, or is it a result of a change in the economy, or a new product roll out? Its difficult to isolate those factors, Posner says.

    And in most cases it would take too much time and effort to compile that data. Self-assessments are equally flawed because people are inaccurate about judging their behavior. But self-assessments are easier to gather, and when you collect enough people to rate the quality of a course, youll get an overall impression of its value, Kelly says. Its not completely accurate but its reliable.

    Cisco uses this approach to gather data on the impact of e-learning on employees and vendors. Internally, through a system developed by DigitalThink, a San Francisco-based provider of custom e-learning solutions, employees whove used a course within 30 days receive e-mail requests to complete a short Web-based survey. The survey asks users to estimate whether the training saved them time on the job, and whether it improved the quality of their performance.

    For each question, employees select a percentage range that reflects the time savings or quality improvement theyve seen since taking the course0-20 percent, 21-40 percent, etc. These are used to calculate ROI. For example, a Cisco recruiter may spend 20 percent of her time doing initial interviews. After taking a course on how to select ideal candidates, she becomes more discerning about whom she sees, reducing the amount of time she spends on first interviews to 10 percent. If she makes $80,000 a year, the return on the training was $8,00010 percent of her salaryminus the cost of the course per user, Posner says.

    He sees about a 10 percent response rate to his surveys, which are immediately fed into a database that he can accesses periodically to calculate ROI. To make the results more conservative, Posner uses the lowest percentage in each range for his measurements.

    Based on the data hes collected over two quarters using this survey, the ROI for e-learning internally at Cisco is 900 percent per course. In other words, for every dollar Cisco spends on training internally, the company sees a $9 gain in productivity.

    On average employees see a 24 percent increase in efficiency as a result of training, he says, and even those who get no time savings see a 17 percent gain in effectiveness. Thats amazing information to have, Posner says. And, even if employees see no improvement in their efficiency or effectiveness, there is still the empirical data that shows the time savings of e-learning over the classroom, so no matter what youll come out ahead.

    The impact of e-learning to Cisco vendors is even more impressive, Kelly adds. Based on self-assessments completed by 3,560 Cisco partners who used Ciscos partner E-Learning Connection training portal in 2001, most of them had observed significant growth in productivity. For example, 74 percent reported an improvement in the ability to sell or service clients, and 76 percent said that customer satisfaction jumped 50 percent or more thanks to the E-Learning Connection customer service training.

    These numbers are a pretty significant indicator of e-learnings impact. Kelly says.