Video Training and Testing Supports Customer Service Goals
In 1986, the customer-service and the domestic ground operations divisions at Memphis, Tennessee-based Federal Express Corp. asked us in the human resources development department to create a mandatory performance-improvement program for all of the company's employees who deal with customers either face-to-face or over the phone. Their primary goals were to completely centralize the development of training content while decentralizing delivery and to audit the employees' ability to retain what was learned.
We developed a pay-for-performance program that consists of Job Knowledge Tests linked to an Interactive Video Instruction (IVI) training curriculum accessed on workstations in more than 700 locations nationwide. More than 35,000 Federal Express customer-contact employees around the country, who fall into one of five job groups, are required to take the Job Knowledge Tests annually via computer terminals at their work locations. The tests, which measure employees' knowledge in their specific jobs, correspond with employees' annual evaluations. In fact, the results of the tests make up approximately one-tenth of the employees' performance ratings.
By testing customer-contact employees on product knowledge, services, policies and various aspects of their jobs, Federal Express incurs several benefits. For one, all employees operate from the "same book," ensuring that all customers will receive accurate and consistent information during each transaction. This helps the company maintain its high service levels and commitment to quality. In addition, managers have an objective way to measure job knowledge for all customer-contact employees.
Design of Job Knowledge Tests allows for constant updating.
Federal Express developed the testing program, called QUEST 2000 (Quality Using Electronic Systems Training), internally. We spent a great deal of time and effort ensuring that all of the Job Knowledge Tests are valid, relevant, fair and meet testing standards outlined by the Federal Uniform Guidelines on Employee Selection for developing instruments used for pay.
Focus groups made up of trainers, managers and job incumbents designed each of the tests, which consist of 90 multiple-choice questions pertaining to all important aspects of employees' jobs. Based on the members' collective knowledge, the focus groups created surveys listing the critical tasks for each job. We then asked workers within those jobs to rate the tasks in order of importance. That's how the objectives for the tests were developed.
Then, the focus groups wrote the test questions based on these issues. We were careful to include only questions that directly pertain to activities in which workers engage.
The final step before implementation was to conduct some pilot testing. At this phase, we had subject-matter experts and an on-staff industrial psychologist examine any questions that might be construed as unfair based on numbers of workers who missed them.
The entire process—from focus-group formations through test validations and implementation, including programming the tests to be taken via CRTs—took approximately 15 to 18 months.
Now that the tests are in place, the current challenge is to keep the tests timely. To accomplish this, the original focus groups meet quarterly. We discuss existing test questions to ensure that they're still valid. We also spend this time writing new questions. We've built up a bank of several hundred questions for each test. If questions are eliminated, we can pull from the bank and insert equally weighted questions on the same topics.
The test's design also allows us to update it daily if necessary. There are two people on staff who are trained in the computer-based testing areas. They can access the QUEST program at any time to take out questions and insert new ones. However, they do this only by approval from the focus group for each specific job group and the industrial psychologist. (Everything that the committee approves has to be sent through the psychologist. He makes sure that the questions have followed the guidelines, that they're written properly and that they aren't discriminatory.)
The QUEST automated program saves hours in clerical and administrative activities because the computer does all of the scoring, recordkeeping, item analysis and score reporting. Additional features of the program are real-time registration, real-time test-score reporting and item analysis.
Interactive video training correlates with job testing.
To prepare employees for their Job Knowledge Tests and keep them consistently informed, Federal Express implemented Interactive Video Instruction in 1987. Using the system, employees can study about their job, company policies and procedures, and brush up on customer-service issues by reviewing various courses.
Currently, we have 1,225 Interactive Video Instruction units placed at more than 700 Federal Express locations. All workstations are linked to the Federal Express mainframe in Memphis. Student and management reports can be stored and generated because of this link. Also, we in human resources development can track usage of the system from a corporate- and an individual-location basis.
Each Federal Express location has 21 video disks that make up the customer-contact curriculum (see "Customer-contact Workers' Training Curriculum"). There is virtually no subject or job-related topic that the customer-contact workers can't find on the Interactive Video Instruction platform.
We develop the IVI programs in the same manner that we developed the Job Knowledge Tests. In fact, we use the same job analysis done by the focus groups in the initial stages of test development. We look at the critical tasks that the focus groups identified to build the basic course objectives. We then design the program using current training materials from the company's stand-up instructional courses.
A vendor produces the video programs after meeting with us. The vendor writes the scripts, develops the video segments and shoots the videos. This whole process takes approximately eight months.
Once the programs have been written, it's imperative to keep them updated. The work force relies daily on the fact that the system can provide accurate and current information: For them, out-of-date information is worse than no information at all. Therefore, changes must be continually incorporated into the curriculum to protect the Federal Express investment.
For these reasons, we send a CD-ROM to each location every six weeks. This CD updates the curriculum through text, PC graphics and digital audio. Approximately 1,200 updates are made on an annual basis.
Video instruction saves time and money.
The benefits of Interactive Video Instruction are numerous, but some of the more apparent benefits are:
- Training time is compressed from stand-up instruction
- Large numbers of employees now can be trained in remote locations
- Information is standardized
- Travel expenses are reduced
- Modeling and simulations can be provided
- Employees have access to real-time training
- Individualized and prescriptive instruction is available
- Competency is measured by a 90% score on all Interactive Video Instruction assessments.
In addition, the system enables the training to be linked directly with the testing system. In fact, it generates two important reports that are specific to the Job Knowledge Test. The Test Score Report shows employees exactly how they scored on their Job Knowledge Tests, indicating if they rated satisfactory or unsatisfactory. This report also gives a breakdown of specific topic areas of the test and the employees' scores as well.
Based on the Test Score Report, the Student Prescription provides the employee with information about specific Interactive Video Instruction courses and manual resources to study for remediation. For example, it will provide a list of video disks and manual resources for every category in which an employee scores low.
Both of these reports print out at the actual site where employees take their tests. Supervisors also receive copies of the reports. The human resources department receives an additional report monthly that lists who has completed their prescriptions.
It's a corporate policy that workers complete their prescriptions within 45 days of their tests. Most workers have no trouble complying. For one, employees are paid for two hours of test preparation or study time prior to each test, two hours of test time and two hours of post-test study time. In addition, internal surveys show that employees prefer interactive training to traditional classroom instruction because it's interesting, fun, requires less time and is more directed toward the individual student.
The current average amount of time that workers nationally use the Interactive Video Instruction program is approximately 132,000 hours per year. Compared to traditional training, this equates to approximately 550 one-day classes, averaging approximately 30 employees per class. Yet, no trainers are necessary and no travel costs are incurred.
The success of the video program prompts future usage.
Internal studies have shown Federal Express that our system for just-in-time training linked to Job Knowledge Tests works. These studies have shown that since the implementation of the system, training time on some modules has been reduced by 50%, with no loss in retention or quality of training.
For example, the training time for the "Journey Toward Quality" program, which is a general course describing the company's total quality management concept, has been reduced from 32 hours of stand-up instruction to eight hours of CD-ROM instruction—a 75% reduction. Training time for a specific job course—Dangerous Goods—has been reduced by 50%, from four hours to two hours.
On top of that, studies indicate that since the implementation of interactive training, Job Knowledge Test scores have increased an average of 20 points. They also show that locations that have higher usage of Interactive Video Training have higher Job Knowledge Test scores. Also, when correlating test scores and performance evaluation ratings, Federal Express learned that, in general, the employees who have the highest scores on the tests are the company's better performers.
The success of the program has been so encouraging that we're currently creating a similar voluntary Job Knowledge Test program for the company's airline mechanics. The vice president of that job area requested a system that would create a consistency of training and evaluation for the people who work on the aircraft, and that would enable management to properly compensate those workers willing to voluntarily take and pass a test.
The program will support the aircraft maintenance department's efforts to improve dispatch reliability, improve the mechanic's technical abilities and provide an incentive-based pay system. It's due for completion sometime this summer.
The process for designing this program is identical to the process we used in creating the testing system for the customer-contact people. During the process, workers from the aircraft maintenance department have expressed similar concerns to those brought up by customer-contact employees. These are issues such as:
- Will the test be fair?
- What are the consequences of the test?
- How are the questions going to be written?
We're addressing these concerns in focus groups, made up again of managers, trainers and job incumbents.
Although the new program will be voluntary, Federal Express is convinced that it will be just as beneficial to our workers as the current Job Knowledge Test program in providing tools for the employees to better serve customers and improve job performances.
Further, the company believes that its philosophy of "train to the job, perform to standards, and test for competency," provides customers with a value-added insurance program that translates into outstanding service and a competitive edge. A well-trained, knowledgeable and empowered employee supports this philosophy and the company's goal of 100% customer satisfaction.
Personnel Journal, June 1994, Vol.73, No. 6, pp.47-51.