RSS icon

Top Stories

0503 Questionmark

March 1, 2005
Recommend (0) Comments (0)
Related Topics: Latest News
Reprints

Scenario
T-Mobile's state-of-the-art technology and highly visible "Get More from Life" advertising campaign have helped this mobile telecommunications provider rapidly grab market share in the United States. Behind the scenes, the legions of engineers who deploy and operate T-Mobile's network face a continuous learning cycle to hone their skills and stay up-to-date on changing technology. Over the past couple of years, the corporate trainers charged with implementing T-Mobile's technical training have had to maximize resources. Looking to develop a large training division at T-Mobile, T-Mobile's senior manager of Engineering and Operations Education Services Mike Bennett had to face some tough economic realities very early in the game.

Although T-Mobile has become one of the country's fastest growing nationwide wireless carriers, providing voice, messaging, and high-speed wireless data services to nearly eleven million customers, the industry grew very quickly and the entire wireless sector faced difficult competitive and economic challenges.

"When we started three years ago, we were going to have instructors, classrooms, and training labs," said Bennett. "Then along came some difficult economic issues in the tech sector and our great dream of a learning organization boiled down to tapping three people in engineering."Bennett's group supplies education services to both engineering and technical services. The training they provide is "99.9%" technical training that they deliver to over 2,000 technicians all over the country. Since keeping technicians current on rapidly changing technology is critical to the company, they needed access to training on a regular basis.

"For every hardware platform we have, we get one to two software upgrades a year," he said. "Some of them are huge and this is just engineering. We have a person tasked with nothing but software upgrade training around here."

Solution
As a result of their budget constraints, Bennett and his team realized immediately that they would need to turn to web-based course delivery. The advantages were clear. They could provide on-going technical education to a far-flung group without incurring travel expenses. The training group has a demanding training requirement that includes orientation on new products and technologies.

Since T-Mobile's engineering group didn't employ any trainers the company had to rely upon its vendors—companies such as Nortel, Ericsson, or Nokia—to provide third-party training materials and instructors.

While the e-learning tool that Bennett selected offered an excellent, real-time method for broadcasting live classroom sessions over the web complete with synchronous slides, Bennett still wanted more.

"I needed something that was going to be able to do surveys, do assessments, do them on the web via web view and get immediate results," Bennett indicated. "I needed to be able to look immediately at our results."

T-Mobile chose Perception from Questionmark as its assessment product. It provides its customers with everything they need to author, administer, deliver, and report on computerized assessments. T-Mobile administers assessments via the Internet using Questionmark's most popular package, Perception for Web.

Instant Results
To illustrate why instant results are so critical, Bennett recalled an example from the first time T-Mobile had used its classroom tool and Perception together. The company had created a course followed by an assessment to determine if the technicians had mastered the material. Almost every single technician got one particular question wrong. When the course was repeated for a different group of students with a different instructor that same day, T-Mobile got the same result.

"The two instructors got together and looked at whether it was the question, the answer, or the content," he said. "They determined it was the content and fixed the problem. When we ran the same course again a week later, we had a good answer dispersal on that question. So, in less than 24 hours, we fixed an important content problem."

Live Reporting
Bennett also likes the feature that enables him to get truly live reporting. "In the middle of an assessment we can look at a report, click the refresh button, and actually see the scores coming in. We want such instant results."

T-Mobile also uses online assessments and testing to save money in a rather interesting fashion. They assign a particular course to technicians and then give a follow-up exam, right after the test and online. Only those who do well on the "readiness" test are eligible to continue in a live course.

"The student takes the first three days of the seven- day class as web-based learning. Then they take a Perception high stakes assessment, which they have to pass at the 85% level," Bennett said. "If they succeed, they become eligible to go to Chicago for the remaining four days of the course. Of course students can review course content and try again. Our main objective is learning and skills improvement."

As a result, only properly prepared employees can take the more expensive portion of the training, which includes travel and hotels. However, because T-Mobile purchases an unlimited, perpetual site license, as many people in the company who want to can take the pre-work and still learn. Therefore, the company gets the best of both worlds, impossible without obtaining instant and accurate test results using a full randomized testing instrument.

Cost Savings
Another cost saving feature for T-Mobile and others relates to web testing. Filling out a paper test sheet, getting it scored, and putting those results into a database is an expensive proposition. Costs range from $15 to $30 per student, simply to administer the test.

When any company uses online assessments security is always a concern, particularly controlling cheating. With his small staff at T-Mobile Bennett was unable to proctor the exams, especially since they are being taken all over the United States. And he does a lot of assessments. Plus, Bennett's success with Perception has caught the attention of other business units within T-Mobile, who sometimes "borrow" it for their own assessment needs.

High-volume Testing
In the course of one week, T-Mobile did 27,574 course completions and 95% of those used Perception. The week before that, there were 12,000. Subsequent weeks brought 7,000 and 11,000.

Questionmark President Eric Shepherd notes that the large number of assessments T-Mobile routinely deploys casts an interesting light on the situation Bennett described in which every technician in two separate classes got the same question wrong on a test. "If you were giving only a single test or assessment, discovering a flaw with a question or a gap in learning content wouldn't pose a major challenge. But on the scale that T-Mobile uses assessments, it's essential to have a reporting capability that enables you to rapidly analyze the validity of questions and assessments."

Given T-Mobile's high demand for assessment delivery, one of the features of Perception that Bennett particularly likes is the fact that every student gets a slightly different exam.

"The randomized, random picked pool type of question, randomized, double randomized, turn-it-upside-down kind of assessment means that no two students get the same test. So your cheating level goes way down," he commented.

Products such as Questionmark Perception settle the most pressing issue faced by trainers who use the web: do students learn more in a classroom or online? Bennett feels that at his company anyway that debate is settled. "Critics say, 'Prove you have learned something,' and Perception is our proof that students have."

Comments

Hr Jobs

Loading
View All Job Listings