Automated Assessment at a Large Company

Capital One moved to online testing mostly in response to the logistical demands of staffing its 8,000 call-center operator positions.

November 30, 2000
Credit card and financial services giant Capital One, which has 25 millioncustomers and $4 billion in revenue, prides itself on being an innovator, on thecutting edge of technology. The company wondered, then, why it was usingpencil-and-paper tests for pre-employment screening for call-center operators,says spokesman Hamilton Holloway.

    Last year, the company switched to automated onlinetesting, with stunning results. Overall cost-per-hire dropped 45 percent, whilethe pass rate increased by 30 percent. With this success, Capital One hasembarked on an ambitious new online testing program, planned for implementationin the second quarter of 2001 and predicted to save $3.7 million next year.

The problem
    Capital One found itself in the midst of rapid growth in1996, with the need for customer-service reps growing 30 to 40 percent eachyear, says Laura Lorenzen, director of global recruiting. Threepencil-and-paper tests were used for pre-employment screening: a cognitiveskills test; a math test; and a "bio-data" job-history test, used topredict future job stability.

    "In Tampa, we were having to process severalthousand people a month just to hire 100," Lorenzen says. Administeringthe test and grading by hand was time-consuming. If timing of the test-giverwas off just a little, comparative results of other tests were skewed."It was cumbersome and inefficient," she says.

    The traditional method caused another dilemma. "Ifcandidates didn’t get a certain minimum on any of the three tests, theyweren’t passed on" to the next level of interviews, Lorenzen says."What you end up doing is turning people down who may have beensuccessful. We wanted to score the tests together in one holistic picture sowe can get a much more powerful pass rate while keeping our standardshigh."

    Another motivation was the traditionally high turnover of customer-servicerepresentatives. "In late 1997, we decided we had to do something verydifferent," Lorenzen says.

The solution
    Using its in-house cadre of IT people, along with acombination of off-the-shelf online software testing programs, Capital One setout to reinvent its pre-employment testing during 1998. An internal validationstudy was conducted, in which call-center reps took a test and the resultswere used to build scoring models around the performances of sales rates,dollars collected per hour, and other criteria.

    From that data, an algorithm was build into the newonline testing system. "The computer can then take the performance on anygiven item on any particular test and run it on the models," Lorenzenexplains. "The models in the system are built around the most powerfulpredictors of the job. It combines a lot of somebody’s performance over anumber of different items. It’s a much more stable indication of performancebased on multiple indicators."

    Capital One’s new online system, rolled out in thefourth quarter of last year after nine months of work, includes theapplication itself and upgraded math and bio-data tests, drops the cognitiveskills test, and has a new feature: a role-playing call simulation. Applicantsdon a headset, and the CD-ROM program plays seven different customersituations. Applicants role-play as operators and answer multiple-choicequestions online as to how they would respond. The application and testresults are graded by the algorithm via an overnight batchprocess.

    The next day, Capital One has a list of who passedand is ready to move to the next stage of the process. "Our pass rateincreased by 30 percent, and we can hire 30 percent more people and still meetour high standards," Lorenzen says. Online testing allowed 22 HRemployees to be redeployed in the company, and it gives Capital One theflexibility to assign personnel according to the ebb and flow of testing.

The future
    While Capital One has experienced great gains with thenew system, the company has been expanding to the United Kingdom and France,and is already in the process of revamping its online testing to accommodategrowth both here and abroad. Lorenzen says that new features are beingdeveloped simultaneously with new technical designs to accommodate overseasnuances. The company is broadening its call-center pre-employment testing tomatch applicants with other Capital One jobs in areas such as administrationand quality assurance.

    Capital One is also adding a real-time dimension byexpanding the role-playing segment to include online, over-the-phoneinteraction, with recruiting specialists posing as customers and applicantsplaying the role of job interviewers. The specialist clicks through abehavioral checklist of responses based on the applicant’s response. Oncerole-play is finished, results are entered with a simple click and scoredtogether with the application and other tests, giving the specialist animmediate "hire" or "don’t hire" message.

    "One ofthe limitations of the current system is we have to wait for overnight batchresults rather than real-time decision-making, where we can make an offer onthe spot," Lorenzen says. "With the new approach, an offer can bemade within seconds of the completion of online testing. This new process willsave us $3.7 million next year."

Workforce, December 2000,Volume 79, Number 12, pp. 102-104 SubscribeNow!