Choosing the Right Assessment Vendor

    Effective vendor selection requires three basic steps:

Step 1: Create a team and a process
    Unless you are addressing a very simple and straightforward staffing need, it is a good idea to create a formal team and project plan to oversee the deployment of assessment tools. The functions of this team include determining assessment tool requirements, researching and gathering information from vendors, reviewing and evaluating vendor solutions, and outlining processes for ongoing use and support of assessment tools once they have been implemented. The team should include representatives from the major stakeholder groups that will be affected by the use of assessment tools.

    In addition to HR, consider including personnel from IT, legal, and operations. It is also useful to include someone with expertise in assessment design and validation to help provide guidance during the process.

Step 2: Perform a needs analysis
    Before speaking with any vendors, clearly define your reasons for using assessment tools. Outline specific business issues you wish to address and the outcomes you expect. Consider operational issues such as where you plan to use assessments, how the assessments will be administered, how you will interpret the results and handle assessment data, and who will provide ongoing support and training for the assessment tools. Define constraints such as budgets, personnel resources, and technology requirements.

    This needs analysis should clearly define what the assessment tools must do in order to be successful. Failure to conduct a good needs analysis will result in vendors telling you what you need instead of allowing you to be in control of the process.

Step 3: Conduct a formal request for proposal (RFP)
    The assessment-tools market is rapidly changing, with new solutions being introduced virtually every month. It is always a good idea to get proposals from a range of vendors before deciding on any single solution. The RFP that you send to vendors should detail the specific business objectives you want to achieve, which jobs you want to support with assessment tools, the number and location of candidates, and any technology requirements.

    The RFP should also ask vendors to supply some very specific information that will help you select the best one for your needs. The following 10 questions can be used to evaluate a vendor’s solutions and their ability to help you meet your goals.

1. What assessment techniques do you provide? It is important to understand the types of assessments offered by a vendor. Do they sell primarily qualifications screens, knowledge tests, talent measures, or background checks? Explore which assessments reflect their core capabilities and which ones they offer through partnerships or outsourcing. Ensure that the assessments offered by the vendor align with your business objectives.

2. How do you determine what are the most critical aspects of job performance? To provide effective and legally sound assessment tools, a vendor must be able to clearly define the critical factors that influence job success. This process of defining job performance is commonly known as job analysis. Many vendors will cut corners in this up-front work, so it is important that they clearly demonstrate the processes they use to develop links between their assessment tools and job performance.

3. What validity data can you provide for this product? Validity data provides the proof that an assessment tool actually predicts job performance. All assessment vendors should have summaries of validity data that are readily available to prospective clients. This data should provide evidence of the tool's performance on jobs similar to those for which you plan to use it. Look for "hard numbers" linked to well-defined measures of performance (e.g., supervisor ratings, tenure).

    Avoid vendors that rely heavily on vague anecdotal statements about their tool's perceived effectiveness and impact. Ask for references from specific clients that you can contact to learn more about the effectiveness of their assessments.

4. Do you have evidence regarding the legal defensibility of this tool? You have a right to know if a vendor’s product has ever been challenged in court. It is also important to determine if the tool displays adverse impact (i.e., members of certain protected classes do not perform as well on it as members of the majority group).

5. How much consulting and customization is required to get your system configured for our organization, and how much time will this take? Some systems are built so that they can be used "off the shelf" with very little customization, while others require more time and effort to configure. In most cases, taking time to create a customized system offers advantages in accuracy of prediction and higher levels of legal defensibility. However, it also increases development costs.

6. What delivery methods are available? Paper and pencil, telephone, and the Internet are the three main ways to deliver assessment tools to applicants. Many companies offer all three options, but differ widely in their level of expertise with each method. The technology around telephone and Internet screening can be a major source of problems, but it can also offer significant benefits in comparison to paper-and-pencil delivery methods. It is a good idea to ask vendors for specific examples of using similar technology to deliver their assessment tools.

7. How easy will the results be for recruiters and HR personnel to interpret? You want a system that provides high-level results that are easy for non-experts to understand but also allow in-depth information for the purposes of documentation and more detailed investigation. Some assessments can even be used for both candidate evaluation and development of newly hired employees.

8. Does the system consider the needs and feelings of the test taker? The experience of applicants is a critical factor in defining the success of an assessment tool. Upsetting applicants is bad recruiting and can lead to legal action. What steps has the vendor taken to ensure that candidates will perceive its tools to be job relevant and culturally unbiased?

9. How is the system priced? It is important to clearly understand the fee schedule as it relates to the manner in which you plan to use the test. Ask vendors to outline up-front fees for system design and configuration, as well as ongoing usage fees.

10. What ongoing support will you receive? What services does the vendor provide to ensure the ongoing effectiveness of its assessment tools? Does it provide quarterly or semi-annual reviews of the performance of its assessment tools, including EEOC summaries? What sort of ongoing customer and help-desk support is offered? You want to look for vendors that will provide active ongoing support and will not simply disappear after you have bought their tools.

    You should also consider the financial and personnel stability of the company. Some assessment companies depend heavily on the expertise of one or two key individuals. Try to find out if this is the case, and ask what transition plans the companies have in place should these individuals leave.

    In general, the RFP process should be approached as a dialogue, not an interrogation. Most assessment vendors are ready and willing to respond to formal RFPs. However, be respectful of the time it requires to respond to your questions, and do not ask for solutions that you are not serious about implementing. It is also helpful to let vendors know in advance about what sort of budget constraints you have. It will save both parties a lot of time, and you may be pleasantly surprised by the alternative solutions they will propose.

Workforce Online, December 2002 -- Register Now!