HR's Burden of Proof

    What works for even the most cynical CFO is an approach borrowed from the world of science, where "split samples" are used to test a hypothesis. It’s a relatively simple approach to understand and to replicate. The only barriers to employing split samples in the HR profession are a socialist mentality and ego. Some argue that providing access to training to one group and not another is unfair. Others are afraid to test their hypothesis because the results may show that what they "really want to do" doesn’t work.

    The process works like this: You implement your HR program with half of the target employee population, and for the other half you temporarily withhold the program. After a period of time has elapsed, you measure the performance differential based on the premise that those with access to the new HR program should demonstrate significantly improved results.

    Let’s take a common premise: Training impacts business results. Assume that you have a group of 30 salespeople in a region and you have a new sales training program that you believe will significantly improve sales. The design of the sales training appears logical and you have heard that it has worked at other companies, but have been unable to find any real proof of that. Select 15 salespeople at random (a critical element) and provide them with the new sales training. Inform the other 15 that the budget is limited and that they will get the training at a later date.

    Nothing about the split sample is made public and nothing else is changed. Document the average sales of each salesperson prior to implementing the training and again at six months. If the sales training is effective, you should be able to see a marked increase in average performance for each individual who attended the training. The concept is quite simple, and hard to argue against. Splitting the training, while keeping everything else constant, allows everyone involved to know whether it worked and how well it worked. In addition to training, there are many other areas in HR where you can provide "dead bang" proof through the use of split samples, including:

• Recruiting: Re-energize the employee referral program at a selected business unit and not in another similar business unit to see if the referral program produces more and higher-quality hires.

• Executive coaching: Provide executive coaches to half of the executive population and see if their business results, 360-degree review scores or other success measures improve or are superior to the uncoached executives.

• Compensation: Add team incentives to the current compensation mix in one plant (or region) and then see if the business results improve compared to similar plants that do not employ team incentives.

    You should enter the process realizing that most senior managers are very cynical. They became successful by being numbers people who demand hard data to prove that any program works, regardless of discipline. Before you start the design process, realize that it’s critical not to develop a split-sample design independently. Instead, involve the CFO’s office, representatives from the Six Sigma team, market research and departmental managers early on. This is so that all of the potential "Yeah, but …" complaints that they might raise about the design or metrics are identified and handled.

    When selecting jobs, start with positions (and business units) that have a distinctly measurable output. Sales jobs are the best, but any job where the output in quality, dollars and quantity are already measured on a regular basis will work. A quick heads up: Over the past 30 years I have seen a number of popular HR programs put to the test. Unfortunately, in a majority of the observed cases, the HR programs did not actually produce significant improvements. Perhaps that’s exactly the reason why most HR people like to remain in the dark and can come up with a hundred reasons why split samples just can’t be done in HR.

Workforce Management, January 29, 2007, p. 26 -- Subscribe Now!