Companies Evaluate Employees From All Perspectives

November 1, 1994
Many supervisors get a little antsy right around performance review time. In the formal performance appraisal system, there's no way for them to know whether an employee is an effective performer in all interactions—or whether the worker is simply an effective performer when the boss is around. What to do if a favored employee receives applause by supervisors but creates an unpleasant buzz among co-workers? How does a supervisor evaluate an employee he or she sees only a few hours each week? Traditional performance appraisals at their worst can be subjective, simplistic and political. Yet the need for accurate, fair performance measurement has increased exponentially as most organizations face increasingly flatter structures, greater internal changes, and more external competitive pressures.

The solution may be provided by 360-degree performance appraisals. Relatively new, they offer an alternative method by which organizations can gain more useful performance information about employees—and make them more accountable to their various customers.

The 360-degree appraisal significantly differs from the traditional supervisor-subordinate performance evaluation.

Rather than having a single person play judge, a 360-degree appraisal acts more like a jury: The people who actually deal with the employee each day create a pool of information and perspectives on which the supervisor may act. This group of individuals is made up of both internal and external customers. Internal customers may include supervisors, top management, subordinates, co-workers, and representatives from other departments who interact with the ratee. External customers may include clients, suppliers, consultants and community officials. Anyone who has useful information on how the employee does the job may be a source in the appraisal.

Using 360-degree appraisals provides a broader view of the employee's performance.
The most obvious benefit of the 360-degree appraisal is its ability to corral a range of customer feedback. Because each customer offers a new, unique view, it produces a much more complete picture of an employee's performance. Karrie Jerman, HR representative at Colorado Springs, Colorado-based Hamilton Standard Commercial Aircraft, says that 360-degree appraisals are becoming imperative in the lean and mean '90s, where managers have less credibility with their employees due to their larger spans of controls. "The thing we gain the most is input from so many people that know [the employee's] work. Now their peers and customers give feedback," says Jerman. "They feel it's more fair." Carol A. Norman, customer service specialist at Maynard, Massachusetts-based Digital Equipment Corp., agrees that 360-degree appraisals are more fair: "Unlike with supervisors, employees can't hide as easily in [360-degree] appraisals because peers know their behaviors best and insist on giving more valid ratings."

For instance, a manager at Denver-based Johnson & Johnson Advanced Behavioral Technology (JJABT) used a 360-degree appraisal to obtain information about an employee with supervisory responsibilities from that employee's direct reports. The feedback revealed that the direct reports believed the employee was not listening to them and was also being overly critical towards them. This allowed the manager to take corrective action. Prior to the appraisal, she could rely only on grapevine murmurs and her own limited observations of the employee.

"Employees can't hide as easily in 360-degree appraisals because their peers know their behavior best and insist on giving more valid ratings."

In addition to providing broader perspectives, the 360-degree appraisal facilitates greater employee self-development. It enables an employee to compare his or her own perceptions with the perception of others on the employee's skills, styles, and performance. And there's a lot of power in peer feedback. "You can change behavior more with feedback coming from your peers," says Karen Ripley, materials manager at Digital. "There is often more power there than in managers' feedback."

Finally, the 360-degree appraisal provides formalized communication links between employees and their customers. It makes the employee much more accountable to his or her various internal and external customers, because these people now have feedback into the employee's performance rating. Employees who previously might have concentrated a great deal on impressing managers now have a powerful motivation to focus on working well with all individuals inside and outside their department with whom they interact.

At Hamilton Standard, the feedback from a number of employees also helped to clarify job roles and expectations—frequent sources of disagreement between employees from different functional areas. Companies can also use feedback from the various raters to create more customer-oriented goals in the next year.

Companies must resolve a number of issues to use 360-degree appraisals effectively.
The first issue employers must solve in implementing 360-degree appraisals is how many raters should be involved, and, more importantly, who should do the rating.

As a rule of thumb, companies generally select between five and 10 raters. Why? Less than five raters unneccesarily limits the perspective on an employee; exceeding 10 raters typically makes the appraisal system too complex and time consuming.

The most important consideration, however, is to choose the right individuals to be raters. One of the first things companies should do is develop a workable definition of what exactly constitutes a peer, an internal customer, etc. Potential raters should be identified as all of those internal and external customers who have significant interactions with the ratee. At JJABT, which has many teams but still retains traditional hierarchical reporting relationships, the ratee develops a list of key internal and external customers that he or she interacts with and then recommends five to 10 individuals to serve as raters. The supervisor still has the ultimate responsibility for the appraisal and ensures that the appropriate raters are selected, thereby preventing the ratee from stacking the deck with supportive customers who will give high ratings.

Unlike JJABT, the Digital Equipment Corporation's and Hamilton Standard's Colorado Springs divisions are organized into self-directed work teams with extremely flat organizational hierarchies. At Digital, the ratee has the primary responsibility for selecting the raters. The Digital ratee works with his or her team leader to select a panel consisting of the coach and three other employees to be objective advocates for the ratee's 360-degree appraisal. Raters are then selected at random from the ratee's team by a computer-generated system and notified by E-mail to participate in the appraisal. The random system ensures that a fair distribution of raters is created.

The most effective 360-degree ap-praisal elicits feedback from external clients. However, Digital's Ripley warns that companies shouldn't survey external customers excessively. The client may feel uncomfortable with the idea, particularly if it's a new situation. For instance, one Digital client was even concerned about any potential legal issues involved if they gave a bad rating. "Remember that [reviewing performance] is not the customer's core business," says Ripley. "Providing feedback for our employees should not take away from the [customer's] profitability. You need to make sure this is a mutually beneficial process." Be strategic in deciding how much information to solicit from clients. When possible, companies may use existing customer satisfaction data or other quantifiable measures of performance in place of a formal appraisal by the client.

Once a company decides who will do the rating, it must create the criteria by which the employee will be judged. The criteria or questions used in 360-degree appraisals should be based on areas with which the rater is familiar. But organizations should fashion the appraisal to fit their unique needs. For instance, in Digital's self-directed teams, each ratee distributes his or her personal-development and work goals to the entire team at the beginning of the appraisal year. Thus, all members of the team have the ability to evaluate each ratee's goals at year end.

With the more traditional hierarchy at JJABT, the supervisor is most aware of the ratee's individual work tasks and goals. Therefore, the various raters ideally evaluate the ratee only on the behaviors or work incidents that they have directly observed.

The JJABT 360-degree appraisal form includes items such as:

Does the employee:

    • Follow up on problems, decisions, and requests in a timely fashion
    • Clearly communicate his or her needs/expectations
    • Share information or help others
    • Listen to others
    • Establish plans to meet future needs
    • Adhere to schedules?

The raters score these items on a scale from 1 (needs improvement) to 5 (outstanding). Space is also provided for the raters to make written comments. The ratee's final performance appraisal consists of a combination of the comments and ratings from the various raters and the supervisor's own feedback on the ratee's performance.

An important consideration involves how many items to include in the appraisal form. A carefully thought out tradeoff must be made between a large number of questions, which provides greater validity, and fewer questions, which require less time. Because each employee is rated by five to 10 other individuals, the appraisal can entail a major time commitment. For this reason, a practical guideline is to keep the appraisal simple by using a one- to two-page form with five to 15 questions taking 10 to 30 minutes to complete.

Effective 360-degree appraisals aren't knee-jerk judgments—they require consideration.
Once the data is collected from the various raters, it must be analyzed and summarized for the ratee's final performance appraisal. At JJABT the employee's supervisor is responsible for summarizing the data and determining the final performance rating, which generally includes a mean score and distribution range for each item. Their experience reveals that feedback can't always be taken at face value. For instance, care must be exercised when only one rater has given highly negative or positive feedback. The JJABT managers stress that the key is to look for trends or patterns in the data. If there are questions or ambiguity in the raters' feedback, the supervisor will often solicit additional feedback from the same or new raters. After summarizing the data, the supervisor conducts the formal appraisal interview with the ratee.

At Digital, where self-directed work teams are used, the ratee is responsible for summarizing the feedback from the various raters. The ratee automatically throws out the lowest and highest overall ratings to ensure more objective overall ratings. The ratee then submits a summary analysis of the remaining ratings to his or her panel of advocates. The ratee and the panel of advocates then meet jointly to determine the ratee's final performance rating and development plan.

Another issue all organizations must face is whether the feedback from the various raters should be kept anonymous or be identified openly to the employee being reviewed. Confidentiality can reduce the possibility that the employee will later confront the raters, and thus encourages raters to be more open and honest with their feedback. Jay Kirksey, a member of the leadership team at Hamilton Standard, agrees that it is difficult to ensure completely honest, open feedback when raters are identified: "Organizational maturity is needed to give and receive constructive feedback. Some people had hidden agendas. We found employees were giving lukewarm and fuzzy feedback because of the fear about the feedback coming back to them. The motto was 'Do unto others as they would do unto you.' "

However, confidentiality has its own baggage: Ratees often try to "hunt the ghost down" or figure out which rater provided the negative feedback. It's also sometimes difficult for the supervisor to give clear and specific feedback without giving away the identity of the original source of the feedback.

In an attempt to deal with these issues, JJABT provides raters with the option of being open or anonymous in their feedback. If the rater requests anonymity, then the supervisor must not compromise his or her identity. However, if the rater is willing to be open, then the supervisor may refer the ratee with questions about his or her feedback to the rater.

In keeping with the self-directed team concept, all ratees at Digital have knowledge of the various raters' comments and ratings. To help make this system work, Digital has instituted a rule that no rater can give negative feedback in the appraisal unless the rater has previously given the feedback directly to the ratee. If a ratee challenges the appraisal feedback, then he or she must face the entire team about the issue. Both Hamilton Standard and Digital stress that it takes time to develop open and effective 360-degree appraisals and suggest that most organizations should start with confidentiality until sufficient understanding, maturity and trust is achieved.

Employers must build a bridge over 360-degree appraisals' potential pitfalls.
Although 360-degree appraisals can be extremely effective, fair and useful at their best, like any form of performance review, they have their own potential weaknesses and disadvantages. For one thing, receiving a performance feedback from a multitude of sources, including one's peers, can be intimidating. Hamilton Standard's Jerman agrees that 360-degree appraisals don't eliminate the sting of criticism: "Feedback is still hard to take. It's not always fun."

While employees may have trouble receiving feedback, providing feedback is often troublesome for some. Says Sandy Bermester, staffing and training manager for financial services at Palo Alto-based Hewlett Packard: "It's hard for people to give constructive feedback when they have to. People have to have the right mindset and skills to do it well. It takes time to internalize." For these reasons, it's important that the company create a non-threatening atmosphere by emphasizing that the major purpose of 360-degree appraisals is to facilitate the employee's development and performance improvement.

Also, companies that use 360-degree appraisals may find that their biggest disadvantage is the time involved to select raters, fill out forms, and analyze the various information. It's imperative that organizations strike a balance: appraisals must be intricate enough to be meaningful, but simple enough to be completed easily. The time commitment involved is also one reason why many companies conduct formal appraisals only once a year, although semi-annual appraisals may be given to low-performing employees. Hamilton Standard does do informal 360-degree appraisals at mid-year to allow employees to hear feedback and make any necessary adjustments in their work or alter their goals.

There's also the problem of different expectations by the raters. Lynda Powell, regional director of sales at JJABT, says, "Raters tend to have different expectations. Some rate very low while others are lenient and rate very high. For example, one rater wrote in the appraisal that the employee was a very good planner, but then gave that employee only a 3 on a 5-point scale on planning."

Finally, 360-degree appraisals, although potentially more accurate, are still only a means to an end. There will never be a cut-and-dried, objective, final judgment. Another senior-level manager at JJABT has several concerns about feedback: "One, does the employee know enough about the person to rate them? The people doing the ratings do not always understand the situation the employee is in. Two, the inputs of all raters are often treated equally regardless of that raters' position or level of knowledge about the person. The feedback is often summarized overall and is not broken down into different areas to facilitate follow up."

Because of these disadvantages and potential employee concerns, it's essential that organizations develop an effective plan and change process to implement 360-degree appraisals.

First, top management needs to buy in to and clearly communicate the goals of the 360-degree appraisal and how it relates to the company's business strategy and competitiveness. Top management should also appoint a committee of representative managers and employees to develop the appraisal forms and process.

Second, perhaps the single most important key is to provide training to employees on:

  • The specific details of the new appraisal process and instrument
  • How to give constructive feedback in a productive, noncritical manner. For example, employees at Ford received training on how to evaluate specific critical incidents and to give feedback before they took part in 360-degree performance appraisals.

Learning to receive feedback is just as important as giving feedback. "What we particularly don't do enough training on is receiving constructive feedback and having to deal with it," says Hamilton Standard's Jerman. "If we don't take it well, people stop giving it. It's a talent that you develop."

The appraisal should first be pilot tested with a select group of employees before it is instituted elsewhere in the organization. Once instituted, it's essential that top management reinforce the goals and responsibilities of employees related to this new appraisal process on an ongoing basis. Tying the appraisal results to the company's reward and recognition systems can also provide added motivation for employees.

An organization must develop an effective change process and orient the appraisal to its particular needs and culture. It takes time and much effort, but when implemented properly, a 360-degree performance appraisal system can enable companies to obtain better performance information and increase employee development and accountability.

Personnel Journal, November 1994, Vol. 73, No.11, pp. 99-103.