Critical Care for Review Process
Performance appraisals are a major tool in the management of employees' development and in setting goals to help spur that development. Appraisal forms range from simple to complex as organizations continuously try to assure compliance with regulatory bodies, while maintaining a meaningful assessment tool. The job description/ appraisal procedure at St. Luke's Hospital had evolved over the years to meet numerous federal, state and health-care industry standards. In doing so, the size of the appraisal and supporting documents had grown exponentially to an average of 20 pages per employee.
The steering committee steps up to the blocks.
The voice of technology was calling. LaJuan Aderhold, the director of compensation and benefits, explains, "We looked at our paper-intensive appraisal process and saw we were near the suffocation point. Because this process had to meet a variety of needs for so many customers, we knew that technology would offer us solutions." So Aderhold assembled a diverse mixture of staff members to form a steering committee. In addition to Aderhold, members included: Doug Cuthbert, the director of information services; Darrell Burke, the director of systems and procedures; Hilary Mathews, the director of training and education; and Nancy O'Keefe, the special projects coordinator.
At the outset in April 1994, the committee approached the new process with three primary requirements: It would be user driven; it would apply technological solutions that required less paper; and it would be a stepping stone toward full automation. Not wanting to fall behind with the next set of annual appraisals, the committee determined it had a three-month window to review the current system, conduct a focus group, and design, train and evaluate the reengineered process.
To establish a common understanding, the steering committee reviewed the chronological sequence of steps required to complete the former appraisal process. A flow chart of the process involved in preparing the appraisal packages showed 36 unique steps. With customer service in mind, the compensation and benefits department had been trying to comply with other departments' special requests for sorting and delivering appraisals. After distributing the appraisal forms, collecting accurate and timely information was critical; performance was a key factor in determining employees' pay increases. Managers manually scored performance based on criteria and weights, and the information was summarized by compensation and benefits. It was then reviewed by executive management to ensure that performance ratings were consistent with business results.
The focus group takes the baton.
The steering committee felt that in order to act quickly and decisively, it was important to involve the users in developing the framework for the reengineered process. A focus group of five middle managers was assembled to establish specific user needs and provide feedback about plans for the new process. The group members represented many clinical and nonclinical departments. Sessions were conducted to brainstorm solutions for bottlenecks with the current paper process, to determine what the ideal automated process would look like and to identify business needs not addressed under the system at the time. The enormous amount of paper exchanging hands during various points in time was a consistent theme. Other issues included: system flexibility; peer-based appraisals; lengthening the time to complete the appraisal process; the need for computerization; and identifying competencies within the framework of the appraisal.
The focus group was pleased about the user-driven approach taken by the steering committee. Debbie Leming, director of rehabilitative services, was happy to be involved: "The appraisal process had become quite cumbersome and time-consuming, and I felt there was much room for improvement. As a member of the focus group, I was able to share my ideas and actively participate in improving the appraisal process."
At this point, the steering committee had a clear knowledge of the current system and what key users wanted. Automation was top priority—it was needed to minimize the manual calculations, tickler files and check lists that were the backbone of the existing system. So the steering committee began searching for an automated appraisal process. This included reviewing current literature, sending out a survey to other hospitals, and reviewing software products advertised in trade journals. The results were disappointing in all three areas. For example: of 167 hospitals surveyed, only two responded—and those two used manual appraisal systems. The focus group also looked into scan readers (bubble sheets), but decided they weren't flexible enough to meet current needs.
Finding no existing applications, the focus group's challenge was to develop an appraisal system internally. The work began to achieve the original goals, noting:
- Constraints of time
- Existing computer hardware
- The need to enhance managers' ability to customize criteria-weights
- Users' request to incorporate peer appraisals.
Using ideas generated by the focus group, the steering committee devised two options and presented them to the focus group for consideration. The first option consisted of moving the numerical criteria scores from the individual pages of the job description to a summary sheet allowing:
- Scoring of up to six employees on one form
- Calculating scores by computer
- Moving handwritten comments, currently appearing next to each criterion, to a separate exception-only summary comment sheet available on the computer or in paper format.
The second option was identical to the first, except the format allowed comments to remain next to each criterion in the appraisal. The focus group overwhelmingly chose the first option because of its superior ability to reduce pages and package the information in a clearer format.
Bringing the focus group back at this point in the project served a number of purposes. Focus group member and director of surgical services Trish Aikenhead explains: "We were introduced to the new technologies available and had a voice in the selection of the new process. We were able to review different options and decide what would work best for our areas. This allowed the departments a better understanding of the system." The steering committee used the needs identified by the focus group to test whether the mock-up solution would be adequate. By presenting the sample packets and obtaining user buy-in, active process design had begun.
In response to Joint Commission on Accreditation of Healthcare Organizations (JCAHO) requirements, a stand-alone competency management program had been in place. A competency profile for each employee was maintained by the hospital's education department. Managers were responsible for updating this information and forwarding it to education. After the profiles were updated, the education department delivered copies to HR. These copies were attached to each employee's appraisal packet. This was a problem. Managers didn't like having separate competency profiles and appraisals.
A detailed review of the JCAHO requirements led to the conclusion that this information didn't need to be stored separately from the appraisals. As a result, the competency profile and the job description/appraisal documentation were combined, eliminating the need to maintain a separate database, and getting rid of unnecessary paperwork and frustration.
The implementation team crosses the finish line.
Now that users had provided feedback to the steering committee and the extra database was eliminated, an implementation group was established to make the project a reality. Composed of HR and data processing staff members, this group worked under the direction of Nancy L. O'Keefe, special projects coordinator and liaison between the steering committee and implementation group. The hardest task was the database design. It had to be developed from scratch and had to interface with existing systems in place. An internal senior programmer analyst, Mario Diaz, was primarily responsible for the database and application design. Diaz relied heavily on the specifications provided by the steering committee and focus group to develop both the database and application. He also solicited input and feedback from Doug Cuthbert, the director of information services, and other IS professionals. To test the database and application, Diaz called in O'Keefe and HR staff members to try out various keying and report generation scenarios.
A program was written to use data already entered into the system to do the calculations previously done by the managers. The forms for the whole process were modified to be produced from the database. With approximately 20 pages per employee under the old system, the new process decreased the number to seven pages, a reduction as high as 70% in some cases.
Managers now had the option of using the computerized comment sheets. The on-screen form is in table format and can be expanded to any length. The form can also simplify year-round activity, serving as a documentation tool for noteworthy incidents as they occur. The manager can make a timely entry and review it later for the appraisal. This eliminates the need for a separate anecdotal-note system which many managers had been maintaining.
Because of the time issue, there was no opportunity to do a true pilot project, sampling one department and working out the problems. Instead, database-design problems were addressed behind the scenes as the hospital went live with the automated system. "Even though it meant more work for the implementation group, the members pulled themselves together to develop and deliver the best process in three short months. All goals were achieved or exceeded," O'Keefe says.
Since this was a new process, manager education was key to the success of the effort. To assure training compliance, managers didn't receive appraisal packets unless they attended a training session. O'Keefe communicated the vision from the steering committee to the implementation group and then managed the training process. Consistency was achieved by having only two instructors teach the 15 classes. One-hour training classes were held up to two times a day for two consecutive weeks. The broad range of times classes were offered enabled all managers to attend, regardless of the shifts they worked. This training blitz achieved a 97% attendance rate (94 of 97 managers attended). The remaining 3% were trained individually.
A sample package consisting of appraisal forms, a check-off time line, a resource list and directions for using online forms was given to each manager at the beginning of the class. Sessions included an explanation of the reason for the change and the steps involved in making the changes. Each new form was reviewed with examples, and at the completion of the class the managers received their actual performance appraisal packets.
St. Luke's comes out the winner.
To determine if the original goals of the steering committee were met, a five-question evaluation tool was developed. This was distributed to managers when they received their score-approval forms. More than 50% of the managers returned the survey. Approximately 90% felt the total process had been streamlined. Specific comments included:
- Easier to understand
- Prevents math errors
- Much more clear and concise
- Recording comments online is significantly reducing paper volume.
Responses to a question concerning additional future improvements included:
- Put everything online
- Start training earlier
- Start the whole process earlier.
Nell E. Robinson, director of food services, is happy with the new system: "I work with a large group of diverse individuals who look forward to their annual performance reviews. The new process allows our supervisors to spend more one-on-one time with their team members, rather than filling out forms." With comments like these, the steering committee felt it achieved the goals for 1994.
In early 1995, the steering committee and implementation group reconvened to identify areas for improvement. Database enhancements were a major issue. HR staff members performing data entry worked with data processing staff to resolve problems they had encountered. The implementation group was included in future meetings to quickly facilitate changes and identify issues. Cuthbert, now the manager of financial and administrative applications, believes that this openness to feedback has enabled the system to continue to evolve, becoming more valuable at every stage. "The key to the success of the project has been the incremental goal-setting by a diverse group, the ability of technical staff to creatively apply technology, and the willingness to seek and respond to continuous feedback," Cuthbert says.
In response to user needs, the 1995 process started in August, versus September as it had in 1994. The large clinical departments also were given four extra weeks to complete the process. Programming efforts centered on assuring speed and flexibility of the data-entry process, simplifying production of reports, and simplifying interfaces to and from the HRIS.
HR staff members who do data entry report that shortcomings have indeed been removed, and trouble-shooting requirements are now only minimal. For 1996, updates to computer hardware and software are under way, and data-entry screens have been further simplified for end-users. At least some supervisors will enter their appraisals on-screen during 1996, and the steering committee has set its goal to complete the automation process by 1997. Darrell Burke, the director of systems and procedures, explains: "Our success is a result of selectively using the right tool at the right time.... We combined pieces of customer service, process analysis, marketing, training and software design using a highly motivated, self-managed work team. I plan to use this approach in future opportunities." With that philosophy, St. Luke's Hospital will continue to be a front-runner.
Personnel Journal, April 1996, Vol. 75, No. 4, pp. 115-120.