Applying Benchmarking in Health
Ed Kelley, Joanne Ashton, Thada Bornstein

What is Benchmarking?
The task of improving quality is a demanding job. It requires focusing on clients, using data, working collaboratively with other team members, and maintaining an overarching view of the health system in which we work. At the Quality Assurance Project (QAP), we have developed quality-improvement technologies that health care managers can use in developing countries. One such approach is benchmarking. While much has been written about benchmarking in domestic business and health literature, virtually no reference works exist on the use of benchmarking in developing countries. This Project Report summary is intended to offer readers a brief background on the approach and some examples of its application in health and development.

Definition
Benchmarking is a process for finding, adapting, and applying best practices. The concept of learning from others’ experience is perhaps as old as human society; however, the first widely publicized use of the term “benchmarking” was by the Xerox Corporation in Rochester, NY, in the 1970s. It was defined by Xerox as “the continuous process of measuring products, services, and practices against the company's toughest competitors or those companies renowned as industry leaders” (Camp, 1994). In general, the two key concepts in benchmarking are the idea of systems or processes and the concept of “benchmarks.”

Benchmarking typically focuses on innovations in managing a given work system or process (see Definitions box), while the “benchmark” is the measure by which that system or process is judged to be successful or effective. Because benchmarking is a quality improvement approach focused on processes, the process of interest must be defined. This is partly why many benchmarking efforts are connected to larger business or strategic planning processes. However, benchmarking can be used as a stand-alone tool for developing a new service or improving an old one. One of the important ideas to keep in mind is that benchmarking does not mean replicating someone else’s process exactly, but rather seeking out aspects of a successful process that could improve your own work. In addition, you may find steps or strategies to avoid when benchmarking.

Definitions
Benchmarking – The process for finding, adapting, and applying best practices.

Process – A series of actions that repeatedly come together to transform inputs into outputs.

System – A system is an arrangement of organizations, people, materials, and procedures associated with a particular function or outcome. A system is usually made up of inputs, processes, and outputs/ outcomes.

Quality improvement - Quality improvement identifies where gaps exist between services actually provided and expectations for services. Quality improvement then lessens these gaps not only to meet customer needs and expectations, but exceed them and attain unprecedented levels of performance. Quality improvement is based on the principles of Quality Management: a customer focus, a systems approach, teamwork, and the use of data.

Types and range of benchmarking
Many authors point out that there are different types of benchmarking: internal, competitive or external, functional, generic process, etc. In general, the difference between these types of benchmarking is the source of information about innovative processes. A manager seeking to improve the client registration system at his or her outpatient clinic may first wish to visit other departments of the hospital to locate “internal” innovations. Secondly, other outpatient clinics in the city may offer information from useful “external” sources. Finally, talking with employees from a hotel known for its client focus and efficient check-in processes may offer some important lessons in client registration from a “generic process” point of view. Distinctions between different types of benchmarking may not, ultimately, be important. For our purposes, we will describe one process that can be applied to numerous situations.

A review of the quality literature reveals that a wide range of processes are referred to as benchmarking. Benchmarking has been used to describe everything from shared learning between colleagues or institutions (through professional conferences or journal articles) to operations research using primary data collection. It may be helpful to think of benchmarking on a continuum that ranges from a sharing of ideas, which may occur through professional conferences, to formal benchmarking, which may rely on structured data gathering and site visits. This continuum is pictured below.

fig1

QAP's benchmarking approach
QAP staff has been gathering literature, books, and material as part of a literature review on benchmarking approaches and applications of best-practices benchmarking in health. This general model is presented below.

Benchmarking : A General Approach
  1. Define the benchmarking team (using criteria such as interest, expertise, authority, time availability, communication skills, etc.).
  2. Define your objectives (i.e., subject area; for instance, creation of a reliable emergency transport system).
  3. Define your criteria for success (i.e., what makes a successful transport system). These criteria could include process measures and/or outcome measures.
  4. Identify premier examples of the process of interest (could be regional, within your own organization, or from a completely different sector).
  5. Gather information. A variety of options exist, including:
    • Mail/e-mail questionnaire
    • Attend workshops/conferences
    • Site visits
    • Existing documentation in journal articles and presentations
  6. Choose elements of the process appropriate to your context.
  7. Develop an improvement strategy based upon benchmarking.

Case Example : Improving Care for Pneumonia through Benchmarking
Not every effort has to consist of one benchmarking team conducting one site visit. The following example illustrates how institutions can work together to learn from each other.

Background and objective: The SunHealth Alliance is a hospital group operating in several states in the southern U.S. Since 1990, SunHealth hospitals have developed more than 15 benchmarking projects in both administrative and clinical areas throughout the hospital system. One such clinical benchmarking project focused on reducing the length of stay and mortality rates for pneumonia patients. The effort involved four hospitals.

Planning: A team was developed composed of three members from each of the four hospitals. These members were a physician leader from the clinical area under study, a senior administrative employee, and another staff member who was designated “clinical benchmarking facilitator” for that hospital. This interdisciplinary team met three times to define objectives of the effort, identify “criteria for success,” or outcome measures of interest, and to review the treatment process for pneumonia. In addition, each of the four hospitals formed internal task forces that would be charged with implementing the solution and tracking data on pneumonia length of stay and mortality over time.

Collecting data: Each benchmarking facilitator led his or her internal team in conducting 15-20 interviews with key players in the treatment process, including laboratory, radiology, pharmacy, and emergency services personnel. In addition, the internal teams reviewed medical records for 30 pneumonia patients across the four hospitals. A set of best practices werethen developed by the overall benchmarking team. These included:
  • Initiating administration of antibiotic dose in emergency room as soon as possible
  • Standardizing sputum collection process
  • Establishing standing orders for post anterior and lateral x-rays
  • Developing clinical path for pneumonia patients
  • PDCA: Once the best practices had been identified, the internal teams selected those practices with the highest priority to be part of the hospital's action plan. Each hospital team then identified key aspects to monitor.

The final step in the benchmarking process involves actually implementing your new process. There are a variety of ways this could be done, from more to less systematic. Depending upon the complexity of the new process and the number of stakeholders involved, you may need to engage in a Plan-Do-Check-Act (PDCA) cycle. On the other hand, if time constraints and simplicity are important, you may need to produce a simple work plan, allocate resources, and begin work.

Implementing benchmarking: some practical advice
A few words of advice when beginning a benchmarking effort:
  • Focus on data. Innovations are defined by their impact on objectives. Teams and their supervisors or leaders will also need data both to verify their suspected problem and confirm that their solution to the problem is actually having an effect. If you are going to look for innovative, rapid ways to register patients in your emergency room, how will you know that a nearby hospital you might visit has a good benchmarking process? The team needs to develop its criteria for success-the indicators that tell them what the team will track when implementing its new plan.
  • Develop and use checklists for your site visits. Once the team has set up its benchmarking site visit, they will need to develop a checklist for the visit. This will contain the items they wish to see and questions they wish to ask during the visit. Benchmarking checklists can help ensure that the team gets the most out of its site visit(s). Although the following is not a full checklist, some key questions to ask include:
    • What process or activity do you use to achieve a certain result?
    • When did you begin to use this process?
    • Why did you begin using this process? What did not work well with the previous process?
    • What benefit (e.g., time, money, health of patients) have you seen since you began using this process? Do you have evidence of this benefit and, if so, what is it?
References and suggested reading
For readers with further interest in this subject, we suggest the following resources:
  • Camp, R., and A. Tweet. 1994. Benchmarking applied to health care. Journal on Quality Improvement. 20(5): 229-38.
  • Franco, L.M., J. Newman, G. Murphy, and E. Mariani. 1997. Achieving Quality Through Problem Solving and Process Improvement. Bethesda, MD: The Quality Assurance Project.
  • Harrington, H.J. 1996. High Performance Benchmarking: 20 Steps to Success. New York: McGraw Hill.
  • Joint Commission on Accreditation of Healthcare Organizations. 2000. Benchmarking in Health Care: Finding and Implementing Best Practices. Chicago, IL: Joint Commission on Accreditation of Healthcare Organizations.
  • McNair, C.J. and K.H.J. Leibried. 1992. Benchmarking: A Tool for Continuous Improvement. New York: John Wiley and Sons.


1