A few key measures can help you gauge whether your practice is headed in the right direction.
Fam Pract Manag. 2003;10(2):43-48
Welcome aboard Flight 111 to Kansas City. We’ll be cruising at … well, I can’t tell you that since our cockpit instruments aren’t functioning. We’re not able to tell you what our airspeed or estimated time of arrival are either, or even what our direction is, but we’re experienced pilots, trained to fly by the seat of our pants. So sit back, relax and enjoy the flight.”
Would you fly this airline again? If you have a basic appreciation for safety and efficiency, probably not. Nevertheless, we ask our patients every day to “sit back and enjoy the flight” and trust that we, as experienced physicians, can guide them on their path to better health – despite the fact that we don’t know how well our practices are actually performing.
Two national initiatives – the Idealized Design of Clinical Office Practices initiative of the Institute for Healthcare Improvement and the AAFP’s Practice 2010 initiative – are proving that office practices can produce health care that is safe, evidence based, satisfying to patients and staff, cost effective and financially viable. Central to our efforts to improve the quality and safety of health care is measurement of our processes and outcomes in a timely and useful way. By incorporating measurement into daily practice, you will be better able to diagnose the strengths and weaknesses in your practice, identify ways to improve your health care delivery processes and evaluate whether changes in your practice have made things better over time.
KEY POINTS
A clinical instrument panel is analogous to a cockpit instrument panel in an airplane or a dashboard display in a car.
Unlike report cards issued by third parties, instrument panels are meant for practice learning and true self-assessment.
For a practice’s instrument panel to have value, its physicians must consult, interpret and act on the data.
Measurement, good; report cards, maybe not
Chances are you’re already measuring a number of aspects of your practice, especially where profitability and productivity are involved. But chances are also good that you have no idea how you are doing in other important respects.
True, third parties are probably monitoring your pharmaceutical prescribing, surveying your patients about their level of satisfaction, tracking your utilization rates for procedures and hospitals and producing “report cards.” But report cards are derived largely from the claims you submit and, therefore, reflect historic, high-level estimates of performance. They are external performance summaries that many physicians fear are a search for bad apples or simply a contract negotiating tool. There is also evidence that report cards are unreliable tools for differentiating one physician from another.1 One effect they have had is to make some physicians resistant to measurement. For whatever reason, few physicians actually use the available data to improve their processes of care.
Don’t confuse report cards with internal measurement, though. Internal measurement is designed to help guide your practice through real-time monitoring of practice trends and outcomes, and to serve as a basis for true practice learning. It is, in the best sense of the word, a self-assessment. When you measure your own performance, you can measure just what is important to you, have confidence in the way the measurements are taken and act on the results as you see fit.
So much to measure, so little time
But what do you measure? How can you and your staff find the time to measure it? And how can you deal with it all? One good answer to all three questions is the clinical instrument panel, analogous to the instrument panel in an airplane or the dashboard display in a car. It is a collection of key measures for your practice, giving you a quick way to assess your practice’s performance and ensure that it is moving in the right direction. What makes the clinical instrument panel so valuable is its focus on simplicity. It requires measurement of only a few parameters – the ones that matter most to you – with minimal data collection, and it displays the results in simple, easy-to-grasp formats. (See the sample instrument panel on page 45.)
Measurement tips
1. “Seek usefulness, not perfection, in measurement.” The goal of internal measurement is not to conduct a randomized clinical trial ready for peer-reviewed publication but to produce “good enough” data you can use in a timely manner to assess how your practice is doing and whether you are making progress toward your goals.
2. “Use a balanced set of measures.” Because health care is complex and dynamic, the more angles you can measure, the more complete a picture you will have of how your practice is doing. If you measure only a few parameters, think carefully about the important aspects of performance, from financial viability to clinical quality and patient satisfaction to staff satisfaction and competence.
3. “Keep measurement simple – think big, but start small.” Don’t try to measure everything in your practice. Instead, start with small sets of measures that are easily understood and collected within your practice. As your practice changes, you can change your measure sets as needed. The table on page 46 offers a starter set of measures that you can choose from or adapt to the unique constraints and characteristics of your practice.
4. “Use qualitative and quantitative data.” Purely quantitative data can lose the richness of your staff’s experiences and your patients’ personal stories. Build in opportunities to get impressions, perspectives, motivations, needs and desires from your staff, patients and colleagues through focus-group discussions or open-ended surveys.
A SAMPLE INSTRUMENT PANEL
Below is a sample instrument panel, constructed in Microsoft Excel, designed to gauge the performance of a family practice in nine key areas. There is nothing magical about the number nine; you may want fewer “gauges.” In constructing your instrument panel, choose the measures and format that are most useful to your group. To download the sample instrument panel, click below.
5. “Write down the operational definitions of the measures.” It is crucial that everyone who collects and uses a specific measure understands the aim of the measure and the specific method for collecting the data and scoring it. As a routine part of practice, your procedures for measurement should be as precise and explicit as any other procedure in your office.
6. “Measure small, representative samples.” A universal sample is rarely available or required for practice improvement. Instead, it is more reasonable to select a sampling strategy, such as surveying every fifth patient or reviewing 20 charts per physician.
7. “Build measurement into daily work.” To the extent possible, involve everyone in your office in collecting data relevant to their work. Use self-scoring work sheets that simplify the process of collecting data. And try to use data that are already being collected (e.g., via vital signs, flow sheets and lab reports), rather than duplicating existing efforts.
8. “Develop a measurement team.” A team approach allows your staff to share not only the workload involved in measuring your processes but also their insights into measurement and practice improvement. A team approach also helps staff members feel more invested in the process. Identify individuals well suited to these tasks and assign them the responsibility of overseeing the measurement activities of your office.
A STARTER SET OF PERFORMANCE MEASURES
The following table presents a variety of sample performance measures, as well as possible definitions and collection methods, to help you get started in constructing your clinical instrument panel. To use these or any other measures in your practice, you’ll need to decide on and write down the fine points, such as how often you’ll survey your patients or whether you’ll collect and display data by physician or for the entire group. But don’t get so bogged down in the fine points that you lose momentum. Once you’ve picked your measures and defined them, simply choose a data collection method that makes the most sense for your group. Then, move on quickly to actually collecting the data and using it to improve performance over time.
| Measure | Definition | How to collect it |
|---|---|---|
| Visit quality | Percentage of patients who report being delighted with their visit. | Conduct a patient satisfaction survey; include every nth patient in the office. |
| Team morale | Percentage of staff who recommend your practice as a great place to work. | Conduct a staff satisfaction survey; include all staff members. |
| Cycle time | Average number of minutes from patient check-in to patient check-out. | Give every nth patient a time card, which the receptionist stamps on registration and check-out. Collect time cards at end of visit. |
| Access to care | Number of days until the third next available appointment. | Consult your appointment calendar. Imagine a patient is calling to request a routine appointment. Calculate the number of days until the third next available appointment slot for each physician. |
| Practice size | Number of patients who belong to your practice or the number of active and unique patient charts seen within the last 18 months. | Consult your practice’s administrative records. |
| Operating cost per visit | Total monthly operating expenses divided by the number of monthly visits. | Consult your practice’s financial and administrative records. |
| Support staff costs | Total monthly cost of non-physician salaries and benefits divided by total monthly operating expenses. | Consult your practice’s financial and administrative records. |
| Preventive care | Select any prevention goal. For example, women age 50 to 64 years on last birthday who have had a mammogram in the last two years. | Review 20 charts that meet inclusion criteria looking for documentation that the preventive service goal has been met. Include charts for each physician within the practice. |
| Chronic disease care | Choose an appropriate measure for the most common chronic disease in your practice. For example, the percentage of patients with diabetes who have an HbA1c < 8.0. | Review 20 charts that meet inclusion criteria looking for documentation that the chronic disease goal has been met. Include charts for each physician within the practice. |
| Patient-physician match | Percentage of patients who see their own physician during their appointment. | Review 20 charts looking to see whether the patient’s last visit was with his or her primary physician. |
| Staffing levels | Number of full-time-equivalent (FTE) support staff per FTE physician. | Divide the number of FTE staff by the number of FTE physicians. |
| Overhead | The percentage of revenue spent on overhead. (This includes all expenses except physician compensation and benefits.) | Consult your practice’s financial and administrative records. |
| Visits per hour | Divide the number of visits provided over a specified time period by the number of hours worked during that same time period. | Consult your practice’s appointment schedule or administrative records for the previous week in practice. |
Constructing your instrument panel
Guided by the above principles, you can begin to create your practice’s instrument panel.
Step 1: State your aims. In other words, be clear about what you are trying to accomplish. Your measurement efforts should be part of an ongoing improvement effort. For each aspect of practice that you are seeking to improve (e.g., patient satisfaction, chronic disease care, practice revenue), clearly define your goals. For example, one of your goals for improving chronic disease care might be “to increase the percentage of patients with diabetes who have a current HbA1c of less than 8 percent.” (At this point, you may not have enough data to determine how much of an increase you expect to accomplish or by when, but once you gather your baseline measurements, you can make your goals more specific and attach a time frame.) Clarifying your purpose is a crucial first step, as it will drive your improvement and measurement activities.
Step 2: Select your measures. Choose multiple measures (no more than eight to 10) that will help you assess whether you are making progress toward the goals you have selected. The table on page 46 provides a starter set that you can pick from, or create your own. Remember to select a balanced set of measures. For example, if you are trying to increase efficiency in the office, you could measure patients’ cycle time (the amount of time from check-in to check-out) as well as their satisfaction with the visit. This will help you assess whether your quest for efficiency is making other parts of your practice suffer.
Step 3: Define your measures and data-collection methods. For each measure you select, write down the operational definition you will use in your practice. Also, write down the protocol or method you will use to collect and analyze the data, including how often the measure will be collected and by whom. Designate a staff person to maintain these definitions and procedures and update them as needed.
Step 4: Decide how your measures will be graphed and displayed. Graphing the data you collect for your instrument panel will help facilitate understanding of how your practice is doing over time and where there might be opportunities for improvement. Run charts are helpful for spotting trends. Bar charts are useful for comparisons. The sample instrument panel on page 45 displays a variety of graphs that help gauge the overall performance of a family practice.
Rather than storing your instrument panel in an inaccessible computer file, consider creating a bulletin board in your office as a “data wall” to allow your staff to view their progress on a regular basis. Also, if possible, display your instrument panel as a one- or two-page handout, which you can easily distribute to your staff or share with patients or third parties as needed.
Step 5: Use the data. As well-functioning as a cockpit instrument panel might be, if the pilots don’t read the data, interpret it and make mid-air flight corrections based on the data, the instrument panel becomes useless. Similarly, for a practice’s instrument panel to have value, its physicians must actually use it for monitoring and improving their practices. Consult, analyze and update your data often. Present the current data at regular staff meetings, and use them as a focal point for discussions on progress and opportunities for improvement. Use them also to celebrate accomplishments.
ADDITIONAL RESOURCES
For more information on using measurement to improve medical practice, consult the following resources:
Clinical Microsystem Action Guide: Improving Healthcare by Improving Your Microsystem. Hanover, NH: Center for Evaluative Clinical Sciences, Dartmouth Medical School; 2002. Available online at: www.clinicalmicrosystem.org.
Practice Redesign Guide. Leawood, Kan: American Academy of Family Physicians; 2000.
Using data to improve medical practice by measuring processes and outcomes of care. Nelson EC, Splaine ME, Godfrey MM, Kahn V, Hess A, Batalden P, Plume SK. Joint Commission Journal on Quality Improvement. 2000;26(12):667–685.
Practice-based learning for improvement: the pursuit of clinical excellence. Staker LV. Texas Medicine. 2000; 96(10):53–60.
Building an idealized measurement system to improve clinical office practice performance. Hess AMR, Nelson EC, Johnson JH, Wasson JH. Managed Care Quarterly. 1999;7(3):26–38.
Quality and outcomes management in the primary care practice. Bender AD, Motley RJ, Pierotte RJ, Bischof RO. Medical Practice Management. 1999;March–April:236–240.
Report cards or instrument panels: who needs what? Nelson EC, Batalden PB, Plume SK, Mihevc NT, Swartz WG. Joint Commission Journal on Quality Improvement. 1995;21(4):155–166.
Step 6: Revise as needed. Regularly assess whether your data collection efforts are answering the critical questions for your practice. If they are not, do not hesitate to discard them and find new measures that might better meet your needs. Likewise, if you have attained the goal you have set in a specific area or if the measure has remained steady over a long period of time, identify new goals and measures that will assist you in moving forward.
A tool for improvement
It’s important to remember that measurement is simply a tool. It is not the end point – practice improvement is. If you approach measurement with a spirit of curiosity (not judgment) and make it practical (not cumbersome), it will enlighten and energize your efforts to create a high-quality medical practice.