For many family physicians, the idea of a chart audit conjures up images of federal investigators or insurance company representatives descending on their offices to look for evidence of wrongdoing. For the most part, however, a chart audit is not so scary. A chart audit is simply a tool physicians can use to check their own performance, determine how they're doing and identify areas where they might improve. The purpose of this article is to describe some scenarios in which a chart audit might be helpful and to offer step-by-step instructions for doing one.

Why a chart audit?

Chart audits can serve many purposes, from compliance to research to administrative to clinical. You can conduct a chart audit on virtually any aspect of care that is ordinarily documented in the medical record. Practices frustrated with clinical processes that don't work well can use chart audits to document that something is wrong, find the defect in the process and fix it.

Perhaps the most beneficial use for a chart audit is to measure quality of care so that you can improve it. Chart audits are often used as part of a quality improvement initiative. For example, a practice might review charts to see how often a particular vaccine is offered, given or declined. If the audit determines that the vaccine is not being offered or given as recommended, then there is room for improvement. The same practice could review the panels of individual physicians within the group to see if they differ in performance on this measure and to give focus to their improvement efforts (for additional chart audit ideas, see below).

A chart audit is one of numerous data sources available for quality improvement efforts. Others include patient surveys, discharge summary reviews, billing/claims data and employee feedback.

How to do it

Below we describe eight steps to a formal chart audit. Although the process is not necessarily linear, we will discuss each step in the order it might typically occur, using the example of a breast cancer screening audit to illustrate each step. Because the audit will involve reviewing confidential data, it is important to check your institutional guidelines regarding patient confidentiality before you get too far into the planning process.

Step 1: Select a topic. The focus of your audit must be clear, neither too narrow nor too broad, and measurable using data available in the medical record. If possible, choose an area that interests you. You will find that you are more able to recognize nuances in your study when you have personal interest in the topic. Of course, your topic should also be of interest to the practice, perhaps a problem or aspect of care that the providers have identified as needing improvement. The Joint Commission recommends studying issues that are high frequency, high risk or both.

You should also consider early in the process how important external comparison is to your purpose. If it is quite important, then choose a topic that has an existing, well-defined measure and available benchmark data – even one you might not choose otherwise – because this will be more practical than developing your own standard for comparison.

Chart auditing is an iterative process – don't be discouraged if you change directions several times before settling on a topic.

Example: Your practice wants to measure how well it's doing on meeting recommendations for preventive care. Since the insurance carriers in the area are focusing heavily on women's health, the group decides to focus its chart review on screening for breast cancer (mammography).

Step 2: Identify measures. Once you're set on a topic, you need to define exactly what you will measure. Criteria must be outlined precisely, with specific guidelines as to what should be counted as a “yes” (criteria met) and what should be counted as a “no” (not met).

For example, if you decided to review the rate at which foot exams were performed on patients with diabetes in the last year, you would need to decide what qualifies as an adequate foot exam. Is it monofilament testing for sensation? Visual inspection? Palpation of pulses? Many would say all three are necessary for a complete foot exam. If only two of the three are documented, how will you count that?

It may be worthwhile to do a literature review to help you define your measures or consult measures used by insurers or accrediting bodies; adopting measures that have been used successfully in the past will make your work easier. A literature review may also help you identify benchmarks for comparison.

Once you've chosen measures that seem workable, it can be helpful to conduct a pilot audit. Just going through a few charts will help to identify issues that need to be clarified before starting a full audit.

Example: For your audit on breast cancer screening, the group considers several measures, including the following:

• Time since last mammogram. This provides the most specific information but would require more analysis.

• Mammogram completed within last year. This measure attempts to assess compliance with clinical guidelines. The U.S. Preventive Services Task Force recommends screening mammography every one to two years for women age 40 and older. However, the Healthcare Effectiveness Data and Information Set (HEDIS) measures, which most health plans use for National Committee for Quality Assurance (NCQA) accreditation purposes, require at least one mammogram completed within the past 24 months.

• Mammogram ordered within last year. Do you want to measure only whether the study was done, or whether it was recommended or ordered by the provider? Should providers be held accountable when patients decline to have the test?

After considerable discussion, the group decides to measure whether a mammogram was completed or recommended within the last 24 months.

Step 3: Identify the patient population. To determine which records to review, you need to define the population you want to assess. Characteristics to consider may include age, gender, disease status and treatment status. In many cases, the focus of the audit and even the measure itself will help to define the population. You'll also need to develop specific inclusion or exclusion criteria.

Example: In keeping with the HEDIS breast cancer screening measure that your group decided to follow, your patient population will be women age 40 to 69. Because you'll be looking for evidence of a mammogram in the past 24 months, the lower age limit for the sample will be 42. Only those patients with at least three visits in the last two years and one in the last 13 months will be included. You decide to exclude women who have had bilateral mastectomies or are terminally ill.

Step 4: Determine sample size. A manual audit of all charts meeting your inclusion criteria will not be feasible in most situations. That's where sampling comes in. For an informal, or “quick and dirty,” audit designed to give you a sense of whether a more sophisticated audit is warranted, you may find it useful to sample a minimum of 20 charts. For better results, a common rule of thumb is to try for 10 percent of the eligible charts. Or you may choose to use a convenience sample: the patients from a single day or all the charts on a single shelf in the records room.

If you want to track a measure over time, or if you want your results to be statistically valid, your sample size is critical. If the sample is too small, the random variability will be too large, and the results will be limited in their applicability.

Example: Using the process outlined below, your group determines that its sample should total 81 charts.

Step 5: Create audit tools. To complete your chart audit, you will need instruments on which to record your findings. How they are structured and the details they include will affect the analysis you can do and the eventual usability of your findings. Data should be collected in a format that keeps all individual records separate but allows for easy compiling.

Many chart audits involve the calculation of a rate, percentage, mean or other statistical measurement. An electronic spreadsheet format can be customized to do these calculations for you. For those more comfortable with paper-based systems, a preprinted form that lists the specific items to check in each chart serves well as an audit tool. One form is completed for each chart, and the forms can then be sorted and counted as desired. A separate form can be used to tabulate results.

Creating clear, simple audit tools will make it possible for nonclinical staff to perform many audits effectively. Once you've developed the forms, if someone other than you will be doing the actual chart reviews, go over a few examples together to be sure the reviewer understands the criteria exactly as you intend.

Example: Your group decides to use paper forms for the chart audit (see the completed forms below).

Step 6: Collect data. Select the date or dates on which you will collect data. Be sure to coordinate the specifics (date, time and number of charts to be pulled) with the medical records staff. Review each chart to determine if the patient meets the selection criteria. The reviewer should complete one audit tool (paper form or row in the electronic spreadsheet) for each patient that meets the criteria. To protect patient confidentiality, patient names should not be included on the review forms.

Example: You instruct your office staff to pull the charts of roughly 100 adult female patients. Once you've identified 81 that meet the selection criteria, your nursing supervisor fills out the audit tool for each one, reserving questionable cases for physician review.

Step 7: Summarize results. Summarizing the data is a little more complex than just counting up all the data sheets. You must consider how the data will be used and make sure the information is presented in a way that will make it meaningful. Inconsistencies here can produce data that can't be interpreted.

Example: Your breast cancer screening audit results show that 57 percent of your sample received mammograms (see the results table below).

Step 8: Analyze and apply results. Once you have compiled your data and calculated the results, you can compare them to local or national benchmarks. There may be multiple benchmarks, depending on your topic and the performance measure you calculated. You should take into account the differences between your population and those you're comparing it with, as appropriate. If the measure is truly important to the group, you may wish to set a performance goal based on what the group feels is appropriate and reasonable and make it the focus of a quality improvement initiative.

Example: At 57 percent, your group's breast screening rate is less than the national benchmark of 68.9 percent. This benchmark is the mean for commercial HMO patients, according to the HEDIS 2007 Audit Means, Percentiles and Ratios, the NCQA's annual report of health plan performance data (view it at http://www.ncqa.org/tabid/334/default.aspx). Of the 35 patient charts that had no documentation of a mammogram, only 10 records showed that the physician had discussed the need for a mammogram with the patient. The challenge is now to drill down to figure out whether the issue was discussed but not documented in those other charts or whether it was simply overlooked. Telephone contact with the 25 identified patients might help you begin to clarify this so that an appropriate intervention can be designed.

Make it count

Chart audits can be useful tools in improvement and safety efforts. It is essential to define precisely what you want to measure and the criteria by which you will measure it. (If you're floundering, you probably haven't defined this well enough.) Sample sizes can be chosen informally or determined in a statistically valid fashion. Summarize your data in a way that makes sense for the problem you're addressing. Make sure to act on problems you find, and remeasure later to see that your changes made a difference. You and your patients will be glad you did.

Editor's note: An expanded and interactive version of this content is available at http://patientsafetyed.duhs.duke.edu.