With more and more family physicians trying to decide whether to move to an electronic health records (EHR) system and which system to choose, we thought it high time we repeated the survey of user satisfaction with EHR systems that Family Practice Management conducted in 2005.¹

We published a revised and simplified version of the 2005 survey instrument in the April 2007 issue.² Over the next several months, we collected 422 usable responses from AAFP members who completed either the print version of the survey or an online version posted on the FPM Web site. As with the 2005 survey, respondents were self-selected. Consequently, it is probably most useful to consider this report as the kind of information you might get if you could ask a few hundred colleagues how they like their EHR systems.

The 422 respondents reported on a total of 61 different EHR systems, the majority of which were reported by three or fewer respondents; on the other hand, the 13 systems most commonly mentioned, each reported by 12 or more respondents, accounted for 80 percent (336) of the responses. In the analysis below, we report system-specific results for these 13 commonly reported systems, using pooled data from all 422 respondents as a point of reference against which to view the system-specific results. We chose to focus on these 13 systems because we believed that we had enough responses on each to represent a reasonable spread of opinions on the system. The 13 systems in question are listed below.

The respondents

Given the wide publication of the survey instrument, we accepted responses only from AAFP members as a way of avoiding frivolous responses, multiple responses per individual and other potential sources of bias. Respondents came from a range of practice situations, with 36 percent (152) in solo or two-person practices, 31 percent (132) in practices of three to 10 physicians, 20 percent (83) in practices of 11 to 50 physicians, and 13 percent (55) in larger practices. Most of the respondents (70 percent, or 295) were in single-specialty family medicine practices.

As you might expect, certain EHR systems were reported more commonly by respondents in smaller practices and other systems more commonly by those in larger practices. See the practice size distribution of the 13 most commonly reported systems below. When choosing a system for your practice, you might find this chart a useful reference, since appropriateness to practice size is an important consideration.

Respondents' range of experience with EHRs was quite wide, with a few reporting more than 10 years of experience with their current system. The bulk of respondents were less experienced, however, with 80 percent (336) reporting 0.5 to 5.0 years of experience with their current systems. Respondents were generally pretty sanguine about their ability to use their EHR system, with 72 percent rating their skill as above average or expert. Since the survey did not specify behaviors or capabilities corresponding to the levels of expertise, the reported levels should be taken as indications of the respondents' confidence in their own abilities rather than as measures of actual abilities.

Some 65 percent of respondents (275) said that they helped select the EHR system they use. Unsurprisingly, users of systems common in small practices were most likely to have helped. While all respondents for Praxis EMR and Amazing Charts said they had been involved in the selection, only 14 percent of EpicCare respondents (3) and no respondents for Cerner PowerChart Office did.

While the survey did collect this sort of limited information about the respondents, its focus, as the name implies, was on the users' satisfaction with the EHR systems they used. It assessed satisfaction in several areas:

• Functions facilitated or performed by the system (22 items),

• Ease of use and flexibility (4 items),

• Service and support (4 items),

• Cost (3 items),

• Interoperability (10 items),

• Security (3 items),

• Overall satisfaction (6 items).

Functionality

The survey asked respondents to report their satisfaction with the functionality of their EHR systems by indicating their degree of satisfaction or dissatisfaction with how their systems allowed them to perform various functions such as “review chart information” or “update and review problem lists.” In each case, the respondent could choose Very satisfied, Satisfied, Neutral, Dissatisfied, Very dissatisfied, Function not installed or Function installed but not used.

“Functionality rankings for 13 EHR systems” shows how the 13 systems ranked on each of the functions asked about, with rankings determined by percentage of Satisfied and Very satisfied responses combined. For each of the functions listed, the four highest ranks are tinted blue and the four lowest tinted red to help show overall satisfaction, and the 13 systems are listed across the top of the table in order of overall average rank, with the ones getting the best evaluations from their users at the left. The 22 areas of functionality are ranked separately in case any should be of particular importance to you.

For a better picture of general satisfaction with these 22 areas, see “Overall satisfaction with functionality of 13 EHR systems.” This chart represents the aggregate of responses to the 22 items in the functionality section. Respondents who use e-MDs Chart seem most satisfied with its functionality, with 80 percent of their ratings being Satisfied or Very satisfied. At the other extreme is Misys EMR, with 44 percent Satisfied or Very satisfied – and 34 percent Dissatisfied or Very dissatisfied.

A word about the chart format used in this article: By displaying the four “satisfaction/dissatisfaction” bars in characteristic colors and arranging them around the center line, the chart gives you a way to compare those ratings at a glance. And by including bars for other responses such as Function not installed, it gives you a more complete picture of the data collected. The total length of each bar represents 100 percent of responses, with some slight variation because of rounding and a handful of blank responses. Just remember that the less colorful bars for Neutral, Function not installed, etc. are not necessarily measures of satisfaction or dissatisfaction; they could as easily be shown on the right as on the left of the satisfaction/dissatisfaction bars.

Ease of use

Respondents were asked to indicate the extent to which they agreed or disagreed with four statements concerning ease of use and flexibility:

• This EHR allows individual user-specific customization.

• This EHR minimizes data input.

• This EHR offers multiple note creation options.

• This EHR is fast (minimal wait between screens, minimal boot-up time, etc.).

Of the 13 systems, three received below-average ratings on all four of the above statements: Cerner Power-Chart Office, Misys EMR and NextGen EMR. On the other hand, six systems received above-average ratings on all four: Amazing Charts, e-MDs Chart, EpicCare, Practice Partner Patient Records, Praxis EMR and SOAPware. Perhaps unsurprisingly, users were generally least satisfied with the amount of data input required by their systems. Only 48 percent of the 422 respondents indicated that they agreed or strongly agreed with the statement that “this EHR minimizes data input,” while 68 percent to 78 percent agreed or strongly agreed with the other three statements regarding ease of use and flexibility. “Overall satisfaction with ease of use and flexibility of 13 EHR systems” shows how the 13 systems did in the aggregate. Praxis EMR and e-MDs Chart came out on top, with Cerner PowerChart Office and NextGen EMR at the bottom.

Support

The four survey items devoted to service and support were these:

• Our vendor provided excellent support during our implementation period.

• Our vendor provides excellent ongoing support and service.

• Our vendor issues at least one significant system upgrade per year.

• Our vendor assists with ongoing training.

Overall, 72 percent of respondents (305) agreed or strongly agreed with the statement about system upgrades, while 50 percent to 57 percent of respondents agreed or strongly agreed with the others. Four systems rated below average in all four areas – Centricity, Cerner PowerChart Office, Misys EMR and TouchWorks – and six rated above average in all four: Amazing Charts, eClinicalWorks EMR, e-MDs Chart, HealthMatics EMR, Praxis EMR and SOAPware. See “Overall satisfaction with support and training for 13 EHR systems” for the aggregate results.

System cost

As in our 2005 survey, we asked respondents for an estimate of the per-physician cost of their system. We learned from the responses that the situation hasn't really changed in the last two years: Physicians generally have no more than a vague sense of EHR cost. Per-physician cost estimates ranged from $1,000 to $350,000, with more than 150 respondents not responding or indicating that they didn't know the cost.

Responses to two other cost-related items, nevertheless, seem to have some value. One asked respondents to indicate their level of agreement with the statement, “This EHR costs more than it's worth.” This item called more for a value judgment than a financial assessment, and we'll cover the responses below, in the section on overall satisfaction.

The other, which gauged agreement or disagreement with the statement, “This EHR has saved or will save my practice money over its first five years of use,” may not reflect reality, considering that many medical practices do not seem to have a good handle on their finances, but it did produce intriguing results. As you'll see from the chart, respondents for some systems were decidedly in agreement with the statement; systems further down the chart show both weaker agreement and larger percentages of “no opinion” responses. But if you compare this chart with the one on practice sizes, you'll see that the systems used primarily by smaller practices are the ones that respondents believe most strongly are saving their practices money. Thus, the high levels of agreement and the relatively small, or in two cases nonexistent, component of No opinion responses suggest that physicians in smaller practices may be both more in touch with their financial performance and more positive that their EHRs are saving them money than physicians in larger groups.

Interoperability

The more systems an EHR directly interfaces with, the greater its usefulness. An EHR interfaced with a lab system, for instance, can obtain patient lab results automatically and easily flowchart the results. Similar benefits accrue from other interfaces. Hence, we asked respondents whether their systems were interfaced with a practice management system, a lab system, a radiology system, a pharmacy system or a hospital system.

The most commonly reported interfaces were with lab systems and practice management systems. We ranked systems by percentage of respondents reporting three or more interfaces. Overall, 58 percent of respondents reported that many interfaces. The six above-average systems, by this measure, were Centricity, Cerner PowerChart Office, eClinicalWorks EMR, EpicCare, HealthMatics EMR and TouchWorks. Perhaps unsurprisingly, most of these are systems used in larger practices. The main exception is eClinicalWorks EMR. Of respondents who reported using it, 63 percent say their installations have at least three interfaces, but 77 percent are in practices with no more than five physicians.

Security

The survey included three items designed to assess EHR system security – yes/no questions asking whether the respondent's system required a password for access, could allow different users different levels of access to records in the system, and kept an audit trail of information about who had accessed which charts. Password protection seems to be generally available, with 98 percent of users of the 13 most common systems reporting that their systems had it. Access restriction was also pretty widely implemented, with 90 percent of respondents for the 13 systems reporting it. The audit trail feature was reported by only 74 percent of users of the 13 systems. Audit trails were most commonly reported by users of eClinicalWorks EMR, e-MDs Chart, EpicCare and HealthMatics EMR.

Overall satisfaction

Since a physician's opinion of his or her EHR system is probably too nuanced to be captured in a single question about overall satisfaction, the survey attempted to approach that issue from three separate directions, with items that asked respondents to agree or disagree with three statements, one of which invited expressions of satisfaction and two that invited expressions of dissatisfaction:

• Given the opportunity to pick an EHR system again, I would pick the same EHR.

• This EHR costs more than its worth.

• If I could go back to paper-based records with no financial penalties, I would do so.

Ranking responses to these three items based on the most favorable responses – Agree and Strongly agree for the first item and Disagree and Strongly disagree for the other two – produces the results shown in the table. Percentages in the table are percentages of favorable responses. To take Praxis EMR as an example, 83 percent of respondents agreed or strongly agreed that they would pick the same system again, 100 percent prefer their EHR system to paper records, and 92 percent believe the system did not cost more than it's worth. The blue tint in the table identifies cases where an EHR system got at least an 80 percent favorable rating. The three systems with the highest overall ratings by these measures, then, are Praxis EMR, Amazing Charts and eClinicalWorks EMR. The three with the lowest ratings are Misys EMR, TouchWorks, and Cerner PowerChart Office.

Of course, these ratings need to be considered not only in context with the others we have presented but also in the context of practice size. Whether coincidentally or not, the three highest rated systems are all among those used primarily by respondents in smaller practices, while the three lowest rated are more common in larger practices.

A note of caution

While we believe that the results of this survey should be helpful to physicians in the process of selecting an EHR system, we need to emphasize several limitations of the survey. That respondents were self-selected may mean that the survey attracted EHR enthusiasts, or at least physicians with particularly strong feelings about their EHRs, positive or negative. That a majority of respondents reported having helped pick their current EHR might mean a relatively high representation of positive opinions, since the results showed that respondents who said they had helped choose their current system were more likely to say they would pick the same one again. Finally, cell size is a problem in two senses. By considering only systems for which we had 12 or more respondents, we necessarily omitted numerous systems; on the other hand, by including systems for which we had as few as 12 respondents, we risked additional bias. As we said to begin with, it's probably best to consider the survey results as input you'd get from a few hundred colleagues who volunteered to report on their EHR experience.