| OCR Text |
Show A Decision-Support System for the Analysis of Clinical Practice Patterns E. Andrew Balas, M.D., Ph.D., Zong Rong Li, Ph.D., Joyce A. Mitchell, Ph.D. Donald C. Spencer, M.D., Edward Brent, Ph.D., Bernard G. Ewigman, M.D., M.S.P.H. Health Services Management Program, Medical Informatics Group, and Department of Family and Community Medicine University of Missouri-Columbia Several studies documented substantial variation in medical practice patterns, but physicians often do not have adequate information on the cumulative clinical and financial effects of their decisions. The purpose of developing an expert system for the analysis of clinical practice patterns was to assist providers in analyzing and improving theprocess and outcome ofpatient care. The developed QFES (Quality Feedback Expert System) helps users in the definition and evaluation of measurable quality improvement objectives. Based on objectives and actual clinical data, several measures can be calculated (utilization ofprocedures, annualized cost effect of using a particular procedure, and expected utilization based on peer-comparison and case-mix adjustment). The quality management rules help to detect important discrepancies aowng members of the selected provider group and compare performance with objectives. 7he system incorporates a variety of data and knowledge bases: (i) clinical data on actual practice patterns, (ii) frames of quality parameters derived from clinical practice guidelines, and (iii) rules ofquality management for data analysis. An analysis of practice patterns of 12 family physicians in the management ofurinary tract infections illustrates the use of the system. INTRODUCTION There continues to be a widespread concern that medical practice variation, outcome variation, and the limited impact of new scientific information on practice patterns indicate an urgent need to improve the quality of health services in many areas [1]. Studies documented that physicians often agree with medical practice guidelines but actual practice patterns remain unchanged [2]. Continuous quality improvement, a major goal of physicians and health care organizations, requires the ability to alter medical practice patterns [3]. Compliance with accepted practice guidelines is an important issue but the use of guidelines to develop and implement quality improvement programs is still difficult. The individual differences in information needs and appropriate actions are often not recognized. In recent years, various interventions have been recommended but information methods seem to play the critical central role (e.g., education, reminders, feedback). Several studies have indicated that health care providers are capable of changing their practice styles when confronted with credible information on how they compare to the practice styles of their colleagues [4]. Therefore, comparison to the performance of colleagues is often part of the analyses of clinical practice patterns. Unfortunately, the amount of information on practice styles is often overwhelming and important differences remain undetected. The aims of this project were to assist groups of health care providers in translating accepted clinical practice guidelines into measurable quality improvement objectives, to compare actual practice patterns with objectives, and measure differences among providers in practice style. 0195-4210/94/$5.00 © 1994 AMIA, Inc. 366 CYCLE OF ANALYSIS Continuous quality improvement focuses on processes instead of individuals, evaluates through measurement and data, and interprets detected defects as opportunities for improvement. The data processing functions, knowledge bases, and inference engine of QFES were designed to support the critical steps of clinical quality improvement (Fig. 1): Figure 1 System Structure I. Definition of a quality improvement plan requires identification of measurable objectives. The system supports the use of a variety of performance-based measures for the purpose of comparing individual data with those of some comparison group. This process is supported by providing a structure for the definition of quality objectives and also a library (knowledge base) of quality parameters (see knowledge representation). The user of QFES can select, edit, and also supplement the practice parameters readily available in the knowledge base (Fig. 2). The quality improvement plan is not only a list of specific objectives but also an important point of reference for subsequent analyses of practice patterns. II. Evaluation of group performance and variation is based on actual practice data and the quality improvement plan. Analyses of practice variation can generate a large number of data and detection of clinically significant or costly differences can be difficult. The production rules of QFES assist interpretation by detecting significant discrepancies between recommendations and practice patterns and by identifying clinical practice variation among providers. The system combines the following methods in the analysis of practice patterns: - Four measures are calculated: (i) crude utilization describes the provider-specific frequency of using a selected procedure without adjusting for case-mix differences; (ii) standardized utilization is the crude utilization divided by the utilization expected on the basis of the case-mix of the analyzed provider and practice pattern of his or her peers, (iii) relevant utilization which measures the use of the procedure in the group of eligible patients, and (iv) deviant utilization which measures the use in the group of ineligible patients, as defined by an established clinical practice guideline. Expected utilization is the average utilization rate of the physician group weighted with the number of patients of the analyzed provider in each severity categories. - The system can express all above listed measures of utilization in three different ways: (i) number of procedures (clinical activities) ordered or performed, (ii) utilization rate (frequency of using a selected procedure), and (iii) cumulative cost effect (annual number of procedures multiplied by the unit costs). For example, the standardized utilization can be expressed as an excess or shortage in the number of procedures performed or ordered by a particular provider. - The knowledge base of quality management rules and corresponding inference engine assist the interpretation of practice data in the analyses of group performance. The calculation of data leads to a data-driven search which generates messages about significant discrepancies and makes recommendations for possible interventions. The group analyses of data can overview the use of several procedures by all participating providers or, alternatively, can focus on a selected procedure. III. Based on the results of group 367 Figure 2 Selection of Quality Parameters analysis, individualized feedback information will be available for the participating providers. Feedback and corrective action are considered the last steps of quality improvement projects and the QFES is designed to support these steps. Providers will be informed only about those aspects of their practice patterns which indicate achievements or need for improvement. KNOWLEDGE REPRESENTATION The unique approach of QFES is the separation of guideline specific and quality management knowledge. Through this separation, the QFES system can effectively handle the recommendations of various and evolving clinical practice guidelines. The user of the system gets full support for modification and supplementation of the structured knowledge which was made available by abstracting clinical practice guidelines. Guidelines can be edited in the knowledge base to make a permanent change or during the process of quality improvement plan development. In the latter case, the knowledge base (library of clinical guidelines) remains unchanged. The current version of the QFES system was built on a Level-5 platform [5] and uses one practice database and two knowledge bases: Guidelines Knowledge Base This knowledge base is a frame-based representation of clinical quality parameters. Definition of the quality improvement plan requires the identification of measurable objectives and widely accepted clinical practice guidelines are available for this purpose in many areas. However, clinical practice guidelines are usually developed for one-on-one patient care and measures of practice patterns are rarely mentioned. Identification of measurable quality improvement objectives in published clinical practice guidelines requires considerable effort. To support the users of QFES, a library of quality parameters is available for the definition of quality improvement objectives. In the Guidelines Knowledge Base, each quality parameter is described by a name, numerator (procedure), denominator (clinical condition), target range, and a few quotations justifying the 368 target range with references. The definition of a parameter also includes supportive evidence beyond the quotation from the guidelines (e.g., results of meta-analyses, randomized clinical trials, or other types of original research). The user can copy selected parameters or entire guidelines (sets of parameters) to the quality improvement plan (Fig. 2). Practice Database The analysis of practice patterns requires cumulative data about the patient care decisions from each participating provider within the specified time period. The input dBASE file consists of seven fields: name of the provider group, beginning and ending dates of the data collection period, provider name or identifier, name of the severity group, number of cases seen by the particular provider in the specified severity group, name of procedure, and number of procedures ordered for the patients of the particular severity group. In comparison to other clinical systems, the size of the practice database is small. Based on six severity groups and 10 parameters, an analysis of the practice patterns of 50 providers would require about 6000 data items (5OX6X10 multiplied by two for the numerator and denominator). The size of practice database does not depend on the length of analyzed period or number of procedures performed. This simple and concise structure of cumulated practice data allows calculation of all major utilization measures, including crude utilization, case-mix adjusted utilization, and relevant/deviant utilization. The Practice Database of the system is a collector of information for the analysis of variation and deviation. The necessary data are defined by the selected clinical practice parameters (measurable objectives). Most commonly used administrative databases do not have sufficient data for the comparison of practice patterns with the specific recommendations of clinical guidelines. However, matching data from different sources can often provide the necessary information. For example, in the analysis of management of diabetes the provider-specific number of patients with diabetes can be identified in the diagnosis-procedure database and the corresponding number of glycosylated hemoglobin determinations can be retrieved from the laboratory database. In developing the QFES system, it was anticipated that the information has to be obtained from variety of sources (e.g., computer file of patient-physician encounters, referral database, or manual chart reviews). It is an obvious advantage to have a comprehensive electronic patient records in place but the lack of such system is not prohibitive. Management Rule Base This knowledge base is a production rule-based representation of the logic of quality management. In the analyses of clinical practice patterns, many different utilization rates need to be evaluated. In addition, variation and deviation are frequent observations but, often, reflect only random variation or practically insignificant differences. The management rule base contains specific production rules to detect substantial variation in the use of observed procedures or deviation from the recommendations of clinical guidelines. The IF part of the rules specifies the deviation or variation in practice patterns which needs attention and the THEN part specifies a corresponding message of recommendation. One group of rules defines criteria for sending the messages specified in the evidence section of the clinical practice parameters (e.g., if the difference between the recommended rate and the utilization rate of the analyzed provider exceeds 20% then present the listed evidence). The other group of rules interprets practice variation and generates additional messages (e.g., if the utilization rate of the analyzed provider is below the average of the physician group by more than two standard deviations then send a statement on that difference). Through the evaluation of a series of production rules, the QFES system can analyze practice data, compare them to quality improvement objectives, and make recommendation for further actions. QFES is based on the integration of object oriented techniques, expert system technology, and traditional procedural programming. Objects in the knowledge base were created via class declarations. To facilitate future knowledge sharing, the Management Rule Base and Guidelines Knowledge Base were 369 structured similarly to the logical modules of the Arden Syntax. QFES operates in a Windows environment and has a graphical user interface. Most interactions are menu- and mouse-driven. CONCLUSIONS The QFES was first used to analyze practice pattern data in the management of urinary tract infections. The source of data was Callaway Physicians, a non-profit family practice center providing fee-for-service care in Fulton, Missouri. The medical office uses the COSTAR system to support the documentation of patient care. The sample of the study consisted of patients who were diagnosed with cystitis, pyelonephritis, or urinary tract infection during a six month period. A retrospective chart review used a form designed for the analysis of urinary tract infection episodes. The case-mix analysis was based on the recently published clinical practice guidelines for the management of urinary tract infections in adults [6]. The analysis documented that several questions and procedures were used with significantly varying frequencies by different providers. Practice parameters for other applications of the QFES are currently under development (diabetes, depression, and HIV infection/AIDS in primary care). According to a survey by the American Medical Association, more than half of physicians are subjected to either clinical or economic profiling [7]. It is anticipated that the prevalence of physician profiling will continue to rise. However, most available physician profiling systems produce reports which have no relationship with established clinical practice guidelines. Furthermore, analyses of clinical practice patterns focus on a single selected procedure and not on the process of care. Expensive or hazardous procedures are often subjected to analysis of practice variation. However, it can be very difficult to interpret variation when other clinical actions surrounding the arbitrarily selected procedure are not analyzed. Continuous quality improvement requires reengineering of clinical processes and, therefore, utilization data are needed on the entire sequence of actions. The QFES supports the full spectrum of process analysis and the evaluation can be based on a series of parameters derived from established clinical practice guidelines. The system provides a new structure for knowledge elicited from the medical literature and takes advantage of the common principles and implementation aspects behind the major content differences of various guidelines. ACKNOWLEDGEMENT This project was supported in part by grant number HS07715 from the Agency for Health Care Policy and Research. REFERENCES 1. Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: A critical analysis of the experimental literature. Milbank Quarterly 1989;67:268-317. 2. Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med 1989;321:1306-11. 3. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989;320:53- 6. 4. Buck CR, White KL. Peer review: Impact of a system based on billing claims. N Engl J Med 1974;291 :877-83. 5. Information Builders. Level 5 Object for Microsoft Windows: Reference Guide, 1994 6. Stamm WE, Hooton TM. Management of urinary tract infection in adults. N Engl J Med 1993;329: 1328-33. 7. Emmons DW, Wozniak GD. Profiles and feedback: Who measures physician performance? In Gonzalez ML (Ed) Socioeconomic Characteristics of Medical Practice, AMA,1994. 370 |
