University of Southern California Dissertations and Theses

Assessment of prognostic comorbidity in hospital outcomes research: Is there a role for outpatient pharmacy data?

(USC Thesis Other)

Assessment of prognostic comorbidity in hospital outcomes research: Is there a role for outpatient pharmacy data?

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UM I films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UM I a complete manuscript and there are missing pages, these w ill be noted. Also, if unauthorized copyright material had to be removed, a note w ill indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Photographs included in the original manuscript have been reproduced xerographicaliy in this copy. Higher quality 6" x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UM I directly to order. ProQuest Information and Learning 300 North Zeeb Road, Ann Arbor, M l 48106-1346 USA 800-521-0600 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ASSESSMENT OF PROGNOSTIC COMORBIDITY IN HOSPITAL OUTCOMES RESEARCH: IS THERE A ROLE FOR OUTPATIENT PHARMACY DATA? by Joseph Paul Parker A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (PUBLIC ADMINISTRATION/PUBLIC POLICY) August 2000 Copyright 2000 Joseph P. Parker R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. UMI Number: 3054893 ___ ® UMI UMI Microform 3054893 Copyright 2002 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. UNIVERSITY OF SOUTHERN CALIFORNIA TH E GRADUATE SCHOOL UNIVERSITY PARK LOS ANGELES, CALIFORNIA 90007 This dissertation, written by Joseph Paul Parker under the direction of h.iS Dissertation Committee, and approved by all its members, has been presented to and accepted by The Graduate School in partial fulfillment of re quirements for the degree of DOCTOR OF PHILOSOPHY ........................ Dean of Graduate Studies D a te.. Au£ust„.4> .„2p00....... DISSERTATION COMMITTEE Chairperson <Zuk a . saiL ^ R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. DEDICATION TO MY FATHER, WHOSE WORK TO FINISH HIS DISSERTATION WAS INTERRUPTED BY AN EARLY DEPARTURE FROM THIS LIFE. TO MY MOTHER, WHO HELPED TO MAKE EVERYTHING POSSIBLE. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. ACKNOWLEDGMENTS The author acknowledges the support of individuals at the USC Department of Pharmaceutical Economics and Policy and the Southern California Region of the Kaiser Permanente Medical Care Program for making these data available and providing guidance in their usage. In particular, Jeffrey S. McCombs, the Principal Investigator for the Kaiser Permanente/USC Patient Consultation Study, provided much critical help. Kathleen A. Johnson also provided much appreciated clinical advice and support. Other faculty, students, and staff in the Department of Pharmaceutical Economics and Policy were also very generous with their time and expertise. The author expresses deep appreciation to his Dissertation Committee Chair, Elizabeth Graddy, at the School of Policy, Planning and Development, for her counsel. Robert Stallings and James Fern's, the remaining two committee members from the same school also provided the questions and some answers that helped bring this task to its completion. Heartfelt appreciation also goes to the close friends and colleagues who persevered beside me and offered encouragement. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. TABLE OF CONTENTS DEDICATION.......................................................................................................................ii ACKNOWLEDGMENTS....................................................................................................iii LIST OF TABLES................................................................................................................vi ABSTRACT........................................................................................................................viii INTRODUCTION.................................................................................................................. 1 Chapter Descriptions...........................................................................................................6 CHAPTER 1 REVIEW OF THE LITERATURE...............................................................9 Policy Issues.......................................................................................................................9 Theoretical Perspectives on Patient Comorbidity............................................................12 Methodological Issues in Risk Adjustment..................................................................... 15 Information on Comorbid Illness..................................................................................... 16 Outcome Measures.......................................................................................................... 41 Conclusion........................................................................................................................ 51 CHAPTER 2 METHODS.................................................................................................. 54 Sample Construction.........................................................................................................54 Data Sources.....................................................................................................................56 Variable Construction...................................................................................................... 57 Measures........................................................................................................................... 61 Predictive Models.............................................................................................................67 Measures of Model Performance..................................................................................... 70 CHAPTER 3 DESCRIPTIVE STATISTICS.....................................................................72 Demographic, Hospitalization and Outcomes Characteristics.......................................73 Prevalence o f Disease Indicated by Comorbidity Measures..........................................77 Agreement Between Comorbidity Measures...................................................................79 CHAPTER 4 COMPARISONS OF PREDICTIVE MODEL PERFORMANCE........... 86 Age/Sex Model Predictive Performance..........................................................................87 Full Information Model Predictive Performance............................................................ 98 Predictive Performance of Revised CDS with Selected Patients.................................I ll iv R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Summary..........................................................................................................................115 CHAPTER 5 DISCUSSION............................................................................................ 119 Comparisons of Overall Model Performance................................................................120 Demographic and Hospitalization Characteristics Related to Outcomes.....................123 Specific Medical Conditions Related to Outcomes.......................................................124 Policy Implications.........................................................................................................125 Study Limitations............................................................................................................ 130 REFERENCES................................................................................................................... 133 v R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. LIST OF TABLES Table 2.1 Study Variables................................................................................................... 59 Table 2.2 Comorbidities Included in Deyo Measure.........................................................62 Table 2.3 Comorbidities Included in Elixhauser Measure.................................................63 Table 2.4 Chronic Conditions Included in Revised CD S..................................................66 Table 2.5 Age/Sex Models to Predict Complications, Readmissions and LOS................68 Table 3.1 Demographic Characteristics of Study Sample.................................................74 Table 3.2 Hospitalization Characteristics of Study Sample...............................................75 Table 3.3 Outcomes and Treatment Models for Study Sample....................................... 76 Table 3.4 Number and Percent of Patients with Deyo Comorbidities.............................77 Table 3.5 Number and Percent of Patients with Elixhauser Comorbidities.....................78 Table 3.6 Number and Percent of Patients with Revised CDS Conditions......................80 Table 3.7 Conditions Held In Common By Three Comorbidity Measures...................... 82 Table 4.1 Prediction of Hospital Outcomes Using Age and Sex...................................... 88 Table 4.2 Deyo + Age/Sex Model Prediction of Hospital Outcomes...............................89 Table 4.4 Revised CDS +Age/Sex Model Prediction of Hospital Outcomes...................94 Table 4.5 Predictive Performance of Revised CDS Controlling for Diagnosis-Based Comorbidity................................................................................................................... 96 Table 4.6 Prediction of Hospital Outcomes Using Full Information model.....................99 Table 4.7 Deyo + Full Information Model Prediction of Hospital Outcomes................102 Table 4.8 Elixhauser + Full information model Prediction of Hospital Outcomes 104 vi R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.9 Revised CDS + Full information model Prediction of Hospital Outcomes... 106 Table 4.10 Predictive Performance of Revised CDS Controlling for Diagnosis-Based Comorbidity..................................................................................................................109 Table 4.11 Predictive Performance of Revised CDS for Patients With No Diagnosis- Based Comorbidity...................................................................................................... 112 Table 4.12 Predictive Performance of Revised CDS for Patients With No Diagnosis- Based Comorbidity (Revised Model)..........................................................................114 Table 4.13 Increases in Overall Model Fit From Addition of Comorbidity Measures.. 116 vii R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. ABSTRACT During the last two decades there has been increasing public interest in providing consumers of health care with quality-based assessments of the medical care provided at hospitals. The success of such profiling efforts depends in part on our ability to adequately adjust for differences in patient severity of illness and physiologic reserve that independently influence hospital outcomes. Failing to adequately control for prognostic conditions will result in the lowest quality rankings being assigned to hospitals with the sickest patients. Current efforts to adjust for patient risk using only hospital administrative data do not appear to account for all prognostic illness found in the patient medical record. This thesis explores the use of automated outpatient pharmacy records as an additional source of chronic disease information in hospitalized patients. It is hypothesized that a measure of patient comorbid illness based on information from both the hospital discharge abstract and patient prescription history will provide a better assessment of prognostic comorbidity than a measure based solely on hospital administrative data. Measures of comorbid illness were developed from automated hospital diagnostic data and patient pharmacy records for a sample of acute hospitalizations (N=3558) in a large Southern California managed care plan. Logistic and ordinary least squares regression models were used to examine the predictive performance of two diagnosis- viii R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. based measures and one pharmacy-based measure of comorbidity. Two hospital outcomes used as potential quality markers (complications of care and unplanned hospital readmission) and another important outcome (patient length of stay) served as dependent measures. Models were also developed to test for improvement in model fit resulting from the addition of pharmacy-based comorbidity to diagnosis-based comorbidity assessments. A pharmacy-based measure of chronic disease provided predictive performance similar to that of a widely used adaptation of the Charlson Comorbidity Index. The combined comorbidity measure significantly improved prediction of complications, but not unplanned readmission or length of stay. The results suggest that a measure of comorbid illness that includes information from two administrative data sources may improve prediction of an important hospital outcome. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. INTRODUCTION Much current and proposed health policy depends on an accurate assessment of patient health status at a specific point in time, especially aspects of health that contribute to future adverse events and increased health services utilization. State level risk-pooling schemes for providing subsidized insurance to the poor, prospective federal payment systems such as Diagnostic Related Groups (DRGs) and state or local proposals to rank acute care hospitals on their in-hospital mortality rates all require an accurate assessment of patient health status to be practicable. Thus, the development of a valid and reliable measure of patient risk is essential to future progress in a number of health policy arenas. The policy context of this thesis is the assessment of medical care quality at the hospital level; however, any methodological improvement in patient risk assessment will have many applications. During the last 2 decades, quality in health care services has generally been conceptualized as resulting from the relationship between structure, processes, and outcomes (Donabedian, 1980). While some researchers have argued that the most direct and relevant measures of quality are to be found in the processes of care, outcomes have been seen as important, though perhaps indirect measures of quality. The emphasis on outcomes assessment in health care mirrors similar movements in other areas of society R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. as the popularity of a quantitative assessment of public and private goals has gained momentum. Several risk-adjusted hospital outcomes have been proposed as markers of hospital quality. These include rates of patient mortality, complications of care, unplanned readmissions and, to a lesser extent, length of hospital stay. There is no consensus that risk-adjusted measures of hospital performance provide evidence o f the quality of medical care provided, but their potential value as quality screening instruments has been widely acknowledged and most researchers have applauded efforts to further their development. A viable risk adjustment methodology in health care will need to be valid and reliable under a variety of conditions, and it will also need to be readily available and relatively easy to apply. The information used to construct measures of risk for hospitalized patients has generally originated from two sources: the patient medical chart and the automated hospital discharge abstract. The patient medical record has been considered the gold standard in terms of documenting patient illness and is referred to as the clinical record. Most research conducted using patient medical records has required the manual abstraction of data from medical charts, limiting its usefulness as a timely source o f information. The hospital discharge abstract, often filed with government agencies, insurers, and other parties by the reporting hospital, is an attractive alternate source o f inpatient information. It is available in a computer-friendly fomat for nearly all patient admissions and can be made available more quickly than medical chart data. Its primary purpose has been administrative, often for use in processing insurance claims, so 2 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. it is commonly referred to as claims or administrative data. Because of the wide availability of automated patient discharge abstracts, there has been a major policy push to use these data in a manner that benefits individual and institutional consumers of health care in their decision making. The list of patient-level characteristics required to successfully predict patient outcomes is probably innumerable, and new research continues to point to new explanatory factors. In most cases, however, the uppermost limit is defined by what exists in the patient medical chart, though supplementary patient questionnaires can tap other important factors such as patient quality of life and functional status. The automated hospital discharge abstract contains only a subset of the information in the medical chart. In quality assessment research using hospital discharge data, commonly included data are patient level characteristics, patient demographic information, characteristics of the admission, severity of primary illness assessments, information concerning health insurance status, and the presence of comorbid illness, among other things. In the last decade, considerable effort has been expended in investigating the influence that comorbid illness has on patient outcomes. This dissertation focuses on the role of comorbid illness and how it relates to common hospital outcomes. Comorbid illness in inpatient populations has been defined as “the state of health at admission apart from the primary diagnosis” (Greenfield, Aronow, Elashoff & Watanabe, 1988, p. 2253) and is recognized as an important factor in determining outcomes. Comorbid illness 3 R eproduced with perm ission o f the copyright owner. Further reproduction prohibited without perm ission. includes both acute and chronic conditions, but the present investigation is primarily concerned with how chronic illnesses influence the results of patient care. Research using both clinical and claims-based measures to adjust for comorbid illness has demonstrated a significant relationship between comorbidities and outcomes such mortality, complications of care, unplanned readmissions, and length of stay (LOS) (Iezzoni, 1997a). This impact has been noted for patients with various primary conditions undergoing both medical and surgical procedures. However, the estimated impact of chronic conditions has varied greatly depending on the type of comorbidity measure used, the reason for admission, and the specific outcome being studied. Many researchers have questioned the ability of claims data to provide an accurate assessment of patient comorbid illness and have noted its poor validity relative to the clinical record. In particular, a high rate of unreported chronic illness has been found when claims databases have been compared with the clinical record. Despite these limitations, policy makers have felt compelled to move forward with risk-adjustment strategies that rely solely on administrative data in assessing the hospital-specific quality of patient care. This has been done despite hospital protests against the ranking methodologies and the discovery of significant limitations in various systems. It appears that the pressure to continue publishing quality “report cards” will not diminish as a more market-oriented approach to health care reform takes hold (Iezzoni, 1997b) This thesis presents an innovative methodology that addresses a stubborn problem faced by researchers when relying on claims data to construct measures of patient risk. It examines whether the merging of automated pharmacy data with patient-level hospital 4 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. discharge data results in improved assessments of patient burden of illness at the time of admission. The patient prescription record, when reasonably complete, should serve as an important additional source of information regarding patient health status when the indications for drug therapy are considered. As part of this exploration, the relative ability of pharmacy and diagnosis-based measures of comorbidity to predict important hospital outcomes is also assessed. Prescription records do not provide patient diagnostic information but several classification systems have been developed that group prescription drugs into therapeutic classes (e.g., AHFS). These classes often relate quite directly to the conditions for which they are being prescribed. For chronic conditions such as diabetes or Parkinson’s disease, the link between drug therapy and medical condition has great face validity. In other cases, the link between drug therapy and medical conditions is less direct. For example, chemotherapy is a commonly prescribed treatment for many types of cancer but it is also used to treat certain rheumatological conditions. A small body of research using pharmacy data to assess health status and relate it to patient outcomes has also appeared in the health services literature. Claims-based prescription data is nearly always automated and is considered to be a reliable and accurate source of information (Christensen et al., 1994). In some instances, where state or federal agencies retain information on both hospitalizations and pharmacy expenditures (e.g., Medicaid and Veterans Affairs programs), linkages between hospital and pharmacy data sources might be made with minimal effort. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The general thesis is that information from the patient prescription record, when combined with comorbid disease diagnoses from the hospital abstract, will provide a more complete picture of patient comorbid illness at time of admission than secondary diagnoses alone. This, in turn, will result in the improved ability of statistical models to explain hospital outcomes such as mortality, unplanned readmissions, complications of care, and hospital length of stay. The hypothesis to be explored may be stated more formally as: A model of patient comorbid illness that combines patient prescription drug use information with hospital administrative data will exhibit superior performance in predicting unplanned hospital readmission, complications of care, and patient length of stay than a model of comorbidity that relies solely on hospital administrative data. Chapter Descriptions The first chapter presents a review of the literature on claims-based measures of patient comorbidity as risk adjusters in assessing hospital outcomes. It begins by reviewing theoretical perspectives on patient comorbidity and current practices in measurement, and introduces some of the methodological issues and statistical problems faced when employing claims-based measures of comorbidity. Studies that have employed the most common patient comorbidity measures are reviewed and the nascent literature on pharmacy-based risk assessment is introduced. Two persistent problems with studies that employ diagnosis-based comorbidity measures are described and 6 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. pharmacy-based comorbidity assessment is introduced as a possible solution for one of these problems. Hospital outcomes that have been used in the literature as proxies for quality of care are reviewed and evidence o f their association with processes of care is summarized. The second chapter describes the methodological strategies employed to answer the overall research question posed. The methods used to derive the population sample are described and the source of data and its original purpose in a pharmaceutical care project are explained. The derivation and source of demographic, hospitalization, outcome, and comorbidity variables are described. The multivariate statistical techniques used to model predictors of complications o f care, unplanned hospital readmission, and hospital length of stay are then reviewed. Chapter 3 presents a description of the sample population of acute adult admissions in the first study year that are used in this study. The demographic and hospitalization characteristics of the 3558 patients who passed the exclusionary criteria and comprise the study sample are presented. The prevalence of disease for the two diagnosis-based measures of patient comorbidity is presented and the prevalence of chronic disease according to the pharmacy-based markers is presented. The frequency of broad disease categories as indicated by the three different measures are then compared and the strength of agreement across specific measures and data sources is assessed. The results of multivariate analyses to predict hospital outcomes using logistic and ordinary least squares regression are presented in Chapter 4. Results of models that employ only age and sex with the three comorbidity measures are presented first, 7 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. followed by models that employ additional patient-level information from the hospital abstract. The relative performance of each of the measures in predicting the hospital outcomes is reviewed and any additional explanatory power contributed by pharmacy data to diagnosis-based measures of patient risk is also assessed. Finally, a subanalysis is presented where the pharmacy-based risk adjuster is employed to predict hospital outcomes for patients who have no prognostic comorbidity according to the hospital abstract. A discussion of the results with an emphasis on the policy implications is contained in Chapter 5. The performance of the pharmacy-based measure of chronic disease was not appreciably better or worse than that of a widely used diagnosis-based measure o f comorbidity. Drug-based comorbidity was an excellent predictor of complications but was a much poorer predictor of LOS and unplanned readmission. The explanatory power of most models compared favorably with that of studies based on administrative data published in the literature during the last decade. Examination of individual disease markers revealed only modest support for the hypothesis that a measure o f chronic disease based on chug prescriptions could help resolve serious problems with claims-based outcomes research. The study limitations are discussed and a possible first step for further research is suggested. 8 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CHAPTER 1 REVIEW OF THE LITERATURE Policy Issues A critical event in the development of claims-based measures of patient risk was the release, in 1986, of mortality rate information for some acute care Medicare hospitals by the Health Care Financing Agency (HCFA). The initial release was obligated by a freedom of information complaint and the rates were not adjusted for patient comorbidity or severity of illness. Among the high death rate outliers was a hospice, whose mission was to provide comfort to the dying. The HCFA released mortality rates for hospitals in 1987 that included some adjustments for patient severity. This release brought severe criticisms of the methods used by the agency to adjust for patient severity of illness in comparing hospital mortality rates, which were felt by many to imply quality. It also sparked an interest in exploring how risk-adjusted mortality rates using administrative data might actually serve as legitimate markers of quality. The HCFA continued to release risk-adjusted mortality rates for hospitals until 1993, when the inadequacies of its risk-adjustment methods had become so apparent that their publication was more misleading than helpful (Berwick & Wald, 1990). For those interested in using risk- adjusted mortality rates as markers of hospital quality, the challenge today remains little changed from the late 1980s. It centers on the development of a valid and easily 9 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. obtainable measure of patient risk at time of hospital admission. What has changed is that many more sources and types of data are being considered for use in risk-adjustment, and many more outcomes are being considered as potential markers of quality. The pressure to publish risk-adjusted hospital outcomes has increased over the last decade even as the inadequacies of claims-based measures of patient risk have come to light. Faced with spiraling medical costs, purchasers and business groups have demanded evidence of quality of care so that their purchasing decisions could be guided by cost-effectiveness considerations. Responding to demands from a number of interest groups, state governments have established health data organizations whose mandate is to collect financial and patient admissions data from all hospitals. Some states have also passed legislation requiring these organizations to release reports comparing hospital- specific quality of care for common surgical and medical procedures. Federal statutes and laws have also been passed that mandate the use of current administrative databases to investigate the quality of hospitals (Iezzoni, 1997b; Romano et al., 1995). The push for greater financial accountability in the medical care industry, as in other sectors of the economy, has brought about an interest in the quantitative assessment of outcomes, but acceptable indicators o f quality based on administrative data have yet to be developed. The HCFA no longer releases risk-adjusted mortality reports for hospitals because of concerns with their ability to adequately adjust for patient case-mix. In California, a similar experience has arisen, as state officials recently put on hold further releases of risk-adjusted outcomes reports for state hospitals using only administrative data. The concern is once again the adequacy of risk-adjustment, despite the fact that California R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. provides relatively high quality data on its hospital admissions and ample room for recording comorbid disease (Iezzoni, 1997b). However, many state-level health data organizations continue to release hospital report cards. States most successful in this effort have usually adopted and forced implementation of data systems for collecting clinical data in their hospitals. These systems, often proprietary, assess patient severity of illness at time of admission and generally document some chronic illnesses. Hospital, state, and federal registries have also been created for specific conditions (e.g., Minnesota and New York CABG registries) that include detailed clinical information on patients. These depositories of clinical information are considered the gold standard by which other methods should be judged. In most states, public assessments of quality rely solely on information found in the hospital abstract. Many states and federal entities have not sought the collection of more extensive clinical data on hospital admissions because of the considerable costs involved (Iezzoni, 1997b). Future quality assessment approaches will almost certainly involve increased efforts to collect inpatient clinical data as well as efforts to improve the risk-assessment methods employed with widely available administrative data. These efforts may involve the abstraction of more data to be included in the hospital abstract and/or the combination of administrative data from different sources. Any incremental improvement in our ability to assess patient physiological reserve at the time of hospital admission will yield benefits not only in terms of improved quality assessment, but also in the areas of provider reimbursement strategies, health care effectiveness research, and risk-pooling and insurance applications. 11 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Theoretical Perspectives on Patient Comorbidity Several taxonomies have been suggested for organizing patient attributes that contribute to increased risk for adverse events during hospitalization, and the schema provided by Iezzoni (1997a) is a widely cited one. These characteristics include: • Acute Clinical stability • Severity of principal diagnosis • Extent and severity of comorbidities • Principal diagnosis • Age and sex • Physical functional status • Psychological, cognitive, and psychosocial functioning • Cultural, ethnic, and socioeconomic attributes and behaviors • Health status and quality of life • Patient attitudes and preferences for outcomes The first three factors relate directly to the patient’s physiological state at the time of admission, given a specific diagnosis. The study of the relationship between severity of illness or clinical stability and adverse outcomes of care is a well-established branch of medical research and various systems and methods of assessing severity of condition have been devised. Most original research in this area has relied on information from the clinical record. In empirical studies using administrative data, assessments of severity, however imperfect, are derived from ICD-9-CM or DRG codes used to document the primary condition. The effect of comorbid illness on patient outcomes has received somewhat less attention, though the severity of the primary diagnosis is sometimes considered a function of coexistent disease. Comorbid illness may be acute or chronic. Both chronic and acute 12 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. conditions may present increased risk to patients undergoing medical treatment, though acute conditions are probably more often recorded and taken into consideration when examining treatment options. Acute conditions can sometimes result from the medical care provided and are then considered complications, but chronic conditions are rarely confused with complications. Prognostic comorbidity refers to comorbid conditions that increase the risk for patient adverse events resulting from treatment. Prognostically important comorbidities have been identified for a number of medical and surgical procedures and these are usually the focus of risk-adjustment methodologies concerning comorbid illness. However, because the relationship between comorbid disease and outcomes has not been explored for all possible combinations of primary and secondary diagnoses, there must certainly exist prognostically important conditions that have yet to be identified. Conceptual models have also been developed concerning the severity of comorbid disease. In one influential application (Gonnella, Hombrook & Louis, 1984), distinct points in the progression of disease, reflected in risk of death or residual impairment, have been identified and assigned severity levels. In this conceptualization, stages of disease are defined “primarily in terms of biologic complications” (p. 639) and the progression of disease through stages is described in terms of increasing involvement of organs and systems, risk of complications and deteriorating prognoses. Another method of assessing severity o f comorbidities is to link the stage of disease to the therapeutic approach being used in treatment. This approach has been used to assign comorbid severity rankings to patients based on the therapeutic class of medication they have been 13 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. prescribed (Von Korff, Wagner & Saunders, 1992). Other approaches have also been employed. An important aspect of comorbid disease is the overall impact that all comorbidities have on a patient’s condition. This corresponds to the expression “burden of illness” that relates to a patient’s physiological reserve. The methodology used by many researchers implies that this is an additive function of the presence and/or severity of the comorbid conditions present, no matter their relative severity. A simple count of the number of comorbid illnesses (or affected body systems) expresses this perspective. The theoretical basis behind more sophisticated attempts to assess physiologic reserve is that the greater the number of body systems involved or affected by disease, the more susceptible the patient will be to adverse outcomes for treatment (Gonnella et al., 1984; Kaplan & Feinstein, 1974; Von Korff et al., 1992). For example, patients with multiple prognostic comorbidities are generally poor candidates for open-heart surgery, since they may not have the physiological reserve to survive the procedure. In medical research, theory designed to explain comorbid disease and its consequences sits comfortably next to purely empirical attempts to discover relationships between comorbid conditions and utilization patterns, response to therapy, or any number of clinical endpoints. A tradition of clinical research has established links between specific comorbidities, severity of conditions, overall physiologic reserve, and important outcomes for a number of treatments and conditions, yet there remains great potential for advances in the modeling of comorbid illness for predicting outcomes of hospital care. 14 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Methodological Issues in Risk Adjustment Judging the adequacy of patient risk measures in predicting hospital outcomes usually takes place in the context of multivariate statistical modeling. Variation in patient outcomes may be thought of as originating from patient level differences, hospital differences, and a random component (Hofer & Haywood, 1996; Smith, 1994). Methods exist to assess the separate contribution that measures of comorbid illness make in explaining outcomes. When differences in hospital quality are large and risk assessment is nearly perfect, then the random component will be relatively small and it should be possible to assess quality differences (in terms of risk-adjusted outcome rates) with considerable success. However, when differences in rates of quality proxies are negligible (e.g., minimal variation in mortality rates across hospitals) or when a poor risk assessment strategy is used, it will be very difficult to find statistically significant differences in quality. In the first instance, very small differences in quality may largely be a function of natural random variation and no matter how accurate the predictive model being used, assessment in terms of risk-adjusted outcomes (or indeed, unadjusted outcomes) will not be meaningful. In the second case, a risk-adjustment strategy that does a poor job of accounting for patient case-mix will leave much variance in outcomes unexplained, and this will give the appearance of greater quality differences than actually exist since actual differences in case-mix will appear as differences in outcome rates. One study suggests that most of the unexplained variation in inter-hospital mortality rates comes from unmeasured patient severity of illness (Silber, Rosenbaum, & Ross, 1995). 15 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The prevalence of a given outcome will also influence our ability to build accurate and useful models. For example, when few deaths exist at a hospital for a given procedure, the power to explain these is diminished. As the frequency of events drop below 5%, much larger samples are required to reliably determine the significance and level o f associations in models. Mortality rates will fall well below that for most conditions at acute care hospitals. These factors may all come into play when attempting to rank hospitals based on their risk-adjusted outcomes. A standard method to achieve these rankings is to use the empirically derived weights from a multivariate equation to assign a predicted rate of death, complications, or unplanned readmission to a particular hospital and compare that with their actual rate. Difference scores or ratios (e.g., observed/expected) are then used to form the rankings and methods for identifying quality outliers are applied. While any o f the above problems might make such a ranking problematic, a poor risk-adjustment method will generally assign hospitals with the sickest patients the lowest quality rankings. In instances where there is sufficient variation in quality between hospitals and sufficient prevalence of an outcome, improvement in risk assessment methods should theoretically lead to improved identification of performance outliers. Information on Comorbid Illness Sources of information on patient comorbidity may be automated or exist only on paper. The medical record of hospitalized patients is usually not computerized so investigation of a patient’s hospitalization requires the abstraction of data from the 16 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. inpatient medical chart. Hospital studies that rely on automated data to determine patient illnesses usually obtain that information from the hospital discharge abstract. Proprietary severity of illness information systems may also provide information concerning comorbidities. Automated outpatient pharmacy information is a potential source of information for assessing patient health, but it has been used infrequently for that purpose and linkages between it and patient-level hospitalizations have rarely been reported. The often-cited advantages of claims-based data are their low cost, ready availability, ease of use, and their suitability for use in population-based or large sample research. Hospital abstract diagnostic information in the United States is uniformly coded using the ICD-9-CM disease classification system and drugs are identified through a unique classifier, the NDC (National Drug Code) number. The advantages of data derived from the medical record are completeness of information, better accuracy and greater clinical detail. The medical record serves as the gold standard against which automated information sources are judged. Assessing Comorbid Illness from the Hospital Abstract Disease states and complications of care recorded for a given patient during a hospital stay are usually abstracted from the medical record by trained coders using the ICD-9-CM methodology. Codes are listed in the primary and secondary diagnosis fields of the patient’s discharge abstract. The primary reason for admission is listed in the principal diagnosis field and comorbid illnesses and complications of care are listed in the secondary diagnosis fields, usually in order of importance. The amount of 17 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. information abstracted to the discharge record may differ greatly from one hospital to the next, depending on the ultimate purpose of the data and state, local, and federal statutes that affect its collection. A critical manner in which automated abstracts differ is the number of fields allowed for the coding of secondary diagnoses. Currently these range from 9 spaces (Medicare’s MEDPAR dataset and many others) to 24 spaces (California’s Office of Statewide Health Planning and Development’s (OSHPD) hospital discharge abstracts). The number of fields provided has gradually increased over the last 2 decades. For example, the MEDPAR dataset at one time included only three fields. Very few discharge systems currently provide information that allows one to distinguish between comorbid conditions and complications of care. Hospital abstracts also differ in terms of the information they provide on patient sociodemographic characteristics, admission status, discharge destination, discharge disposition, and other characteristics. The reliability of discharge abstracts compared to the medical record has been the subject of considerable investigation in the health services literature. Several reabstraction studies have been published in which the prevalence of comorbid disease reported in both sources has been compared. Nearly all of these studies have noted a lower prevalence of comorbid disease reported in the hospital abstract (especially chronic diseases), though studies that have examined the incidence of disease reported over time have noted significant improvements in the completeness of diagnostic coding at the state level ( Romano, Roos, Lufit, Jollis & Doliszny, 1994; Schwartz, Iezzoni & Moskowitz, 1996), at individual medical centers (Jollis et al., 1993) and within the Medicare claims system (Fisher et al., 1992). Because the time periods considered in these studies and the 18 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. sources of data (including the number of secondary fields in the abstract) have varied widely, it is difficult to derive an overall estimate of agreement between the two data sources, though study-specific estimates have been provided. Jollis and coinvestigators, using Duke University patient data from 1985 to 1990, found that claims data failed to identify more than one half of the patients with prognostically important conditions. Several additional studies have provided similar results. However, in a review of reabstracted data used in OSHPD’s quality assurance program, Romano et al. (1994) found that the sensitivity of coding for seven of eight comorbid conditions (used by HCFA in their risk-adjustment) exceeded 85%, suggesting much better agreement. Comorbidity Measures Adapted From Clinically-Based Research Some of the current claims-based indices of comorbidity were originally created with and validated using clinical data. The major outcome of concern in these studies was short-term mortality, following the HCFA reports, but recently more interest has been directed towards outcomes that are reported with greater frequency. The Charlson Comorbidity Index (Charlson, Pompei & MacKenzie, 1987) is perhaps the most widely reported comorbidity index in the health services literature today and is a good example of how such measures were created. The original index was created by abstracting all medical chart information on comorbid illness from patients admitted to New York Hospital during a one-month period in 1984. Charlson and colleagues then estimated the association between these comorbidities and one-year survival using Cox proportional hazards analysis. The resulting weights associated with each comorbidity were then 19 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. summed to create individual comorbidity scores. These scores were then validated with a sample of breast cancer patients. The weighted score, as well as a simple count of the 19 comorbid conditions considered, were predictive of one-year mortality. The Charlson disease identification criteria have been translated into ICD-9-CM codes since that time and two somewhat different versions have dominated the literature. Deyo, Cherkin, and Ciol (1992) adopted the Charlson Cl for use with claims data in a sample of 27,111 Medicare beneficiaries who underwent lumbar spinal surgery in 1985. In their version, 17 comorbid conditions were identified using combinations of specific ICD-9-CM codes and an index of comorbid illness was created using the original severity-scoring algorithm. Increasing Charlson Cl scores on an ordinal scale (0, 1, 2, 3+) were found to be significantly associated with in-hospital complications, blood transfusion, discharge to a nursing home, 6-week postoperative mortality, mean LOS and total hospital charges, after controlling for patient age. The other popular adaptation of the Charlson Cl is the Dartmouth-Manitoba (D- M) version, derived independently by Romano, Roos and Jollis (1993a, 1993b). They used somewhat different ICD-9-CM codes to identify 15 specific comorbidities and found that their measure provided considerable power in explaining in-hospital complications in both lumbar discectomy patients in California and 1-year mortality in Coronary Artery Bypass Graph (CABG) patients in Manitoba, Canada. In adapting the Charlson Cl to administrative data, both teams of researchers devised their own method, based on clinical judgment, of separating comorbidities existing prior to hospitalization from complications of care. 20 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Subsequent studies have validated these adaptations of the Charlson Cl with different populations and different medical conditions and surgical procedures. Most researchers have opted for the Deyo method of disease identification in their risk- adjustment strategies, because it is considered less apt to misclassify complications as comorbidities. In an investigation of the factors that determine unanticipated hospital readmission for Medicare patients with congestive heart failure (CHF) patients, Krumholz et al. (1997) found the Deyo comorbidity measure to be a significant predictor of that outcome. They used the original scoring of the index but included it as a dichotomous measure (scores > 1) in multivariate models. Ghali et al. (1996) employed the Deyo Cl in exploring the outcomes of CABG, developing study-specific comorbidity weights for each of the conditions. In multivariate models, several conditions were significantly related to the occurrence of postoperative complications and in-hospital death. In a recent study, the Deyo and D-M adaptations were combined so that a broader range of conditions would be included (Librero, Peiro, & Ordinana, 1999). In a population of hospital admissions in Valencia, Spain, the researchers found high scores on their Charlson index to be significantly associated with longer LOS, in-hospital mortality and increased probability of readmission at both 30 and 365 days. Other adaptations of the original Charlson Cl have also been reported. D’ Hoore and colleagues (D’Hoore, Sicotte, & Tilquin 1993; D’Hoore, Brouckaert, & Tilquin 1996) found their simplified version, employed as an index using the original Charlson scoring, to be a 21 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. statistically significant predictor of in-patient mortality in five common hospital admission diagnoses. Other measures that focus on the assessment of comorbid illness have also been translated for use with automated discharge abstract data. One of the more influential of these has been Staging of Disease, a case-mix measure developed by Gonnella, Hombrook & Louis (1984) that reflects “severity in terms of risk of death or residual impairment” by assigning different scores to diseases based on the stage of their progression. Adaptations and extensions of the original measure has been used with considerable success in risk-adjustment (Greenfield, Apolone, McNeil, & Cleary, 1993; Naessens, Leibson, Krishan, & Ballard, 1992; Thomas & Holloway, 1991), but the proprietary nature of the scoring algorithms has made it less accessible than the Charlson C l. Comorbidity Measures Originally Developedfor Use With Automated Claims Data Many current approaches to assessing comorbid illness in hospital patients have been developed specifically for use with automated hospital data. That is, identification of comorbid disease relies on the ICD-9-CM classification scheme used in hospital discharge abstracts, and not evidence from the medical chart. Excluded from consideration are systems whose primary purpose is predicting resource utilization. Such resource-based measures are not usually concerned with the timing of hospital events and so have limited utility in assessments of provider quality. 22 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Selection of the medical conditions that comprise claims-based measures may be based on clinical judgement and/or informed by relevant clinical literature, though purely empirical methods for identifying prognostic comorbidities have gained many adherents in recent years. In practice, the creation of new measures usually results from a mix of clinical judgement and empirical modeling. The often-cited 1988 study by Jencks, William and Kay is representative of an empirically driven research methodology that remains popular today. They investigated the contribution of comorbidities to 30-day mortality in Medicare patients diagnosed with pneumonia, myocardial infarction (MI), stroke and CHF. To determine which comorbidities might be important determinants o f mortality they selected 16 chronic conditions that occur in at least 1% of all Medicare admissions and calculated their odds ratios with two mortality outcomes using logistic regression, adjusted for age, sex and other chronic diseases. They found that only a few chronic conditions, such as cancer and renal failure, were consistently associated with increased mortality while many of the diseases had suspiciously low odds ratios or counterintuitive effects, a finding that raised many concerns about the use of hospital abstract data. In a series of frequently cited articles on risk-adjusted outcomes, Deshamais and colleagues (DesHamais, Chesney, Wroblewski, Fleming, & McMahon, 1988; DesHamais, McMahon, Wroblewski, & Hogan, 1990; DesHamais, McMahon, & Wroblewski, 1991) explained the derivation of, and explored the relationship between three different risk-adjusted indices: Risk-adjusted Mortality Index (RAMI), Risk- adjusted Readmission Index (RARI), and Risk-adjusted Complications Index (RACI). 23 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The authors used the 1983 National Commission on Professional and Hospital Activities’ database to first aggregate DRGs into clusters with similar conditions. Expert medical opinion was then used to exclude from consideration 70 comorbidities that might represent complications of care. Using contingency table analysis and logistic regression, comorbidities that were associated with the risk of each outcome for 64 DRG clusters were then identified. Their empirically estimated risk indexes were effective in predicting mortality, unanticipated readmissions and complications. Goodness of fit measures indicated better predictive power for RAMI and similar predictive ability for RACI and RARJ. Correlational analyses suggested that each measure provided a separate, unique assessment of hospital performance. In a study to investigate the effects of preexisting conditions on in-hospital death in adult trauma patients, Morris, MacKenzie, and Edelstein (1990) used California state data to match all trauma deaths occurring in 1983 with a group of trauma survivors in a case-control design. The relative odds of dying for patients with 1 or more of 11 pre existing conditions were then estimated. Five of the conditions significantly increased the relative odds of in-hospital death for patients with the disease, after making adjustments for age. In two articles appearing in 1994, the relationship between 13 chronic conditions and 2 potential measures of hospital quality were explored by Iezzoni and colleagues. In the first article, the researchers (Iezzoni et al., 1994c) selected 13 chronic conditions “that were either common or clinically judged to be important sources of risk for short-term mortality” (p. 440) in a study using California hospital discharge data. They excluded 24 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. some diagnoses such as hypertension and prior myocardial infarction because of suspected undercoding and prior counterintuitive findings by researchers. They selected all general and medical adult admissions in California in 1988 and estimated the impact of the comorbidities on in-hospital death in the population. They found that across all case types (groupings based on mortality experience), 8 of the 13 comorbid conditions increased the odds ratio for in-hospital death after adjusting for age and other chronic conditions. In the second article using 1988 California discharge abstract data, Iezzoni et al. (1994a) sought to identify potentially avoidable complications of care and develop a risk- adjustment strategy for ranking hospitals on this outcome measure. They defined 27 complications that could indicate poor quality across six risk pools, defined by DRG and ICD-9 codes and thirteen chronic conditions were included to control for patient risk due to a priori health, along with other patient characteristics. Some conditions were highly predictive o f complications across five of six risk pools, while other appeared significant in only one or two risk pools. Hospitals were ranked by their observed-to-expected rates and these ranking generally correlated across risk pools. Analyses also indicated that hospital performance assessed by complication rates was generally not correlated with the HCFA risk-adjusted mortality rates. Other analyses indicated that “heavy coding” at some hospitals was partially responsible for higher complication rates at some hospitals. More recently, Elixhauser and colleagues (Elixhauser, Steiner, Harris, & Coffey, 1998) sought to extend the range of diagnoses considered in claims-based comorbidity measures beyond those commonly used. They defined their list using a set of a priori 25 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. criteria and refined it through a series of univariate and multivariate evaluations, finally settling with 30 disease states. To avoid confusing comorbidities with complications of care, they excluded all secondary diagnoses that could be directly related to the principal diagnosis. Using California hospital data for all adult hospital stays in 1992, they found that 10 comorbidities had substantial effects on LOS, hospital charges and in-hospital death. These effects persisted in analyses across six disease subgroupings. Some of the best individual predictors were conditions that had not been examined by previous researchers. Typical of recent research, the authors decided against combining the individual disease markers into an index. They also found unexpected inverse relationships between several comorbid conditions and the probability of dying in the hospital, suggesting that the problem was a result of undercoding of chronic disease. Empirical Explorations O f Claims-Based Measures O f Comorbid Illness The use of a validated risk-adjustment methodology is a necessary element in all hospital profiling efforts using administrative data. However, it is not well understood how the choice o f a specific measure will affect model performance. Some researchers have compared the performance of multiple claims-based measures in order to better understand critical differences between measures. Their results suggest that differences in predictive ability can often be attributed to the different methods used in deriving scoring weights and the degree to which complications contaminate comorbidity estimates. 26 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. In two separate studies that compared the predictive ability of the Deyo and D-M versions of the Charlson (Cleves, Sanchez, & Draheim, 1997; Ghali et al., 1996), both found that indices derived using study specific weights performed superior to the original Charlson index in explaining patient mortality. Researchers have urged their colleagues to derive their own empirical weights when adopting the Charlson comorbidity index (Cleves et al., 1997; Ghali et al., 1996; Romano et al., 1993b). The Ghali et al. study and another (Romano et al., 1993a) also found greater prevalence for several diseases (and higher average risk scores) when using the D-M version compared to the Deyo version. This greater frequency may have contributed to the improved explanatory ability of the D-M version in one comparison (Cleves et al., 1997). The creators of the D-M version of the Charlson have acknowledged that their index may include some comorbid factors that are actually complications of care (Romano et al., 1993b), and other researchers (Ghali et al, 1996) have also suggested that the Deyo Cl provides a more conservative assessment of comorbidity. In an exploration of the extent to which the inclusion of complications affects the explanatory ability o f Charlson measures, Roos and colleagues (Roos, Stranc, James, & Li, 1997) compared the performance of slightly different versions of the original D-M Cl in predicting one-year mortality for three surgical procedures. The Manitoba data source used in the study distinguishes complications of care from comorbid conditions, allowing for an accurate assessment of comorbidity at time of admission. They found that in CABG patients, the D-M version misclassified the complications of CHF and Cerebrovascular Disease as comorbidities with some regularity. However, they found 27 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. that the exclusion of complications from the original D-M Cl had a minimal effect on the size of the C-statistic attained in multivariate models. However, they found that the addition of three prognostically important acute conditions (AMI, CHF and cerebrovascular disease) to the D-M, originally excluded because they may represent complications, had a large positive effect on variation explained. Using the Manitoba database, such increases in explained variance were obtained without loss of clinical validity (since complications are clearly indicated), however inclusion of the three acute conditions without regard for the time of their occurrence (which is the typical situation with hospital discharge abstracts) led to ‘spuriously high discrimination’ in models for at least one surgical procedure. Comparisons between Charlson measures and other claims-based measures along several dimensions have also been performed. Melfi and colleagues (Melfi, Holleman, Arthur, & Katz, 1995) compared the Deyo Cl with two measures based on Patient Management Categories (PMCs) (Young, Kohler, & Kowalski., 1994) and a simple count of comorbidities in predicting LOS and 30-day mortality for patients undergoing total knee replacement surgery. They found that the highest R-square values and C-statistics were obtained using PMC-based comorbidity measures, with the Charlson measure performing even worse than a simple comorbidity count. However, unexpected signs for the PMC indices for the lowest-risk patient groups caused the authors to question their clinical validity. The authors applauded the virtues of the simple count of comorbidities, yet it suffers the same deficiency as the PMC, and the only measure that attempts to avoid including complications is the Deyo CL 28 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Polanczyk and colleagues (Polanczyk, Rohde, Philbin, & DiSalvo, 1998) compared the performance of the Deyo Cl with their own study-specific measure in predicting in-hospital mortality for CHF patients. Their measure, derived empirically in a sample o f4608 Massachusetts General Hospital admissions from 1990-1996, contained patient status and chronic disease information, but also contained acute conditions that are common complications of CHF. In their internal validation sample, logistic regression models yielded C-statistics of .78 for the CHF-specific index and .66 for the Deyo Cl. Since the measure was intended to adjust for case-mix and not for risk stratification of patients at the time of admission, they were relatively unconcerned with the probable inclusion of complications in the measure. In a study that utilized 14 different illness severity measures, Iezzoni et al. (1996) assessed the relative ability of 10 claims-based measures to predict in-hospital death in 12,016 adults hospitalized in 1991 for pneumonia. In their study, clinically based measures generally achieved higher R-square and C-statistics, though some measures based on administrative data performed nearly as well. A simple count of body systems affected by illness did not provide better discrimination over a baseline model (C = .71) while the Deyo et al. index registered a C-statistic of .74. Addition of the Deyo measure increased the R-square from .04 to .06. Resource-based measures of severity and clinically based measures generally performed better, as did the Patient Management Categories (PMCs) severity score, which is similar to Deyo Cl in its intended purposes. 2 9 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Understanding the Limitations of Administrative-data Based Measures Inconsistencies in results and counterintuitive findings in studies that relied solely on administrative data to risk-adjust for hospital outcomes have concerned researchers from the very start. During the 1980s, a number of articles appeared in which the prevalence of disease in claims and clinical-based data systems was compared. Most compared the incidence of comorbid conditions, by data source, for the same individuals. Many of these studies were made possible by reabstractions of state and federal-level administrative data that were intended to check the quality o f such data. Other studies have compared the data collected by special clinical registries at hospitals with the data from discharge abstracts produced by these hospitals. Underreporting o f Chronic Illness Most early studies indicated that comorbid disease was underreported in the administrative record when compared to the clinical record, but the degree of undercoding and its effects were topics of disagreement. Data used in these studies was often quite old, often from the 1970s, before the advent of DRG reimbursement and accompanying incentives (including legislation) to provide more accurate coding of disease. Also, the discharge abstracts used commonly featured five or fewer fields for the coding of secondary diagnoses. Several studies have appeared since the late 1980s that shed more light on the limitations of claims databases. Some studies have been concerned not only with the extent of miscoding or undercoding of comorbid disease in claims files, but the effect this has on risk-adjusted outcomes. Researchers have focused 30 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. on the association between specific comorbid conditions and outcomes when individual comorbidities are included as covariates in multivariate models, as well as the overall variance explained or discrimination achieved by comorbidity indexes. Almost without exception, researchers in the 1990s have found that the prevalence of all diseases in administrative databases is less than that found in the medical record. This finding holds across state and federal record sources in the United States and even provincial and federal systems in Canada. In several studies, these rates were found to be quite low. Kieszak and associates (Kieszak, Flanders, Kosinski, Shipp, & Karp, 1999) found the mean number of secondary diagnoses to be .55 for the claims source compared to 1.31 for the medical chart in a population of Medicare beneficiaries hospitalized for carotid endarterectomy. Schwartz et al. (1996) found that the average record-based score on a measure of comorbid illness was 50% greater than the average claims-based score for patients hospitalized with eight common conditions in 1985. Hawker and colleagues(Hawker, Coyte, Wright, Paul, & Bombardier, 1997) found a mean false negative rate of 63% for all comorbid conditions considered, when comparing claims records against the clinical gold standard. However, Romano & Mark (1994) found only small differences in the total number of listed diagnoses between abstracts (4.0) and the medical record (4.3) in a reabstraction study using 1988 California hospital discharge data. Kappa values, used to express agreement while controlling for random error, have often been used to describe the degree of agreement between clinical and claims records. Levels of agreement have ranged from low to moderate for most comorbidities in recent 31 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. studies (Hannan, Racz, Jollis, & Peterson, 1997; Jollis et al., 1993; Malenka, McLerran, Roos, Fisher, & Wennberg, 1994; Weintraub et al., 1999). Low kappa values, indicating poor agreement between the clinical and claims data source, have also been noted when comparing comorbidity index scores (NewschafFer, Bush, & Penberthy, 1997). Sensitivity and specificity, widely used measures in epidemiological research to explain the performance of a test for the diagnosis of disease (when a gold standard is available), have also been reported in clinical vs. claims comparisons. Sensitivity in this context is the percentage of persons with the disease of interest (according to the clinical record) that are correctly identified by claims data. Specificity is the percentage of persons without the disease who have no indication for that disease in the claims file. High sensitivity scores indicate a low proportion of false negatives and high specificity scores indicate a low proportion of false positives. Nearly all clinical vs. claims comparisons have reported low to moderate sensitivities for most conditions and high specificity for all conditions being studied (Fisher et al., 1992; Jollis et al., 1993; Roos, Sharp, & Cohen,, 1991; Weintraub et al., 1999). In one exception, Romano & Mark (1994) found relatively high sensitivity for eight comorbid conditions, with the exception of hypertension, while specificity was quite high for all conditions. These studies together suggest that considerable underreporting of comorbidity in claims data exists, yet very few conditions are reported only in claims data that are not found in the clinical record. Several articles that compare the predictive ability of clinical and claims-based measures of comorbidity were published in the 1990s, most of which used logistic and 32 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. multiple regression to explain variation in hospital outcomes while controlling for common patient characteristics such as age and sex. Most published research has found a significant association between claims-based measures of comorbid illness and outcomes of care, after adjusting for confounding variables, though these were usually weaker than associations demonstrated using clinical data. However, in a recent study no significant association was found between comorbid illness indexes based on administrative data and adverse hospital outcomes (Kieszak et al., 1999). In another study comparing claims and clinical versions o f three different comorbidity indices (Newschaffer et al., 1997), the addition of the claims-based D-M Cl to a baseline model added significant improvement to a proportional hazards model predicting mortality, while the Satariano index (Satariano & Ragland, 1994) added significant explanatory power only when medical records were used and the Kaplan index (Kaplan & Feinstein, 1974) failed to improve the fit of the baseline model in all cases. A surprising result in several recent studies is that claims-based measures performed as well, or even better in some cases, than clinical measures. For example, two studies (Iezzoni, Ash, Shwartz, & Mackieman, 1997c; Pine, Norusis, Jones, & Rosenthal, 1997) found that administrative data performed slightly better in predicting in- hospital mortality. Newschaffer et al. (1997) found that use of the claims-based version of the D-M Cl brought a slightly higher Chi-square value than use of the clinically based version of the same index in a proportional hazards model to predict mortality. Another two studies found general parity in the predictive ability of the two types of measures (Iezzoni et al., 1996; Weintraub et al., 1999). 33 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Several authors have attributed this enhanced explanatory power to the claims measures’ inappropriate inclusion of conditions that represent outcomes of care. To test this argument, Pine et al. (1997) devised a “restricted” administrative model, wherein acute conditions that might represent complications were removed, and found that this model provided “strikingly worse” mortality predictions than the “unrestricted” administrative model, which included all conditions listed as secondary diagnoses. They also demonstrated that in patients whose risk of dying as calculated by the unrestricted model was much greater than that using clinical data, there were a large number of shock and hypotension diagnoses without clinical evidence of these conditions at hospital admission. Hannan et al. (1997) also demonstrated a substantial drop in the predictive ability of a claims-based measure to predict in-hospital mortality in CABG patients after using corroborating clinical information to exclude complications. Iezzoni and colleagues (1995) examined whether abstract-based measures relied heavily on complications in rating the severity of patients. As part of a larger study, they compared patients with higher severity ratings as judged by Disease Staging (claims- based) than on MedisGroups (clinical-based) and found that the frequency of cardiac arrest codes in these patients was 16.2%, but when the comparison was reversed only .4% of the patients had such codes. Weintraub and colleagues (1999) also argued that much of the explanatory power found in their claims-based measures was due to the inappropriate inclusion of complications. In a comparison of claims and clinical data in forecasting mortality for CABG and PTCA patients, they found higher sensitivity of the claims measures in detecting heart failure in the patients who died than in those who 34 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. survived. They concluded that this is because heart failure is a common complication in patients who die, so most of those diagnoses represent complications. In survivors, however, a larger percentage of heart failure diagnoses would represent a comorbid condition. Coding Bias The contention that systematic bias in the reporting of certain types of comorbidities in claims databases exists has drawn a great deal of attention from health services researchers in the last decade. Interest in this topic was originally sparked when a few studies in the late 1980s using claims data found protective effects against the risk of death for several conditions in hospitalized patients. These counterintuitive results were puzzling since clinically based research generally indicated that such diseases were risk factors for adverse outcomes. The clinical validity of claims-based measures of comorbidity has suffered because of these frequently reported findings and researchers have tried to understand how coding practices might explain such results. The study by Jencks and colleagues (1988) was the first to notice and attempt to explain some of the paradoxical results that can be obtained when relying on claims data for risk adjustment. They investigated the contribution of comorbidities to 30-day mortality for Medicare patients diagnosed with pneumonia, Myocardial infarction (MI), stroke and CHF. In a claims-based study that explored the effect of specific comorbidities on hospital outcomes in Medicare patients with pneumonia, they found that some chronic conditions had odds ratios substantially below 1, indicating decreased risk 35 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. for 30-day and long-term mortality. The significant, counterintuitive relationships were especially consistent for adult-onset diabetes, essential hypertension, obesity and osteoarthritis. Other conditions such as cancer and chronic renal failure demonstrated clinically reasonable relationships. The authors presented several lines of evidence to support their contention that “chronic conditions are substantially underreported in patients who die or who are most likely to die after discharge” (p. 2246). That is, a coding bias occurs when fewer chronic conditions are recorded for patients who die because acute, life-threatening diagnoses take precedence. This bias was exacerbated by the five-field limit for recording secondary diagnoses in Medicare discharge abstracts. Subsequently, several studies have appeared that bring empirical evidence to that argument. Iezzoni et al (1992) sought to understand how expanding the number of available spaces for coding secondary diagnoses might affect the results obtained by Jencks et al., using data from California, where 24 fields are available. They also found protective effects for chronic conditions that should have increased the mortality risk and these effects persisted even in the hospitals where the average number of diagnoses coded was the highest. They conclude that the bias noted by Jencks et al. persists despite more than an adequate number of fields for the recording of diagnoses. In a study providing contrary evidence, Romano & Mark (1994) also tested the hypothesis that chronic conditions are selectively underreported when patients die, with somewhat advantaged data. Using data from a California reabstraction study, they estimated the sensitivity of coding for eight chronic conditions using claims and analyzed 36 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. whether this sensitivity differed by patient death status. After truncating the number of secondary diagnosis fields to five to replicate the Medicare database, they did find underreporting of chronic conditions for nonsurvivors, but at 9 or 25 fields no such evidence was found. They concluded that there was limited support for the coding bias theory and that it was an artifact of truncation of diagnosis fields. Additional evidence in support of these opposing arguments has not been offered. Yet numerous studies since the mid 1990s have found results consistent with the original bias argument and most researchers have concluded that the peculiar finding that some chronic diseases, like hypertension, are predictive of good hospital outcomes is because they coded most often in healthier patients. Using Automated Pharmacy Records to Assess Chronic Disease Automated pharmacy records are an important source of information for epidemiological, postmarketing surveillance, and drug effectiveness research, but interest in using them to provide patient health status information is a recent phenomenon and relatively few studies have explored their potential in risk adjustment for adverse outcomes. Perhaps the first use of pharmacy data to assess health status in patients was the creation of the Illness Scale (Mossey & Roos, 1987). Using data from the Manitoba health insurance program, the authors incorporated drug-utilization data into a multifaceted claims-based health status measure that was strongly predictive of death and hospitalization over time. 3 7 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Beginning in the early 1990s, a series of articles appeared that incorporated pharmacy data in a measure of chronic disease burden. In 1992, Von Korf, Wagner, and Saunders (1992) developed the chronic disease score (CDS) using automated data with a managed care outpatient population. Using the American Hospital Formulaiy System (AHFS) therapeutic classification system, medications associated with the treatment of 17 different chronic conditions were identified and a weighting system for assigning severity based on clinical judgment was devised. The scoring methodology followed the general rule that individual scores should reflect the nature (life threatening, stable, benign) of the diseases for which they are indicated and should increase as the complexity o f the therapeutic regimen increased. The index score was significantly associated with physician-rated disease severity, self-reported health status, hospitalization and mortality, after controlling for patient age, gender, and some prior utilization. Johnson, Hornbook, & Nichols (1994) also validated the measure with future hospitalization and total number of office visits. They found evidence of reliability over time and support for the construct validity of the CDS in associations with other health status indicators. An expanded and revised CDS with empirically derived weights has also been reported (Clark, Von Korff, Saunders, Baluch, & Simon, 1995). In this study, the authors extended the number of chronic conditions considered to 28 and the predictive ability of the original CDS index and another popular outpatient case-mix measure (ADGs) were compared with their expanded measure. In a prospective analysis of total patient costs, outpatient costs and the total number of outpatient visits, the revised CDS demonstrated 38 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. improved predictive ability relative to the original. The Revised CDS also performed slightly better than ADGs in this respect. The CDS with empirically derived total cost weights also provided moderate predictive ability with regards to hospitalization and mortality. Subsequent studies using pharmacy data as a risk adjuster have generally taken place in the managed care outpatient setting and focused on estimation of costs and patterns of utilization within organizations. Roblin (1996) demonstrated the usefulness of pharmacy data as a case-mix method for projecting global budgets in a managed care organization, using an empirical approach to risk assessment unrelated to the CDS. Lamers (1999) adopted the revised CDS for use with a Dutch sickness fund population and found that it substantially improved predictive accuracy regarding charges compared to the use of a demographic model. Two researchers have also created a pediatric version of the index (Fishman & Shay, 1999). They found that the Pediatric CDS performed far superior to a demographic model in forecasting health service utilization and costs. It also performed only slightly worse than ACGs (based on ICD-9-CM diagnostic information) in various tests of forecasting ability. Automated pharmacy data is one of the most reliable and consistent sources of patient-based data available in the health care market. Its content is highly standardized, its reliability is rarely questioned, it is well understood, and its collection for use in health services research is achieved at minimal cost. However, adapting this information for use in assessing chronic disease severity or complexity in patients presents several challenges. 39 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. A major problem the researcher confronts is that identification of disease states using only therapeutic drug class information is highly problematic. Some medications are used to treat a variety of conditions (e.g., inhaled beta-agonists for COPD, asthma, or severe allergies) while others can be reliably linked to a single condition (e.g., insulin for diabetics and antiparkinsonian agents for Parkinson’s disease). A variety of drug classification systems exist, and some may provide a better classification scheme for linking drug groupings to diagnoses than the AHFS, but the essential problem will remain. Patterns of medication usage across and within therapeutic classes may also be better markers for certain medical conditions than a single prescription and have greater face validity. The clinical epidemiology literature provides relatively few examples of how patterns of medication usage relate to the presence of chronic disease. However, in one recent investigation (Malone et al., 1999), researchers were able to identify 6 common chronic diseases with a specificity of > 60% (a specificity of > 75% for 3 conditions) using criteria very similar to that of the CDS. Another substantial problem is that not all diseases are treated medically. For example, diabetes and hypertension may be controlled with dietary restrictions and lifestyle changes, though this generally happens with milder cases of these diseases. Other problems may also skew estimates of comorbid illness using pharmacy data, such as patient nonadherence to regimens, pharmacy utilization outside of the health system being analyzed, and differences in physician prescribing practices. 4 0 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Outcome Measures Risk-adjusted mortality, unplanned hospital readmission and complication rates have all been explored as potential markers of hospital quality of care. Hospital length of stay has generally been considered an indicator of resource consumption and unrelated to quality of care. Numerous studies over the last 2 decades have also shown a relationship between the presence of comorbid illness and the occurrence of death, readmissions, complications and extended length of stay in hospitalized patients, though these relationships have not been consistent across investigations. Such empirical studies have employed both clinical and claims-based data sources, a number of different comorbidity indices or comorbid disease indicators, and a range of statistical methods to investigate the link between comorbid disease and outcomes, and they provide a useful background against which newer results might be judged. Mortality Most quality-related outcomes research, especially that appearing in academic journals in the 1980s, used risk-adjusted mortality as the primary outcome of interest. This was, and continues to be the most publicized measure of putative quality and is commonly employed when examining acute medical and high-risk surgical procedures. Mortality during or shortly after hospitalization has also been the principal validation criterion against which the explanatory value of measures of comorbid illness has been assessed. Generally, researchers have found a strong and consistent relationship between the occurrence of comorbid illness and death for a variety of conditions and surgical 41 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. procedures, no matter what the data source (Iezzoni, 1997a). However, the use of mortality as a quality indicator in many situations presents significant problems. Recently, researchers have shown that the low prevalence of death for most hospitalized patients makes it unsuitable for use as a quality marker across hospitals (Rosenthal, Amrik, Way, & Harper, 1998). Certainly, when the inpatient population of interest consists primarily of medical cases for which mortality is an unexpected and rare event, mortality as a quality indicator has limited face validity (Hofer & Hayward, 1996). The problems with, and limited applications of mortality as an outcome have spurred interest in other potential measures of quality, outcomes that occur more frequently yet still have a plausible relationship to processes of care. Complications o f Care Medical and surgical complications have great face validity as quality markers because they can often be directly related to the process of care. A 1988 Office of Technology Assessment report states that “Complications such as drug toxicities and nosocomial infections are much more frequent than in-hospital death and also have face validity as outcomes that ought to be avoided whenever possible” (US Congress, 1988: p. 190). Nonfatal complications for certain procedures have also been published recently as indicators of quality in states such as Florida and California (Jost, 1994). Recent research also indicates a link between the occurrence of complications and low provider adherence to standards (Kuykendall, Ashton, Johnson, & Geraci, 1995) and overall treatment quality (Geraci et al., 1999). 42 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Researchers have employed various methods of identifying in-hospital complications in recent studies, and no one method has dominated in published research. In studies using data from clinical registries or patient medical charts, the identification of complications presents relatively few problems when these are dutifully recorded. However, when administrative data are employed, complete and accurate identification is not possible. This stems from the fact that acute and chronic conditions, as well as complications, are all recorded as secondary diagnoses in the hospital abstract. While the ICD-9-CM system explicitly provides for the coding of certain types of complications, such as adverse drug events, misadventures to patients during medical or surgical care, and others, and these specific codes are easily identified, they fail to account for a large number of events that are precipitated by inadequate care. Many of the complications of surgery, for example, are recorded as additional procedures made necessary because of problems with the initial treatment. Studies that have assessed the accuracy of claims data in identifying medical and surgical complications, using clinical data as the gold standard, have reported generally low to moderate sensitivities for claims-based ICD-9- CM codes (Geraci et al., 1997; Hartz & Kuhn, 1994; Kuykendall et al., 1995). In studies using hospital administrative data where a conservative approach to identification of in-hospital complications is taken, authors rely primarily on the ICD-9- specified complication codes. In validating their measure of comorbid illness in patients undergoing lumbar surgery, Deyo et al. (1992) used specific ICD-9-CM codes and a small number of diagnoses that should only occur postoperatively to mark the occurrence of a complication. To assess the ability of diagnosis codes from the hospital abstract to 43 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. identify actual in-hospital complications, Geraci et al. (1997) used an inclusive set (including some diagnoses that might represent acute conditions at admission) and an exclusive set of diagnoses, (consisting primarily of designated ICD-9-CM complication codes). Their inclusive set of code-based complications resembled many that have been used recently in the literature. The authors found that the exclusive set, while identifying fewer complications, also resulted in fewer false positive cases. A number of approaches to identifying complications that are not identified as such by the ICD-9-CM system have also been employed. Generally these involve the use of clinical experts who review the hospital abstract and make a judgement as to whether, given the principal diagnosis or procedure, a secondary diagnosis represents a complication of care. For example, Brailer, Kroch, Pauly, & Huang, (1996) adopted an approach whereby each combination of admitting diagnosis and secondary diagnosis was assigned a probability of being a complication or a comorbidity in constructing their comorbidity-adjusted complication risk (CACR) index. There is conflicting empirical evidence as to the impact that comorbid disease has on the occurrence of complications, but a wealth of clinical experience suggests that there should be strong and positive effects in most cases. Rosen and colleagues (Rosen, Geraci, Ash, McNiff, & Moskowitz, 1992) examined the relationship between patient severity at admission and adverse events for four surgical procedures using medical chart-abstracted data and found that the presence of significant comorbidities were only moderately predictive of postoperative adverse events, after controlling for other patient characteristics. Pompei and co-authors (Pompei, Charlson, Ales, MacKenzie, & Norton, 44 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. 1991) found that a popular weighted measure of comorbidity (Kaplan-Feinstein) was not predictive of in-hospital morbidity in a general hospital population, but was strongly predictive of 1-year mortality. Deyo et al. (1992) found that their claims-based measure was significantly associated with in-hospital complications and blood transfusion in patients undergoing spinal surgery, after controlling for age. In studies that have relied on clinical data in predicting surgical complications, positive results have often been reported. Silber and colleagues (Silber, Williams, Krakauer, & Schwartz, 1992) investigated the impact that five specific comorbidities and a clinical measure of illness severity have on the surgical outcomes of cholecystectomy and transurethral prostatectomy (TURP) in Medicare patients. They found that while four of the five comorbidities, along with admission severity, were predictive of an adverse occurrence (complication), comorbidities were generally not related to death or failure to rescue. In two studies where the original Charlson Cl was employed, higher Charlson scores were associated with increased risk for adverse events in patients undergoing carotid endarterectomy after adjustment for age and sex (Karp, Flanders, Shipp, Taylor, & Martin, 1998) and were independently associated with pulmonary complications in patients undergoing abdominal surgery (Lawrence, Dhanda, Hilsenbeck, & Page, 1996). Comorbidity as quantified by another popular measure, the ICED (Index of Co-Existent Diseases), was also found to be a significant predictor of serious complications following total hip replacement in British, but not Japanese patients, though the predictive power was weak in the English population. 4 5 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Claims data have only recently been employed in the creation of binary measures of complication occurrence and there is continuing exploration of new measures and approaches for identifying complications from this data. The most conservative measures restrict themselves to designated ICD-9-CM complication codes. Most health services researchers currently develop study-specific measures of complications occurrence since no preferred method has emerged. Unplanned Readmissions The belief that unscheduled hospital readmission shortly after discharge might be indicative of poor initial treatment of a medical condition, or at least poor discharge planning, has generated interest in the use of unplanned readmission rate as a hospital quality marker. This interest may also be related to concern over the effects that increasingly shorter hospital lengths of stay are having on patient care in the move towards prospective payment systems (Epstein, Bogen, Thorpe, Dreyer & Conolly, 1991). While unplanned readmissions cannot generally be linked directly to processes of care during the index admission, they have considerable face validity and are already being used as quality indicators in health care purchasing decisions (Thomas, 1996). Readmission rates following discharge for many conditions and procedures have been found to be quite high. Several studies have investigated the relationship between unplanned readmissions and processes of care or indicators of problems in the care of patients. An early literature review of such studies (Soecken, Prescott, Herron, & Creasia., 1991) 46 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. found only weak evidence of such a link and Thomas (1996) found no such evidence using Medicare claims data and the results of Michigan Peer Review Organization (PRO) reviews. A recent meta-analysis of the evidence (Ashton et al., 1997), however, concluded that substandard care increases the risk of early readmission by 55%, yet the authors judged the usefulness of the measure as an indicator of quality to be quite low. Weissman et al. (1999) also found evidence of an association between lower adjusted quality of care and readmissions, but judged that the outcome was not a useful tool for identifying patients experiencing inferior care. The identification of a patient readmission is relatively straightforward but it can be made more difficult when hospital use by individual patients varies across health plans or geographic areas. However, the accurate identification of unplanned readmissions related to an initial episode of care presents many difficulties to researchers. The task is made even more difficult when only administrative data are available. Because o f such difficulties, some studies have considered all readmissions in their attempts to understand influential patient and hospital characteristics, though most researchers have looked at a narrower range of readmission types. Time periods from 7 days to 90 days have commonly been used to define the readmission period, with the most common readmission time window being 30 days. The strategy used in identifying unplanned readmissions does not differ much whether administrative or clinical data is being used, though clinical strategies are advantaged by more information. In both cases clinical judgement is used to determine whether the processes of care associated with an index admission could be reasonably 47 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. related to the reason for a subsequent readmission given the time elapsed between discharge and readmission. A shorter time period lends greater face validity to such judgements, but may sharply reduce the incidence of readmission, which reduces the utility of the outcome. A fair amount of educated guesswork is required in the linking process when only claims data are available though important conceptual progress has been made in identifying sound approaches. Riley et al. (1993) demonstrate an approach using Medicare claims for eight procedures through the use of specific diagnosis and procedure codes. Wray et al. (1995) provide a conceptual framework for evaluating the feasibility of using administrative data for quality purposes using DRGs as the linking device. The work by Thomas et al. (1996) is typical of recent approaches using claims data, where various time periods are considered (15, 30, 60 and 90 days) and linking is accomplished via ICD-9-CM codes using clinical judgement. Evidence regarding the validity of this general approach to identifying unplanned readmissions using claims data is just beginning to appear. A recent European evaluation using discharge abstract and matching claims data for all Geneva Hospital patients during a recent year (Kossovy, Sarasin, Bolla, Gaspoz, & Borst, 1999) found that accurate determinations of unplanned readmissions were possible in only 64% of the cases. Most studies that have investigated the determinants of readmission have included some measure of patient severity and/or comorbidity as covariates. In the 1990s, there was mixed evidence as to whether the presence of comorbid illness increased the risk of readmission, though clinical experience would certainly argue for such a finding. The 48 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. review by Soecken and colleagues (1991) found that readmissions in the literature were primarily associated with clinical characteristics, some which indicate chronic illness. Marcantonio and colleagues (Marcantonio et al, 1999) recently found that multiple comorbidities, depression and other factors such as a recent previous admission were all associated with readmissions in a heterogeneous group of Medicare patients. In contrast, Ludke and colleagues (Ludke, Booth, & Lewis-Beck, 1993) found that none of the patient characteristics, including the PMC severity score, were related to readmission within 14 days for a randomly selected group of patients at Veterans Affairs (VA) hospitals. Most studies using administrative data have found relationships between comorbidity and readmission. In the study by Thomas and colleagues (1996), the authors found that a count of comorbid conditions, along with 5 of 12 PMC severity scale interaction terms, were significant predictors of readmissions in all 12 medical conditions considered. Camberg and colleagues (1997) found a significant association between certain comorbidities and time to readmission in a group of elderly VA patients with stroke, COPD and dementia, and Philbin & DiSalvo (1999) found seven comorbid conditions predictive of readmission for CHF in a 1995 population of all CHF patients in the state of New York. Studies that have explored the relationship between other patient-level characteristics and hospital readmissions have found that lower socioeconomic status is related to increased risk for readmission (Weissman et al., 1994) and that discharge destination is also a determinant (Camberg et al., 1997). In the latter study, the authors 49 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. found that patients with COPD and dementia were less likely to be readmitted if they were discharged to nursing homes or community care versus personal homes. Length o f Stay Risk-adjusted patient length of stay is not typically considered a measure of quality, though it is often included in empirical studies of quality of care. LOS is routinely used to approximate hospital charges or resource consumption and is of such relevance to health policy decisions that its inclusion is often automatic. It also provides information concerning the predictive power of patient-level models, which is useful in comparing performance across different measures of comorbid illness. One group of researchers (Kuykendall et al., 1995) has proposed longer than expected LOS as a flag (a dichotomous one, in this case) for potential medical care problems, and its relationship to the occurrence of adverse events in hospitalized patients has been demonstrated. However, this research exists in relative isolation. There is a large body of research that has documented a significant and sometimes strong relationship between comorbid illness and LOS and many commercial health status measures have been developed to take advantage of this fact. Validation studies concerning these instruments are beyond the scope of this review, though studies do routinely compare the performance of these resource-based measures with other measures of prognostic comorbidity. One property of these resource-based measures that makes them unsuitable for use in risk-adjusted quality research is their reliance on postadmission information, such as complications, to derive patient risk assessments. 50 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Given their purpose to estimate costs and resource use during the hospital stay, such an approach is reasonable. Conclusion The underreporting of comorbid disease is the major problem uncovered by health service researchers investigating the utility of the hospital abstract in assessing patient comorbid illness. The sensitivity of coding for certain chronic conditions in claims databases is extremely low. Current evidence suggests that this bias is systematic, reflected in the underreporting of chronic conditions for the severely ill, but whether this bias exists or not the underreporting problem remains. This underreporting is aggravated by the limited number of spaces provided by some hospital abstract systems for the recording o f secondary diagnoses. This problem, along with the inability to distinguish much acute comorbidity from complications, has led to inaccurate estimates of patient health status when using administrative data. Despite this, the low cost of hospital abstract data and their easy availability continue to invite their employment in assessments of hospital quality and this use will only grow in the future. The desirable traits of a pharmacy-based comorbidity measure include the accuracy o f the information contained, as well as availability and low cost. For patient groups that do not fill prescriptions outside their health plan, it also represents a complete record of medications filled (not necessarily prescribed or taken). Given these caveats, the pharmacy record would appear to be a reliable source of information concerning therapeutic treatment for a number of chronic conditions in certain populations (e.g., a 51 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. managed care group with a prepaid drug benefit). When one considers the documented undercoding of chronic disease in the hospital discharge abstract, the pharmacy record seems a natural response to this problem. The most widely used marker of hospital quality is the risk-adjusted mortality rate. However, the low incidence of mortality for most types of admissions makes this indicator suitable for relatively few conditions, most of them surgical procedures. In the 1990s, several researchers focused on risk-adjusted complications and readmissions as alternative outcomes, given their more desirable statistical properties. Complications have great face validity as quality indicators but the nature of the hospital abstract and the ICD-9-CM coding system limit the ability of researchers to accurately identify complications of care. A similar problem presents itself when attempting to identify unplanned readmissions using data from the hospital abstract. The few studies that have attempted to relate complications and unplanned readmissions to process of care criteria, considered quality gold standards, have generally provided positive results. Length of stay has several desirable properties as an outcome measure, one of those being its close association to hospital costs, but it is not generally considered a quality marker. Numerous clinical and claims-based empirical investigations have demonstrated a link between the presence/severity of comorbid illness and the four outcomes above. However, several studies, some using clinical data sources, have failed to find such a link and there is no satisfactory explanation for these unexpected results. Presently, most health services research proceeds on the assumption that comorbid illness increases the risk of adverse events in hospitalized patients and that risk-adjusted mortality, 52 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. complications and unplanned readmission rates are related to the adequacy of patient care. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CHAPTER 2 METHODS Sample Construction The study population was a convenience sample of 3811 members of the Southern California region Kaiser Permanente Medical Care Program (Kaiser Permanente) who were hospitalized between April 1, 1993 and March 31, 1994 for acute medical conditions. This population was drawn from a random 10% longitudinal sample of the entire adult Kaiser Permanente membership in six southern California regions. All of these patients had obtained at least one prescription from a Kaiser pharmacy within two years prior to their hospital admission. At the time of the study, the Southern California Region of Kaiser Permanente provided prepaid comprehensive health care to 2.2 million members and approximately 75% of these had an outpatient pharmacy benefit. The region was organized into 11 service areas that usually consisted of a hospital, a large outpatient medical office complex, and smaller satellite medical offices. These data come from the Kaiser-Permanente/USC Patient Consultation Study, a large prospective study that compared the impact of three different models of pharmaceutical care in the outpatient pharmacy environment on patient health outcomes. The three models examined were the KP model (designed by Kaiser Permanente and USC to correspond closely to the paradigm of pharmaceutical care), the state model 54 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. (pharmacist consultation as mandated by a California state law implemented in 1992) and the control model (the standard of pharmacy practice as specified in regulations prior to 1992). The researchers found that the KP and state models had positive impacts on patient satisfaction and utilization/reduced costs, but had no impact on self-reported patient quality of life (McCombs, Liu et al., 1998; Johnson, Parker et al., 1998; Cody, McCombs et al., 1998). The database created by the Kaiser Permanente (KP)/USC study covers a baseline period consisting of the calendar year 1992 and two demonstration periods -April 1,1993 to March 31, 1994 and April 1, 1994 to February 28, 1995. The study period for this investigation corresponds to the first demonstration year of the KP/USC study. The present research uses the 10% sample from the KP/USC study, a random geographic sample. Prescription drug usage and hospitalization information for these patients were obtained from the computerized files of the Kaiser Permanente data systems. Several types of patients were excluded from the larger KP/USC consultation study sample to make the study sample more homogeneous and to avoid potentially confounding influences in multivariate models. Patients removed from the sample include those who: 1) were admitted for maternal, mental health, or substance abuse DRGs, 2) were admitted on an outpatient or day-surgery basis, 3) were discharged against medical advice, or 4) were discharged to or admitted from another acute care hospital. Only data from the first demonstration year were used. The resulting sample consists of 3558 patients (including in-hospital deaths) with at least one hospitalization during the study period who meet the selection criteria. 55 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Data Sources All data came from the offices of Kaiser Permanente in an automated format. Two separate data files, the hospital abstract and the outpatient pharmacy claims were obtained for all patients in the 10% sample during the baseline and demonstration periods. Only individuals with a hospitalization during the first demonstration year (April 1,1993 to March 31,1994) and a pharmacy claim in the baseline period or the first year were retained. This strategy had been employed in the KP-USC study since the research targeted patients receiving prescription drugs. The researcher was unable to obtain data for individuals with no prescription claims in the baseline year but who were hospitalized in the first year study period. Hospitalization data included patient medical record number (MRN), gender, race/ethnicity, date of birth, dates of admission and discharge, length of stay, source of admission, admission type, primary and secondary pay source, DRG, admission diagnosis, up to 9 additional secondary diagnoses, up to 10 procedures codes, case type, disposition, hospital location and specialty of the admitting M.D. These files were converted into two patient-level data files using the member’s unique MRN, each file corresponding to the baseline or first year demonstration period. Records were sorted by admission date and variables for multiple hospitalizations were created (e.g., admission diagnosis for first, second, third, etc., hospitalization during the study year). In the few instances where discharge dates were prior to admission dates for a given hospitalization (fewer than 10 instances), the records were deleted. A senior official Kaiser vouched for 56 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. the overall quality of the hospital abstract data, expressing high confidence in the accuracy of the diagnosis data. He felt least confident in the quality of the procedure codes, which were not used in this analysis. Pharmacy claims records included patient MRN, sex, date of birth (DOB), date of dispensing, name and strength of drug dispensed, quantity dispensed, new prescription identifier, NDC (National Drug Code) number, drug copayment amount, average wholesale cost, and other pertinent information. Two files, corresponding to the baseline and first year periods were created and information was collapsed to the patient level using the MRN. Records were sorted by dispensing date and drug name and a number of summary variables (e.g., total number of prescriptions filled for insulin) were created. The hospitalization data were then linked to the pharmacy data, by study year, and records were retained for all patients with a hospitalization in the first year period. The study sample consisted of 3558 patients who were hospitalized at some point during the first year of the demonstration period. Variable Construction Two patient-level summary files, one corresponding to the baseline study period and the other to the first study year, were created with identical data elements from the hospital and pharmacy files. These were linked by MRN to construct the final analytical file and its variables. 5 7 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Demographic Variables The demographic variables defined for each sample member are listed in Table 2.1. Patient DOB and sex were most complete in the hospital records, so age in years, created from DOB, and the gender variable were created from this source. Previous research has indicated that the effects of age on most hospital outcomes is not linear, so a series of binary variables was created to aggregate age into 4 mutually exclusive categories (ages 18 to 45,46 to 64,65 to 75, and 76+). This division of age is similar to many used in the risk-adjustment literature and has policy relevance. A dummy variable was created to indicate female sex. Information from the hospital file also placed patients into eight ethnic/racial groupings. Because of low frequencies in four groups (Native American/Alaskan Indian, Southeast Asian, Other, Unknown), these were collapsed into a single “Other Race” grouping, leaving the five categories in total. Hospitalization Variables The Kaiser hospital files contain somewhat more information than is usually contained in state and federal hospital abstracts, so an effort was made to reduce the number of variables created for the study to those most commonly available (Table 2.1). Information concerning type of admission was transformed into three binary variables, indicating emergency, urgent, or elective admission status. The DRG was used to collapse patients into 25 mutually exclusive Medical Diagnostic Categories (MDCs). MDCs cluster patients into disease groupings with similar resource-utilization characteristics and represents case-mix adjustment at a fairly gross level. Eighteen binary 58 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 2.1 Study Variables V ariable D escription V a riab le Nam e D e m o g ra p h ic M ember Record N um ber M RN Age expressed in 4 multi-year categories A G E 18 45 (referent), AGE46 64, AGE65 75, A G E76+ Sex FEM ALE Racial Categories A FRO , ASIAN, HISPAN, RACEOTH, WHITE (referent) H o s p ita liz a tio n Admission Type EM ERG, URGENT, ELECTIV (referent) Major Diagnostic Categories based on M DC1-M DC4 MDC5 (referent) MDC6-MDC13 DRG M D C 16-M DC18 MDC21, MDCOTHR Surgery Performed M DCSURG Primary Pay Source M EDICARE, KAISER (referent) Disposition D HO M E, D OTHR (referent) Died in hospital D_DEAD O u tco m es Lognormal transform ation o f Length o f LOGLOS Stay in Days Unplanned Readm ission READM ITU Complication(s) o f Care a GENCC O th e r Model o f Pharmaceutical Care KP, STATE, CONTROL (referent) a Any of the following 1CD-9-CM codes in the secondary diagnosis fields: 415.1,451.1, 960-979, 995, 996.62, 998.5, 998.8, 998.9, 999, E850-858, E870-888, and E930-949. MDC classification variables were created and four categories with low frequencies were combined into an “Other MDC” category. The largest number of patients belong to the reference category, MDC 5 (Diseases and Disorders of the Circulatory System). The admission diagnosis was used to create a variable indicating whether a surgical procedure was performed during the hospitalization. Primary pay source information was used to create a binary variable indicating whether Medicare or Kaiser was the responsible payee. Nearly all payment types fell within these two categories. Categorical information concerning the disposition of the 59 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. patient was reduced to two binary variable, one indicating whether the patient was discharged to their own home or not and the other indicating whether the patient died in the hospital. The reference category is any other type of discharge. Outcome Variables The three outcome measures used in this study are also detailed in Table 2.1. Patient LOS in days was available from the hospital abstract. LOS exhibited a strongly right-skewed distribution so a lognormal transformation of LOS was used in analyses. The unplanned readmission indicator was created by first identifying any second hospital admission for an individual within 30 days of the index admission discharge date. Of these, only admissions coded as emergency or urgent in type were deemed to be unplanned. No attempt was made to link the diagnosis for the second admission to that of the first to determine the relatedness of the admissions. A binary indicator to mark the occurrence of at least one in-hospital complication was created from the secondary diagnoses of the hospital abstract. Each o f the nine fields for the index admission was scanned for any of the ICD-9-CM codes listed in the footnote to Table 2.1 and when a code matched, a complication was denoted. These complication codes were identified by Geraci et al. (1997) as an exclusive set that ‘seemed most likely to represent in-hospital complications in a general medical population’ (p. 593). Code-based complication definitions in other studies have tended to be more inclusive of conditions that might represent comorbidities. No attempt was made to identify other possible complications not explicitly labeled as such by the ICD-9- 60 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CM terminology (e.g., secondary diagnoses of bacteremia, pressure ulcer, or cardiac arrest). Other Variables The study for which these data were originally collected was an evaluation of the effect of different models of pharmacy practice on patient outcomes, including those related to hospitalization. Patient exposure to the various models of pharmacy practice was controlled for by including the original treatment variables indicating model of pharmaceutical practice. Measures Information from the nine secondary diagnosis fields was used to create binary disease indicators for the diagnosis-based comorbidity measures. Pharmacy-based chronic disease indicators were based exclusively on data from prescription records. Deyo (Deyo et al., 1992) comorbidity indicators were derived following the method described in the original article and required hospital data from both the baseline and first year periods. The secondary diagnoses of the index admission of the first year were scanned for any of the ICD-9-CM codes corresponding to the diseases listed in Table 2.2. Baseline year data, linked by MRN, was also scanned for chronic disease information from previous hospitalizations. The decision logic employed to exclude new acute conditions in the index admission is noted in the table. Seventeen binary chronic disease variables were created. 61 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 2.2 Comorbidities Included in Deyo Measure C om orbidities IC D -9-C M Codes* 1. M yocardial Infarction 410-410.9 412* 2. Congestive Heart Failure 428-428.9 3. Peripheral Vascular Disease 443.9*, 441-441.9*, 785.4*, V43.4* 4. Cerebrovascular Disease 430-438 5. Dem entia 290-290.9* 6. Chronic pulmonary disease 490-496*, 500-505*, 506.4* 7. Rheumatological disease 710.0, 710.1, 710.4, 7144.0-714.2, 714.81, 725 8. Peptic Ulcer Disease 531-534.9, 531.4-531.7, 532.4-532.7, 533.4- 533.7, 534.4-534.7 9. M ild liver disease 571.2*, 571.5*, 571.6*, 571.4-571.49 10. Diabetes 250-250.3 240.7 11. D iabetes with chronic complications 250.4-250.6* 12. Hemiplegia or paraplegia 344.1,342-342.9 13. Renal disease 582-582.9, 583-583.7, 585, 586, 588-588.9 14. Any malignancy, including leukemia 140-172.9, 174-195.8,200-208.9 and lymphoma 15. M oderate or severe liver disease 572.2-572.8 456.0-456.21 16. M etastatic solid tumor 196-199.1 17. AIDS 042-044.9 Note. Codes with asterisk were included if listed during index or prior admissions. Other codes were included only if recorded prior to the index admission. * ICD-9-CM, International Classification of Diseases, 9th Revision, Clinical Modification Elixhauser (Elixhauser et al., 1997) comorbidity indicators were created using information from the index admission of the first study year. The process followed replicates that of the authors in their article. The individual binary comorbid markers were created by scanning the secondary diagnoses of the index admission for any of the eligible ICD-9-CM codes and applying the appropriate DRG screening criteria. The 30 comorbidities, their corresponding ICD-9-CM codes and DRG screening criteria are detailed in Table 2.3. 62 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. T able 2 .3 Comorbidities Included i n Elixhauser Measure O T * on P o Q P s n C 8 u C5 a s 8 D l 8 IS O P 1 I - p p 8 C / D + 5 a s! § s c n o o — o o w q ! u l/l A > C L h O U o ea * o T 3 i - CO C O U U o CO U o CO o CO CO O c o c o e o I ■ 't f * -5 .S- eo u U S c/i c/i C G _ « P P C 3 ts t: c p p p o. c l c 2 K K o no c o £ p 4-» c/i > o C/D c/i 3 O £ P £ OO O n i VO w C O E < v_ o o o o o Q O h o C J n o n o ON ON <N CN i i O n O n CN CN O o T 3 C P © O n CM CN O c o C /l ^ SO s « .2 is CL ^ 5 c/i p 2 -2 rs < = > o < ? t o _ .3! . S J ? ® 1 2 < 1 > .C Q Q h C . ^ « l J p p ■ 4 — » W o) p X > jo *«S G 68 — 8 p o o p oo a 1: o E P f f i cO E o • s © i" AQ C O a> L»S p • © 0 U 2 u 1 A U ON OO * ( N <N 2 * CN O CO Q ON ^ S d o oo m m CN ON ^ vq O V O * no " > CN CN N - > VO v d N - c o v d c o . <o *n 2 ^ CN — C N r t vd O CN N* ~ C O O n ~ CN vd O CN ^ N* CN V O O C N —~ © O n ~ OO vd ON C N c o ^s* c o c o on i^s w ’ N “ CN 0 *1* n o © N f o _ ■ “ 2: „ c o On q 2 O n ^ CO On ° - o - " & ^ 2 o n TJ- o *, I — o CO CN . _ < o 2 v d CN C £ 5 O > O n v d ‘I 1 CO •O NO O n O s O p NO o o r - * NO o CO CO TT O n o f o 0 0 r - o f NO 3 - CO CO CO N* i o •N* o C 1 o o CO *3* v d CO NO CO — r o NO N* o o 0 0 3 * o f N * 3 * CO r r CO NO o NO O n p d 3 * NO CO NO N * o CO NO O n O CO o N - d ■ 'd O n CO 0 0 ON CO NO •3* N f O N * o ' ON NO CO CO CO p CO CO i CO NO O n CN N * O O n CN NO " T o mm 3 * CO CO o o ' CO CO CO t o c d ON O s O n (N CN p NO O n O 3 - T CO Tj* > o O N * o " CN CO 1 CN CO CO TT CO OO CN O n o o C — o o f *r3* o CN CN mrn NO o N* NO o O ’ «3* CO 3 * ON NO CO CO CO O NO < ? O N O CN NO CN OO r f CN c o c o c o o o n o NO CN CN CN o o o -< r o o NO NO CO TJ- o v d N O > NO > o CN r r > VO o o NO NO o o NO ^3- O N - CN O n O ’ o N - CO o s § C N N O C O o' ( N m m r- N O N O p C N C N r ~ v d N O N O N T p O r - C N N O v d • K N O O O N Tm m t > - N O N O r . N O p X d r - N O v d N O N O m l N O r * N O d o * o r > N O C O C O d d r * ~ - o N O C N C O C O d m m t o N O o O n CO NO o CO N O o " ON CO CO NO <=T r«# c o CO NO o ' ON CN CO NO o CN CO NO o ' ON CO NO P - * 5 ^ CN ^ CN ? c o r > " NO >■ r - o CO r - CN v d o o c o CN O O c o o CN CO o CN o CO o CN i o N O CN o CN _ On o o P CO o c n r OS 0 > | 3 t f O o . ( N O ® ^ o r “ O CN > *o -f i i m o B o u (I n -p £ U a U U CO a> C O -S CO s a> c/i * a u- c w o * p » O -a a / s p s « c o c c a> O J 5 E .£■ 3 a> P-. CL p ^ 2 " E , E o o G c .2 ’w G n> t : a> CL X a> E o a . E o p c o * « G P t : p CL P - a .S£ 5 Cfl G o u ^ 3 -C C3 ^ cu O p C/i « N CO P *T3 o j f - G cO fe g-,a § l a E I § e ^ i C O = 3 > S ‘o CL i m w to >. .2 M & J a * 2 *-» < 4 — < ^ 3 C < U O o p j i £ 2 i s n S; u Q b a O co o j E f t * G T 00 o H 3 8 P P > OO •S - 3 2 8 p p c/i 3 P c/i P P P M .2 ^ 03 E g Q & H <. es E o 4 3 O . £ no VO O * —• CN CO 6 3 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. Table 2.3 Comorbidities Included in Elixhauser Measure (Continued) Com orbidities8 ICD-9-CM Codesb DRG Screen: Case Does Not Have the Following (DRGs) 18. Metastatic Cancer 196.0-199.1 Cancer 19. Solid Tumor w/out Metastasis 140.0-172.9,174.0-175.9, 179-195.8, VI 0.00-V 10.9 Cancer 20. Rheumatoid arthritis/collagen vascular diseases 701.0, 710.0-710.9,714.0-714.9,720.0-720.9,725 Connective tissue (240-241) 21. Coagulopathy 2860-2869,287.1,287.3-287.5 Coagulation (397) 22. Obesity 278.0 Obesity procedure (288) or Nutrition/metabolic (296-298) 23. Weight loss 260-263.9 Nutrition/metabolic (296-298) 24. Fluid and Electrolyte Disorders 276.0-276.9 Nutrition/metabolic (296-298) 25 Blood loss anemia 2800 Anemia (395-396) 26. Deficiency anemias 280.1-281.9,285.9 Anemia (395-396) 27. Alcohol abuse 291.1,291.2,291.5,291.8,291.9, 303.90-303.93,305.00-305.03, V I13 Alcohol or Drug (433-437) 28. Drug Abuse 292.0,292.82-292.89,292.9, 304.00-304.93, 305.20-305.93 Alcohol or Drug (433-437) 29. Psychoses 295.00-298.9, 299.10-299.11 Psychoses (430) 30. Depression 300.4, 301.12,309.0,309.1,311 Depression (426) Note: DRG = diagnosis-related group; COPD = chronic obstructive pulmonary disease; GI = gastrointestinal; AIDS = acquired immune deficiency syndrome; HIV = human immunodeficiency virus. Definitions of DRG groups: Cardiac: DRGs 103-108, 110-112, 115-118, 120-127, 129, 132-133, 135-143; Renal: DRGs 302-305, 315-333; Liver: DRGs 199-202, 205-208; Leukemia/lymphoma: DRGs 400-414, 473, 492; Cancer: DRGs 10, 11, 64, 82, 172, 173, 199, 203, 239, 257-260, 274,275, 303, 318, 319, 338, 344, 346, 347, 354, 355, 357, 363, 366, 367, 406-414. " A hierarchy was established between the following pairs of comorbidities: If both uncomplicated diabetes and complicated diabetes are present, count only complicated diabetes. If both solid tumor without metastasis and metastatic cancer are present, count only metastatic cancer. b ICD-9-CM, International Classification of Diseases, 9lh Revision, Clinical Modification; The general procedure described by Clark and colleagues (1995) was used to derive the 28 drug-based chronic disease indicators, with some modifications (Table 2.4). Prescription drug use information from one-year previous to the index admission date was used to construct the Revised CDS markers. The Kaiser prescription claims files do not contain the AHFS classification information used by the CDS authors so it was necessary to link the prescription files with an external file via the NDC in order to obtain the AHFS codes. This process resulted in identification of nearly 80% of the unique drugs from the KP data files and identification of the AHFS codes for the remaining drugs was done manually using the 1998 AHFS reference guide (American Society of Hospital Pharmacists). Newer, unclassified therapeutic agents not appearing in the Revised CDS coding schema were also matched to AHFS categories via the drug name if they were indicated for the conditions covered. Several pharmacists from the Department of Pharmaceutical Economics and Policy at the USC School of Pharmacy aided in this effort. Translation of the AHFS codes into the binary disease markers in the KP/USC study revealed two problems with the Revised CDS approach when applied to these data. Specific methodologies were created, based on route of administration and drug name, to resolve these problems. In the first instance, the revised CDS assigns both rheumatologic and respiratory illness status to patients using adrenals (AHFS code 68:04), which may result in both misclassification and double counting of disease. Route of administration information was used to separate these drugs into topical preparations (for which no 65 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 2.4 Chronic Conditions Included in Revised CDS Conditions AHFS Drug Classes AHFS Categories 1. Cardiac Disease, ASCVD, CHF Cardiac drugs 24:04 2. Coronary and Peripheral Anticoagulants, Hemorrheologic 20:12.04, 20:24, Vascular Disease agents, Unclassified therapeutic agents 92:00 (various) 3. Respiratory Illness, Sympathomimetic agents, Respiratory 12:12 (route = Asthma smooth muscle relaxants, Unclassified therapeutic agents inhalation), 86:16, 92:00 (various) 68:04 (route = inhalation) 4. Rheumatologic Adrenals, Gold compounds, 68:04 (route = oral or conditions Antimalarial agents po), 60:00, 8:20 5. Acid Peptic disease Misc. GI drugs 56:40 6. Liver Failure Ammonia detoxicants 40:10 7. Diabetes Antidiabetic agents 68:20 8. Renal Disease Potassium removing resins 40:18 9. ESRD Hematopoietic agents 20:16 10. Malignancies Antineoplastic agents, Hematopoietic antiemetics 10:00, 20:16, 56:22 11. HIV Antivirals, Misc. antiinfectives 8:18, 8:40 12. Epilepsy Anticonvulsants 28:12 13. Hyperlipidemia Antilipemic agents 24:06 14. Hypertension Hypotensive agents 24:08 15. Glaucoma Carbonic anhydrase inhibitors, Misc. EENT drugs, Miotics 52:10, 52:36, 52:20 16. Cystic Fibrosis Mucolytic agents, Digestants 48:24, 56:16 17. Transplantation Unclassified therapeutic agents 92:00 (selected drugs) 18. Thyroid Disorders Thyroid agents, Antithyroid agents 68:36.04, 68:36.08 19. Parkinson’s Disease Antiparkinsonian agents 12:08.04 20. Gout Unclassified Therapeutic Agents 92:00 (selected drugs) 21. Crohn’s and ulcerative Sulfonamides, Misc. GI drugs 8:24, 56:40 colitis 22. Pain Opiates 28:08.08 23. Pain and inflammation Non-steroidal anti-inflammatory agents 28:08.04 24. Tuberculosis Antituberculosis agents 8:16, 6804 25 Depression Antidepressants 28:16.04 26. Psychotic illness Tranquilizers 28:16.08 27. Bipolar Disorders Antimanic agents 28:28 28. Anxiety and tension Benzodiazepines, Misc. anxiolytics, sedatives, and hypnotics 28:24.08, 28:24.92 Note. AHFS = American Hospital Formulary Service. 66 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. disease was assigned), oral drugs (indicated for rheumatological conditions) and inhaled adrenals (indicating respiratory illness). In the second case, use of sympathomimetic agents (AHFS code 12:12) assigns the patient respiratory illness status, yet some of the agents are anti-inflammatory creams and lotions. So, drugs whose route of administration was not inhalation or oral were excluded in assigning this condition. Predictive Models A series of logistic and OLS regression models were developed to assess the performance of the three different measures of patient comorbidity in predicting future adverse events and LOS. The simplest of these were age/sex models, with independent variables restricted to patient sex and the 4 binary age variables. The age/sex model has policy relevance and allows for comparisons of model performance with those published in the health services literature. Predictive modeling was performed on the same patient sample when complications and length of stay were the outcomes of interest. In modeling unplanned readmissions, in-hospital deaths were removed from the sample since they did not constitute an eligible group for readmission. A series of 3 predictive models were fitted using the patient sex and age variables and each of the three comorbidity measures (Deyo, Elixhauser, Revised CDS). Each of these age/sex models (Table 2.5) was used to predict the 3 study outcomes, resulting in a total of 9 equations. The overall goodness of fit measures associated with these equations provided an indication of the relative ability of each comorbidity measure to predict the 67 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. three outcomes, after accounting for patient age and sex. Statistical tests were also conducted to evaluate whether each comorbidity measure significantly improved the goodness of fit for these models after controlling for age and sex. Table 2.5 Age/Sex Models to Predict Complications, Readmissions and LOS M easure Covar. Hospital Diagnostic Data Pharm acy Data 1. Deyo age/sex 17 Comorbidity M arkers None 2. Elixhauser age/sex 30 Comorbidity M arkers None 3. Rev. CDS age/sex None 28 Drug Class Markers Models were then developed to answer the central research question, “Does the addition of pharmacy-based chronic disease information improve the prediction of outcomes after accounting for diagnosis-based measures of patient comorbidity?” In these analyses, the Revised CDS comorbidity markers were added to the Deyo and Elixhauser age/sex models listed in Table 2.5 and any increase in predictive performance resulting from their inclusion was assessed for practical and statistical significance. Two series o f models, indicated below, were fitted for each of the three outcome variables, resulting in 6 separate equations to test whether pharmacy data had explanatory power above and beyond that provided by diagnostic data, while controlling for age and sex. 1. Deyo-Rev. CDS Age/sex+ Deyo (17 Markers) + Rev. CDS (28 Markers) 2. Elix.-Rev. CDS Age/sex + Elixhauser (30 Markers) + Rev. CDS (28 Markers) A series of predictive models utilizing more patient demographic and hospitalization information were also constructed to explore essentially the same 68 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. hypothesis as posed with the age/sex models. These are referred to as full information models since they include much more patient-specific information. Three general arguments support their inclusion in these analyses: 1) they provide estimates of overall goodness of fit based on reasonably complete administrative data models, which are of interest to both researchers and policy makers; 2) they better control for patient differences that might independently influence outcomes and/or be related to comorbidity, providing a stricter test of the research hypothesis; and 3) they enable us to compare the explanatory value of comorbidity information with other types of available information (e.g., MDC, admission status). The specific variables (detailed in Table 2.1) were included because either previous research has shown them to have an independent influence on the outcomes studied here or because strong clinical rationales argue for their inclusion: • Patient sex • Age • Race/Ethnicity • Admission type • Primary pay source • Medical Diagnostic Category • Surgery indicator • Home discharge indicator • KP-USC intervention model The same sets of analyses were performed with the full information series of models as were performed with the age/sex models. The only difference between the two model types is that the full information models, in addition to having age and sex, contain the patient and hospitalization variables listed above. 69 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Measures o f Model Performance The overall fit of the linear multiple regression models was assessed with the F test and the log likelihood ratio test of goodness of fit (G2 ) was used with multiple logistic models. Two measures of explanatory power were used in comparing the performance of multivariate predictive models: the coefficient of determination (R2 ) and the C-statistic. The R2 is reported for LOS models and the C-statistic is reported for equations modeling complications and unplanned readmissions. i?2is commonly described as the percentage of variance in the dependent variable explained by the linear combination of the independent variables and is employed with continuous dependent measures. The simple R2 is not a valid statistical measure for comparing model performance when the number of independent variables varies across models, as in this analysis. Instead, an adjusted R2 , which takes into account the number o f independent variables relative to the number of observations, is reported in the results. The C-statistic provides an estimate of the discriminatory power of the risk- adjustment models and is appropriate when dichotomous dependent measures are employed. The C-statistic is equal to the area under the receiver operator curve (Hanley & McNeil, 1982) and measures how well the model discriminates, for example, between patients who had complications and those who did not. A C-value of .50 indicates no ability to discriminate and a value of 1.0 indicates perfect discrimination. To test whether the addition of the pharmacy-based data to the Deyo and Elixhauser age/sex models resulted in significantly greater explanatory power, F tests and 70 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Chi-square tests were used. For LOS models, an F-test was used to determine if the 28 Clark variables as a group contributed significantly more to the variance explained in models already containing demographic and diagnosis-based comorbidity information. The Wald chi-square test was used in a similar fashion for models predicting complications and readmissions. All statistical analyses were performed using the REG and LOGISTIC procedures from SAS PC Version 6.12. 71 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CHAPTER 3 DESCRIPTIVE STATISTICS Descriptive statistics for the study sample are presented in this chapter. First, the demographic characteristics of the sample are presented, followed by the hospital characteristics, study outcomes and other characteristics of the sample. Next, the frequency of specific diseases identified by the Deyo and Elixhauser diagnosis-based comorbidity measures is provided, along with the frequency of conditions identified using the Revised CDS. Finally, the prevalence of disease within 15 general categories common to the three measures are presented. The level of agreement between these three measures regarding the presence/absence of disease is also assessed. This final examination of disease across measures is motivated by two related purposes. First, it allows for a general contrast of disease prevalence by instrument and data source. As noted in Chapter 1, the literature suggests that chronic disease is underreported in the hospital abstract when compared to the clinical record. A pharmacy- based measure of comorbidity should indicate greater prevalence of specific chronic diseases than the hospital abstract if it is expected to ameliorate underreporting. Secondly, given the potential unreliability of pharmacy markers as disease indicators, it is helpful to understand where pharmacy and diagnostic data agree best on the presence of disease in patients. For example, in instances where similar prevalences for a given 72 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. disease are noted using the two data sources but agreement is low, pharmacy data may be a poor marker for a given disease. The principal hypothesis being tested in this dissertation does not state the mechanism by which pharmacy data is expected to improve prediction because the sample data do not support such a test. But the literature does suggest how such improvement might be realized, and this comparison provides preliminary evidence regarding such arguments. Demographic, Hospitalization and Outcomes Characteristics The demographic characteristics of the 3558 patients in the study sample are presented in Table 3.1. The study sample ranged in age from 18 to 97 years with a mean age of 57.8 years and a median of 59.6 years. The distribution of ages shows that more than one-third of the patients (34%) fall within the 46-64 age range. The sample is largely White (66%) with a roughly equal number of Blacks and Hispanics (14%) forming the next largest racial/ethnic grouping. Asians and Other races comprise much smaller groupings (5% and 2%, respectively). There are slightly more women (54%) than men. The mortality rate for these patients, which only includes deaths occurring within the index admission, is 2%. Hospitalization characteristics are presented in Table 3.2. Approximately two- thirds of the admissions were non-elective, with the majority (55%) being urgent and a smaller percentage (13%) being emergency-related. Elective admissions constitute a third of the total. The number of patients for whom the Kaiser plan is their primary pay source (68%) is double the number of those who have Medicare as their primary 73 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 3.1 Demographic Characteristics o f Study Sample (N=3558) Measure Number Percent or Mean (SD) Age Overall 57.8(17.5) 18-45 years 922 26 46-64 years 1220 34 65-75 years 818 23 76+ years 598 17 Sex (Female) 1932 54 Race/Ethnicity White 2318 66 Black 506 14 Hispanic 485 14 Asian 170 5 Other 55 2 Inhospital Mortality 86 2 pay source. Most patients (82%) were discharged to their homes following hospitalization. The breakdown of patients by Medical Diagnostic Category reveals an array of diagnosis groups typical of an acute hospital population. Twenty-percent of patients, the largest single group, were admitted for MDC 5, Diseases and Disorders of the Circulatory system. Admissions for MDC 6 and MDC 8, Diseases and Disorders of the Digestive System and Diseases and Disorders of the Musculoskeletal System Connective Tissue, were similar in size, each representing 13% of the patient sample. The only other category comprising 10% or more of the sample was MDC 4, Disease and Disorders of the Respiratory System (10%). A full 45% of all patients underwent some surgical procedure during their hospital stay. 7 4 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 3.2 Hospitalization Characteristics o f Study Sample Measure Number Percent Admission Type Emergency 449 13 Urgent 1926 55 Elective 1160 33 Primary Pay Source Medicare 1128 32 Kaiser 2407 68 Disposition Home Discharge 2928 82 Medical Diagnostic Category (MDC) Diseases and Disorders MDC 1 Nervous System 249 7 MDC 2 Eye 31 1 MDC 3 Ear, Nose and Throat 75 2 MDC 4 Respiratory System 338 10 MDC 5 Circulatory System 723 20 MDC 6 Digestive System 454 13 MDC 7 Hepatobiliary System and Pancreas 177 3 MDC 8 Musculoskeletal System Connective Tissue 462 13 MDC 9 Skin, Subcutaneous Tissue and Breast 157 5 MDC 10 Endocrine, Nutritional and Metabolic System 123 4 MDC 11 Kidney and Urinary Tract 171 5 MDC 12 Male Reproductive System 92 3 MDC 13 Female Reproductive System 292 8 MDC 16 Blood and Blood Forming Organs 34 1 MDC 17 Myeloproliferative 30 1 MDC 18 Infectious and Parasitic 59 2 MDC 21 Injuries, Poisonings and Toxic Effects of Drugs 30 1 MDCOTHR 33 1 Surgical Procedure ICD-9-CM Code 1593 45 The distributions of outcomes and intervention characteristics are presented in Table 3.3. Eleven percent of patients were readmitted, regardless of reason, within 30 days after discharge from the index admission. This figure may include readmissions R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. planned for during the initial stay. The figure for unplanned readmissions (8%) includes only those classified as urgent or emergency in nature. Approximately 9% of patients experienced a complication of care in the hospital. The average length of stay was approximately 4 days (median = 3, mode = 1), though the standard deviation of 5.0 shows considerable variability in LOS. LOS ranged from 1 to 120 days and 21% of hospital stays were for only one day. The pharmaceutical care model variables indicate the type of care provided to the patients at pharmacies in their geographical area. Models of pharmaceutical care were assigned geographically but patients were able to visit pharmacies outside their area, so patients may have been exposed to more than one model of care. More patients lived in State model areas (42%) than in the KP or Control model areas (29% each). Table 3.3 Outcomes and Treatment Models for Study Sample Measure Number Percent or Mean (SD) Outcomes Any Readmission 400 11 Unplanned 286 8 Readmissions Complications 333 7 Length of Stay 4.3 (5.0) Model of Pharmaceutical Care KP 1037 29 State 1497 42 Control 1025 29 7 6 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Prevalence of Disease Indicated by Comorbidity Measures The prevalence of specific diseases that constitute the Deyo Cl is shown in Table 3.4. Of the 17 diseases, the three occurring most frequently in the sample are diabetes (10.2%), chronic pulmonary disease (9.2%), and myocardial infarction (6.3%). All other diseases occur in less than 5% of patients. Four diseases that occur in less than 1 % of all patients include: AIDS (.9%), moderate or severe liver disease (.5%), mild liver disease (.8%), and peptic ulcer disease (.4%). Table 3.4 Number and Percent o f Patients with Deyo Comorbidities Comorbidities Number Percent 1. Myocardial Infarction 223 6.3 2 . Congestive Heart Failure 87 2.4 3. Peripheral Vascular Disease 52 1.5 4. Cerebrovascular Disease 77 2.2 5. Dementia 48 1.3 6. Chronic pulmonary disease 328 9.2 7. Rheumatological disease 52 1.5 8. Peptic Ulcer Disease 13 0.4 9. Mild liver disease 30 0.8 10. Diabetes 363 10.2 11. Diabetes with chronic complications 75 2.1 12. Hemiplegia or paraplegia 101 2.8 13. Renal disease 87 2.4 14. Any malignancy, including leukemia and lymphoma 138 3.9 15. Moderate or severe liver disease 17 0.5 16. Metastatic solid tumor 149 4.2 17. AIDS 33 .9 The prevalence of diseases included in the Elixhauser comorbidity measure is displayed in Table 3.5. Of the 30 diseases listed, only 2 occur in more than 10 percent of 77 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. patients: hypertension (24.5%) and fluid and electrolyte disorders (17.8%). Other diseases that occur in more than 5% of the sample include uncomplicated diabetes Table 3.5 Number and Percent o f Patients with Elixhauser Comorbidities Comorbidities Number Percent 1. CHF 130 3.7 2. Cardiac arrhythmias 238 6.7 3. Valvular disease 43 1.2 4. Peripheral Vascular Disorders 51 1.4 5. Chronic Pulmonary Disease 281 7.9 6. Pulmonary circulation disorders 6 0.2 7. Rheumatoid arthritis/collagen vascular diseases 44 1.2 8. Peptic Ulcer disease excluding bleeding 10 0.3 9. Diabetes (uncomplicated), 315 8.9 10. Diabetes (complicated) 98 2.8 11. Renal failure 73 2.1 12. Lymphoma 17 0.5 13. Liver Disease 27 0.8 14. Metastatic Cancer 70 2.0 15. Solid Tumor w/out Metastasis 201 5.6 16. AIDS 11 0.3 17. Hypertension (uncomplicated): 873 24.5 Hypertension (complicated): 3 0.1 18. Hypothyroidism 127 3.6 19. Other neurological disorders 66 1.9 20. Paralysis 43 1.2 21. Depression 64 1.8 22. Psychoses 29 0.8 23. Coagulopathy 56 1.6 24. Drug Abuse 20 0.6 25 Weight loss 11 0.3 26. Blood loss anemia 65 1.8 27. Deficiency anemias 211 5.9 28. Alcohol abuse 87 2.4 29. Fluid and Electrolyte Disorders 634 17.8 30. Obesity 138 3.9 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. (8.9%), chronic pulmonary disease (7.9%), cardiac arrhythmia (6.7%), deficiency anemia (5.9%) and solid tumor without metastasis (5.6%). Diseases occurring in less than 1% of the sample include complicated hypertension (.1%), pulmonary circulation disorders (.2%), AIDS (-3%), peptic ulcer excluding bleeding (.3%), weight loss (.3%), lymphoma (.5%), liver disease (.8%) and psychoses (.8%). The number and frequency of chronic conditions using the Revised CDS measure are shown in Table 3.6. The suggested prevalence of chronic conditions is generally much greater when using the pharmacy-based markers. Three conditions occur in more than one third of all patients: hypertension (39.6%, pain and inflammation (38.4%), and cardiac conditions (36.6). Another seven conditions occur in more than 10% of all patients, including pain (29.3) anxiety and tension (20.3), Respiratory illness (15.9%), rheumatologic conditions (15%), malignancy (12.1%), depression (11.7%), and diabetes (11.4%). The least frequently occurring therapeutic classes are Crohn’s and ulcerative colitis (.5%), transplantation (.5%), cystic fibrosis (.3%), bipolar disorders (.3%), ESRD (.3%), and renal disease (.2%). Agreement Between Comorbidity Measures The three comorbidity measures used in this study differ in two very basic ways: the total number of comorbidities that comprise the measure and the different types of comorbid conditions included. Both the Revised CDS and Elixhauser measures include at least 10 conditions more than does Deyo. The Deyo and Elixhauser measures, both 79 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. based on the ICD-9-CM codes, contain approximately 13 shared diagnostic groupings. The Revised CDS and Elixhauser measures have approximately 15 conditions in common while the CDS and Deyo share 12 conditions in common. Table 3.6 Number and Percent of Patients with Revised CDS Conditions Conditions Number Percent 1. Cardiac Disease, ASCVD, CHF 1304 36.6 2. Coronary and Peripheral Vascular Disease 161 4.5 3. Respiratory Illness, Asthma 557 15.9 4. Rheumatologic conditions 535 15.0 5. Acid Peptic disease 690 19.4 6. Liver Failure 27 0.8 7. Diabetes 407 11.4 8. Renal Disease 7 .2 9. ESRD 12 .3 10. Malignancies 431 12.1 11. HIV 30 .8 12. Epilepsy 117 3.3 13. Hyperlipidemia 199 5.6 14. Hypertension 1410 39.6 15. Glaucoma 130 3.7 16. Cystic Fibrosis 11 0.3 17. Transplantation 17 0.5 18. Thyroid Disorders 259 7.3 19. Parkinson’s Disease 30 0.8 2 0. Gout 82 2.3 21. Crohn’s and ulcerative colitis 19 0.5 2 2. Pain 1043 29.3 23. Pain and inflammation 1367 38.4 24. Tuberculosis 34 1.0 25 Depression 415 11.7 26. Psychotic illness 58 1.6 27. Bipolar Disorders 12 0.3 28. Anxiety and tension 723 20.3 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The maimer in which the code-based measures assign disease status to patients differ in two regards: the specific ICD-9-CM codes employed to assign disease, and the algorithms used to distinguish complications from comorbidities. For example, using the Deyo method a diagnosis of CHF is assigned only when codes 428.0-428.9 appear as secondary diagnoses in a previous hospitalization for a given patient (Table 2.2). CHF codes appearing on the index admission abstract are ignored because they may represent complications. A diagnosis of CHF is assigned under the Elixhauser framework when codes 428.0-428.9, as well as several others (Table 2.3), appear on the index admission abstract and the index DRG does not represent a cardiovascular condition. The differences in prevalence between these two measures for CHF (2.4% for Deyo vs. 3.7% for Elixhauser) is explained by these two factors. The fundamental difference between the two code-based measures and the Revised CDS is, of course, that the CDS disease markers are based on pharmacy data. This makes comparisons of disease prevalence across these instruments difficult. However, if one keeps in mind the caveat that pharmacy-based markers only indicate possible disease, such a comparison helps us understand how differences in disease prevalence across measures may relate to covariate-level differences in predictive performance in multivariate models. Table 3.7 provides such a comparison across 15 disease categories, showing the percentage of cases within disease categories and a measure of agreement across instruments. Only comorbidities common to the Revised CDS and at least one of the diagnosis-based measures are included. The kappa coefficient reported (Cohen, 1960) compares observed agreement to agreement expected 81 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. by chance. The target set for kappa is the upper left to lower-right diagonal of a two-by- two matrix as these cells indicate agreement. A coefficient of 1 indicates perfect agreement while a coefficient of 0 indicates no agreement. The kappa coefficient is preferred in epidemiological research since it does not suffer the same problems as the Chi-square test of independence and related measures of association: these statistics may be misleading when some cells in a two-by-two matrix have fewer than five cases. Table 3.7 Conditions Held In Common By Three Comorbidity Measures Percent of Cases Agreement (Kappa) Disease Categories Deyo CDS Elix. CDS-D CDS-E D-E 1. AIDS/HIV 0.9 0.9 0.3 .66 .49 .50 2. Cardiopulmonary disease 7.9 36.2 9.6 .19 .16 .09 3. Chronic pulmonary disease 9.2 15.9 7.9 .41 .37 .92 4. Neurological disorders8 NA 4.1 1.9 NA .30 NA 5. Depression NA 11.7 1.8 NA .15 NA 6. Diabetes6 11.7 11.3 11.4 .73 .75 .90 7. Hypertension6 NA 39.3 24.4 NA .47 NA 8. Liver diseasec 0.5 0.8 0.8 .31 .29 .45 9. Neoplasms* 1 6.7 12.2 8.0 .15 .16 .21 10. Peptic ulcer disease 0.4 19.4 0.3 .01 .01 .08 11. Peripheral vascular disease 1.5 4.5 1.5 .07 .09 .46 12. Psychotic illness6 NA 1.8 0.8 NA .23 NA 13. Renal diseasef 2.5 0.5 2.1 .13 .17 .59 14. Rheumatologic disease 1.5 15.0 1.3 .08 .07 .85 15. Thyroid disorders NA 7.3 3.6 NA .52 NA ‘ For Revised CDS, Epilepsy and Parkinson’s disease markers combined. b For Deyo and Elixhauser, both complicated and uncomplicated diagnoses combined. c For Deyo, Moderate or severe liver disease and Mild liver disease combined. d For Deyo, Any malignancy and Metastatic soiid tumor combined. For Elixhauser, Lymphoma Metastatic cancer, and Solid Tumor w/out metastasis combined. ' For Revised CDS, Bipolar disorders and Psychotic illness combined. f For Revised CDS, Renal disease and ESRD combined. 82 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. In creating the disease categories listed in Table 3.7, individual comorbidity markers were sometimes collapsed to make categories more comparable. The footnotes to the table explains how this was done. The prevalence of disease is generally similar across the diagnosis-based measures, with categories such as peripheral vascular disease, renal disease, rheumatologic conditions, peptic ulcer disease, and liver disease showing very similar levels of occurrence. Other differences are fairly small in absolute terms, though three times as many patients appear to have AIDS/HIV using the Deyo measure (.9%) compared to the Elixhauser measure (.3%). This difference can be completely explained by the each instrument’s use of a different screening criteria. For many disease categories, the incidence of disease according to the Revised CDS is much greater than that noted using the diagnosis-based measures. There are large absolute differences in the frequency of hypertension and depression between the Revised CDS and the Elixhauser measure (39.6% vs. 24.6%, and 11.7% vs. 1.8%, respectively). Also, ten times as many patients appear to have a rheumatological condition when using the Revised CDS (15%) than when using the Deyo (1.5%) or Elixhauser (1.2%) measures. A very large difference exists between the rate of peptic acid disease using the Revised CDS (19.4%) and either the Deyo (.4%) or Elixhauser (.3%) measures. For the remaining disease categories, the incidence of disease recorded by the Revised CDS is generally greater than, or very similar to, the incidence found using the diagnosis-based measures with one notable exception: renal disease occurs in only .5% of patients according to the CDS, compared to 2.4% for Deyo and 2.1% for Elixhauser. 83 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The greatest agreement occurs between the two diagnosis-based measures. For COPD, diabetes, and rheumatologic disease, this level of agreement is quite high (.92, .90, and .85, respectively). Only slight to moderate agreement is shown for the rest of the disease categories. Agreement between the Revised CDS and the two code-based measures ranges from very weak to moderate. The highest level of concordance across the two types o f measures is for diabetes, though moderate agreement on AIDS/HIV and COPD is also seen. Moderate agreement also exists between the Revised CDS and Elixhauser on thyroid disorders (.52) and hypertension (.47). In the case of AIDS/HIV, Deyo and the Revised CDS show better agreement (.66) than do the code-based measures (.50). Again, this difference can be attributed to the different methods used to screen for complications. For three conditions (rheumatological, peripheral vascular disease (PVD), and renal) there is scant agreement between the hospital and pharmacy-based measures on the presence of disease. In the case of peptic ulcer disease, there is only a chance level of agreement across the three measures. Overall, much more potential comorbidity is identified in the sample when using the pharmacy record. If prescription chug usage is strongly related to actual health status, then the Revised CDS indicates a sicker patient population that do diagnoses recorded in the hospital abstract. The two measures based on ICD-9-CM codes identify similar incidence of disease. Only in the case of diabetes (AIDS/HIV to a lesser extent) do the chug and diagnosis-based measures show much concordance in terms of individuals affected by disease. Much better agreement exists between the two diagnosis-based 84 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. measures, though differences in screening criteria and actual ICD-9-CM codes included in disease definitions result in low agreement for a few conditions. 8 5 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CHAPTER 4 COMPARISONS OF PREDICTIVE MODEL PERFORMANCE A series of logistic and OLS regression models were developed to assess the performance of the three measures of patient comorbidity in predicting future adverse events and LOS. The simplest of these models was an age/sex model that included only the categorical age variables and sex as predictors. The age/sex model has policy relevance and allows for comparisons of model performance with those published in the health services literature. The model was used to predict the three outcomes of interest. The comorbidity markers of the Deyo, Elixhauser and Revised CDS measures were then added, separately, to the age/sex model and the performance of each model in predicting complications of care, unplanned readmissions, and LOS was assessed. Tests were also performed to determine if the comorbidity measures significantly improved prediction beyond that achieved with age and sex alone. A similar series of models was then developed to answer the question, “Does the addition of drug class information from the patient pharmacy record improve the prediction o f outcomes after accounting for diagnosis-based measures of patient comorbidity?” In these analyses, the Revised CDS drug class markers were added to the Deyo and Elixhauser models (including age and sex) and any increase in predictive performance resulting from their inclusion was assessed for practical and statistical 86 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. significance. The entire set of analyses was then repeated with a full information model, which makes use of more of the information available in administrative data sets to better derive risk estimates. In a final set of analyses, the value of pharmacy data in predicting hospital outcomes was assessed in patients who had none of the comorbid conditions listed in the Deyo measure. In this group, substantial disagreement concerning patient health status appears to exist between the pharmacy and diagnosis-based measures of comorbidity and we were interested to see if drug-based chronic disease information was predictive in this instance. Age/Sex Model Predictive Performance The performance of the age/sex model in the prediction of in-hospital complications, unplanned readmission, and length of stay during the index hospital stay is presented in Table 4.1. Overall logistic regression results show that the predictor variables are more successful in classifying patients with future complications (C = .64) than unplanned readmissions (C = .57). Individual odds ratios show that older age was generally an important predictor of adverse events. Patients aged 76 or older were nearly four times as likely to have a complication (OR 3.76,p = .0001) and twice as likely to have an unplanned readmission (OR 1.99, p = .0003) as patients aged 45 and under. Advanced age also extended length of stay in the hospital, increasing it by 42% (p = .0001) for the oldest group. Female gender was associated with a 7% {p = .01) reduction 8 7 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. in average length of stay, holding age constant, but was unrelated to the occurrence of complications or unplanned readmission. Table 4.1 Prediction o f Hospital Outcomes Using Age and Sex Variables Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (N = 3558) Parameter AGE46 64 1.72* 1.45* 0.18*** AGE65 75 3 38*** 1.35 0.29*** AGE76+ 3.76*** j 99*** 0.42*** FEMALE 1.27 0.88 -0.07* Overall Model G2 = 57.0 (pc.0001); C= .64 G2 = 14.4 (p = .006); C = .57 F = 32.5 (p<0001); R2 = .04, Adj. R2 = .03 Note. G1 - Log-Likelihood Ratio Chi-Square. * p <.05. **/?<.01. ***p<.001. Predictive Performance o f Comorbidity Measures With Age/Sex Model The predictive performance of the three comorbidity measures was assessed by adding the disease markers from each to logistic and multiple regression models containing the age and sex variables. The Wald Chi-square statistic was used to determine if the addition of these variables resulted in statistically significant improvements in predictive ability. Diagnosis-Based Measures of Patient Comorbidity The results of the diagnosis-based models are presented in Tables 4.2 and 4.3. Addition of the Deyo comorbidity markers as a group (Table 4.2) resulted in improved prediction of readmission and LOS, but not complications. The C-statistic for 88 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. readmissions showed a 6-point increase (C = .63) over the age/sex model while both R2 and adjusted R2 increased by 3% (.7 and .6, respectively) for LOS. Table 4.2 Deyo + Age/Sex Model Prediction o f Hospital Outcomes Variables Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (logN) (N = 3558) Param eter AGE46 64 1.69* 1.22 0.15*** AGE65 75 3 20*** 1.04 0.25*** AGE76+ 3.55*** 1.62* 0.39*** FEMALE 1.30 0.94 -0.06* AMIYY1 0.90 1.43 -0.10 CANCYY1 1.26 1.22 0.27*** CHFYY1 1.83 1.35 0.03 COPDYY1 1.09 1.58* 0.14** CVDYY1 1.30 1.55 -0.16 DEMYY1 0.89 0.59 0.06 DIABYY1 1.07 2.00*** 0.09* DMCCYY1 1.30 1.14 0.13 HIVYY1 2.42 2.17 0.6*** MLDYY1 0.94 2.67 0.62*** MSLDYY1 0.98 2.68 -0.03 NEOYY1 1.15 2.34** 0.32*** PARAYY1 0.73 0.46 0.12 PUDYY1 0.67 a 0.13 PVDYY1 1.41 1.72 0.03 RAYY1 1.33 1.30 0.03 RENYY1 1.41 1.30 0.10 Overall Model G2 = 68.7 (pc.0001); C = .65 G2 = 72.2 (p<.0001); C = .63 F= 11.8 (p<.0001); R2 = .07, Adj. R2 = .06 A Model Wald f = 13.2 (p = .66) Wald x2 = 63.6 (p<.0001) F= 7.10 (pc.0001) Note. G2 = Log-Likelihood Ratio Chi-Square. “Variable removed to allow for model convergence. * p <.05. **p<.01. ***/?<.001. None of the Deyo diagnosis markers were statistically significant predictors of complications. A diabetes diagnosis doubled the risk of unscheduled readmission (OR 8 9 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. 2.00,p = .0001), neoplasms more than doubled the risk (OR 2.34,p = .0001), and COPD increased readmission risk by nearly 60% (OR 1.58, p - .014). These three diseases also contributed to significantly longer hospital stays, as much as 32% (p ~ .0001) longer for a neoplasm diagnosis. In addition, cancer, HIV, and mild to severe liver disease also contributed to longer LOS. All significant odds ratios and parameter estimates associated with comorbidities were positive in these models, indicating increased risk of adverse outcomes or longer LOS when these conditions are present. About one-fifth of the odds ratios for diseases were below 1.0, indicating reduced risk of outcomes, but none of these were statistically significant and many of these would be expected when testing large models. Of course, when testing coefficients in models with many predictors, there is always the possibility that some nonsignificant coefficients will be found to be significant by chance alone. Table 4.3 shows the multivariate results for models including age, sex and the Elixhauser measure. For all three outcomes, addition of the Elixhauser diagnosis markers as a group resulted in increased predictive performance. A 4-point increase in discrimination over the age/sex model was obtained for complications (C = .68) while a 10-point increase in explaining unplanned readmissions was obtained (C = .67). Improvements in explaining LOS were considerable, with a 9-percent increase in adjusted R2 (.12) when compared to the age/sex model. In each of the logistic regression models, three or more of the disease markers were statistically significant and each of these increased the probability of a complication or unplanned readmission. Two conditions, coagulopathy and HIV, were significant for 90 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. both outcomes, increasing the risk of complications (OR 2.4 ,p = .021 and OR 3.2, p = .002, respectively) and unplanned readmissions (OR 5.56, p = .034 and OR 3.98, p = .048, respectively) by similar amounts. In terms of chronic conditions, the largest odds ratio reported was an five-fold increase in probability of complication for patients with HIV (OR 5.56). Uncomplicated diabetes nearly doubled the risk of readmission (OR 1.97,/? = .0002) while complicated diabetes nearly doubled the risk of complication occurrence (OR 1.96,/? = .028). CHF also more than doubled the risk of readmission (OR 2.08,/? = .007). Twelve of the 31 diagnosis markers contributed significantly to explaining length of stay, and only valvular disease had a negative and significant association with the criterion (b = -.24,/? = .038). Fluid and electrolyte disorders, a significant predictor of readmission, was a strong predictor in the OLS model (b = .42,/? = .0001). Other diagnoses significant in the OLS model and at least one of the logistic models include blood loss anemia (b = .21,/? = .028), CHF (b = .34,/? = .0001), and both complicated diabetes (b = .16,/? = .007) and uncomplicated diabetes (b = .12,/? = .037). A diagnosis of cancer resulted in an average 50% (p = .0001) increase in hospital stays and three other chronic conditions, liver disease, neoplasms, and COPD also resulted in longer lengths of stay. Overall, the Elixhauser index contributed substantially to the prediction of the study outcomes, though some of the most consistent impacts across outcomes came from conditions that may represent in-hospital complications, such as coagulopathy, blood loss anemia, and congestive heart failure. 91 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.3 Elixhauser + Age/Sex Model Prediction o f Hospital Outcomes Variables Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (logN) (N = 3558) Parameter AGE46 64 1.60 1.30 0.13*** AGE65 75 2.75*** 1.03 0.17*** AGE76+ 2.74*** 1.31 0 .22*** FEMALE 1.30 0.89 -0.06* ALCX1 1.02 0.97 0.11 ARRX1 1.50 0.78 0.08 BLAX1 1.46 2.20* 0.21* CANCX1 1.24 1.18 0.5*** CHFX1 1.41 2.08** 0.34*** COAGX1 2.41* 3.21** 0.38*** COPX1 1.31 1.32 0.09* DAX1 a 2.20 0.22 DEPRX1 1.81 1.23 0.17 DFAX1 1.19 0.82 0.27*** DIABCX1 1.96* 1.13 0.16* DIABX1 0.96 1.97*** 0.12** FEDX1 1.20 2.03*** 0.42*** HIVX1 5.56* 3.98* -0.06 HTNCX1 a a 0.07 HTNX1 1.25 i.ii 0.01 LVRX1 0.85 1.38 0.33* LYMX1 a a 0.23 NEOX1 1.28 1.55 0.12* NEUX1 1.14 1.60 0.01 OBESX1 1.10 1.05 0.13 PARX1 1.37 1.53 0.17 PCDX1 1.73 1.25 0.14 PSYCX1 2.50 1.45 0.2 PVDX1 0.93 1.45 0.15 REN XI 1.32 0.95 0.03 RHEUX1 0.74 0.72 -0.09 THYX1 1.03 1.67 0.02 ULCRX1 1.19 3.42 0.27 VLVX1 0.38 1.85 -0.24* WGTX1 0.82 a 0.07 Overall Model G 2 = 94.9 (p = G2 = 101.2 (p<.0001); F= 15.3 (><.0001); .0001); C = .68 C = .67 R2 = .13, Adj. R2 = .12 A Model W _ . _ _ . -1. Wald y? = 43.0 (p = .042) Wald x2 = 93.6 O<.0001) F= 12.7 (j?<.0001) Note. G = Log-Likelihood Ratio Chi-Square. “ Variable removed to allow for model convergence. * p <.05. ** p<.01. ***p<.001 9 2 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Revised CDS from Pharmacy Data The results of multivariate models to predict study outcomes using the Revised CDS comorbidity markers and age and sex are presented in Table 4.4. The addition of the Revised CDS markers significantly improved model performance for all outcomes. Five and 6-point increases in discrimination over age and sex alone were attained in the complications (C = .69) and readmissions (C = .63) models. There was a one-percent increase in LOS variance explained (R2 = .05, Adj. R2 = .04). Odds ratios and parameter estimates associated with all significant comorbidity markers showed an increased risk of adverse events. Cardiovascular medications were associated with a 58% (p = .018) increase in risk of complications and a 92% (p = .0005) increase in unplanned readmissions. Diabetes medication was the only other significant predictor of readmission, increasing the probability of occurrence by 58% (p = .009). The renal medication marker was associated with a 5-fold increase in complications (OR 5 3 6 ,p = .049) while the organ transplantation marker was associated with a 449% (p = .012) increase in the risk of complications. O f the 28 Revised CDS markers, only HIV and COPD were significant predictors of LOS, associated with 39% (p = .02) and 8% (p = .03) increases in length of stay, respectively. In all cases, significant drug class markers increased the probability of adverse events or longer LOS. 93 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.4 Revised CDS + Age/Sex Model Prediction o f Hospital Outcomes Variables Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (logN) (N = 3558) Parameter AGE46 64 1.45 1.27 0.18*** AGE65 75 2 78*** 1.09 0.29*** AGE76+ 307*** 1.64* 0.44*** FEMALE 1.33 0.91 -0.07** TANX 1.09 1.00 -0.02 TCF 2.89 2.04 0.01 TCOPD 0.85 1.33 0.08* TCROHN 1.07 0.59 -0.24 TCVD 1.58* 1.92*** 0.05 TDEPR 1.24 0.97 0.04 TDIAB 0.87 1.58** 0.08 TEP1 1 . 1 1 0.77 0.08 TESRD a 0.53 -0.16 TGLAU 0.98 0.71 -0.08 TGOUT 0.88 1 . 2 0 0.06 THIV 3.05 3.08 0.39* THTN 0.93 0.77 -0.05 TLIP 1.53 0.81 -0.06 TLIVR 0.66 1.90 0.03 TMANC 1.62 1.23 0.00 TNEO 1.06 1.34 0.06 TNSAID 0.84 0.89 0.03 TPAfN 0.83 1.14 0.05 TPRK 1.82 1.79 0.04 TPSYC 1.31 0.53 0.07 TPVD 1.37 1.03 - 0 . 0 2 TRA 1.28 1.13 0.07 TREN 5.36* 1.45 0.45 TTB 1.14 0.85 - 0 . 0 2 TTHY 0.81 0.98 - 0 . 0 2 TTRNS 4.49* 1 . 0 1 0.32 TULCR 1 . 2 1 1.23 0.03 Overall Model G2= 102.0 (p = .001); C= .69 G2 = 64.9 (p<0001); C= .63 F=5.7(p<.0001); R2 = .05, Adj. R2 = .04 A Model Wald x2 = 50.9 (p = .004) Waldx2 = 53A (p = .003) F = 1.81 (p = .006) Note. G2 = Log-Likelihood Ratio Chi-Square. “Variable removed to allow for model convergence. * p <.05. **/?<.01. ***/K.001. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Predictive Performance o f Pharmacy-Based Comorbidity Controlling for Diagnosis- Based Comorbidity Table 4.5 details the predictive performance of the Revised CDS when added to models containing age, sex and either the Deyo or Elixhauser comorbidity measures. Only the odds ratios and parameter estimates for the Revised CDS comorbidity markers are displayed since the primary interest was in changes in explanatory power that may result from their inclusion. Only in the case of complications with Deyo and length of stay with Elixhauser does the addition of the pharmacy data as a whole significantly improve overall model performance. A 5-point increase in the C-statistic from .65 to .70 was seen with complications and increase in adjusted R2 of one-percent to .13 was seen with LOS. Medication prescriptions for cardiovascular disease increased the probability of a complication by 53% (p = .03) after controlling for Deyo comorbidity and 48% (p = .046) after controlling for Elixhauser comorbidity. In both diagnosis-based models, transplantation-related medications increased the risk of complications more than 5-fold. HIV and non-steroidal anti-inflammatory medication (NSAID) markers were the only two CDS variables positively and significantly associated with LOS. In predicting readmissions, the cardiovascular disease pharmacy marker was the only significant CDS variable, increasing the probability of readmission by more than 70% in both models. However, the addition of pharmacy markers did not significantly improve overall model fit beyond the age/sex + diagnosis-based comorbidity models. 95 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4 .5 Predictive Performance o f Revised C D S Controlling for Diagnosis-Based Comorbidity o JZ X U J V o Q £ o " O z ^ ■S ° I I b O '— - _ = z © c _ .2 C N _ v i r- o s .2 2 £ S e n w ii rt o Z g C /5 c ^ o o o 4 - * * i ~ > 0 3 u -i .2 c n o , I I i s O 03 B 5 G O T S I s * £ o z „ £ w o o rn 5 © I I c Z Q > ^ n J /-v O C N * « S 5 H : c # c n i i I I S § in £ o o o ■ a 10 e s u -i •H cn n. I I E Z o C Z f u £ o o > s •S * u 93 > t-** ^ (N c n o o h * © — © — © © m < N r- r- O O O < N O 0 0 0 * 0 0 0 0 0 0 * * v o • — « m ^ — « CN » — 1 O O O O 0 0 0 0 * 0 0 0 0 0 0 0 s o o o o o oocNsosooomovOscoOvmossn n vo oo ^ (S r- OvCNCNV^r^OOsnr^vOCNU^r^OOOOOOCNON — oo O (N ^ O ’— < — ~ ® © © — 4 0 0 © — 4 © 1 — 4 1 — . © OO c n c n * 0 0 © * 00 r f o <N — 4 p c4 © *-4 — m *-S © 1 — ^ V O o ^ o O s O s so O s © © <N © Nr^t^Ov^vomn w^sqoosooooo^ — ©~© © ©<N~ n n m — m'tnovvoi^vooo p — 4©<soooocnooo o o o o o o o o o o o o o lA^OOfN— '^(N^VO o o o o o o o o o *0 0 ( 0 0 0 0 0 * * o o o o n ^ r n < n < s m ONvoqvo^poooint^^o © r-J — © ^ ^ ©‘ o’ © 0 0 CO OO OS ^ r^c^mo>jppsosri — © pJ ^ © r- m - n v o ^ r ^ * vo * — i n vo Or^COCNU->(NVOCN„ © oo o O s sn — ^ c v i o ^ ^ — ^ o ^ o o ^ o ^ o ^ Tt n O v vo -n m r*' ^ \ o o s o o o o o o f S © © © 1— 4 *— 4 > < £ O Q z u, o o a > < u o u u h H H h h S§ § 3 = b S E S j O d d b b o o H H H H H H y o n o 5 S * > ? w j 5 < 2 m . ZZfcftiai h h P h H h 9 6 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. TPV D 1 .3 5 0.98 -0.03 1 .2 7 1 .0 5 -0.04 Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. Table 4.5 Predictive Performance o f Revised CDS Controlling for Diagnosis-Based Comorbidity (Continued) Variables Deyo Cl Elixhauser Cl Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (logN) (N = 3558) Parameter Complications (N = 3558) Odds Ratio Unplanned Readmission (N = 3472) Odds Ratio Length of Stay (logN) (N = 3558) Parameter TRA 1.27 1.06 0.05 1.28 1.15 0.04 TREN 4.55 1.49 0.42 3.69 1.12 0.08 TTB 1.07 0.97 -0.14 0.72 0.91 0.04 TTHY 0.83 0.95 -0.01 0.72 0.62 -0.05 TTRNS 5.34** 0.97 0.35 5.47** 0.96 0.25 TULCR 1.19 1.19 0.02 1.12 1.19 0.0 Overall Model G2= 112.3 G2 = 95.7 F= 5.6 (p<.0001); G2= 131.5 G2 = 132.9 F= 9.2 (p<.0001); C = .70 (p<.0001); R2 = .07, Adj. R2 (p<.0001); ip<.m\y, fp<.0001); C= .66 = .06 C= .7 1 C= .69 R = .14, Adj. R2 = .13 A Model Wald-f = 44.7 Waldy} = 23.7 F= .99 (p = .48) Waldx2 = 39.3 Wald x2 = 31.0 F= 1.50 (p = .004) (p = . 70) 1 1 O (p = . 32) (p = .046) Note. G2 = Log-Likelihood Ratio Chi-Square. "Variable removed to allow for model convergence. * p <.05. **p<.01. ***p<.001. v o -o Full Information Model Predictive Performance The analyses performed with the age/sex model and comorbidity measures just described were repeated with a full information model including age, sex, and the patient demographic and hospitalization variables reported in Chapter 4. These models provide estimates of the overall predictive performance that might be expected from models using comorbidity information based on administrative data. They also control for variables that have been shown to independently influence hospital outcomes. Additionally, they aid us in understanding the relative importance of information derived from administrative data in predicting adverse outcomes of hospitalization and LOS. In deciding whether to collect additional information to improve risk adjustment strategies, one seeks a measure that is unbiased, reliable, relatively free from gaming strategies, and which performs better in predictive models than alternative measures. The performance of the full information model is shown in Table 4.6. In all cases, the addition of the patient characteristic and hospital variables resulted in large improvements in model discrimination and calibration when compared to a model containing only age and sex. The C-statistic associated with complications increased by 9 points to .73 and a 12-point increase in the same was noted with readmissions (C = .69). R-square values related to the LOS model more than tripled (R2 = .13, R2 adjusted = .12). / 9 8 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.6 Prediction o f Hospital Outcomes Using Full Information model Variables Complications (N = 3531) O dds Ratio Unplanned Readmission (N = 3445) Odds Ratio Length of Stay (N = 3465) Param eter Demographic AGE46 64 2.00** 1.31 0.18*** AGE65 75 4 34*** 1.16 0.17*** AGE76+ 4.34*** 1.26 0.25*** FEMALE 1.21 0.94 -0.07* AFRO 0.92 0.98 0.01 HISPAN 0.67 0.72 -0.07 ASIAN 1.00 0.64 -0.06 RACEOTH 1.93 0.38 0.05 Hospital EMERG 2.64*** 1.48 0.55*** URGENT 2.12*** 1.65* 0.37*** D HOME a 0.42*** a MEDCARE 0.86 0.88 0.17*** MDC1 0.74 0.45** 0.17** MDC2 1.82 0.48 -0.60*** MDC3 0.91 0.50 -0.45*** MDC4 0.81 0.90 0.28**** MDC6 1.13 0.96 0.11* MDC7 0.42 0.94 0.08 MDC8 2.79*** 0.44** 0.08 MDC9 0.41 0.33* -0.20** MDC10 1.55 0.54 -0.06 MDC11 0.67 0.91 -0.14* MDC12 0.42 0.45 -0.03 MDC13 0.71 0.63 0.04 MDC16 7.39*** 0.82 0.19 MDC17 0.71 4.06** 0.87*** MDC18 0.75 1.10 0.35*** MDC21 12.23*** 0.55 -0.26 MDCOTHR 4.07* 0.51 0.70*** MDCSURG 1.54* 0.79 0.33*** O ther STATE 1.44* 0.81 0.05 KP 1.46 0.7* 0.13*** Overall M odel G2= 192.1 (p= .0001); G2 = 125.2 (p<.0001); F = 16.3 (p<.0001); C = .73 C= .69 R2 = .13, Adj. R2 = A2 A Model Wald x2 = 72.7 Wald = 60.] F = 12.6 (p = .0001) / n 2 _ r _ i m (p=. 0001) li © o o 'w ' Note. G = Log-Likelihood Ratio Chi-Square. “Variable not included in model. * p <.05. **p<.01. ***d<.001. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Individual odds ratios show that, as in the age and sex model, older age was an important predictor of complication occurrence and length of stay. Of note, the older age categories were no longer significant predictors of unplanned readmission. None of the ethnicity/racial variables were significant in the models and female gender was only significant in the LOS model, associated with shorter length of stay as in the age/sex model. Urgent admission status more than doubled the chance of a complication (OR 2.12 ,p = .0003) and increased the probability of readmission by 65% (p = .023), when compared to elective admissions. Emergency admission patients were more than twice as likely to have a complication than elective admission patients (OR 2.64, p = .0005), but emergency admission was not a significant predictor of readmission. Home discharge decreased the risk of unplanned readmission by more than half (OR .42, p = .0001). In the OLS model, emergency and urgent admission status resulted in 55% (p = .0001) and 37% (p = .0001) increases in LOS, respectively. Medicare insurance coverage was associated with a 17% (p = .0003) increase in length of stay when compared to Kaiser group coverage. Many of the Medical Diagnostic Category dummy variables showed significant associations with outcomes, some in a positive and some in a negative direction. Receiving KP-model pharmaceutical care was associated with a 13% (p = .0004) increase in the hospital stay and a 30% (OR = .70, p = .048) reduction in the probability of readmission, when compared to control model care. State model pharmacy patients had a 44% (p = .04) increased risk of complication. 100 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. There were many more significant predictors of LOS than complications and the fewest number of significant variables was seen in readmissions. The only consistent effect across all three outcomes in the full information models was seen in urgent admission status, which was positively associated with outcomes. The MDC control variables showed inconsistent patterns of association across the outcomes when contrasted with the referent category (MDC5, Disease and Disorders of the Circulatory System) which is not surprising given the complexity of the relationships that exist between diagnostic categories and other model variables. Addition of hospital and patient characteristics data improved both model discrimination and calibration substantially, and raised the predictive performance of the readmissions model closer to that of the complications model. Predictive Performance o f Comorbidity Measures with Full information model Diagnosis-Based Measures of Patient Comorbidity Addition of the Deyo comorbidity markers (Table 4.7) resulted in significantly improved prediction of readmission and LOS over the full information model. The C- statistic for readmissions increased by two points to .71 and a 2% increase in both R2 (.15) and adjusted R2 (.14) was seen for LOS. A one point increase in the C-statistic for complications (C = .73) was also noted, but this was not significant. A diabetes diagnosis 101 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.7 Deyo + Full Information Model Prediction of Hospital Outcomes Variables Complications (N = 3531) Odds Ratio Unplanned Readmission (N = 3445) Odds Ratio Length of Stay (logN) (N = 3531) P aram eter AM1YY1 1.00 1.45 -0.07 CANCYY1 1.48 1.14 0.29*** CHFYY1 1.66 1.17 0.02 COPDYY1 1.19 1.28 0.11* CVDYY1 1.28 1.56 -0.15 DEMYY1 1.02 0.40 0.00 D1ABYY1 1.14 1.78** 0.10* DMCCYY1 1.32 1.34 0.17 H1VYY1 1.19 2.38 0.35* MLDYY1 1.08 2.83 0.64*** MSLDYY1 a 2.60 0.00 NEOYY1 1.20 1.98* 0.27*** PARAYY1 0.75 0.39 0.05 PUDYY1 1.07 a 0.21 PVDYY1 1.62 1.61 -0.01 RAYY1 1.28 1.50 0.00 RENYY1 1.51 1.25 0.09 O verall Model G2= 198.8 (/X.0001); C = . 73 G2 = 163.4 (/K.OOOl); C= .7 1 F = 12.8 (p<.0001); R2 = A 5, Adj. R2 = .14 A Model Wald%2= 12.1 (p = .74) Wald x2 = 42.5 (p = .0003) F = 5.9 (p = .0001) Note. G1 = Log-Likelihood Ratio Chi-Square. "Variable removed to allow for model convergence. * p <.05. ** p<.01. ***p<.001. increased the risk of unscheduled readmission by 78% (p — .001) and increased LOS by 10% (p = .018) on average. A diagnosis of neoplasm nearly doubled the risk of readmission (OR = 1.98,p = .015) and resulted in 27% (p = .0001) longer hospital stays. Cancer, COPD, liver disease and HIV also significantly increased the LOS for patients with those secondary diagnoses. With minor exceptions, these results are very similar to those obtained using the age/sex model and the Deyo comorbidity measure. That is, R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. addition of important additional information concerning the patient and his/her hospitalization did not alter the pattern of relationships between specific comorbidities and outcomes. Table 4.8 shows the multivariate results for models including the baseline variables and the Elixhauser comorbidity markers. For all three outcomes, addition of the Elixhauser diagnosis markers as a group resulted in improved model predictive performance. A 2-point increase in discrimination over the full information model was obtained for complications (C = .74) while a 3-point increase over baseline for unplanned readmissions was obtained (C = .72). Improvements in explaining LOS were considerable, with an 7 percent increase in adjusted R2 (.19) over the full information model. In each o f the logistic regression models, five odds ratios were statistically significant and each of these indicate an increased probability of complication or unplanned readmission. Coagulopathy was significant in both logistic models and more than doubled the risk of complications (OR 2 .7 \,p = .022) and unplanned readmissions (OR 2.32, p = 035). In terms of chronic conditions, the largest odds ratio reported was an eight-fold increase in complications for patients with HIV (OR = 3.32, p = .012). Uncomplicated diabetes increased the risk of readmission by 173% (p = .044) while only complicated diabetes increased the risk of complication occurrence (OR 2.1 l,p = .012). CHF was responsible for nearly doubling the risk of readmission (OR 1.77, p = .046). 103 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.8 Elixhauser + Full information model Prediction o f Hospital Outcomes Variables Complications (N = 3531) Odds Ratio Unplanned Readmission (N = 3445) Odds Ratio Length of Stay (logN) (N = 3531) Parameter ALCX1 1.13 0.89 0.17 ARRX1 1.71* 0.82 0.05 BLAX1 1.31 2.10* 0.19* CANCX1 1.19 1.19 0.51*** CHFX1 1.42 1.77* 0.33*** COAGX1 2.71* 2.32* 0.29** COPX1 1.37 a 0.08 DAX1 a 1.14 0.26 DEPRX1 1.87 1.17 0.14 DFAX1 1.36 0.70 0.24*** DIABCX1 2.11* 1.13 0.17 DIABX1 1.09 1.73** 0.15*** FEDX1 1.38 1.72*** 0.39*** HIVX1 8.32* 3.63 0.08 HTNCX1 a a 0.04 HTNX1 1.33 1.07 0.01 LVRX1 0.77 1.79 0.37** LYM X1 a a 0.26 NEOX1 1.25 1.52 0.10 NEUX1 1.21 1.35 0.08 OBESX1 1.20 1.04 0.15* PARX1 0.88 1.29 0.12 PCDX1 2.80 1.09 0.04 PSYCX1 1.59 1.08 0.28* PVDX1 1.00 1.29 0.13 RENX1 1.54 0.93 0.02 RHEUX1 0.72 0.75 -0.10 THYX1 0.98 1.63 0.0 ULCRX1 1.37 2.75 0.33 VLVX1 0.46 2.30 -0.27* W GTX1 0.75 a 0.04 Overall Model G2 = 235.2 (p<.0001); C= .74 G2 = 173.9 <><.0001); C= .72 F= 14.5 (p<.0001); R2 = .21, Adj. R2 = .19 A Model Wald x2 = 47.4 (p = .012) Waldx2 = 53.5 (p = .002) F= 5.9(p= .004) N ote. G2 = Log-Likelihood Ratio Chi-Square. ‘V ariable removed to allow for model convergence. * p <.05. **p<.01. ***p<.001. 1 0 4 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Eleven of the 31 diagnosis markers contributed significantly to explaining length of stay, and only valvular disease showed an inverse association with the criterion (J b = - .27, p = .016). Most of the significant comorbidities were the same as those noted in the age/sex analysis, with minor variations. Revised CDS from Pharmacy Data The results of multivariate models combining the Revised CDS comorbidity markers and baseline variables are presented in Table 4.9. The addition of the Revised CDS markers significantly improved overall performance in only the complications model, with the C-statistic rising 4 points to .76. The increase in discriminatory ability related to readmissions (C-statistic increased 3 points to .71) was not significant (Wald Chi-square = .09). Odds ratios associated with all significant drug therapy classes in the table show an increased risk of adverse events. Cardiovascular medication increased risk of complications by 76% (p = .006) and readmission by 63% (p = .012). Diabetes was the only other significant predictor of readmission, increasing the probability of occurrence by 50% (p = .029). The renal medication marker was associated with a nearly nine-fold increase in complications (OR 8.89, p = .018) while the transplantation marker was associated with a near quadrupling of risk for complications (OR 3.84, p = .039). 105 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.9 Revised CDS + Full information model Prediction o f Hospital Outcomes Variables Complications (N = 3531) Odds Ratio Unplanned Readmission (N = 3445) Odds Ratio Length of Stay (logN) (N = 3531) Parameter TANX 1.03 0.98 -0.02 TCF 4.01 2.00 0.13 TCOPD 0.86 1.28 0.03 TCROHN 1.02 0.49 -0.20 TCVD 1.76** 1.63* 0.04 TDEPR 1.14 0.88 0.02 TDIAB 0.95 1.50* 0.11* TEPI 1.30 0.76 0.03 TESRD a 0.29 -0.29 TGLAU 0.89 0.65 -0.10 TGOUT 0.83 1.01 0.09 THIV 1.84 3.26 0.18 THTN 0.95 0.81 -0.04 TLIP 1.63 0.86 -0.05 TLIVR 0.62 2.15 0.06 TMANC 2.25 1.33 0.14 TNEO 1.01 1.24 0.03 TNSAID 0.74 0.90 0.02 TP AIN 0.68* 1.19 0.07* TPRK 1.50 1.95 0.05 TPSYC 1.13 0.45 0.0 TPVD 1.32 1.08 -0.03 TRA 1.33 1.18 0.06 TREN 8.89* 1.70 0.39 TTB 1.30 0.82 -0.02 TTHY 0.82 1.01 0.0 TTRNS 3.84* 1.01 0.31 TULCR 1.22 1.23 0.01 Overall Model G2 = 236.7 (p<.0001); G2 = 162.9 (p<.0001); F=9.2(p<.0001); C = . 76 C = .71 R2 = .14, Adj. R2 = .13 A Model Waldy? = 5\.Z(p = .003) iValdx2 = 38.3 (p = .09) F= 1.3 (p = .13) Note. G2 = Log-Likelihood Ratio Chi-Square. “Variable removed to allow for model convergence. * p <.05. **p<.01. ***p<.001. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The only drug class associated with a lowered risk of complications was pain medication (OR .68,/? = .027). The pain marker was also a significant predictor of LOS, however, it was associated with a 6% (p = .02) increase in LOS. Diabetes was associated with an 11% (p = .011) increase in average LOS. In the logistic analyses, most of the significant comorbidities were the same as those noted in the age/sex models when the CDS was included, however in the LOS analysis entirely different coefficients were significant. Predictive Performance o f Pharmacy-Based Comorbidity Controlling for Diagnosis- Based Comorbidity Table 4.10 details the predictive performance of the Revised CDS when added to models consisting of baseline variables and the diagnosis-based comorbidity measures. Because the ratio of positive responses on the dependent variable to the number of predictor variables was too small to make analyses reliable, the Deyo and Elixhauser measures were entered as empirically derived indices in these analyses, conserving degrees of freedom. Only for complications does the addition of the pharmacy data significantly improve overall model performance. When combined with the Deyo index, the Revised CDS markers contributed to a rise in the C-statistic from .73 in the full information model to .76 (p = .007). When combined with the Elixhauser comorbidity index, the C-statistic rose from .74 to .77 (p = .042). Medications indicated for cardiovascular disease increased the probability of a complication by 75% (p = .007) after controlling for the Deyo index and 51 % ( p - .036) 107 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. after controlling for the Elixhauser index. Prescriptions for transplantation-related medications increased the probability of complication nearly five-fold in both the Deyo model (OR 4.51, p - .025) and the Elixhauser model (OR 4.77,/? = .021). Pain-related medications reduced the risk of complications by more than 30% in both models. Renal medication was associated with more than a seven-fold increase in the probability of complication, but only with the Deyo measure. In predicting readmissions, the cardiovascular disease marker was the only significant CDS variable, increasing the chance of readmission by more than 50% after controlling for either Deyo (OR 1.51, p = .036) or Elixhauser (OR 1.55,/? = .014) comorbidity. None of the CDS markers were significant predictors of LOS after controlling for the diagnosis-based comorbidity measures. The odds ratios shown in Table 4.10 exhibit patterns similar to those seen with the age/sex model in Table 4.5. The impact of cardiovascular medications on adverse events, in particular, was quite consistent across analyses. Pharmacy data, when combined with the Elixhauser model to predict LOS, made a significant contribution in the age/sex model but was no longer significant when additional patient information was included. Conversely, the Revised CDS as a whole made a significant contribution to explaining complications in this analysis when combined with either diagnosis-based measure but was not significant in the age/sex model when combined with the Elixhauser measure to predict complications. 108 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. T able 4 .1 0 Predictive Performance o f Revised C D S Controlling for Diagnosis-Based Comorbidity o W o 0 01 Q C O in ■s&S t d ! g s ► J o c C eg cx c D ^ o o n *«S ^ C S cn ^ i i M I I ' ■ 5 5 S § o ’ 5 ^ ^ s S s 1 - j » i s u lr 00 ^ ^ ‘s S s t j 1 1 = ~ s o > w “ O 0 c in .2 c 'tfl « c e g C / 3 E cn P S 0 3 a. •0 I I ■ © c e g a > 0 * S O C / 3 5 = .2 cn eg m 0 cn "EL I I E 0 s O A « -fi s *c es > <N cn — ® © © cn ^ — © O O CN cn cn i n r - r ' - r ^ — c n w n o v n ^ ^ 0 0 —0 0 0 0 —0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 * oei^vcMr)ino\^^fnooi^oooe'-o On — C N ^ i n 0 N © V 0 c n V © O ^ r * ^ 0 0 © © © — © © © — C N O © < N — O (N — O — cn CN vo t*- Os C N O N — 00 cn — © — — © cn o O V O 00 On r - C N — no © in cn * © — cn o — — — * r- — ©oecNcninr-^©^© ooooooo'OvoviaM^'O^ © O —* © ~ © ' — © O O — — cn — o o 0 0 0 — p"g, cnts*g, cnO\ooov 0 — 0 0 0 0 ^ 0 0 0 * 0 0 0 0 0 0 0 0 0 «n v© no cn cn in 0 0 0 - 0 0 0 0 0 0 0 0 0 0 0 0 0 0 On C N O N O N © — — r-* — rj* vo m in on o ' — : O — 00 vo r t cn 00 cs vo ov g- 0 0 0 0 ^ in cn cn — vo p-* 00 oooovqm-ov-^g; m o o — — — o — — o * * — t j - — r ^ - i n o o — O n © O n o o © p- * —< r - ; c n , — c n o — — — o — oocNOvincNor^-cninvo — — oooomCNNqvo — ovr^vovncN © © C N © — © C N © © © — — X S o Q ^ ^ S ^ s g . g | o < £ * £ Z f c O « > « S E M J O g H £ : i S w w < 2 M < O O U O Q Q w w O O S S J j 2 Z Z ( i , a l B. E—1 H H t - ' E - ' H H E — ' H F - H f - t — 1 ' H f r - H H H H H H 1 0 9 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. TPV D 1 .2 7 1 .0 6 -0.02 1 .1 9 1 .1 4 -0.04 Reproduced w ith permission o f th e copyright owner. Further reproduction prohibited without permission. Table 4.10 Predictive Performance o f Revised CDS Controlling for Diagnosis-Based Comorbidity (Continued) Deyo Cl Elixhauser Cl Variables Complications (N = 3531) Odds Ratio Unplanned Readmission (N = 3445) Odds Ratio Length of Stay (LogN) (N = 3535) Parameter Complications (LogN) (N = 3531) Odds Ratio Unplanned Readmission (N = 3449) Odds Ratio Length of Stay (LogN) (N = 3535) Parameter TRA 1.30 1.15 0.05 1.31 1.24 0.05 TREN 7.72* 1.76 0.37 5.79 1.53 0.07 TTB 1.25 0.96 -0.09 0.93 0.81 0.02 TTHY 0.82 0.97 0.0 0.72 0.63 -0.03 TTRNS 4.51* 0.88 0.33 4.77* 0.95 0.23 TULCR 1.26 1.19 0.01 1.26 1.20 0.0 Overall Model G2 = 249.9 G2 = 184.6 F=SA Q rc.O O O l); G2 = 272.5 G2 = 201.0 F= 10.3 (p<.0001); C= .76 [p<.0001); C=. 73 R2 = . 16, Adj. R2 = .14 (p<.0001); C=. 77 (/K.0001); C= .74 UX.0001); R = .21, Adj. R2 = .19 A Model Waldy? = 48.4 (p = .007) Wald i 2 = 20.8 Ip - .84) F= .83 (p = .73) Waldf) = 40.9 {p = .042) Wald x1 = 26 A ip = . 55) F=M(p = .65) N ote. G 2”Log-Likelihood R atio C hi-Square. "V ariable rem oved to allow for m odel convergence. * p <.05. * * p<.01. ***p<.001. o Predictive Performance o f Revised CDS with Selected Patients A full 65% of study patients had a Deyo comorbidity score of zero. That is, they had no secondary diagnosis recorded in the hospital abstract that constituted part of the Deyo measure. For the Elixhauser measure, the corresponding figure was 41%, and for the Revised CDS, the figure was only 12%. Since pharmacy data is a source of information entirely independent of the hospital abstract, it should be especially useful in identifying illness in patients with little or no diagnosis-based comorbidity. To test its utility in such situations, a separate set of three models was run with patients who had none of the 17 Deyo comorbidities recorded in their hospital abstract. This test could not be replicated with the Elixhauser measure because of insufficient sample size. Results of the logistic and OLS regression models with patient age and sex are detailed in Table 4.11. Only the complications and LOS models were statistically significant overall. The C-statistic obtained with complications (.72) is a 7-point increase over that obtained with a simple age/sex model (C = .65) in this sample, though that increase was not statistically significant (Wald yj = 35.5,p = .10). A five-point increase in the C-statistic (C = .62) regarding unplanned readmissions was also realized by including the CDS variables with these patients, but it was not significant (Wald % 2 = 23.4, p = .44). In the LOS model, the adjusted coefficient of determination remained at .03 despite the addition of the CDS markers. I ll R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.11 Predictive Performance o f Revised CDSfor Patients With No Diagnosis- Based Comorbidity Variables Complications (N = 2306) Odds Ratio Unplanned Readmission (N = 2280) Odds Ratio Length o f Stay (logN) (N = 2306) P aram eter AGE46 64 1.63 1.24 0.15*** AGE65 75 2.89*** 0.91 0.27*** AGE76+ 3 79*** 1.52 0.43*** FEMALE 1.11 0.99 -0.04 TANX 0.95 1.02 -0.04 TCF 13.31* 9.79 0.34 TCOPD 0.59 1.05 0.05 TCROHN 1.17 a -0.01 TCVD 1.55 1.86* 0.06 TDEPR 1.53 0.94 0.09 TDIAB 0.68 0.47 0.04 TEPI 1.27 0.24 0.19 TESRD a a -0.67 TGLAU 0.87 0.78 -0.07 TGOUT 0.60 1.7 0.08 THIV 4.19 3.27 0.12 THTN 0.89 0.82 -0.05 TLIP 1.97* 0.84 -0.06 TLIVR 1.87 a 0.09 TMANC 0.84 a -0.06 TNEO 1.16 1.00 0.03 TNSAID 0.79 0.92 0.01 TP AIN 0.58* 1.15 0.01 TPRK 2.27 6.8* -0.06 TPSYC 1.89 0.55 0.08 TPVD 0.65 1.05 -0.03 TRA 1.70 1.12 0.14* TREN a a 1.31* TTB 1.07 1.16 0.07 TTHY 1.03 1.10 -0.04 TTRNS 1.92 1.82 0.23 TULCR 1.29 1.24 0.03 Overall Model G2 = 70.9 (pK.0001); C G2 = 27.7 (p = .45); F = 3.6 OX.0001); = .72 C = .62 R2 = .05, Adj. R2 = .03 A Model Wald x2 = 35.5 (p =. 10) Wald % 2 = 23.4 (p = .44) F = 1.2 (p = .21) Note. G2 = Log-Likelihood Ratio Chi-Square. “ Variable removed to allow for model convergence. * p <.05. **p<.01. ***p<.001. R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. As in the full information model with the entire sample, pain medication was associated with a reduction in the probability of complication (OR .58, p = .028). Two markers not significant in previous models, cystic fibrosis and lipid medications, increased the probability of complications by approximately 13 and 2 times, respectively. Cardiovascular disease, a significant factor for readmission in previous models, also increased the risk of unplanned readmission in this group of patients (OR 1.86 ,p = .028), though the readmission model itself was not significant. Two pharmacy variables not significant in predicting LOS in earlier models, renal and rheumatological conditions, both increased hospital stays, with renal medication increasing average LOS by a full 31%. While some of the odds ratios are fairly large in these models, a closer inspection of results showed that the largest are based on small numbers of individuals affected by these diseases. Contingency table analysis of the dependent measures with the CDS binary markers also revealed problems with individual cell sample sizes. For 11 of the 28 CDS markers there were fewer than three patients who were both positive for that condition and an adverse hospital event. This appeared to violate the conditions under which chi- square approximations for goodness of fit tests are appropriate. This might also explain why substantial increases in the C-statistic for some models were not significant according to the Wald Chi-square statistic. The models for adverse events were then run again after removing all CDS markers for which the count in any cell was less than 3. This resulted in more than 80% 113 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. of the cells in the model having counts of 5 or more (a commonly applied rule of thumb in logistic regression modeling). Table 4.12 Predictive Performance o f Revised CDS for Patients With No Diagnosis- Based Comorbidity (Revised Model) Variables Complications (N = 2306) Odds Ratio Unplanned Readmission (N=2280) Odds Ratio AGE46 64 1.58 1.23 AGE65 75 2.81*** 0.92 AGE76+ 3.66*** 1.51 FEMALE 1.10 0.98 TANX 0.95 1.00 TCOPD 0.60 1.04 TCVD 1.53 1.81* TDEPR 1.52 0.88 TDIAB 0.69 0.45 TEPI 1.24 a TGLAU 0.95 0.85 TGOTJT a 1.66 THTN 0.94 0.84 TL1P 1.97* 0.85 TNEO 1.16 0.99 TNSAID 0.81 0.93 TPAIN 0.59* 1.16 TPRK a 6.60* TPSYC 2.27 0.51 TPVD 0.64 1.06 TRA 1.74* 1.14 TTHY 1.03 1.09 TULCR 1.30 1.22 Overall Model G2 = 65.0 OK.0001); C=.71 G2 = 20.3 (p = .56); C = .6 1 A Model Wald % 2 = 28.5 (p = .04) Wald x2 = 16.6 (p = .55) Note. G2 = Log-Likelihood Ratio Chi-Square. ‘Variable not included because of small cell sample size. * p <.05. **p<.01. ***p<.001. The results of the revised analysis are shown in Table 4.12. Despite the removal of 11 markers in the complications model, measures of overall fit did not suffer greatly. 1 1 4 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. The C-statistic dropped only .01 to .71 and the nested CDS model became significant (p = .039). Individual coefficients in the model changed only slightly, though the marker for rheumatological conditions was now significant (OR 1.74,/? = .044). The removal of 10 chronic disease markers from the unplanned readmissions model also resulted in slight degradation in model fit. However, in this instance chi- square values associated with the nested model did not improve. Covariate-level coefficients, odds ratios, and /^-values were largely unaffected. Summary The addition of comorbidity information to simple models based on age and sex resulted in significantly improved predictive performance for all three outcomes in most instances (See Table 4.13). The largest increases occurred with the Elixhauser measure in predicting unplanned readmissions (C-statistic from .57 to .67) and LOS (Adj. R2 from .03 to .12). The Elixhauser measure contributed to a .04 increase in the C-statistic for complications (.64 to .68), the CDS improved the prediction of complications by .05 (.64 to .69) and the small increase seen with Deyo (.01) was not significant. Both the Deyo and Revised CDS showed identical increases in discrimination in predicting readmission (.06) and the Deyo improved prediction of LOS (A Adj. R2 = .03) relative to the CDS (A Adj. R2 = .01). 115 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Table 4.13 Increases in Overall Model Fit From Addition o f Comorbidity Measures Models Complications Unplanned Readmissions Length of Stay C AC P C AC P A dj.# A R2 P Age/Sex + Deyo .65 .01 .66 .63 .06 .0001 .06 .03 .0001 Elixhauser .68 .04 .042 .67 .10 .0001 .12 .09 .0001 CDS .69 .05 .004 .63 .06 .003 .04 .01 .006 Deyo + CDS .70 .05 .004 .66 .03 .70 .06 .00 .48 Elix. + CDS .71 .03 .07 .69 .02 .32 .13 .01 .045 Full Infonnation + Deyo .73 .00 .74 .71 .02 .0003 .14 .02 .0001 Elixhauser .74 .01 .01 .72 .03 .002 .19 .07 .004 CDS .76 .03 .003 .71 .02 .09 .13 .01 .13 Deyo + CDS .76 .03 .007 .73 .02 .83 .14 .00 .73 Elix. + CDS .77 .02 .04 .74 .02 .55 .19 .00 .65 When the comorbidity measures were added to the full information models, improvements in fit were generally much smaller. A likely explanation for this is that some severity of illness information was captured by hospitalization characteristics (e.g., emergency admission status) and major diagnostic categories. As with the age/sex models, the largest increases in explanatory power for readmission and LOS were achieved with the Elixhauser measure. Significant, but somewhat smaller increases in model fit for readmissions and LOS (.02) were also seen with the Deyo measure. The Revised CDS provided the best discrimination in terms of complications (C = .76) but was not a significant predictor of unplanned readmissions or LOS (despite improvements in overall model fit). In the full information models, the Elixhauser measure was clearly the best overall predictor. This may result because it includes the most conditions, is the least 116 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. restrictive measure, or a combination of both. Less restrictive measures include acute conditions that are actually in-hospital complications and as such provide better, though biased, prediction of hospital outcomes. The pharmacy record was the most restrictive measure since it only contained information prior to admission. When restricting the comparison of predictive performance to the Revised CDS and the Deyo measure, neither appears to have a clear-cut advantage since each failed to significantly improve predictive performance for at least one adverse outcome. Addition of the CDS markers as a group did not result in significantly improved prediction of readmissions or LOS (except in the age/sex models), but did contribute to the largest increase in model discrimination for complications, with the C-statistic rising from .73 to .76 when included with Deyo and rising from to .74 to .77 when combined with Elixhauser. This identical increase in discrimination is noteworthy, since the Elixhauser measure appears to include some information on complications. In the subanalysis, where the CDS was used to help predict future hospital outcomes for patients with no significant diagnosis-based comorbidity, increases in model fit over the age/sex model for the same group were noted for all three outcomes, but this increase was not significant for the pharmacy measure as a whole in initial models. In fact, the readmissions model failed the overall chi-square test of significance. However, at least two comorbidity coefficients were significant in all models and contributed to relatively large overall increases in model performance. Moreover, when CDS markers with low cell frequencies were removed from the analysis, the improved discrimination in complications due to the pharmacy measure was found significant. The 117 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. subanalysis appears to have provided somewhat less reliable results because it was based on a smaller sample and the prevalence of disease markers was quite low (<5) for several conditions. 118 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. CHAPTER 5 DISCUSSION This study provides limited support to the hypothesis that the addition of pharmacy-based information to diagnosis-based measures of patient comorbidity will result in improved prediction of hospital outcomes. Consistent evidence for this hypothesis was only found in the case of in-hospital complications. Evidence o f the predictive validity of the Revised CDS for three hospital outcomes in multivariate models adjusted for age and sex was also demonstrated. As a predictor of adverse hospital events in more complex models, the CDS performed comparably to the Deyo measure. The Elixhauser measure performed the best in these comparisons, though it appears to include postadmission information, which invalidates its use as a risk-adjuster for quality assessment. The analyses broke new ground in demonstrating the performance of a pharmacy-based risk-adjuster in the inpatient setting, comparing its performance to one of the most widely used measures of patient comorbidity in health services research today. The study results do not support an immediate change in local, state or federal policy to mandate the reporting of patient pharmacy data with hospital administrative data. Rather, the results demonstrate the need for continued exploration of the predictive value o f pharmacy records with public data sources. An application with high potential 119 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. benefits would be the development of study-specific measures of pharmacy-based chronic illness within a narrow range of primary diagnoses or procedures. Comparisons of Overall Model Performance The performance of diagnosis-based comorbidity measures in this study compares favorably with results recently published in the health services literature. Comparisons with the Revised CDS are limited, however, since this is the first study in which a pharmacy-based measure was used to risk-adjust for hospital outcomes. Iezzoni and colleagues (1994) noted a C-statistic of .66 in predicting complications in six medical/surgical populations using California administrative data. Their model included considerable patient-level information and 13 diagnosis-based dummy indicators of prognostic comorbidity. The Deyo full information model of this study, similar in many regards, yielded a C-statistic of .73. In two other studies that included limited patient and sociodemographic information to predict complications in surgical patients (Ghali et al., 1998; Kieczak et al., 1999), C-statistics of only .60 were reported. Both of these studies included a code-based version of the Deyo measure, which was only significant in the first study. Three studies using hospital abstract data to risk-adjust for readmissions in hospitalized patients achieved lower levels of discrimination than the present study. Philbin and DiSalvo’s (1999) model to predict readmissions (within the calendar year) for heart failure after CHF in New York state patients showed only modest discrimination (C = .62). The Deyo full information model of this study, which includes patient 120 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. demographic and hospital information similar to their model, had a C-statistic of .71. In a Spanish study employing a simple age/sex model and an extended version of the Charlson Cl (Librero et al., 1999), readmission within one year of discharge was predicted with even less accuracy (C = .58). In another study (Thomas, 1996), the average C value was less than .60 in predicting 30-day readmissions for Medicare patient across twelve primary conditions. That study utilized a count of comorbid conditions to control for case-mix. The literature documents varying success in efforts to model LOS using hospital administrative data. Melfi and colleagues (1995) explained 17% of LOS variation in one- quarter million Medicare beneficiaries hospitalized for total knee replacement surgery. They used models that included patient demographic and sociodemographic characteristics as well as the Deyo CI. The partial R2 attributed to the Deyo measure was one-half of one percent (.005). It is interesting to note that a clinically derived Deyo measure showed the same partial R2 (.005) in predicting LOS for patients admitted with chest pain, (Matsui et al, 1996), though the overall R2 achieved was .51 in that study. The most comparable model in this study (Deyo full information) achieved an adjusted R2 of .14. Two-percent of the explained variation could be attributed to the Deyo measure. In a study involving endarterectomy patients (Kieczak et al., 1999), a simple demographic model with the Deyo CI achieved an R2 of .036 in predicting LOS (dichotomized), which is somewhat lower than the R2 of .06 achieved in this study with a comparable model. In that study, a claims-based version of the Deyo CI was not a significant predictor but a chart-based index was. 121 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Pharmacy-based measures o f risk have most frequently been employed to predict outpatient or total medical costs. Hospital LOS, often considered a proxy for hospital charges, has not been employed, but studies that have utilized total costs as the dependent measure provide an indication of expected performance. Four studies have used some variant of the CDS to predict future medical expenses and most of these used simple demographic models with the measure. Data for all of these studies came from managed care organizations or a single health insurance plan. The overall R2 achieved in these studies were .024 (Johnson, Holbrook & Nichol, 1994), .061 (Fishman et al., 1999), .10 (Clark et al., 1995), and .109 (Lamers, 1999). Partial R2 values, provided in the last three studies, were .06, .07, and .053, respectively. The R2 achieved for LOS in this study using the age/sex + Revised CDS (adjusted R2 = .04) was lower by comparison, as was the partial R2 of only 1% for the CDS. It may be more difficult to explain in-hospital costs (LOS as a proxy) using pharmacy data than to explain total medical costs, given that prescription fills are directly related to total drug costs and should have some relationship to outpatient visits and their cost. The three measures of comorbid illness examined in this study often provided impressive boosts in predictive performance of hospital outcomes when added to simple age/sex models. When combined with the more complete information, the increases were more modest, though still noteworthy in certain instances. Based on recent claims-based research in the literature, we would expect only modest increases in predictive power resulting from the addition of comorbid illness information to baseline predictive models. The diagnosis based measures generally performed better than expected in this regard. 122 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Pharmacy-based chronic disease explained somewhat less variance in LOS than was expected, though comparisons with the literature are limited to total cost studies. Demographic and Hospitalization Characteristics Related to Outcomes Of the demographic variables included in our models, only advanced age was consistently and strongly related to two study outcomes, complications and LOS. The significant relationship between age and unplanned readmissions seen in the simple age/sex model was no longer present in the full information models. Most, but not all previous studies have found a significant relationship between advanced age and early readmission though the strength of this relationship was often trivial (Soecken et al., 1991). Race was never a significant factor in models and female sex was significantly associated with reduced LOS in only some models. Hospitalization characteristics exhibited inconsistent effects across the three outcomes, with perhaps the most consistent patterns seen for admission status. The pharmacy model variables, meant to control for patient exposure to different models of pharmacy practice, showed sometimes significant and unexpected relationships with the outcomes, given results from previous studies that used these intervention variables. These inconsistent relationships may be explained by case mix differences across the geographically based pharmacy models, differences that were inadequately controlled for in this study. 123 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Specific Medical Conditions Related to Outcomes In most models, individual markers for disease were significant predictors of only one or two of the three outcomes. With few exceptions, significant markers were associated with an increased probability of adverse events or longer LOS. The following remarks focus on chronic conditions that have been shown to be prognostically important predictors of hospital outcomes in the clinical and health service research literatures. Acute conditions that could represent hospital complications are not discussed. In the age/sex models, diagnosis markers that were positively associated with one of the outcomes using the Deyo measure often showed the same pattern of association with the Elixhauser measure. This was true for neoplasms, cancer, COPD, diabetes, HIV, and liver disease. Of these, the corresponding COPD, diabetes, and HIV markers of the CDS demonstrated similar relationships. The condition most consistently associated with an increase in adverse events and longer LOS in the study was diabetes. It was a consistent predictor of all three outcomes when the Elixhauser measure was used. HIV was associated with relatively large effects in many models. In Chapter 1, it was argued that pharmacy-based data might ameliorate some of the problems encountered in diagnosis-based assessments of comorbidity, specifically the documented underreporting of chronic disease in the hospital abstract. Chapter 3, Table 3.7 presented the prevalences of disease indicated by diagnosis and the suggested prevalence of disease indicated by Rx-based comorbid disease markers. The sample appeared to suffer more from comorbid disease than the hospital abstract indicated when medications were taken as an indicator of disease. The rates of disease based on 124 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. medication usage are not dissimilar to those published in the clinical literature and one study has corroborated pharmacy-based and epidemiological estimates of disease prevalence (Lamers, 1999). Among the conditions where estimates varied the greatest were prognostically important conditions such as cardiopulmonary disease, COPD, hypertension, neoplastic conditions and PVD. Of these, only the COPD and cardiovascular markers were significantly associated with outcomes in the age/sex + CDS models. In the full information models (Table 4.9 and 4.10), only cardiovascular disease (CVD) remained significant. This condition significantly increased the probability of both complications and readmissions. These results suggest that receipt of medications for many prognostically important conditions may not contribute to an increased risk for adverse events. It also suggests that many of the reported deficiencies of diagnosis-based comorbidity measures will not be resolved through the addition of pharmaceutical information. Nonetheless, it was consistently demonstrated that medications associated with cardiovascular disease increased the risk of in-hospital complications and unplanned readmissions after controlling for diagnosis-based comorbidity. Policy Implications Pharmacy claims data have not previously been explored as risk adjusters for quality of care assessments. Most explorations have focused on the utility of these data in predicting future costs within managed care organizations. In terms of public policy, there is limited recent evidence to support their use in health status assessments for risk- 125 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. based and risk-pooling reimbursement schemes (Lamers, 1999). However, this study sought to employ these data in a novel way to address problems with current diagnosis- based strategies of risk adjustment in hospital performance profiling. It was hypothesized that properties of pharmacy data such as uniformity, completeness, accuracy, ready availability, and relative invulnerability to gaming strategies would make them a valuable addition to risk assessment methods for hospital patients. The results of this study support the validity of the Revised CDS as a risk-adjuster for three important hospital outcomes in predictive models including age and sex. Results also demonstrate the validity of the Revised CDS in predicting in-hospital complications in models that include most demographic and hospitalization information available from the hospital abstract. The results of this study are relevant to two possible policy proposals. The first is the use of pharmacy-based comorbidity as a substitute for diagnosis-based comorbidity in controlling for hospital case-mix in risk-adjusted outcomes assessment. The second is the use of pharmacy data as an addition to diagnosis-based assessment of comorbidity in models to predict adverse hospital events. Can a pharmacy-based assessment of risk by itself substitute for or improve on current diagnosis-based methods of risk adjustment in the inpatient setting? This question is best posed in terms of a comparison between the Deyo CI and the Revised CDS. We have much more experience with the Deyo measure in that role and the Elixhauser measure has not yet been investigated sufficiently to rule out problems of post-admission complications contaminating risk estimates. Based on this study’s results, 126 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. neither the Deyo nor the Revised CDS has a clear-cut advantage in predicting adverse hospital outcomes. Clearly, the Deyo measure suffers greatly from its inability to improve the prediction of complications in any of the models tested. In the age/sex models, the Revised CDS does improve prediction of both complications and unplanned readmission, but this explanatory power is reduced to non-significance ip = .09) in the full information models (See Table 4.9) when unplanned readmission is the dependent variable. A similar situation occurs in the prediction of LOS, where the Revised CDS is a significant predictor in age/sex models but not in the full information models. Also, in the age/sex models, the partial R2 attributed to the CDS is one third of that attributed to the Deyo measure (.01 vs. .03). Of course, LOS is not intended to be a proxy for quality, but better capture of comorbidity should result in better explanation of LOS; a contention well supported in the literature. Regarding the first policy question, there is no clear criteria established for judging these results. In terms of outcome measures, neither rates of complication nor readmission appear to be more reliable or valid proxies for hospital quality. The fact that neither measure is a significant predictor of both outcomes in the full information models may speak for deficiencies in either the outcomes chosen, the comorbidity measures, or both. At this stage in the development of claims-based quality indicators, it is not possible to answer such questions with confidence. The second policy issue relates to the central hypothesis of this investigation: Was pharmacy data able to explain variation in hospital outcomes beyond that accounted for by ICD-9 diagnoses from the hospital abstract? This study provides only partial 127 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. support for that hypothesis. The Revised CDS significantly improved the discrimination of complications models containing the Deyo or Elixhauser comorbidity index and extensive patient-level information from the hospital abstract. If one considers the complications indicator a good proxy for poor quality in acute hospitalizations then the answer may be positive. If one considers the unplanned readmission indicator to be the better quality proxy, then the answer is negative. Rates o f unplanned hospital readmission and in-hospital complications have been proposed as alternatives to mortality rates in quality assessment during the last 15 years. Much more uncertainty concerning the formulation of these measures exists than with mortality or LOS, but they possess qualities that make them preferred alternatives for many conditions and procedures. Chief among these is their greater prevalence when compared to mortality and their clinical appeal for many conditions (Rosen et al., 1992; Silver et al, 1992; Ashton et al., 1997). Our experience in modeling these two outcomes using hospital administrative data, however, has been quite limited. Therefore, it is significant that a combined diagnosis and pharmacy-based model in this study provided superior discrimination regarding complications than has generally been seen in the health services research literature. Statistical significance is not equivalent to significance in the public health policy context. To understand whether the increase in discrimination achieved by adding pharmacy data to hospital profiling efforts based on complication rates is worthwhile, increases resulting from other proposed policies were examined. A currently debated policy is to require the recording of a minimal set of clinical values for hospitalized 128 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. patients. In a relevant example, Hannan and colleagues (1997) estimated the increase in the C-statistic from adding two important clinical elements to a claims-based model in predicting mortality in New York CABG patients. They noted an increase of approximately .04 in the C-statistic, which was termed “considerable” improvement. This study is of note because it concerns one of the most successful hospital profiling efforts in the country and because the comparison used administrative data in which complications of care were carefully excluded. In the present research an increase of .03 was demonstrated when the Revised CDS was added to the relatively complications-ffee Deyo measure in a similar model. Despite the improvement in discrimination demonstrated here for one study outcome, these results should be considered preliminary in nature and not sufficiently compelling to initiate public action. Examination of the results at the coefficient level revealed limited support for the suggested mechanism by which improvements in prediction were achieved. The majority of improvement in risk assessment, coming as it did from a limited number of prescription history markers, also suggests a relatively narrow range for application of such data. This runs contrary to initial expectations. The recommendation at this time, then, is to extend and replicate the findings of this study. The findings so far support an initial effort to link public databases that contain pharmacy records data with hospital abstract data. A viable California effort would entail linking the MediCal claims database maintained by California Department of Health Services with hospital abstract data maintained by OSHPD. Currently both 129 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. systems use a scrambled SSN to uniquely identify patients and these two data files could be linked at minimal cost. Researchers have maintained that hospital profiling efforts should generally be pursued within narrow diagnostic or surgical categories and this advice should be adhered to in subsequent studies. One procedure that has been extensively studied in the last decade with both administrative and clinical data is CABG. The separate and combined predictive performance of diagnosis and pharmacy-based measures of comorbidity might be successfully explored within the context of this procedure. Changes to the Revised CDS and diagnosis-based comorbidity measures such as the Charlson CI should also be contemplated given our clinical understanding of cardiovascular disease. The high volume of this procedure, its high costs, and the fact that the procedure is commonly performed on Medicare-aged patients, lend great policy relevance to this suggested study. Study Limitations This study was based on a convenience sample of patients with acute conditions hospitalized at Kaiser Permanente hospitals in Southern California in the early 1990s. Kaiser is a large, well-established health management organization that ran its own hospitals during the study period. Because few HMOs exist in the United States with similar characteristics, it is difficult to generalize this study’s findings to managed care organizations in general. The study findings may not be representative of similar large health plans in other parts of the country or in other time periods. The exclusion of 130 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. patients under the age of 18 also limits the comparability of these results to similar, adult populations. Generalizability of these results to public and private hospital patients is questionable. The incentives and reporting requirements faced by managed care organizations regarding coding of hospital data differ markedly from those faced in the fee-for-service sector. The integrated nature of the services provided within Kaiser may have resulted in more reliable coding of patient comorbidities in the hospital abstract (and perhaps more complete pharmacy data) than in other settings. Prescribing practices for the control of disease are likely to vary considerably across health care sectors and regions of the country, also limiting the generalizability of these results. Additionally, the medication records for all hospitalized patients in this study may not have been complete, since they reflect only prescriptions filled at Kaiser system pharmacies. While patients were required to have a pharmacy benefit to participate in the original study, we are unable to demonstrate that they filled all prescriptions at Kaiser pharmacies, despite strong financial incentives to do so. The study sample size also limited the types of analysis that could be performed and their reliability. The ratio of observations to predictor variables in multivariate models was not always optimal and may have influenced results in some models. Logistic regression diagnostics did not reveal significant violations, but model convergence was sometimes a problem and necessitated the removal of comorbidity markers in several analyses. The low incidence seen for certain diseases and drug class indicators (e.g., HIV, Renal disease) made estimates of their impact on hospital outcomes 131 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. less than optimal. Small sample size precluded the creation of models that might include important explanatory factors such as individual DRGs or primary diagnosis. There is currently no consensus on the best way to define complications of care or unplanned readmissions using administrative data and the formulation of these outcomes has generally been study-specific. Such problems do not affect the definition of length of stay. This study followed the approach outlined by Geraci and colleagues (1997) to define complications and used a general approach to define readmissions, both conservative in nature. Less conservative approaches would have increased the incidence of these outcomes and resulted in fewer problems with logistic model convergence, possibly changing study results. It is not possible to determine if these were the most appropriate outcome definitions for this study. 132 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. REFERENCES American Society o f Hospital Pharmacists. (1998). A m erican hospital fo rm u la ry service drug inform ation. Bethesda, M D : Published by authority o f the Board o f Directors o f the American Society o f Hospital Pharmacists. Ashton, C. M., Del Junco, D. J., Souchek, J., Wray, N. P., & Mansyur, C. L. (1997). The association between quality o f inpatient care and early readmission: A meta-analysis o f the evidence. M edical Care, 35(10), 1044-1059. Berwick, D. M., & Wald, D. L. (1990). Hospital leaders’ opinions o f the HCFA mortality data. JA M A , 263-247. Brailer, D. J., Kroch, E., Pauly, M. V., & Huang, J. (1996). Comorbidity-adjusted complication risk: A new outcome quality measure. M edical Care, 34(5), 490-505. Camberg, L. C., Smith, N. E., Beaudet, M., Daley, J., Cagan, M., & Thibault, G. (1997). Discharge destination and repeat hospitalizations. M edical Care, 35(8), 756-767. Charlson, M.E., Pompei, P, & MacKenzie, C.R (1987). A new method o f classifying prognostic com orbidity in longitudinal studies: Development and validation. Journal o f Chronic D isease. 40(5), 373-83. Christensen, D. B., Williams, B., Goldberg, H. I., Martin, D. P., Engelberg, r., & LoGerfo, J. P. (1994). Comparison o f prescription and medical records in reflecting patient antihypertensive drug therapy. Pharm acoepidem iology, 28,99-104. Clark, D. O., Von KorfF, M., Saunders, K., Baluch, W. M., Simon, G. E. (1995). A chronic disease score with empirically derived weights. M edical Care, 33(8), 783-795. Cleary, P. D., Greenfield, S., Mulley, A. G., Pauker, S. G., Schroeder, S. A., Wexler, L., & McNeil, B. J. (1991). Variations in length o f stay and outcomes for six medical and surgical conditions in Massachusetts and California. JAM A, 266(1), 73-79. Cleves, M. A., Sanchez, N., & Mayumi, D. (1997). Evaluation o f two competing methods for calculating Charlson’s comorbidity index when analyzing short-term mortality using administrative data. Journal o f Clinical Epidemiology, 50(8), 903-908. Cohen, J. A. (1960). A coefficient of agreement for nominal scales. Educational Psychology M easures, 20,37-46. 133 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. D’Hoore, W. D., Sicotte, C., & Tilquin, C. (1993). Risk adjustment in outcome assessment: The Charlson Comorbidity Index. Methods o f Inform ation in Medicine, 32,382-387. D’Hoore, W. D., Bouckaert, A., & Tilquin, C. (1996). Practical considerations on the use o f the Charlson Comorbidity Index with administrative data bases. Journal o f Clinical Epidemiology, 49(12), 1429-1433. DesHamais, S. I., Chesney, J. D., Wroblewski, R. T., Fleming, S. T., & McMahon, L. F. (1988). The risk-adjusted mortality index: A new measure o f hospital performance. M edical Care, 26(12), 1129-1145. DesHamais, S. I., McMahon, L. F., Wroblewski, R. T., & Hogan, A. J. (1990). Measuring hospital performance: The development o f mortality, readmissions, and complications. M edical C are, 28(12), 1127-1141. DesHamais, S., McMahon, L. F., & Wroblewski, R. (1991). Measuring outcomes o f hospital care using multiple risk-adjusted indexes. Health Service Research, 26(4), 425-445. Deyo, R. A., Cherkin, D. C., & Ciol, M. A. (1992). Adapting a clinical comorbidity index for use w ith ICD-9-CM administrative databases. Jo u rn a l o f Clinical Epidemiology. 45(6): 613- 9. Donabedian, A. (1980). The definition o f quality a n d approaches to its assessm ent. Ann Arbor: Health Administration Press. Donahue, J. G., Scott, T. W., Goetsch, B. S., Livingston, J. M., Greinder, D. K. & Platt, R. (1997). Assessment of asthma using automated and full-text medical records. Journal o f A sth m a , 34(4), 273-281. Epstein, A. M, Bogen, J., Thorpe, K. Dreyer, P. and Conoily, K. (1991). changes in length o f stay and rates o f readmission in response to prospective payment. Inquiry, 28, 19-28. Elixhauser, A., Steiner, C., Harris, D. R. & Coffey, R. M . (1998). Comorbidity measures for use w ith administrative data. M edical Care, 36(1), 8-27. Fink, A., Yano, E. M., & Brook, R. H. (1989). The condition o f the literature on differences in hospital mortality. M edical Care, 27(4), 315-336. Fisher, E. S., Whaley, F. S., Krushat, W. M., Malenka, D. J., Baron, J. A. & Hsia, D. C. (1992). The accuracy o f Medicare’s hospital claims data: Progress has been made, but problems remain. A m erican Journal o f Public Health, 82(2), 243-8. Fishman, P. A., & Shay, D. K. (1999). Development and estimation o f a pediatric chronic disease score using automated pharmacy data. M edical Care, 37(9), 874-883. 13 4 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Geraci, J. M., Ashton, C. M., Kuykendall, D. H., Johnson, M. L., & Wu, L . (1997). International Classification o f Diseases, 9th Revision, Clinical Modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients. M edical Care, 35(6), 589-602. Geraci, J. M., Ashton, C. M., Kuykendall, D. H., Johnson, M. L., Souchek, J., Junco, D. & Wray, N. P. (1999). The association o f quality o f care and occurrence of in-hospital, treatment- related complications. M edical Care, 37(2), 140-48. Ghali, W. A., H all, R. E., Rosen, A. K., Ash, A. S., & Moskowitz. (1996). Searching for an improved clinical comorbidity index for use with 1CD-9-CM administrative data. Journal o f C linical E pidem iology, 49(3), 273-278. Gonnella, J. A ., Hombrook, M . C., & Louis, D. Z. (1984). Staging of disease: A case-mix m easurement. Jo u rn a l o f the A m erican M edical Association, 251(5), 637-644. Greenfield, S., Aronow, H. U., Elashoff, R. M., & Watanabe, D. (1988). Flaws in mortality data: The hazards o f ignoring comorbid disease. JA M A , 260(15), 2253-2255. Greenfield, S., Apolone, G., McNeil, B. J., & Cleary, P. D. (1993). The importance o f co-existent disease in the occurrence o f postoperative complications and one-year recovery in patients undergoing total hip replacement: Comorbidity and outcomes after hip replacement. M edical Care, 31(2), 141-154. Hanley, J. A., & McNeil, B. J. (1982). The meaning and use o f the area under the receiver operating characteristic (ROC) curve. R adiology, 143, 29-36. Hannan, E. L., Racz, M. J., Jollis, J. G., & Peterson, E. D. (1997). Using M edicare data to assess provider quality for CABG surgery: Does it w ork well enough? H ealth Services R esearch, 31(6), 659-78. Hartz, A. J.& K uhn, E. M. (1994). Comparing hospitals that perform coronary artery bypass surgery: The effect o f outcome measures and data sources. American Journal o f P ublic H ealth, 84(10), 1609-14. Hawker, G. A., Coyte, P. C., Wright, J. G., Paul, J. E. & Bombardier, C. (1997). Accuracy o f adm inistrative data for assessing outcomes after knee replacement surgery. Jo u rn a l o f C linical E pidem iology, 50(3), 265-73. Hofer, T. P., & Hayward, R. A. (1996). Identifying poor-quality hospitals: Can hospital mortality rates detect quality problems for medical diagnoses? M edical Care, 34(8), 737-753. Iezzoni, L. I. (1997a). Dimensions of risk. In R isk Adjustm ent f o r Measuring Health Care Outcomes, L. I. Iezzoni, Ed. Ann Arbor: Health Administration Press. 135 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Iezzoni, L. I. (1997b). Risk adjustment and current health policy initiatives. In R isk Adjustm ent fo r M easuring H ealth Care Outcomes, L. I. Iezzoni, Ed. Ann Arbor: Health Administration Press. Iezzoni, L. I., Ash, A. S., Shwartz, M., Ash, A. S., Hughes, J. S., Daley, J. & Mackieman, Y. D. (1996). Severity measurement methods and judging hospital death rates for pneumonia. M edical C are, 34(1), 11-28. Iezzoni, L. I., Ash, A. S., Shwartz, M., & Mackieman, Y. D. (1997c). Differences in procedures use, in-hospital mortality, and illness severity by gender for acute myocardial infarction patients. M edical C are, 35(2), 158-171. Iezzoni, L. I., Daley, J., Heeren, T., Foley, S. M., Hughes, J. S, Fisher, E. S., Duncan, C. & Coffman, G. A.. (1994). Using administrative data to screen hospitals for high complication risks. Inquiry, 31,40-55. Iezzoni, L. I., Foley, S. M., Daley, J., Hughes, J., Fisher, E. S. & Hereen, T. (1992). Com orbidities, complications, and coding bias: Does the number o f diagnosis codes matter in predicting in-hospital mortality? JA M A , 267(16, 2197-2203. Iezzoni. L.I., Ash, A.S., Shwartz, M., Daley J., Hughes, J.S., & Mackieman Y.D. (1995). Predicting who dies depends on how severity is measured: implications for evaluating patient outcomes. A nn a ls o f Internal M edicine, 123(10), 763-770. Iezzoni, L. I., Heeren, T., Foley, S. M, Daley, J., Hughes, J., & Coffman, G. A. (1994c). Chronic conditions and risk o f in-hospital death. H ealth Services Research, 29(4), 435-460. Jencks, S. F., W illiam, D. D., & Kay, T. L. (1988). Assessing hospital-associated deaths from discharge data: the role o f length o f stay and comorbidities. JAM A, 1988,26 Johnson, K.A., Parker J.P., Venturini, F., Kamath ,T., Rudolph, M. &ChaikIedkaew U. Impact o f Pharm acists’ Interventions in a Pharmaceutical Care Demonstration Project. Podium Presentation. Am erican Pharm aceutical A ssociation A nnual Meeting. San Antonio, Texas. M arch, 1999. Johnson, R. E., Hom brook, M. C., & Nichols, G. A. (1994). Replicating the chronic disease score (CDS) from automated pharmacy data. J o u rn a l o f C linical Epidemiology, 47(10), 1191- 1199. Jollis, J. G., Ancukiewicz, M., DeLong, E. R., Pryor, D. B., Muhlbaier, L. H. & Mark, D. B. (1993). Discordance o f databases designed for claims payment versus clinical information systems. Annals o f Internal M edicine. 119: 844-50. Jost, T. S. (1994). Health system reform: Forward or backward with quality oversight? JA M A , 271(19), 1508-1511. 136 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Kaplan, M. H., & Feinstein, A. R. (1974). The importance o f classifying initial co-morbidity in evaluating the outcome o f diabetes mellitus. Jo u rn a l o f C hronic D isease, 24, 387-404. Karp, H. R., Flanders, W. D., Shipp, C. C., Taylor, B, & Martin, D. (1998). Carotid endarterectomy among Medicare beneficiaries: A statewide evaluation of appropriateness and outcome. Stroke, 29(1), 46-52. Kieszak, S. M., Flanders, W. D., Kosinski, A. S., Shipp, C. C. & Karp, H. (1999). A comparison o f the Charlson Comorbidity Index derived from medical record data and administrative billing data. Jo urnal o f Clinical Epidemiology, 52(2), 137-42. Kossovy, M. P., Sarasin, F. P., Bolla, F. Gaspoz, J. M., & Borst, F. (1999). Distinction between planned and unplanned readmissions following discharge from a department o f internal medicine. M ethods o f Information in M edicine, 38, 140-143. Krumholz, H. M., Parent, E. M, Tu, N., Vaccarino, V., Wang, Y, Radford, M. J., & Hennen, J. (1997). Readmission after hospitalization for congestive heart failure among Medicare beneficiaries. A rchives o f Internal M edicine, 157, 99-104. Kuykendall, D. H., Ashton, C. M., Johnson, M. L., & Geraci, J. M. (1995). Identifying complications and low provider adherence to normative practices using administrative data. H ea lth Services Research, 30(4), 531-554. Lamers, L. M. (1999). Pharmacy costs groups: A risk-adjuster for capitation payments based on the use o f prescribed drugs. M edical Care, 37(8), 824-830. Lawrence, V. A., Dhanda, R., Hilsenbeck, S. G., & Page, C. P. (1996). Risk of pulmonary complications after elective abdominal surgery. C hest, 110(3), 744-750. Librero, J., Peiro, S. & Ordinana, R. (1999). Chronic com orbidity and outcomes of hospital care: length o f stay, mortality, and readmission at 30 and 365 days. Journal o f Clinical Epidem iology, 52(3), 171-179. Ludke, R. L, Booth, B. M. & Lewis-Beck, J. A. (1993). Relationship between early readmission and hospital quality o f care indicators. Inquiry, 30, 95-103. Malenka, D. J., M cLerran, D., Roos, N., Fisher, E. S., & W ennberg, J. E. (1994). Using administrative data to describe casemix: A comparison with the medical record. Journal o f C linical E pidem iology, 47(9), 1027-1032. Malone, D. C., Billups, S. J., Valuck, R. J., & Carter, B. L. (1999). Development of a chronic disease indicator score using a Veterans Affairs medical center medication database.. Journal o f C linical Epidemiology, 52(6), 551-557. 137 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Marcantonio, E. R., McKean, S, Goldfinger, M ., Kleefield, S., Yurkofsky, M., Brennan, T. A. (1999). Factors associated with unplanned hospital readmission among patients 65 years o f age and older in a Medicare managed care plan. The A m erican Jo u rn a l o f M edicine, 107(1), 13-17. Matsui, K., Goldman, L., Johnson, P. A., Kuntz, K. M., Cook, E. F. & Lee, T. H. (1996). Com orbidity as a correlated o f length o f stay for hospitalized patients with acute chest pain. J o u rn a l o f G eneral Internal M edicine, 11, 262-68. McCombs, J. S., Cody, M., Besinque, K., Borok, G., Ershoff, D., Groshen, S. Hay, J., Johnson, K. A., N ichol, M. B. & Nye, M. T. (1995). Measuring the impact o f patient counseling in the outpatient setting: the research design o f the Kaiser Permanente/USC Patient Consultation Study. C linical Therapeutics, 17(6), 1188-1207. McCombs, J. S., Liu, G. L., Feng, W., Cody, M., Parker, J. P., Nichol, M. B., Hay, J. W., Johnson, K . A., Groshen, S. L., & Nye, M. T. (1998). The Kaiser Permanente/USC Patient Consultation Study: change in use and cost o f health care services. A m erican Jo u rn a l o f H ealth System Pharm acy, 55, 2485-99. Melfi, C., Hollman, E., Arthur, D., & Katz, B. (1995). Selecting a patient characteristics index for the prediction o f medical outcomes using administrative claims data. Journal o f C linical E pidem iology, 48(7), 917-926. Morris, J. A., M acKenzie, E. J., & Edelstein, S. L. (1990). The effect o f preexisting conditions on m ortality in trauma patients. JAM A, 263(14), 1942-1946. Mossey, J. M., & Roos, L. L. (1987). Using insurance claims to measure health status: The illness scale. Jo u rn a l o f C hronic Disease, 4 0 ,41S-50S. Naessens, J. M., Leibson, C. L, Krishan, 1., & Ballard, D. J. (1992). Contribution o f a measure o f disease complexity (COMPLEX) to prediction o f outcome and charges among hospitalized patients. M ayo C linical Proceedings, 67, 1140-1149. Newschaffer, C. J., Bush, T. L., & Penberthy, L. T. (1997). Comorbidity measurement in elderly female breast cancer patients with administrative and medical records data. Journal o f C linical E pidem iology, 50(6), 725-733. Park, R. E., Brook, R. H., Kosecoff, J., Keesey, J, Rubenstein, L., Keeler, E., Kahn, K. L., Rogers, W. H., & Chassin, M. R. (1990). Explaining variations in hospital death rates: Randomness, severity o f illness, quality o f care. JAM A, 264(4), 484-490. Philbin, E. F., & DiSalvo, T. G. (1999). Prediction o f hospital readmission for heart failure: development o f a simple risk score based on administrative data. Journal o f the American College o f Cardiology, 33(6), 1560-1566. 138 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Pine, M., Norusis, M., Jones, B. & Rosenthal, G. E. (1997). Prediction o f hospital mortality rates: A comparison o f data sources. A nnals o f Internal M edicine, 126, 347-54. Polanczyk, C. A., Rohde, L. E., Philbin, E. A., DiSalvo, T. G. (1998). A new casemix index for hospital mortality among patients with congestive heart failure. M edical Care. 36(10), 1489-99. Pompei, P., Charlson, M . E., Ales, K.., MacKenzie, C. R., & Norton, M. (1991). Relating patient characteristics at the time o f admission to outcomes o f hospitalization. Journal o f C linical Epidem iology, 44(10), 1063-1069. Quam, L., Ellis, L. B., Venus, P., Clouse, J., Taylor, C. G. & Leatherman, S. (1993). Using claims data for epidemiologic research: The concordance of claims-based criteria with the medical record and patient survey for identifying a hypertensive population. M edical Care, 31(6), 498-507. Riley, G., Lubitz, J. Gomick, M., Mentnech, R., Eggers, P. & McBean, M. (1993). Medicare beneficiaries: Adverse outcomes after hospitalization for eight procedures. M edical Care, 31(10), 921-49. Roblin, D. W. (1996). C ase M ix m easurem ent using outpatient p h arm acy data. In Physician P rofiling a n d R isk A djustm ent, N. Goldfield & P. Boland, Eds. Gaithersburg, Maryland: Aspen. Romano, P. S., Zach, A., Luft, H. S., Rainwater, J., Remy, L. L., & Campa, D. (1995). The California hospital outcomes project: Using administrative data to compare hospital performance. Jo u rn a l on Q uality Im provem ent, 21(12), 668-682. Romano, P. S., Roos, L. L., et al. (1993a). Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: Differing perspectives. Journal o f Clinical Epidem iology. 46(10): 1075-9. Romano, P. S., Roos, L. L., et al. (1993b). Further evidence concerning the use o f a clinical comorbidity index with ICD-9-CM adm inistrative data. Journal o f Clinical Epidem iology. 46(10): 1075-9. Romano, P. S., Roos, L. L., Luft, H. S., Jollis, G. G., Doliszny, K. (1994). A comparison o f administrative versus clinical data: Coronary artery bypass surgery as an example. Jo urnal o f C linical E pidem iology, 47(3), 249-60. Roos, L. L., Sharp, S. M . & Cohen, M. M. (1991). Comparing clinical information with claims data: Some similarities and differences. Jo u rn a l o f C linical Epidem iology, 44(9), 881-8. Roos, L. L., Stranc, L, James, R. C., & Li, J. (1997). Complications, comorbidities, and mortality: Improving classification and prediction. H ealth Services Research, 32(2), 229-242. 139 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Rosen, A. K., Geraci, J. M., Ash, A. S., McNifF, K. J. & Moskowitz, M. A. (1992). Postoperative adverse events o f common surgical procedures in the Medicare population. M edical C are, 30(9), 753-65. Rosenthal, G. E., Amrik, S., Way, L. E., & Harper, D. L. (1998). Variations in standardized hospital mortality rates for six common medical diagnoses: Implications for profiling hospital quality. M edical Care, 36(7), 995-954. Satariano, W. A., & Ragland, D. R. (1994). The effect o f comorbidity on 3-year survival o f women with primary breast cancer. Annals o f Internal M edicine, 120(2), 104-110. Schwartz, M. & Ash, A. S. (1997). Evaluating the performance o f risk-adjustment methods: Dichotomous outcomes. In R isk Adjustm ent f o r Measuring Health Care Outcomes, L. I. Iezzoni, Ed. Ann Arbor: Health Administration Press. Schwartz, M., Iezzoni, L. I., Moskowitz, M. A., Ash, A. S. & Sawitz, E. (1996). The importance o f comorbidities in explaining differences in patient costs. M edical Care, 34(8), 767-782. Silber, J. H., Williams, S. V., Krakauer, H., & Schwartz, J. S. (1992). Hospital and patient characteristics associated with death after surgery. M edical Care, 30(7), 615-629. Silber, J. H., Rosenbaum, P. R., & Ross, R. N. (1995). Comparing the contributions o f groups o f predictors: Which outcomes vary with hospital rather than patient characteristics? Journal o f the Am erican Statistical A ssociation, 90(429), 7-18. Smith, D. W. (1994). Evaluating risk adjustment by partitioning variation in hospital mortality rates. Statistics in Medicine, 13, 1001-1013. Soecken, K. L., Prescott, P. A., Herron, D. G., & Creasia, J. (1991). Predictors o f hospital readmission: A meta-analysis. E valuation a n d the health Professions, 14, 262-281. Thom as, J. W., & Holloway, J. J. (1991). Investigating early readmission as an indicator for quality o f care studies. M edical C are, 29(4), 377-394. Thomas, J. W. (1996). Does risk-adjusted readmission rate provide valid information on hospital quality? Inquiry, 28, 258-270. Thomas, J. W. & Hofer, T. P. (1998). Research evidence on the validity o f risk-adjusted mortality rate as a measure of hospital quality o f care. M edical Care R esearch a n d R eview , 55(4), 371-404. U.S. Congress, Office o f Technology Assessment (OTA) (1988). The quality o f medical care: Information for consumers. W ashington, DC: U.S. Government Printing Office. Von Korff, M., Wagner, E. H., & Saunders, K. (1992). A chronic disease score from automated pharmacy data. Journal o f Clinical Epidemiology, 45(2), 197-203. 140 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission. Weintraub, W. S., Deaton, C., Shaw, L., Mahoney, E., Morris, D . C., Saunders, C., Canup, D., Connolly, S., Culler, S., Becker, E. R., Kosinski, A., & Boccuzzi, S. J. (1999). Can cardiovascular clinical characteristics be identified and outcome models be developed from an in-patient claims database? The Am erican Journal o f Cardiology, 84, 166-169. Weissman, J. S., Ayanian, J. Z., Chasan-Taber, S. C., Sherwood, M . J., Roth, C., & Epstein, A. M. (1999). Hospital readmissions and quality o f care. Medicalcare, 37(5), 490-501. Wen, S. W ., Hernandez, R. & Naylor, C. D. (1995). Pitfalls in nonrandomized outcomes studies. T he case o f incidental appendectomy with open cholecystectomy. JA M A , 274(21), 1687- 91. Wray, N. P., Ashton, C. M., Kuykendall, D. H., Petersen, N. J., Souchek, J., & Hollingsworth, J. C. (1995). Selecting disease-outcome pairs for monitoring the quality of hospital care. M edical Care, 33(1), 75-89. Young, W.W., Kohler, S., & Kowalski, J. (1994). PMC patient severity scale: Derivation and validation. Health Services Research, 23,367-. 141 R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Controlling for biases from measurement errors in health outcomes research: A structural equation modeling approach

PDF

Effects of a formulary expansion of the use of SSRIs and health care services by depressed patients in the California Medicaid program

PDF

Cost analysis of three pharmacy counseling programs for diabetics in a health maintenance organization

PDF

A methodology to identify high -risk patients with diabetes in the California Medicaid populations (Medi -Cal)

PDF

Assessing the cost implications of combined pharmacotherapy in the long term management of asthma: Theory and application of methods to control selection bias

PDF

Compliance study of second-generation antipsychotics on patients with schizophrenia

PDF

A new paradigm to evaluate quality-adjusted life years (QALY) from secondary database: Transforming health status instrument scores to health preference

PDF

Biopharmaceutical strategic alliances: Interorganizational dynamics and factors influencing FDA regulatory outcomes

PDF

Hospital conversions: The California experience

PDF

An analysis of third -party regulatory audit systems for medical device safety

PDF

An analysis of health risk selection and quality of care under Medicare fee -for -service and Medicare managed care health care systems

PDF

Executive development and effectiveness: A study of mobility assignment experiences of career executives in federal research and development agencies

PDF

A case/parental/sibling control study of Ewing's sarcoma/peripheral primitive neuroectodermal tumor (pPNET)

PDF

Health care finance reform and market failure at King/Drew Medical Center, Los Angeles: An analysis of pediatric social outcomes in an urban public safety -net hospital

PDF

A search for theory: Performance management to improve transportation safety

PDF

Business political influence on agency regulatory decision making: The efficacy of corporate political activity in Taiwan's CATV license awarding

PDF

Essays on organizational forms and performance in California hospitals

PDF

Education and intelligence test scores: Predictors of dementia?

PDF

A study of employee health plan choice and medical cost: Panel data probit regression and sample selection model

PDF

Care management for the uninsured: A force field analysis of the business case

Asset Metadata

Creator Parker, Joseph Paul (author)

Core Title Assessment of prognostic comorbidity in hospital outcomes research: Is there a role for outpatient pharmacy data?

School Graduate School

Degree Doctor of Philosophy

Degree Program Public Administration/Public Policy

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag health sciences, health care management,Health Sciences, Pharmacy,health sciences, public health,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Graddy, Elizabeth (committee chair), Ferris, James M. (committee member), McCombs, Jeffrey S. (committee member), Stallings, Robert A. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c16-178196

Unique identifier UC11339054

Identifier 3054893.pdf (filename),usctheses-c16-178196 (legacy record id)

Legacy Identifier 3054893.pdf

Dmrecord 178196

Document Type Dissertation

Rights Parker, Joseph Paul

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA