Screening Algorithms to Assess the Accuracy of Present-on-Admission Coding

by Michael Pine, MD, MBA; Donald E. Fry, MD; Barbara Jones, MA; and Roger Meimban, PhD

Abstract

Present-on-admission modifiers for International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes are rapidly becoming a standard coding requirement. Inaccurate coding of these modifiers can distort analyses of risk-adjusted outcomes and determinations of hospital reimbursement. A set of 12 screens for the plausibility of present-on-admission designations was developed and tested using New York State claims data for 2003, 2004, and 2005. Application of these screens uncovered numerous potential problems in coding with 39 percent of hospitals achieving a composite score higher than 90 percent and 36 percent of hospitals scoring 80 percent or less. Whether data quality control personnel adopt the screens employed in this study or develop similar sets of their own, the analytic approach used in this study provides a cost-effective method of assessing the quality of coding and the integrity of clinical performance reports based on enhanced claims data.

Introduction

Present-on-admission (POA) modifiers for International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes now are required on hospital claims submitted to the Centers for Medicare and Medicaid Services (CMS) and are rapidly becoming a standard coding requirement. Because these modifiers distinguish potentially avoidable hospital-acquired complications from unavoidable comorbidities that often increase lengths of stay, costs, and risks of adverse outcomes, inaccuracies in their application can distort analyses of risk-adjusted outcomes and determinations of hospital reimbursement. This paper describes a set of easily applied screens to assess the accuracy of POA coding and demonstrates their ability to detect differences in coding practices among hospitals in New York State.

Background

The use of POA modifiers to distinguish diagnoses that are present on admission from diagnoses that are hospital-acquired complications was first described in 1991 and has been required by California and New York State since the mid-1990s.^{1, 2} There was little further adoption of these modifiers until they were incorporated into the UB-04 claims form and a provision in the Deficit Reduction Act required their use on claims submitted to Medicare for discharges on or after October 1, 2007.³ 

Increasing interest in using these modifiers in public reporting of comparative hospital quality, pay-for-performance programs, and quality improvement initiatives has been accompanied by growing concern about the potential effect of inaccurate coding on clinical, managerial, and purchasing decisions. Incomplete coding of secondary diagnoses generally results in underestimates of the severity and complexity of a hospitalized patient’s condition, which rarely if ever benefits a hospital. On the other hand, improper coding of hospital-acquired complications may improve a hospital’s measured clinical performance, enhancing both its reputation and its bottom line. To ensure the validity of analyses of hospital performance based on POA-coded diagnoses, the accuracy of POA coding must be independently established.

Accurate POA coding depends upon proper documentation by clinicians and good coding practices by medical record personnel. Chart reviews by clinically sophisticated health information professionals can detect problematic documentation and improper coding, but these reviews are too expensive to be employed routinely. Automated screens designed to detect inconsistencies between POA coding and other data contained in billing records could provide valuable insights into the validity of conclusions about hospital performance drawn from these data without costly chart reviews. Although some coarse automated screens have been developed and applied to detect potentially inaccurate POA coding, a sophisticated set of screens capable of identifying subtle but important errors and pinpointing potential problems requiring remediation has not been described. ⁴

Development and Description of POA Screens

The most recently available claims data from New York State’s SPARCS database for hospital discharges in 2003, 2004, and 2005 were used to create and evaluate a set of 12 screens for the quality of POA coding. New York requires that hospitals designate all secondary diagnoses in claims submitted to this database as present on admission, hospital-acquired, or unknown. 

To assess the quality of POA designations in a variety of frequently occurring types of admissions, three categories of admissions were screened: (1) high-risk admissions identified by principal diagnosis (e.g., septicemia, respiratory failure), which included 22 percent of discharges and 70 percent of deaths (9.2 percent mortality rate); (2) elective surgical admissions for one of seven low-risk procedures performed within two days of admission for specified principal diagnoses (e.g., hysterectomy for leiomyoma, knee replacement for osteoarthrosis); and (3) inpatient childbirth admissions. These three categories of admissions were chosen because they represented unique areas that would examine the breadth of the coding skills within a given hospital. Screens were applied to all secondary diagnoses that are not exempted from POA coding by ICD-9-CM Official Guidelines for Coding and Reporting.⁵ Standards used to evaluate the individual hospitals’ data quality and to establish specific thresholds for the designation of criteria and thresholds for unsatisfactory performance were based on clinical judgment and examination of aggregate data. 

Screens 1, 2, and 3-Secondary Diagnosis Codes for Chronic Conditions: We identified a set of diagnosis codes for chronic conditions that should almost always be present on admission (e.g., malignancy, osteoporosis). Rates at which these codes were designated as unknown or as hospital-acquired are shown in Table 1 for secondary diagnoses assigned to high-risk admissions (Screen 1), to elective surgical admissions (Screen 2), and to inpatient childbirth admissions (Screen 3). For high-risk admissions, these codes were designated as unknown 5.8 percent of the time and as hospital-acquired 1.1 percent of the time; for elective surgical admissions, these codes were designated as unknown 4.5 percent of the time and as hospital-acquired 0.6 percent of the time; and for inpatient childbirth admissions, these codes were designated as unknown 8.9 percent of the time and as hospital-acquired 1.9 percent of the time. POA coding was considered unsatisfactory when these secondary diagnosis codes were designated as unknown more than 10 percent of the time or when they were designated as hospital-acquired more than 2 percent of the time.

Table 1: Overall and Hospital-level Performance of Screens to Assess the Accuracy of Present-on-Admission Modifiers of Secondary Diagnoses That Are Almost Always Present on Admission

Screen 4-Secondary Diagnosis Codes for Chronic Conditions with Exacerbation or Associated Complications in High-Risk Admissions: We identified a set of diagnosis codes each of which specified both a chronic condition and either a potentially acute exacerbation (e.g., obstructive chronic bronchitis with exacerbation) or a potentially acute associated complication (e.g., chronic hepatitis with coma). Because exacerbations and complications of chronic conditions will sometimes be hospital-acquired, the rate at which these combination codes are designated as hospital-acquired should be somewhat greater than the corresponding rate for codes specifying these complications without exacerbations or complications. For high-risk admissions in this study (Table 2), these combination codes were designated as hospital-acquired 3.3 percent of the time, while corresponding codes for the same chronic conditions without potentially acute exacerbations or complications were designated as hospital-acquired only 1.1 percent of the time. POA coding was considered unsatisfactory when secondary diagnosis codes specifying potentially acute exacerbations or complications of a chronic condition were designated as hospital-acquired less than 2 percent of the time in high-risk admissions, or when this rate was less than twice the rate at which codes for corresponding uncomplicated chronic conditions were designated as hospital-acquired.

Table 2
Overall Performance of Nine Additional Screens to Assess the Accuracy of Present-on-Admission Modifiers

Screen 5-Secondary Diagnosis Codes That Frequently Are Hospital-acquired in High-Risk Admissions: We identified three sets of diagnosis codes for conditions that frequently are acquired during hospitalization in high-risk admissions (e.g., transfusion reaction, acute respiratory failure). These three categories were chosen from a continuum of codes that were most to least likely to be coded as hospital-acquired. Secondary diagnosis codes in Set A were designated as hospital-acquired more than 50 percent of the times they were coded (mean 63.5 percent); secondary diagnosis codes in Set B were designated as hospital-acquired between 30 and 50 percent of the times they were coded (mean 34.7 percent); and secondary diagnoses in Set C were designated as hospital-acquired between 20 and 30 percent of the times they were coded (mean 24.8 percent). POA coding was considered unsatisfactory when secondary diagnosis codes in Set B were designated as hospital-acquired less than 15 percent of the times they were coded or when the percent of times that secondary diagnosis codes were designated as hospital-acquired was not greater for Set A than for Set B and greater for Set B than for Set C.

Screen 6-Secondary Diagnosis Codes Associated with Higher Mortality Rates When Hospital-acquired Than When Present on Admission: We identified three sets of secondary diagnosis codes that are associated with higher inpatient mortality rates in high-risk admissions when these diagnoses are acquired during hospitalization than when they are present on admission (e.g., gangrene, acute renal failure). Secondary diagnosis codes in Set A were associated with inpatient mortality rates of 27.0 percent when they were hospital-acquired and 12.6 percent when they were present on admission (odds ratio 2.57); secondary diagnosis codes in Set B were associated with inpatient mortality rates of 25.2 percent when they were hospital-acquired and 15.3 percent when they were present on admission (odds ratio 1.87); and secondary diagnosis codes in Set C were associated with inpatient mortality rates of 30.5 percent when they were hospital-acquired and 21.2 percent when they were present on admission (odds ratio 1.64). POA coding was considered unsatisfactory when the odds ratio for mortality rates associated with secondary diagnosis codes was less than 1.30 for all three sets of diagnosis codes combined or when it was less than 1.60 for Set A and Set B combined and for all three sets combined. 

Screen 7-Secondary Diagnosis Codes That Generally Are Contraindications to Performing Scheduled Procedures in Elective Surgical Admissions: We identified a set of diagnosis codes for conditions that generally are contraindications to performing scheduled procedures in elective surgical admissions (e.g., septicemia, shock). In the elective surgical admissions used for this screen (Table 2), these codes were designated as hospital-acquired only 68.3 percent of the time. POA coding was considered unsatisfactory when these secondary diagnosis codes were designated as hospital-acquired less than 65 percent of the time. 

Screen 8-Secondary Diagnosis Codes for Chronic Conditions with Exacerbation or Associated Complications in Elective Surgical Admissions: We used the same set of diagnosis codes that specified both a chronic condition and either a potentially acute exacerbation or a potentially acute associated complication in Screen 4 for high-risk admissions to screen the POA coding for elective surgical admissions. Because exacerbations and complications of chronic conditions often will result in postponement of elective procedures, the rate at which these combination codes are designated as hospital-acquired should be much greater for elective surgical admissions than for high-risk admissions. For elective surgical admissions in this study, these combination codes were designated as hospital-acquired 18.7 percent of the time, while corresponding codes for the same chronic conditions without potentially acute exacerbations or complications were designated as hospital-acquired only 0.4 percent of the time. POA coding was considered unsatisfactory when secondary diagnosis codes specifying potentially acute exacerbations or complications of a chronic condition were designated as hospital-acquired less than 12 percent of the time in elective surgical admissions or when this rate was less than three times the rate at which codes for corresponding uncomplicated chronic conditions were designated as hospital-acquired.

Screen 9-Elective Surgical Admissions without Hospital-acquired Secondary Diagnosis Codes That Have Prolonged Risk-adjusted Postoperative Lengths of Stay: We identified cases with prolonged risk-adjusted postoperative lengths of stay for all elective surgical admissions and for elective surgical admissions without any secondary diagnosis codes designated as hospital-acquired.⁶ To do this, predictive equations for postoperative length of stay were derived using stepwise linear regression on all uncomplicated discharges for each of the seven elective surgical procedures included in this study. XmR control charts of differences between observed and predicted postoperative lengths of stay at each participating hospital were used to identify cases for which this difference exceeded a three-sigma upper threshold for all live discharges and for only live discharges without any secondary diagnoses designated as hospital-acquired (Figure 1 and Figure 2.)

Figure 1: Illustrative Control Chart for Risk-adjusted Lengths of Stay for All Live Discharges Notes: OBS LOS = observed postoperative length of stay PRED LOS = predicted postoperative length of stay Average = mean risk-adjusted postoperative length of stay for cases with normal postoperative lengths of stay 3 Sigma = three-sigma upper bound for normal postoperative length of stay Normal LOS = cases for which OBS LOS minus PRED LOS is less than the three-sigma upper bound Long LOS w Cpl = cases for which OBS LOS minus PRED LOS exceeds the three-sigma upper bound and at least one secondary diagnosis is coded as hospital-acquired Long LOS w/o Cpl = cases for which OBS LOS minus PRED LOS exceeds the three-sigma upper bound and no secondary diagnosis is designated as hospital-acquired

Figure 2: Illustrative Control Chart for Risk-adjusted Lengths of Stay for All Live Discharges without Hospital-acquired Complications Notes: OBS LOS = observed postoperative length of stay PRED LOS = predicted postoperative length of stay Average = mean risk-adjusted postoperative length of stay for cases with normal postoperative lengths of stay 3 Std Dev = three-sigma upper bound for normal postoperative length of stay Normal LOS = cases for which OBS LOS minus PRED LOS is less than the three-sigma upper bound Long LOS w/o Cpl = cases for which OBS LOS minus PRED LOS exceeds the three-sigma upper bound and no secondary diagnosis is designated as hospital-acquired

Prolonged risk-adjusted postoperative lengths of stay generally are associated with one or more postoperative complications and should rarely occur in uncomplicated elective surgical admissions. Therefore, high rates of prolonged risk-adjusted postoperative lengths of stay in uncomplicated cases suggest either undercoding of hospital-acquired complications or improper designation of these complications as having been present on admission. The rates of prolonged risk-adjusted postoperative lengths of stay were 5.9 percent for all live discharges and 3.9 percent for live discharges without a secondary diagnosis code designated as hospital-acquired. Coding of hospital-acquired complications was considered unsatisfactory when the rate of prolonged risk-adjusted postoperative lengths of stay among live discharges without a secondary diagnosis code designated as hospital-acquired exceeded 5.4 percent (the median rate of prolonged risk-adjusted postoperative lengths of stay for all live discharges at each participating hospital). 

Screen 10-Obstetrical Secondary Diagnosis Codes for Conditions Almost Always Present on Admission: We identified a set of diagnosis codes for obstetrical conditions (e.g., multiple gestation) that almost always are present on admission. In inpatient childbirth admissions in this study (Table 2), these codes were designated as hospital-acquired 5.2 percent of the time. POA coding was considered unsatisfactory when these codes were designated as hospital-acquired more than 3 percent of the time.

Screen 11-Obstetrical Diagnosis Codes with Fifth Digits That Are Incompatible with Inpatient Delivery: We identified a set of obstetrical diagnosis codes with fifth digits that indicate that an inpatient delivery did not occur. These codes were present in 0.3 percent of claims for inpatient childbirth admissions in this study. Coding was considered unsatisfactory when these codes were present in more than 0.5 percent of documented inpatient childbirth admissions.

Screen 12-Obstetrical Diagnosis Codes with Fifth Digits That Indicate the Occurrence of a Postpartum Complication in the Absence of a Diagnosis Designated as Hospital-acquired: We identified a set of obstetrical diagnosis codes with fifth digits that indicate that a postpartum complication occurred. No diagnosis code was designated as hospital-acquired in 36.5 percent of inpatient childbirth admissions with one or more diagnosis codes with fifth digits indicating the occurrence of a postpartum complication. POA coding was considered unsatisfactory when more than 20 percent of cases with one or more secondary diagnoses indicating the occurrence of a postpartum complication did not have any secondary diagnosis designated as hospital-acquired.

Hospital Performance on POA Screens

Hospital performance was assessed using every screen for which there were sufficient numbers of coded diagnoses or eligible cases to permit application of the screen. 

Hospital performance on the first three screens is presented in Table 1. Nine percent of participating hospitals failed Screen 1, for high-risk admissions, because of high rates of unknown designations, and 8 percent failed because of high rates of hospital-acquired designations. Thirteen percent of participating hospitals failed Screen 2, for elective surgical admissions, because of high rates of unknown designations, and 10 percent failed because of high rates of hospital-acquired designations. Twenty-one percent of participating hospitals failed Screen 3, for inpatient childbirth admissions, because of high rates of unknown designations, and 43 percent failed because of high rates of hospital-acquired designations. 

Hospital performance on the remaining nine screens is presented in Table 2. For high-risk admissions, 28 percent failed Screen 4 (for codes describing chronic conditions with potentially acute components), 17 percent failed Screen 5 (for codes describing frequently hospital-acquired conditions), and 17 percent failed Screen 6 (for codes with higher mortality rates when hospital-acquired than when present on admission). For elective surgical admissions, 39 percent failed Screen 7 (for codes for conditions that generally are contraindications to elective surgery), 3 percent failed Screen 8 (for codes describing chronic conditions with potentially acute components), and 19 percent failed Screen 9 (for prolonged risk-adjusted postoperative lengths of stay in the absence of secondary diagnoses designated as hospital-acquired). For inpatient childbirth admissions, 37 percent failed Screen 10 (for obstetrical codes almost always present on admission), 13 percent failed Screen 11 (for fifth digits of obstetrical codes that are incompatible with delivery), and 59 percent failed Screen 12 (for absence of a diagnosis designated as hospital-acquired when the fifth digit of one or more obstetrical codes indicated the occurrence of a postpartum complication). 

Of the 226 hospitals with adequate data to apply one or more of the 12 screens, composite scores were computed for the 204 that designated as unknown fewer than 10 percent of secondary diagnoses that are almost always present on admission. Weights from 1 to 10 were assigned to each of the 12 screens. Criteria for partial scores were established for some screens. For each hospital, only screens with sufficient numbers of codes or cases were scored. Average scores were computed for each screen and were used to impute missing values based on each hospital’s performance on screens for which adequate data were available. Examples of aggregate hospital scores are presented in Table 3. The final distribution of adjusted scores as percentages was as follows: 39 percent had a score higher than 90; 25 percent had a score greater than 80 but not greater than 90; 16 percent had a score greater than 70 but not greater than 80; 12 percent had a score greater than 60 but not greater than 70; and 8 percent had a score of 60 or less.

Table 3: Hospital-level Performance of Nine Additional Screens to Assess the Accuracy of Present-on-Admission Modifiers

Table 4: Example of Hospitals’ Aggregate Scores

Discussion

Hospitals in New York State have been required to use POA modifiers since 1993, but rigorous screening suggests that the quality of POA designations remains problematic. Furthermore, variations in coding practices among hospitals threaten to undermine analyses of clinical performance and reimbursement arrangements that require differentiation between unavoidable comorbidities and potentially avoidable hospital-acquired complications.⁷ The availability of guidelines for POA designations may aid in achieving better results than were found in this study, but shortages of trained personnel and challenges associated with the planned adoption of ICD-10 coding make labor-intensive quality control initiatives difficult to justify. 

On the other hand, healthcare consumers and payers need some assurance that the data on which they base their decisions are reliable and accurate. As POA designations become more influential in purchasing and payment decisions, external audits of data quality will increase in importance. As with Diagnosis Related Group (DRG) coding, poor performance in audits of data quality may result in substantial embarrassment and financial penalties. Therefore, it will be important for hospitals to initiate cost-effective screening protocols that can identify potentially problematic areas and suggest improvement strategies.

The screens described in this paper cover a wide range of conditions and employ relatively sophisticated clinical logic to detect implausible relationships between POA designations and other information routinely provided in hospital claims. For these reasons, they provide important insights into the accuracy of the diagnostic information included in hospital claims. Because these screens apply to aggregate data, it is very difficult, if not impossible, to selectively code individual records to avoid detection of inaccuracies in POA designations. Therefore, these screens can serve both external auditors and internal personnel concerned with the maintenance and improvement of data quality.

While previous investigators have developed simpler algorithms to detect egregious problems in data quality, this study demonstrates the power of combining a larger, more sophisticated set of screens. Selection of cases to be screened, codes to be included in screens, and criteria to identify problems and score aggregate performance are somewhat arbitrary. Whether data quality control personnel adopt these screens or develop similar sets of their own, the method of screening employed in these analyses provides the most cost-effective method of ensuring the quality of POA designations and the integrity of clinical performance analyses that utilize these data.

Conclusion

A set of 12 screens to assess the accuracy of POA designation was developed and applied to a large claims database. There was substantial variation among hospitals in the plausibility of their POA designations. The screening technique employed in this study represents a cost-effective method to ensure the credibility of claims data used for performance assessment, reimbursement, and quality improvement.

Michael Pine, MD, MBA, is the president of Michael Pine and Associates in Chicago, IL, and a lecturer in the Department of Medicine, Division of Cardiology, at the University of Chicago.Donald E. Fry, MD, is the executive vice president for clinical outcomes of Michael Pine and Associates in Chicago, IL; adjunct professor of surgery in the Feinberg School of Medicine at Northwestern University; and professor emeritus in the Department of Surgery at the University of New Mexico.Barbara Jones, MA, is the vice president of data management at Michael Pine and Associates in Chicago, IL. Roger Meimban, PhD, is a senior SAS programmer at Michael Pine and Associates in Chicago, IL.

Notes

1. Naessens, J. M., M. D. Brennan, C. J. Boberg, et al. “Acquired Conditions: An Improvement to Hospital Discharge Abstracts.” Quality Assurance in Health Care 3, no. 4 (1991): 257–62.
2. Coffey, R., M. Milenkovic, and R. M. Andrews. The Case for the Present-on-Admission (POA) Indicator (HCUP Methods Series Report No. 2006-01). June 26, 2006. U.S. Agency for Healthcare Research and Quality. Available at http://www.hcup-us.ahrq.gov/reports/2006_1.pdf.
3. Bowman, S. “Are You Ready for POA Reporting?” Journal of AHIMA 78, no. 1 (2007): 70–72.
4. Hughes, J. S., R. F. Averill, N. I. Goldfield, et al. “Identifying Potentially Preventable Complications Using a Present on Admission Indicator.” Health Care Financing Review 27, no. 3 (2006): 63–82.
5. National Center for Health Statistics. Present on Admission Reporting Guidelines. 2006. Available at http://www.cdc.gov/nchs/data/icd9/POAguideSep06.pdf.
6. Pine, M. “Crafting Valid, Relevant Measures of Clinical Performance.” In P. R. Kongstvedt and D. W. Plocher (Editors), Best Practices in Medical Management. Gaithersburg, MD: Aspen Publishers, 1998, chapter 35.
7. National Center for Health Statistics. Present on Admission Reporting Guidelines.