Assessing Coder Change Rates as an Evaluation Metric
Author: Stoner, Jean; Nossal, Michael; Resnik, Philip; Kapit, Andrew; Toren, Richard
Source: AHIMA Computer-Assisted Coding Meeting | Perspectives in Health Information Management
Publication Date: September 2006
In a typical model for computer-assisted coding, the system automatically identifies codes, which are then improved by a human review coder in order to more closely approach the codes that would be assigned to the note by an idealized "gold standard" coder. Assuming such a model, a natural method for evaluating system accuracy might be the coder change rate—specifically, the percentage of notes approved by a coder without modification in a day-to -day business setting. In this paper we look more carefully at change rates as a measure of coding accuracy.
Using Intrinsic and Extrinsic Metrics to Evaluate Accuracy and Facilitation in Computer-Assisted Coding
Author: Resnik, Philip; Niv, Michael; Nossal, Michael; Schnitzer, Gregory L.; Stoner, Jean; Kapit, Andrew; Toren, Richard
Source: AHIMA Computer-Assisted Coding Meeting
Publication Date: September 06, 2006
Computer-assisted coding (CAC) evaluation can be viewed from two related but distinct perspectives: accurately identifying codes, and more generally facilitating the process of human coding and information flow within and among healthcare organizations. The first perspective calls for intrinsic metrics, grounded in comparison with a gold standard. The second perspective calls for extrinsic techniques, grounded in the
real-world task. We discuss this distinction and present an experimental evaluation of CAC using both intrinsic and extrinsic measures.