A diagnostic expert system, Iliad, as a quality review screen.

A diagnostic expert system, Iliad, was evaluated as a quality review tool in the Medicare inpatient review performed by Peer Review Organizations (PROs). The current PRO review appeared to underdetect diagnostic errors. The PRO review started with a nurse review using generic quality screens. Only cases flagged by the nurse review as containing a potential quality problem were then referred for a physician review. The quality screens focused only on treatment, discharge and documentation. Physician review, directed by the concerns raised in the nurse review, might also have failed to detect diagnostic errors. Thus, the expert diagnostic capability of Iliad could be useful as a nurse review screen for diagnostic errors. A discrepancy between Iliad's diagnosis and that made by the attending physician for a case would flag a potential diagnostic error. The case could then be referred to a physician for confirmation. Two experiments were performed. The first compared Iliad review with the current PRO review. On 100 Medicare inpatient cases flagged by the Utah Peer Review Organization (UPRO) nurse review as containing potential problems, and which contained diagnoses recognized by Iliad, Iliad review found 17% of the cases to have contained a diagnostic error, significantly higher than the 0% Diagnostic Error Rate (DER) found by UPRO. The Iliad nurse review True Positive Rate (TPR) was also significantly higher than that of the UPRO nurse review. The second experiment compared Iliad review to the computerized UCDS (Unified Clinical Data Set) review undergoing testing at UPRO. In a random sample of 326 Utah Medicare inpatient cases containing diagnoses recognized by Iliad, Iliad review found a significantly higher DER of 6%, compared to that found by the UCDS review (0.8%). The Iliad nurse review TPR was also significantly higher than that of the UCDS nurse review. The effects of varying the 'strictness' of Iliad review, different diagnosis matching strategies, different nurse reviewers, and different physician reviewers were explored. A cost-benefit estimation was also performed. It was concluded that Iliad review was effective and efficient in detecting diagnostic errors causing quality of care problems in Medicare hospitalizations.