Using natural language processing to assist automatic decision support systems and quality assurance in radiology

The output of a natural language processor can be used to generate higher level concepts useful for decision support and quality assurance in radiology. This dissertation describes three studies in which the output of a natural language processor called SymText was used to generate concepts that can be used to support automated clinical systems. All three studies addressed acute bacterial pneumonia and used a test set of 292 chest x-ray reports from the HELP hospital information system at LDS Hospital. The first study used SymText's output to determine whether radiologic evidence of pneumonia existed in chest x-ray reports. The accuracy of a rule-based system, a probabilistic system (Bayesian network), and a machine learning system (decision tree) were compared. All three systems performed similarly to each other and to physicians. Second, the usefulness of SymText's output at identifying characteristics of clear chest x-ray reports was examined. Thirty percent (89/292) of the reports were disagreed on by at least one of seven physicians. Reports were categorized by the number of dissenting votes the reports received. Reports with zero dissenting votes were considered the most clear; reports with three dissenting votes were considered the least clear. Using a corrected version of SymText's output, characteristics from the radiologic literature believed to be associated with clarity were quantified and analyzed with ordinal logistic regression. Five characteristics generated by SymText's output were significantly associated with clear reports. Third, the accuracy of clarity characteristics generated from SymText's uncorrected output was evaluated. The variable "interpretive report" was very accurately generated. However, the remaining characteristics were not accurately generated, in part because reports that confused physicians also confused SymText; SymText's accuracy significantly decreased on unclear reports. In summary, SymText's output can be combined with an expert system to generate an inference of radiologic support for pneumonia. Additionally, SymText's output can be used to quantify characteristics of clear chest x-ray reports that could be used for quality assurance or education in radiology. However, SymText is not currently accurate enough, especially on unclear reports, to be used to automatically generate the characteristics.