Publication Type |
Journal Article |
School or College |
College of Engineering |
Department |
Computing, School of |
Creator |
Riloff, Ellen M. |
Other Author |
Light, Marc; Mann, Gideon S.; Breck, Eric |
Title |
Analyses for elucidating current question answering technology |
Date |
2001-12 |
Description |
In this paper, we take a detailed look at the performance of components of an idealized question answering system on two diff erent tasks: the TREC Question Answering task and a set of reading comprehension exams. We carry out three types of analysis: inherent properties of the data, feature analysis, and performance bounds. Based on these analyses we explain some of the performance results of the current generation of Q/A systems and make predictions on future work. In particular, we present four fi ndings: (1) Q/A system performance is correlated with answer repetition; (2) relative overlap scores are more effective than absolute overlap scores; (3) equivalence classes on scoring functions can be used to quantify performance bounds; and (4) perfect answer typing still leaves a great deal of ambiguity for a Q/A system because sentences often contain several items of the same type. |
Type |
Text |
Publisher |
Cambridge University Press |
Journal Title |
Natural Language Engineering |
Volume |
7 |
Issue |
4 |
First Page |
325 |
Last Page |
342 |
DOI |
10.1017/S1351324901002819 |
citatation_issn |
1351-3249 |
Subject |
TREC Question Answering task |
Subject LCSH |
Question-answering systems |
Language |
eng |
Bibliographic Citation |
Light, M., Mann, G. S., Riloff, E. M., & Breck, E. (2001). Analyses for elucidating current question answering technology. Journal of Natural Language Engineering, 7(4), 325-42. |
Series |
University of Utah Computer Science Technical Report |
Rights Management |
(c) Cambridge University Press http://dx.doi.org/10.1017/S1351324901002819 Permission granted by Cambridge University Press for non-commercial, personal use only. |
Format Medium |
application/pdf |
Format Extent |
213,578 bytes |
Identifier |
ir-main,12452 |
ARK |
ark:/87278/s6b85sq6 |
Setname |
ir_uspace |
ID |
706669 |
Reference URL |
https://collections.lib.utah.edu/ark:/87278/s6b85sq6 |