An Interlingua for Electronic Interchange of Medical Information: Using Fames to Map between Clinical Vocabularies

Biomedical Informatics

COMPUTERS AND BIOMEDICAL RESEARCH 24, 379-400 (1991) An lnterlingua for Electronic Interchange of Medical Information: Using Frames to Map between Clinical Vocabularies FRED E. MASARIE, JR.,* RANDOLPH A. MILLER, *·:j: 0MAR BOUHADDOU, t NUNZIA B. GIUSE,* AND HOMER R. WARNERt *Section of Medical Informatics, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, and tDepartment of Medical Informatics, University of Utah, Salt Lake City, Utah 84112 Received September 27, 1990 The proliferation ofmedical knowledge ha:s led to the development of extensive dictionaries for electronically accessing information resources. The task of standardizing terminology used for electronic hospital records and for knowledge bases for medical expert systems and indexing the medical literature cannot easily be met by developing a single, monolithic ''official'' medical vocabulary. Developing a monolithic vocabulary would require a massive effort, and its existence would not guarantee its use by third-party payors, by practicing clinicians, or by developers of electronic medical information systems. Recognizing this, the National Library of Medicine (NLM) has begun to develop the Unified Medical Language System (UMLS) as a means of promoting electronic information exchange among systems with controlled vocabularies. The authors describe a frame-based system developed as an experimental approach to mapping between controlled clinical vocabularies. © 1991 Academic Press, Inc. 1. BACKGROUND The need for standardization in medical terminology has been long recog­nized. Several large, disparate (but partially overlapping) controlled medical vocabularies have been developed over the years. At present, the National Library of Medicine (NLM) is developing the Unified Medical Language System (UMLS) in order to facilitate electronic information exchange among systems with controlled vocabularies (J). Some of the first controlled vocabularies were developed as classification schemes to facilitate statistical analysis of disease information (2, 3). The World Health Organization (WHO) assumed responsibility in 1946 for periodically t Author to whom correspondence should be addressed at Section of Medical Informatics, B50A Lothrop Hall, 190 Lothrop St., University of Pittsburgh, Pittsburgh, PA 15261. 379 - 0010-4809/91 $3.~ Copyright © 1991 by Academi~ P•ess,lt1e. All rights of reproduction in any form reserved. 380 MASARIE ET AL. revising a Qis.ease classification scheme originally developed by Dr. Jacques Bertillon in 1893. The result of the WHO decennial conferences has been the Manual of the International Statistical Classification of Diseases, Injuries, and Causes of Death (ICD) which is now undergoing its tenth revision (3). In describing the ICD system, Cote makes the distinction between "a classifica­tion, which is a systematic division of a series of related phenomena into groups or classes, and a nomenclature, which is a system of names used in a particular branch of knowledge" (4). He contends that the ICD classification scheme is not adequately robust to be useful in clinical settings. Recognizing that ICD had some deficiencies, the College of American Pathol­ogists independently developed the Systematized Nomenclature of Pathology (SNOP) in 1965. SNOP is based on a multiaxial coding scheme allowing for a rich network of anatomical, etiological, and functional concepts. SNOP has been well accepted in the realm of pathologic diagnosis (5). In 1979, the College of American Pathologists published the more comprehensive Systematized No­menclature of Medicine (SNOMED) which expanded the SNOP coding scheme to include nomenclature relevant to other medical disciplines (6). The SNOMED system of terminology has been used successfully in a number of settings, including automated autopsy protocols, medical auditing, and regional patient data banking (7, 8). Even with a number of enthusiastic supporters, the SNOMED vocabulary has not yet reached the status of an "industry standard." It is possible that in its future releases, the improved SNOMED vocabulary will become such a standard. Another stimulus for the development of standardized medical vocabularies has been the desire to index and retrieve articles in the medical literature. The Medical Subject Heading (MeSH) vocabulary has evolved at the National Library of Medicine (NLM) as a controlled keyword vocabulary for indexing biomedical literature. MeSH was developed in the early 1960s as part of prepara­tion for automating the production of its written precursor, Index Medicus. Since then, MeSH has been continuously revised and expanded to keep up with the growing body of medical information. Since the 1960s, MeSH has been employed as an on-line tool to index and retrieve millions of articles in the computerized MEDLARS bibliographic database retrieval system (9, 10). The Quick Medical Reference (QMR)(R) vocabulary, which is a superset of the original INTERNIST-I vocabulary, is an example of a large controlled medical vocabulary created for use in a diagnostic decision support system (11-14). The QMR vocabulary terms are a key component of the QMR knowl­edge base. The wording of the vocabulary terms has been carefully supervised by Dr. Jack Myers, the principal developer of the INTERNIST-I project and a key participant in the Quick Medical Reference project. Myers chose clinical terms for the INTERNIST-I and QMR vocabularies based on their ability to capture clinical diagnostic concepts concisely. Although the terminologies used in the INTERNIST-I and QMR knowledge bases have been exact and specific, they rarely have paralleled natural language syntax. In a 1983-1984 evaluation of the ease of use of the INTERNIST-I I l 1 1 I l I J I '1 I 'J 1 I I 1 A FRAME-BASED INTERLINGUA 381 vocabulary as part of an "on-line" textbook for residents, Dr. Michael First found that naive users had significant difficulty in accessing the INTERNIST- 1 knowledge base through its controlled vocabulary (15), even when it was supplemented by a spelling corrector and a synonym vocabulary. This discovery led to the additional development of a key-word completer and the use of pull­down menus as alternative methods of identifying vocabulary terms within the QMR program. Additional problems in the INTERNIST-I vocabulary were analyzed by R. Miller from 1981 to 1983. At that time, he extended the INTERNIST-I knowledge base (KB) to facilitate generation of logically consis­tent artificial patient cases for teaching diagnosis to medical students. The resulting computer program was called the Clinical Patient Case Simulator (CPCS) (16, 17). The CPCS project involved significant restructuring of the INTERNIST-I knowledge base in order to represent disease concepts more thoroughly. In order to generate logically consistent patient cases, information regarding pathophysiological mechanisms and syndromes, as well as severity of illness, had to be added as superstructure to the existing INTERNIST-I knowledge base. A key feature of a clinical vocabulary, required for synthesis of realistic patient case simulations, is explicit representation of standard (or default) values for "normal" findings (i.e., the usual state of a patient parameter in "healthy" individuals). The original INTERNIST-I terminology did not possess this char­acteristic. Of note, many of the other major controlled medical vocabula­ries- including the MeSH, ICD-9, SNOMED, HELP (18), and DXplain (19) terminologies-do not explicitly represent ''normal'' values for patient parame­ters either. Although it may be possible to represent default "normal" findings in some of these systems, their relationship to corresponding "abnormal" find­ings is not well represented for computational purposes. For example, the finding "melena" is abnormal; its corresponding "normal" findings are "feces normal on gross inspection" and "feces guaiac test negative." Few controlled vocabularies represent such relationships between terms explicitly, even though it may be possible to construct each of the text strings (or their synonyms) in a given controlled vocabulary. Not many systems, when presented with the expression, ''the patient had intermittent right upper quadrant pain but no other abdominal pain," would be able to answer the question "did the patient have colicky left lower quadrant pain?'' This problem was partially solved through the CPCS project. Another problem with the INTERNIST-I terminology for patient findings was that several separate findings existed, often with quite different terminol­ogy, which described attributed or possible states of a single medical concept. For example, the concept "liver size" was represented in INTERNIST-I via the findings "LIVER SMALL BY PERCUSSION," "HEPATOMEGALY PRESENT," "LIVER ENLARGED SLIGHT," "LIVER ENLARGED MODERATE," and "LIVER ENLARGED MASSIVE." Solutions to the prob­lems of representing medical concepts as discrete entities and representing "normal" findings explicitly were implemented in CPCS. The solutions involved 382 MASARIE ET AL. reorganization of the INTERNIST-I manifestations (patient findings) into find­ing concept frames (also called "generic concept frames," or simply "generic frames"). The CPCS generic frames allowed concepts implicit in the INTERNIST-I manifestations to be more directly represented. For example, there are over 60 INTERNIST-I manifestations beginning with "ABDOMEN PAIN .... " Thus, to indicate that a patient had acute periumbilical colicky abdominal pain, the INTERNIST-I user would identify three separate manifestations, namely ABDOMEN PAIN ACUTE, ABDOMEN PAIN PERIUMBILICAL, and ABDOMEN PAIN COLICKY. Ifthe patient also had chronic epigastric pain, the user would additionally identify the INTERNIST-I manifestations ABDOMEN PAIN CHRONIC and ABDOMEN PAIN EPIGASTRIUM. Thus, the INTERNIST-I system would be presented with five separate mani­festations of abdominal pain and had no intrinsic way of clarifying or inter­nally representing whether the epigastric pain was acute or chronic. The addition of finding concept frames in the CPCS project allowed the program and the user to refine medical concepts such as "abdominal pain" more directly through the use of qualifiers like ''site of abdominal pain,'' ''severity of pain," and "clinical time course." Each qualifier could be used to explic­itly modify a specific central concept (e.g., abdominal pain). In 1984, Dr. Masarie manually reviewed the existing INTERNIST-I patient findings, attempting to identify names for potential generic finding frames that could represent each finding (without actually constructing the generic frames). He estimated that the entire set of approximately 4000 INTERNIST-I findings could be captured using approximately 1800 finding concept frames. Using the limited number of finding concept frames actually created during the CPCS project, it was possible for the CPCS program to synthesize realistic patient case scenarios that took into account severity of illness, pathophysiology, and default "normal" values for findings (16, 17). UsE OF FINDING CONCEPT FRAMES AS AN INTERLINGUA FOR CoNTROLLED VocABULARIES With the initiation of the Unified Medical Language System Project by the National Library of Medicine in 1986-1987 (1 ) , Miller and Masarie realized that the finding concept frames created during the CPCS project might be modified and expanded to serve as an interlingua between controlled clinical vocabula­ries. Of note, the term "interlingua" is a relatively recent addition to the vocabulary of computer scientists and linguists. However, it has gained common usage, even in the lay press. The term"interlingua" and the utility of the in terlingua concept were discussed in an article, ''Computers Gain New Respect As Translators'', which appeared in the Science Times section of the New York Times (20). The use of frames originated with Quillian's implementation of Minsky's theory of frames for natural language understanding (21, 22). Brach­man significantly added to the understanding of frames, the use of frames in A FRAME-BASED INTERLINGUA 383 lassification systems, and the limitationsofframe-based systems (23,24). His ~L-ONE system was a seminal work in the field. During the 1980s, many of the approaches to machine translation between languages such as English, German, and Japanese used an interlingua to internally represent the concepts expressed by text from the target vocabularies (25). When word-by-word or phrase-by-phrase methodologies are used to translate between spoken lan­guages, idiomatic expressions such as "I caught a cold" and "I can see it in my mind's eye" are often poorly handled. In such an approach, an interlingua is created that can represent the meaning and/or definition of expressions in source vocabularies (such as "contracting a viral upper respiratory infection"). Translations from the source vocabularies to the interlingua are created on a pairwise basis. By translating a source vocabulary utterance into the interlingua, and then mapping from the central concepts represented in the interlingua to the appropriate terminology in the target vocabulary, the in terlingua approach offers the potential to overcome the problem of translating most idioms. An added advantage of the interlingua approach is that when translations between large numbers of source vocabularies are desired, the burden of the translations is significantly diminished. While mapping between controlled vo­cabularies can be accomplished straightforwardly using brute force (by de­termining for each term in one vocabulary its closest match in another), the brute force technique only works reasonably well for a "world" of two vocabularies. However, when the brute force method is used for direct pairwise translation between N source vocabularies, the number of pairwise translation tables re­quired is N * (N- 1)/2. Given this combinatorial explosion, with four or more vocabularies it becomes clear that a more extensible solution must be found. Fortunately, using the interlingua approach, only N translation tables are re­quired to translate among N separate vocabularies, one from each source vocab­ulary into the interlingua. Figure 1 illustrates graphically how the interlingua approach simplifies the burden oftranslation when many languages are involved. METHODS Generic Concept Frames Database The authors have created an experimental frame-based interlingua for map­ping between clinical vocabularies. In doing so, several simplifying assumptions were made. The authors recognized that medical terminology can be loosely organized into three categories: clinical observations or attributes (e.g., tachyp­nea, blood pH 7.2); clinical interpretations (e.g., metabolic acidosis, dehydra­tion); and clinical diagnoses (e.g., Type I Diabetes Mellitus presenting with diabetic ketoacidosis). These distinctions, although somewhat arbitrary, help in approaching the problem of mapping between vocabularies. The authors believe that clinical observations serve as the building blocks from which higher level concepts (interpretations and diagnoses) are defined. For example, when asked to define metabolic acidosis, a clinician usually describes the constellation of clinical attributes associated with the interpretation. Therefore, the first step 384 MASARIE ET AL. Number of Pairs = 15 Brute Force: Pairwise Translation Number of Pairs = 6 Interlingua Approach: Tramlation thrwgh central intermediar FIG. l. Combinatorial advantages of the interlingua approach to translation. in creating an interlingua between clinical terminologies should address the level of observable clinical manifestations of disease. The authors initially focused on the terminology of clinical manifestations of disease and developed a prototype system for translating between disparate clinical vocabularies. Four large controlled vocabularies-QMR finding names; relevant HELP PTXT terms (18); DXplain (19) finding names; and Medical Subject Headings (MeSH)-were chosen as a test environment. To further restrict the domain to a size conducive to experimentation, the initial prototype was limited to manifestations (history, symptoms, signs, and laboratory tests) of cardiopulmonary diseases. In developing the frame-based finding concept representation of clinical manifestations a final assumption was made-that clinically relevant statements about patients contain at least one identifiable central concept and central concepts can serve as focus for mapping between medical vocabularies. Each vocabulary is organized differently. The QMR manifestation names consist of non parsed text strings which are represented internally by the QMR system as single unique numbers. They are often noun phrases with multiple modifiers. Several examples of QMR manifestation names are included in Fig. 2. r. . ~· A FRAME-BASED INTERLINGUA 385 ~: CIGARETTE SMOKING HX CHEST PAIN SUBSTERNAL BURNING RALES LOCALIZED CHOLESTEROL SERUM INCREASED ABDOMEN XRAY COLON DISTENTION WITH GAS PROLACTIN SERUM GTR THAN 100 NG PER ML FIG. 2. QMR manifestation names. The HELP PTXT dictionary is a compendium of terminology which can be used to encode patient information into electronic hospital records for the HELP hospital information system. The HELP system has been developed over the past 20 years at the Latter Day Saints Hospital in Salt Lake City. The dictionary is primarily hierarchically organized. Terms are represented internally by an eight (8) byte code which represents both simple semantic and syntactic informa­tion about the term. HELP PTXT terms tend to be more atomic than QMR manifestations since they can be concatenated in the patient record. A part of the PTXT dictionary is shown in Fig. 3. DXplain is a mainframe-based diagnostic program developed at the Labora­tory for Computer Science at the Massachusetts General Hospital in Boston (19). DXplain covers over 2000 medical disorders, using a controlled vocabulary of over 6000 findings. Its finding vocabulary is organized hierarchically but not to the extent of HELP PTXT. The terms tend to be more atomic than QMR, but not as atomic as HELP PTXT. 7 1 11 1 1 0 0 0 COUGH 7 1 11 2 1 1 0 0 CONSTANT 7 1 11 2 1 2 0 0 PAROXYSMAL 7 1 11 2 1 3 0 0 MILD 7 1 11 2 1 4 0 0 MODERATE 7 1 11 2 1 5 0 0 SEVERE 7 1 11 2 1 6 0 0 WORSE LYING DOWN 7 1 11 2 1 7 0 0 WORSE SITTING UP 7 1 11 2 1 8 0 0 NOT POSITIONALLY RELATED 7 1 11 2 1 9 0 0 NON-PRODUCTIVE 7 1 11 2 1 10 0 0 SPUWM PRODUCTION 7 1 11 3 1 10 1 0 MINIMAL AMOUNT 7 1 11 3 1 10 2 0 MODERATE AMOUNT 7 1 11 3 1 10 3 0 COPIOUS AMOUNT 7 1 11 3 1 10 4 0 WHITE FIG. 3. Part of the HELP PTXT dictionary. 386 MASARIE ET AL. The Medical Subject Heading (MeSH) vocabulary has evolved at the National Library of Medicine (NLM) as a controlled keyword vocabulary for indexing biomedical articles. In general, the MeSH vocabulary does not contain the depth of coverage as the other vocabularies in the study domain. MeSH terms tend to be quite atomic; for example, DYSPNEA and CHEST PAIN. After selecting the vocabularies to be used, and limiting the test domain to cardiopulmonary findings, the next step in creating the frame-based interlingua was identifying the clinical concepts that actually appeared as terms in the sample controlled vocabularies. By manually reviewing each cardiopulmonary finding, first in the QMR lexicon and then in the HELP vocabulary, a name for each concept encountered was created, serving as the basis for definition of each "generic concept frame." (At a later date, the cardiopulmonary findings of DXplain and MeSH were manually reviewed and generic frames for any concepts not represented in QMR or HELP were created). The process of creating generic frames involved parsing medical terminology into ''clinically meaningful units.'' The functional definition of'' clinically mean­ingful information'' employed was that if a medical student presenting a case· said that the patient's chief complaint was "X," and "X" was a valid medical concept name, an experienced physician would not be likely to laugh or to be confused about the patient's condition. For example, the term PAIN has little clinical relevance when taken alone, whereas the terms HEADACHE, CHEST PAIN, and ABDOMINAL PAIN provide the physician with enough information to begin formulating hypotheses. Therefore, the latter group of terms would be the central concepts around which "generic frames" would be built. Several examples further clarify what is meant by "valid" concept names. Consider three phrases: HEARTBURN, PLEURITIC PAIN, and ANGINA. What is their (common) central concept? Each term is really a specific instance (or instantiation) of the clinical manifestation Chest Pain. How are the terms LIVER SMALL BY PALPATION and HEPATOMEGALY PRESENT re­lated? They both are specific instances of the concept Liver Size. However, while ABNORMAL ELECTROLYTES is a valid medical concept, it is too vague to be used as a basic, definition-level generic concept name. More specific terms, such as Serum Sodium and Serum Potassium, are used as definition-level generic concept names. ABNORMAL ELECTROLYTES, however, would be a valid higher-level concept in a hierarchy that organized and classified the lower, more basic, definition-level generic concepts. Generic concept frames were developed by Drs. Masarie, Miller, and Giuse to serve as the interlingua between the clinical vocabularies used in the current study. As such, each generic frame is intended to capture ultimately all the lexical variations in terminology that might be used to describe a clinical con­cept. Generic frames have a superstructure and fine details. The fine details are encapsulated in the form of "item lists," which each have a name (e.g., "severity") and a list of allowed values (e.g., mild, moderate, severe). The superstructure of a generic frame includes its concept name, its status descrip­tor, its potential site descriptor, its potential subcategory descriptor, and its list ' I J 1 .1.. A FRAME-BASED INTERLINGUA 387 of potential qualifiers. The descriptors consist of item lists. In addition, each generic frame has an associated ''method of elicitation'' descriptor, or multiple "methods of elicitation" if there are multiple ways in which the finding can be measured-for example, liver size, which can be measured by percussion, palpation, by ultrasonography, by CT or MRI scanning, or by weight at autopsy. Slots are used to define the structural components of each generic concept; most slots can contain an item list name. The status slot is used to indicate how one would describe the state of the concept in a patient (or in a healthy individual)-for example, the status for ABDOMINAL PAIN was a list of the words "present" and "absent." The potential site slot can be empty (NIL) if a finding concept is not associated with a specific body location (for example, JAUNDICE), or can contain a named list of the sites where the finding might occur in a patient when naming such sites is clinically relevant. For ABDOMINAL PAIN, the site slot list name is "Superficial Abdominal Sites", and the list's elements include epigastrium, right upper quadrant, left upper quadrant, periumbilical area, suprapubic area, and so on. The subcategory slot, like the site slot, can be empty (NIL) for concept frames where it is irrelevant, but otherwise lists the important kinds (or subcategories) of occurrence of the finding that occur clinically. The site slot and subcategory slot can be viewed as specially reserved "most important qualifier" slots. For example, the HEART MURMUR concept frame includes a site descriptor list to describe the site of maximum intensity of the murmur, and a subcategory list to describe the timing of the murmur in the cardiac cycle-systolic, diastolic, or continuous. Thus, the generic frames provide a template for describing specific medical terms in a standardized manner. Once key concepts were identified on manual review as candidate names for "generic concept frames," the potential qualifiers for each diagnostic medical concept were collected from review of the test vocabularies. The qualifiers slot in a generic concept frame lists all the "adjective" or "adverb" phrases that might be used to clarify, or more specifically delineate a finding. Each qualifier itself consists of a list of the values (terms) that might be used to describe the qualifier's topic. For ABDOMINAL PAIN, "qualifier" values include the severity of pain, exacerbating factors, the pattern of radiation of the pain, and others (the site slot mentioned above is used for the site of the pain). Just as the possible values for the site slot are contained in a separately specified list, the allowed values for each qualifier, such as pain's possible exacerbating factors (meals, swallowing, fasting, postprandial timing, etc.) are also defined separately in item lists. A physician might describe a patient's chest pain as substernal (site slot), crushing (quality qualifier), radiating to the left arm (site of radiation qualifier), and unrelieved by nitroglycerin (influencing factors quali­fier). Figure 4 shows the generic frame for the concept CHEST PAIN. In constructing each generic frame, the authors attempted to codify exhaus­tively all of the clinically valid attributes for each concept, and to represent explicitly how physicians might "modify" the central clinical concept during medical discourse. As such, the generic frames are intended to represent defini- 388 MASARIE ET AL. Generic Concept: Chest Pain Status: Present or Absent Normal: Absent Site: Chest Topographic Site Subcategory: Nil Method: Cardiopulmonary Symptom Qualifiers: Chest Pain Quality Severity Chest Pain Radiation Qualitative Time Duration Quantitative Time Duration Influencing Factors FIG. 4. Sample generic frame. tions of all of the axes and terms that might be used to describe a medical concept. Functionally, the generic frames are made to be comprehensive through manual review of terms from each new ''target'' vocabulary. Whenever a term from a target vocabulary does not have a corresponding generic concept frame, a new concept frame is created. Whenever a qualifier (modifying or adjectival phrase) has a value in a target vocabulary which has not been repre­sented in the set of values allowed in the qualifier's (item list) definition within the generic frame system, the new value is added. The process described above was used to create over 750 generic frames for concepts represented in the test set of 1500 QMR manifestation terms. Between 90 and 95% of the 1500 terms can be represented using these 750 generic frames. The generic frames were extended to cover terms from the HELP, DXplain, and MeSH lexicon as described below. Each generic finding frame is, in effect, a definition for how clinicians might compose clinically meaningful patient descriptors with respect to a given patient parameter. Because generic concept frames must incorporate broad-based definitions for all the potential ways in which a concept might be modified, they can be used to construct non-sensical medical terms as well as the terms that make sense. For example, the generic frame for ABDOMINAL PAIN would allow one to describe an "epigastric," "colicky" abdominal pain that "radiates to the epigastrium," is "exacerbated by meals," and is "relieved by food." Instantiated Concept Frames Database Generic frames provide the templates for describing controlled vocabulary terms such as "Chest Pain Substernal Burning" or "Sharp Or Stabbing Chest Pain'' in a standardized format. At the next level of specificity, called an l r . . ;j: t~'· A FRAME-BASED INTERLINGUA Instantiated Concept: Chest Pain Substernal Relieved By Antacids (QMR) Generic Concept Name: Chest Pain Status: Present Site: Substernal Method: Cardiopulmo~ry Symptom Qualifiers: Influencing Factor: Relieved by Antacids FIG. 5. Sample of an instantiated frame. 389 instantiated frame, medical reality becomes a constraint. The process of select­ing specific values for the qualifiers and slot descriptors in a generic frame, in order to describe a particular patient finding, is referred to as building instanti­ated frames. Through this process, terms from "target" lexicons (such as HELP or QMR) are linked to the interlingua (the system of generic frames). Each instantiated frame (corresponding to a term in a target vocabulary) thus consti­tutes a definition for that target term, stated in the "controlled vocabulary" of the generic frame system. Instantiation involves identifying which portion of the information in a generic frame is applicable to a given controlled vocabulary term. Only vocabulary terms that occur (or might occur) in patients are allowed to be built as instantiated frames. The controlled vocabularies of the four systems used in the current project fulfill this requirement, as they contain no "nonsense" terminology. Figure 5 shows an example of an instantiated frame. The success of building instantiated frames for controlled vocabulary terms depends on the extent of the information included in the generic frames. The authors will briefly describe the process of building an instantiated frame for a controlled vocabulary term. First, the relevant generic concept name or names for the vocabulary term must be identified. In rare cases, one target term, such as "Dyspnea improvement following gross hemoptysis," may involve two generic concepts, such as dyspnea and sputum production. Mter the relevant generic concept name is identified, its associated generic frame then provides the template for describing the modifiers implicit or explicit in the target vocabulary term. The person building the instantiated frame is sequentially prompted to enter information about (1) the status of the target finding (the user selects one value from the status descriptor list specified in the generic frame definition); (2) the subcategory or site information (if included in the generic frame defini­tion); (3) the method used to elicit the finding (selected from a list of methods if more than one method exists in the generic concept definition); and (4) the relevant qualifiers, which then must be assigned appropriate values. The process of instantiation can be accomplished with variable degrees of success. Often, the terminology used for concept names and item names closely parallels that used in the sample controlled vocabularies. However, in many 390 MASARIE ET AL. cases, the names of generic frames capture an implied central concept, rather than one which is explicitly stated in the target vocabulary term. For example, breath sound character is the unifying concept for rales, rhonchi, and wheezes. Under these circumstances, it is difficult for the person building the instantiated frame to identify the central concept from the list of available generic concept names, especially if the individual did not participate in generic frame construc-tion. This problem cannot be solved by altering the terminology used within the frame system. The solution involves taking advantage of the organization of modifiers within generic frames. The instantiated frame corresponding to "chest pain substernal relieved by antacids'' appears in Fig. 5. Instantiated frames can be created manually using an Instantiated Frame Editor created by the authors. Following the steps out­lined above, this process takes about 2 min per term. Instantiated frames can also be created automatically by taking advantage of the highly structured nature of the generic frames. Experiments with Automatic Instantiation The goal of using the generic frame system as an interlingua for mapping between terms in target vocabularies involves two phases of construction. 1 During the initial review of all terms from the target vocabularies, generic 1 concept names are identified and the corresponding generic frame definitions are created. The second pass through target vocabulary terms uses the generic frames to create an instantiated frame corresponding to each target vocabulary term. Automatic instantiation is the process of building instantiated frames for medical phrases (controlled vocabulary terms or free text) in an automated fashion-i.e., with minimal human intervention. Mter generic frames have been constructed, terms from target medical vocabularies can be fed to the automatic instantiator, and the automatic instantiator program will attempt to construct instantiated concept frames which capture the intended meaning of the target vocabulary term. Mapping between separate controlled vocabularies is the result of finding the "best fit" between instantiated frames representing terms from separate target vocabularies. The success of our approach to mapping between medical lexicons depends on the following assumptions: 1. The generic concept database is exhaustive and adequate-that is, the database contains enough depth to cover all concepts expressed in the medical dictionaries mapped to it; 2. The descriptors (adjectival finding modifiers) associated with a concept (generic frame) represent the union of all descriptors used across the medical dictionaries involved; and 3. For every term in the participating dictionaries, it is possible to construct a frame representation (instantiated frame). r .. . ,t,l A FRAME-BASED INTERLINGUA 391 The first two hypotheses constitute the underlying principles adopted for the development of the medical concept frames database (i.e., generic frames, item lists, method lists). In other words, the concept frames database is not intended 10 stand independently as a medical vocabulary (i.e., to provide a complete, definitive medical lexicon in its own right). Instead, the concept frames represent an organized, highly structured collection of all semantic nuances as that occur in the real world, as exemplified in the lexicons that are included as "targets." Given a database developed following these guidelines, assumption three above is directly satisfied. This assumption amounts to mapping from a particu­lar element of a set to its location in that set. However, various refinements are required to overcome the problems of syntactic and semantic variations that exist among the allowed values of a field. The development of the database must be the result of successive revisions in order for an accurate representation to evolve-accurate with respect to the medical content, as well as to making the automatic instantiation process feasible. The automatic instantiation process involves parsing a text phrase from an external source (such as a controlled vocabulary) into its component words. To be useful to the automatic instantiation program, each word must ultimately correspond to some information in a generic frame. Useful words are those contained in the generic frames system thesaurus, which is composed of -the lexicon: the list of words collected from all mapped controlled vocab­ularies; -the concept list: the list of all medical diagnostic concepts; -the item list: the list of all qualifier names and qualifier values used in the generic frames to describe a concept; -the method list: the list of all methods used in the generic frames; -the synonym list: internal classes of words within the lexicon (e.g., abdomen/abdominal); and -the definition dictionary: a correspondence table between a word and a phrase (e.g., splenomegaly = spleen size increased). The automatic instantiation algorithm works as follows. First, the text phrase is broken into words, and stop words (e.g. "the," "of," "by") are eliminated. Words not found in the generic frames system lexicon are ignored (dropped from further consideration). Two steps, automatic concept identification and automatic qualifier identification, then follow. Finally, scoring procedures are employed to identify the best concept name, the appropriate item names, and optimal word groupings. Some heuristics are employed when ambiguities are encountered. These steps in automatic instantiation are described in detail below. Automatic Identification of Instantiated Frame Concepts Using hierarchical pointers (KWIC lists), the list of concepts potentially linked to each word in the text entry is determined. To illustrate this, Fig. 6 392 MASARIE ET AL. "Rates Basilar Fine" "Rates" (participates in the following list of generic frames) 1. Breath Sound Character "Basilar" -> ( = "Base") 1. Lung Perfusion Scan Uptake 2. Breath Sound Expiratory Phase Duration 3. Breath Sound Intensity 4. Chest Resonance To Percussion 5. Fremitus Intensity 6. Pleural friction Rub 7. Breath Sound Character "Fine"-> 1. Breath Sound Character FIG. 6. Mapping words in "rales basilar fine" to generic frames. shows how words in the phrase "Rales Basilar Fine" map to generic concept names such as ''Breath Sound Character,'''' Breath Sound Intensity,'' and other candidates. Next, a "concepts list" is built where each concept evoked in the previous step is represented only once, along with a count of the distinct words from the text entry that evoked the concept. This process considers mappings from the text entry phrase to words that appear in (a) a generic concept name; (b) the item list names and values associated with the generic finding's status, site and subtype; (c) the qualifier names associated with the generic concept definition; and (d) the values associated with each qualifier. In the example of Fig. 6, the merged list is (3) Breath Sound Character (1) Breath Sound Expiratory Phase Duration (1) Breath Sound Intensity (1) Chest Resonance To Percussion (1) Fremitus Intensity (1) Lung Perfusion Scan Uptake (1) Pleural Friction Rub The count facilitates identification of the generic concept name. If all the words in the phrase are related to the same concept, then the count associated with this concept is equal to the number of words in the phrase. Otherwise, the program employs heuristics in an attempt to differentiate between the different competitors. If application of the heuristics fails to identify a single (unambigu­ous) concept, the user is prompted to select a concept from the list of candidate concepts, thus resolving the ambiguity. In the example above, there is no A FRAME-BASED INTERLINGUA 393 ambiguity: "Breath Sound Character" is the only concept with a count equal to the number of words in the phrase. This is because the generic frame definition for Breath Sound Character includes a Subtype Slot, which is the item list, "Breath Sound Quality," and a Site Slot, which is the item list, "Site of pulmonary Examination." In turn, one of the allowed values for Breath Sound Quality is "Fine Crepitant Rales," and one of the allowed values for Site of Pulmonary Examination is "Base" (which maps via a lexical variants link to "Basilar"). This (first) step in automatic instantiation is completed when a concept is identified. The identified concept's constituent words are then re­moved from the entry phrase. In the example above, no words from the concept name appear in the phrase and therefore the program further investigates "Rales," "Basilar," and "Fine" as potential qualifiers (including site and sub­type) of the concept name "Breath Sound Character." Automatic Identification of Instantiated Frame Qualifiers Once the generic concept name is identified, the remaining words from the entry phrase are assumed to be associated with qualifiers. Identifying the quali­fiers consists of matching the words with the item names and/or item list values that have been defined as "allowed qualifiers" within the generic frame associated with the identified concept (e.g., "sharp" in the context of the concept "Chest Pain" is a"Chest pain quality"). This process is similar in design to the process in the first step of automatic instantiation, and uses the same routines. For each remaining word, a list is constructed containing all the item names that both include the word and that are used in the generic concept frame. Mapping Between Controlled Vocabularies Using Instantiated Frames By representing all controlled vocabulary terms as instantiated frames based on clinically meaningful concepts, the authors have been able to develop map­ping routines that allow comparison of intermediate representations (instanti­ated frames) for similarities. The net result is the ability to "translate" between medical vocabularies by finding the best-matched pairs of instantiated frame representations for terms from two target vocabularies. The translation routines assess the degree of match between terms taken from target vocabularies by comparing the instantiated frames that represent the terms. The routines assess the degree of match from three perspectives: COARSE MATCHING (noting differences in scope, where one term is simply broader than the other term-e.g., "chest pain" is conceptually broader than "chest pain substernal"); FINE TUNING (recognizing semantic differences, which occur when one lexi­con uses a term synonymous with or closely related to a term in the other lexicon, in order to describe a concept-e.g., "stabbing chest pain" and "knife­like chest pain"); and, COMPOSITE MAPPING (aggregation of one vocabu- 394 MASARIE ET AL. Example of COARSE mapping: (QMR) Chest Pain Substernal Relieved By Antacid (HELP) Chest Pain Substernal Examples of FINE mapping: (QMR) Chest Pain Substernal Knife-Like Or Tearing (HELP) Sharp Or Stabbing Chest Pain (QMR) Chest Pain Substernal Unrelieved By Nitroglycerin (HELP) Chest Pain Relieved By Nitroglycerin Example of COMPOSITE mapping: (a single term from one vocabulary mapping to several terms in another) (QMR) Chest Pain Substernal Relieved By Nitroglycerin (HELP) Chest Pain Substernal Chest Pain Relieved By Nitroglycerin FIG. 7. Different mapping perspectives. lary' s atomic terms to match another vocabulary's more complex phrases-e.g., combining "chest pain substernal" and "chest pain relieved by nitroglycerin" from one lexicon to match "substernal chest pain relieved by nitroglycerin" in another lexicon). Figure 7 includes examples of these different perspectives. Routines were developed which allowed system users to ask which terms in a target vocabulary (i.e., HELP) matched a specific term in a source vocabulary (i.e, QMR). Scoring algorithms were developed empirically and remain to be tested rigorously as our data base increases in size and complexity. A script of one such match is included in Figure 8. The scores in front of each target vocabulary term reflect its "degree of match" with the source term (refer to Fig. 8). A score which is a multiple of 100 is considered an exact match. Target terms are penalized by - 5 points if they are more or less specific than the source term for a given descriptor (i.e., when the COARSE MATCHING routine detects a difference in scope). Target terms which address the concept name, status, site, and subtype as the source term are given a score which reflects the level of conflict or match with the source term's qualifiers. A distance metric was developed which penalizes terms that are not exact matches at the FINE TUNING level. Thus, "knife-like" pain and ''sharp or stabbing'' pain might be rated as 7 5% similar, whereas ''dull pain'' and "sharp or stabbing pain" might be rated as 100% dissimilar (opposites). RESULTS Examples of Automatic Instantiation The following examples are from experiments with three medical controlled vocabularies: HELP, QMR, and DXplain. The examples fall into three separate categories: successful instantiations; examples which show some imperfections A FRAME-BASED INTERLINGUA Mapping from QMR to HELP: "Chest Pain Lateral Sharp" Potential Matches identified by mapping algorithm (first pass) 95 Chest Pain, Left-sided 95 Chest Pain, Right-sided 95 Sharp Or Stabbing Chest Pain 90 Pleuritic Chest Pain (with Breathing) 90 Pleuritic Chest Pain (with Coughing) Mapping algorithm identifies the following pairs (taken together), with each pair as an exact match for the Source term: Chest Pain, Left-sided Sharp Or Stabbing Chest Pain Chest Pain, Right-sided Sharp Or Stabbing Chest Pain FIG. 8. Sample match. 395 of the present generic frame database; and examples which point out some limitations of the algorithm/thesaurus used. The first set of instantiated frames, illustrated in Fig. 9, present cases where the automatic instantiations are as accurate as if they were manually built. The second set of examples, shown below, illustrate cases where the auto­matic instantiator was only partially successful, as the result of incomplete generic frame definitions or inadequacies in the concept hierarchy. "Notes" explain the categories of problems encountered. "Cough paroxysmal" (DXplain) CONCEPT NAME: Cough METHOD NAME: Cardiopulmonary Observation Note. The automatic instantiator was unable to determine the meaning of "paroxysmal," which did not appear in generic frame definition for "cough." To correct this problem, the authors should add "paroxysmal" as a timing qualifier to the ''cough'' frame. "Heart sound irregularity" (DXplain) Note. The automatic instantiator was unable to determine the meaning of "heart sound irregularity," and did not match it to the generic frame "heart sound.'' In this case, it is not clear whether the term ''irregularity'' is (medically speaking) so vague as to be meaningless (and therefore not worth representing in the "heart sound" frame), or whether "irregularity" should be considered a synonym for "abnormal" in the finding's "status" slot. 396 MASARIE ET AL. "Splenomegaly" (HELP) CONCEPT NAME : Spleen Size DIRECTION OF CHANGE : Increased METIIOD NAME : Unspecified "Is your chest pain increased by breathing deeply" (HELP) CONCEPT NAME : Chest Pain INFLUENCE ON CHEST PAIN TYPE OF INFLUENCE : Initiated or Exacerbated by FACTOR INFLUENCING CHEST PAIN : Breathing METIIOD NAME : Cardiopulmonary Symptom "Chest Pain Pleuritic" (DXplain) CONCEPT NAME : Chest Pain INFLUENCE ON CHEST PAIN TYPE OF INFLUENCE : Initiated or Exacerbated by FACTOR AFFECTING CHEST PAIN: Breathing METIIOD NAME : Cardiopulmonary Symptom "Conjunctival Bleeding" (DXplain) CONCEPT NAME : Conjunctival appearance TYPE OF CONJUNCTIVAL ABNORMALITY : Large Hemorrhage METIIOD NAME: Eye inspection "Abdominal Bruit Systolic Epigastrium" (QMR) CONCEPT NAME : Abdominal Bruit TIMING WITIIIN CARDIAC CYCLE: Systolic ABDOMINAL TOPOGRAPHIC SITE: Epigastrium FIG. 9. Examples where automatic instantiator functioned accurately. "Drug Abuse Hx" (QMR) Note. Here, the automatic instantiator could not determine which generic frame was more appropriate: "Drug Chronic Abuse or Overuse History" or "Drug IV Abuse flistory"-not enough input information was provided to resolve this dilemma. The last two examples show some of the limitations of the present approach to automatic instantiation. The authors could resolve some of these problems by embedding the generic frame concepts in a multilevel hierarchy. Such a hierarchy would have a higher level node for "drug abuse" with subnodes for "drug chronic abuse or overuse history" and "drug IV abuse history." The 1 A FRAME-BASED INTERLINGUA 397 TABLE 1 SAMPLE AUTOMATIC INSTANTIATOR PERFORMANCE ON VARIOUS CONTROLLED VOCABULARIES Average Number of Number of time to terms able terms with Automatic Overall process Number of to be automatic vs. automatic term, sec, terms instantiated matching manual,% success 80286 CPU examined manually manual matching rate,% HELP 12 87 43 34 79 39 DXplain 12 148 92 86 93 58 QMR 9 287 234 220 94 77 automatic instantiator could then map "drug abuse hx" unambiguously to the higher level concept node, if such a hierarchy were to be constructed. ''Do you get short of breath with exertion'' (HELP) Note.The automatic instantiator failed to identify the correct generic frame concept, dyspnea, because the synonym mapper was not capable of matching multi-word terms (short of breath) with single word terms (dyspnea). The tech­nology to do this is not complex, but has not yet been implemented as part of the automatic instantiator or the generic frames system. "Spine xray vertebral body (ies) sclerosis" (QMR) CONCEPT NAME: Vertebral body erosion Note. Here the program identified the wrong generic frame, vertebral body erosion, because the correct generic frame, vertebral body sclerosis, had not yet been constructed. The algorithm to find a best match did not perform incorrectly, as the absence of the correct term in the concepts database made the match found the "best" one. The example points out the need for "post hoc'' checking to make sure that a match is valid and does not contain un­matched words not understood by the system. Performance The performance of the automatic instantiation program measured in terms of speed and accuracy on a random sample of terms from HELP, DXplain, and QMR is presented in Table 1. In conclusion, the automatic instantiation program in its present state reduces significantly the instantiation effort from approximately 120 sec per term (manu­ally) to about 12 sec per term (automatic instantiator). Such a reduction in effort can be achieved for between 39 and 77% of all the terms in a controlled clinical vocabulary. This should provide an appropriate stimulus for instantiating a large number of controlled dictionaries. Another stimulus would come from the design of experiments that would demonstrate the usefulness of intervocabulary map- 398 MASARIE ET AL. ping (e.g., obtaining DXplain and QMR consultations on a set of HELP patient cases automatically translated from one dictionary to the others). The automatic instantiation program provides a fast and exhaustive way to browse through the frame associated data structures. The building and refine­ment of generic frames has been significantly improved through using the auto­matic instantiation program. Indeed, once a first generation of generic frames has been created manually from a specific "target" lexicon (e.g., QMR), the program can be run on a new "target" dictionary (e.g., HELP or DXplain) in order to point out necessary modifications to the generic frames. The modifica­tions indicated by the failures of the automatic instantiator are those which are needed to cover or include the terms of the new dictionary(ies). Also, when updates are made to the generic frame database, the automatic instantiation process has been used to verify the validity of the existing instantiated frames. In summary, the automatic instantiation program, through its functionality, validates the authors' hypothesis that a frame-based system is useful for map­ping among target medical vocabularies. The automatic instantiator provides the added benefit of facilitating generic frame construction, validation, and maintenance. CONCLUSIONS The authors have developed and tested, in a preliminary manner, a frame­based system for mapping among controlled clinical vocabularies. The potential for this methodology to enhance mapping between a large number of medical lexicons in a manner that avoids combinatorical complexity has been demon­strated. Nevertheless, there are a number of significant limitations in the frame­based approach described in this report. First, construction of generic frames is extremely labor intensive, and requires the efforts of a number of clinically experienced physicians. The authors devoted two years to creating a system that covered cardiopulmonary manifestations of disease; to cover all clinical findings in a broad field such as internal medicine would require at least five to ten years of effort. Second, it is difficult to construct generic frames in a "standardized" manner, i.e., in such a way that all observers intuitively accept the names selected for generic frames. It is also difficult to obtain general consensus on the definitions for those frames as being appropriate or correct. This problem occurs when any group attempts to create a standard lexicon to be shared among a diverse set of individuals. Third, the generic frames and their instantiated counterparts are difficult to maintain. The target vocabularies represented in the instantiated frames all continue to evolve independently, creating the need for frequent and vigilant updating. This problem is both a strength and a weakness of the generic frame system described by the authors, since the automatic instantiator provides at least one tool which can lessen this cumbersome burden. Despite the limitations of the authors' approach, there are a number of inter- ·1···0. .·....·. ·· ) ' I A FRAME-flASED INTERLINGUA 399 esting avenues for future research extending the generic frames utility and applicability. An additional use of the automatic instantiated program as a medical language processor remains to be explored. Indeed, the automatic instantiation could be used to dynamically capture a "free text" input into the generic frame structure. This would facilitate "free text" queries of instantiated dictionaries (e.g., input of patient cases in QMR or DXplain; query of HELP patient database or of MeSH indexed literature databases). The group at the University of Utah, through its ongoing involvement in the UMLS project, is exploring how to recast the generic frame system into a more readily useable format that can be more easily maintained (26). Investigators at the University of Pittsburgh plan to explore the utility of the frame system and the automatic instantiator as a means of recognizing terms from controlled medical vocabula­ries embedded in existing clinical records (e.g., hospital charts). One of the most innovative applications of the generic frames system described in this report has been the efforts of Fagan, Shiffman, et al. at Stanford University to adopt the frame system as part of a speech recognition system. The goal is to facilitate capturing spoken input in the form of a controlled medical vocabulary (27). The generic frame system has been demonstrated to have interesting potential as an interlingua to facilitate mapping among electronic medical vocabularies. It also has potential applicability in the fields of speech recognition and pro­cessing of clinical records. Experiments in progress at a number of universities will ultimately determine the true utility and generalizeability of this approach. AcKNOWLEDGMENTS Dr. Miller's work was partially supported through National Library of Medicine (NLM) New Investigator Award R23-LM-03589 and through Research Career Development Award K04-LM- 00084. NLM Contract N01-LM-6-3522 supported work on the UMLS project. The DXplain vocabu­lary was contributed to the UMLS effort by Drs. G.Octo Barnett and James J. Cimino of Massachu­setts General Hospital. Drs. Gregory F. Cooper and Daria A. Giuse have provided valuable insights by reviewing earlier versions of this manuscript. While the work described was funded by NLM as part of its UMLS initiative, the system described is not, per se, directly part of the current UMLS metathesaurus. REFERENCES 1. HuMPHREYS, B. L., AND LINDBERG, D. A. B. Building thet Unified Medical Language System. In "Proceedings of the Thirteenth Annual Symposium on Computer Applications in Medical Care" (L. C. Kingsland, Ed.), pp. 475-480. IEEE Comput. Soc. Press, Washington, DC, 1989. 2. THOMPSON, E. T. (Ed.) "Standardized Nomenclature of Diseases and Operations," 5th ed. McGraw-Hill, New York, 1961. 3. "The International Classification of Diseases," 9th revision, Clinical Modification, 2nd ed. Vols. 1-3. U.S. Department of Health and Human Services, September, 1980. 4. CoTE, R., AND ROBBOY, S. Progress in medical information management: Systematized nomen­clature of medicine (SNOMED). lAMA, J. Am. Med. Assoc. 243: 756 (1980). 5. WELLS, A. H. Systematized nomenclature of pathology. Conversion to the computer language of medicine. Minn. Med. 55(6), 585 (1972). 6. COTE, R. (Editor-in-Chief) "SNOMED, Systematized Nomenclature of Medicine," 2nd ed. Am. Coll. of Pathologists, April, 1979. 400 MASARIE ET AL. 7. GANTNER, G. E. SNOMED and the pathologist. Pathologist 31, 398 (1977). 8. DoNNELLY, W. H. Application of SNOMED to state-wide data banks. Pathologist 31, 404 (1977). 9. "Medical Subject Headings-Annotated Alphabetical List," National Library of Medicine published annually. ' 10. LINDBERG, D. A. B., AND ScHOOLMAN, H.M. The National Library of Medicine and Medical Informatics, West. J. Med. 145, 786 (1986). 11. MILLER, R. A., PoPLE, E., AND MYERS, J.D. INTERNIST-I, an experimental computer-based diagnostic consultant for general internal medicine. N. Engl. J.Med. 307, 468 (1982). 12. MILLER, R. A., McNEILL, M.A., CHALLINOR, S.M., MASARIE, F. E., JR., AND MYERs, J. o. The INTERNIST-1/Quick Medical Reference Project-Status report. West. J. Med. 145, 816 (1986). 13. MILLER, R. A., AND MASARIE, F. E., JR. Use of the Quick Medical Reference (QMR) (R) program as a tool for medical education. In "Proceedings of the International Symposium on Medical Informatics and Education, Victoria, British Columbia, May, 1989," pp. 247-252. 14. MILLER, R. A., AND MASARIE, F. E., JR., Quick Medical Reference (QMR): An evolving, microcomputer-based diagnostic decision-support program for general internal medicine. In "Proceedings of the Thirteenth Annual Symposium on Computer Applications in Medical Care" (L. C. Kingsland, Ed.), pp. 947-948. IEEE Comput. Soc. Press, Washington, DC, 1989. 15. FIRST, M.D., SoFFER, L. J., AND MILLER, R. A. QUICK (QUick Index to Caduceus Knowl­edge): Using the INTERNIST-1/Caduceus knowledge base as an electronic textbook of medi­cine. Comput. Biomed. Res. 18, 137 (1985). 16. MILLER, R. A. A computer-based patient case simulator. Clin. Res. 32, 651A (1984). 17. PARKER, R. C., AND MILLER, R. A. Using causal knowledge to create simulated patient cases: The CPCS project as an extension of INTERNIST-I. In "Proceedings of the Eleventh Annual Symposium on Computer Applications in Medical Care" (W. W. Stead, Ed.), pp. 473-480. IEEE Comput. Soc. Press, Washington, DC, 1987. 18. PRYOR, T. A., GARDNER, R. M., CLAYTON, P. D., AND WARNER, H. R. The HELP system, J. Med. Syst. 7, 87 (1983). 19. BARNETT, G. 0., CIMINO, J. J., HUPP, J. A., AND HOFFER, E. P. DXplain: An evolving diagnostic decision-support system. lAMA, J. Am. Med. Assoc. 258, 67 (1987). 20. "Computers Gain New Respect as Translators." Science Times. New York Times. pp. B5-B6. March 5, 1991. 21. MINSKY, M. "Computers and Thought. McGraw-Hill, New York, 1963. 22. QUILLIAN, M. R. Word concepts: A theory and simulation of some basic semantic capabilities. Behav. Sci.l2 (1967). 23. BRACHMAN, R. J., AND ScHMOLZE, J. G. An overview of the KL-ONE knowledge representa­tion system. Cognit. Sci. 9, 170 (1983). 24. BRACHMAN, R. J. "On the Epistemological Status of Semantic Networks" Ph.D. Dissertation, Harvard University, May 1977. 25. CARBONELL, J. G., CuLLINGFORD, R. E., AND GERSHAM, A. G. Steps towards knowledge­based machine translation. IEEE Trans. Pattern Anal. Mach. Intelligence PAMI-3 (4) (1981). 26. Fu, L. S., BouHADDOU, 0., HuFF, S.M., SoRENSON, D. K., AND WARNER, H. R. Toward a public domain UMLS patient database. In "Proceedings of the Fourteenth Annual Symposium on Computer Applications in Medical Care." IEEE Press, Washington, DC, 1990. 27. SHIFFMAN, S., Wu, A. W., PooN, A. D., LANE, C. D., MIDDLETON, B., MILLER, R. A., MASARIE, F. E., CooPER, G. F., SHORTLIFFE, E. H., FAGAN, L. M., Building a speech interface to a medical diagnostic system. IEEE Expert, pp. 41-50, (February, 1991). I l l