00001.tif

PDF

Download

Open document

Transcript (if available)

Content I A MARKER-PASSING MODEL OF DEFINITE REFERENCE RESOLUTION IN NATURAL LANGUAGE DISCOURSE by I Seungho Cha i I i I l A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL i ! UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the j I I Requirements for the Degree i DOCTOR OF PHILOSOPHY ; I 1 (Computer Science) I i November 1993 I Copyright 1993 Seungho Cha UMI Number: D P22861 All rights reserved INFORMATION TO ALL U SE R S The quality of this reproduction is depen d en t upon the quality of the copy submitted. In the unlikely even t that the author did not sen d a com plete manuscript and there are m issing p a g es, th e se will be noted. A lso, if material had to be rem oved, a note will indicate the deletion. Dissertation Publishing UMI D P22861 Published by ProQ uest LLC (2014). Copyright in the Dissertation held by the Author. Microform Edition © ProQ uest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United S ta tes C ode ProQ uest LLC. 789 E ast E isenhow er Parkway P.O. Box 1346 Ann Arbor, Ml 4 8 1 0 6 - 1346 UNIVERSITY OF SOUTHERN CALIFORNIA TH E GRADUATE SCHOOL UNIVERSITY PARK LOS ANGELES, CALIFORNIA 90007 This dissertation, w ritten by C h a ....... under the direction of h Dissertation Committee, and approved b y all its members, has been presented to and accepted b y The Graduate School, in partial fulfillm ent of re -. quirem ents for the degree of D O C TO R OF PH ILOSOPH Y Dean o f Graduate Studies D a te ................ DISSERTATION COMMITTEE C hairperson PW -P- CpS >cj3 CAZifi Dedication my mother Duk-Young and my wife Eunha Acknowledgements I am grateful to my adyisor, Professor Dan Moldovan, for his guidance throughout my graduate studies at the University of Southern California. This work has been strengthened thanks to the technical directions and insight he provided. I have learned a lot other than my research topic from his enthusiasm toward research and learning. I would like to thank Professor Shankar Rajamoney for being my official ad- ■ visor in the Computer Science Department, and to thank Dr. Kevin Knight for \ ' ■ serving on my dissertation committee. I specially want to thank Dr. Eduard [ ; | I Hovy for lots of invaluable advice while serving on my qualifying committee, and i | to thank Professor Paul Rosenbloom and Professor Jean-Luc Gaudiot for serving ■ on my qualifying committee. I I | I also want to thank all the members who contributed to the SNAP project: | Juntae Kim, Min-Hwa Chung, Sang-Hwa Chung, Changhwa Lin, Chin-Yew Lin, i Steve Kowalski, Ken Hendrickson, Tony Gallippi, Adrian Moga, Traian Mitrache, Sanda Harabagiu, Wing Lee, and Haigaz Farajian. All of them were so help ful in many ways. I also thank our secretary Dawn Ernst for her kind help in i ; administrative chores and especially for correcting my English grammar. I would like to acknowledge the continuous support of the National Science , 1 Foundation grants MIP-89-02426 and MIP-90-09109. I also would like to thank j I , | my friends at USC for helping me with computers and always having lunch with j : me. Most of all, I thank my wife and my mother for their long periods of patience ! ] and devotion, and I thank my parent-in-laws for their continuous prayers for me. [ Contents D ed ica tio n ii A ck n ow led gem en ts iii L ist O f T ables vii L ist O f F igu res v iii A b stra ct x 1 In tro d u ctio n 1 1.1 M otivation................................................................................................... 1 1.2 Definite refe ren c e ...................................................................................... 3 1.3 Definite a n a p h o ra ...................................................................................... 4 1.4 Scope of the thesis .................................................................................. 6 1.5 Organization of d isse rta tio n ........................................................... 7 2 R ela ted W orks 9 2.1 Researches for anaphora resolution ..................................................... 9 2.1.1 Non-focusing approaches........................................................... 9 IV 2.1.2 Focusing a p p ro a c h e s ........................ 11 2.2 Other researches based on a c tiv a tio n .................................................. 15 2.3 Marker-passing, memory-based parsing and S N A P ........................... 16 2.3.1 M arker-passing............................................................................. 16 2.3.2 Memory-based p a rs in g ................................................................ 17 2.3.3 Semantic Network Array P ro c e s s o r......................................... 19 3 In form ation for R eferen ce R eso lu tio n 22 3.1 Constraint and p re fe re n c e ........................................... ....................... 22 3.2 Syntactic c o n stra in ts..................................................... ....................... 23 3.3 Semantic c o n s tra in ts ..................................................... ....................... 29 3.4 Discourse p re fe re n c e ..................................................... ....................... 33 3.5 World k n o w led g e........................................................... ....................... 33 3.6 Preference on linguistic p a ra lle lis m .......................... ....................... 33 : 4 A ctiv en e ss as D iscou rse P referen ce 36 4.1 D e fin itio n ............................................................................ .................... 38 4.2 Activeness a s s ig n m e n t..................................................... .................... 39 4.3 P ro p a g a tio n ........................................................................ .................... 40 4.4 Knowledge representation .............................................. .................... 47 4.5 Strengthening and decaying a c tiv e n e s s....................... .................... 53 4.6 Global order in a c tiv e n e s s .............................................. .................... 55 4.7 Examples of activeness c h an g e ........................................ .................... 59 ! 5 M ark er-P assin g A lg o rith m 62 t I 5.1 Recalling active co n cep ts........................................................................ 62 5.2 Refer a b i l i t y ................................................................................................ 64 5.3 Global order for constraints and p re fe re n c e s.................................... 66 5.4 Algorithm description............................................................................... 68 5.5 Case study of important exam ples........................................................ 75 6 P arallel Im p lem en ta tio n 83 6.1 Parallel Implementation on S N A P ........................................................ 83 6.2 Performance a n a ly sis ............................................................................... 85 6.2.1 Initial Experiment ...................................................................... 85 6.2.2 Analysis of result and h e u ristic s............................................... 85 6.3 Comparison with other approaches ..................................................... 91 6.3.1 Quantitative com parison............................................................ 92 6.3.2 Qualitative com parison................................................................ 93 7 S u m m ary and C on clu sion s 98 I 7.1 Result of d issertatio n ............................................................................... 98 ! j I 7.2 Future w o r k ................................................................................................ 100 ! I ! i A p p en d ix A j Test examples from internet news group ..................................................... I l l ! L ist O f T ables 1.1 Types of anaphora based on grammatical character 1.2 Types of pronominal anaphora. ................ .. 2.1 SNAP instruction set................................................... .. 4.1 Activeness propagation example 1................................. 4.2 Activeness propagation example 2................................ 4.3 Activeness propagation example 3................................ 6.1 Assigned Values................................................................ 6.2 Different Values................................................................ 6.3 Result without heuristics................................................ 6.4 Pronoun reference without h e u ristic.......................... 6.5 Result with intra-sentential heuristics......................... 6.6 Result with heuristics for proper name reference. . . 6.7 Final result........................................................................ 6.8 Pronoun reference............................................................ 6.9 Comparison among discourse approaches................... 6.10 Comparison among approaches..................................... 5 6 21 59 60 61 84 85 86 89 90 91 91 92 96 96 vii L ist O f F igu res 2.1 Marker-passing model................................................................................ 16 2.2 Memory-based parsing............................................................................... 18 2.3 Case grammar representation. . . . .................................................... 19 2.4 SNAP-1 Prototype..................................................................................... 20 3.1 Gender and number constraint............................................................... 25 3.2 Person constraint........................................................................................ 28 3.3 Semantic constraint: Verb Case............................................................... 30 3.4 Semantic constraint: A ttribute................................................................. 31 3.5 Semantic constraint: Subsumption.......................................................... 32 4.1 E x p e c ta tio n ............................................................................................... 42 4.2 Confirmation ............................................................................................ 45 4.3 Concept Hierarchy in WordNet .......................................................... 48 4.4 Concept S eq u en ces.................................................................................. 49 4.5 Relations between events......................................................................... 51 4.6 Partial order graph.................................................................................... 57 4.7 Activation levels......................................................................................... 58 5.1 Discourse graph............................................................................................ 63 v i i i 5.2 Memory partition and transition............................................................ 64 5.3 Flow chart of anaphora resolution model............................................. 69 5.4 MUC4 tex t.................................................................................................. 70 5.5 MUC4 tem plate.......................................................................................... 71 5.6 C-commanding case................................................................................... 73 5.7 Resolution by merging nodes.................................................................. 74 5.8 MUC5 tex t.................................................................................................. 76 5.9 Definite Noun Anaphora (a)................................................................... 77 5.10 Definite Noun Anaphora (b)................................................................... 78 5.11 Syntactically Ambiguous Anaphora...................................................... 80 5.12 Related events and concepts................................................................... 82 5.13 Linguistic Parallelism............................................................................... 82 6.1 Input for parsing output generator........................................................ 97 Abstract i i J Definite reference is a natural language phenomenon, which is frequently used to denote a certain entity in natural language discourse. This includes the use of pro noun, noun phrases proceeded by the definite article the, and proper name. In or der to understand natural language discourse, a computational system has to find out which entity is referred by the definite reference used. The definite reference resolution involves many problems related to syntax, semantics, discourse, and pragmatics which requires reasoning on world knowledge. However, this disserta tion is focused on the discourse related problems and proposes a marker-passing model of the definite reference resolution. Previous approaches to the discourse processing which uses local focus fail to resolve references in some cases due to the lack of consideration about a more global discourse topic. In the proposed model, ] activeness is defined as a notion of discourse preference, and this is integrated into | J another defined term referability to represent scores as a referent of a given definite ! reference. Other syntactic and semantic information is represented as factors in the referability term. Two important processes — expectation and confirmation \ — contribute to find out the discourse topic by spreading the activeness of rele- ] vant concepts using marker-passing paradigm. This model has been implemented ; on SNAP (Semantic Network Array Processor) and experiment shows over 92% . (96% for singular pronouns) of successful resolution in news article type discourse. This thesis emphasizes on the importance of concepts’ activation in reference res olution, and proposes the computational model based on linguistic theories and activation theory in psychology. Chapter 1 i Introduction 1.1 Motivation i ! I • • 1 j In everyday life, we listen to, speak, read, and write our own languages such as | j English, French or Korean. These languages are not made artificially, as computer j languages are, and they don’t have a very strict form to be fit in. Because they j are not fit into some strict form, it has been a big challenge to make a computer ! to process human languages even syntactically. However, the goal of natural j language processing is not just syntactic parsing of sentences, but acquiring the | semantic understanding of large texts. Since the natural language understanding is mingled with the human knowledge, which is constructed inside the brain over several decades, this task becomes even more difficult. ; Natural language understanding requires lots of very difficult tasks, and it is re- ; f ! lated to many research areas such as artificial intelligence, linguistics, psychology, and so on. This thesis proposes a solution from an artificial intelligence point of view to one of many problems, and shows the experiment results which are better : than the ones from previous approaches. The main topic of this thesis is natural language discourse processing, and more specifically, it is reference resolution. In most languages, several words form a phrase, several phrases form a clause or a sentence, and sentences form a discourse, which is to transfer the consistent and | meaningful information to the listener or reader. Therefore, discourse processing | in a natural language is an inevitable task to solve in order to achieve one of the I goals in artificial intelligence — to make machines which are able to understand j natural languages. { There have been many approaches to understand natural language phenomenon related to natural language discourse. [54] Though the discourse processing is a part of the natural language problem, it also is a very broad problem itself. Among | various problems in natural language discourse, reference is a very important phe- j nomenon to understand natural language. The reference, which will be defined 1 ' < later, is a problem related to the discourse processing of a natural language, be- j cause this problem occurs not only within a sentence but also in several sentences which form a small discourse. In all natural language discourses, whether written ! or spoken, references are frequently used. Thus, a discourse cannot be under- ! ^ f stood without resolving references, and the natural language processing system should resolve references1, especially anaphoras which consist of using abbreviated expressions (e.g. pronoun) to refer to something mentioned earlier in discourse. ; There have been many computational approaches to anaphora resolution using local focus, which is an entity at the center of attention in a sentence [112, 42, 61, | 13]. These approaches fail to resolve anaphora in some cases, either because of , choosing a wrong focus and/or not considering a more global discourse topic. In j this thesis, I present an approach to reference resolution based on the activation of ! relevant concepts in discourse. A parallel marker-passing paradigm is used to find a referent by calculating the activation levels of all relevant concepts and other i , information [15, 17, 16]. I (i.i) A good share of the amazing revival of commerce m ust be credited to the ease and security of communications within the empire. The 1This will be defined in Section 1.2. Imperial fleet kept the Mediterranean Sea cleared of pirates. In each province the Roman emperor repaired or constructed a number of skill fully designed roads. They were built for the army but served the mer chant class as well.2 i 1 Kantor points out the importance of the discourse topic in anaphora through the example (1.1) where an anaphor tends to be interpreted as a discourse topic (the ease and security of communications), rather than the local focus (a number of skillfully designed roads) which is nearer to the anaphor [62]. He claimed they in i the discourse above is not clear enough, because they tends to be understood as j ease and convenience of transportation which is a discourse topic, until it is found | semantically wrong and they are actually the roads. | In other words, the reference using a pronoun as the local focus is not clear ! sometimes, and the pronoun can be understood more clearly as a discourse topic. j ; When assuming a clear discourse, a discourse topic plays a very im portant role in | the resolution of references. This argument implies a very im portant premise: the j j listeners/readers may try to resolve the reference as a concept which gets the most \ j attention in a discourse — the discourse topic — and it maybe different from the | local focus. Thus, the discourse topic must be considered in a reference resolution j process. This is the basis of the new reference resolution model. j | 1.2 Definite reference i t Reference is a linguistic process, by which a speaker/writer denotes an entity or entities. There are two kinds of references: one is the definite reference and the other is the indefinite reference. This thesis is restricted to the part of definite ■ reference problems. I i (1-2) The chinchilla ate m y portrait of Richard N ixon... 3 ! ! _______________________________ I 2This is taken from [62], I 3This is taken from [54], j In (1.2), all the noun phrases are definite descriptions which refer to some entity. The chinchilla denotes the writer’s pet, m y denotes the writer, Richard Nixon denotes the ex-president of the United States, and m y portrait of Richard Nixon j denotes the portrait of Nixon that belongs to the writer. As in this example, a definite reference denotes a certain entity, which is known to (or assumed to be known to) both of the speaker/writer and the listener/reader. On the other hand, a certain entity is not denoted specifically by an indefinite reference as in the following example. In other words, an indefinite reference one does not denote a specific TV, but any TV. (1.3-1) Tom ’ s T V is out of order. (1.3-2) He will buy a new one. | 1.3 Definite anaphora i i I i Anaphora is a part of a broader term reference. Instead of a full expression, an j abbreviated expression is often used as a reference to some entities. This case occurs when a speaker/writer thinks that: (1) the level of speciality is sufficient and (2) he/she expects the perceiver(s) to understand the abbreviated expression. There are several types of abbreviated expressions, and they can be classified in several ways. Though Clark and Marshall [27], or Pinkal [99] classified them in more detail from the different point of views, Hirst’s simpler classification is quoted here and will be used for further discussion. E n d o p h o r refers to an entity which appears in the discourse. This may be further classified as: A n a p h o r refers to an entity which appears earlier in the discourse. (1.4-1) Tom bought a new car, (1.4-2) and he likes it very much. i i 4 type of anaphora referent (grammatically) example sentences pronominal Noun Phrase Tom bought a car. He likes it very much. prosentential Sentence Tom loves Judy, and she knows it. proverbial/ proactional Verb Phrase Tom went shopping. John did, too. proadjectival/ prorelatival Adjective Phrase Tom has a palm-size PC. John has never seen such a PC. locative Adverb Phrase Tom has been to Korea. John once lived there. temporal Adverb Phrase Tom fought John, (timel) Since then, they’ ve never talked to each other. ellipsis something omitted Ross folded his trousers and climbed into bed. j Table 1.1: Types of anaphora based on grammatical character. He refers to Tom, and it refers to the car which Tom bought. C a ta p h o r refers to an entity which appears later in the discourse. (1.5-1) When he entered the room, (1.5-2) Tom found a broken vase. i j He in (1.5-1) refers to Tom in (1.5-2). j j E x o p h o r refers to an entity in the real world not mentioned in the discourse. j (1.6) Pick that up. (while pointing out a book on the desk) ! That refers to a book on the desk. Thus, anaphora is a phenomenon of subsequence reference to a previously mentioned entity in a discourse. Through anaphora, a speaker or a writer makes . an abbreviated reference to some entity or entities. If this anaphora is a part of definite reference, it is called definite anaphora. Indefinite anaphora (e.g. one ! anaphora) is out of the scope of this thesis. Anaphora resolution is a process 5 type what refers to NP? example sentence pronoun. anaphora pronouns (he, it, one, ...) Tom bought a new car. He likes it a lot. definite noun anaphora definite noun phrase Tom loves a sales girl at Broadway. He is a student at USC. But the girl hates him. epithet Noun Phrase As Tom used his credit card too much, the poor guy went bankrupted. surface count the former, the first, ... Lynn has two boyfriends, Mark and Kevin. She likes the former better. i Table 1.2: Types of pronominal anaphora. j ! ' j through which the perceiver of the discourse disabbreviates the reference and \ I perceives the identity of the entity (or entities) which the writer/speaker wants to j refer [54]. To understand the anaphora problem more clearly, it’s necessary to know what kinds of anaphoras are used in a discourse. There are several criteria to classify anaphora, and they are well explained in [54]. The most common criterion is the grammatical character of the referred concept(s) in the discourse. The classifi cation using this criterion is given in Tables 1.1 and 1.2 [54]. Identity of sense anaphora (descriptional anaphora) and identity of reference anaphora (denota- tional anaphora) [54, 94] are not considered in this thesis, as these classifications are beyond the scope of this thesis. 1.4 Scope of the thesis Reference resolution involves many aspects of natural language phenomena. Many i reference problems cannot be resolved without considering discourse pragmatics | which is based on a huge amount of delicate and complicated world knowledge. 1 [ However, these kinds of problems are not in the scope of this thesis. This thesis I mainly deals with the discourse aspect upon reference problems. In other words, the cases that can be solved without considering the above complicated knowledge are the main concern of this thesis, because a human recognizes instantly what the reference is about in many cases — by recognizing the discourse topic. I 1 I As this thesis is about the discourse aspect of reference, the m ajor concern ! j in this thesis is inter-sentential reference (mostly pronoun and definite noun anaphora). This thesis is not concered much about syntactic and semantic mat- j ters, but small portions of syntactic and semantic aspects are considered. Many ' of intra-sentential anaphora require much more syntactic analysis than what is I described in Chapter 3 of this thesis. Thus, detail solution for intra-sentential anaphora is out of the scope of this thesis, though they are included in the exper iments. ! 1.5 Organization of dissertation The dissertation is organized as following. In Chapter 2, the related works to this research are discussed. Several previous approaches to definite reference resolution ! and their weak points are discussed. Also, SNAP (Semantic Network Array Pro cessor) is discussed with marker-passing paradigm. The memory-based parsing, which is to provide the parsing output to the definite reference resolution module, is discussed as well. In Chapter 3, various types of information, which is required to be considered in reference resolution, are discussed. Information such as syntactic constraint, semantic constraint, discourse preference, and so on, is considered and the marker- passing solutions are presented. I In Chapter 4, activeness is defined, and it is discussed how all the operations J I I I on activeness are carried out. The goal of activeness is to provide the means to ■ | handle the concepts, which receive attention in the current discourse, and the [ I change of attention to those concepts. Activeness is assigned to every relevant ! I I concept, propagated to the related concepts, and strengthened or weakened when t 7 the concept is realized or not. Global order of activeness is presented. Through the notion of activeness, discourse topic of the discourse can be considered, and it helps to find out the correct referent of a definite reference. In Chapter 5, referability is defined to integrate the various types of informa tion which are required to compute the correct referent. The referability value is based on the syntactic and semantic constraints, the discourse preference (which is activeness) and the preference on linguistic parallelism. Partial order and global ' order related to these constraints and preferences are presented. Using the refer ability value, the marker-passing paradigm computes the referent from the all relevant concepts in parallel. The concept with the highest referability value is chosen as a referent. The algorithm of the proposed model is presented. This algorithm is explained with various definite reference problems. In Chapter 6, the parallel implementation on SNAP is discussed with experi ment result. The experiment on news article type discourse shows that this model achieves around 92 percent of overall success and 96 percent success in singular pronoun resolution, provided that required knowledge is in the knowledgebase, j Qualitative and quantitative comparisons with other models are discussed as well. : In Chapter 7, the contribution of the research is summarized with the conclu- I sion. Test examples are attached in an appendix. Chapter 2 Related Works 2.1 Researches for anaphora resolution 2.1.1 N o n -fo cu sin g ap proach es There have been various approaches to resolve anaphora problems, and they are mostly concerned with the pronominal anaphora resolution. The followings are some of the im portant approaches [54]. • h eu ristics: This approach uses some heuristics without any theory. For example, one of heuristics used in W inograd’s SHRDLU system [124, 125] is if it occurs twice in a same sentence, they are co-referential. Too many heuristics are required to cover all kinds of problems, though a limited number of heuristics may still be required, due to the nature of natural language problems [54]. • sy n ta c tic parse tr e e search: This approach simply searches for the syntactically valid referents in the order of recency (or by some heuristic order according to syntactic parse 9 tree [55, 58]). This is the simplest and fastest way, and shows good capa bility in intra-sentential anaphora, but its resolution capability is limited in intersentential anaphora. i ] • case gram m ar: This approach uses the information provided by the cases [114, 69]. Case grammar provides more information besides the syntax, and is more flexible than the syntactic approach [31]. However, this has a problem when there are multiple candidates which satisfy case constraints. I | • an a ly sis by synth esis: I This approach generates all possible surface sentences and then analyzes their validity [128, 69]. This consumes too much tim e for the sentence gen eration and this doesn’t consider the intersentential aspect of anaphora at all. It is just for intrasentential anaphora. i ! • inferences: i j This approach includes Rieger’s conceptual memory [101] and W ilk’s pref erence semantics [123]. The conceptual memory approach uses conceptual dependency and lots of undirected inferences. This approach resolves only a part of the definite reference cases and is limited by the large cost of undi rected inference. Preference semantics use word meanings and inferences, [ and is more flexible and controlled in inference than the conceptual memory I approach. This approach compares among referent candidates and selects the one which is more semantically acceptable. This also fails when there is j more than one candidate equally semantically valid. j In addition to these approaches, there were several other approaches for limited j domain (e.g. database query [38, 49]), but they have very limited resolution ■ capability. (2.1-1) John left the window, and drank the wine on the table. 1 (2.1-2) It was good.1 1This is taken from [54]. 10 In the example above, many simple approaches, which consider only one aspect of the anaphora problem (e.g. syntax or semantics), fail to find the referent of it in the third sentence. Even the combination of several approaches may fail, ; because both the wine and the table are syntactically correct for it, and both are I j semantically correct for good. But, it refers to the wine. It’s an easy task for humans to find the referent, because there is more attention on the wine than on the table. J In (2.1), the anaphora occurs intersententially in a discourse. The inability ' to consider discourse is the major reason why many approaches failed to find the | referent. One of the most im portant pieces of information in a local discourse j is the local focus, which is the concept the local discourse is talking about. The | importance of the focus in anaphora resolution has been addressed well by many I researchers. [54, 112, 40, 41, 42, 61, 13, 1, 45, 116] 2.1.2 F ocu sin g ap p roach es A ssu m p tio n s and ad van tages in focu sin g As discussed before, some discourse information, such as focus, is very impor tant in the resolution of intersentential anaphora. The term focus, used in this paper, denotes focus of attention which has been used by AI researchers [40, 112]. Though there is some confusion in terms such as topic, focus, given, new, theme, and rheme, focus in linguistics research usually denotes new information which is the predicate part of the sentence, while topic denotes the given information which is the subject part [54, 62, 46]. From now on, focus denotes the AI term, unless it is explicitly mentioned as a focus in linguistics research. The focus may be classified as global and local focus, depending on the scope of focusing. Basically, a local focus is found in an utterance, while a global focus is found in a subdiscourse. The local focusing theory is based on the fact that the perceiver of the discourse, as well as the speaker/writer, maintains some important concepts (foci) in mind, and pays less attention to the minor things [54, 62, 112]. 11 I As a result, it takes longer for listeners to process a pronominalized noun phrase that is not in the focus, than one that is in the focus. Also, it takes longer to process a non-pronominalized noun phrase that is in the focus, than one that is : not in the focus [13, 62, 45]. This is the base of the focusing theory for anaphora j ! . . . 1 j resolution. There are several assumptions made in a focusing theory, as explained in [112]. • The speaker/writer is assumed to be communicating about something, which is called focus. • The speaker/writer assumes that the listener/reader can identify the focus ' of the discourse given. I ! • The speaker/writer is not trying to confuse or deceive the listener/reader. J • The speaker/writer assumes that listener/reader has enough world knowl edge to reason about the referred expressions. i | By using the focus idea, several big advantages can be attained. First, the focus ! limits the search space in anaphora resolution by maintaining a focus space. This reduces the processing time significantly. Second, the focus helps the ambiguity resolution where there are many candidates for a referent, because the focused | concepts are preferred as referents. Third, the focus can guide the inference, so I j that inference is performed only for the focused concepts. j ' I I J But it is not an easy task to pick up a correct focus from a sentence of a , discourse, and none of the previous approaches have been completely successful for , a broad range of anaphora problems. One of the reasons is that the speaker/writer ; does not always make discourses in the way defined in their focusing models, even | ; though the discourses axe made clearly for humans. I i # I G ro sz’s ap proach to g lob al focu sin g I | This research is basically applicable to the definite noun anaphora in a task- oriented discourse with a predefined well-structured task knowledge. In a task 12 oriented knowledge hierarchy, each task and subtask represents the global focus in a discourse. When a definite noun is used, its referent is selected from the 1 entities in the focus of attention, which contains all the related entities in that ; task/subtask knowledge [40, 41]. Her idea about global focusing was extended to a discourse structure theory which considers discourse segments, discourse intention and focus of attention [44]. Though Grosz and Sidner’s discourse structure in [44] is very powerful and well formalized, it doesn’t help anaphora resolution much in local discourses, and 1 it doesn’t resolve pronoun anaphora. | j I : S id n er’s ap p roach to lo ca l fo cu sin g | I I Unlike Grosz’s work, Sidner’s work doesn’t depend heavily on discourse struc- ! tures, instead it takes a more general approach to anaphora resolution. This j generality comes from using local focusing instead of global focusing which Grosz i used, because it’s much easier to grab a local focus than to grab a global focus without any prior domain specific knowledge about what will be said in a dis- j course. A discourse focus and an actor focus are maintained. The agent in an utterance is chosen as an actor focus, while a discourse focus is chosen from a potential foci list which is ordered in Cobjectl, object2, other noun phrases, verb 1 phrase> [112]. The discourse focus seems to be m aintained as a frame represen- [ ! . I : tation. 1 1 i I \ i Though [112] is more general than Grosz’s approach, several problems can be found in her approach, including the problems which Hirst [54] and Brennan [13] pointed out. i i i • No consideration for multi-frame focus (i.e. multiple pronominalization in a sentence). j i • Frame-selection (initial focus selection) problem. i • The pronoun used as an actor strictly prefers the actor focus over the dis- i course focus. 13 • There are problems with the sentence, which has more than one subject and/or object. • The algorithm keeps one discourse focus and one actor focus, but the chosen j discourse focus cannot be guaranteed to be correct. When the chosen focus is wrong, resolution in the following discourse becomes very poor. The actor focus and the discourse focus are strictly exclusive. A potential foci list consists of only the concepts which appear in the pre vious sentence. B ren n a n e t a l’s ce n terin g approach ! Among the techniques using local focusing, one of the most recent works is the 1 centering approach [13, 61, 42], In Brennan’s centering approach [13], the forward- looking center list contains the candidates for the backward-looking center in the order of <subject, objectl, object2, others>. In this approach, the backward- looking center corresponds to the discourse focus in [112], and the forward-looking \ center list corresponds to the potential foci list. 1 The difference, however, is that [13] keeps only one focus, which is the backward- | looking center. When selecting a backward-looking center, several rules and con- | straints are used with the forward-looking center list. This approach is only for i j the pronoun anaphora, and doesn’t consider the definite noun anaphora cases. ! Brennan’s approach also has several deficiencies similar to those found in Sid- | ner’s approach, though it is claimed to handle the complicated sentences with j | multiple subjects better. j • Focus does not need to be a noun or simple noun phrase. ! I | ! • No consideration for multi-frame focus. ■ I i j ; • More serious frame-selection problem. ' 14 • The algorithm finds one focus according to the given ordering and heuris tics. When the chosen focus is wrong, resolution in the following discourse becomes very poor, just as in Sidner’s approach, i i • Forward looking center list consists of only the concepts which appear in the j current sentence. • It considers only nouns phrases as a referent, thus, it is mainly for he and she anaphora. 2.2 Other researches based on activation ! i i j K a n to r’s ap proach to glob al a c tiv a te d n e ss j ' i K antor’s research [62] is about comprehensibility of the discourse. He defines | the activatedness of a concept to be a global focus as a continuum, unlike the usual ^ | ! binary distinction about focus — in the focus or out of the focus. He claimed th at ! i some pronouns are more comprehensible than others, and only the highly activated concepts may be pronominalized in an order that a speaker/writer makes a clear discourse. I But he didn’t mention how to get the global activatedness computationally, j i and it’s very difficult to compute the global activatedness from the discourse topic, j context and so on without knowing what the discourse topic is. It also requires to check if a current discourse topic is realized in the proceeding discourse. This is another difficult task, as the checking process includes analysis of collaboration j and incorporation. \ \ H a jico v a ’s ap proach to a c tiv a tio n h ierarchy ! Hajicova claimed that concepts in the discourse have activation levels which j > keep changing. By defining several rules for deciding a concept’s activation level, the discourse can be segmentized by the activation level of concepts, and the acti- , vation level offers better choices in the pronominalization process for the discourse 15 Global Inference Engine i 1 i t r Permanent Knowledge ( Semantic Network ) Temporary Knowledge ( Markers) Figure 2.1: Marker-passing model. | generation [46, 47]. Though her approach has a similar basic idea (activation level) | I to the idea proposed in this approach, it has a different goal (pronominalization j in discourse generation and discourse segmentation) and a different scheme for I setting and changing activation level. j j The interesting points of her research include: (1) it is claimed that concepts 1 j in the focus (predicate) part — focus in linguistics — of the utterance has a higher ; activation level than the concepts in the subject part, (2) the activation level for j the referent of a definite noun phrase is higher than the referent of the pronoun, | and (3) the decaying of activation may be different between the concepts according to their position (subject or predicate). However, the activation levels between a descriptor and a non-descriptor is not distinguished in this research. I 2.3 Marker-passing, memory-based parsing and | ! SNAP f i M ark er-p assin g j 16 2.3.1 Marker-passing is a technique developed to reason on semantic networks and is suitable to parallel processing [84, 89]. Marker-passing was introduced by Quil- lian [100] and developed by Charniak [20, 21]. In marker-passing, permanent i knowledge is represented as a semantic network. The nodes in the semantic net- ' j work represent concepts and their properties. The arcs in the network represent 1 interrelation among the nodes. Usually, the network forms a hierarchical struc- ! ture. Markers constitute tem porary knowledge which overlaps with the permanent i knowledge. (Figure 2.1 [91]) These markers can represent various types of infor mation. For example, in this research, markers can represent which concepts are more in attention. Reasoning is achieved by changing the states of tem porary and/or permanent knowledge. The reasoning process is done by propagating markers, performing set operations, and checking the existence of certain markers. Since markers and arcs can include numerical values or weights, performing numeric computation on the markers and arcs contributes to the reasoning process. | The major advantage of marker-passing paradigm is parallelism. (1) The oper- j ations on markers (reasoning processes) are processed in a massively parallel way, j 1 and (2) marker-passing paradigm fits into many artificial intelligence applications, which require complicated search and computation. This research also requires complicated computation in the resolution of references where the knowledgebase is very huge, and marker-passing is the best approach for this purpose. j i 2.3.2 M em o ry -b a sed p arsin g ! : I Memory-based, parsing is an approach to natural language processing, where parsing is performed directly in the knowledge base by the marker-passing memory , search. There have been several systems based on this approach, such as DMAP, ; DMTRANS, ^DMDIALOG, and DMSNAP [102, 70, 67, 69]. The basic idea i behind this approach is similar to the memory-based reasoning paradigm, which [ | uses memory intensively for realizing intelligent systems [113]. In memory-based 17 (^see-event^) prediction ^ next prediction last first next object agent syntai syntactic constraint semantic constraint istraint semantic constraint physical objei NP isa animate activation activation isa ‘ John’ person isa c-john Figure 2.2: Memory-based parsing. I parsing, parsing is performed directly in the memory by matching the linguistic pattern of an input sentence to templates, which are called concept sequences. Marker-passing performs memory-based reasoning to handle multiple hypotheses, while it performs massively parallel pattern matching throughout the memory. The parsing algorithm used in [91] is illustrated in Figure 2.2. Linguistic knowledge is represented in a semantic network using case grammar representation | i [31] for linguistic patterns, while world knowledge is represented in a semantic I I network hierarchy [12]. Both syntactic and semantic constraints are represented | j under each case role node, which is concept sequence element. At the beginning of processing a sentence, the first role nodes of all concept se- | quences are predicted with prediction markers. When a word is input, it activates j the corresponding node. The activation is propagated up through the knowledge ( j hierarchy to the role nodes, using activation markers. If a predicted role node is I 18 | mtrans-event#! agent object John#! see-event#l experiencer experiencer Paula#! agent object ISA bomber#! house#! Case Representation of “John told Ron that Paula saw a bomber bombing a house.' Figure 2.3: Case grammar representation. activated, then it predicts the next role node. This procedure is repeated, and if the last role node is predicted and activated, the input sentence is interpreted ! as an instance of the concept sequence root node [88]. The parser generates the ! output as shown in Figure 2.3, and this form of parsing output is used as input to this anaphora resolution model. i I ^ i i i i 2.3.3 S em a n tic N etw o rk A rray P ro cesso r i , | SNAP (Semantic Network Array Processor) is a parallel marker-passing archi- . tecture, which was developed at the University of Southern California. [32] The j architecture of SNAP-1, which is a SNAP prototype, is shown in Figure 2.4. I SNAP-1 is made of a SNAP array (multiprocessor array) and a SNAP controller (array controller), and the SNAP array consists of 144 Texas Instrum ent DSP 1 microprocessors (TMS320C30). The array is 32 tightly-coupled clusters of 4-5 , 1 1 i DSP microprocessors, which act as Processing Elements. Each cluster covers IK : ! nodes and 10K links. Semantic network is stored in the memory of the processing 19 Host Computer Hardware Environment Host Software Environment Program development using SNAP instruction set Physical Design SUN 4/280 SNAP-1 Controller VME Bus Controller Compiled SNAP code Program Control Processor Sequence Control Processor ..Q rie.Sy-size bffajrd. SNAP-1 Array Custom Backplane 144 Processor Array Knowledge base SNAP instruction execution Eight 9U-size boards Four clusters per board our to five processors per cluster Figure 2.4: SNAP-1 Prototype. elements in the SNAP array. The SNAP prototype is shown in Figure 2.4, and the instruction set for SNAP-1 is shown in Table 2.1. 20 Type Instruction Operands Node Maintenance create delete set-color source, relation, weight, destination source, relation, destination node, color Marker Node Maintenance marker-create marker-delete marker- set- color marker, relation, reverse-relation, destination marker, relation, reverse-relation, destination marker, color Search search-node search-relation search-color node, marker, value relation, marker, value color, marker, value Propagation propagate markerl, marker2, function, rule, relation Boolean and-marker or-marker not-marker test-marker markerl, marker2, marker3, function markerl, marker2, marker3, function markerl, marker2 markerl, marker2, value, condition Set/Clear set-marker clear-marker func-marker marker, value marker marker, function Retrieval collect-marker collect-value collect-relation collect-color marker marker marker, relation marker, color Table 2.1: SNAP instruction set. 21 Chapter 3 Information for Reference Resolution 3.1 Constraint and preference Though reference resolution seems to be just a small part of natural language understanding, it is a very difficult and complicated task to achieve. As briefly discussed before in Chapter 2, various types of information are required to resolve various types of reference cases. The resolution task cannot be done by a method for one aspect of the problem, though the discourse aspect of the reference problem will be emphasized in this thesis. The information is classified into two basic types in this thesis: constraint and preference. Though it’s hard to draw a border line between constraint and preference, syntactic and semantic information are thought to be constraints, and discourse information and linguistic parallelism are thought to be preferences. But complicated world knowledge which requires complicated inference is not considered in this thesis. Some of the world knowledge is considered as a semantic constraint. In this approach, the following categories are considered: (1) number, (2) gen der, (3) c-commanding for syntactic constraint checking, (4) verb case constraint, 22 (5) property constraint, (6) subsumption constraint for semantic constraint check ing, (7) the activeness values and the expectation as discourse information, and (8) a portion of the linguistic parallelism. In the following sections, issues about (1) - (6) and (8) will be discussed and the solutions for each of these problems are presented. All solutions using marker- passing will be integrated into one and will be discussed in Chapter 5. The details about (7) is the key issue in this thesis and will be discussed in detail in the following chapter. 3.2 Syntactic constraints One of the constraints, which should be satisfied in the reference resolution task is syntactic constraint. Assuming the given discourse is w ritten/spoken correctly in grammar, this constraint should be satisfied in the reference resolution process. In other words, the concepts which don’t satisfy the syntactic constraints cannot be the candidate of the referent, which is referred by the given reference. There are several types of syntactic constraints: they are gender, number, person, and c-commanding. G en d er One type of syntactic constraints is imposed by satisfying the gender of the noun phrases and pronouns. There are male (e.g. boy, he), female (e.g. girl, ship, she) and neutral (e.g. car, baby, it) genders. The gender constraint should always be satisfied, if it is not the case that the writer/speaker is confused about the gender of an entity which he/she wants to refer, as shown in (3.1). But this case will not be considered in this thesis. (3.1-1) Dr. Brown is a famous computer scientist. (3.1-2) Is he the man with the wine glass over there? (3.1-3) No, Dr. Brown is that lady in a white cashmere sweater. 23 S o lu tio n for g en d er co n stra in t : 1. While parsing, mark each concept’s gender with M(ale), F(emale) and N(eutral) marker. 2. When a pronoun anaphora is used, M, F or N gender marker is marked while parsing, too. 3. Find out concepts with the same gender marker. Concepts found are the candidates for the referent. 4. The proper value is assigned to the concepts which satisfy these concepts.1 5. The concept with the highest value is selected as a referent. There can be multiple concepts selected, because this solution only considers gender constraint. The same solution can be applied for the other references, too. The referent of the lady should be a female and the referent of Bob should be a male. (3.2) and Figure 3.1 shows the example, along with the number constraint discussed later. (3.2-1) John bought a car. (3.2-2) He likes it. U n k n o w n g en d er Some cases requires more careful consideration to handle the gender constraint. In the next example, because the gender of the singer in (3.3-2) is not known and treated as neutral, Mike in (3.3-1) can be found as a referent of him (3.3-3). This should be avoided. (3.3-1) Mike went to the rock concert. (3.3-2) The singer was a teenager’ s idol. (3.3-3) A lot of boys and girls came to see him. 1This value is for the algorithm which integrates various types of information. This will be discussed later in Chapter 5. 24 buy-event object agent John car Male Singular Neutral Singular C like-state j ^ like-state buy-event experiencer. object object agent object he John car Male Singular Neutral Singular Figure 3.1: Gender and number constraint. To avoid this problem, neutral gender is divided into two types — one for the real neutral gender concepts like stone or water and the other for the tentative neutral gender concepts like singer or reporter — in the model. When finding a referent, the tentative neutral gender concept can be a referent of a reference which has either male or female gender. Once the gender is found, the tentative neutral gender is changed into its own gender given by the reference. Thus, the previous solution can be rew ritten as below. R ev ise d so lu tio n for g en d er co n stra in t : 1. While parsing, mark each concept’s gender with M, F, N, and TN (Temporary Neutral) marker. 2. When a reference is found, the M, F, N or TN gender marker is marked while parsing, too. 25 3. Find out concepts with the same gender marker. Or, find out concepts with TN marker, if the referring concept has an M, F or TN marker, or find out concepts with M, F or TN markers, if the referring concept has a TN marker. 4. Proper value is assigned to the concepts which satisfy these concepts. 5. The concept with the highest value is selected as a referent. 6. If a referent is chosen and the referent (or the referring concept) has a TN marker, replace it with a marker of the referring concept (or the referent). N u m b er Similar to the gender constraint, another type of syntactic constraints is im posed by satisfying the number of the noun phrases and pronouns. There are singular and plural numbers. The number constraint, as well as the gender con straint should always be satisfied, except the plural referring expression refers to the class of the referred concept, as in (3.4). (3.4-1) M y neighbor has a monster Harley 1200. (3.4-2) They are really huge, but gas efficient bikes.2 In the example above, the referent of they cannot be found in (3.4-1). Definitely the referent is not a monster Harley 1200 which my neighbor has, but the class of Harley 1200 motor cycles, but this case will not be considered in this thesis, either. This number constraint is processed in the same way explained with the gender constraint. In other words, the original solution for gender constraint can be applied to the number constraint, using SG and PL markers instead of M, F and N markers. 2This example is taken from [111]. 26 In (3.2), as shown Figure 3.1, John, car, he and it all get SG markers, thus, all of them get some value (10 in this example). Due to gender markers, John attains 20 as a referent of he, while car attains only 10. P erso n The person constraint is a little different from the two constraints discussed above. The person constraint plays a very im portant role in the anaphora reso lution in discourses with the narration or the conversation where the first person and the second person keep changing. In the narration or dialogue, the first per son denotes the speaker, the second person denotes the listener, and the third person denotes the other persons besides the speaker and the listener. Though the person constraint is not im portant in the indirect narration like (3.5-1), it is very im portant for anaphora resolution in the discourse with the direct narration (3.5-2). (3.5-1) John told M ary that he had seen her. (3.5-2) John said to Mary, “ I saw you.” (3.5-3) John: I saw you. W ith case grammar representation [31], which is used in parsing output, the first person usually denotes the agent case of the event, and the second person denotes the experiencer case. As shown in Figure 3.2, in a direct narration case, the system propagates the person markers from the mtrans-event through the agent and the experiencer links, and does the same procedure in the gender and number constraint checking. In (3.5-2), John attains 20 for I and 0 for you, while Mary attains 0 for I and 10 for you. (3.5-3) shows the pure dialogue. The dialogue between two participants can be processed in the same manner of the direct narration, provided that the system can identify the participants. However, in this thesis, the conversational discourse is not considered, though this type of constraint can be checked, as described above. C -co m m a n d 27 itrans-event object agent. experience! John Q M ary Male experien( Second Singular __ see-even) object Male First Singular you Singular First Second itrans-event object agent experiencer John, see-even) object Figure 3.2: Person constraint. (3.6) Tom killed him. The other im portant syntactic constraint is about the use of reflexive pronouns. In (3.6), it is said that Tom c-commands him [58], and in this case, him cannot be Tom. In order for Tom to be the object of kill, the reflexive pronoun him self should be used instead of him. In the marker-passing approach, c-commanding status can be easily identified by propagating a marker from the subject Tom through other case links such as object or experiencer cases. S o lu tio n for c-co m m a n d in g ch eck in g : 1. While parsing, a subject concept is marked with a SBJ marker, and an object concept is maxked with an OBJ marker. 2. A C-CMD marker is propagated from the SBJ marked concept through the proper case links. 28 3. If the C-CMD marked concept is a pronoun, which is not a reflexive pronoun, the SBJ marked concept cannot be a referent. 3.3 Semantic constraints The semantic constraints usually must be satisfied, except in m etaphorical or humorous discourses, which are not considered here. Semantic constraints are classified into three types in this thesis, which are discussed below. B e tw e e n a verb an d its case One type of constraint exists between the verb and the noun/pronoun, which is bound as a value of each verb case role. For instance, the agent case of drive should be a person, and the object case of it should be an automobile. This constraint is represented in Figures 2.2 and 2.3, and the constraint is checked while parsing, except when a pronoun is used. When a pronoun is used, constraint checking is done in the reference resolution process by the marker propagation through the case link. For example, a memory- based parser generates a concept sequence instance drive-event#l, when parsing a sentence He drove it. It is represented as an object case of drive-eventfyl. This it should be an automobile by the constraint between a verb and its case. The solution for checking this type of constraint is like below, and the example is shown in Figure 3.3. S o lu tio n for ch eck in g v erb -ca se sem a n tic co n stra in t : 1. Check if a pronoun is used or not. If a pronoun is used: 2. Propagate a marker M l through case link, concept sequence instance link. 29 drive- event instance hypemym (^automobile^) person drive-| even t#! instance car#4 (^person^^) Figure 3.3: Semantic constraint: Verb Case. 3. Propagate a marker M2 from M l through hypemym3 link (if it exists) and the same case link. 4. Propagate a marker M3 from M2 through the concept instance link. 5. Concept instances with M3 markers are the candidates for the referent. B e tw e e n a co n c e p t and its a ttr ib u te The second type of constraint exists between the concepts and their attributes or properties. For instance, a car cannot be sad, nor happy, and a car cannot have a house as its property. Constraints between verbs and adverbs, and be tween adverbs and adjectives are not considered in this thesis, because this thesis mainly deals with pronominal anaphora and does not deal with proajectival or proadverbial anaphoras. The constraint check requires another relation link constraint link in addition to the links described in Section 4.4 for the knowledge representation of relations between concepts. In this thesis, the constraint link represents the constraint of 3This is equivalent to the usual ISA link. This will be discussed in Section 4.4. 30 constraint Cghysical-objec£>^— C^^ight-descnpt^ <f~rock~^>— — ►<T~heavy~''} ----------------attnbute —------ Figure 3.4: Semantic constraint: A ttribute. the concept which represents an attribute. For example, in the utterance i t ’ s heavy, it has heavy attribute, and heavy should be an attribute of a physical- object (constraint link). So, it should be a physical object. This example is shown in Figure 3.4. Constraints about properties can be checked in the same way. S o lu tio n for co n stra in t check b etw een a co n cep t and its a ttr ib u te s : 1. If a pronoun with an attribute concept is found, a marker M l is prop agated from the attribute through the constraint link and concept in stance link. 2. Concept instances with M l markers are the candidates for the referent. B e tw e e n co n ce p t and its su b su m er The third type of constraint is defined between the concepts and their sub- sumers, which are nouns, as complements in a sentence. For instance, in a sen tence, it is a large boat, the referent of it is searched among instances of boat and its subsumee. In some cases, a specialization process maybe required. (3.7-1) There’ s a large ship at the dock. (3.7-2) I t ’ s a U.S. N avy’ s new cruiser. 31 state-description#l new ' object object attribute ship#0 ). cruiser#3 dock#4 location owner (^ m v y ^ ^ ^ ^ n a tio n a lity United States Figure 3.5: Semantic constraint: Subsumption. It in (3.7-2) refers to a large ship, where ship is not a subsumee of the cruiser, but a subsumer. Therefore, if a referent is not found among subsumed concepts, a referent is looked for among the subsumer. The nearest concept in a subsumption hierarchy gets the highest value, and is selected as a referent. Then, the referent concept is specialized as a referring concept. In (3.7), ship instance is substituted with the more special cruiser concept. Figure 3.5 shows the result after reference resolution. W ith definite noun anaphora, this constraint is very im portant. In the sentence the car was new, the referent of the car should be a subsumee of the car concept or a very close subsumer (e.g. automobile). S o lu tio n for co n stra in t check in su b su m p tio n h ierarch y : 1. Propagate marker M l through the concept instance link and hyponym link. 2. Propagate marker M2 through the hypemym link and concept instance link with reduced value for every level of hyponym link. 3. Let M l have a higher value than the initial value of M2. 4. After reference resolution, a more specialized concept represents the referring and referent concept. 32 3.4 Discourse preference Discourse information serves as a preference in the reference resolution. The importance of this information is discussed in Chapter 2 in detail. However, as discussed before, previous researches about discourse for the reference resolution have several deficiencies, and a new marker-passing model is proposed in this thesis. Since this is the key issue in this dissertation, this issue is discussed separately in Chapter 4. 3.5 World knowledge The world knowledge includes many things, such as subsumption hierarchy with properties, attributes and semantic constraints, discourse pragmatics, and rela tions between concepts. This thesis does not consider complicated discourse prag matics, which requires complicated inference mechanisms beyond the scope of this thesis. However, some simple inference on the world knowledge is performed in the proposed model. For example, when a restaurant is mentioned and the waitress is mentioned successively, the waitress refers to the waitress of the restaurant which is mentioned just before, and this can be recognized by associating concepts. This is related to the implicit definite reference and is discussed in Chapter 4. 3.6 Preference on linguistic parallelism In some cases, pronoun interpretation is influenced by the syntactic parallelism and semantic parallelism. The effect of parallelism is pretty strong, so the result of parallelism sometimes can override the interpretation of a pronoun derived from discourse preference. We will see one example in (3.8). (3.8-1) The green W hitierleaf is commonly found near the wild rose. 33 (3.8-2) The wild violet is found near it, too. 4 Though the descriptor the wild rose has less discourse preference,5 it in (3.8-2) is interpreted as the wild rose instead of the green Whitierleaf, due to the parallelism on the phrasal structure. Thus, the preference for the parallelism on the phrasal structure should be stronger than the preference on the activeness, and this should be considered in the reference resolution. The recognition of a linguistic parallelism is a very difficult task6, because many kinds of world knowledge and inference are required to detect parallelism, and in extreme cases, the whole paragraph can have parallelism with another paragraph in a discourse [54]. Linguistic cues (e.g. too) are very im portant to catch this kind of parallelism, but it is not considered in this thesis. (3.9-1) Lynn raced her. (3.9-2) She defeated her. 7 Here, only the parallelism on the subject position and a simple parallelism on phrasal structure are considered. In the above example, she and her in (3.9-2) can refer to either of Lynn or her in (3.9-1). By the parallelism on the subject position, she is interpreted as Lynn. S o lu tio n for th e sim p le p a ra llelism : If a pronoun anaphora is found on the subject position, concepts with SBJ marker get higher points as candidates for the referent. A phrasal struc ture can be easily compared by propagating markers from concept sequence instances through concept sequence instance links. 4This is taken from [54], 5The reason is discussed in Chapter 4. 6There’s no system which can detect the parallelism generally [112]. 7This exam ple is taken from [13]. 34 In the next chapter, the issues related to the discourse preference is discussed, and all these constraints and preferences are integrated into a marker-passing algorithm to establish a marker-passing model for definite reference resolution. 35 Chapter 4 Activeness as Discourse Preference (4.1) In Columbus, Ohio ... police said a 14-year-old boy was beaten and stabbed to death ... because he refused to give up his Los Angeles Raiders’ Starter jacket to drive-by attackers. Dewayne Williams Junior was on his way to the school bus when he was killed. Four people were later charged in the murder. 1 W hen reading this news article, a human can recognize that the 14-year-old boy is Dewayne Williams Junior without any difficulty. This is true even without because he ... drive-by attackers, where the pronominalization, which is used for previous local focusing approaches, occurs for the 14-year-old boy. It is because 14-year- old boy is the topic entity of this article. In other words, when reading the first sentence, readers expect to read more about the 14-year-old boy. This example tells us how im portant the discourse topic is in the definite reference resolution. Therefore, the basic concept of this model is: • keep the discourse topic in attention and • utilize it in the reference resolution. '-This example is taken from internet news group clari.news.cast. 36 The premise of the proposed model can be described as below. Before presenting this model for the reference resolution with the premise, let me define the term activeness in this model. To define activeness, several linguistic and psychological aspects should be considered. In psychology research concerning verbal learning [3], Anderson stated that • Traces once formed are not lost, but the strength of it decays. • The strength of the trace decays in the following formula. S = t~b • A memory unit called the cognitive unit has the retention represented by the following formula. s = T .ii: h This says th at a memory unit becomes more active when the same unit is accessed repeatedly, and a memory unit becomes less active when the unit is not accessed. A discourse topic and local focus can be represented by one of these highly active memory units, because the concepts which draw more attention are in the highly ( active memory units, At this point, a new definition of activation fo r the reference resolution is required because of the following reasons. • The reference can be used only for the concepts which is activated with I reasonable strength. j i • Even at the same time, each concept is at a different activation level in a natural language discourse. • Activation level can change in a different way from the given formula due to natural language discourse phenomena. j l • In natural language processing, artificially established knowledge networks ; j are used, and the representation can be very different from the real human ; memory architecture. ' 37 ! ___ i Therefore, a different definition from activation in psychology is required to handle reference problems, and this results in the following definition of activeness. i ! ; 4.1 Definition In tu itio n : The activeness of a concept is the artificially defined degree of atten tion by the listener to that concept in a discourse. P re m is e : The most active concept in the listener’s mind is considered as a referent, unless it is not syntactically or semantically acceptable. • The concepts in the discourse perceiver’s mind have different degrees of activeness in a discourse. i i ] • Activeness of a concept decays, if the concept is not brought into at- j tention again with direct or indirect mentioning. j • Activeness of a concept is strengthened, if it comes out more to the ; | foreground by direct or indirect mentioning. , D efin itio n : The quantified degree of activeness is defined as partial orders and j a global order discussed in the following sections. j ! Based on the above assumptions, a model which achieves better reference ! resolution is proposed. As told in the premise, the most plausible concept as ! a referent of the pronoun reference is the most foregrounded concept, which is j * I ‘ acceptable either syntactically or semantically. This concept is considered as a discourse topic in the subdiscourse. To find out the most foregrounded concept(s), ! the activeness of each concept is calculated. In this approach, the activeness of all relevant concepts is evaluated, instead of computing one local focus. Then, the ' referent is chosen according to the computed activeness with other syntactic and : semantic constraints. 38 4.2 Activeness assignment In previous approaches, the preference on discourse is put in order to decide the fo cus. For example, < objectl, object2, other NP, VP > [112] or < subject, objectl, object2, other NP > [13] are those orders. In the proposed approach, each concept, which appeared in a discourse, is treated in two groups at first. The descriptor t j concepts, which are concepts as modifiers, get less activeness. The non-descript or j concepts, which are not modifiers, get more activeness [54]. The descriptor con cepts include the prepositional phrases and the relative pronoun clauses, and the descriptor concepts in the descriptor clause have even less activeness2. Then, as the discourse proceeds, the activeness of each concept is changed — strengthened, | retained or weakened. ! (4.2-1) Tom drank the wine on the table. (4.2-2a) It was good. (4.2-2b) It was brown and round.3 j Hirst said that (4.2-2b) was described as an absurdity by some listeners [54]. In the experiment on a small group, someone felt (4.2-2b) even as a m etaphor of the taste of wine. This means the table is not in the listener’s topic, and cannot be pronominalize naturally. To bring table into the topic naturally, the table should be used instead of it. Hirst explains why it in (4.2-2a) is easily comprehensible, and it in (4.2-2b) is not. T h at’s because the table is a descriptor of the wine and j I it is not a concept in its own right. In this thesis’s term , th a t’s because the table \ i is less active than the wine. j I I (4.3a) j Bush’ s successor Bill Clinton prom ptly reaffirmed the president’ s ac- : tions, but indicated he supported even broader regional agreements. I ------------------------------------------------------------------------------------------------------------------------------------------- j | 2This is not distinguished in the im plem entation, because the concepts in the descriptor ' ' clause are rarely referred by pronouns in a clear discourse. j ! 3This is taken from [54]. 1 39 In (4.3a), he is naturally interpreted as Bill Clinton instead of Bush which is a descriptor of Bill Clinton. I ; ( 4 . 3 b ) Gary Steven Foster who made a threat to kill President-elect Clinton has been sentenced to 18 months in prison. He pleaded guilty in federal court in Chicago yesterday.4 In (4.3b), he is naturally interpreted as Foster, which is not a descriptor, instead of Clinton, which is a concept in a descriptor clause (relative pronoun clause). This gives the following partial order of activeness denoted here as Act. P a rtia l O rder 1 : j A ct descriptor < A ct non-descriptor This activeness is represented with ACT marker in the semantic network, and j the ACT marker contains the appropriate activeness value. Then, as mentioned before, the activeness of each concept is changed — strengthened or weakened — as the discourse proceeds. Strengthening and weakening of activeness is discussed later in Section 4.5. I ! 4.3 Propagation i When a human hears about some entities, he/she not only thinks about that entity, but also think about other related entities, even episodes, in th at context. ' Similarly, once a concept becomes active, its activeness is propagated to related , I concepts. The following formula about spreading activation among memory units | j was suggested in [3]. I 4This article is taken from internet newsgroup clari.news.cast. 1 40 , Jxy&x + where node ny receives activation a,\y - aiy from nodes n\ - rii, I is the loss in activation, Sk/Yijsj is the relative strength of node rik from node nx, and f xj = l$k/I2jSj if node nj is connected to nx f xj = 0, otherwise. However, this formula is not proper for this thesis about definite reference resolution in a computational way. First, it’s difficult to define a knowledge base, which not only represents human memory but also matches the suggested m ath ematical model in psychology. Second, the m athem atical model for spreading activation is overly complicated for the reference resolution model. For spreading activation, two kinds of simplified activeness propagation are defined in the proposed model: expectation and confirmation. Both propagations are to find the concepts which are related to currently active concepts. In other words, these propagations associate the concept, which has just appeared in a discourse, with the concepts, which have already appeared in a discourse. An expectation procedure anticipates the concepts which may become active in the following discourse, while a confirmation procedure confirms the previously active concepts to be still active or expected, because those are closely related to the currently active concepts. (4.4-1) Ben had dinner at Red Lobster. (4.4-2) The chef cooked very well. (4.4-3a) It is the best restaurant near here. (4.4-3b) It was the most delicious dinner he has ever had. At the point of (4.4-1), a discourse topic can be Ben or dinner, or less probably it can be Red Lobster or eating-event. At the point of (4.4-2), the discourse topic seems to be chef, dinner or Red Lobster. Finally, at (4.4-3a), the discourse topic appears to be Red Lobster, or dinner (which Ben had) at (4.4-3b). This process can be well explained using activeness propagation. 41 ISA link (hypem ym ) eat-event animal food case link Expected Expected other inheritable relations dinner A ctive 1 i l <^t-evenST ^i Expected Expected (^w aitress} Expected A ctive dinner#! menu Expected estaurant A ctive Expected Figure 4.1: Expectation E x p e c ta tio n An expectation can be interpreted as a weak activation. W hen a concept be comes active, it can affect other related concepts, which are linked to the active concept by arrows, as shown in Figure 4.1. In the first sentence, when the concept Red Lobster becomes active, it’s natural for the related concepts (which are inheri table from restaurant concept) like waitress, menu, chef, etc. to be expected in the next discourse. (Or, when chef becomes active, restaurant is expected.) W hen the Red Lobster concept becomes active, its activeness is propagated to the chef, wait ress and menu concepts, by the expectation process. The expectation is cleared, if it is not realized directly or indirectly in the subsequent discourse. One im portant issue here is how much activeness will be increased by this ac tiveness propagation. As discussed before, Anderson’s m athem atical model cannot be applied. Activeness increased by the expectation process is represented as a partial order below. This is because direct realization has a bigger effect than indirect realization. 42 P a rtia l O rder 2 Act ^creased by expectation < descriptor Though the activeness values of the concepts, which are expected by this type of activation propagation, are only slightly increased, (1) the expectation broadens the search scope for definite noun anaphora by adding expected concepts to the search scope, and (2) the expectation relates the expected local focus (e.g. chef) . to the current more global discourse topic (e.g. dinner (at Red Lobster)). One very im portant thing to note here is that the direct realization and the in direct realization (expectation) are treated differently when computing activeness. If this is explained in the Anderson’s psychological model, the direct realization corresponds to the trace, which are not lost, but the indirect realization only contributes to increase the activation level. Therefore, as shown in the Figure 4.1, expectation is represented by EXPECT markers, not by ACT markers. Activeness value of a concept is computed by adding values in the ACT marker and the EXPECT marker, which are marked on the concept. The ACT marker is not removed throughout processing the whole discourse, as the trace is not lost. But, the EXPECT marker is removed if the concept is not realized. E x p e c ta tio n R u le : All subsumers, inheritable properties and attributes of the active concepts are expected. Also, all subsumed concepts are expected. The constraints of unfilled m andatory cases (in intransitive usage of transitive verbs) represent default values, when they are not mentioned explicitly. For ex ample, food is a default value in H e’ s eating, now, and an automobile is a default value in (4.5-1). If event nodes are active, semantic constraints of m andatory cases (grammatical subject and object) are expected. This is because this type of constraints are inheritable from the event concept to the event instance. (4.5-1) Tom was driving along the freeway. (4.5-2) Suddenly, the engine made a funny noise. (4.5-3a) He was very upset, because the car was new. (4.5-3b) He was very upset, because it was a new car. 5 This example requires us to solve several problems: (1) two N P’s — the engine in (4.5-2) and the car (or it in (4.5-3)) — should be resolved, though the referents are not explicitly mentioned, (2) Tom should be in attention throughout the dis course so that the later he in (4.5-3) can be resolved, (3) the whole discourse is about Tom’s driving episode, and (4) in the following discourse, multiple frames (Tom and the car) should be in attention. First, an automobile is a default object value of driving-event in (4.5-1). It’s natural th at this concept is expected, and the expectation rule provides engine concept for the anaphora resolution. By creating an instance of the default value as an object case of driving-event and applying expectation rule, the first goal is achieved in the model. The second part will be discussed in the next type of propagation — confirmation. Third, by expecting and confirming through the link between related events and concepts about driving, the proposed model can solve the third problem. This will be discussed later. Fourth, computing the activeness of concepts avoids this multi-frame focus problem, because m ultiple concepts can have high activeness. Another im portant thing to discuss about expectation is the range of active ness propagation. More active concepts propagate their activeness (expect) in wider range theoretically. However, the expectation range is limited to one link, because this is performed in an artificially formed knowledge base. In other words, various kinds of artificial knowledge base is not appropriate to apply the active ness strength dependent propagation. This will be discussed more in the Section 4.4. 5This exam ple is inspired from [54]. 44 ISA link (hypemym) case link other inheritable relations eat-event animal dinner Expect Active confirm person Active g^eat-event#T^) waitress Expected Ben#l Active dinner# Restaurant ed Lobster# Active confirm! menu Expected confirm Expected cook- event confirm links which created newly after reference resolution <5ook-event#J>- Newly Active <^chef#r^) Newly Active Figure 4.2: Confirmation In the expectation process, the following partial order of activeness values can be drawn. However, the second partial order is not considered in the implementa tion, since it doesn’t influence much and is not required for the reference resolution goal — highly active concepts are referred in a clear discourse. P a rtia l O rder 3 : A rt expected concepts '> A c tno^ eXpecfed concepts A rteX pecfed fry highly active concepts ^ A d eX pected by lowly active concepts C on firm a tio n First, the confirmation process helps find a discourse topic. In the second sentence of the previous example, because the Red Lobster and dinner concepts 4 5 are currently active and local discourse about the chef and cook event is initiated, the global discourse (more global than about the chef) may still be about the Red Lobster or dinner. (This is shown in the third sentence.) Second, the confirmation resolves associative anaphora (implicit reference by using the + noun, which is related) by cooperating with expectation. As the process finds the relation between the newly active chef and the previously active Red Lobster, (chef was expected by Red Lobster) chef is resolved as the chef at the Red Lobster where Ben had dinner. The confirmation rule described later considers the above concepts as candidates for the discourse topic. This is shown by the newly created links in Figure 4.2. In (4.5-2), driving-event of (4.5-1) is confirmed by the active concept the en gine, its agent Tom is still active in (4.5-3). Also, the instance of automobile is confirmed to be still expected in (4.5-2), therefore the car in (4.5-3) can be resolved. C on firm ation R u le : The active concept(s) which expect the newly active concept(s) is confirmed to be still active or expected. Also, the active concept(s) which is expected by the newly active concept(s) is confirmed. The effect of the confirmation reduces the weakening of the activeness strength, because the confirmed concepts should be still in attention. In other words, if a concept is not realized by direct mentioning, it loses its activeness and becomes inactive. W ith confirmation, it does not lose the whole activeness and remains still active. The confirmation also spreads activeness as expectation does. The difference is that the confirmation is propagated in the reverse direction of the expectation. j Confirmation and expectation are equivalent to the indirect realization of a [ concept, but confirmation follows the direct realization. Obviously, directly re alized concepts are more active than indirectly realized concepts. Therefore, the 4 6 following partial orders hold. The first inequality is a re-statem ent of Partial Order 2. P artial O rder 4 : Act directly realized concepts realized concepts Act confirmed concepts Act expected concepts 4.4 Knowledge representation Knowledge representation plays an im portant role in the propagation of activeness. Though the basic idea of the activeness propagation discussed in the previous section does not depend on the knowledge representation scheme very much, the actual implementation depends on the representation scheme. The knowledge base consists of two major parts: semantic concept hierar chy and concept sequences. The semantic concept hierarchy consists of semantic concept nodes and relation links, and concept sequences consist of many phrase patterns for memory-based parsing. These two m ajor parts are linked through case links which represent the semantic constraints for concept sequences. The current knowledge base uses the semantic concept hierarchy shown in Figure 4.3, which is based on Wordnet [82]. It consists of actual words, synsets and links. Synset represents the semantic meaning of the word, and multiple words are linked to one synset, and one word which has multiple meanings is linked to m ultiple synsets. Also, a synset can be linked to another synset, which is its hypernym. L inks in W o rd N et : • hypernym (isa), hyponym (animal is a hypernym of dog.) 4 7 / hypernym/ part-meronym ___ __ _ — — "w Q_synset #1 j) hypernym / word (^sy n set C word#4 J ) f word hypernym / ([[^synset #3~~^) (^~ w ord#3~ ^^) / word word / ( ^ [ [ ) w o r d # l C ^ ^ w o r d # 2 Figure 4.3: Concept Hierarchy in WordNet • word (lexicon) • member-meronym, member-holonym {family is a member-meronym of sister.) • substance-meronym, substance-holonym {juice is a substance-meronym of water.) • part-meronym (has-part), part-holonym {automobile is a part-meronym of engine.) • antonym Concept sequences shown in Figure 4.4 consist of two m ajor subparts, basic concept sequences, which include verbs in them and are used to parse clauses or phrases including verbs, and auxiliary concept sequences, which are for adverb phrases. [24] Each of the concept sequences consists of CSR layer and CSE layer, which are linked to semantic concept hierarchy. Parsing a sentence results in one or more instances of concept sequences and instances of concepts. One thing to note here is the importance of the links between CSE layer and the semantic concept hierarchy. As described before, one word has multiple meanings, 4 8 Concept Sequence Layer [agent, murder, object] murder-event next murder Semantic Concept Hierarchy semantic animate > r inanimate isa human) /'non-human uman-groups rperson <J^orism -organization> R ation J s a JW (sM ^ng-pa£) (FMLN r Syntactic Constraint Layer instance Lexical Entry the shining path Figure 4.4: Concept Sequences thus linked to many synset nodes (e.g. noun concepts) or basic concept sequences (e.g. verbs). The links between concept sequences and semantic concept hierarchy enforces to satisfy the constraints for each verb case while parsing, and this results in word sense disambiguation. It is meaningless to define how far the propagation should spread without knowing what concepts are there in the knowledge base and how they are linked to each other. There is a big difference between the knowledge base with auto mobile has-part m otor and motor has-part piston and the knowledge base with automobile has-part piston explicitly. In the current implementation, activeness is propagated through one link (besides j'sa-link), regardless of the strength of the activeness. Thus, in the knowledge base without automobile has-part piston, activeness of automobile is propagated upto motor. If piston is strongly related to the automobile concept and should be in attention, the knowledge base should 4 9 be designed in the way that all strongly related concepts are linked directly, in order to work with the proposed model.6 R e la tio n b e tw e e n ev e n ts (4.6) Kevin took out Cathy yesterday. They had a great dinner at Laguna Beach, and walked along the seashore. They went to the movies at night. It was a great date. (The date was great.) Previous computational models cannot resolve the references to the subdiscourse or subdiscourse related associative anaphora problems shown in (4.6). However, the proposed model finds out the expected discourse topic by activeness prop agation through the semantic network, when there is proper knowledge about relations between events. These relations between the events look like simplified script [107] representation. Key events or concepts are linked through the proper relations and nodes which represent scripts. A script node can be linked to other closely related script nodes. And the activation of the key event or key concept spreads its activation to the other related key events and concepts through nodes, which represent scripts, during the expectation process. As long as the discourse remains inside the same context, related concepts and events are expected, and the reference which refers to the concepts related to the expected script can be resolved. In this model, there is no script instantiation problem (which script should be instantiated at the beginning of discourse?), because scripts are not instantiated, but expected. Therefore, as discussed in the expectation process, scripts which are not related to the current context are not expected any more as discourse proceeds. 6W ithout considering much about the knowledge base structure, this problem can be solved i by adding a little inference routine. In other words, then piston is referred, the inference routine ] can find out the path to the automobile and related them. This is out of this thesis’s scope. 5 0 loc. meeting-script key. role meeting agt. inv. mv. exp. agt. person Figure 4.5: Relations between events. In order to represent simplified script, three more types of links are used to rep resent relations between key events and key concepts, besides the links which are used for the semantic concept hierarchy. These are: involve, role and environment links. The involve link represents the related key events to a key concept. For example, the m eeting concept involves discuss-event. The role link represents the role characters of the key concept or key event. For example, the role characters of the restaurant concept are the chef, waitress and customer. The environment link is a reverse relation of the involve link. An example is shown in Figure 4.5. One thing to note in the figure is that some have one directional links (e.g. property) and some others have bidirectional links (e.g. property and environment). This is very im portant in activeness propagation. The reason is that the chef, menu, and customer are expected by the activation of restaurant, but the restaurant is not expected by the activation of the customer. As discussed before, the figure shown is a kind of simplified script representa tion, but this representation cannot specify the constraints between related events and concepts. For example, the experiencer case of meeting-event should be the same as the agent case of discuss-event. But this cannot be represented with this simplified representation. However, it is thought to be sufficient for the reference resolution at this stage. 51 E x te n d e d E x p e c ta tio n R u le : Active key concepts or key events expect other related concepts and events. Thus, an active concept expects (1) its subsumers, (2) the all inherited attributes and properties of its subsumers, (3) its subsumees, and (4) other related scripts and their concepts events, if any exist. R efere n c e to a su b d isco u rse In the example (4.6), it refers to the whole discourse about Kevin and C athy’s date, or the date refers to the central concept date expected by the whole discourse. In the activeness propagation process, the key concept date keeps being expected with appropriate relations between concepts and events, and all related concepts to the key concept date are expected, too. Therefore, either it or the date can be resolved. E la b o r a tio n p ro b lem Another advantage of the proposed model is that the extended expectation rule also helps to solve the elaboration problem in the reference resolution, because it prevents the weakening of a concept activeness throughout the long elaboration. Just as in previous approaches, without script-like knowledge, this model with the expectation and confirmation processes can have problems in the long elabo ration which doesn’t associate concepts directly. In other words, since the basic expectation and confirmation processes associate only concepts which are directly related, some im portant concepts (one which maybe a discourse topic) can lose their activeness in a discourse with a long elaboration process. (4.6) is an ex treme case of the above problem, where the main concept (discourse topic) date has not been mentioned until the reference occurs, and other related events are continuously mentioned. However, this problem is solved by using the extended propagation rule based on the simplified script-like representation with additional links between concepts and events. Key events expect a key concept and all concepts and events related to the expected key concept. Therefore, as long as the discourse remains inside the 5 2 context related to the expected key concept, other related concepts and events are expected or confirmed, avoiding the elaboration problem described before. Thus, anaphora, which refers to the concepts related to the expected concept, can be resolved. However, the lim itation of this model is th at it cannot handle different kinds of elaboration, such as theoretical writings, because it is a very difficult problem to find out an elaboration with only a discourse and relate concepts. In the following example, this model cannot find the elaboration of the ease and security of communication by clearing pirates and building and repairing the roads. This type of elaboration requires very sophisticated reasoning, which is beyond the scope of this thesis. (4.7) A good share o f the amazing revival of commerce m ust be credited to the ease and security of communications within the empire. The Imperial fleet kept the Mediterranean Sea cleared o f pirates. In each province the Roman emperor repaired or constructed a number of skillfully designed roads.7 4.5 Strengthening and decaying activeness As stated in the premise and discussed with Anderson’s model, the activeness of a concept decays, unless that concept is brought directly or indirectly into the listener’s attention again. Though the retention function of the memory unit discussed in Section 4.1 never goes to 0, the proposed model uses the different value to construct a reference resolution model. The retention function shows that the strength of the trace reduces very much at first, then reduces by less amounts gradually. In the reference resolution model, it is assumed that the concept does not have enough activeness after the first drop of activeness (i.e. the strength of 7This is taken from [62], 5 3 trace). For convenience, this level of activeness is represented with 0, and later decay of the activeness is not in concern, because it doesn’t affect reference at all. A concept can be brought directly into attention by mentioning it again, or referring to it with an anaphora or other means of the definite reference. When a direct realization (e.g. pronoun anaphora in (4.8)) is found, the activeness of the concept becomes higher than before. This fact is also shown in Anderson’s retention function. A direct successive realization of a concept is equivalent to repeated verbal training, but another factor should be considered in natural lan guage processing. A repetition of the same concept implies that a speaker/writer wants to talk about that concept — a discourse topic or a local focus. Therefore, some extra strength should be added to the concept. (4.8-1) Tim went to a restaurant. (4.8-2) He ate too much. The indirect realization (e.g. mentioning closely related concept) is equivalent to the confirmation process. In the following two sentences, the waitress is an indirect realization of a restaurant, and this is found by the confirmation process. (4.9-1) Tim went to a restaurant. (4.9-2) The waitress was very kind. W hen an indirect realization is found, there is one similarity to the direct realization case, while there is one difference from the direct realization. First, the difference is th at the activeness cannot be increased more than the previous activeness. This is obvious from two aspects. The first one is stated in Partial Order 4, and the second one is shown in the retention function. The strength drops sharply, but the activation propagated from other related concepts are not as much as the drop. The similarity is that some extra strength should be added to the concept, because of the activeness propagation to the indirectly realized concepts. 5 4 The examples collected from the news articles show that the concept referred by a pronoun is always active (at least expected) in case that the pronoun refers to the subdiscourse. Thus, as discussed before, it seems to be sufficient to use a model where all active concepts lose their activeness to 0, if they are not realized directly or indirectly in the next sentence. Activeness does not decay below 0, as the retention function implies that the strength of trace is never lost, and the ACT marker which represents activeness is not cleared throughout the discourse. Thus, the following partial ordering is considered. The last one is obvious, because confirmation is also a same kind of activeness propagation as expectation. Act = 0 in this model means active, but not active enough to be the referent. In other words, in Anderson model’s term , the activation strength of inactive concepts is lower than the strength, which is desirable to be referred by some reference. The second inequality is a re-statem ent of second inequality of Partial Order 1. P a rtia l O rder 5 : Act mentioned before, but not realized ~ ^ A d re-mentioned successively ^ A ctnewiy appeared W ith this approach, a discourse topic can be found among active concepts which include expected concepts, even though they may not have the highest activeness. By using activeness instead of one or two local foci, the proposed approach provides a broadened scope in searching for a referent. It can handle multi-focused discourses which cannot be handled in the previous local focusing approaches, it doesn’t suffer from the frame-selection problem, and consequently, it can provide more resolution capability. 4.6 Global order in activeness In order to define the degree of activeness for the reference resolution model, a global order in various cases should be defined based on the partial orders discussed before. The classification of the concepts according to the activeness level is: 5 5 • descriptor vs. non-descriptor • has been realized vs. has not been realized at all • directly realized vs. indirectly realized • successive realization vs. first realization The combination of these classes makes the following classes of concepts, (i) successive direct realization as non-descriptor concepts, (ii) first direct realiza tion as non-descriptor concepts, (iii) successive direct realization as descriptor concepts, (iv) first direct realization as descriptor concepts, (v) successive indirect realization (confirmed concepts), (vi) first indirect realization (expected concepts), (vii) concepts realized before but not realized currently, and (viii) inactive con cepts. According to the partial orders 1 through 5, • (i) > (iii) : by Partial Order 1 • (ii) > (iv) : by Partial Order 1 • (i), (ii), (iii), and (iv) > (v) and (vi) : by Partial Order 2 • (v) and (vi) > (vii) : by Partial Order 3 • (i) > (ii) : by Partial Order 5 • (iii) > (iv) : by Partial Order 5 • (v) > (v0 : by Partial Order 4 This results in the partial order graph in Figure 4.6. As we can observe, the relation between (ii) and (iii) is not clear. In (4.10), he in the third sentence is likely to be recognized as John instead of Jeff (the referent of his), where the subdiscourse is (4.10-1) = 4 - (4.10-2a) = 4 - (4.10-3). However, if the subdiscourse is (4.10-1) = 4 - (4.10-2b) = 4 * (4.10-3), it’s more confusing. In either case, John or Jeff should be used instead of he to be a clear discourse. Anyway, if he is used, it’s 5 6 (iv) (vi) ,(vii). Figure 4.6: Partial order graph. very difficult to decide a referent without inference. However, in the analysis of news articles, it was observed that (ii) class concepts are more likely to be the referent than (iii) class concepts. Therefore, (ii) > (iii) seems to be a reasonable assumption in this approach, though inference should be the m ajor factor to decide the referent in this case. (4.10-1) Jeff is a very popular guy. (4.10-2a) John is his best friend. (4.10-2b) John likes his friendliness. (4.10-3) He is a student at USC. One more thing to be noted here is th at directly realized concepts (and also indirectly realized concepts) are treated equally, no m atter how many times they are repeatedly realized. Actually, in the case of direct realization, repeated real ization should fortify the activeness more, and in case of indirect realization, it depends on which of the following two factors are considered more critically: (1) ! the extra strength related to the possibility as a discourse topic, or (2) the differ ence between the sharp drop in the first time period and the activeness increase by 5 7 high in activeness low in activeness (vii) (viii) Rest of the KB (inactive) (iii) (vi) (iv) Figure 4.7: Activation levels. activeness propagation (expectation). This is a very difficult thing to decide, but not as useful in the reference resolution. Therefore, this has not been considered in the thesis. A pronoun reference and a proper noun reference are treated differently, though both of them are direct realizations of a concept. T hat is: a proper noun is not treated as a repeated direct realization unless it is successively used, while a pronoun reference is always treated as a repeated direct realization. Actually, if a pronoun reference can be recognized clearly, it is used either successively or in very close range. 5 8 S i: B en h ad dinner at R ed L obster. S2: T h e ch ef cooked very w ell. S3a: It is th e b est restau ran t near here. S3b: It w as th e m o st d eliciou s dinner. A c tiv a tio n lev el after each sen ten ce SI S2 S 3 a /S 3 b p e r s o n a l (B en ) (ii) (v ii) (v ii) / (v ii) dinner # 2 (ii) (v ) (v ) / (i) restau ran t # 3 (iv ) (v ) (i) / (v ) e a t-e v e n t# 4 (iv ) (v ) (v ) / (v ) ch ef (p e r s o n # 5 ) (v i) (i) (v ) / (v ) c o o k -e v e n t# 6 (v i) (iv ) (v ) / (v ) st ate- d e sc r ip tio n # 7 (v iii) (v iii) (iv ) / (iv ) Table 4.1: Activeness propagation example 1. The levels of activeness and the transition of activeness according to the partial orders discussed axe shown in Figure 4.7. The boxes higher in the figure have more activeness than the ones lower in the figure. The concepts over the horizontal line have ACT markers with a value 0 or higher. This can be explained as they have traces in the memory unit in the psychology research. The alphabet marks on arrows are the condition where the transition occurs, and they are described below. (a) first realization (mentioning) of a concept (b) indirect realization by expectation process (c) direct realization by definite reference (d) indirect realization by confirmation process (e) no realization 4.7 Examples of activeness change Table 4.1 and 4.2 shows the change of activeness for several key concepts in a small discourse. In the examples, activeness is represented with concept classes 59 S i: T om w as driving alon g th e freeway. S2: T h e en gin e sta lled . S3: H e w as d isa p p o in ted , since it w as a n ew car. A c tiv a tio n lev el after each sen ten ce S I S2 S3 p e r s o n a l (T o m ) (ii) (v i) (i) freew ay # 2 (iv ) (v ii) (v ii) d rivin g-even t # 3 (iv ) (v) (v ) a u to m o b ile # 4 (v i) (v) (i) (m erged to c a r # 8 ) en gin e (e n g in e # 5 ) (v i) (i) (v) sta ll-ev en t # 6 (v iii) (iv) (v ii) st ate- d escrip tion # 7 (v iii) (v iii) (iv ) c a r # 8 (v iii) (v iii) (i) Table 4.2: Activeness propagation example 2. (i)-(viii), as discussed before. The used value will be discussed in detail in Chapter 6. W ith the centering algorithm presented in [13], it seems that the first example cannot be resolved, due to the frame-select ion problem and no consideration about definite noun anaphora. In the second example, both algorithms in [13] and [112] seem to have problems in the resolution of the third sentence, and multi-frame focus problem (should the focus be he or it?) may occur after S3 [54]. The proposed model shows the correct resolution, as well as no suffering from the multi-frame focus problem, through using activeness of the relevant concepts. Also, especially in the discourse like (4.6), neither of the techniques in [13] and [112] cover this type of reference problem. Table 4.3 shows how this kind of problem can be solved through the activeness propagation in the semantic network of related key events and key concepts. Also, activeness propagation through key concepts and events helps to avoid an elaboration problem. 6 0 SI: Kevin took out Cathy yesterday. S2: They had a great dinner at Laguna Beach, S2: and walked along the seashore. S3: They went to the movies at night. S4: It was a great date. Activation level after each sentence SI S2 S3 S4 person# 1 (Kevin) (iii) (i) 0) (v) person# 1 (Cathy) (iii) 0) (i) (v) date (vi) (vi) (vi) (i) restaurant (vi) (vi) (vi) (vi) dinner (vi) (iii) (v) (v) date-script (vi) (v) (v) (v) walk-event (vi) (iv) (v) (v) seashore (viii) (iv) (vii) (vii) movie (vi) (vi) (iv) (v) Table 4.3: Activeness propagation example 3. Chapter 5 Marker-Passing Algorithm 5.1 Recalling active concepts Before discussing details of the marker-passing algorithm of the computational model for definite reference resolution, what kinds of concepts are referred by definite reference, and what kinds of definite references are used are discussed in this section. Active concepts can be referred by various kinds of reference. However, pronoun references are mostly used for the highly active concepts, while definite references using the -f NP are used for lowly active concepts and some other definite references, such as references using proper nouns, can be used for both highly and lowly active concepts, as well as for introducing new concepts. Highly active concepts include successively realized non-descriptor concepts and newly appeared non-descriptor concepts. Lowly active concepts include the rest of the concepts: newly appeared descriptor concepts, indirectly realized (con firmed) concepts, expected concepts, and concepts with 0 activeness value, which were usually the highly active concepts at one tim e in a discourse, as in (5.1). W hen selecting from 0 activeness concepts, recency information is im portant, because there can be multiple instances of the same class (meeting in (5.1)). Since the retention function (whose domain is time) is not used, recency is considered 62 current discourse event node Figure 5.1: Discourse graph. by some other way — a discourse graph. The discourse graph will be discussed later. (5.1-1) Tom will have a meeting with Ira. (5.1-2) They will use Tom ’ s office. (5.1-3) I t ’ s kind of small, but the meeting won’t take long. The recency is traced by linking event nodes, as in Figure 5.1 (a), in the order of recency. If a discourse segmentation process is combined, this graph can generate much better discourse structure, as in Figure 5.1 (b), but the discourse segmentation is out of the scope of this thesis. Recency will be discussed more in Section 5.4 with the algorithm description. However, many inactive concepts are also referred by the references using the+NP. Some of these are concepts in the m utual knowledge of the speaker/writer and the listener/reader. The examples of the reference to the concepts in mutual knowledge are shown in (5.2). (5.2) Tom was driving along the freeway. 63 Active memory (Instances + expected) expected expectation A CT = 0 cleared instantiation Inactive memory (Knowledge base) Figure 5.2: Memory partition and transition. The knowledge base is partitioned into two parts from the reference resolution point of view: active memory and inactive memory. Figure 5.2 shows the parti tions. The active memory consists of all instances of concepts and events which are generated by parsing. All instances in this memory partition are active and recallable. 5.2 Referability Before discussing the marker-passing algorithm for the reference resolution, the simple m athem atical term referability is introduced in this section. Based on various constraint and preference categories discussed in Chapter 3, referability is defined and used in computing the referent of a reference. The basic idea of the referability value is: for all appropriate concepts mentioned and for all constraint and preference categories, the system simultaneously computes the score as to how valid each concept is as a referent. 64 R efera b ility : The referability R e f of concept C by a reference R , which occurs in the utterance Ui, is the measurement which shows how much C is valid as a referent of R. For a utterance £/;, a concept C and a reference R , let me define: S xc(U i): score for syntactic validity of C as a referent of R in Ui N utuq: score for number constraint SexQ\ score for gender constraint P su q: score for person constraint SrriQ^Ui): score for semantic validity of C as a referent of R in Ui CC§(Ui): score for case constraint of C in Ui, substituting R ACc(Ui): score for attribute constraint of C as a referent of R in Uz S C q (U{): score for subsumption constraint of C as a referent of R in Ut A c te ■ score for activeness of C including expectation Subj§(Ui, Ui-1 ): score for parallelism on the subject position P hre(U i, Ui-i): score for parallelism on the same phrasal pattern R e f is computed as R & fc{U i) = Sxc(U i) + Smc{Ui) + A ctc + Para§(Ui, Ui-1 ), where Sxc{U i) = NurriQ + S ex§ + P suq Sm §(U i) = CC§(Ui) + AC§(Ui) + SC c(U i) Para%(Ui,Ui-i) = Subj§(Ui, U ^ ) + Phr*(U i, Ut- X) It should be made clear why the addition (instead of m ultiplication or some thing else) is chosen without any factors with the term s inside the formula. There 65 may be some complicated way which is perfect, however addition is sufficient to find the reference, provided that the values are properly selected. The reasons are: (1) all constraints and preferences are independent from each other, (2) it is good enough for the reference resolution to assign larger values for the critical categories, such as constraints, and smaller values for the less critical categories, such as preferences, and (3) the addition of values is the simplest computation. 5.3 Global order for constraints and preferences The most critical point in using R e f is how to decide the values for each constraint and preference categories properly. To choose the proper values, a global order for all constraints and preferences should be made, and then, values should be chosen according to th at order. Several partial orders should be set before setting up the required global order. First, assuming all constraints are satisfied in all clear discourse, high values should be assigned for the constraints and low values should be assigned for the preferences. By this, the concepts which satisfy the constraints can be picked. Because all the constraints should be satisfied, we cannot discriminate the impor tance of each constraint category. The partial ordering is given as: P a rtia l O rder 6 : score for Constraint » score fo r Preference C onstrainti — C onstraintj Second, because the linguistic parallelism on phrasal structures can override the increased discourse preference by reference, the first of partial orders below can be considered. However, linguistic parallelism is not completely checked in the implementation of this model, but the parallelism on phrasal structure (i.e. using same verbs) is detected. The parallelism on the subjects is considered using the second partial order. This preference is used as a tie-breaker, because it does 66 not play as a big role in preference as activeness. Parallelism is checked only for the adjacent two sentences. P a r tia l O rder 7 : P arallelism pfirasai structure ^ Actstrengthened by reference AActsmallest difference - > ^ >ara^ e^ sm subjects Third, the strength of recency effect should be considered. Usually, the active ness value reflects the recency of the concepts. However, activeness in this model cannot reflect the recency information, if the concept is not brought into attention for the tim e being. By putting time stamps on each concept mentioned (in other words, concepts which get some strength of trace), recency can be considered when necessary. Recency only m atters in the resolution of the + N P reference. G lo b a l ord er for referen ce reso lu tio n : each syntactic and semantic constraint > Act directly realized non-descriptor > A ctnewiy appeared non-descriptor ^ Act directly realized descriptor - > A c t n e w i y a p p e a r e d descriptor ->- Act indirectly realized concepts - > Act expected ^ Act directly realized before ^ P arallelism mbject From partial orders discussed so far, a global order shown above can be con sidered. However, the preference on linguistic parallelism on the phrasal structure is not shown here, because its partial order is given regarding the activeness dif ference. 67 5.4 Algorithm description O v erv iew Here is an overview of the marker-passing algorithm and the detail will be discussed later in this section. The proposed model considers the resolution of: (1) pronoun anaphora, (2) the + N P reference, (3) proper noun reference, and (4) reference using occupation of the person, but the main concerns are on the pronoun anaphora and the -f NP references. The resolution is carried out by computing R e f values of the relevant (active) concepts, and the flow chart is shown in Figure 5.3. W hen parsing output is fed to the reference resolution module, the module first checks if there is a proper name or occupation. If there exists any, it is processed by special routines. Then, the next block checks for a pronoun reference (or pronoun references). If one or more pronoun references are found, it is resolved by computing R e f values of relevant (i.e. active) concepts. Then, activeness is propagated from active concepts to perform the confirma tion process. The pronoun should be resolved before this process to provide an appropriate confirmation process. In other words, activeness propagation from a pronoun instance doesn’t mean anything. The activeness should be propagated from the referent concept. When a definite noun anaphora is found, a semantic constraint check and a recency check are processed. A recency value, however, is considered only if there are multiple candidates with the same scores. If the referent is not found, it means that the definite noun is not an anaphor. In other words, the definite noun is used for denoting a concept, which is in the mutual knowledge between the speaker/w riter and the listener/reader. [27] This is not considered in this thesis. Then, the current activeness of all concepts are re-evaluated. In this update, the activeness transition rules, in Figure 4.7, are applied. After the activeness update process, activeness is propagated to perform the expectation process. 68 proper name handling profession handling semantics, syntax, activeness checking find out active or expected concepts in attention unfilled case handling expectation confirmation pronoun resolution definite NP resolution update activeness sentence Figure 5.3: Flow chart of anaphora resolution model. P ro p er n ou n an d p ro fessio n h a n d lin g Proper nouns and professions are handled differently in two cases: human name and organization name. These two groups should be treated differently according to the convention of their use. In other words, in case of human: (1) full name, (2) first name, (3) last name, (4) profession, (5) profession and full name, or (6) profession and last name are used to denote a person. In case of organization: (1) full name, (2) abbreviation, or (3) their variations are used. In searching process, the algorithm finds out the most active or recent (if all names are inactive) one. (5.3) is a simple example of this category. 69 Salvadoran president-elect Alfiedo Cristiani condemned the terrorist killing of attorney general Roberto Garcia Alvarado and accused the FMLN of the crime. Legislative assembly president Ricardo Valdivieso and vice president-elect Francisco Merino also declared that the death o f the attorney general was caused by what Valdivieso termed the guerrillas’ “irrational violence.” Garcia Alvarado, 56, was killed when a bomb placed by urban guerrillas on his vehicle exploded as it came to a halt at an intersection in downtown San Salvador. “We have to condemn this incident, it is a guerrilla act,” Alfredo Cristiani, Nationalist Republican Alliance (Arena) president-elect, who will replace Christian Democrat Jose Napoleon Duarte on 1 June, stated. Cristiani said that “these are the risks faced by someone who enforces the law.” He noted that “the guerillas’ irrational attitude make it increasingly difficult to believe they want peace.” According to Cristiani, the attack took place because attorney general Garcia Alvarado warned that “he would take measures against urban terrorists.” Vice president-elect Francisco Merino said that when the attorney general’s car stopped at a light on a street in downtown San Salvador, an individual placed a bomb on the roof o f the armored vehicle. “The driver told the attorney general about the bomb. The vehicle swerved and the bomb exploded, causing the top of the vehicle to collapse on the attorney general’s head,” Merino stated. Guerrillas attacked Merino’ s home in San Salvador 5 days ago with explosives. There were seven children, including four o f the vice president’s children, in the home at the time. A 15-year-old niece of Merino’s was injured. Garcia Alvarado, father of six, was appointed attorney general on 23 December 1988. He was considered to be closely linked to arena. On several occasions, however, he said he did not represent any party and was carrying out his job “impartially and with the intention of enforcing the country’s laws.” Figure 5.4: MUC4 text. (5.3-1) Mayor Bradley lives in Los Angeles. (5.3-2) He has a big house. (5.3-3) It is very expensive. (5.3-4) Mayor will resign next month. Let’s see Figure 5.4. The task is to process the message and generate a tem p late^), as shown in Figure 5.5. W ithout resolving the definite reference to Merino, the information given b y vice president-elect Francisco Merino, M erino’ s 70 0. message: Id TST2-MUC4-0048 1. message: template 2 2. incident: date 14 APR 89 3. incident: location El Salvador: San Salvador (city) 4. incident: type bombing 5. incident: stage of execution accomplished 6. incident: instrument Id “explosives” 7. incident: instrument type explosive: “explosives” 8. peip: incident category terrorist act 9. perp: individual Id “guerrillas” 10. perp: organization Id “FMLN” 11. perp: organization confidence suspected or accused by authorities: “FMLN” “Merino’s home” 12. phys tgt: Id 13. phys tgt: type government office or residence: “Merino’ s home” 14. phys tgt: number 1: “Merino’ s home” 15. phys tgt: foreign nation - 16. phys tgt: effect of incident - 17. phys tgt: total number - 18. hum tgt: name - 19. hum tgt: description “children” “vice president’s children” “15-year-old niece of Merino’s” / “15-year-old niece” 20. hum tgt: type civilian: “children” civilian: “vice president’ s children” civilian: “15-year-old niece” 21. hum tgt: number 7: “children” 4: “vice president’ s children” 1: “15-year-old niece” 22. hum tgt: foreign nation - 23. hum tgt: effect of incident injury: “15-year-old niece” 24. hum tgt: total number 7 Figure 5.5: MUC4 tem plate. home, vice president’ s children, and 15-year-old niece o f M erino’ s cannot be inte grated to generated the tem plate shown. [86, 87] Another difficult problem is found frequently in the news articles. (5.4) In Columbus, Ohio ... police said a 14-year-old boy was beaten and stabbed to death ... because he refused to give up his Los Angeles Raiders’ Starter jacket to drive-by attackers. Dewayne Williams Junior was on his way to the school bus when he was killed. Four people were later charged in the murder. 71 This is one of many difficult problems in definite reference resolution. In the news article above, Dewayne Williams Junior refers to a 14-year-old boy who was killed. The reference resolver should identify th at 14-year-old-boy, he, his, and Dewayne Williams Junior are the same entity. This seems to be impossible to resolve in other computational approaches. However, the proposed model can resolve Dewayne as a 14-year-old boy using activeness, when assuming the sentence with Dewayne is in the same subdiscourse with the sentence with 14-year-old boy. Since the second sentence has no connection to the first sentence if Dewayne does not connect both sentences, this assumption is true, and the problem is solved. In some cases, hum an names are used with a person’s professions. This type of problem is discussed later. In organization name cases, a lot of abbreviations are used. Either having the company’s abbreviated name (e.g. GE for General Electric Company) in the knowledge base or a special inference routine can solve this abbreviation problem. [85] P ro n o u n referen ce r e so lu tio n W hen a pronoun anaphora is found from the parsing output, syntactic and semantic constraint check and linguistic parallelism check are carried out. To attain referability values of all active concepts, the scores for these checks are summed with the scores for the activeness. If two pronouns in the same syntactic category (e.g. she and her) are found and one is c-commanding the other, the referents are chosen so th at the sum of the referability values for two pronouns are maximum. (5.5) shows the example case. She and her in (5.5-4) can be either of Susan or Lynn, and she c-commands her. Thus, the algorithm compares two cases ({she = Susan, her = Lynn} and {she — Lynn, her = Susan}), and chooses the higher value case. Figure 5.6 shows how this is represented in the semantic network. (5.5-1) Susan drives an Alfa Romeo. (5.5-2) She drives fast. 72 drive-event drive-event [3-a] C-command her ) Female Singular fast Lynn ) Susan Alfa Active Subject Female Singular Active Female Singular Subject ipete-evenj [3-b] Active v Female C N Singular Street Active Alfa Susan fast Female Singular Active drive-event drive-event [4-a] fast Active Female Singular ( Lynn S u b j e c t - Active Subject Female V^he Singular Singular Female C-command Susan Alfa her Active [4-b] ipete-evenj Active i f f Female Singular JT Female Lynn ) Singular y Subject Active fast Alfa Figure 5.6: C-commanding case. (5.5-3) Lynn races her on weekends. (5.5-4) She often beats her. The resolution of pronoun consists of 4 phases. First, assign the activeness to the newly mentioned concepts. Second, check the c-commanding status. Third, compute the referability value for all concepts and find the concept with the high est referability value. Finally, by merging two nodes which represent the referent concept and the referring concept (pronoun), pronoun reference is resolved. Be cause the activeness value is a component of the referability value, it can achieve 73 Mayor Bradley lives in Los Angeles. C ^~~ mayor < Z Los Angeles ^ person#2 Bradley He has a big house. Il is very expensive. <Tstate-descrip lio n # 6 ^ it#7 Mayor will resign next month. <^~resign-event#8P^> < T mayor#9 ~^> Figure 5.7: Resolution by merging nodes. the same effect of searching active concepts first as well as checking other in formation. Various cases of the pronoun reference are discussed later in Section 5.5. B in d in g reso lv ed referen t The result of reference resolution is merging two nodes, each of which rep resents the referent concept and the referring expression respectively, into one. This is shown in Figure 5.7. For example, h e # 4 , i t # 7 and m ayor#9 nodes are merged into person#2, house#5 and person#2, respectively. More general con cept instance is merged into a more specific concept instance. For example, in (4.5) (also, in Table 4.2), the more general automobile is merged into the more specific car. Thus, more specific information (car) can be accessed later. 74 the + NP r e so lu tio n The resolution of the + N P also consists of 4 phases. First, all active con cepts (including expected concepts) which satisfy semantic constraint given by the subsumption hierarchy are found. Second, the modifier preference is checked. This process is to find out the concepts which satisfy the modifier of the referring expression the + NP. For example, in Figure 5.8, the proposed joint venture is more likely to refer to some joint venture concept instance which is proposed. The satisfied concept instances get the same value as the value for each syntactic or semantic constraint. Third, the referability value is computed for each of the active concepts. If there are ties, then recency is considered. Finally, the concept with the highest referability value is merged to the referring concept node. 5.5 Case study of important examples D e fin ite n ou n an ap h ora Since associative anaphora cases are already discussed with the activeness propagation, several other examples are discussed here. (5.6) Six California condors got their freedom today when th ey were released from a wildlife sanctuary today. The condors can have wingspans of up to 10 feet and are listed as endangered species. T he birds were hatched last spring at the Los Angeles Zoo. The zoo is one of a handful of captive breeding programs in this country. As shown in the above example, there are two kinds of definite nouns covered in this category. The condors and the zoo are the ones using same words, and the birds is the one using the subsumer concept. Since the subsumers are expected in the expectation process, the birds are resolved as condors. (5.7-1) Tom wants to attend the conference. 75 The Federal Trade Commission moved to block a proposed joint venture between General Electric Co.and Union Carbide Corp. in the more than $3 billion-a-year marker for silicones world-wide. The commission, which authorized its staff to seek a preliminary injunction blocking the proposal, said the joint venture could substantially lessen competition in the production and sale of silicone products. Silicones and silicone-based products are widely used in industrial and consumer products, ranging from deodorants and carpet fibers to certain types o f insulation and plastic materials. One agency official said the proposed joint venture, announced in May as a way of expanding research and development efforts, “amounts to an outright merger o f the silicones business” of the giant companies. The companies previously said the joint venture would have initial annual sales o f about $750 million, with GE slated to receive 70% o f the venture’s profits and Carbide 30%. The government estimates that in 1986, sales of silicone products in the U.S. totaled about $1.1 billion. In Danbury, Conn., Union Carbide maintained that the proposed venture “would be in the best interest of our customers world-wide.” The two companies “intend to work with the FTC in resolving the issue,” said H.W. Lichtenberger, president of Union Carbide’s chemical and plastic group. The commission said GE ranks No. 2 in sales and production of silicones, both in the U.S. and world-wide. Union Carbide ranks third among silicones producers in the U.S. and sixth world-wide, according to the commission. Dow Coming Corp., a joint venture of Dow Chemical Co., Detroit, and Corning Glass Works, Corning, N.Y., is the world’ s largest silicones producer. If a federal district judge grants a preliminary injunction, the commission would have 20 days to file administrative charges against the proposed venture. A spokesman for General Electric Silicones said: “General Electric disagrees with the initial decision o f the FTC regarding the proposed joint venture of Union Carbide Silicones and GE Silicones. We remain committed to working with the FTC to resolve any existing issues, since we feel the planned merger is good for both companies and for U.S. industry.” Figure 5.8: MUC5 text. (5.7-2) He sent a paper yesterday. (5.7-3) It was the due date, (5.7-4) but the conference will be held in July. As described in the algorithm, when a definite noun is encountered, a marker is propagated from the current utterance node through discourse and inner case links to find the conference concept. The concept with a closer discourse distance (more recent tim e stamp) is selected as a referent. The discourse link can be re-arranged to generate Figure 5.1 (b), though it is not implemented. As shown 76 Active evenF) Singular Neutral [1 ] [3-a] Active C^attend-eventji Singular Neutral Active Singular Male Subject Active Singular Neutral want-event Active Cattend-everjt) Singular Neutral Active Singular Male Subject conferen Active Singular Neutral agnt Active ^send-evenT^) Singular — Neutral time thesis Active Singular Neutral Active Singular Neutral want-event <^aftend-even£> Active Singular Male Subject _______ Active o '^ ^ s e n d ^ e v e n O Singular agnt — Neutral time thesis time Active Singular Neutral Active Singular Neutral Subsumed Cstate-descrigF) due-date Singular Neutral Singular Neutral discourse link Figure 5.9: Definite Noun Anaphora (a). in Figures 5.9 and 5.10, the conference in the (5.7-4) can be resolved with the presented algorithm. U n k n o w n g en d er (5.8) A Colombian judge is getting a first-hand look at the U.S. criminal ju s tice system following her arrest on charges of smuggling 2.2 pounds of heroin into M iami in her luggage. Esperanza Rodriguiz-Arevalo, whose papers identified her as a juvenile court judge in Colombia, was ar rested yesterday at Miami International Airport after a drug-sniffing dog singled out her luggage. 77 <^want-evenT^) want-event Cattend-event) conferenc attend-event Active Singular Male Active Active Singular Male Singular Neutral send-event^) ^-Oend-evenO agnt ^ — thesis thesis Active Singular Neutral Active Singular Neutral Subject Singular Neutral Active Active Singular Neutral Active Singular Neutral Cjio ld -event time event time Active Singular Neutral Subject <^due-date) Singular Neutral Active discourse link Figure 5.10: Definite Noun Anaphora (b). Though the gender information is not known for the first her, as described before, Colombian judge with high activeness value is found as the referent using the tentative neutral gender. C o m m o n sen se rea so n in g (5.9a) President Bush phoned Perez to express his support. (5.9b) President Bush phoned Perez to request his support. The resolution of his totally depends on the verb sense, and it’s not possible to resolve correctly with the current implementation without reasoning about verb sense. However, as discussed later, this model uses the heuristic that the agent case of the sentence is the referent of the possessive cases, if there is no pronoun reference found in the last sentence. W ith this heuristic, (5.9b) cannot be resolved correctly. 78 P ro n o u n an ap h ora (5.10) California, officials have suspended a psychologist for 60 days for re ferring one of her patients to an exorcist. The Board of Psychology ordered Doris Crane suspended for her decision to send the patient to a Hawaiian Huna priest. Crane said she referred the patient to the priest because he showed signs of being possessed by a demon. The first her is also solved by the tentative neutral gender approach. However, he has another problem to solve, because both of the patient and the priest have tentative neutral gender. (Crane is not tentative neutral any longer, since the gender is decided by the reference she.) But he is easily resolved as the patient, as the priest is a descriptor with less activeness, If she is used instead of he, the discourse becomes not clear enough. The reader thinks she as Crane until the reader reads the rest (showed signs of being possessed by a demon), and then a little bit of tim e is required to get out of the confusion — the patient should be used instead of she. Anyway, if she is used, the current model without reasoning on common sense cannot resolve she correctly. R efere n c e to a class (5.11-1) M y neighbor has a m onster Harley 1200. (5.11-2a) They are really huge, but gas efficient bikes. (5.11-2b) The Harley 1200 is a huge, but gas efficient bike. (5.11-2c) It is a huge, but gas efficient bike. They in (5.11-2a) definitely refers to the class of Harley 1200 motor cycle. Thus, a plural noun which co-specifies [111] a singular noun seems to be a reference to the class of the co-specified singular noun. The Harley 1200 also definitely refers to the class of Harley 1200. Thus, a definite noun which co-specifies a highly active noun seems to be a reference to the class of the co-specified noun. It in (5.11-2c) 7 9 [1} Active(2) Keutral(lO) ^Smgular(10) Active(2) Neutral(lO) §ingular(10) [2-a] Active(3) Active(3) Subject Male(lO) Singular(lO) Active(3) Active(3) Male(lO) Subject Male(lO) Singular(10) Singular(lO) Subject thuik-event study-event . Subject he J Male Singular , N Active(2) ^(^thmk-evem; Neutral(lO) Smgular(lO) l>b] (^^dy~event^ MaleflO) J Active(3) Activef3 — s mgma£(lQ)— Neutral(lO) Active(3.5) Singular(lO) Male(lO) . Singular (10) Acnve<3) Subject Subject E3-a] Active(2) Neutral(lO) S i n g u l a r ( l O ) t h i n k - e v e n t think-event go-event study-event Active(3) Neutral(10 Singular (10) Subject Male(lO) Active(3) Singular(lO) Subject Active(2) Neutral(lO) Singular(lO) think-event study-event S i n g u l a r Male Subject (3-b] Active(3, Male(l , Singular^lO) Subject Active(3) Neutral(lO) Singular(10) A c t i v e ( 2 ) Neutral(lO) Singular(lO) Figure 5.11: Syntactically Ambiguous Anaphora. can be considered to refer to the motor cycle which my neighbor has. Reference to a class is not considered in this thesis. S y n ta c tic a lly a m b ig u o u s an ap h ora (5.12-1) I haven’t seen Jeff for several days. (5.12-2) Carl thinks that h e’ s studying for his exams. (5.12-3) B ut I think he went to the Cape with Linda. 8 0 Figure 5.11 shows the snap shots of the resolution process of the example (5.12). For simplicity, let’s not care about syntactic and semantic constraints, because both Carl and Jeff satisfy the constraints. He in (5.12-2) is resolved by the heuristic that prefers continuing the current discourse over introducing a new discourse. In other words, selecting Carl for he in (5.12-2) cannot connect (5.12- 1) and (5.12-2), thus Jeff is selected. The same heuristic is used in Brennan’s centering approach. He in (5.12-3) is resolved as Jeff, since Jeff is previously referred and thus, has higher activeness. A n a p h o ra re so lu tio n u sin g re la ted e v e n ts an d c o n c e p ts (5.13-1) Tom wants to attend the conference. (5.13-2) He registered by phone yesterday. (5.13-3) It will be held in Boston. This case is explained with (5.13). W ith the high level knowledge about confer ence, it is obvious that the proposed model resolves it as conference. In (5.13-1), by the expectation rule, the conference concept expects register-event. In (5.13-2), the register-event confirms that conference concept should be still active, by the confirmation rule. Therefore, the conference concept doesn’t drop to 0 activeness state even after (5.13-2). In (5.13-3), it is resolved as conference, since the more active Tom cannot be the object case of the hold-event. A n a p h o ra w ith L in g u istic P a ra llelism (5.14-1) The green W hitierleaf is commonly found near the wild rose. | (5.14-2) The wild violet is found near it, too. 1 I ; i 1 This problem is already discussed before — though W hitierleaf is more active, the wild rose gets higher referability value due to the parallelism in the structure. This procedure is shown in Figure 5.13. Also, (5.5) shows the example on the parallelism on subject. 1This is taken from [54]. 81 object — -<^want-state^) Sttend-evCTT) s ' j o S . •egister-evenj agent Active agent expridncer Expected agent locadj resterda; tone, Active Active ^onferenSy Active Confirmed mferenci Tom Expected Active Active jtttend-evenT^ agent Active agent yday) hold-event~5^'cl* ve tone Tom object location Active Boston [ister-evenj .attend-eventT) agent >hone /dav hold-evcnt^) V Active location^ Tom object Active Active Figure 5.12: Related events and concepts. find-event find-event find-event#!, find-event#!. find-event# L location object location object location object object Whitii violet Whitiej violet rose rose Neutral Singular Active Neutral Singular Neutral Singular Active Neutral Singular Active Parallel Neutral Singular Active Neutral Neutral Singular Singular Figure 5.13: Linguistic Parallelism. 82 Chapter 6 Parallel Implementation 6.1 Parallel Implementation on SNAP The proposed model is implemented on the parallel marker-passing computer, SNAP (semantic network array processor). To utilize its parallelism, the values are used to represent activeness, and other constraints and preferences, instead of the proposed global order. By using values instead of the symbolic global order, SNAP can perform parallel evaluation of R e f value for all relevant concepts, by parallel marker adding and parallel searching of the highest value. In a very huge and complicated semantic network, the parallelism is very im portant, because activeness can be propagated to a very wide range. The values assigned are showed in Table 6.1. They are selected to satisfy the global order discussed in Chapter 4. The value range for the syntactic constraints is 0 < S x < 30, the value range for the semantic constraints is 0 < S m < 30, and the value range for the activeness is 0 < Act < 3. Thus, in our experiment, the range of R e f is: 0 < R e f < 63 + 0S(paraU elism ). I would like to note th at the different values can be used without any problem j if they are kept in the global order. One example of different values are shown in Table 6.2. As discussed in the previous chapter, one assumption is made in 8 3 Each constraint (number, gender, verb case, etc.) 10 directly realized non-descriptor 3 new non-descriptor 2.5 directly realized descriptor 2.0 new descriptor 1.5 indirectly realized concepts (previous strength of activeness is not considered) 1 expected (expectation level is not distinguished) 0.5 linguistic parallelism on structure 0.6 linguistic parallelism on subject 0.1 Table 6.1: Assigned Values. the global ordering between new non-descriptor concepts and the directly realized descriptor concepts. If the order between these two are changed, different results can be received. However, the current order gives the slightly better result in the experiment. All the procedures below the sentence block in Figure 5.3 are processed by parallel marker-passing on SNAP architecture. The parallelism does not exist between the procedures. They are processed in sequential flow. However, the parallelism lies on the computation, which is performed in each procedure. These computations are parallel propagation of the activeness through the huge semantic network and the parallel evaluation of the referability value. As discussed before, the proposed model can get superior resolution capability when the knowledgebase is very large and has complicated links among relevant concepts (including simplified script-like representation). However, the proposed model doesn’t sacrifice speed even for a very large and complicated knowledgebase, due to the parallelism provided by the parallel marker-passing model. 8 4 Each constraint (number, gender, verb case, etc.) 10 directly realized non-descriptor 9 new non-descriptor 8 directly realized descriptor 7 new descriptor 6 indirectly realized concepts (previous strength of activeness is not considered) 5 expected (expectation level is not distinguished) 4 linguistic parallelism on structure 4.8 linguistic parallelism on subject 1 Table 6.2: Different Values. 6.2 Performance analysis 6.2.1 In itia l E x p erim en t Table 6.3 shows the experiment results on one hundred news articles selected from the internet news group clari.news.cast. These news articles are shown in Appendix A. Because the current memory-based parser (or any parser) cannot produce the perfect output, the output is modified to a correct form manually, or the parsing output generator generates the parsing output from hand generated input in LISP form for parsing output, as shown in Figure 6.1. Also, required world knowledge (e.g. Yeltsin is a president of Russia.) and nodes for the script representation are added to the current knowledgebase, which has more than 50k semantic nodes. As shown in Table 6.3, the proposed approach achieves an average success of more than 88% overall definite reference and over 93% in pronoun resolution. 6 .2 .2 A n a ly sis o f resu lt an d h e u ristics S u ccess a n a ly sis 85 category total appearances correct failure h e/she/his/ h im /... 183 173 (94.5%) 10 (5.5%) it / its 53 47 (88.7%) 6 (11.3%) proper noun / occupation 129 113 (87.6%) 16 (12.4%) the+ N P reference 173 141 (81.5%) 32 (18.5%) all definite references 538 474 (88.1%) 64 (11.9%) Table 6.3: Result without heuristics. (6. 1) President-elect Bill Clinton will treat Saddam Hussein the same way President Bush does. He says he won’t resume normal relations with the Iraqi dictator. He can be resolved as either of Clinton, Hussein or Bush. But, Bush is in the descriptor clause, thus less active than the other two candidates. Also, by the linguistic parallelism, Clinton is selected as a referent. (6.2) The suspect... described in his 20s and armed with a ride... fled in a van or stationwagon. A witness says the gunman opened fire at auto mobiles parked at the C IA ’ s main gate. He says the man was QUOTE “ shooting at everything.” Another witness says he was wearing an A rm y field jacket. I W hile other discourse approaches fail in the resolution of two he’s, the proposed approach succeeds with the help of phrasal parallelism analysis. In other words, , there is a linguistic parallelism in the last 3 sentences, as agent 1 says agent2 ... j appear in all sentences. Similar situations are found in some other examples. 8 6 F ailu re a n a ly sis (6.3) Chief Justice William Rehnquist paid homage to his former colleague and friend, Justice Thurgood Marshall. Marshall was 84 when he died yesterday. In (6.3), all discourse approaches, including the proposed approach, fail in re solving he. Reading up to Marshall was 84 when he (i.e. before reading died), he tends to be interpreted to Marshall by a human, even though he in the first sentence may mislead the focus to Rehnquist. All discourse anaphora resolution approaches are weak in intrasentential anaphora resolution. The re-appearance of Marshall in the second sentence shows the change of discourse — from Rehnquist to Marshall. (6.4) Cuban track star Ana Fidelia Quirot who was badly burned in a fire at her home Friday gave birth to a baby girl. Cuba’ s official news agency says the bum s induced the birth and that the baby girl was born in perfect health. The child was taken to an intensive care unit. She won a bronze medal in the women’ s 800 m eter at the Olympics last sum mer in Barcelona. All discourse and sentential approaches fail in this example. In this example, she is m isinterpreted as the baby girl (the child) without the reasoning that a baby girl cannot win a medal in the Olympics. Actually, this pronominalization is improper and unclear. However, if a proper knowledgebase and a fairly good inference mechanism is provided, and integrated with the proposed model, most of inference related failures in he/she and other definite reference will be resolved successfully. (6.5) Gilbert Robinson, who witnessed the shooting told UPI he watched 8 7 the man firing at the cars. He says he didn’t think anyone realized what was happening. Robinson says “It was like in a dream.” In (6.5) all approaches cannot specify clearly what it is. In other words, it is re solved as something which was happening. It should be the man firing at the cars, but this is a very difficult problem to solve, and considered as correct resolution. (6.6) If the new first lady’ s role in the W hite House hasn’t been clear, i t ’ s been given some definition today. Hillary Clinton is shouldering a na tional health care plan. Mrs. Clinton attended a m eeting o f Cabinet- level health care advisers today. The president says h e ’ s grateful his wife agreed to take the chairmanship. (6.6) also requires a complicated reasoning. First, Hillary Clinton refers to the same person as the new first lady, but this may not require the knowledge that Hillary Clinton is the first lady. Even without that knowledge, human can recognize both are the same person. This is explained with (4.1) already. But Mrs. Clinton and his wife requires world knowledge and reasoning th at the president is Clinton and his wife has same last name. (6.7) California’ s state Senate Insurance Committee is considering legisla tion that would pay for auto insurance at the gas pum p. Proponents say the cost of providing total liability no-fault protection for Califor nia drivers would be roughly 40 cents a gallon. They say th a t’ s about half o f what Californians now pay for insurance. Under the plan ... This is another very difficult case which requires a sophisticated common-sense reasoning. It should be reasoned th at considering legislation can be a plan. 8 8 inter-sentential anaphora intra-sentential anaphora total pronoun anaphora total appearance 69 (29.2%) 167 (70.8%) 236 activeness approach 63 (91.3%) 157 (94.0%) 228 (96.6%) Brennan’s approach 60 (87.0%) 143 (85.6%) 203 (86.0%) Sidner’s approach 59 (85.5%) 143 (85.6%) 202 (85.6%) Hobbs’ approach 59 (85.5%) 164 (98.2%) 223 (94.5%) Table 6.4: Pronoun reference without heuristic H e u r istic s for p erfo rm a n ce im p ro v em en t Most of the failure in pronoun resolution occur in the intra-sentential anaphora, as shown in Table 6.4. Though the proposed model performs better than other discourse approaches, as shown in Table 6.4, the discourse approaches like this model are relatively poor in intra-sentential anaphora, compared to the sentential approaches like Hobb’s model. Because intra-sentential anaphora takes more than 70% portion of all pronoun anaphora in the experiment, performance in intra- sentential anaphora resolution plays an im portant role in the overall performance. To improve the performance in intra-sentential anaphora resolution, 100 more news articles are analyzed and the following heuristic is used in the new experi ment. Table 6.5 shows the improved result in the pronoun anaphora resolution. H eu r istic 1 : The referent of a possessive case in a sentence is the agent case concept of the sentence, unless there is a definite anaphora in the previous sentence. (If there is a definite anaphora in the previous sentence, it is selected as a referent due to higher activeness.) 8 9 category total appearances correct failure h e/she/his/ h im /... 183 180 (98.4%) 3 (1.6%) it / its 53 48 (90.6%) 5 (9.4%) all singular pronouns 236 228 (96.6%) 8 (3.4%) Table 6.5: Result with intra-sentential heuristics. Also, definite reference using proper noun can be improved with a heuristic. Though the references using proper names, as in (4.1), (i.e. Dewayne, which appears later, refers to the 14-year-old boy, which appears earlier) require inference, the discourse preference helps the resolution using activeness values. Many definite references (proper names, the + N P s, etc.), other than pro nouns, indicate the change of discourse, while some are used to refer clearly with out confusing the listener. The following heuristic can be found in many examples besides (4.1). W ith this heuristics, considerable improvement is shown in the re sult in Table 6.6. H e u ris tic 2 : In consecutive two sentences, • If (1) a non-pronominal reference in the second sentence is found at a given (theme) position (usually the subject), (2) the referent can be clearly found in the first sentence, and (3) no other pronominal reference to the concepts in the first sentence occurred, it’s considered th at the subdiscourse topic is changed. • If (1) a non-pronominal reference in the second sentence is found at a given position and (2) a referent of the reference in the second sentence cannot be clearly found in the first sentence, it’s considered that the referent is the discourse topic, or a concept related to the discourse topic at the first sentence. 9 0 category total appearances correct failure proper noun / occupation 129 127 (98.4%) 2 (1.6%) Table 6.6: Result with heuristics for proper name reference. category total appearances correct failure he/she/his/ h im /... 183 180 (98.4%) 3 (1.6%) it / its 53 48 (90.6%) 5 (9.4%) proper noun / occupation 129 127 (98.4%) 2 (1.6%) the+N P reference 173 141 (81.5%) A ) all definite references 538 496 (92.2%) 42 (7.8%) Table 6.7: Final result. Table 6.7 shows the final result of the proposed model with heuristics for intra-sentential anaphora and proper noun reference. It shows 3% improvement in pronoun anaphora resolution, 9% improvement in proper noun resolution and 4% improvement overall. Among the 7.4% of failure, a large portion due to the lack of the inference mechanism. Especially, references using the + NP, and it require inference to be resolved. 6.3 Comparison with other approaches As discussed before, the proposed approach deals with the pronoun, the+ N P and proper noun reference, which most of other approaches onlu deal with the pronoun anaphora. Therefore, comparison should be carried out only on pronoun anaphora 91 inter-sentential anaphora intra-sentential anaphora total pronoun anaphora total appearance 69 (29.2%) 167 (70.8%) 236 (100%) activeness approach 64 (92.8%) 164 (98.2%) 228 (96.6%) Brennan’s approach 60 (87.0%) 143 (85.6%) 203 (86.0%) Sidner’s approach 59 (85.5%) 143 (85.6%) 202 (85.6%) Hobbs’ approach 59 (85.5%) 164 (98.2%) 223 (94.5%) Table 6.8: Pronoun reference. in order to have meaning. In this section, a quantitative comparison is carried out for the 100 news articles (Appendix A), and a qualitative comparison is discussed. 6.3.1 Q u a n tita tiv e com p a riso n Hobbs’ approach is intrinsically strong for the intra-sentential anaphora, and Sid- ner’s and Brennan’s approach is strong for the inter-sentential anaphora. The same analysis was also reported by Walker [116]. When compared to the other three approaches, the proposed approach performs equivalent or better in both inter-sentential and intra-sentential anaphora as shown in Table 6.8. In inter- sentential anaphora, the proposed model performs better than other discourse approaches. In intra-sentential anaphora, the proposed model performs at the same success rate as the sentential approach. W ith the nature of algorithm, this model is poorer than the sentential approach in intra-sentential anaphora, but shows a better resolution capability than other discourse approaches. In the inter-sentential anaphora, the proposed model shows the best performance among the mentioned approaches. 9 2 6.3.2 Q u a lita tiv e com p a riso n S id n e r ’s approach Sidner’s approach falls into trouble with the sentence, which has more than one subject and/or object. The approach succeeds in the resolution of intersentential anaphora he in (6.8-2), since the approach (and other focusing approaches) prefers continuation of focus, over introduction of a new focus. However, it fails with another intersentential anaphora he in (6.8), because he in the agent position prefers the actor focus Carl. (6.8-1) I haven’ t seen Jeff for several days. (6.8-2) Carl thinks h e ’ s studying for his exams. (6.8-3) B ut I think he went to the Cape with Linda. 1 In (6.9), since her approach picks the freeway as a discourse focus, and doesn’t consider car or driving-event in (6.9-1) and (6.9-2), it cannot resolve the anaphoras the engine and the car. (6.9-1) I was driving along the freeway. (6.9-2) Suddenly, the engine made a noise. (6.9-3) The car was brand new. [112] picks the conference as a discourse focus of (6.10-1), and paper at (6.10- 2). It at (6.10-3) cannot be resolved, even though the conference should be in the attention throughout the discourse. This is because her approach considers only the concepts in the previous utterance. This case is not so clear as the case where the pronoun refers to the concept in the previous utterance, but this kind of discourse is frequently used by many people and understood without any problem. 1This is taken from [13]. 9 3 (6.10-1) Tom wants to attend the conference. (6.10-2) He sent a paper yesterday. (6.10-3) It will be held in Boston. Sidner’s approach also fails on (6.11) which was used by the centering approach [13]. Besides this approach cannot handle subdiscourse related problems nor the definite reference using proper nouns. (6.11-1) Susan drives an Alfa Romeo. (6.11-2) She drives fast. (6.11-3) Lynn races her on weekends. (6.11-4) She often beats her. B r e n n a n ’s ap p roach Though Brennan’s approach succeeds in (6.8), it fails in (6.10). The reason is that it considers only the concepts in the previous utterance, just Sidner’s approach does. The sentences in this example will be processed like below with centering approach, where Un is the n-th utterance, Cb is the backward-looking center and (7/ is the forward-looking center list. W hen the algorithm finds that paper is semantically wrong, it doesn’t have any candidates for it, because only Tom has been chosen as Cb and kept on the focus stack. Clearly, it should be conference, instead of paper, and conference has to be in the attention of the discourse perceiver throughout all of the three sentences in order to resolve it. The same problem will occur in (6.9), if the third sentence has a pronoun anaphora to the car which Tom was driving. (6.12-1) The green W hitierleaf is commonly found near the wild rose. (6.12-2) The wild violet is found near it, too. 9 4 This example is another case of failure of the centering approach and Sidner’s approach. It in (6.12-2) does not refer to the green Whitierleaf, which is a backward-looking center, but it refers to the wild rose. This is due to the lin guistic parallelism. Brennan’s approach is strictly for the pronoun anaphora, thus it cannot handle other definite reference cases using the + N P or subdiscourse related reference cases. H o b b s’ ap p roach His approach shows very good performance in intra-sentential anaphora, though it shows relatively poor performance in various kinds of inter-sentential anaphora. As in (6.8), if inter-sentential anaphora occurs in a complicated sentence, this approach can easily fail to resolve it. However, since intra-sentential anaphora is found more frequently than inter-sentential anaphora, the overall performance of this approach is very good. Another advantage of this approach is that it’s very simple. Any system with a parser, which can generate syntactic parse tree can use this approach without any difficulty. P ro p o sed m o d el As shown before, the proposed model successfully handles various reference cases. Main advantages over previous focusing approaches are: (1) no frame selection problem, which results in a wrong choice of focus, (2) no multi-frame focus problem, which results in confusion in the later discourse, (3) more capability in definite NP anaphora, (4) more capability in elaboration type discourse, and (5) more capability in the subdiscourse related reference problems. Moreover, the proposed model can handle all the problems which can be han dled by the previous focusing approaches. The summary of comparison is shown in Table 6.9 and Table 6.10. 95 problems activeness Sidner’s Brennan’s frame-selection problem no yes yes multi-frame focus problem no yes yes elaboration can handle sometimes cannot handle cannot handle subdiscourse insertion problem can handle sometimes cannot handle cannot handle complicated sentence can handle cannot handle can handle linguistic parallelism can handle sometimes cannot handle cannot handle other definite references can handle more can handle some types pronoun only Table 6.9: Comparison among discourse approaches. category activeness model discourse model Hobbs’ model intra-sentential good worse better inter-sentential better good worse Table 6.10: Comparison among approaches. 96 ( ; THE G U A TEM A LA A R M Y DENIED TO D A Y THAT GUERRILLAS ATTACKED THE "SANTO TOM AS" ; FARM, W H E R E PRESIDENT CEREZO H A S BEEN STAYING. C(deny-event#0 :cs_instance (deny-event) :subj (army#l) :agent ((army#l :nationality ((guatemala)) :instance (army) :number singular :article the)) :time ((today#2 :instance (today))) :obj (attack-event#3) :object ((attack-event#3 :cs_instance (attack-event) :subj (guerrilla#4) :agent ((guerrilla#4 :instance (guerrilla) :number plural)) :obj (faxm#5) :object ((farm#5 :instance (farm) :name ((santo_tomas)) :number singular :subclause ((stay-event#6 :cs_instance (stay-event) :subj (person#7) :agent ((p erson#7 :instance (human) :profession ((president)) :number singular :name ((cerezo)))) :location ((farm#5 :instance (farm location))))))))))) :instance food)))))))))) ) Figure 6.1: Input for parsing output generator. 97 Chapter 7 Summary and Conclusions 7.1 Result of dissertation A human makes consistent discourse through references, and the discourse per- ceiver not only finds out what the speaker/writer wants to say, but also integrates the information by the resolution of references. Therefore, reference resolution is a very im portant task in Natural Language Understanding. Text understanding and its applications, such as information extraction, cannot be achieved without the proper understanding of reference problems. There have been many approaches to solve these problems. Approaches which don’t consider the discourse phenomenon suffer from poor resolution of inter sentential references. Previous local focusing approaches, which are to solve dis course related reference problems, perform better than other non-focusing ap proaches in the resolution of intersentential anaphora. However, they suffer from several critical problems, due to the rigid focus selection scheme, and not consid ering more global discourse topic. These problems are the initial frame-selection problem, multi-frame focus problem and narrow coverage of reference resolution. 98 This thesis emphasizes the importance of the concepts’ activation in reference resolution, and proposes the computational model based on the activation the ory in psychology and other linguistic theories. The proposed model for better reference resolution with wider problem coverage shows promising results. By defining the activeness of all concepts and providing the integration mech anism — referability — of various information, the problems, which previous approaches suffer from, are overcome. The activeness and its propagation provide that the discourse topic is always in consideration, when finding out the refer ent. In other words, the expectation and confirmation processes keep track of the discourse topic and make it active always. In the news article type discourses, the proposed model achieved over 92% success in the resolution of various kinds of definite references. This model shows around 96% success in the pronoun anaphora resolution, and the result shows the best performance in intersentential anaphora resolution, when compared with other im portant approaches. Also, the proposed model shows better performance than other discourse approaches in intrasentential anaphora resolution. The over all success rate is the best among im portant approaches. The main contributions of this thesis are: • The notion of activeness and its definition for reference resolution which are based on psychological and linguistic phenomena. • The utilization of activeness in the reference resolution task, which avoids various problems that previous approaches suffer from. (Activeness and its propagation works in a way that they keep track of the context of the discourse so that the discourse topic is always one of the active concepts.) • A global order based on the activeness theory proposed in this thesis. (According to the global order, appropriate values are chosen to define and utilize referability in marker-passing computation.) 99 7.2 Future work First, further research will be carried out on the analysis of natural language discourse phenomenon, which is not yet well-defined. Natural language discourse requires more research in psychology, as well as linguistics, because the language is an expression of human consciousness. By doing this, the model will be established on a more concrete base, and will result in a much better capability to handle various difficult problems. As stated before, this thesis is focused on intersentential reference, and the proposed model is not well-defined for the intra-sentential references. The resolu tion of intra-sentential reference, which takes the major portion of references used in usual discourse, requires extensive analysis on syntactic problems, sentence style and also common sense reasoning. Common sense reasoning is required for many cases of failure in the experiment of the proposed model. An appropriate reasoning mechanism will be surveyed, and a much better result is expected with the integration of the proposed model and the reasoning mechanism. Finally, issues about knowledge representation are to be solved. Human knowl edge is gradually build up throughout a long period of tim e through lots of expe rience, and natural language is based on that knowledge. Therefore, knowledge representation is the most basic thing to attack natural language problems. The artificial knowledge which is stored in the knowledge base should have a very ef fective and efficient form, which is enough to represent complicated human knowl edge. Otherwise, the computational models which are based on a weak foundation cannot perform correctly to generate good results. 100 B ibliography [1] J. Allen. “ Natural Language Understanding,” chapter 11-15. The Ben jam in/Cum m ings Publishing Company Inc., 1987. [2] J. R. Anderson. “A theory of language acquisition based on general learning principles.” In Proceedings o f the Seventh International Joint Conference on Artificial Intelligence, IJCAI-81, 1981. [3] J. R. Anderson. “A Spreading-Activation Theory of Memory.” Journal of Verbal Learning and Verbal Behavior, 22:261-295, 1983. [4] J. R. Anderson. “The Architecture of Cognition.” Harvard Press, 1983. [5] D. Appelt and A. Kronfeld. “A Computational Model of Referring.” In Proceedings of the Tenth International Joint Conference on Artificial Intel ligence, IJCAI-87, pages 640-647, 1987. [6] B. W. Ballard. “The syntax and semantics of user-defined modifiers in a transportable natural language processor.” In Proceedings of International Conference on Computational Linguistics, COLING-84, 1984. [7] J. B arnett, K. Knight, I. Mani, and E. Rich. “Knowledge and natural language processing.” Communications of the ACM, 33(8), 1990. [8] J. D. Becker. “The phrasal lexicon, ” volume Report No. 3081. Bolt Beranek and Newman Inc., 1975. [9] T. J. M. Bench-Capon. “ Knowledge representation — An approach to arti- , ficial intelligence.” Academic Press Inc., 1990. [10] D. G. Bobrow and T. Winograd. “An overview of KRL: A knowledge rep resentation language.” Cognitive Science, 1, 1977. [11] R. J. Brachman and H. J. Levesque. “The tractability of subsumption in frame-based description languages.” In Proceedings of the National Confer ence on Artificial Intelligence, AAAI-84, 1984. 101 [12] R. J. Brachman and J. Schmolze. “An Overview of the KL-ONE Knowledge Representation System.” Cognitive Science, 9:171-216, 1985. [13] S. Brennan, M. Friedman, and C. Pollard. “A Centering Approach to Pro nouns.” In Proceedings of the 25th Annual Meeting of the A C L, pages 155- 162. Association for Computational Linguistics, 1987. [14] J. G. Carbonell. “Towards a self-extending parser.” In Proceedings of the 17th Annual Meeting o f the A C L, Association for Computational Linguistics, 1979. [15] S. Cha and D. Moldovan. “Parallel Anaphora Resolution on Marker-Passing Architecture.” Technical Report PKPL 91-6, University of Southern Cali fornia, 1991. [16] S. Cha and D. Moldovan. “A marker-passing algorithm for reference reso lution.” In TAI-93. IEEE International Conference on Tools with Artificial Intelligence, November 1993. [17] Seungho Cha and Dan Moldovan. “Reference Resolution in a Discourse Using Marker-Passing.” Technical Report PK P 93-2, University of Southern California, 1993. [18] W. L. Chafe. “Language and Consciousness.” Language, 50, 1974. [19] W. L. Chafe. “Givenness, Contrastiveness, Definitiveness, Subjects, Topics, and Point of View.” In C.N. Li, editor, Subject and Topic, pages 25-56. Academic Press, 1976. [20] E. Charniak. “Passing Markers: A Theory of Contextual Influence in Lan guage Comprehension.” Cognitive Science, 7, 1983. [21] E. Charniak. “A Neat Theory of Marker Passing.” In Proceedings of the National Conference on Artificial Intelligence, AAAI-86, 1986. [22] M. Chung, J.-T. Kim, D. Moldovan, and F. Parisi-Presicce. “Linguistic knowledge representation on SNAP.” Technical Report PKPL 91-10, Uni versity of Southern California, 1991. [23] M. Chung and D. Moldovan. “Memory-based parsing on SNAP: Integrated syntactic and semantic analysis.” Technical Report PKPL 91-4, University of Southern California, 1991. [24] M. Chung and D. Moldovan. “Parallel Natural Language Processing on Se mantic Network Array Processor.” Technical Report PK PL 93-3, University of Southern California, 1993. 102 [25] S. Chung and D. Moldovan. “Modeling Semantic Networks on the Connec tion Machine.” Journal of Parallel and Distributed Computing, 1993. [26] K. Church and P. Hanks. “Word association norms, m utual information and lexicography.” In Proceedings of the 28th Meeting of the A C L, Association for Computational Linguistics, 1990. [27] H. H. Clark and C. R. Marshall. “Definite Reference and M utual Knowl edge.” In A.K. Josh, B.L. Webber, and I.A. Sag, editors, Elements in Dis course Understanding, pages 10-63. Cambridge University Press, 1981. [28] H.H. Clark and C.J. Sengul. “In Search of Referents for Nouns and Pro nouns.” Memory and Cognition, 7:35-41, 1979. [29] Cohen and Perrault. “Elements of a Plan-based Theory of Speech Acts.” Cognitive Science, 3, 1979. [30] A. M. Collins and E. F. Loftus. “A Spreading-Activation Theory of Semantic Processing.” Psychological Review, 82:407-428, 1975. [31] W. A. Cook. “Case Grammar: Development of the Matrix Model.” George town University Press, Washington D.C., 1979. [32] R. DeMara. “ Parallel, Design, and Performance of a Marker-Propagation Reasoning System. ” PhD thesis, University of Southern California, 1992. [33] R. DeMara and D. Moldovan. “The SNAP-1 parallel AI prototype.” In Pro ceedings of the Annual International Symposium on Computer Architecture, 1991. [ 3 4 ] K. S. Donnellan. “Reference and Definite Description.” Philosophical Re- j view, 7 5 : 2 8 1 - 3 0 4 , 1 9 6 6 . j [35] R. C. Schank et al. “W hat’s the point.” Cognitive Science, 6, 1982. | [36] S. E. Fahlman. “ NETL: A system for representing and using real-world ! knowledge.” MIT Press, Cambridge, MA, 1979. j [37] G. Gazdar. “ Natural Language Processing in LISP. ” Addison-Wesley, 1900. [38] K. Godden. “Computing Pronoun Antecedents in an English Query Sys- I tern.” In Proceedings of the Eleventh International Joint Conference on J Artificial Intelligence, IJCAI-89, pages 1498-1503, 1989. [39] R. Granger. “FOUL-UP: A program that figures out meanings of words \ from context.” In Proceedings of the Fifth International Joint Conference 1 on Artificial Intelligence, IJCAI-77, 1977. 103 ! [40] B. J. Grosz. “The Representation and Use of Focus in a System for Under standing Dialogs.” In Proceedings of the Fifth International Joint Confer ence on Artificial Intelligence, IJCAI-77, pages 67-76, 1977. [41] B. J. Grosz. “Focusing and Description in Natural Language Dialogues.” In A.K. Joshi, B.L. Webber, and I. A. Sag, editors, Elements o f Discourse Understanding, pages 84-105. Cambridge University Press, 1981. [42] B. J. Grosz. “Providing a Unified Account of Definite Noun Phrase in Discourse.” In Proceedings of the 22nd Annual Meeting of ACL, pages 44- 50. Association for Computational Linguistics, 1983. [43] B. J. Grosz. “TEAM: A transportable natural language interface system.” In Proceedings of the Conference on Applied Natural Language Processing, 1983. [44] B. J. Grosz and C. L. Sidner. “Attention, Intention, and the Structure of Discourse.” Computational Linguistics, 1986. [45] R. Guindon. “Anaphora Resolution: Short-term Memory and Focusing.” In 24th Annual Meeting o f the AC L, pages 218-227. Association for Com putational Linguistics, 1985. [46] E. Hajicova. “Focusing — A Meeting Point of Linguistics and Artificial Intelligence.” In Artificial Intelligence II, pages 311-321. Elsevier Science Publishers B.V. (North-Holland), 1987. [47] E. Hajicova. “From Language Structure to Discourse Strategies.” In Lin guistics and Phonetics, 1990. [48] J. A. Hawkins. “ Definiteness and Indefiniteness: A Study in Reference and Grammaticality Prediction.” Croom Helm, London, 1978. [49] P. J. Hayes. “Anaphora for Limited Domain Systems.” In Proceedings of the Seventh International Joint Conference on Artificial Intelligence, IJCAI-81, pages 416-422, 1981. [50] J. Hendler. “Marker-passing and microfeatures.” In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, IJCAI-87, 1987. [51] J. Hendler. “ Integrating marker-passing and problem solving.” Lawrence Erlbaum Associates, Inc., 1988. [52] G. Hendrix and W. Lewis. “Transportable natural language interfaces to databases.” In Proceedings of the International Conference of Computa tional Linguistics, COLING-84, 1984. 104 [53] D. Hillis. “The connection machine.” The MIT Press, 1985. [54] G. Hirst. “ Anaphora in Natural Language Understanding: A Survey,” vol- I ume CSLI 119 of Lecture Notes in Computer Science. Springer-Verlag, 1981. [55] J. Hobbs. “Resolving Pronoun Reference.” Lingua, 44:311-338, 1979. [56] J. Hobbs. “Conversation as Planned Behavior.” Cognitive Science, 1980. [57] J. Hobbs, D. Appelt, M. Tyson, J Bear, and D. Israel. “Fastus: System summary.” In Proceedings of the fth Message Understanding Conference, 1992. | [58] R. P. Ingria and D. Stallard. “A Computational Mechanism for Pronominal Reference.” In Proceedings of the 28th Annual Meeting of the A C L, pages 262-271. Association for Computational Linguistics, 1989. [59] A. Joshi and S. Weinstein. “Control of Inference: Role of Some Aspects of Discourse Structure — Centering.” In Proceedings of the Seventh Interna tional Joint Conference on Artificial Intelligence, IJCAI-81, pages 385-387, 1981. [ 6 0 ] Joshua-Laird. “Mental Models in Cognitive Science.” Cognitive Science, 4, ^ 1 9 8 0 . j : [61] M. Kameyama. “A Property-sharing Constraint in Centering.” In Proceed ings of 25th Annual Meeting of the ACL, pages 200-206. Association for ! Computational Linguistics, 1986. 1 [62] R. N. Kantor. “The Management and Comprehension of Discourse Connec tion by Pronouns in English.” PhD thesis, Ohio State University, Columbus, Ohio, 1977. I [63] E. O. Keenan and B. B. Schieffelin. “Topic as a Discourse Notion: A Study \ of Topic in the Nouns in English.” In C. N. Li, editor, Subject and Topic, pages 335-384. Academic Press, 1976. [64] J.-T. Kim and D. Moldovan. “Parallel knowledge classification on snap.” In | Proceedings of the International Conference on Parallel Processing, ICPP- ; 90, 1990. ; [65] J.-T. Kim and D. Moldovan. “Acquisition of semantic patterns for infor mation extraction from corpora.” In Proceedings of 9th IEEE conference on A I applications, 1992. 105 [66] J.-T. Kim and D. Moldovan. “Classification and retrieval of knowledge on parallel marker-passing architecture.” IE E E Transactions on Knowledge and Data Engineering, 1992. [67] H. Kitano. “A Massively Parallel Model of Simultaneous Interpreta tion: The phiDmDIALOG System.” Technical Report CMU-CMT-89-116, Carnegie Mellon University, 1989. [68] H. Kitano. “$DM-Dialog: An experimental speech-to-speech dialog trans lation system.” IE EE Computer, 1991. [69] H. Kitano, D. Moldovan, and S. Cha. “High Performance Natural Language Processing on Semantic Network Array Processor.” In Proceedings of the 12th International Joint Conference on Artificial Intelligence, IJCAI-91, 1991. [70] H. Kitano, H. Tomabechi, and L. Levin. “Ambiguity Resolution in Dm- TRANS Plus.” In Proceedings of the fth Conference of the European Chap ter of the ACL. European Chapter of the Association of Computational Linguistics, 1989. [71] S. Kowalski. “SNAP Simulator and Development Library Version 6.” Tech nical Report PKPL 92-5, University of Southern California, 1992. [72] A. Kronfeld. “Donnellan’s Distinction and a Computational Model of Ref erence.” In Proceedings of the 2fth Annual Meeting of ACL. Association for Computational Linguistics, 1986. [73] M. Lebowitz. “Memory-based parsing.” Artificial Intelligence, 21, 1983. [74] W. Lee. “ A Parallel Marker-Passing Computer for Knowledge Processing.” PhD thesis, University of Southern California, 1992. [75] W. Lee and D. Moldovan. “The Design of a Marker Passing Architecture for Knowledge Processing.” In Proceedings of the National Conference on Artificial Intelligence, AAAI-90, 1990. [76] W. Lehnert. “Knowledge-based natural language understanding.” In H. Shrobe, editor, Exploring Artificial Intelligence. Morgan Kaufmann Pub lishers, Inc., 1986. [77] W. Lehnert, C. Cardie, D. Fisher, J. McCarthy, E. Riloff, and S. Soderland. “Description of the CIRCUS system used for MUC-4.” In Proceedings of the 4th Message Understanding Conference, 1992. 106 [78] C. Lin. “SNAP Simulator User M anual.” Technical Report PK P 90-5, University of Southern California, 1990. [79] C. Lin and D. Moldovan. “SNAP Simulator Results” . Technical Report CENG 89-11, University of Southern California, 1989. [80] J. Lyons. “Deixis as the Source of Reference.” In E.L. Keenan, editor, Formal Semantics of Natural Language. Cambridge University Press, 1975. [81] J. Lyons. “Semantics.” Cambridge University Press, 1977. [82] George Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller. “Five papers on W ordNet.” Technical Report CSL Report 43, Princeton University, 1990. [83] M. Minsky. “A framework for representing knowledge.” In P. W inston, editor, The Psychology of Computer Vision. McGraw-Hill Book Co., 1975. [84] D. Moldovan. “An Associative Array Architecture for Semantic Network Processing.” Technical report, University of Southern California, 1983. [85] D. Moldovan, S. Cha, M. Chung, T. Gallippi, K. Hendrickson, J. Kim, Changhwa Lin, and Chinyew Lin. “Description of the SNAP System Used for MUC-5.” In The Fifth DARPA Message Understanding Conference, McLean, Virginia, August 1993. DARPA. [86] D. Moldovan, S. Cha, M. Chung, K. Hendrickson, J. Kim, and S. Kowalski. “USC: Description of the SNAP System Used for MUC-4.” In The Fourth DARPA Message Understanding Conference, McLean, Virginia, June 1992. DARPA. [87] D. Moldovan, S. Cha, M. Chung, K. Hendrickson, J. Kim, and S. Kowalski. “USC SNAP: MUC-4 Test Results and Analysis.” In The Fourth DARPA Message Understanding Conference, McLean, Virginia, June 1992. DARPA. [88] D. Moldovan, S. Cha, I. Um, R. Demara, and J. Kim. “Direct Memory Access Translation on SNAP.” Technical Report PKPL 90-9, University of Southern California, 1990. [89] D. Moldovan and J. Hendler. “Why Marker-Passing Architectures are Good for Knowledge Processing.” Technical Report PKPL 90-12, University of Southern California, 1990. [90] D. Moldovan, W. Lee, and C. Lin. “SNAP: A marker propagation archi tecture for knowledge processing.” Technical Report 89-10, University of Southern California, 1989. 107 [91] D. Moldovan, W. Lee, C. Lin, and M. Chung. “SNAP: Parallel Process ing Applied to AI.” Technical Report PKPL 91-3, University of Southern California, 1991. [92] D. Moldovan, W. Lee, C. Lin, and S. Chung. “Parallel knowledge pro cessing on SNAP.” In Proceedings of International Conference on Parallel Processing, ICPP-90, 1990. [93] Dan I. Moldovan. “ Parallel Processing: From Applications to System s.” Morgan Kaufmann Publishers, 1993. [94] B. L. Nash-Webber. “Anaphora and Logical Form: On Formal Meaning Representations for Natural Language.” In Proceedings of the Fifth Interna tional Joint Conference on Artificial Intelligence, IJC AI-77, pages 121-131, 1977. [95] P. Norvig. “Inference in text understanding.” In Proceedings of the National Conference on Artificial Intelligence, AAAI-87, 1987. [96] P. Norvig. “Marker-passing as a weak method for text inferencing.” Cogni tive Science, 13, 1989. [97] G. D. Nunberg. “The Pragmatics o f Reference.” PhD thesis, City University of New York, 1977. Unpublished. [98] A. Ortony and R. C. Anderson. “Definite Descriptions and Semantic Mem ory.” Cognitive Science, 1:74— 83, 1977. [99] M. Pinkal. “Definite Noun Phrases and the Semantics of Discourse.” In Proceedings of International Conference on Computational Linguistics, COLING-86, pages 368-373, 1986. [100] Quillian. “Semantic Memory.” In M. Minsky, editor, Semantic Information Processing, pages 216-270. The MIT Press, 1968. [101] C. J. Rieger III. “Conceptual Memory and Inference.” In R. C. Schank, editor, Conceptual Information Processing, pages 157— 288. North-Holland, 1975. [102] C. Riesbeck and C. Martin. “Direct Memory Access Parsing.” Technical Report Yale University Report 354, Yale University, 1985. [103] C. Riesbeck and R. Schank. “ Inside case-based reasoning.” Lawrence Erl- baum Associates, 1989. 108 [104] C. K. Riesbeck. “From Conceptual Analyzer to Direct Memory Access Pars ing: An Overview.” In N. E. Sharkey, editor, Advances in Cognitive Science, chapter 8, pages 236-258. John Wiley and Sons, 1986. [105] P.S. Rosenbloom and A. Newell. “Learning by Chunking: A Production- System Model of Practice.” In R. Neches, D. Klahr, P. Langley, editor, Production System Models o f Learning and Development, pages 221-286. Bradford Books/M IT Press, Cambridge, MA, 1987. [106] R. C. Schank. “Language and Memory.” Cognitive Science, 4, 1980. [107] R. C. Schank and R. Abelson. “ Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures.” Erlbaum, Hilldale, NJ, 1977. [108] S. R. Schiffer. “ Meaning.” Oxford University Press, 1972. [109] J. Schmolze and T. Lipkis. “Classification in the KL-ONE knowledge rep resentation system.” In Proceedings of the Eighth International Joint Con ference on Artificial Intelligence, IJCAI-83, 1983. [110] Searle. “Intentionality of Intention and Action.” Cognitive Science, 4,1980. [111] C. L. Sidner. “Focusing for Interpretation of Pronouns.” American Journal of Computational Linguistics, 7:217-231, 1981. [112] C. L. Sidner. “Focusing in the Comprehension of Definite Anaphora.” In M. Brady and R.C. Berwick, editors, Computational Models of Discourse”, chapter 5, pages 267-330. The MIT Press, 1981. [113] C. Stanfill and D. Waltz. “Toward Memory-based Reasoning.” Communi cation of ACM, pages 29-42, 1986. [114] B. H. Taylor and R. S. Rosenberg. “A Case-driven Parser for Natural Lan guage.” American Journal of Computational Linguistics, 1975. [115] H. Tomabechi. “Direct Memory Access Translation.” In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, IJCAI-87, pages 722-725, 1987. [116] M.A. Walker. “Evaluating Discourse Processing Algorithms.” In Proceed ings o f the 27th Annual Meeting of the ACL, pages 251-261. Association for Com putational Linguistics, 1989. [117] D. Waltz. “Massively parallel AI.” In Proceedings o f the National Confer ence on Artificial Intelligence, AAAI-90, 1990. 109 [118] D. Waltz and J. Pollack. “Massively parallel parsing.” Cognitive Science, 9, 1985. [119] D. L. Waltz. “TINLAP-2: Theoretical Issues in Natural Language Processing-2.” 1978. [120] Wilensky. “Why John married Mary.” Cognitive Science, 2, 1978. [121] Wilensky. “Planning and Natural Language”. Cognitive Science, 5, 1981. [122] Y. A. Wilks. “Preference Semantics.” In E.L. Keenan, editor, Formal Se mantics o f Natural Language, pages 329-348. Cambridge University Press, 1973. [123] Y. A. Wilks. “A Preferential Pattern-seeking Semantics for Natural Lan guage Inference.” Artificial Intelligence, 6:53-74, 1975. [124] T. Winograd. “Understanding Natural Language.” Academic Press, 1972. [125] T. Winograd. “W hat does it mean to understand.” Cognitive Science, 1980. [126] T. Winograd. “ Language as a cognitive process - Vol. 1, Syntax.” Addison- Wesley Publishing Co., 1983. [127] W. A. Woods. “Transition Network Grammars for Natural Language Anal ysis.” Communications of the ACM, 1970. [128] W. A. Woods. “Lunar Rocks in Natural English: Explorations in Natural Language Question Answering.” In A. Zampolli, editor, Linguistic Structure Processing, pages 521-569. North-Holland, 1977. [129] D. Wu. “ A probabilistic approach to marker propagation.” In Proceedings of the 11th International Joint Conference on Artificial Intelligence, IJCAI- 89 1989. 110 Appendix A Test examples from internet news group 1. The ex-Soviet republic of Armenia is paralyzed today after an explosion in neigh boring Georgia damaged the pipeline that supplies it with natural gas. A Russian state news agency says the metro was shut down, most telephone lines are dead, newspapers are NOT publishing and radio stations stopped broadcasting. Resi dents are also withOUT heat in 14-degree temperatures. 2. The 1993 entertainment awards season kicks off tonight with the 50th annual Golden Globe Awards. The military drama “A Few Good Men” was expected to win best picture, best actor for Tom Cruise and best supporting actor for Jack Nicholson. Other contenders for best actor in a drama are Robert Downey Jr. for “Chaplin,” Nicholson for “Hoffa,” AI Pacino for “Scent of a Woman” and Denzel W ashington for “Malcolm X .” 3. President Clinton’s only been in the W hite House a few d a y s .... and already some Republicans are lining up to challenge him in 1996. Among th em .. . . Senate leader Bob Dole. Dole says (in a C-Span interview) lots of people are talking to him about running. Dole, who’ ll be 72 in four years, says he’s keeping his options open and if he does run he’d only seek one term. Jack Kemp is another prominent Republican considered a possible presidential candidate in ’96. 4. Illinois Congressman Dan Rostenkowski could face more trouble for charging tax payers to lease three vehicles he now owns. A Chicago newspaper (The Sun- Tim es) says the deals apparently violate House guidelines on how members use their official expense accounts.. . . and raise questions about possible federal vio lations. Rostenkowski refuses to answer questions on the charges. Records show it ’s cost taxpayers over one thousand dollars every m on th .. . . for almost four years.. . . to lease Rostenkowski’s “mobile offices.” 5. A national auto safety group plans to look at thousands of air bag deployments for clues on how to improve the life-saving devices. The move by the Insurance Institute for Highway Safety is prompted by concerns stemming from reports of occasional injuries from inflating air bags. I l l 6. The first Hollywood awards event of the new year was held last night. The Golden Globes, awarded by the foreign press corps, featured lots of surprises. The pre-event favorite for dramatic picture “A Few Good Men” starring Tom Cruise and Jack Nicholson did not receive a single award. AI Pacino and his movie, “Scent of a Woman,” were surprise winners. Clint Eastwood was best director for his action-western “Unforgiven.” Gene Hackman won a Golden Globe for his role in the film. Emma Thompson won as best actress in a drama for her role as a warm and loving gentlewoman in turn of the century England in “Howards End.” “The Player” was named best comedy or musical film for its cynical and satirical look at Hollywood’s motion picture jungle as viewed from the tow n’s executive suites. Walt D isney’s animated film “Aladdin” ran away with both Golden Globe musical aw ard s.... for best score and best song: “A W hole New World.” In television “Roseanne” swept the three top awards for a comedy series, posting victories for best series, best actor and best actress. “Northern Exposure,” essentially a comedy, won the Golden Globe for best dramatic series. Best actor in a dramatic series was Sam W aterston for “I’ll Fly Away.” His co-star, Regina Taylor, won best actress honors. The show has been cancelled by NBC. “Sinatra” was named best mini series or m otion picture made for television. 7. Experts say the latest attack on Virginia forests by the southern pine beetle is the worst in recent memory. State entomologist Tim Tigner says this infestation may be one of the worst. Beetles killed trees valued at more than nine-Million- dollars in 1982. Tigner says this outbreak is much more severe and may be the biggest y e t.. . . in terms of the value of lost timber. 8. A Georgia man has been charged with stealing more than 50-thousand limited edition sports cards worth more than 350-thousand dollars. Gary Moran allegedly stole the cards last year. 9. W ill Madonna show up in Atlantic City today or won’t she. T h at’s one of the big questions as organizers hurry to finish last-minute preparations for a massive treasure auction. The rock star is said to be interested in several golden baubles being auctioned off at the Trump Regency hotel and casino. Bidders from as far away as Japan are flying in for the ev en t.. . . when more than 50-Million dollars worth of treasure recovered from shipwrecks is sold. Also for sa le .. . . a 29-Million- dollar set of gold silverware a gift the King of Siam intended for Abraham Lincoln that got waylaid during the Civil War. 10. President Clinton’s first summit may be coming soon. The W hite House says Clinton telephoned Russian President Boris Yeltsin today. Both agreed they want to meet as soon as p ossible.. .. and ordered their top officials to work out the place and time. Clinton spent 30 minutes talking with Y eltsin .. .. and reaffirmed his support of the Russian leader’s economic and political reforms. 11. The Vatican City newspaper L’Osservatore Romano (Loh-ser-vah-TOHRR-ray Roh-MAH-noh) today strongly criticized President Clinton’s action in easing 112 abortion restrictions. The paper says the lifting of the restrictions mean Clinton’s campaign for renewal “is taking place on the pathways of death.” The newspaper’s brief comment was the first by Vatican sources since Clinton signed five pro-choice memoranda yesterday. M eanwhile.. . . smaller ANTI-abortion demonstrations continued in Washington D-C today. Tens of thousands demonstrated yesterday as President Clinton eased Reagan- and Bush-era restrictions on abortions.... including the gag rule on abortion counseling at federally funded clinics.. . . and the use of fetal tissue for research to cure diseases. 12. 13 Palestinian deportees Israel admits it expelled by m istake.. .. and four others who are sick .. . . were plucked by British helicopters from their frigid tent camp in south Lebanon today. The 13 are headed back to Israel.. . . but most face trial on other activities. Nearly 400 other deportees remain in the NO-man’s land. Israel deported the men last m on th.. . . saying they were connected to fundamentalist groups that killed several soldiers and police. 13. Jesse Jackson is in Haiti to push for the return of deposed President Jean-Bertrand Aristide (Zhawn-bayr-TRAWN’ ah-REEST’-eed ).. . . now in exile in the United States. Jackson met with Aristide supporters to d a y .... and said the United States should work for democracy in Haiti. Aristide is a Roman Catholic priest who was overthrown as president in a bloody military coup in September 1991. 14. Texas Senator Phil G ram m .. . . of Gramm-Rudman law fame. . .. and former presidential candidate Ross Perot are joining forces to form a grassroots campaign to fight the deficit. Gramm has met with Perot to discuss proposing a balanced budget amendment to the Constitution. He says the Dallas BILLIONaire’s group “is going to support and mount the grassroots effort” aimed at passage of the amendment. The Unlikely alliance apparently was initiated by G ram m .. . . who has never received Perot’s support. 15. Cuba seems to have sprung a serious lead. A record 13 Cubans have asked for political asylum during interviews on a Miami radio station in the last week. Those announcing their defection at station W -Q-B-A include a Cuban dancer, a shipping official, two journalists, and at least four government officials. Legally all they have to do is file an asylum request with immigration officials. But the Miami stations can be heard in C uba.. . . so announcing defections over the radio gives them a chance to speak out against the Castro government and let relatives at home know they made the journey safely. 16. A standoff is brewing between the W hite House and the Pentagon over President Clinton’s promise to lift the ban on gays. In the wake of military opposition, Defense Secretary Les Aspin today said he wants a six-month review of the con troversial question. On CBS’s “Face the Nation” p rogram .... Aspin said he’s 1 sent Clinton a letter asking for six months to -quote- “see whether we can do it.” But, Aspin said he would work with Clinton to lift the ban. M eanw hile.... reports are circulating that General Colin Powell chairman of the Joint Chiefs 113 of Staff may resign if Clinton allows gays in the military. But, a spokesman for Powell denies the report. And, Clinton spokesman George Stephanopoulos says Clinton will stick to his promise despite opposition. A new poll out today in Newsweek Magazine also shows that most people 53 percent do NOT think Clinton should change the military policy. But, an overwhelming majority think gays can effectively serve in the military if they keep their sexual orientation private. 17. A published report today says there are as many as 70-thousand prisoners being held in war-torn Bosnia. The New York Times quotes U-S intelligence officials as saying there are about 135 detention camps in and around B osn ia.. .. with most of them but NOT all controlled by Serbian forces. Such detention camps reportedly exist, despite a Serbian agreement to shut down the facilities. Bosnian Serbs backed by the republic of Serbia have siezed most of Bosnia in their campaign to dominate the Balkans. 18. Congressman Dan Rostenkowski of Illinois denies he used taxpayers’ money for his personal use. The Chicago Sun-Times today reports Rostenkowski charged taxpayers more than 68-thousand dollars to lease three cars that later became his personal property. The vehicle transactions reportedly may violate House rules. This report comes while Rostenkowski chairman of the powerful House Ways and Means C om m ittee.... is under investigation for possible fraud, election and income-tax violations in regard to the House Post Office scandal. 19. Irene Seale will be sentenced tomorrow in a New Jersey district court for her part in the brutal kidnapping of Exxon executive Sidney Reso. The sentencing focuses on one question: was she her husband’s accomplice. . .. or was she, like Reso, a victim? Arthur Seale, a former Exxon security officer, is serving life in prison for Reso’s kidnapping and death in captivity. His wife is likely to get a much shorter term, because she cooperated with authorities and pleaded guilty after her arrest in June. Her sentencing is set for tomorrow morning. 20. Biologists in Florida are discussing several possible solutions to the problem of whales being accidentally killed by ships. The most recent incident occurred Jan uary 5th off the coast of St. Augustine when the captain of a Coast Guard cutter reported hitting and killing a northern right whale calf, which is an endangered species. National Marine Fisheries Service biologist Terry Henwood of St. Pe tersburg said it was the first tim e a ship had reported such an accident but many others go unreported. 21. The search is on for a man with a rifle who opened fire outside the main gate of the C-I-A in Virginia. Two people were killed and three others were wounded. Police say the incident took place at 8 a-m. The su sp e c t.... described in his 20s and armed with a rifle fled in a van or stationwagon. A witness says the gunman opened fire at automobiles parked at the CIA’s main gate. He says the man was QUOTE “shooting at everything.” Another witness says he was wearing an Army field jacket. 114 22. The circumstances surrounding former attorney General-designate Zoe Baird’s withdrawal areN’T going away. This week Peruvian couple who worked in Baird’s home have to report to Immigration in Hartford this week for questioning. An I-N-S spokesman says Lillian and Victor Cordera are both Illegal aliens. Depend ing on their sta tu s.. . . they’ll either have to leave the country or stay and face proceedings. Baird withdrew her nomination when opposition to her hiring of the couple mounted. 23. F-B-I Director W illiam Sessions says he’s a target of political opponents who want him out. Sessions is disputing a Justice Department report accusing him of IMproperly billing the government for personal expenses and other abuses. In an interview with the New York T im es.. . . Sessions pointed to Attorney General W illiam B a r .. .. his boss in the Bush Administration. He says his conduct was NOT improper. Sessions can only be removed by the president. President Clinton says he wants to hear Sessions out before taking any action. 24. The Supreme Court Monday said today Texas can execute a man convicted in the 1981 murder of two police officers withOUT holding a hearing on newly discovered evidence that attorneys claim would prove he’s innocent. By a 6-3 vote, the court cleared the way for the state to execute Leonel Herrera. Herrera’s scheduled execution had been stayed by a Texas state court last year after the Supreme Court agreed to hear his case. In an opinion by Chief Justice W illiam R ehnquist.. . . the court says executing an innocent person would be U Nconstitutional but Herrera apparently isN ’T innocent. 25. More problems with Iraq. Today Defense Secretary Les Aspin says there’s some indication Iraq is moving missiles back into a “no-fly” zone in the southern part of the country. He says U-S officials are monitoring the zone for placement of surface-to-air missiles. Missiles moved in the disputed zone is what sparked the latest military confrontations with Iraq. Aspen says the administration is waiting “a couple of days” to make sure Saddam is indeed breaking the cease-fire they declared. 26. Trouble on peace talks in other parts of the w orld.. . . former Yugoslavia. Today Bosnian president Alija Izetbegovic (ah-li-jah eye-zet-BEG-o-vich) flatly rejected boundaries proposed by peace mediators for 10 semi-autonomous provinces in a future decentralized state of Bosnia-Herzegovina. Izetbegovic.. . . who is a M uslim .. . . says a draft map presented by mediators indicates areas seized by Serbs as part of “ethnic cleansing” would be legal. 27. It could be a heated session today when President Clinton meets with the Joint Chiefs of Staff. T hey’ll be discussing a plan to lift a ban preventing gays from serving in the military. Clinton vowed during the campaign that gays should NOT be discriminated against in the armed forces. But Pentagon spokesman Colonel Bill Smullen says General Colin Powell is against it and intends to tell the president so. 115 28. It’s the first day of school in Washington for Chelsea Clinton. Today Hillary Clinton escorted her 12-year-old daughter to the Sidwell Friends School on her first day of classes. The Clintons arrived with Secret Service protection at 7:30 a-m. Chelsea attended a public school in Little Rock, Arkansas.. . . where she was in the eighth grade. But President and Mrs. Clinton decided to send Chelsea to a private school in the nation’s capital. 29. State and local police are on alert today after a gunman killed two people and wounded three others at the C-I-A headquarters outside Washington. Police and eyewitnesses say the man was firing randomly at cars as they entered the main entrance of the building at 8 o ’clock this morning. Police are searching for the suspect. He’s thought to be in his 20s and wearing an Army jacket. He fled in a van or stationwagon. Gilbert Robinson, who witnessed the sh ootin g.. . . told UPI he watched the man firing at the cars. He says he didN’T think anyone realized what was happening. Robinson says QUOTE “It was like in a dream.” 30. The Supreme Court says trying multiple defendants IS legal. Today the high court ruled that co-defendants are NOT entitled to separate trials even if one suspect blames the other for being the guilty party. The justices upheld a lower court ruling.. .. which refused to let four drug suspects have individual tria ls.. .. even though they pointed a finger at one another. 31. President Clinton says he will NOT budge from his campaign promise to allow gay men and women to serve in the military. Clinton is meeting this afternoon (at 4pm est) with the Joint Chiefs of S taff.. .. and he says he’ll listen to their objections. Clinton says he wants their input on how the government should proceed with lifting the military ban. But the president says make NO mistake about i t . . . . he plans to keep his commitment to allow gays to serve. 32. Chief Justice W illiam Rehnquist paid homage to his former colleague and friend whom he often disagreed with philosophically Justice Thurgood Marshall. Marshall was 84 when he died yesterday. Rehnquist said the court will miss Mar shall’s “wit, warmth, and charm.” He discussed Marshall’s lifelong fight for racial eq u ality.... and called him the “most influential civil rights lawyer of our cen tury.” Meanwhile President Clinton all American flags be flown at half-staff until the former Supreme Court justice is buried. Clinton called Marshall “a fundamen tal force of change in this nation.” Funeral arrangements for Marshall are still pending. 33. Irene Seale has been sentenced to 20 years in federal prison for her part in the kidnapping and death of Exxon International President Sidney Reso. She has another hearing this afternoon in M orristown.... where she is to plead guilty to a New Jersey kidnapping charge and to receive another prison term. Seale’s 116 husband, Arthur, is already serving consecutive state and federal sentences that give him NO hope of release. The two kidnapped Reso for ransom and kept him in a box, where he died just days later. 34. One of the nation’s oldest department stores is in financial trouble. Sears, Roeback and Company announced plans to lay about 50-thousand full-time and part-time employees, close more than a hundred of UNprofitable retail and specialty stores and discontinue its catalog operations.... a marketing tool that helped it become the nation’s top retailer. Sears officials say store closure dates have NOT been set. At least 34-hundred full tim e and 16-thousand-five-hundred part-time jobs will be eliminated this year because of the catalog shut-down. The Spring 1993 catalog will be its last. 35. President Clinton is playing down a signal sent by Treasury Secretary Lloyd Bentsen that a tax hike was in the offing. Clinton reacted to Bentsen’s favor able comments about a broad-based energy consumption tax today saying NO decision has been made. Bentsen had singled out an energy tax on gasoline and oil as a possiblity. He also praised the benefits of higher taxes on alcohol and tobacco. Clinton’s communications director George Stephanopoulos said the en ergy tax was being considered. He pledged the economic package would promote investment and bring the deficit down. Meanwhile. . .. Clinton’s staff is busily trying to narrow the selection of an attor ney general. Last week Clinton’s nominee, Zoe Baird, withdrew her nomination after wide public condemnation for her hiring of illegal workers at her home and her acknowledged failure to pay Social Security taxes. Stephanopoulos said Clin ton planned to meet with prospective candidates but had NOT done so yet. He said all candidates must endure an FBI background check. 36. President Clinton’s staff is growing. Today he appointed three more deputies to assist in inter-agency affairs and press relations today. He named Washington lawyer John D. H art.. . . a senior transition officer.. . . to be deputy director of the Office of Intergovernmental Affairs. Clinton also appointed Lorraine Voles and Arthur Jones as deputy press secretaries. Voles served as press secretary to Iowa Senatore Tom Harkin. Jon es.. . . a former reporter for the Boston G lobe.... was press secretary to Boston Mayor Ray Flynn. 37. F-B-I director W illiam Sessions continued his public campaign against a scathing Justice Department report today. Appearing on the N-B-C Today show this I m orning.. . . he said he hopes to make his case personally to President Clinton. The report charges Sessions with IMproperly billing the government for personal expenses and alleged he consistently took improper advantage of the government through personal trips. Sessions says it’s politically motivated and he’s sure he’ll be vindicated. 38. A status conference is being held today in the federal civil rights trial of the Los Angeles police officers accused of beating Rodney King. A judge is expected 117 to rule on whether testim ony delivered during the state trial can be admitted. Attorneys will also address the issue of issuing questionnaires to potential jurors. 39. California’s state Senate Insurance Committee is considering legislation that would pay for auto insurance at the gas pump. Proponents say the cost of providing total liability NO-fault protection for California drivers would be roughly 40 cents a gallon. They say th at’s about half of what Californians now pay for insurance. Under the plan someone who gets 20 miles a gallon and drives about 15 thou sand miles a year would pay about 300 dollars a year for insurance. The Senate Insurance Committee will hear the proposal February 3rd. 40. Many key allies aligned with the U-S against Iraq appear to be distancing them selves from the current round of action aimed at forcing Saddam Hussein’s com pliance to U-N resolutions. Great Britain, France and R ussia.. . . all permanent U-N Security C ouncil.. . . feel Bush personalized the conflict with Saddam and would do anything to triumph during the waning days of his presidency. These countries have the right to veto any action action proposed in the U-N. A veto could make it impossible for the United States to finesse its global policies through the international body. 41. The W hite House finds an economic proposal to raise energy consumption taxes appealing.. . . but is soft peddling the idea because it could become contro versial. President Clinton said he’s considering options but NO decision has been made on the notion discussed yesterday by Treasury Secretary Lloyd B entsen.. . . that an energy tax hike was in the offing. Clinton’s spokesman----- George Stephanoupolas was probed on Bentsen’s assertion a consumption tax is likely. He confirmed such taxes were under consideration. 42. If the new first lady’s role in the W hite House hasN ’T been clear it’s been given some definition today. Hillary Clinton is shouldering a national health care plan. Today Clinton said he’s putting his Hillary in charge of a health care task force whose job it is to produce a universal health coverage plan within 100 days. Mrs. Clinton attended a meeting of Cabinet-level health care advisers today. The president says he’s grateful his wife agreed to take the chairmanship. 43. The public will have a chance to pay their respect to former civil rights leader and Supreme Court Justice Thurgood Marshall. The court announced he will lie in repose for a public viewing Wednesday in the court’s ornate Great Hall. It’s is tentatively scheduled to last from 10-30 a-m until 9 p-m (E S T ).. . . with the public adm itted through the court’s front doors. A funeral service is scheduled for Thursday at the Washington Cathedral with a private burial Friday at Arlington National Cemetery. The 84-year-old Marshall died Sunday of heart failure. 44. Two men fatally shot and three men injured outside C-I-A headquarters have been identified as employees of the agency. The two men who died are 28-year-old Frank Darling and 66-year-old Lansing Bennett, both of Reston, Virginia. Two other 118 victims are in critical and serions condition at Fairfax Hospital. Authorities are looking for a man believed to be in his 20’s. W itnesses say the gunman slowly and methodically shot the victims while in their cars. 45. A British nuclear weapons plant may be a safety hazard. The environmental group Greenpeace said today more than 100 people have been either contaminated, injured or killed at the nuclear weapons plant where they worked in Southern England. The report says the Aldermaston plant the hubb of Britain’s nuclear arms industry has an “appalling record” of UNreported deaths and fires, radioactive leaks and worker contamination. 46. In San Francisco today Miss America Leanza Cornett presented a panel to the Names P roject.... the group responsible for the national AIDS quilt. The p an el.. . . which contains a quote from Mother T heresa.. . . honors Florida women who died of the disease. Among the victims whose name appears on the panel is Kimberly B ergalis.. .. who delivered an emotional plea to Congress for more AIDS funding before dying in December 1991. Cornett, who’s from Jacksonville, has been working as a voluntter for AIDS organizations in Central Florida. 47. In F lorida.. .. a 17-year-old boy accused in a brutal carjacking agreed to plead guilty and testify against three other suspects. They are charged with armed carjacking and shooting two victims to death after forcing them to strip and lie on the ground. In exchange.. . . state prosecutors will NOT seek the death penalty against Leondra Lamont Henderson.. . . who was to have been tried as an adult. He faces a maximum sentence of life in prison withOUT parole. The suspects are the first to be indicted in a deadly carjacking under the Anti-Car Theft Act of 1 9 9 2 .... which took effect in late October. 48. Police in Tustin, California, are searching for the man who sold a baby for 10- dollars in a market parking lot. Authorities are also looking for the baby’s mother. The newborn baby boy was only hours old when a teenager turned him over to police on Saturday night. 18-year-old Robert Garcia says a man approached him in a shopping center parking lot and offered to trade the baby for drugs or cash. Garcia says he paid the man 10-dollars, took the baby and called authorities. Police say the infant is in good condition. 49. Even if California didN’T send a team to the Super Bowl this y ear.. . . southern California will reap the benefits of Sunday’s showdown between the Dallas Cow boys and the Buffalo Bills. This year’s game, in Pasadena, will add 180-Million dollars to the battered Southern California economy. Standard and Poor’s says the Super Bowl could NOT come at a more opportune tim e for Los A ngeles.... following last year’s devastating riots and the resultant drop in tourism. Southern California has been in a recession for two and half years. 50. The President is sticking to his plan to end the ban on gays in the m ilitary___ and says he’ll begin taking the steps to implement i t . . . . despite considerable i opposition from Capitol Hill and the Pentagon. Yesterday Clinton’s plan met 119 with strong objection from military leaders. Today Clinton will probably face more opposition when he meets with congressional leaders. 51. Treasury Secretary’s announcement of a possible tax on oil and gas has met with praise and criticism from congress. Lloyd Bentsen says an energy tax on gasoline and oil would raise revenue and encourage conservation. Two senators from each party jointly endorsed the idea of shifting tax collection from income to consum ption.. . . saying it would boost investment and savings. But Clinton’s proposal took some heat from Republicans on Capitol Hill. Representative Newt Gingrich accused the W hite House of “abandoning” the middle class tax cut and replacing it with a tax increase on everyone. 52. In Israel, with a Supreme Court decision looming on the legality of the govern m ent’s expulsion of Palestinians.... the army says it will allow the 396 deportees to meet with attorneys and appeal their banishment. Military officials say all the remaining deportees will be allowed to contact a lawyer a meet with them outside their desolate camp in southern Lebabon. The suspected Muslim militants were deported five weeks ago afer a series of attacks on Israeli soldiers. 53. Virginia police are conducting a all-out manhunt for a lone gunman who killed two Central Intelligence Administration employees at the agency’s headquarters yes terday. Three others were wounded in the shooting spree outside of C-I-A’s Lang ley complex. Eyewitnesses say the gunman displayed no emotion as he walked up to cars getting ready to pull into the spy headquarters.. . . and fired point-blank into cars with a hunting rifle. Police suspect the man might have had a vendetta against the spy agency. 54. Washington State’s only openly-gay state lawmaker has introduced a gay and lesbian civil rights bill in the Legislature. Seattle Democrat Cal Anderson says the civil rights bill serves as a “preemptive strike” against any anti-gay initiatives that may be proposed. Representative Anderson’s bill would give gays and lesbians the same protections against discrimination that other minority groups enjoy. The legislation covers discrimination in housing, employment, public accomodation, and credit. The bill is a top priority for W ashington’s gay community. T hey’re concerned about anti-gay initiatives spreading north from Oregon. 55. A Wisconsin lawmaker says she wants to strengthen the state’s stalking law. Senator Margaret Farrow says current law only punishes stalking if the victim already has a restraining order or court injunction against the stalker. Last year a woman was killed at the Milwaukee County Courthouse as she was going to get a restraining order. Farrow says stalking itself should be a crim e.. . . and the ! victim shouldN’T have to get court protection first to stop it. 56. A National Association of Private Enterprise survey claims 80 percent of small business owners blame Congress NOT the presidents for the record budget deficits in the last 12 years. 56 percent think the president and Congress should take drastic m easures.. . . such as deep cuts in defense and domestic programs and tax 120 increases, to balance the deficit. Even if the cuts mean a short-term economic slowdown. 37 percent of the entrepreneurs said the deficit should be Clinton’s top priority, followed by job creation (24 percent), health care (17 percent), education (11 percent) and crime (9 percent). 57. Fighting in Somalia today 23 miles northeast of Kismayu. A Belgian armored vehicles backed by U-S helicopter gunships engaged forces of a Somalian warlord who had moved toward Kismayu to fight forces of other Somalian warlords that had taken position near the town. U-S forces sent an ultim atum to the w arlord .... General Mohamed Said Hersi M orgon.... to evacuate. When the deadline passed withOUT response U-S and Belgian forces issued warning shots and then open fired. Unconfirmed reports say the fighting left 42 Somalian casualties. There was NO breakdown of dead and wounded. NO U-S or Belgian casualties were reported. 58. The European Community is calling on all former Yugoslav republics to end fight ing which recently erupted in C roatia.. .. around the port city of Zadar. Today two French U-N peace-keepers died and three others were injured as Croatian forces pressed against Serb-held territory there. The 12-member E-C warns the consequences of fighting in Croatia could be extremely grave. In a statem en t.... the E-C warned recent action risks harming peace talks in Geneva which have shown promise of making headway recenlty. 59. Defense Secretary Les Aspin says there’s a possiblity Iraqi missiles are moving into Iraq’s southern NO-fly zone. He says U-S officials are monitoring the movement. Meanwhile Iraq’s deputy prime minister Taraq Aziz denied any military ag gression. Aziz appeared on C-N-N earlier today. He say he’s hoping President Clinton will renew good relations between Washington and Bahgdad.Clinton’s spokesm an.. . . George Stephanoupolos reiterated the administration’s position that good relations can only be establish if Iraq complies with all U-N resolu tions. 60. A meeting is underway between President Clinton and the Joint Chiefs of Staff, (starts 4 p.m. E.S.T) They’re discussing Clinton’s promise to lift the ban on gays in the military. Hours before the meeting Clinton said he’d listen to their objections.. . . but that he intends to keep his commitment. He said he did want their imput on what the best plan would be. The president declined to say how soon he’d lift the b a n .. . . which is strongly opposed by the Joint Chiefs. However | Clinton’s spokesm an.. .. George Stephanopoulos.. . . later said it could take a few months before the ban is lifted. 61. Secretary of State Warren Christopher says he want the U-N to reflect the new I world order. He says Germany and Japan should be included in an expanded | U-N Security Council. Their inclusion would more accurately reflects post-Cold J War realities. Christopher made his remarks today at a “get-acquainted” meeting I 121 with State Department employees. He also said among the adm inistration’s top priorities are resolving both the Yugoslav conflict and relations between the State Department and C ongress.. . . which are often stormy. 62. A manhunt continues today after a shooting spree at the C-I-A headquarters. The gunm an.. . . thought to be in his 20s and wearing an Army jacket killed two people and wounded three others. Police and eyewitnesses say the man was firing randomly at cars as they entered the main entrance of the building at 8 o ’clock this morning. The land surrounding the Northern Virginia facility is NOT pa trolled by government security. State police are on the lookout for two vehicles.... a white van and an older model black station wagon. Gilbert Robinson, who witnessed the shooting told UPI he watched the man firing at the cars. He says he didN’T think anyone realized what was happening. Robinson says QUOTE “It was like in a dream.” 63. Sears is in financial trouble and today the retail giant announced it was cutting back. The company will lay-off about 50-thousand full-time and part-time em ployees, close more than a hundred of UNprofitable retail and specialty stores and discontinue its catalog operations a marketing tool that helped it become the nation’s top retailer. Shutting down the catalog wil claim at least 34-hundred full tim e and 16- thousand-five-hundred part-time jobs in itself. The last catalog will be the Spring 1993 edition. 64. Today’s New York Post headline blared “Never say die!” after New York businessman Steven Hoffenberg breathed new life into the tabloid. Hoffenberg stepped in last night just minutes before Post Publisher Peter Kalikow was to announce suspension of the publication. Hoffenberg met today with Kalikow and bankers to hammer out a deal that could lead to his buying the New York Post the nation’s oldest daily newspaper. For now, Kalikow will pay the paper’s expense while he investigates the possibility of buying it. 65. Nominees for the Director Guild Awards were named today and on the heels of his Golden Globes trium ph.. . . Clint Eastwood is among them for his slam- bang anti-violence Western, “Unforgiven.” Also nominated are Rob Reiner for the military courtroom drama “A Few Good Men” and Neil Jordan for his stark drama “The Crying Game.” Others included James Ivory for his classic English story manners and morals “Howard’s End,” and Robert Altman for his cynical inside look at Hollywood through the eyes of a hustling executive, “The Player.” The 45th annual D-G-A Awards will held March 6th. 66. Anti-abortion demonstrations that loomed over the nation’s capital last week con tinued yesterday in Tallahassee. In the pouring rain more than 60 anti-abortion picketers squared off against 120 abortion rights activists at an abortion clinic. The two sides held sign s.. . . stared at each o th er.. .. once in a while injected their points of v iew .. . . and after an hour went home. The confrontation ended a 122 weekend of protests and reaction marking the 20th anniversary of the Roe versus Wade decision Supreme Court decision that legalized abortion. 67. Prosecutors in Florida say they’ll seek the death penalty for a school dropout convicted over the weekend in the strangulation of a Florida A-and-M University student. 20-year-old Richard Tony Robertson was found guilty in the August 1991 death of 18-year-old Caarmela Fuce. Defense attorneys argued Roberstson was NOT responsible for the death because he was drunk on alcohol and stoned on drugs.. . . and police coerced a confession from him. The sentencing phase of the trial begins today in Tallahassee. 68. Mia Farrow is in the news today but not because of her custody battle with Woody Allen. A man allegedly posing as Farrow’s personal shopper tried to walk out of Ralph Lauren’s Polo store in Beverly Hills with ten-thousand-dollars worth of merchandise. 26-year-old Julian Lee Hobbs is charged with burglary and grand theft. He’s being held on 10-thousand-dollars bond. 69. Cuban track star Ana Fidelia Quirot who was badly burned in a fire at her home Friday.. .. gave birth to a baby girl. Cuba’s official news agency says the burns induced the birth and that the baby girl was born in perfect health. The child was taken to an intensive care unit. She won a bronze medal in the women’s 800 meter at the Olympics last summer in Barcelona. The 29-year-old middle distance runner was hospitalized Friday with burns to 38 percent of her body. 70. The Big Three automakers are reportedly poised to go on the offensive. The New York Times reports today the Big Three will be asking the Commerce Depart ment for punitive duties on all imported cars. IF successful the move could add thousands of dollars to the sticker prices of imported vehicles. The Times says G-M, Ford and Chrysler will charge that Japanese and European automakers are dumping cars on the American m arket.. . . violating U-S law by selling their cars for less in the United States than they do in their home markets. 71. President Clinton is pushing against two very tough obstacles as he tries to lift the ban on gays in the m ilitary the military brass and the top powers in Congress. S till.. . . Clinton insists he’ ll move within a week to gradually end the ban. The president held a two-hour session yesterday with the Joint Chiefs of Staff. A pres idential spokesman calls the meeting “cordial, honest and respectful”. . . . though George Steph-a-nopoulos admits there’s a conflict of opinion between Clinton and the Pentagon officials. The president is set to meet with congressional leaders to d a y .. .. and he’s expected to meet with stiff opposition. 72. The Clinton administration is trying to tiptoe around a proposal for a broad- based energy tax. Treasury Secretary Lloyd Bentsen sent a signal Sunday that the tax hike was in the offing. But when asked about the tax yesterday I Clinton wouldN’T rule it in and wouldN’T rule it out. The president said, | “We have a lot of options under consideration but NO decision has been made.” 123 Then Clinton went out of his way NOT to disagree with B entsen.. . . saying he did NOT wound to contradict him. 73. The economists keep saying the recession is long o v e r .... and key American companies keep announcing massive job cuts. Once-thriving Sears, Roebuck and Company says it will scrap its 100-year-old catalog business.. . . shut down more than 100 Unprofitable stores and eliminate 50-thousand jobs. 16-thousand of those disappearing jobs are full-time. From St. L ouis.. . . McDonnell Douglas says it will lay off about 10 percent of its work force this year to cut costs. T hat’s about 87-hundred people. McDonnell had already announced on Friday the expected reduction of 4- to 5-thousand jobs at its Douglas Aircraft commercial operations so yesterday’s announcement adds 37-hundred 47-hundred job cuts on its defense side. A spokeswoman says the company hopes most of the cuts on the defense side can be made through attrition. 74. Northern Virginia police are still hunting that gunman who killed two C-I-A em ployees.. . . and wounded three other workers as they sat in traffic yesterday morning. Officers are describing the shootings outside the gates to C-I-A head quarters as random .. .. but they’re investigating whether the killer has a vendetta against the intelligence agency. Two of the dead and two of the wounded were C- I-A em ployees.. .. all men who were analysts for covert operations or intelligence officers. A third man who was injured worked for a C-I-A contractor who helped maintain the agency’s facilities. 75. The Philippines says its ambassador to the United States died while under treat ment at a Washington D-C hospital. Pablo Suarez was 65 years old. The foreign affairs department in Manila says the veteran diplomat had been confined to the Providence Hospital in Washington for two days before his death early today Philippine-time. The government says Suarez died of internal bleeding. 76. Like restaurant chains? Wondering about where to eat? The Olive Garden has snapped up honors for the fourth straight year as the nation’s favorite dinner- house restaurant. T hat’s according to a nationwide survey conducted for Restau rants and Institutions Magazine. When it comes to hamburgers, W endy’s grilled top honors for a fifth year in a row. Chi-Chi’s regained top honors for Mexican food, Pizza Hut held on to the favorite pizza place title for a ninth year, Red Lobster is the favorite seafood chain and Baskin-Robbins is the favorite sweet tooth spot. The chains are graded in seven categories.... food quality, service, menu variety, atmosphere, convenience, value and cleanliness. 77. Like many other d ad s.. . . President Clinton saw his daughter off to school this morning. UNlike other schoolkids.. .. 12-year-old Chelsea had to take along her Secret Service agents. Just before she left Chelsea hit up her dad for an allowance. C linton.... with a blue mug of coffee in one h an d .. . . dug into his pockets and came up with a 20-dollar bill. A smiling president told watching reporters, “T hat’s all I had.” I 124 I 78. More good news on the econony just out today. Sales of previously owned homes rose for the third consecutive month in December. The National Association of Realtors says home sales were buoyed by rising consumer confidence and low interest rates. The association said sales of existing homes rose 5 percent last month. 79. Sales of existing homes rose 10 percent in October, another five percent in Novem ber, and ANOTHER five percent last month. Economists watch this report from the National Association of Realtors and compare it to the government’s NEW home sales index. These numbers suggest old home resales are stronger than new home sales so far. It’s a good sign because it means people who already own homes are trading up. And it will be better when new home sales catch u p .. .. because that will mean more first-time homebuyers. 80. Boeing says it ’s cutting back commercial aircraft production. No layoff notices y e t.. . . but Boeing, a major employer in the Seattle area, expects its decision to have a significant impact. McDonnell Douglas, which employs lots of people in St. Louis, says it ’s reducing its workforce 10 percent, laying off almost nine-thousand. And you’ve heard about the Sears plan to close more than 100 stores, terminating 50-thousand jobs. Analysts say the trend in major corporations is to solve any bottom -line problems by slashing payroll. 81. I-B-M did the expected this morning reducing its regular quarterly dividend. It cut the payment about 55 percent to 54 cents a share from a dollar and 21 cents ($1.21) as it struggles to retain its place in the international computer industry. The dividend is payable March 10th to shareholders of record February 10th. 82. President Clinton says he’s intent on lifting the ban on gays in the m ilitary.... despite the controversy it ’s causing on Capitol Hill and in the Pentagon. Clin ton met with a bi-partisan group of congressional leaders this morning and heard more complaints about his plan to remove the ban. Georgia Senator Sam Nunn, a Democrat, reiterated his adamant opposition to lifting the ban as did Republican leader Robert D o le.. .. who says there’s strong support for the present policy. 83. Lawsuits have been filed over an outbreak of food-poisoning at Jack-in-the-Box restaurants in Washington State. About 200 cases have been reported.. . . and at least one death, that of a two-year-old.. .. has been blamed on the poisoning, i The federal suits say an adult woman and young child got sick after eating tainted hamburgers. Jack-in-the-Box officials say the bad meat came from a supplier in ! Los Angeles. | 84. A Massachusetts boy who needed 400 stitches to close wounds inflicted when he [ was attacked by his two Great Dane dogs is finally home. Eight-year-old Jonathan ' Caldwell spent six days in a hospital. His dad says the dogs will probably be i stroyed and Jonathan wants another d o g .. . . just a smaller one. 125 ! 85. The search is still on for the man who opened fire at the C-I-A in Virginia yester day. Two employees were killed and three were injured after the gunman shot at cars turning into the agency’s main gate. Police released a composite drawing and description of the killer. He’s described as a white male with dark complexion---- in his 2 0 s.. . . about 5 feet 8 to 10 inches tall weighing around 155 pounds. He has dark brown or black hair and was wearing a tan jacket and dark colored p a n ts.. . . possibly blue jeans. 86. Police are trying to find out who placed a bomb in a high school bathroom in Mandarin, Florida yesterday. It went off. No injuries.. . . but students in the vicinity were startled. Since last month, several bottle bombs made from an acid and aluminum mixture have exploded in mailboxes in the community just south of Jacksonville. 87. The Texas campaign chairman for former President Bush is seeking documents about the F-B-I’s investigation into Ross Perot’s allegations of dirty tricks. The Bureau investigated Perot’s allegations that Jim Oberwetter and the Republican Party tried to get information about the Perot family to use in a smear campaign. 88. The Rhode Island Supreme Court has adopted new rules to remove the veil of secrecy from the process in which lawyers are disciplined. For the first tim e, non lawyers will serve on the court’s disciplinary board. Chief Justice Thomas Fay says the new procedures open the disciplinary process to public scrutiny from beginning to end. Previously, charges of misconduct were not made public until after the discplinary board held a hearing and made a decision. 89. It’s Dan Quayle’s turn. The former vice president has sold his memoirs about his four years in the W hite House for a reported seven-figure sum to Harper Collins. Reports (in The Washington Post) say the book will begin with Quayle’s selection as George Bush’s running mate and end with November’s defeat. The book will be sold in Christian-oriented markets through Harper’s religious subsidiary. It’s expected to be published next year. No title yet. 90. Texas’ lieutenant governor says he was just teasing a female lawmaker when he said she’d be more effective wearing shorter skirts and high heels. Bob Bullock put his foot in his mouth last week when he suggested state Senator Judith Zaffirini could pass more bills if, in Bullock’s words “she’ ll cut her skirt off about six inches and put on some high heels.” Zaffirini says she wasn’t offended by the remark. But her defense of Bullock failed to stem a torrent of protests from women’s groups. Bullock now vows not to make similar jokes publicly in the future. Again, using his w ords.. . . “I can assure you that I’m going to be leaving dress fashions up to Vogue magazine and the governor of Texas from here on.” The governor is a woman Ann Richards. 91. President Clinton is exploring avenues to spend upwards of $20 BILLION to spark economic activity. His aides said today he’ll ask Congress to finance targeted 126 programs that would initiate creation of jobs. Clinton gives his first State of the Union Message in three weeks (Feb. 17th). Today Clinton’s spokesman refuted assertions by a many economists that 20 BILLION isN ’T enough and would reap few visible dividends. A number of Democratic lawmakers support Clinton’s plan. The announcement came amid bleak news for his Clinton who’s trying to reduce the national debt while maintaining economic recovery. The Treasury Department says the U-S budget deficit is bigger than first thought. For fiscal 1 9 9 3 .... the deficit widened to $38.9 BILLION in December, putting the red ink at $120.5 BILLION for the first three months of this year. The quarterly deficit is $36.7 BILLION higher than the gap for fiscal 1992’s comparable period. That wasn’t good news for President Clinton. 92. House Republican Newt Gingrich said today an executive order lifting the ban on gays in the military would be overturned by Congress but the W hite House says President Clinton is determined to follow through on his campaign promise. Texas Senator Phil Gram says Clinton’s decision to make the issue a priority in his first week is “surprising and worrisome.” In Gram’s w o rd s.... “That the president would overrule America’s joint chiefs of staff and erase a long-standing defense policy withOUT even guessing at the consequences of his action s.. . . is UNfathomable. The W hite House says lifting the ban will involve two steps an initial presi dential order followed up by a strict code of conduct that will likely need action by Congress. 93. The Senate Labor and Human Resources Committee has approved a “family leave” measure, providing employees 12 weeks of unpaid leave in the event of a birth, adoption or illness for them or their immediate family. As the Senate committee approved the legislation, the House Education and Labor Committee began hearings on the same measure, with Labor Secretary Robert Reich as the leadoff witness. An indentical bill was vetoed by President George Bush last year, but President Clinton is on record as favoring the legislation and promises to sign it when it reaches the W hite House. 94. Thomas Pickering.. .. the career diplomat who helped engineer the international effort against Iraq is President Clinton’s choice to serve as ambassador to Russia. Pickering holds the highest rank in the Foreign Service and speaks Span ish, French, Swahili and Arabic. He was tapped by President Bush to serve at the United Nations in 1988. He was most recently ambassador to India. 95. In former Yugoslavia to d a y .. . . a renegade Serbian barge captain has threatened to spill oil into the Danube River. The State Department says he escaped from Romanian sanctions enforcers and is steaming toward Belgrade with a load of illicit diesel fuel. Romania and Bulgaria are responsible for monitoring traffic 127 in an area bordering Serbia they share. American diplomats in Belgrade have been instructed to inform Serbian authorities they will be held responsible for any willful environmental disaster. 96. I-B-M did the expected today reducing its regular quarterly dividend. It cut the payment about 55 percent to 54 cents a share from a dollar and 21 cents ($1.21) as it struggles to retain its place in the international computer industry. The dividend is payable March 10th to shareholders of record February 10th. 97. A British Department of Transport report says the damage to the Shetland Islands from the wreck of the tanker Braer poses far lower threats to the environment than originally feared. The Department hasN ’T closed the book on the disaster---- and monitoring will continue for some tim e to assess the environmental effects of the oil spill. Some 25 Million gallons of light oil were released after the tanker ran aground January 5 north of Scotland. Meantime an oil slick from a Danish supertanker threatening an Indian island was brought under control by the Indian Coast Guard. A fire raging on board the vessel since Thursday was finally extinguished. 98. In Pittsburgh.... a 62-year-old man is in critical but stable condition after re ceiving a Baboon liver 16 days ago. A spokeswoman at Presbyterian University Hospital said today the liver is functioning much better and that the quality of bile the organ is producing has improved. She says the liver which can grow to fit a recipient’s cavity.. .. has grown to the expected volume. The man is the second person to receive a baboon liver transplant. He was dying from the liver disease hepatitis B. 99. In F lorida.. .. two men are in the Hernando County Jail for the weekend beating death of a man who thought he was helping an accident victim. 20-year-old James Lindsey Howze and 18-year-old W illiam Scott Russell were arrested yesterday for the death of Harry Reeves. The victim ’s 12-year-old son found him lying on the road Sunday morning. Authorities say Howze and Russell got into an argument with R eeves.. . . who had pulled over to check a disabled car. Howze and Russell’s car reportedly was on the side of the road because of a flat tire. 100. Mud-wrestling can be hazardous to your health. T hat’s the finding of researchers who studied an outbreak of rashes among college students. Researchers say mud- wrestling can cause pus-filled skin lesions. Doctors at the University of Wash ington in Seattle reported the findings in the Journal of the American Medical Association. They made the discovery when seven students at the University complained of rashes within 36 hours of a “mud-wrestling social event” between men and women. 128

Linked assets