University of Southern California Dissertations and Theses

A representation for semantic information within an inference-making computer program

(USC Thesis Other)

A representation for semantic information within an inference-making computer program

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content A REPRESENTATION FOR SEMANTIC INFORMATION WITHIN AN INFERENCE-MAKING COMPUTER PROGRAM by Robert Wesley Lingard, Jr. A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In'Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Computer Science) June 1975 UMI Number: DP22714 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is d ep en d en t upon the quality of the copy subm itted. In the unlikely ev en t that the author did not sen d a com plete m anuscript and th ere are missing pag es, th e se will be noted. Also, if m aterial had to be rem oved, a note will indicate the deletion. Dissertation Publishing UMI DP22714 Published by P roQ uest LLC (2014). Copyright in the Dissertation held by the Author. Microform Edition © P roQ uest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United S tates C ode P roQ uest LLC. 789 E ast E isenhow er Parkway P.O. Box 1346 Ann Arbor, Ml 4 8 1 0 6 -1 3 4 6 \ o U N IV E R SITY O F S O U T H E R N C A L IF O R N IA T H E G R A D U A T E S C H O O L U N IV E R S IT Y PA R K L O S A N G E L E S . C A L IF O R N IA 9 0 0 0 7 j P H Cf>S ' ‘75 L 755 This dissertation, written by Kobert..WesleY..Ijingard,__J_r ............... £ under the direction of Ais.... Dissertation Com mittee, and approved by all its members, has been presented to and accepted by The Graduate ^ School, in partial fulfillment of requirements of ' the degree of D O C T O R OF P H I L O S O P H Y Dean n.„ . DISSERTATION COMMITTEE O M & y-d L- Chairmaj A B S T R A C T The understanding of "natural" language by computer requires extensive capabilities for drawing inferences on known information. This research is an attempt to develop such a computer program, capable of making inferences on a large and complex collection of semantic information derived from natural language statements. Not much more than simple information retrieval is possible on such a data base without knowing a great deal about the implications of the statements expressed. Real understanding of a body of knowledge implies the ability to answer questions about the information, and hence the implications and interactions of the relations used must be known. That is to say, there must be some "rules of inference" to relate the words and phrases in the questions to the given body of knowledge. In this approach, the program accepts a parsed form of English as input and makes inferences on it based on advice given by the user. The development of a suitable language for advising the program of needed inference rules has been a major part of this ef fort. Also of significance has been the design of an internal re p re sentation for the knowledge base which facilitates both the accessing of information and the process of making inferences on it. ii A C K N O W L E D G E M E N T S I would like to express my gratitude to Professor Frederick G. Knirk for his interest in my work, to Professor F. Roy Carlson for his many helpful suggestions, and especially to Professor Albert L. Zobrist who provided me with constant advice and encouragement in the completion of this dissertation. Also, I would like to thank Ellen, for without her years of patience and understanding, none of this would have been possible. iii T A B L E O F C O N T E N T S page ACKNOWLEDGEMENTS.................................................................. ii ABSTRACT....................................................................................................................... iii LIST OF FIG U RES...................................................................................................... vi CHAPTER 1 INTRODUCTION........................................................................... 1 CHAPTER 2. BACKGROUND .................................. 9 Linguistics . . ........................... 9 P sychology........................................................ 11 Computer Science....................................................................... 12 E d u catio n ................................................ 14 Current O bjectives................................................................... 15 Previous Systems ................................................................... 18 CHAPTER 3 GENERAL APPRO A CH .......................................................... 25 CHAPTER 4 REPRESENTATION OF KNOWLEDGE...................... 42 The Form of the Internal Represen tation ................................................................................. 47 The Knowledge N e t.................... 52 Time and T e n s e .................................................................. 53 The Context M echanism........................................................ 57 N egation......................................................................... 64 Quantification............................ 65 CHAPTER 5 IN FEREN CE.................................................................................. 69 Retrieval of Information......................................................... 72 The Advice Giving Language.............................................. 74 The Inference Mechanism................. 86 CHAPTER 6 CONCLUSION.................................................................................. 99 iv p ag e BIBLIOGRAPHY............................................................................................................. 107 APPENDIX PROGRAM LOGIC DESCRIPTION................................ I l l v LIST OF FIGURES Page Figure 1. Chomsky's View of Linguistic Theory............................... 10 Figure 2. Components of an Automatic P rogram ming System ........................... 13 Figure 3. A Complete Language Understanding S y stem ................................................................................................... 16 Figure 4. A Conversation with the Computer................... 31 Figure 5. A Simple Deduction T r e e .................................................. 32 Figure 6. The Introduction of a New R u le ............................................. 33 Figure 7. A Computer Conversation Involving Family R e la tio n s................................................ 35 Figure 8. The Introduction of a New Relation...................................... 36 Figure 9. The Resolution of Ambiguities................................................ 37 Figure 10. A Complex Deduction T re e ....................................................... 40 Figure 11. A Simplified Problem Solver.................................... 43 Figure 12. A Knowledge Net............................................................................... 54 Figure 13. The Organization of Inform ation......................................... 59 Figure 14. Events in a Baseball Context................................................... 62 Figure 15. A Knowledge Net Containing Two C ontexts.................... 63 Figure 16. Rules in General F o r m ................................. 77 Figure 17. Differences in Grammatical Structure between Roughly Equivalent S en te n ces......................... 84 Figure 18. Use of the Context Mechanism in Disambiguation................................................................................. 90 Figure 19. Supplying a Definition to the Program .............................. 92 vi p a g e Figure 20. Supplying an Implication of a R elatio n................................ 93 Figure 21. The Use of the "And" O p e ra to r............................................. 94 Figure 22. Specifying Grammatical Transfor mation R u le s ............................... 96 Figure 23. A Quantification E xam ple.......................................................... 97 Figure 24. A Negation Example. . ....................................... 98 Figure 25. Overview of the MAIN P ro g ram ...................... 112 Figure 26. Flow Diagram of the VERIFY Sub program ............................................................................. . . . . . 114 Figure 27. Flow Diagram of the MATCH Sub program ............................................................................................... 116 Figure 28. Flow Diagram of the TRANSFORM S ubprogram ........................................................................................ 118 Figure 29. Flow Diagram of the INSERT Sub program ...................................... 120 vii CHAPTER 1 INTRODUCTION No one can deny that computers have revolutionized our society. Their speed and memory capacity have enabled us to solve countless large and difficult problems. In the field of science, computers have had such a great effect that scientific procedures themselves have been changed. In the business world, computers have become a ne cessity, and new applications are continually being discovered. In spite of this, computers are still limited in their usefulness by their inability to understand natural language. In order to use computers, humans must communicate with them. Many languages have been de veloped for this purpose, but most are nontrivial. The typical user must expend considerable effort to gain the necessary degree of pro ficiency in one of these languages to be able to solve even a simple problem. Clearly, many more useful applications would follow if man could interact with computers more easily. Instead of requiring that humans learn a language the computer can understand, why not make computers understand the language of man or at least a m ore "natural" language? Much has been written both supporting and opposing the use of natural language for commu nicating with computers. Without going into great detail, an attempt will be made to summarize both points of view. More detailed a c counts of this debate are given in articles by Halpren [l], Sammet [2], 1 2 and Hill [3], Those seening advantages in "Natural Language Processing" feel that such systems would significantly increase the number of p e r sons able to utilize the capabilities of the computer. If this were true then certainly a natural language understanding computer would be a highly significant contribution for the field of Computer Science to b e stow on society. Opponents, however, argue that English is a poor language for expressing procedures to a computer. They claim fur ther that programming languages do not necessarily restrict one's capability of expression. While there are many things that can be said in English that can't be said in a programming language, the pe culiar properties of programming languages make some things easier to express. Furtherm ore, in learning a programming language one learns more than a new language; one learns how to express the so lution to a problem. Even with the capability to use English, most people would have to learn how to solve problems on a computer be fore the computer could be of use. In fact Hill [3] has suggested that English is so complicated and ambiguous that we should not only write instructions for computers in programming languages but we should use programming languages for giving instructions to people as well. One of the many examples he gives to support his position is the fol lowing set of instructions on a shampoo bottle: For best results wet hair with warm water. Gently work in the first application. Rinse thoroughly and repeat. The question is from where to repeat? Certainly one does not have to 3 wet his hair again after just rinsing it as is implied. Hill gives the following program as a substitute: begin wet hair with warm water; for j : = 1, 2 do begin gently work in application (j) ; rinse thoroughly end end This may seem a bit ridiculous in this case but in something like a legal document such precision might be essential. What then can be concluded about the value of machines that can understand English? Probably both sides in this argument are co r rect, at least as far as they go. Certainly for many applications, programming languages are desirable or even necessary and there is no reason to even think of replacing them with English. In addition, anyone wanting to solve complex problems on computers (or by any means for that matter) will certainly have to know or learn problem solving techniques. However there are many computer custom ers, or potential computer custom ers, who are perfectly capable of perform ing the hard part of programming — that of problem solving — but are frustrated as potential program m ers by their unfamiliarity with the easy part — the translation of their problem solution into a machine acceptable language. It is difficult at best for non-programming p ro fessionals to practice their own demanding disciplines and at the same time keep their knowledge of available programming systems and lan guages current. Either they must depend on professional program - 4 m ers or their effectiveness in their own profession will suffer. The value of a natural language processing system cannot be determined solely on its capabilities for automatic programming (i. e. the ability to produce computer program s from natural language de scriptions). There are whole classes of users, or potential users, who need access to a computer not to solve complex problems but only to obtain information. With the current explosion of information, computer controlled data bases are continually being created and ex panded. But how does one get access to the information needed with out a fam iliarity with computers and programming? The ability to interrogate computer managed data bases without detailed knowledge of programming seems essential. Similarly in the field of education, computers would be of great value if they could understand questions asked or responses given by students. More and more effort is being devoted to developing capabilities for instruction by computers. But effective computer assisted instruction for m ost subjects will c e r tainly depend upon some capability for the processing of natural lan guage. Some people, notably Winograd [4], even argue that such sys tems for language understanding might be of help to a professional program m er in solving complex problems. If the program m er pro vided a description, or semantic model, of this problem solution along with his program , then conceivably some semantic understanding sy s tem could be used interactively to help him develop, debug, or modify his program by answering questions about the functions and intentions 5 behind various portions of his code. Such a capability would also be of great value whenever someone other than the original program m er needed information about the inner workings of the program. Finally, many people see language understanding systems as valuable because of the possible light they might shed on human intel ligence. Since the use of languages seems to be closely related to in telligence, the development of systems that understand language may be of great help in understanding the nature of human intelligence it self. Being convinced then that the development of a natural language system would be a good thing, one might wonder why such things were not developed long ago. What is it about the problem that makes it so difficult? Why haven't computers been programmed to do such things? As has been noted, language understanding is closely related to intel ligence. Herein lies the difficulty. Computers are obedient slaves. They are accurate, reliable, and fast. Unfortunately, they don't really know what they are doing. They do what they are told without understanding why. Typically they have no knowledge of anything be yond what is necessary to perform a requested task. That is, they have no intelligence. One dictionary definition of intelligence says it is "the capacity to m eet situations, especially if new or unforseen, by a rapid and effective adjustment of behavior. " This suggests that for a computer to perform intelligent tasks, it must contain vast amounts of knowledge just waiting to be used should the need arise as well as a program that can effectively utilize this information. 6 The problems involved in developing a language understanding system are representative of those in artificial intelligence in general. The essential issues are the internal representation of knowledge with in a computer and the effective utilization of that knowledge by a pro gram. Before examining how this might be accomplished, some of the different types of knowledge one might wish to represent should be considered. F irst, there is simple encyclopedic information. This is the factual, state of the world, static type of knowledge. Statements such as, "George Washington was the first president of the United States, " "John threw the ball to Mary, " "human beings have two arm s, " and the like are examples. A second type of knowledge can be called rules of inference. This information tells how to transform a given statement, phrase, or word into another of approximately the same meaning. Rules of in ference can tell how to deduce the truth of one statement from others. For example, a rule of inference that says "to fly is to travel by plane" allows one to see the equivalence of the statements "John flew to New York" and "John traveled to New York by plane" and therefore deduce one from the other. As another example, consider the rule, "if A is the father of B and B is the parent of C then A is the grand father of C. " This perm its the deduction that if John is the father of someone who is the parent of Mary, then John is the grandfather of Mary. Such inferential knowledge can be considered as encylopedic to be sure, but it must also be recognized as having special use in 7 understanding other encyclopedic knowledge. A third type of knowledge could be called procedural informa tion or commands. These tell how to do things. For example, the instructions on a bottle of medicine, "take one capsule three times a day, " or the rules for a game of cards, "deal five cards to every player, etc. " are procedural statements. As with rules of inference this type of knowledge can have a special function. There are obviously many m ore distinctions that could be made among different types of knowledge, or different ways could be spe cified for subdividing the knowledge to be represented within the com puter. The point is that a uniform and consistent means for rep re senting all knowledge is needed which still perm its the distinctions among the different functional characteristics which portions of that knowledge may possess. That is to say, the system utilizing the knowledge base m ust have some way of knowing what information can be used as rules of inference, what is procedural, and so on. But since much of the information may have more than one possible func tion, a uniform representation is, if not essential, at least highly de sirable. How can all this knowledge be represented in a computer? C er tainly it can't just be copied from encyclopedias, dictionaries or other books into the computer's memory. The information put into a computer cannot be considered knowledge unless the computer has some way of using it. This implies there must be some organization to the data and some procedures for accessing it. But before much 8 can be said about the representation of the knowledge, it must be known how it is to be used. The simplest use one could make of the knowledge would be sim ple retrieval. This would require only that the data be organized in a manner that would perm it easy location of the desired information. If, however, one wants to be able to ask questions about the stored know ledge, much m ore is needed. Some means must be available to relate or interpret questions in relation to the information stored in the com puter. If p art of the knowledge base were procedural, one might want to execute such statements to change the state of the world (i. e. , the data base). Conceivably one might want to use the stored information to help generate a procedure or program as in the case of automatic programming systems. The internal representation chosen must be a function of the in tended use of the information. Care m ust be taken to capture, within the representation, all the essential knowledge for the purpose inten ded, while at the same time putting it into a form easily usable by a computer program . This research represents an attempt to accom plish these objectives relative to the inference making process. CHAPTER 2 BACKGROUND As an area of stddy natural language processing lies on the boundaries between computer science, linguistics and psychology. In other fields, notably education, it has become an essential part of a great deal of research. It is not surprising, therefore, that it en compasses many different types of problems and areas of concern. The relationship of the present research to each of these fields, as well as its applicability to some specific problems, will be discussed. Linguistics Linguistics is the scientific study of the nature and structure of human language. The goals of linguistic theory, according to the views of Chomsky [5], are illustrated in Figure 1. He holds that the observation of a language leads to a description of it (a gram m ar). This will shed light on the nature of language itself (a universal gram mar) which may provide information concerning the nature of mental processes. Typically linguists have concentrated on how language- is gener ated as opposed to how it is understood. Hence, their emphasis has been on syntactical structures instead of semantics. Much of the early work on machine translation of languages was based on this lin guistic approach. Since linguists were interested in discovering or establishing a universal gram m ar, it was natural to propose that 9 10 Language Data Gram m ar of Language Nature of Language (Universal Grammar) I I t__ Innate Intellectual Structure Observation Description Explanation Figure 1. Chomsky's View of Linguistic Theory 11 translation proceed by transforming the input language into its "uni versal" or "deep structure" form (see Chomsky [6]) and then m erely generate the desired output language from it. These efforts at m a chine translation, though unsuccessful, were significant. They con vincingly demonstrated the hopelessness of performing translation without considering semantics. In a recent effort Wilkes [7] has pointed out that although a deep and complete understanding of textual m aterial is not needed for adequate translation, it is certainly true that something more than syntactical analysis is necessary. In fact, it has been demonstrated that syntactical analysis itself is heavily dependent upon semantics. The current research has application in precisely this area. The system developed can work effectively to provide the semantic information necessary for the proper analysis of syntax. F u rth er m ore, the system has been designed in such a way that it can be used as a tool by both the linguist and the computer scientist for working on the problem of language understanding. Montgomery [8] has pointed out the complementary nature of linguistics and computer science in the area of automated language processing and has urged an integra ted approach toward solving the problems involved. The present work has been done in that spirit. Psychology Much of psychology today views the human mind as a sophisti cated information processing system. A computer system for proces 12 sing information, particularly one utilizing natural language would seem to provide an excellent model for human thought processes. In fact, Colby [9] has used such a program to simulate human belief system s. The effectiveness by which this can be done depends upon how successfully natural language can be processed. But just as the psychologist is depending upon computer science to produce a model for language understanding, the computer scientist looks to psychol ogy for advice on how to develop such an intelligent system. As with linguistics, there is clearly a need for an integrated approach between psychology and computer science toward the development of language understanding system s. Computer Science Computer science naturally leans heavily on the use of compu ters to solve complex problem s. Systems for automatic programming lie alm ost wholly within this field. The fundamental concern is the development of systems to enable non-professionals to effectively use computers to solve problems. B aizer's model [10] of an automatic programming system, depicted in Figure 2, contains three essential elements. The first is called "problem acquisition" and is concerned with obtaining an understanding of the u ser's problem in the domain in which it exists. The next is referred to as "model completion. " This phase fills in pieces m issing from the model of the u ser's prob lem as generated by the "problem acquisition" phase. Finally, the "interpreter" executes the completed model thus solving the problem. 13 Knowledge Base Interpreter Problem Acquisition Model Completion Figure 2. Components of an Automatic Programming System 14 The first two elements of this system m ust rely heavily on some in ference making mechanism in order to understand and complete the u ser's problem specification. Even the "interpreter" will probably need some means for making inferences on the data base. The de velopment of such inference making capabilities is the object of the current research. Education Current theories within the field of education indicate that im provement of the educational process can best be achieved through greater individualization of instruction. The high cost of teachers leaves little hope of accomplishing this with human instructors. The computer provides an alternative. As Carlson [ll] points out com puter assisted instruction (CAI) can provide the solution. If instruc tional m aterials can be stored in the computer so that the student can access them directly in an interactive manner, then the groundwork for individualized instruction has been laid. For now the computer can decide, based on the student's characteristics and performance, how best to structure and present the instructional m aterial. Many system s, such as those of Carbonell [12], Simmons [13], and Wexler [14], have been developed which to varying degrees accomplish this. But the subjects and extent to which such systems can be applied are greatly limited by their capability to understand student generated in puts. Effective computer assisted instruction will certainly require the ability to understand and make inferences on student responses. 15 A good teacher (or CAI program) must be able to answer most ques tions a student asks and m ust be able to determine when a student's answer to a question is correct. It was from a study of this question answering aspect of CAI that the present research was initiated. Current Objectives The main thrust of this research is in developing a computer system capable of "understanding" the information contained in a given data base. That is, the system must be able to effectively use the data base to answer questions or verify statements made about the in formation contained in it. A complete system would have a module which could accept nat ural language statements and parse them. Next some component would be able to take a set of parsed sentences and produce an organ ized structure or semantic net containing the "knowledge" embodied in them. A third piece of the system would be capable of accessing and manipulating this stored information in an efficient and effective m anner in order to answer questions about the data base. Finally, there would need to be some component in the system to formulate responses in natural language. Such a system is depicted in Figure 3. Although a great deal of effort has been spent on systems of this type, their overall capabilities are somewhat disappointing. The only com plete systems developed so far either operate on only extremely small data bases or provide only shallow analysis capabilities. In general, m ore success has been attained with the first and fourth components no yes SEMANTIC NET OUTPUT Natural Language INPUT in Natural Language ' i s it a N Question Organizes and structures in formation into a semantic net BUILDER Translates from internal form to Natural Language RESPONDER Retrieves ans wer using infer ence if necessary RETRIEVER Syntactically Analyzes Sentences PARSER Figure 3. A Complete Language Understanding System 17 than with the second and third. Although it is impossible to com plete ly divorce the first component from the second, since the output from the p arser m ust be used by the analyzer, the present research will assume the existence of a p arser. Assumptions about the capabilities of the p arser will be discussed later. Likewise, this effort will ig nore the last component. The focus of this work is on the highly interdependent modules which build and use a knowledge data base. Obviously, one cannot design a program to access and manipulate a data base without know ing something about the organization and structure of the information. But the form of the representation of the data should be a function of its anticipated use. Clearly, the development of a suitable internal representation for the data and the design of the module to analyze it m ust proceed concurrently. As has been stated before, in order to effectively utilize the in formation in a data base, considerable additional knowledge m ust be available. This knowledge includes meanings of words, the interac tions between relations, and real world knowledge. Human beings bring this knowledge with them. In most previous efforts in language understanding systems this knowledge has either been programmed in, or it has been lacking. So far it has not been possible to provide the knowledge necessary for making inferences in the same way as other knowledge. In this research an effort has been made to develop a capability for supplying this type of knowledge in a manner sim ilar to that for encyclopedic information. A language has been developed which gives the user a means by which he can advise the program as 18 to how to make inference. It is hoped that this approach will make it possible for experts in an application area who have little knowledge of programming to easily supply any necessary inferential information required for understanding the domain data. In this way it will be possible to take advantage of a u ser's expertise in some area without requiring the user to learn how to program. Previous Systems Certain aspects of previous efforts in the field have had signifi cant impact on the direction of the current research. A brief survey of developments which relate to the present work follows. The earliest program s for representing knowledge and answ er ing questions about it could deal only with simple information, usually about a single subject. The information would have to be expressed in a restrictive fixed format, and the program s usually amounted to simple information retrieval. G reen's Baseball program [15] and Lindsay's SAD SAM [16] are examples. The deductive capabilities were in the way the data base was organized. These program were neither capable of understanding over m ore than a specific subject nor were they capable of any depth of understanding. They are signi ficant, however, because they represent the first efforts to achieve language understanding by computer. Since these first attem pts, many developments have taken place toward achieving greater depth of understanding or understanding over a wider domain or both. Weizenbaum's ELIZA program [17] has 19 probably both the widest breadth of understanding and the least depth of any developed. The program tries to emulate a Rogerian psycholo gist by responding in a non-directed way to statements made or ques tions asked. It accomplishes this by making simple transformations on input sentences and using the results as its responses. The pro gram operates purely at the syntactical level and has no "understand ing" of the information it deals with. In efforts to develop general language understanding systems some researchers concentrated on the problems associated with p a rs ing English. The augmented transition networks of Woods [18], the MIND system of Kay [19], and Kaplan's General Syntactic Processor (GSP) [20], fall into this category. These systems are designed to accept arb itrary English sentences and produce parses. Woods' sy s tem provides a computationally efficient method for analyzing English sentences. Furtherm ore, as Kaplan [2l] has shown, the left to right analysis method of the transition network seems analogous to the way humans process sentences. Hence, this method may have value as a model of human understanding. The MIND system is a combination of many linguistic processors. It is highly interactive, allowing users to modify rules, monitor progress of the parse, and aid in disam bi guating sentences. The General Syntactic P rocessor of Kaplan is derived from an abstract conception of syntactic processing tasks and serves as a tool for comparing and evaluating various parsing schemes. These system s all lack any capabilities for deep understanding 20 or any significant inferencing capabilities. In fact, because of this deficiency, these systems cannot always produce an accurate parse. The MIND system attempts to overcome this by allowing the user to interact and help the system when it encounters difficulties. While such a feature is extrem ely useful, it is one of the goals of the p re s ent research to provide an inference making capability that such a parsing system could use to lessen its need to ask for outside help. In efforts to achieve a deeper understanding of input information many researchers have ignored the problems associated with parsing English. By starting with a parsed form or a highly restricted sub set of English, they were able to concentrate on making inferences on the data base. Even so, successful efforts usually required fu r ther restrictions on the problem. Raphael's SIR (Semantic Informa tion Retriever) program [22] could only understand six semantic r e lations, and Bobrow's STUDENT program [23] was limited to the do main of high school algebra problems. Both program s could make simple deductions only because the knowledge of how to do them was programm ed into the system. The addition of new deductive capabili ties would have required both programming additions and modifica tions. The systems of Black [24] and Shapiro and Woodmansee [25] overcame this difficulty by allowing deductive rules to be supplied as input. The program of Shapiro and Woodmansee experimented with inductive reasoning as well. Although these systems provided ad ditional flexibility they were still limited in the type and complexity 21 of inference rules that could be input. Some research ers tried to increase deductive capabilities by using a predicate calculus form for the internal representation of facts and applying theorem-proving techniques. Coles [26] developed a system for translating a subset of English into a fully quantified predicate calculus, while Darlington [27] and Green and Raphael [28] worked on systems for applying the resolution method of Robinson [29] to a first order predicate calculus representation of natural language. Although many are hopeful that such techniques will eventually achi eve success, there are still many difficulties to be overcome. Even the best automatic theorem-proving techniques based on Robinson's resolution approach work only for first order predicate calculus. As Simmons states in his review of natural language system s [30], "those concerned with question-answering. . . have encountered well-known shortcomings in the first order predicate calculus, with reference to representing equality, recursiveness, modality, tense, and the de tailed quantification structure of English." Although some systems, like that of Coles, have gone beyond the first order predicate calculus, no really effective automatic theorem-proving techniques have yet been developed for these higher order logic system s. A second dif ficulty with the application of general theorem-proving techniques is that, without a good set of heuristics to guide the search, proofs tend to be very time consuming. Unfortunately, there is no easy way to utilize subject m atter or contextual knowledge to guide searches when a uniform theorem-proving procedure is used. Finally, what might 22 be considered a difficulty with such an approach is that it seem s in consistent with the way humans analyze language. Not that a machine need operate in the same way humans do, but it seems a mistake to ignore one of the best examples of an intelligent system in trying to develop another one. Humans clearly do not analyze sentences by constructing proofs, rather they use a large and complex set of h eur istics. It seems that m ore effort should be devoted toward determ in ing and utilizing heuristic techniques for language understanding. In the current research attempts have been made to allow certain search guiding information to be input as advice as well as organizing the data base in a form that perm its utilization of contextual information. A system with significant deductive capabilities for question- answering is Protosynthex III (Simmons, et al [31] and Schwarcz, et al [32]). .This system represents English sentences internally as nested (Concept-jRelation-Concept) triples. It allows deductive inference rules and certain semantic advice to be input. The main disadvan tages to the system are its inability to effectively handle quantification and its limitation to nested triples for the representation of inform a tion. This latter problem limits the complexity of sentences and rules of inference that can be expressed since only binary relations are possible. In the present research an internal representation has been developed which generalizes this nested triple approach. Yet another approach to language understanding has been taken by Schank [33]. It is based on the view that all sentences can be rep resented by using only a sm all number (presently about fourteen) of 23 "prim itive" actions in an actor-action-object framework. Since there are only a few actions the inference mechanism can be fairly simple and straightforward. Input statements and questions are all reduced to their "primitive" form, and ail inferencing takes place at this level. The main problem with this approach is that although the inferencing is straightforward, the number of inferences that have to be made at this "primitive" level to answer even a simple question can easily get out of hand. The present research rejects this approach maintaining that questions can usually be answered at a level close to the level in which they are posed and that the analysis of statements and questions into their basic of fundamental meanings (if such really exists) is in efficient and unnecessary. Winograd [34] has developed one of the most complete systems for natural language understanding. It is capable of parsing English sentences of extremely complex form. It is able to make complex in ferences and even make responses in English. To a large extent all this is possible because of the limited domain over which the program operates. It functions in a "Blocks World" in which there are only a few objects and only a few actions or relations possible. But it is the approach taken in this system that is of real significance. Winograd recognized the limitations of the theorem-proving approaches and sought to develop a way to provide heuristic information to the p ro gram making inferences. By using PLANNER, a goal-oriented pro cedural language developed by Hewitt [35], Winograd was able to sup ply the heuristics needed for inference making. Since in PLANNER 24 one does not have to be concerned with interactions between proce dures, many of the problems encountered by Raphael in his SIR p ro gram were overcome. These heuristics still have to be programmed, however. It is not possible in Winograd1 s system for a non-program m er to supply information to the program as to how to make inferen ces. It is the prim ary goal of the present research to provide the capability by which such non-programming individuals can supply at least certain types of inferential advice. CHAPTER 3 GENERAL APPROACH If there is anything in common among all the previous efforts in natural language processing, it is that they point out the futility of programming into a system all the knowledge necessary to function "intelligently" over any domain of significant size. The problem is just too complex. The standard approach has been to try to achieve a uniform representation of knowledge and break apart the program necessary to understand that knowledge into separate, independent, easily understood modules. The basic assumption is that the in ter actions between pieces of knowledge and between program modules can be handled in a simple straightforward manner. Unfortunately such is not the case. It is precisely in these interactions that most of the complexity lies. Another hope, although quite vague and as yet un tried, is that the problem might be solved by developing a program that can learn. But learning itself is a complex, and as yet unex plained, process. There is no reason to believe that developing a learning program would be an easier task. It is clear, at the present state of the art, not enough is known to develop a complete language understanding program . A great deal can be done, however. The key to success is in recognizing what the computer can be made to do and what it cannot (at least for the p re s ent). The most promising approach seems to be in achieving a re a s onable separation between the tasks the computer is to perform and 25 ________ 26 those required of the user. If a reasonable means can be provided for a user to supply the computer with second order knowledge (i. e. , the knowledge of interactions needed to understand other knowledge) in a form that can be effectively used, a significant step toward de veloping a general language understanding system will have been achieved. This is the advice giving approach first proposed by McCarthy [3 6] which has been used in other program s in the field of artificial intelligence. The chess playing program of Zobrist and Carlson [37] is a recent example of the use of advice giving. The program has all the knowledge of the rules of chess built in. But the knowledge how to play well is input by a user. It is data to the p ro gram and hence can easily be modified or supplemented. The user m ust supply this advice in a form which can be understood and used effectively, but he need not know anything else about the program. It is thus possible for a non-programming chess expert to supply know ledge about chess to the program . This is basically the approach taken in this research. A p ro gram has been developed which is capable of making inferences on an arb itrary data base according to advice given by a user. The p ro gram knows the basics of how to access information in the data base and how to utilize rules of inference. The rules themselves can be supplied as data by experts in linguistics or by various subject m at ter experts as required. A second class of user can then use the program to answer his questions about the information in a given data base. This research centers around the means of specifying this in- 27 ferential advice. It became clear early that the key to being able to see equival ences among statements or being able to infer one statement from another lay in understanding the relations existing in them. Raphael's SIR program [22] had separate subprograms for understanding each of the relations it could handle. F o r example, one subprogram under stood the relation "left-of. " It could deduce that if A was to the left of B, and B was to the left of C, then A was to the left of C. The b a sic concept understood by the subprogram might be written as left-of if left-of left-of. (3. 1) That is, the program knew the recursive nature of the concept. It was capable of applying the rule expressed by (3. 1) any number of times to provide a necessary deduction. The form of (3. 1) is suggestive of a syntax rule. In fact, the basis of the user language for giving advice to the program is in expressing concepts like (3. 1) as production rules in a gram m ar for natural language relations. The analog of (3. 1) would be the production rule, left-of -» left-of left-of. (3. 2) There are several advantages to this way of supplying inferential knowledge to the computer. 1) The language is concise and powerful in that concepts that might be difficult to program are easy to state. 28 2) It is a fairly simple and "natural" language. Hopefully non-programming users can find it easy to supply advice to the program. 3) It is general in that it is not restricted to any particular domain of discourse and is applicable over a wide range of natural language relations. 4) The advice given in this form is easily usable by the program in making inferences. 5) The uniform representation of the advice makes the rules easily inspectable by the program . That is, it could be possible to ask questions about the rules themselves or make decisions about which rule to use at a given point. 6) It is easy with this approach to specify new concepts about new or existing relations and to specify any in ter action between new and existing relations. To see how new interactions can be added consider the following ex ample. Assume the following three sentences are given: "The pen is to the left of the pot. " (3. 3) "The pot contains the flower. " (3.4) "Is the pen to the left of the flower? " (3. 5) Using only the advice expressed by (3. 1) or (3. 2) it would not be pos sible to answer the question in (3. 5). However, if we could add the concept that "if A is to the left of B, and B contains C, then A is to the 29 left of C" to our set of inferential knowledge, the question could be answered. In the advice giving language of this paper this concept is expressed as the simple syntax rule, left-of -* left-of contains. The addition of this one rule makes possible the deduction necessary to answer the question in (3. 5). Some of the power and ease of use of this approach for advice giving can be demonstrated by a series of examples. The examples which follow are actual computer results from a program written in the early stages of this research. A complete description of this program is given in Lingard and Wilczynski [38]. In each of the examples the computer responses are preceeded by The rules providing the inferential advice in this program are given in the form, RULES R ^ i* ~ N, R£, R^, . . . , R ^» where R^ is the left side of the rule, R^,, R^, . . . , R ^ +^, are the r e lations on the right side, and N is their number. The symbol "#" attached to a relation indicates the converse relation. For example, subset# means superset, left-of# means right-of, and so on. For each rule input the program automatically generates the correspond ing rule for the converse relation. For example, if left-of -* left-of contains 30 is input then left-of# -♦ contains# left-of# (i. e . , right-of -+ contained-in right-of) is automatically generated. The first example, Figure 4, shows a dialogue between some user and the computer involving the subset and part-of relations (ab- \ breviated by S and P, respectively). The first three statements en tered by the user are advice to the program . They indicate the m an ner in which the relations subset and part-of interact with each other and thus give the necessary knowledge to the program to enable it to make certain inferences about statements involving those relations. In particular, by the first rule the program can determine that a BOY is a subset of a PERSON which is a subset of HUMAN. The steps of this deduction are printed along with the answer, YES. The program then answers SOMETIMES to the reverse question and r e plies that it knows nothing about ARM to the third. The last question, "IS A FINGER A PART OF JOHN? involves a slightly m ore com plicated deduction. The tree used to make the proper deduction is given in Figure 5. The next example, Figure 6, is a continuation of the same con versation. Now, when asked if an arm is part of a person, the com puter answers INSUFFICIENT INFORMATION. For although it now knows something about ARM, namely that a hand is part of an arm , it still lacks sufficient knowledge to make the necessary deduction. The 31 RULES < S > = 2, S, S/ RULES <P>= 2, P, P / RULES <P>= 2, P, S#/ EVERY BOY IS A PERSON. **** I UNDERSTAND A PERSON IS A HUMAN. **** i UNDERSTAND IS A BOY A HUMAN? **** BOY/S PERSON/S HUMAN > ! < > ! < YES IS A HUMAN A BOY? **** HUMAN/S# PERSON/S# BOY **** SOMETIMES JOHN IS A BOY. **** I UNDERSTAND A HAND IS PART OF A PERSON. **** i UNDERSTAND A FINGER IS PART OF A HAND. **** I UNDERSTAND IS AN ARM PART OF A PERSON? **** I KNOW NOTHING ABOUT ARM. **** INSUFFICIENT INFORMATION IS A FINGER A PART OF JOHN? **** FINGER/P HAND/P PERSON/S# BOY/S# JOHN **** y es Figure 4. A Conversation with the Computer 32 FINGER P HAND P S# JOHN PERSON S# BOY Figure 5. A Simple Deduction Tree 33 A HAND IS PART OF AN ARM. **** I UNDERSTAND IS AN ARM PART OF A PERSON? **** INSUFFICIENT INFORMATION AN INDEX-FINGER IS A FINGER. **** I UNDERSTAND IS AN INDEX-FINGER PART OF JOHN? **** INSUFFICIENT INFORMATION RULES < P > = 2, S, P / IS AN INDEX-FINGER PART OF JOHN? **** INDEX-FINGER/S FINGER/P HAND/P PERSON/S# BOY/S# JOHN **** YES Figure 6. The Introduction of a New Rule 34 program is also unable to answer the next question. The reasons for this seem less obvious. On close inspection, however, it becomes clear that a crucial fact about the interaction between the subset and part-of relations has been omitted, namely that a subset of a part-of something is also a part of that thing, or in the advice notation of this program , part-of -4 subset part-of. When this piece of information is supplied to the computer, the proper deduction can be made. The next example, Figure 7, is a conversation involving family relations (PAR, F, M, and G denote parent, father, mother, and grandfather, respectively) and dem onstrates the capability of the p ro gram to answer questions requiring m ore than just a simple yes or no. As has been previously indicated, it is easy to add new relations and specify their interactions with existing relations. In Figure 8, the brother-of (B) relation is introduced. The rules indicating its interactions with the father-of and m other-of relations are given and the conversation is continued. The last example, Figure 9, dem onstrates how the knowledge put into the computer can be used to resolve ambiguities. In this case, the knowledge is relative sizes of objects. The necessary rules are: CC SZ LT# SZ# (3. 6) CC S# CC (3. 7) 35 RULES < PAR > = 1, F / RULES < PAR > = 1, M/ RULES < G > = 2, F, PAR/ JOHN IS THE FATHER OF FRANK **** x UNDERSTAND FRANK IS THE FATHER OF MIKE. * * * * I UNDERSTAND JANE IS THE MOTHER OF MIKE. **** i UNDERSTAND GEORGE IS THE FATHER OF JANE * * * * I UNDERSTAND WHO IS THE GRANDFATHER OF MIKE? **** MIKE/F# FRANK/F# JOHN **** MIKE/M# JANE/F# GEORGE Figure 7. A Computer Conversation Involving Family Relations RULES < F > = 2, F, B/ RULES < M > = 2, M, B/ RULES < B > = 2, B, B/ MIKE IS THE BROTHER OF PETER. **** I UNDERSTAND PETER IS THE BROTHER OF FRED. **** I UNDERSTAND PETER IS THE BROTHER OF SALLY. **** I UNDERSTAND WHO ARE-.THE GRANDCHILDREN OF JOHN? **** JOHN/F FRANK/F MIKE **** JOHN/F FRANK/F MIKE/B PETER/B SALLY **** JOHN/F FRANK/F MIKE/B PETER/B FRED Figure 8. The Introduction of a New Relation RULES<CC>= 3, SZ, LT#, SZ#/ 37 RULES< CC > = 2, S#, CC/ RULES< CC > = 2, CC, S/ RULES< LT > = 2, LT, LT/ A PLAY-PEN IS A PEN. **** i UNDERSTAND A FOUNTAIN-PEN IS A PEN. **** I UNDERSTAND A BREAD-BOX IS A BOX. **** i UNDERSTAND A CARDBOARD-BOX IS A BOX **** I UNDERSTAND THE PLAY-PEN HAS SIZE 500. **** I UNDERSTAND THE FOUNTAIN-PEN HAS SIZE 10. **** i UNDERSTAND THE BREAD-BOX HAS SIZE 23. **** I UNDERSTAND THE CARDBOARD-BOX HAS SIZE 200. **** x UNDERSTAND 10 IS LESS THAN 23. **** I UNDERSTAND 23 IS LESS THAN 200. **** I UNDERSTAND 200 IS LESS THAN 500. **** I UNDERSTAND Figure 9. The Resolution of Ambiguities 38 THE PEN IS IN THE BOX. * * * * PEN/S# FOUNTAIN-PEN/SZ 10/LT 23/SZ# BREAD-BOX/S BOX **** PEN/S# FOUNTAIN-PEN/SZ 10/LT 23/LT 220/SZ# CARDBOARD-BOX/S BOX **** THE m e a n in g o f t h is s e n t e n c e m u s t CORRESPOND TO 1 OF THE 2 GIVEN CASES THE BOX IS IN THE PEN. **** BOX/S# CARDBOARD-BOX/SZ 200/LT 500/SZ# PLAY-PEN/S PEN **** BOX/S# BREAD-BOX/SZ 23/LT 200/LT 500/SZ# PLAY-PEN/S PEN **** THE MEANING OF THIS SENTENCE MUST CORRESPOND TO 1 OF THE 2 GIVEN CASES Figure 9. The Resolution of Ambiguities (continued) 39 CC -* CC S (3.8) LT -» LT LT (3.9) where CC means can-contain SZ means has-size LT means less-than S means subset Rule (3. 6) can be interpreted as follows: A can contain B if A has size Xj which is greater than X2 which is the size of B. Notice that since the program has no knowledge of the behavior of numbers, it m ust be told their relative ordering. Now, when given the sentence, the program correctly deduces that the pen m ust be a fountain pen. Conversely, when the sentence, is input, it "knows" pen m ust be a play-pen. Figure 10, shows the tree generated in deducing that a (fountain) pen can be contained in a (cardboard) box. Although the power of this approach is im pressive, there are many limitations to this initial version of the system. F irst, the rules of inference are limited to simple compositional rules. Deduc tions requiring the intersection operator, such as the rule, THE PEN IS IN THE BOX THE BOX IS IN THE PEN 40 CC# CC# BOX CC# PEN SZ# FOUNTAIN-PEN CARDBOARD-BOX SZ LT 23 200 10 LT LT Figure 10. A Complex Deduction Tree 41 If X is perpendicular-to Z, and Y is perpendicular-to Z, and X is coplanar with Y, then X is parallel-to Y. cannot be expressed. Likewise there is no way to state a rule such as, To fly is to travel by plane. Another limitation is the inability to handle sentences involving any thing other than binary relations. In the sentence, JOHN GAVE THE BOOK TO MARY. the relation "gave" relates the three item s, "John, " "book, " and "Mary. " This system can accept only that factual knowledge which can be expressed in the form of simple sentences having an unquali fied subject and object separated by a simple relation. The rem ainder of this dissertation describes the research which has taken place toward overcoming the limitations of the original p ro gram. The result has been the development of a system based on the rule structure described above but providing the advice giver with far greater power of expression and permitting m ore complex forms of factual knowledge to be input. CHAPTER 4 REPRESENTATION OF KNOWLEDGE Although the emphasis in this research is the development of a language for giving inferential advice, a great deal of attention has been devoted,to the internal representation of semantic information. It is probably wise at this point to examine in some detail the needs for and purpose served by such an internal representation. Ernst and Newell [39] devoted considerable attention to the role of an inter nal representation in developing a general problem solver. They described a simplified problem solver as shown in Figure 11. The task of developing a natural language understanding system can be put easily in this framework. The external representation for the infor mation is natural language and the problem solving techniques are the set of computer program s which operate on some internal rep resen tation of the problem to enable questions to be answered about the in formation. Certainly one could consider the internal representation to be the same as the external representation. In this case no tra n s lator is needed, but the problem solving techniques required are much more complex. An internal representation is really just a conven ient means for dividing a complex problem into sm aller pieces. By so doing it is possible to concentrate on techniques for making infer ences without having to be concerned with parsing natural language. The subject of natural language cannot be avoided entirely, however. 42 43 External Representation Trans lator Internal R epre s entation Problem - Solving Techniques Solution Figure 11. A Simplified Problem Solver 44 The effectiveness by which a computer program can operate is a func tion of the usefulness of the internal representation, but the extent to which it can understand natural language depends upon how accurately natural language is represented. Of couse, if natural language can be automatically translated into the internal representation, there is no question. Without the existence of such a translator, however, one m ust be convinced that for the internal representation chosen such a translator is possible. This makes the selection of the internal representation for nat ural language extremely critical. Not only m ust it be close to natural language, but in order for inferences to be effectively made, the in ternal representation m ust capture much m ore than just the syntactic structure of each of the individual sentences comprising the given set of knowledge. It is not sufficient to know for each sentence what the verb phrase is, what words are adjectives, adverbs, and the like. It is necessary to know what function each word and phrase of a sentence serves. That is, the semantic relationships which exist between the various noun phrases and other portions of the sentence m ust be known. An approach for analyzing sentences according to these ideas is suggested by the work of Fillm ore [40], Each noun phrase is labelled with the case relationship which holds between it and the verb of the sentence. The cases represent certain types of judgements that can be made about the events the sentences describe. These judg ments are of the type — "who did it, " "what was affected, " "who it happened to, " "how it was accomplished, " when it happened, " and the 45 like. T h e r e a r e s e v e r a l a d v a n t a g e s t o t h i s a p p r o a c h . F i r s t , s e n t e n c e s h a v i n g t h e s a m e o r a p p r o x i m a t e l y t h e s a m e m e a n i n g w i l l t e n d t o e n d u p w i t h t h e s a m e i n t e r n a l r e p r e s e n t a t i o n . F o r e x a m p l e , c o n s i d e r t h e f o l l o w i n g s e n t e n c e s . John hit the ball. (4. 1) The ball was hit by John. (4. 2) The ball was hit with the bat. (4. 3) The bat made contact with the ball. (4. 4) Even though John is the subject of sentence (4. 1) and the object of the preposition "by" in sentence (4. 2), John is the "who" that did it in both. Likewise, in all four examples ball is the "what" that was af fected, and in the last two sentences the bat is suggestive of "how" it was accomplished. Some might argue that there can be subtle differences in m ean ing between even the active and passive forms of a sentence, and hence, any method of analysis that ignores these differences may not always accurately represent the meaning of a sentence. But to make inferences on the data base, at some point attempts must be made to determine equivalences between sentences or sets of sentences. By parsing a sentence using the notions of a case gram m ar, one is m e re ly deciding to do part of the task before forming an internal structure of the data. If nothing else, this manner of operating seems to con 46 form with the way in which humans function. Human beings can se l dom repeat word for word anything they hear. Rather, they are cap able of giving only some account which approximates the original v e r sion. Obviously, what has happened is that they have analyzed the sentences as they heard them and have recorded only some rep resen tation of the information. F rom this they are able to produce a p a ra phrase of what they have been told. It is not the intent here to justify case gram m ar notions by claiming that human beings analyze senten ces in this way but only to justify the use of an internal representation which may not always completely capture all the details and subtleties in meaning of the sentences being analyzed. Another advantage in using a case gram m ar analysis is that it makes it easier to combine sentences describing a single event into a coherent piece within the internal semantic structure making up the knowledge base. F or example, the sentences, John threw the ball. (4. 5) He threw it to Sally. (4. 6) He threw it with his left hand. (4. 7) obviously all refer to the same event, and are equivalent, at least approximately, to the single sentence, John threw the ball to Sally with his left hand. (4. 8) In the analysis of sentence (4. 5), "John" would be labelled as the agent, "threw" as the verb, and "ball" as the object. Sentence (4. 6) 47 would m erely add "Sally" as the dative to the existing structure for the event. Similarly, sentence (4. 7) would cause the addition of "left hand of John" as the instrument. This is all possible since each case relationship occurs only once, except for noun phrase conjunction, in a simple sentence. Hence, whenever the cases in common between two sentences match, it is in general true that they represent a single event. A third reason this case structure analysis seems particularly useful in a question answering environment is that by its very nature it anticipates questions that can be asked about events. The cases actually correspond to the answers to these questions. The agentive case answers the question, "who did it? " The dative case answ ers, "to whom was it done? " The others follow sim ilarly. The number of cases one might wish to define is, in part, a function of the number of questions that need to be answered about the events. Although no attempt will be made to exhaustively cover all lin guistic issues, the sections that follow are presented as an indication of how this approach toward an internal representation can handle some of the m ore important features of natural language. The Form of the Internal Representation A unique feature of this system is that there is no fixed form of representation. For as much as has been possible, the structuring of semantic information within the computer has been left up to the user. F or the examples presented in this paper, a representation 48 based on the case gram m ar notions of Fillm ore [40] has been chosen. The basic elements in the internal representation are called event tuples. Every phrase or set of phrases representing a complete idea, event, thought, state, or condition forms a tuple. As has been d is cussed, Fillm ore analyzes a simple sentence by assigning to each noun phrase a unique case relationship with the verb. Consider the following sentence, J o h n t h r e w t h e b a l l t o M a r y . According to Fillm ore, "John" would be of the agentive case (A), "the ball" would be of the objective case (O), and "Mary" would be of the dative case (D). If the verb "threw" is identified by R (for relation), this sentence can be represented by the following event tuple. (A-John, R-threw, O-ball, D-Mary) The number and types of cases actually used are completely arb itrary as far as the program is concerned. The only assumptions are that 1) it is possible to parse natural language sentences into the case structure used, 2) each tuple contains exactly one verb or relation and one or m ore associated cases, and 3) except in the case of compound phrases, cases within a given tuple are unique. Of couse, a noun phrase might itself be an event tuple having its own relation and cases. (Compound sentences are split into separate event tuples. ) 49 A slightly m ore complicated event is represented by the following sentence, The girl who was carrying the sm all dog kicked the old man. Here, "the small dog" could be represented as if it were "the dog was small, " that is, (A-dog, R-was, O-small). Similarly, "the old man" could be represented as (A-man, R-was, O-old). The whole sentence could then be represented by (A-(A-girl, R-was carrying, 0-(A-dog, R-was, O-small)), R-kicked, 0-(A -m an, R-was, O-old)). For the examples used in this paper a case structure slightly differ ent from that of Fillm ore has been chosen. More precisely it is an expansion of F illm ore's cases. The prim ary cases to be used in this discussion are as follows: 1) Agentive (A) — This is the who of the event, the usually animate perceived instigator of the action. 2) Instrumental (I) — This is the how of the event in the sense of an inanimate force or object causally involved in the action. 50 3) Dative (D) — This is the whom that something was done to, that is, the animate being affected by the state or action. 4) Locative (L) — This is the where of the event, the loca tion or spatial orientation of the state or action. 5) Objective (O) — This is the what of the event, the object or being affected by the action or state. 6) Reasonitive (Y) — This is the why of the event, the reason or purpose for the action. Additional cases will be added to this list as they are needed in the following sections. Before pursuing further the internal structure for semantic in formation, a slightly different view of this semantic analysis of "events" will be presented. It turns out to be useful to consider each event to be composed of one operator and a set of one or m ore oper ands. The operator specifies a relation or condition on the operands and is strongly related to the verb occurring in the event, and there is one operand for every case present. In the event described by the sentence, John threw the ball to Mary. the operator is the verb "throw, " and the operands are "John, " "ball," and "Mary. " The operands are related to each other by the action of throwing. In the case of a sentence like 51 John loves Mary. the operator is "love." "John" and "Mary" are operands related by the state of loving. In a sentence of the type, The house is red. ther operator is "red, " or maybe m ore accurately the condition or state of being red, and there is just one operand, "house. " This turns out to be much m ore meaningful than letting "is" be the opera tor and having both "red" and "house" be operands. The reason for this will become clear when the inference mechanism is discussed. Consider the sentence, Dogs are m am m als. In this case the operator is the subset relation and the operands are "dogs" and "mam mals. " Finally consider the sentence, John is the father of Bill. The operator here is the relation of "being father to" and the operands are "John" and "Bill. " However, this sentence implies another event, namely one having as the operator the subset relation and "John" and "father" as the operands. Possibly the p arser in a complete system would generate both of these events from the sentence given. If not, 52 it might be possible, as will be seen later, to provide a rule of infer ence to allow deduction of the second event from the first. The Knowledge Net Probably the key to intelligent behavior in either man or m a chine lies in the organization of the known information. Bits and pieces of information and isolated events m ust be organized into a co herent logical structure. Not only must all words and phrases a sso ciated with a particular event be connected, but also it m ust be pos sible, from a phrase or a single word, to obtain all pieces of infor mation of which it is a part. The word "ball'1 m ust not only be con nected to "John, " "threw, " and "Mary" as the object of an event, but it m ust be possible to reach, from "ball, " other events in which the word is used. An intelligent system m ust be able to associate a giv en event with other events involving the same words, phrases, or concepts. F urtherm ore, it m ust be capable of answering questions about concepts implied by the events or states existing in the data base. F or example, one might ask the system a question of the type, What can be done with a ball? Assuming the answer was not explicitly contained in the data base, events involving the word "ball" would have to be examined before a response could be given. Thus, there m ust exist a path in the data base from the word "ball" to each event containing it. These consid erations have dictated the structuring of the knowledge net used in this research. 53 Each distinct word or event occurs only once in the network of information. There is a pointer from that word to every event con taining it. Furtherm ore, the pointer is labelled with the case re la tionship which holds between the word and the verb for each event. Events themselves are organized sim ilarly. One event may be an element of several other events and would have appropriate pointers in the knowledge net. To illustrate this organization a conceptual diagram (Figure 12) is provided. The sentences (events) that went into constructing the net are as follows: 1) John hit the ball. (A-John, R-hit, O-ball) 2) John flew to Los Angeles. (A-John, R-flew, L-Los Angeles) 3) The girl who lived in Los Angeles flew to New York. (A-(A-girl, R-lived, L-Los Angeles), R-flew, L-New York) 4) Bill threw the red ball to P eter. (A-Bill, R-threw, 0 -(R -red , O-ball), D -Peter) 5) P eter loves tall girls. (A -Peter, R-loves, 0 -(R -tall, O-girls)) 6) Sally is a tall girl. (A-Sally, R-subset, 0-(R -tall, O-girl)) Time and Tense In addition to the cases described previously, there is a time 54 Nl J o h n ) hit ball red New Y o r k flew N5 N2 N4 Bill N6 lived N3 threw] loves girl P e t e r N7 N8 tall N9 Sail Subse Figure 12. A Knowledge Net 55 element in each tuple. This item is designated by the letter T and in dicates when the event took place or will take place. In many cases it simply indicates the tense of the verb. If all that is known is that the event took place sometime in the past, a P is in the T element of the event tuple. If the event is in the present tense the date and time at which the event is entered is placed in the T element. Future events are indicated by F. In the sentence, "John threw the ball to Mary, " the event tuple would be (A-John, R-threw, O-ball, D-Mary, T-P). If the sentence were, "John will throw the ball to Mary, " it would be (A-John, R-throw, O-bail, D-Mary, T-F), the only difference in the two events being in the T element. Now in the case of the sentence, "John loves Mary, " where the tense is the present, the event tuple would be (A-John, R-love, D-Mary, T-1974. 165. 7. 32. 10), where the number in the T element is the year, julian date, hour, minute, and second at which the event was entered into the data base. This means that at this time it is known that this event is true. More will be said about how this information is used when the inferencing mechanism is discussed. 56 If the date today were 1974. 165 and the sentence, "John gave the book to Mary yesterday, 1 1 was input, one day would be subtracted from the current date and the event tuple, (A-John, R-give, O-book, D-Mary, T-1974. 164), would be stored in the knowledge net. Similarly if the input sentence were, "John will given the book to Mary tomorrow, " the event tuple stored would be, (A-John, R-give, O-book, D-Mary, T-1974. 166). Of course whenever an actual date and/or time is given for an event it is used directly. It is also often the case that an event occurs over a period of time. To handle this situation another tuple element has been added. This is the completing time of the event and is indicated by the letters TE. "John watched television from seven o'clock to nine o'clock" is an example of such a case. The event tuple corresponding to this sen tence would be, (A-John, R-watch, O-television, T-1974. 165. 7, TE-1974. 165. 9). If the TE element is not present it is assumed that either the event occurred instantaneously or that the completion time is unknown. For example, in the case of "John loves Mary, " (A-John, R-love, D-Mary, T-1974. 165. 7. 45. 26), 57 the time assum ed for the T element is the time the event is entered into the data base. But there is no way of knowing, from this one sen tence, how long this condition rem ains to be true, that is, how long the event lasts. The problems this situation creates in making infer ences will be discussed later. As is true for the other elements of the event tuples, multiple tim es can exist within a single event. For example, the sentence "John went to the store yesterday and today" can be represented by the event tuple, (A-John, R-go, L -sto re, Tl-1974. 164, T2-1974. 165). Context Mechanism One of the m ost difficult problems any language understanding system must reckon with is that of handling, in some reasonable way, large volumes of data. No system can be of any practical value unless it can be used on large data bases effectively. But as a data base grows in size, the problem of combinatorial explosion arises. Most system s developed to date have operated only on small data bases. Those that have not have handled only a sm all subset of the problems associated with understanding natural language. Techniques are needed to facilitate intelligent interrogation of a data base rather than relying on exhaustive search. The data m ust be organized in a logical structure rather than stored randomly. Words, phrases, and ideas which are logically associated m ust be stored in the same logical sub 58 set of the data. Efforts have been made in this research to provide a means for such a structuring of information. In the system described here, a m echanism has been developed which enables a logical context or set of contexts to be specified for each event tuple in the data base. This perm its organization of the stored information into logical subsets. These subsets may be nes ted, disjoint or even intersecting. Such a structure is depicted by the example in Figure 13, where "American Presidents" is a subset of the "History of the United States, " which is a subset of the "History of - the World. " The "History of the United States" is disjoint from the "Early Roman Em pire, " but "Great Battles in History" intersects with both. Each piece of information stored in such a data base would be flagged according to the context to which it belonged. (Obviously, it would not be necessary to indicate that information about George Washington was part of United States or World History, since the con text of A m erican Presidents lies properly in both. ) This type of organization provides three important advantages to a system for understanding semantic information. 1) The first and m ost important advantage is that it helps to lim it and direct searches in the data base. If some con text is known, or can be inferred, regarding the infor mation sought only that context and subsets containing it need to be searched. If the information wanted is not in the context directly, nested chains of contexts can be examined in order of expanding size, thus giving a 59 History of the United States A m e r i c a n residents The History of the World Great Battles in History The Early Roman Empire Figure 13. The Organization of Information reasonable direction to the search. 2) The second advantage is closely related to the first. In any computer system dealing with large volumes of data only a small portion can be kept in mem ory at any one time. The rest m ust reside on some secondary storage device. In order to efficiently access information it is essential that it be brought in and out of memory in blocks of data which are likely to be used together. Thus, the system described here for logically organizing information suggests an effici ent physical organization as well. 3) The third advantage represents alm ost the opposite use of the context mechanism. Instead of needing a context to help locate information, it might be the case that the information has been easily found but is ambiguous. Here, context might be used to decide on the proper meaning by inferring which context is m ost likely. In the present system the context for information must be given explicitly. This is done by stating a context name or list. All infor mation following is identified with that context or list of contexts un til a new context is specified. By way of example suppose one first specifies as a context — "Baseball. " If then the sentences, J o h n i s a b a s e b a l l p l a y e r . John hit the baseball. 61 Bill caught the baseball. J o h n w e n t t o t h e d u g o u t . were input, a structure like that of Figure 14 would be formed. (For simplicity the time element has been omitted, although it would c e r tainly be of great significance in understanding these events. ) Now if the context were to be changed to "crim e" and the follow ing sentences were input, J o h n r o b b e d a b a n k . J o h n w e n t t o j a i l . the structure would be modified as in Figure 15. Suppose the question "where did John go? " were to be asked. If it were known that the discussion pertained to baseball, only that p o r tion of the data base pertaining to baseball would need to be examined. If on the other hand the context was not known, the data base would be searched for "John" and all the places he went. In this case there are two possibilities. If there is some way, based on previous dialogue, for example, to determine which context is m ore likely, crim e or baseball, the ambiguity can be resolved. In order to be able to use context in this manner, however, the logical organization of the data m ust be inverted. Instead of events being stored within contexts, the contexts relevant to a given event must be stored within the event. The advantages of the first organiza tion scheme have already been explained. It has also been shown how 62 El John play baseball go dugout E4 E3 E2 Bill catch] Figure 14. Events in a Baseball Context 63 E5 bank rob E6 E l jail John play baseball go dugout hit E4 E2 E3 Bill catch] Figure 15. A Knowledge Net Containing Two Contexts 64 the second scheme can be used to disambiguate possible meanings, but it is essential for yet another reason. If information is sought and no context is known it is unrealistic to suppose that searching through each context one by one can be an effective way to proceed. If, however, that subset of the data base having to do with the desired information can be accessed, as is possible under the second method of organization, efficient search is still possible. Since there are numerous examples indicating why each of these organizations is important, both are used in the present system. This necessitates, of course, the double storing of information. However, the extra storage space should be m ore than made up for by the reduction in search time. Negation Associated with each event is a true/false element. This allows negation of the proposition expressed by the event. If this element is not present the event is assumed to be true. This element is desig nated by N and can have either the value T (true) or F (false). For example, the sentence, "Boys are not adults, " can be represented by the tuple, (A-boys, R-subset, O-adults, N-F). Similarly the sentence, "John did not fly to New York by plane" has the tuple representation, (A-John, R-fly, T-P, L-New York, I-plane, N-F). 65 This negative element will be discussed further in the next section. Quantification The analysis of quanitfication in natural languages is extremely difficult. F or a sentence of the type, Children like toys. at least four meanings are possible, namely, All children like all toys. All children like some toys. Some children like all toys. Some children like some toys. In the present work problems associated with determining the proper quantifiers from im precisely stated sentences have been ignored. It will be assum ed that all necessary quanitifiers have been explicitly given. F or reasons that will become clear when the inference m ech anism is discussed, omitted quantifiers will be considered to be ex istential. That is "children like toys" is taken to mean only that "some children like some toys. " Since this is somewhat counter in tuitive care m ust be taken when using the system to be sure ail ne cessary universal quantifiers are explicitly stated. Another difficulty occurs under mixed quantification which causes an apparent difference in meaning between active and passive 66 forms of the same sentence. Consider the following statem ents. Everybody in the room speaks two languages. Two languages are spoken by everybody in the room. The first sentence seems to imply that everyone in the room is b i lingual while in the second it appears that everyone speaks the same two languages. To admit that these sentences are really different weakens the case for considering semantic structure to be indepen dent of gram m atical order. However, as Leech [41] points out, both of these sentences are really ambiguous. • That is, it is possible to read either meaning for either sentence. He states, "even though the choice of active or passive appears to predispose the interpreter to one meaning rather than another, active and passive sentences under mixed quantification, as elsewhere, have the same choice of meaning. " Furtherm ore, other factors, such as stress and intona tion, seem to be much m ore important in the selection of an in terp re tation than does the active or passive nature of the sentence. One way to handle quantifiers would be to insert them as extra elements in the given event tuple labelled according to the case to which they are related. For example, "all children like some toys" might have a tuple counterpart of the form, (AQ-all, A-children, R-like, OQ-some, O-toys), where AQ identifies the agentive quanifier and OQ the objective quan tifier. Unfortunately this approach turns out to be inadequate for two 67 reasons. F irs t it fails to account for the ambiguity discussed above and secondly it is incompatible with the way in which negation has been defined. Unlike m ost events, quantified events can be negated in two different ways. The assertion "all children like toys" has two nega tives, namely, "all children do not like toys" (meaning "no child likes toys") and "not all children like toys" (meaning "some children do not like toys"). In order to properly account for ambiguity and negation in quantified expressions it is necessary to represent such statements as complex events. In fact, for reasons which facilitate inference it becomes useful to consider quantifiers to be relations. For example, the statement, All children like toys. has the tuple representation, (A-(R-all, O-children), R-like, O-toys). The two negative forms of this event are, Not all children like toys. (Some children do not like toys. ) (A-(R-all, O-children), N-F), R-like, O-toys), and All children do not like toys. (No child likes toys. ) (A-(R-all, O-children), R-like, O-toys, N-F). The ambiguous sentence, "all people in the room speak two languages! has three possible representations, 68 (A-(R-all, O-people), L-room , R-speak, 0-(R-tw o, O-languages)) (R-all, A-people, L-room , 0-(R -speak, 0-(R-two, O-languages))) (A-(A-(R-all, A-people), L-room , R-speak), R-two, O-languages) The first of these retains the two possible meanings expressed by the next two. CHAPTER 5 INFERENCE The previous chapter dealt with the representation of knowledge. Here, the use of that knowledge is treated. The fundamental question is how to effectively manipulate the represented facts in such a way that all the knowledge embodied by them may be inferred. Rather than writing program s to make particular inferences as was done by Raphael [22], Bobrow [23], and others, a general inference making procedure has been developed which utilizes a separate body of know ledge called rules of inference. These rules specify ways in which the stored information can be operated on to derive new knowledge. Rules of inference have the same form of representation as all other information and internally are only logically separated from other data. They consist mainly of definitions, information regarding r e lational interaction, and rules for syntactic restructuring. Internally they are identified as inferential information by their presence in a context called "rules of inference. " The program , in trying to verify facts or answer questions, considers knowledge so identified as advice. The general inference making m echanism draws upon this set of advice in order to make inferences. In this sense it operates in the spirit of the "Advice Taker" proposed by McCarthy [36]. That is, instead of having pro grammed procedures to instruct the machine as to how to make 69 _________________ 70 inferences, advice is given to the computer in the form of declarative statem ents. McCarthy has pointed out several advantages to this ap proach. For one thing declarative statem ents, unlike im peratives, can generally be made without knowledge of the current state of the system . The im perative, "place the red cylinder on the blue block, " makes sense only under certain conditions. In particular, the state m ent assum es the existence of the objects in question. The simple declarative, "the block is red, " or even the conditional statement, "if there is a block on the table, then it can be picked up, " require much less knowledge of the system 's current state. Another advan tage of declaratives is that the meanings of such statements are usu ally independent of order. This simplifies the task of adding to or modifying the set of inferential advice. Although a machine, when given a sequence of im peratives, can proceed quickly toward a solu tion without being concerned with previous knowledge, it is precisely the advantage of declarative statements that use can be made of exis ting knowledge. The use of declaratives sacrifices efficiency for some degree of intelligent behavior on the part of the program . In stead of having to tell the program explicitly each action it is to p e r form, the program is given a goal to fulfill and asked to use some "common sense, " that is, examine the existing set of knowledge, in order to decide what to do. The following example can probably best exhibit the value of such an approach. Suppose it is desired to use the computer to han dle a scheduling problem. Assume a schedule for a baseball league 71 were needed. The natural way to express the knowledge a program would need is by use of declaratives. The following are examples. The league has eight team s. Each team m ust play each other team ten tim es. Each team m ust have one day off each week. No team is to play the same team for m ore than four or less than two consecutive games. If a general program can operate on such knowledge and produce the desired result, much has been accomplished. Certainly expressing all the necessary information as declaratives ( i.e ., advice) is sim p le r than devising the necessary algorithm. This offers great poten tial for the effective use of computers by individuals with little or no knowledge of programm ing. In addition, the declarative statements are independent of order and can easily be modified. This gives an added degree of flexibility over writing procedures. The development of a program capable of dealing with such problems can be divided into two parts. The first is that of under standing the declarative information provided and the second is con structing a program to solve the problem. Although these pieces would interact heavily in a complete automatic programming system, the current research deals only with the first. The approach used in this research employs advice taking tech niques but the motivation differs slightly from M cCarthy's. He p ro posed the "Advice Taker" as a step toward developing a program 72 capable of learning from its experience. This is a necessary step as McCarthy points out, for "in order for a program to be capable of learning something it m ust be capable of being told it. " While neither the goal nor the approach can be argued against, advice taking was adopted in the present research because of m ore practical consider ations. In particular, this approach was chosen out of the firm b e lief that programming in all the inferential knowledge needed for un derstanding natural language is impossible within the current state of the art. Advice taking seem s to be the shortest path toward develop ing an effective language understanding system . It may turn out, as McCarthy has indicated, that in so doing a significant step has been taken toward developing an "intelligent" machine. Retrieval of Information Before discussing the advice giving language and the process of making inferences, an explanation of the basic information retrieval capabilities will be helpful. After a data base has been built by in putting a set of event tuples, questions may be asked regarding the information. Questions are stated in event tuple form and are fol lowed by a question m ark. F or example, (A-Mary, R-love, O-John)? asks, "does Mary love John? " If the event tuple, (A-Mary, R-love, O-John), had previously been input the program would respond "yes. " It is 73 also possible to ask, "who does Mary love? " This is done with the tuple, (A-Mary, R-love, 0-*X)? Here, the "*" indicates that this is an item to be found. The variable X is arb itrary and any string of characters can be used. In this case the program will respond with all (or up to some specified number) of the values of X for which there is such a tuple in the data base. F u r therm ore, even a question of the type "who loves whom? " can be asked. The event tuple for expressing this is (A-*X, R-love, 0-*Y)? This will cause the program to search for all pairs, X and Y, for which such a tuple exists. It is possible, of course, to have a very complex pattern to match and fill in. Consider the sentence, V Napoleon commanded the French arm y which lost the battle of Waterloo in 1815. Its tuple representation might be as follows: (A-Napoleon, R-command, 0-(A -(R -French, O-army), R-lose, 0-(R -battle, O-Waterloo), T-1815), T-P). Here one might ask, Who did Napoleon command? (A-Napoleon, R-command, 0-*X, T-P)? or one could even ask, • 74 Who commanded the French what which lost what in 1815? (A-*X, R-command, 0-(A -(R -French, 0-*Y), R-lose, 0 -* Z , T-1815), T-P)? In general, any tuple elements in an event can be sought. The search ing algorithm matches those elements explicitly given and finds values it can assign to those asked for. It is possible also to ask compound questions like Does M ary love John and does John love Mary? In tuple form this is (A-Mary, R-love, O-John) (A-John, R-love, O-Mary)? The two tuples w ritten together imply that they m ust be anded together. The answer will be "yes" only if both tuples are true. Even compound questions of the type Who does Mary love and John hate? (A-Mary, R-love, 0-*X) (A-John, R-hate, 0-*X)? can be answered. In this case the program will attem pt to find an X which satisfies both tuples. The answer to the question will be "yes" only if the same value of X is valid in both tuples. Much m ore will be said about this question answering capability when the inference m echanism is discussed. The Advice Giving Language One. of the prim ary goals of this research has been to develop a language which can be used to give advice to the program . Since 75 the power of this system to make inferences comes from the "rules of inference" or advice given to it by the user, a convenient and effec tive means of expressing such information was essential. As was seen earlier the somewhat sim plistic approach of expressing inferen tial knowledge as production rules in a gram m ar for natural language relations was certainly convenient and powerful. Unfortunately, that approach was greatly limited in the types of inferential advice that could be expressed. Only simple compositional rules involving b i nary relations could be handled. The advice giving language to be described now captures much of the simplicity and power of this o rig inal approach while adding significant new capabilities. These perm it a far greater range of expression for inferential knowledge. The m ost general way in which advice can be given is in the form of event tuples. F or this purpose two new tuple element classes have been created. They have been given the labels H and C and c o r respond to the hypothesis and conclusion clauses of a conditional statement, that is, a statement of the type "if H then C, " "H implies C, " "C if H, " or the like. The statem ent of advice, "to travel by plane is to fly, " which partially defines the verb "fly, " thus, has the following tuple re p re sentation. (C-(R-fly), R-if, H -(R-travel-I-plane)). (5. 1) The fact that if A is a subset of B which is a subset of C then A is a subset of C 76 can be represented as (C-(A-*A, R-subset, 0-*C), R-if, H-(A-*A, R-subset, 0-*B) (A-*B, R-subset, 0-*C)). (5.2) It is clear that any of the simple production rules that were used in the original approach can be expressed in this form. Figure 16 lists both forms for many of these. The new representation has far great er capabilities of expression as the example (5. 1) above indicates. The advice givento>the program is thus a set of conditional statem ents. This has the desired feature that advice can be given in a form close to English. It m ust be pointed out, however, that English might not be the best way to give advice. Compare, for ex ample, statement (5. 2) with the following statement of the same rule. subset -* subset subset. (5. 3) Both of these express the sam e concept. The shorter form is c e r tainly easier to write and even seems clearer. For this reason this means of expressing advice has been retained in the present system. The program will automatically generate something of the form (5. 2) from (5.3). Other simplifications have also been added. The hypo thesis and conclusions labels can be omitted and the relation can be replaced by an arrow . For example, the rule (5. 1) becomes (R-fly) -* (R-travel, I-plane). (5. 4) A final simplification allows omission of the relation labels. This perm its rule (5. 4) to be rew ritten as fly travel (I-plane). (5. 5) 77 left-of -4 left-of left of (C-(A-*X, R-left-of, 0-*Y), R-if, H-((A-*X, R-left-of, 0-*Z) (A-^Z, R-left-of, 0-*Y)) left-of - 4 left-of contains (C-(A-*X, R-left-of, 0-*Y), R-if, H-((A-*X, R-left-of, 0-*Z) (A-*Z, R-contains, 0-*Y)) can-contain -» has-size greater-than is-size-o f (C-(A-*X, R-can-contain, 0-*Y), R-if, H-((A-*X, R -has-size, 0 -* Z l) (A-*Z1, R -greater-than, 0-*Z2) (A-*Z2, R -is-size-of, O-* Y)) can-contain can-contain ,subset (C-(A-*X, R-can-contain, 0-*Y), R-if, H-((A-*X, R-can-contain, Q-*Z) (A-*Z, R-subset, O-^Y)) less-than -+ less-than less-than (C-(A-*X, R -less-than, 0-*Y), R-if, H-((A-*Y, R -less-than, 0-*Z) (A-^Z, R -less-than, 0-*Y)) father -4 father brother (C-(A-*X, R-father, 0-*Y), R-if, H-((A-*X, R-father, Q-*Z) (A-*Z, R -brother, 0-*Y)) mother -♦ m other brother (C - (A - *X, R -m other, 0-*Y), R-if, H-((A-*X, R-m other, 0-*Z) (A-*Z, R-brother, 0-*Y)) brother -+ brother brother (C-(A-*X, R -brother, 0-*Y), R-if, H-((A-*X, R -brother, 0-*Z) (A-*Z, R -brother, 0-*Y)) Figure 16. Rules in General Form 78 This advice giving language provides a convenient means for expressing the dictionary definitions of relations. As an example, consider a definition for "command. " 1) to order or direct with authority. 2) to demand. 3) to be in control over. 4) to dominate by reason of location. 5) to deserve and get. 6) to have charge of and authority over. 7) to issue commands. 8) to occupy a dominating position. The rules of inference needed to express this are as follows: command -> order | direct (M-authority) (5.'6) command -♦ demand (5.7) command - ► control (5. 8) command -» dominate (W-location) (5.9) command deserve & get. (5.10) command R-(R-have, O-charge) & R-(R-have, O-authority) (5.11) command -* issue (O-command) (5. 12) command - + occupy (0-(R-dom inate, O-position)) (5. 13) Notice the use of the "or" operator in rule (5. 6). This rule is equi valent to the following two rules. command -* order command -» direct (M-authority) A new case, indicated by M, was added to express the manner in which something was done. Rule (5. 8) is straightforward. In rule 79 (5.9) the right side became simply "control" since in the represen ta tion scheme adopted there is no effective way to distinguish between "control" and "to be in control over. " In (5. 9) another new case was created for "the reason something was able to be done, " as distin guished from "the reason it was done. " The label for this new case is W. Rule (5. 10) required that two relations be anded together. The special "and" operator is required for this purpose to distinguish this from a complex product rule like (5. 3). This rule m eans, in effect, that a statement involving the relation "command" can be verified if it can be verified with "command" replaced with both "deserve" and "get. " In rule (5. 11) two relations have again been anded together. In this case, however, they are both complex relations. The relation label, R, m ust be explicitly stated when complex relations are in volved. The remaining two rules, (5. 12) and (5. 13), are relatively straightforward although (5. 13) does involve a deeper nesting. Similarly, consider a definition of the relation "fly. " 1) to move through the air or rise in the air. 2) to cause something to move through the air. 3) to manage an a i r plane. 4) to travel through the a ir in an airplane. It can be expressed by: fly -4 move (L-air) | rise (L-air) (5.14) fly (0-*X) -» cause (0-(0-*X , R-move, L-air)) (5.15) fly -+ manage (O-airplane) (5. 16) fly -4 travel (L -air, I-airplane). (5.17) 80 Notice that in rule (5. 15) an object is required before the rule can be applied. This rule causes both the relation and the object to be tra n s formed as indicated on the right side. The other rules are sim ilar to those discussed above. As a last example of a dictionary definition, consider the "grandfather" relation. It is defined as the father of one's father or mother. In the advice language that becomes grandfather - ► father father | father mother. Here, the relations written together, with no operator in between, indicate the complex product of relations. Complex product rules are always expanded by the program by adding agentive and objective cases as required. That is, a rule of the form r l -* r2 r3 always becomes (A-*X, R - r l, 0-*Y) -♦ (A-*X, R -r2, 0-*Z) (A-*Z, R -r3, 0-*Y). If cases other than the agentive and objective are required, they must be explicitly given. The advice language is used for much m ore than giving diction ary definitions of relations. As Winograd [34] has stated, "the real 'meaning' of a word or concept cannot be defined in simple dictionary term s, but involves its relationship to an entire vocabulary and structure of concepts. " Many of the implications and interactions of 81 relations are not always clear, if derivable at all, from their dic tionary definitions. Even in cases where an implication of a relation could be obtained through a combination of dictionary definitions, it might be advantageous to make such an implication explicit. For example, "to fly" implies "to travel" which implies "to go" accord ing to their dictionary definitions, but it might be desirable to have directly the implication expressed by the rule, go -» fly. As another example, consider the quantifier "many" as in the sen tence, Many children like toys. Certainly this implies that some children like toys, and therefore a rule like some -» many is needed so that if asked, Do some children like toys? an affirm ative response can be given when it is known that many children like toys. It is hard to imagine how the implication many = > some could be derived from any set of dictionary definitions. It is also necessary many times to specify interactions among relations. It was seen earlier that the set relations interacted with many other relations. Rules such as, 82 part-of -» subset part-of part-of ■ + part-of superset can-contain -» superset can-contain were used to express these interactions. If the subject m atter at hand were geometry one might wish to express the fact that two lines which are coplanar and perpendicular to the same line are parallel. The interaction between these three geometric relations could be ex pressed as follows: (A-*X, R -parallel, 0-*Y) -» (A-*X, R-perpendicular, 0-*Z) (A-*Y, R-perpendicular, 0-*Z) (A-*X, R-coplanar, 0-*Y). So far the types of advice discussed have related to the meanings of words and their relationships with other words. It is also possible to give advice to the program at another level, namely at the level of gram m atical structure. One of the m ajor problems encountered in trying to determine the equivalence between two statements lies in the fact that significantly different gram m atical structures can be used to express the same thought even when the semantic components are essentially the same; For example, consider the following pairs of sentences. The janitor opened the door with a key. (5. 18) The janitor used a key to open the door. (5. 19) The president made an official statem ent of his position. (5. 20) 83 The president stated his position officially. (5.21) John’s love for Mary will increase. (5. 22) John will love Mary increasingly. (5. 23) While it can be argued that there are at least subtle differences be tween the sentence pairs, it would certainly be hoped that a reason able program could infer the one from the other. That is, if (5. 18) were known, certainly an "intelligent" program would have to answer "yes" if it were asked whether (5. 19) was true. As was pointed out earlier, a case structure representation was adopted in order to avoid many of the problems associated with the wide range of gram m atical structure possible with natural language. But a case gram m ar alone, at least the one used here, is not suffi cient for handling all problems of this type. The case structure rep resentations for the sentences (5. 18) through (5. 23) given in Figure 17 dem onstrate this. It is not clear, however, that all such problems should be done away with. It might often be the case the sm all dif ferences in meaning between roughly equivalent statements should be preserved. As long as it is possible to make the desired inferences under the control of user supplied advice, it is possible to see such equivalences when necessary, and at other times answer questions requiring m ore precise information by limiting the inferential advice that the program can use. A rule of inference to make it possible to deduce sentence (5. 18) from (5. 19) would be 84 The janitor opened the door with a key. (A-janitor, R-open, O-door, I-key, T-P) The janitor used a key to open the door. (A-janitor, R-open, O-door, W-(A-janitor, R-use, O-key), T-P) The president made an official statem ent of his position. (A-president, R-make, 0-(R-official, 0-(R -(R -president, O-position), O-statement)), T-P) The president stated his position officially. (A-president, R -state, 0-(R -president, O-position), M-official, T-P) John's love for Mary will increase. (0-(R-John, O-love), D-Mary, R -increase, T-F) John will love Mary increasingly. (A-John, R-love, D-Mary, M -increase, T-F) Figure 17. Differences in Grammatical Structure between Roughly Equivalent Sentences 85 (W-(A-*X, R - * Y, 0-*Z)) -¥ (I-*Z). In order to deduce (5. 20) from (5. 21) a rule like (0-*X, M-*Y) (0-(R-*Y, 0-*X) (5.25) might be needed in addition to the rule, state (0-*X) -+ make (0-(R-*X, O-statement)), (5.26) defining "state. " Application of this latter rule would transform the internal representation of (5. 21) into (A-president, R-make, 0-(R -(R -president, O-position), O-statement), M-official, T-P). Then application of the structural transform ation rule (5. 25) would give (A-president, R-make, 0-(R-official, 0-(R -(R -president, O-position), O-statement)), T-P). Finally, the sentence (5. 22) would imply the sentence (5. 23) given the rule (R-*X, 0-(R-*Y , 0-*Z) ■ + (A-*Y, R-*Z, M-*X). (5.27) Care m ust be taken in using rules for altering gram m atical structure in order to avoid implications not intended. For example, suppose sentence (5. 19) had been instead, The janitor kicked the handle to open the door. and the program were asked, How did the janitor open the door? in the sense of what instrum ent he used, that is, 86 (A-janitor, R-open, O-door, I-*X, T-P)? Rule (5. 24) would allow the program to answer "with the handle" or (I-handle). Unfortunately, the correct answer in this case is prob ably the janitor's foot. Rule (5. 24) could be rewritten as use (W-(A-*X, R-*Y, 0-*Z))--* use (I-*Z), and thus re stric t it to event tuples with "use" as the relation. This would prevent the above incorrect deduction from occurring. Inference Mechanism The program operates as follows. When given an event tuple to verify or fill in, it first attempts to do it by direct look-up in the data base. If the information sought is not present, it uses the body of knowledge in the context labelled "rules of inference" as advice. The relation in the tuple under consideration is matched against the rules of inference, and the subset of the rules involving that relation is extracted. One of these is selected and the tuple is transform ed as indicated by the chosen rule. The new tuple, or possibly tuples, are then sought for in the data base. If this fails, a new rule is chosen and the process is repeated. When all rules have been ex hausted, then all possible consecutive applications of two rules are attempted and so on until the tuple is verified or the answer is found or until no m ore infreences are possible or some specified lim it to the depth of searching has been exceeded. Notice that this searching process is breadth first. This guarantees that the shortest deduction possible will be found first. There are two reasons why this is im- 87 portant. F irst, m ost deductions involving natural language relations tend to be short. Thus, checking possible deductions in order of their length would seem to be m ore efficient. The second reason is related to the "exception principle" as defined by Raphael [22]. That is, the design of the system allows for the presence of contradictory inform a tion. It is perfectly acceptable to tell the program that "mammals live on land, " that "whales are m am m als, " and that "whales live in w ater," What this means is that "mammals live on land, except for whales, w h i c h live in w ater. " These exceptions which contradict general rules can usually be deduced using fewer rules than can the general facts. Hence, a breadth first approach usually assures the discovery of an exception, when there is one, before a deduction involving a general rule is produced. Although the system can be used for simple interrogation of the data base, as was seem earlier, its real power lies in the ability to make inferences on the stored knowledge. While m ost inferencing is done based on advice supplied to the program , some capabilities for deduction are built-in. Many questions relating to time and tense fall into this category. Consider the following sentences and their tuple repre s entations. John has been employed. (5. 28) (D-John, R-employ, T-P) John was employed until yesterday. (D-John, R-employ, T-P, TE-1974. 146) (5.29) 88 John has been and will continue to be employed. (D-John, R-employ, T-P, TE-F) (5. 30) John was employed in 1942. (D-John, R-employ, T-1942) (5. 31) John was employed from 1942 to 1972. (D-John, R-employ, T-1942, TE-1972) (5. 32) John is employed (today). (D-John, R-employ, T-1974.165) (5. 33) John will be employed beginning today. (D-John, R-employ, T-1974. 165) (5. 34) John will be employed until 1976. (D-John, R-employ, T-F, TE-1976) (5. 35) John will be employed. (D-John, R-employ, T-F) (5. 36) A question involving time or tense can be answered by checking the interval specified in the question against the time intervals of the event tuples in the data base. For example the question, Was John (ever) employed? (D-John, R-employ, T-P)? can be answered in the affirm ative if any of (5. 28) through (5. 34) are known since the time interval specified in the question, namely any interval in the past, is contained in each of the time intervals for the events (5. 28) through (5. 34) (assuming the time the question is asked 89 is after 1974. 165). If the question, Was John employed in 1948? (D-John, R-employ, T-1948) were asked, the only event from the above list that could perm it an affirmative response would be (5. 32). The question, Is John employed now? (D-John, R-employ, T-1974.165)? could be answered "yes" if event (5. 33) were known. Notice that (5. 30) does not perm it the deduction since there is no way to tell whe ther the indefinite future completion time for the event has passed b e tween the time the event was entered and the question was asked. Similarly only event (5. 35) of this set could be used to answer the question, Will. John be employed? (D-John, R-employ, T-F)? A statem ent like (5. 36) is difficult to use since there is usually no way to determ ine, at some time subsequent to its entry into the data base, whether or not the event has occurred. Another built-in feature of the program is the context m echan ism . It was shown earlier how this mechanism can help to organize and subdivide the data base to lim it and direct searching, to provide effective data management, and to aid in the disambiguation of sen tences. An example of this latter use of the context mechanism is given in Figure 18. The presence of conflicting or ambiguous in- 90 CONTEXT Cl (A-BOX, R-CONTAIN, O-PEN) CONTEXT C2 (A-PEN, R-CONTAIN, O-BOX) (A-BOX, R-CONTAIN, O-ROCK) CONTEXT RULES CONTAIN -> CONTAIN CONTAIN CONTEXT Cl (A-BOX, R-CONTAIN, 0-*X)? **** X-PEN (A-PEN, R-CONTAIN, 0-*X)? **** INSUFFICIENT INFORMATION CONTEXT C2 (A-BOX, R-CONTAIN, 0-*X)? **** x = ROCK (A-PEN, R-CONTAIN, 0-*X)? X = BOX **** (A-PEN, R-CONTAIN, 0-*Q l) (A-*Q1, R-CONTAIN, 0-*X) **** (A-BOX, R-CONTAIN, 0-*X) **** X = ROCK Figure 18. Use of the Context Mechanism in Disambiguation 91 formation creates no problem as long as the proper context is speci fied for the question. The next three examples dem onstrate some of the m ore funda mental capabilities for supplying inferential advice to the program . In the first, depicted in Figure 19, a partial definition of the relation "hit" is given. The information that John dealt Tom a blow and the supplied rule allow the deduction that John hit Tom. The next ex ample, Figure 20, shows how the program can be advised of the im plications of relations. The rule given says that if X is the father of Y then X is a father. This allows the deduction that John is a father from the fact that John is Bill's father. Finally, Figure 21 shows how the "and" operator can be used in giving advice. The rule supplied allows the program to deduce that John commanded respect from knowing that he both deserved it and got it. Although these examples all seem trivial, it is interesting to note that they are beyond the deductive capabilities of many systems including the previous version of this program . This is not to sug gest that the approach taken in these examples is necessarily the best means toward the desired deductions. In the case exhibited in Figure 20, for example, the deduction might better have been done during the parsing phase. The goal of this research is not to determine how or when deductions should be made but rather to provide sufficient capabilities by which a user can express any desired inference rule. One of the m ore powerful capabilities of the system is the abil ity to specify rules for transform ing gram m atical structure. The ex- 92 CONTEXT RULES HIT -v DEAL(O-BLOW) CONTEXT U (A-JOHN, R-DEAL, O-BLOW, D-TOM, T-P) (A-JOHN, R-HIT, D-TOM, T-P)? **** (A-JOHN, R-DEAL, O-BLOW, D-TOM, T-P) **** y es Figure 19. Supplying a Definition to the Program 93 CONTEXT RULES SUBSET(0-FATHER) -» FATHER(0-*Y) CONTEXT U (A-JOHN, R-FATHER, O-BILL) (A-JOHN, R-SUBSET, O-FATHER)? **** (A-JOHN, R-FATHER, O-BILL) vi* ‘V T T 'O 't' T T * ! ' Y H .o Figure 20. Supplying an Implication of a Relation 94 CONTEXT RULES COMMAND -♦ DESERVE & GET CONTEXT U (A-JOHN, R-DESERVE, O-RESPECT, T-P) (A-JOHN, R-GET, O-RESPECT, T-P) (A-JOHN, R-COMMAND, O-RESPECT, T-P)? **** (A-JOHN, R-DESERVE, O-RESPECT, T-P) (A-JOHN, R-GET, O-RESPECT, T-P) **** (A-JOHN, R-GET, O-RESPECT, T-P) > ! " n - YES Figure 21. The Use of the "And" Operator 95 ample shown in Figure 22 dem onstrates the use of such a rule. An interseting application of rules of this type with regard to quantifica tion can be seen in Figure 23. These rules, while specifying stru c tural transform ations, effectively serve to define the universal and existential quantifiers "all" and "some. " In general, questions are answered in the affirm ative or they are not answered at all. That is, if a tuple can be verified in the data base, a "yes" answer is made and if it cannot, the response "insufficient information" is given. The latter response indicates only that based on the known information it was not possible to verify thetuple. An exception to this situation occurs if explicit information is available contradicting the tuple to be verified. That is, if a tuple with the opposite value in the negation element can be verified, a "no" response can be made. Figure 24 illustrates such a case. 96 (A-KNIGHT, R-SAVE, O-MAIDEN, . M-(A-KNIGHT, R-KILL, O-DRAGON, I-SWORD), T-P) CONTEXT RULES (I-*Z) - ► (M-(A-*W, R-*X, 0-*Y , I-*Z)) CONTEXT U (A-KNIGHT, R-SAVE, O-MAIDEN, I-SWORD, T-P)? **** (A-KNIGHT, R-SAVE, O-MAIDEN, M-(A-*Q1, R-*Q2, 0-*Q 3, I-SWORD), T-P) *** y e s (A-KNIGHT, R-SAVE, O-MAIDEN, M-*X, T-P)? **** X = (A-KNIGHT, R-KILL, O-DRAGON, I-SWORD) (A-KNIGHT, R-SAVE, O-MAIDEN, I-*X, T-P)? **** X = SWORD Figure 22. Specifying Grammatical Transformation. Rules 97 (A-(R-ALL, O-BOY), R-LIKE, O-BALL) (A-BALL, R-SUBSET, O-TOY) (A-JOHNNY, R-SUBSET, O-BOY) CONTEXT RULES (A-*X, R-*Y,- 0-(R -S0M E, 0-*Z)) - ► (A-*X, R-* Y, 0-*W) (A-*W, R-SUBSET, 0-*Z) (A-*X, R-*Y, 0-*Z) - ► (A-*X, R-SUBSET, 0-*W) (A-(R-ALL, 0-*W), R-*Y, 0-*Z) CONTEXT U (A-JOHNNY, R-LIKE, 0-(R-S0M E, O-TOY))? **** (A-JOHNNY, R-LIKE, 0-*Q l) (A-*Q1, R-SUBSET, O-TOY) **** (A-JOHNNY, R-SUBSET, 0-*Q2) (A-(R-ALL, 0-*Q2), R-LIKE, 0-*Q l) (A-*Q1, R-SUBSET, O-TOY) **** (A-(R-ALL, O-BOY), R-LIKE, 0-*Q l) (A-*Q1, R-SUBSET, O-TOY) **** (A-BALL, R-SUBSET, O-TOY) **** YES Figure 23. A Quantification Example 98 {A-JOHN, R-LIKE, 0-(R -A L L, 0-(R-TA LL, O-GIRL)), N- P) (A-JANE, R-SUBSET, 0-(R-BASKETBALL, O-PLAYER)) (R-TALL, 0-(R -A LL, 0-(R-BASKETBALL, O-PLAYER))) (A-JANE, R-SUBSET, O-GIRL) CONTEXT RULES (A-*X, R - * Y, 0-*Z) -> (A-*X, R-*Y, 0-(R -A LL, 0-*W)) (A-*Z, R-SUBSET, 0-*W) (A-*X, R-SUBSET, 0-(R-*Y , 0-*Z)) - 4 (A-*X, R-SUBSET, 0-*Z) (R-*Y, 0-*X) (R-*X, 0-*Y) ■ 4 (A-*Y, R-SUBSET, 0-*W) (R-*X, 0-(R -A LL, 0-*W)) CONTEXT U (A-JOHN, R-LIKE, O-JANE)? **** (A-JOHN, R-LIKE, 0-(R-A LL, 0-*Ql)) (A-JANE, R-SUBSET, 0-*Q l) **** (N-F) (A-JANE, R-SUBSET, 0-(R -TA LL, O-GIRL)) **** (N-F) (A-JANE, R-SUBSET, O-GIRL) (R-TALL, O-JANE) **** (N-F) (R-TALL, O-JANE) **** (N-F) (A-JANE, R-SUBSET, 0-*Q2) (R-TALL, 0-(R -A LL, 0-*Q2)) **** (N-F) (R-TALL, 0-(R -A LL, 0-(R-BASKETBALL, O-PLAYER))) 'i- “ i " t* ^ ■ J h ' | 'I' ^ ^ Figure 24. A Negation Example C H A P T E R 6 CONCLUSION The system, as it presently exists, deals successfully with many problems associated with language understanding. Its signifi cant features are as follows: 1. It is general. That is, it is domain independent. Both the rep resentation of information and the process of making inferences are independent of any particular area of discourse. 2. The information expressed by sentences of complex structure can be represented. Problem s associated with quantification, negation, and time and tense have been dealt with. Moreover, the internal representation is flexible enough to accommodate alm ost any degree of complexity and has been designed to allow the user certain control over the internal representation. For the m ost part, gram m atical cases or tuple elements can be user defined. 3. The internal representation for semantic information has been designed to perm it its organization into a hierarchical structure. Logical subsets of the information and the relationships among them can be defined by the user. The data base so organized can be m ore effectively used in making inferences as well as p ro vide the basis for efficient handling of large volumes of data on 100 the computer. 4. Finally, the system provides an inference mechanism which is driven by advice supplied by the user. This m echanism oper ates on information provided in a specially designed advice giv ing language to interpret the knowledge in the data base. Although the system effectively handles many language processing problem s, there are limitations to both the present implementation and the general approach. Some of these are typical of the difficul ties suffered by all Natural Language Processing efforts. One such problem is speed. Even though the system is quite fast when com pared to efforts utilizing m ore general artificial intelligence lan guages, such as QA4 [42], it is still too slow to be considered useful for m ost practical applications. The current slowness of the system is due to a variety of reasons, some easily remedied and others not. F irst, the program has been w ritten in a straightforward manner in a high level language (SNOBOL/SPITBOL). Since the real purpose of this research has been to investigate certain techniques for language understanding little attention has been given to issues related to com puter speed or efficiency. Reprogramming, with particular regard to the storage and retrieval of the data item s, could result in signifi cant savings of both computer resources and time. A second reason for the slowness of the system is that there is no built-in mechanism for handling synonyms. To substitute a synonym for another word presently requires a deductive inference. A m ore direct manner for 101 treating synonyms could greatly speed up many inferences. The third and m ost critical problem relates to a basic deficiency of the system. It has no really good searching mechanism. Although the context mechanism helps to lim it and direct searches, contexts are, at p re s ent, completely user supplied. A truly effective system, capable of making "intelligent" searches through the data would be able to logi cally organize the stored information automatically. Another deficiency of the system is that, in its present form, it deals only with a data base that can be kept within the com puter's memory. Although the present system does not contain the necessary coding, a great deal of attention has been given to problems asso ci ated with handling large data bases. The design of the data base, which doubly stores information, and the context mechanism which organizes it were developed with the efficient paging of data in mind. It should also be pointed out that the present system is cap able of only deductive inference. Most of the design of the system, however, is not so limited. Extensions are planned to allow for in ductive inference as well. Inductive rules would be written in a sim ilar m anner except a different replacem ent symbol ('-I> ' instead of ) would be used and two integers would be provided to indicate how to judge the truth of the inductive hypothesis. For example, one might consider a rule like (A - , R - live, L-*Y)-I > (A-*X, R-subset, 0 - * Z l) (A -^Z l, R-subset, 0 - * Z 2 ) (A-*Z2<9/10 >, R-live, L-*Y). 102 The meaning of this rule is sim ilar to the deductive case. Namely, the tuple, (A-*X, R-live, L-*Y), can be replaced by the three tuples on the right side of the rule. However, here it is not sufficient to find a single value of Z2 which makes the tuples true. As w ritten the rule states that nine distinct elements out of the first ten chosen as values for Z2 in the second tuple m ust be valid in the third. In other words, suppose the tuple (A-elephant, R-live, L-land)? were to be verified. The inference rule given would allow an affirm ative response if, for example, it could be verified that an elephant was a m ammal, and that nine of the first ten mam m als found in the data base lived on land. The program would, of course, attempt to verify the tuple deductively before using other inference rules and would point out whenever it has resorted to inductive reasoning. As with the earlier version, this system has no built-in m ech anism for comparing numeric quantities or performing calculations. As was seen in an early example, the relative ordering of numbers has to be explicitly given when needed. Furtherm ore, the system provides no direct way to do arithm etic. It is not difficult to think of cases where it might be necessary to perform some calculation in order to infer the truth of a statement. For example, suppose both the weight and volume of an object were known and the question were asked as to whether the object would float in water. The inference rule needed would be something like the following: 103 (A-*X, R-can-float, L-water) -4 (A-^X, R-has-density, 0-*Y) (A-*Y, R -is-le ss- than, O^Z) (A-water, R-has-density, 0-*Z). This says that an object whose density is less than water can float in it. Now, if the density of the object is not known directly, how can it be determined? It would be convenient to be able to invoke, from some rule, a routine to perform the needed calculation. For exam ple, suppose a rule like (A-*X, R-has-density, 0-*Y) -+ call density(*X, *Y) could be w ritten where the routine 'density' would calculate the den sity of X and assign that value to Y. The 'density' routine would know what param eters it needed, say the volume and weight of the object in question, and would obtain these from the knowledge net by re c u r sively invoking the question-answering program . If the density rou tine were unable to obtain the values it needed or for some other reason were unable to determ ine the density of the object, the the rule which invoked it would fail. Notice that the tuple, (A-* Y, R -is-less -than, 0-*Z), could be treated sim ilarly. In fact, a relation could, by this means, invoke any kind of a routine. Conceivably, inferring the truth of some statement could depend upon whether some complex procedure could be carried out. The methods of expressing and interpreting such procedures are beyond the scope of this research but are of fun 104 damental importance in the larger problem of developing a complete automatic programming system. Wilczynski [43] has presented one approach for the representation of such procedures. It is important to note that while the methods of inference are expressed in a non-procedural way in this research, procedural in formation could easily be incorporated, as outlined above. This sep aration between the inference m echanism and the mechanism for fol lowing procedures can simplify both development tasks without any loss in capabilities. Just as the procedural mechanism can be in voked as needed when performing inferences, it can likewise call upon the inference mechanism when it needs information from the knowledge base. Hence, this lack of procedural capability is actually an advantage rather than a disadvantage of the system. Two of the m ore disturbing deficiencies with the system relate to the more fundamental problems associated with language proces sing. F irst, it usually takes a great deal of information to be able to make inferences. Some seemingly simple deductions require that many inference rules and general facts be supplied, as evidenced by some of the examples in the previous chapter. What is m ore d is turbing, though, is that it is difficult to anticipate what information or rules of inference are needed without also anticipating what ques tions will be asked. The examples in this paper, and probably many other works, have been rigged in the sense that they have been de veloped by starting from the question and working backwards. The real test would be to let the questions be asked by someone other than the one providing the information to the system. How one de 105 term ines what supplemental information m ust be provided with a giv en data base to perm it its effective interrogation is a key to the u se fulness of the system and hence a question of fundamental importance. The second fundamental problem relates to the specific ap proach taken in this research. Although there seems to be great power in the advice giving language, some inference rules are very difficult to express. Whether this is just a deficiency of the user language or due to m ore fundamental problems associated with lan guage understanding (i.e ., there is no easy way to express such things) is unclear. What is clear is that this issue needs further study. In spite of these difficulties and the current prim itive state of the field, there are many potential applications for even such limited methods. Although the practical, impossibility of many applications is apparent, the works of Woods [44] and Martin [45] show how useful language processing system s can be developed for certain specific and limited applications. With continued research it should be pos sible to extend natural language processing capabilities to practical applications of even greater complexity. In summary, the present research has attacked the problem of representing knowledge within a computer and has given some insight into the process of making inferences on a knowledge base. But p e r haps its greatest significance has been in providing the framework and the basis on which further investigations into such issues can be conducted. In Dostert, et al [46] it is commented that "one of the 106 immediate tasks of Computational Linguistics is to make it possible for a research er untrained in Computer Science to communicate with the computer. " But in order to provide a general means for man- computer communication, much m ore m ust be understood about hu man communication and human language. Applicable research m ust be continued in Linguistics and Psychology in addition to Computer Science. This suggests that before a general language processing capability can be provided, an effective means of communication with the computer m ust be developed for those research ers working in the area of language understanding. This approach is effectively a boot strapping technique. By first developing a limited, somewhat "nat ural" language directed toward language processing problem s, it might be possible for research ers untrained in programm ing to work in concert with computer scientists to develop m ore general language processing system s. The language processing model presented in this research was developed with the goal of providing such a "natur al" means for the expression of linguistic and inferential knowledge. The advice giving capability developed provides not only a basis for the accomplishment of this initial objective but a direction of r e search for the ultimate goal of producing a complete Natural Lan guage Processing System. BIBLIOGRAPHY [1] Halpren, M. , "Foundations of the Case for Natural-Language Program m ing, 1 1 P roc. FJC C , 1966, pp. 639-649. [2] Sammet, J. , "The Use of English as a Programm ing Language,1 Communications of the ACM, vol. 9, no. 3 (March 1966), pp. 139-143. [3] Hill, I. , "Wouldn't It Be Nice If We Could Write Computer P rogram s in Ordinary English — Or Would It? , " The Computer Bulletin, vol. 16, no. 6 (June 1972), pp. 306-312T [4] Winograd, T. , "Breaking the Complexity B arrie r (again), " A rtificial Intelligence Project, Stanford University, July 1973. [5] Chomsky, N. , Topics in the Theory of Generative G ram m ar, Mouton, The Hague, 1966. [6] Chomsky, N. , Aspects of the Theory of Syntax, MIT P re ss, Cambridge, M assachusetts, 1965. [7] Wilkes, Y. , "An A rtificial Intelligence Approach to Machine Translation, " in Schank and Colby (eds. ), Computer Models of Thought and Language, W. H. Freem an & Company, San Francisco, 1973. [8] Montgomery, C. , "Linguistics and Information Science, " Journal of the ASIS, vol. 23, no. 3 (May-June 1972), pp. 195-219. [9] Colby,- K. , "Simulations of Belief Systems, " in Schank and Colby (eds. ),. Computer Models of Thought and Language, W. H. Freem an & Company, San Francisco, 1973. [10] Balzer, R. , "A Global View of Automatic Program m ing, " USC Information Sciences Institute, ISl/RR-73-9, Marina del Rey, California, February 1973. [11] Carlson, F . , "A Design P rocess for Computer Based Instruc tional Systems, " Ph.D . dissertation, University of Southern California, Los Angeles, California, 1970. [12] Carbonell, J. , "AI in CAI: An Artificial-Intelligence Approach to Com puter-Assisted Instruction, " IEEE Trans, on Man- 107 108 Machine Systems, vol. MMS-11, no. 4 (December 1970), p p . 1 9 0 - 2 0 2 . [13] Simmons, R. , "Linguistic Analysis of Constructed Student Responses, " in Holtzman (ed. ), Com puter-Assisted Instruction, Testing, and Guidance, H arper and Row, New York, 1970. [14] Wexler, J. , "A Generative Remedial and Query System for Teaching by Computer, " Ph.D. dissertation, University of Wisconsin, Madison, June 1970, pp. 181-190. [15] Green, P ., A. Wolf, C. Chomsky, and K. Laugherty, "BASEBALL: An Automatic Question Answerer, " in Feigen- baum and Feldm an (eds. ), Computers and Thought, McGraw- Hill, New York, 1963. [16] Lindsay, R. , "Inferential Memory as the Basis of Machines Which Understand Natural Language, " in Feigenbaum and Feldman (eds. ), Computers and Thought, McGraw-Hill, New York, 1963. [17] Weizenbaum, J . , "ELIZA, " Communications of the ACM, vol. 9, no. 1 (January 1966), pp. 36-45. [18] Woods, W. , "Transition Network G ram m ars for Natural Lan guage Analysis, " Comm^;ni£aL tions_cT_tlm_ACM, vol. 13, no. 10 (October 1970), pp. 591-'606. [19] Kay, M. , "The MIND System, " in Rustin (ed. ), Natural Lan guage P rocessing, Algorithmics P re ss, Inc. , New York, 1973. [20] Kaplan, R. , "A General Syntactic P rocessor, " in Rustin (ed. ), Natural Language Processing, Algorithmics P re ss, Inc., New York, 1973. [21] Kaplan, R. , "Augmented Transition Networks as Psychological Models of Sentence Comprehension, " A rtificial Intelligence, vol. 3, no. 2 (1972), pp. 77-100. [22] Raphael, B. , "SIR: A Computer P rogram for Semantic Infor mation Retrieval, " in Minsky (ed. ), Semantic Information P ro cessing, MIT P re ss, Cambridge, M assachusetts, 1968. [23] Bobrow, D. , "Natural Language Input for a Computer Problem Solving System, " in Minsky (ed. ), Semantic Information P ro cessing, MIT P re ss, Cambridge, M assachusetts, 1968. 109 [24] Black, F . , "A Deductive Question Answering System ," in Minsky (ed. ), Semantic Information P rocessing, MIT P ress, Cambridge, M assachusetts, 1968. [25] Shapiro, S. , and G. Woodmansee, "A Net Structure Based Relational Question Answerer: Description and Examples, " Proc. IJCAI, 1969, pp. 325-346. [26] Coles, S. , "An On-Line Question-Answering System with Natural Language and Pictorial Input, " Proc. ACM National Conference, 1968, pp. 157-167. [27] Darlington, J. , "Theorem Proving and Information Retrieval, " in M eitzer and Mitchie (eds. ), Machine Intelligence 4, Edin burgh University P re ss, Edinburgh, 1969. [28] Green, C ., and B. Raphael, "The Use of Theorem-Proving Techniques in Question-Answering Systems, " Proc. ACM National Conference, 1968, pp. 169-181. [29] Robinson, J. , "A Machine-Oriented Logic Based on the R eso lution Principle, " Journal of the ACM, vol. 12, no. 1 (January 1965), pp. 23-41. [30] Simmons, R. , "Natural Language Question-Answering Systems: 1969, " Communications of the ACM, vol. 13, no. 1 (January 1970), pp. 15-30. [31] Simmons, R. , J. Burger, and R. Schwarcz, "A Computational Model of Verbal Understanding, " P roc. FJCC, 1968, pp. 441- 456. [32] Schwarcz, R. , J. Burger, and R. Simmons, "A Deductive Question A nsw erer for Natural-Language Inference, " System Development Corporation, SP-3272, Santa Monica, California, November 1968. [33] Schank, R. , "The Fourteen Prim itive Actions and Their In ferences, " Stanford A rtificial Intelligence Laboratory, Memo AIM-183, M arch 1973. [34] Winograd, T. , "Procedures as a Representation for Data in a Computer P rogram for Understanding Natural Language, " Project MAC, TR-84, MIT, Cambridge, M assachusetts, Feb ruary 1971. [35] Hewitt, C ., "PLANNER: A Language for Proving Theorems in Robots, " Proc. IJCAI, 1969, pp. 295-301. 110 [36] McCarthy, J. , "Program s with Common Sense, " in Minsky (ed. ), Semantic Information P rocessing, MIT P ress, C am bridge^ M assachusetts, 1968. [37] Zobrist, A ., and F. Carlson, "An Advice-Taking Chess Com puter, " Scientifm_Am££i:cari, no. 228 (June 1973), pp. 92-105. [38] Lingard, R. , and D. Wilczynski, "A Syntax Directed Approach for Handling Natural Language Relations, " Proc. of the ACM National Conference, 1972, pp. 118-128. [39] E rnst, G. , and A. Newell, GPS: A Case Study in Generality Problem Solving, ACM Monograph Series, Academic P re ss, New York, 1969. [40] Fillm ore, C ., "The Case for C ase," in Bach, and Harms (eds.)j Universals in Linguistic Theory, Holt, Rinehart and Winston, Inc., New York, 1968. [41] Leech, G. , Towards a Semantic Description of English, Indiana University P re ss, Bloomington, Indiana, 1970. [42] Rulifson, J. , et al, "A Language for Writing Problem-Solving Program s, " P roc. IFIP Congress, 1971, pp. 201-205. [43] Wilczynski, D. , "A P rocess Elaboration Form alism for P ro gram Writing and Analysis, " unpublished Ph.D . dissertation, University of Southern California, Los Angeles, California, 1975. [44] Woods, W ., R. Kaplan, and B. Nash-Webber, "The Lunar Sciences Natural Language Information System: Final Report, " Report 2378, Bolt, Beranek and Newman, Inc. , Cambridge, M assachusetts, June 1972. [45] Martin, W. , "Interactive Design in Protosystem I, " Project MAC, Automatic Program m ing Group, Internal Memo No. 4, MIT, Cambridge, M assachusetts, August 1972. [46] Dostert, B. , et al, "Verb Semantics in a Relational Data Base System, " REL Project Report No. 6, AD-759345, California Institute of Technology, Pasadena, California, Janurary 1973. A P P E N D IX PROGRAM LOGIC DESCRIPTION The MAIN program , depicted in Figure 25, controls the entire process. It operates as follows. After performing the necessary in itialization (box 1), it reads a statement (box 2) and determ ines whe ther the statement is a question to be answered (box 3). If it is, the subprogram, VERIFY, is invoked (box 4). If the statem ent contains information to be added to the data base (box 5), then the subprogram, INSERT, is invoked (box 6). This information could be statements in event tuple form or it could be rules in the form allowed by the advice giving language. In the latter case the information is converted to tuple form before INSERT is invoked. Finally, the statem ent could be context information (box 7). In this case the current context indi cation within the program is updated (box 8). Any other kind of state ment is invalid and an e rro r comment is generated (box 9). If there are m ore statements to read (box 10), control is transferred back to box 2. The heart of this program is the VERIFY routine. VERIFY is called with three arguments — the entire statem ent to be verified, a substring of it to be considered immediately, and a pointer indicating where the substring begins within the whole statement. Initially, VERIFY is invoked by MAIN with the original statement as the first argument, the leftmost tuple as the second argument, and a 1 as the third. VERIFY always attempts to verify the tuple given in the sec- 111 1 112 Initialize Read a statem ent no Call VERIFY to answer question 'D ata baseN information no yes Call INSERT to add it to data base / Context ^ informatio no Write e rro r m essage — Illegal state ment Update current context list 10 ' More \ statements yes no Stop F i g u r e 25. O v e r v ie w of th e M A IN P r o g r a m o 113 ond argument. If it is successful, the verified tuple is replaced by its name and VERIFY is recursively invoked to verify the resulting tuple. If the second argument cannot be verified a failure return oc curs. The VERIFY subprogram is illustrated in Figure 26. It first checks to see if there is anything left to verify (box V. 1) and if not prints the answer (box V. 2) and returns with an indication of success. If there is m ore to verify, the relation in the tuple (of the second a r gument) is found (box V. 3). If the relation is itself a complex tuple as opposed to being a simple element (box V. 4), the VERIFY sub program is recursively invoked with this relation tuple as the second argument (box V. 18). The first argument remains the same and the third is appropriately adjusted. If the verification is successful (box V. 17), a return is generated. Otherwise, control transfers to box V. 12 to try to apply rules of inference. If the relation is a simple element then the tuple is scanned for another element (box V. 5). The program will use this element and the relation to search the data base. An attem pt is made to find an element that will help in searching, that is, one which is not apt to have many tuples containing it. This is done by considering the ele ment types in a reasonable order. The time element, for example, is a poor choice since many tuples will have a P, say, in this ele ment. An attempt is made also to find an element which is simple and not itself a complex tuple. If this is not possible (box V. 16) the VERIFY program is recursively invoked with the complex tuple as 114 V ^ M o re —/ to N/verify no yes V * 2 V. 3 Print answ er(s) Find relation of tuple Return Relation' com plex V. 5 Find another elem ent Call VERIFY with relation V. 6 Elem ent ^complex, yea. V. 16 Make lis t cf all tuples containing relation & elem ent Call VERIFY with elem ent -xi V, IZ S u c c e s s / - ^ ----- no L ist empty es no Return V. 9 Choose tuple and rem ove from lis t / D e p th \ lim it exceeded yes no V, 10 V- 13 Call MATCH to m atch against data base tuple F ailure return Increment depth by 1 V. 14 V . 1 Call TRANSFORM to apply rules of inference Success, yes V, 15, yes no Success Failure return Return F i g u r e 26. F lo w D i a g r a m of th e V E R IF Y S u b p r o g r a m 6 115 was done in the case of a complex relation tuple (box V. 16). If a simple element has been found, the data base is searched for all tuples, according to the current context limitations, which contain that element and the current relation (box V. 7). A list of these tuple names is formed. A check is made to see if the list is empty (box V. 8). It is is not, one is chosen and is removed from the list (box V. 9). Then subprogram MATCH is invoked to try to match the data base tuple with the input tuple (box V. 10). If the match is successful (box V. 11), the VERIFY subprogram returns with an indication of success. If the match fails control transfers back to box V. 8. When there are no m ore data base tuples to try a test is made to see if the lim it to the allowed depth of inference has been exceeded (box V. 12). The depth limit, DLMT, is initially set to one and is incremented only when all inferences involving applications of exactly that number of rules have been attempted. This feature insures the breadth first nature of the search. If the depth limit has been exceeded, a failure return occurs. Otherwise, the depth of the current inference is in creased by one (box V. 13), and the subprogram, TRANSFORM, is invoked to attempt to apply a rule of inference to the tuple to be v e ri fied (box V. 14). If the transform ation is successful (box Y. 15), the VERIFY routine returns with an indication of success. Otherwise, a failure return occurs. The subprogram, MATCH, attempts to match an input tuple against a tuple in the data base. It is illustrated in Figure 27. It first checks to see whether all elements of the input tuple have been 116 M. 1 yes no M. 2 M. 4 M. 5 M. 3 yes no yes Success no Failure return M. 6 Return no yes M. 7 Failure return Element type OK ' Tuple > elements . match j f All N elements matched Select a tuple element Remove elements from tuples Call VERIFY with revised tuple F i g u r e 27. F lo w D i a g r a m o f th e M A T C H S u b p r o g r a m 117 matched against the data base tuple (box M. 1). If so VERIFY is in voked with the tuple replaced by the name of the data base tuple (box M. 2). If VERIFY returns successfully (box M. 3), then so does MATCH. Otherwise, a failure return occurs. It is important to note that MATCH only checks to see if all elements of the input tuple oc cur in the data base tuple and not the converse. All appearances of extra elements in the data base tuple, with the exception of the nega tion element, N-F, are ignored. The negation element, if it is the only extra element, may allow a negative response to the question asked. If not all elements of the input tuple have been matched, MATCH selects one (box M. 4) and checks to see if the data base tuple has an element of the same type (box M. 5). If it does not a failure return occurs. Otherwise, the two elements are compared (box M. 6). If they match they are removed from their respective tuples (box M. 7) and control is passed back to box M. 1. If they don't match a failure return occurs. (It may be the case that these tuple elements are themselves complex tuples, in which case, MATCH is recursively called to compare them. ) TRANSFORM is the subprogram which finds and applies rules of inference to the tuple to be verified. This routine is illustrated in Figure 28. It first finds and makes a list of all applicable rules (box T. 1). It then checks to see if the list is empty (box T. 2). If it is, a test is made to see if the current depth of inference is one (box T. 3), and if so, the depth lim it is increased by one (box T. 4). Control is 118 T. 1 Make list of applicable rules T. 2 no T. 3 T. 6 C urrent' depth = 1 Select rule and remove it from list yes no T. 5 T. 7 / \ / Left \ ^side match" X. tuple / Increment depth lim it by 1 Decrement depth by 1 no yes T. 8 Failure return Replace tuple with right side of rule T. 9 Call VERIFY with resulting tuple T. X -X Su c c e s s no Failure return Return F i g u r e 28. F lo w D i a g r a m of th e T R A N S F O R M S u b p r o g r a m 119 then transferred back to box T. 1. If the depth is greater than one it is decreased by one (box T. 5) and a failure return occurs. This ef fectively unwinds the program after it has tried all applications of rules up to the current depth limit. When it has unwound all the way, the depth lim it is increased and the program proceeds again. If the list of rules is not empty, one is selected and removed from the list (box T. 6). The left side of the rule is compared with the tuple to be verified (box T. 7), and if it matches, the tuple is r e placed by the right side of the rule (box T. 8). In so doing, any v a ri able elements in the right sidejof the rule are replaced by the appro priate values, if any, from the original tuple. The VERIFY subpro gram is then invoked with the resulting tuple (box T. 9)* If VERIFY returns successfully then so does TRANSFORM. Otherwise, a fail ure return occurs. If the left side of the rule doesn't match the tuple, control returns to box T. 2. Finally, subprogram, INSERT, illustrated in Figure 29, is used to add elements to the data base. Given a statement in tuple form, it first finds the leftmost simple tuple (box I. 1). (A simple tuple is one which contains no other tuples. ) The data base is then searched to see if this tuple is already contained within the knowledge net (box I. 2). If it is not, a tuple name is created for it (box I. 3), and the tuple is stored in the applicable contexts (box 1.4). Next an element l of the tuple is selected and removed (box I. 5). A link, labelled with the elem ent's type, is then created in the data base between the ele ment and the tuple name (box I. 6). This is done for each applicable no yes I. 4 yes no More tuples yes no Return /M o re \ tuple .elements /AlreadyN. in data base1 Replace tuple with its name Store tuple in data base Find leftmost s imple tuple Create name for tuple Create link between element & tuple name Select and remove an element of the tuple F i g u r e 29. F lo w D i a g r a m o f th e IN S E R T S u b p r o g r a m 120 , 121 context. A test is made to see if.there are m ore tuple elements to store (box I. 7). If there are, control is given back to box I. 5. If no m ore elements exist, the tuple is replaced by its name in the original statem ent (box I. 8). If there are more tuples in the statem ent to be processed (box I. 9), control passes back to box I. 1. Otherwise, INSERT returns.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Analysis of information process for software development productivity

PDF

Determination Of Media Materials And Methods For Teaching English As A Second Language In Nigeria

PDF

An investigation of the effects of output variability and output bandwidth on user performance in an interactive computer system

PDF

Communication strategies in barricade-hostage confrontations: Theory, research, and police experience

PDF

A divide-and-conquer computer

PDF

A unified approach to the construction of correct concurrent programs

PDF

A process elaboration formalism for writing and analyzing programs

PDF

An extensible object-oriented framework for engineering design databases

PDF

An integrated systems approach for software project management

PDF

A learning-based object-oriented framework for conceptual database evolution

PDF

A study of the intonation patterns of black and Standard English speaking children in formal and informal situations

PDF

A federated architecture for database systems

PDF

A unified model and methodology for conceptual database design and evolution

PDF

A hierarchical knowledge based system for airplane recognition

PDF

An architecture for parallel processing of "sparse" data streams

PDF

An analysis of bit blitting and polygon clipping

PDF

Communications-efficient architectures for massively parallel processing

PDF

An algebraic view of protection and extendibility in abstract data types

PDF

A study of the factors affecting software testing performance and computer program reliability growth

PDF

A framework for dynamic load balancing and physical reorganization in parallel database systems

Asset Metadata

Creator Lingard, Robert Wesley (author)

Core Title A representation for semantic information within an inference-making computer program

Degree Doctor of Philosophy

Degree Program Computer Science

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Computer Science,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Zobrist, Albert F. (committee chair), Carlson, Roy F. (committee member), Knirk, Frederick G. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c17-764188

Unique identifier UC11347689

Identifier DP22714.pdf (filename),usctheses-c17-764188 (legacy record id)

Legacy Identifier DP22714.pdf

Dmrecord 764188

Document Type Dissertation

Rights Lingard, Robert Wesley

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA