University of Southern California Dissertations and Theses

Toward A Systems Analysis Approach To Engineering Education; A Heuristic Model For The Scheduling Of Subject Matter

(USC Thesis Other)

Toward A Systems Analysis Approach To Engineering Education; A Heuristic Model For The Scheduling Of Subject Matter

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content This dissertation has been microfilmed exactly as received 66-5497 TAFT, Martin Israel, 1930- TOWARD A SYSTEMS ANALYSIS APPROACH TO ENGINEERING EDUCATION; A HEURISTIC MODEL FOR THE SCHEDULING OF SUBJECT MATTER. University of Southern California, Ph.D., 1966 Engineering, general University Microfilms, Inc., Ann Arbor, Michigan C o p y rig h t by MARTIN ISRAEL TAFT 1966 TOWARD A SYSTEMS ANALYSIS APPROACH TO ENGINEERING EDUCATION; A HEURISTIC MODEL FOR THE SCHEDULING OF SUBJECT MATTER by Martin Israel Taft A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Engineering) January 1966 UNIVERSITY O F SOUTHERN CALIFORNIA T H E G R A D U A T E S C H O O L U N IV E R S IT Y PA R K LO S A N G E L E S , C A L IF O R N IA 9 0 0 0 7 This dissertation, written by ........................M ^ T I N I S R ^ L T A P I .......................... under ihe direction of h%§....Dissertation Com mittee, and approved by all its members, has been presented to and accepted by the Graduate School, in partial fulfillment of requirements for the degree of D O C T O R O F P H I L O S O P H Y Dean Date....... PLEASE NOTE: Figure pages are not original copy. They tend to "curl". Filmed in the best possible way. University Microfilms, Inc. DEDICATION This dissertation is dedicated to my family: to my wife, Ettie, who proffered much-needed encouragement at crucial moments, served patiently as a sounding board for ideas that ofttimes emerged as unintelligible technical jargon, and who selflessly carried most of the burden of child rearing, housekeeping, and even financial support during the nine long years of my academic hibernation in graduate school; to my daughter, Esther, who was born when I began my graduate work and would like to know what will happen "after Daddy gets his Ph. D. t o Noah, who told his teacher that his father "only studies for a living"; and to my father and mother who deserve to have the satisfaction of saying, "my son, the Dpctor. . . . " ACKNOWLEDGMENTS This work is the result of many years of study and interest in the problems of education and the persistent thought that significant contributions towards the solution of these problems can be derived from the application of certain engineering techniques. Encourage ment to attempt an interdisciplinary dissertation, and in particular, to apply systems analysis methods to educational institutions came from lengthy discussions with my friend and colleague, Professor Arnold Reisman. This attempt to apply systems analysis techniques to institu tions of higher learning was made possible by the farsightedness and cooperation of my dissertation committee and in particular by its chairman, Professor P. Roy Choudhury. I am deeply indebted to Professor Alan J. Rowe for giving generously of his time and experience and for guiding me through the myriad of technical details involved in carrying the research to its completion. Many thanks are due Dr. Stephen Abrahamson for his critical comments in areas related to specific problems in education methodology and research; John Duvick at California State College at Los Angeles, and Hal Roach at University of Southern California for their help in writing the computer program; and to Ann Gahan and Louis Seitz for typing and drawing the illustrations for the disserta tion,. The author and his family would like to express their appreciation to the Ford Foundation which provided a forgiveable loan that made it possible to devote fifteen months of full-time effort to this work. And finally, it is with a deep sense of gratitude that I acknowledge the creative contributions of my dear friend, Jack Kirschenbaum. The dissertation reflects the many hours of stim ulat ing discussions with him regarding psychological principles, statistical analysis, and research methodology. iv TABLE OF CONTENTS Page DEDICATION.................................................................................... ii ACKNOWLEDGMENTS.................................................................... iii LIST OF TABLES.............................................................................. viii LIST OF ILLUSTRATIONS.............................................................. ix LIST OF SYMBOLS............................................................................ xi PART I. SYSTEMS ANALYSIS IN HIGHER EDUCATION Chapter I. SURVEY OF RESEARCH AND APPLICATIONS OF SYSTEMS ANALYSIS .............................. 1 Review of systems analysis Systems research in engineering education IL THE SCHOOL AS A SYSTEM...................................... 9 A conceptual model of engineering-type system The resource allocation problem in a school Inputs for decision rules The curriculum: a major input to decision making Curriculum content transm ission and allocation PART II. OPTIMUM SCHEDULING OF SUBJECT MATTER III. STATEMENT OF THE PROBLEM.............................. 33 v Chapter Page IV. A MATHEMATICAL MODEL FOR ESTIMATING STUDENT MASTERY OF SUBJECT MATTER ................................................... 37 V. BACKGROUND IN LEARNING THEORY.................... 45 Curve-fitting; empirical and rational Learning theory models from other fields A summary of relevant principles of learning VI. DEVELOPMENT OF A GENERAL MATHE MATICAL LEARNING FUNCTION ................... 76 Specification of variables and aggregate param eters A general learning function and its properties Additional assumptions The interaction factor, I VII. A HEURISTIC METHOD FOR SUBOPTIMAL SCHEDULING......................................................... 99 The combinatorial problem Review of other methodologies A heuristic algorithm The computer program “ “ Additional computer program capabilities VIII. STATISTICAL EXPERIMENTAL DESIGN............... 115 IX. AN APPLICATION OF THE COMPUTERIZED SCHEDULING MODEL TO THE LAST TWO YEARS OF COURSES IN AN ENGINEERING SCHOOL ................................................................ 120 vi Chapter Page X. SOME IMPLICATIONS OF THE DISSERTATION. 105 Immediate applications of the computer program Potential research projects XI. CONCLUDING REMARKS......................................... 150 BIBLIOGRAPHY........................................................................... 153 w vii LIST OF TABLES Table Page 1. Symbols, Nomenclature, and Functions of Components of a S y stem ...................................... 13 2. Factors Considered in the Sequencing Model. . . . 117 3. Data for Three-Factor Analysis of Variance for Level of Mastery, P ...................................... 127 4. Summary of Analysis of Variance for Computer Experiment.............................................................. 129 viii LIST OF ILLUSTRATIONS Figure Page 1. A total systems c o n c e p t............................................... 10 2. A conceptual model of any system ............................ 12 3. A schematic representation for Systems Analysis R esearch................................................... 16 4. Increase in level of m astery due to sequence changes of subject m a tte r ...................................... 43 5. Time reduction curves, arithmetic s c a le .................. 48 6. Time reduction curves, logarithmic s c a l e s 48 7. Estimated and actual time reduction curves used in long-range planning and production of airplanes at Douglas Aircraft Com pany.............. 50 8. The fit of an ogive to learning data for the learning of three-phase symbols obtained by Culler (7:371)......................................................................... 53 9. Forgetting as related to the method of m easure ment ............................................................................ 59 10. Retention of one elementary school subject, one high school subject, and three college subjects from sixteen to eighteen months after the end of the course ........................................................... 61 11. Forgetting after different degrees of overlearning . 64 12. Typical learning curve trends for variations in the type of learner, L; type of subject m atter, S; and type of teaching method, M.............................. 73 ix Figure Page 13. Final level of m astery as a result of learning- forgetting interactions............................................ 74 14. Teaching methodology in three dim ensions 81 15. Interactions between param eters S, M, L 97 16. Flow diagram of the computer p ro g ram .................. 106 17. Computer printout of initial schedule. Last two years of mechanical engineering at CSCLA. . . 120 18. Levels of m astery as function of time for four courses in Engineering at California State College at Los A ngeles......................................... 122 19. Computer printout of an improved schedule for last two years of mechanical engineering at CSCLA .................................................................... 123 20. Computer printout of param eters used in the calculation of P ...................................................... 125 21. Computer printout of final table of information showing input data for the p ro g ram ................... 125 22. Graphic presentation of interactions between learner, teaching method, and schedule (cont.) 130 23. Graphic presentation of interactions between learner, teaching method, and schedule . . . . 131 x LIST OF SYMBOLS A exponential coefficient of decay or forgetting B em pirical constant for forgetting H total time allocated to given subjection the curriculum I interaction factor K curvature of learning curve at t = 0 c " L ^ type of learner M type of teaching method n number of courses P total educational potential for a given schedule Pj value of P for the jth schedule p educational potential or level of m astery for one course R total number of repetitions or reinforcements r number of courses exchanged at one time S type of subject m atter t time current decay or forgetting time y cumulative learning time nCr number of combinations of n courses exchanged r at a time xi PART I SYSTEMS ANALYSIS IN HIGHER EDUCATION CHAPTER I SURVEY OF RESEARCH AND APPLICATIONS OF SYSTEMS ANALYSIS Review of systems analysis Systems analysis has been used extensively in many areas. It has been used in the design of machines, electric circuits, defense and weapons systems, communications, industrial management, international relations, and in very recent years in the management of educational institutions. Hopkins has defined a system as "a bounded complex of elements (men, machines, objects or forces of nature, or any combination of these), interrelated by processes, which respond to events to achieve an objective” (52:25). We can also think of a system as a set of interrelated factors that may be used together to produce an output. The literature of operations research is full of different definitions which vary in accordance with the needs and objectives of the field under consideration. A general survey of recent work in the area of systems analysis is presented in Chapter 5 of Progress in Operations Research (22). 1 During the past few years concerted efforts to simulate socioeconomic systems have been undertaken with appreciable success. Many individuals and specific companies have developed simulation models for individual cases.. For example, Professor Jay F orrester, Massachusetts Institute of Technology, has developed a model based upon feedback control systems to simulate the dynamics of industrial concerns (6). Dr. Alan J. Rowe, while at the General Electric Company in 1958, simulated a machine shop and used his model to test alternate dispatch and decision rules (63). Earl LeGrande, while at Hughes A ircraft Company, worked with a large-scale simulator of the Hughes Company operations (57). A concentrated effort to study management controls in the context of a total, complex, industrial system and to develop a general-purpose computer model for simulation was carried on at System Development Corporation from 1959 through 1962. The purpose, history, areas of investigation, and accomplishments of the Management Control Systems (MCS) Project, together with a comprehensive annotated bibliography, is given in a report by J. Kagdis and M. R. Lackner (53). In his Ph. D. dissertation in engineering, University of California, Los Angeles, Arnold Reisman developed a general analytical model for service and/or goods-producing enterprises. * The model presented structural, mathematical, and financial evalua tion frameworks which were shown to include as special cases systems analyses in the areas of manufacturing, hydraulics, heat transfer, and electrical circuits (72). At the present time, the Canadian Energy Board, an arm of the government of Canada, is applying this model to the dynamics of the orderly exploitation of that country's energy resources, i.e ., coal, oil, gas, hydropower, etc. A doctoral student at U, C. L. A. is applying the model to the opera tions of the U.S. Borax Corporation, and a proposal has been sent to the Public Health Service requesting funds to apply the general model to the operations of a class of community hospitals. At the present time, North American Aviation Corporation, Los Angeles, is completing a report for the state of California entitled "California Integrated Transportation Study. " The study involves the development of six submodels into one large system. The submodels are concerned with population, land usage, econo m etrics, transportation^ demand, transportation simulation, and evaluation. Other large-scale systems analyses are underway in California in the areas of water resources, air pollution, and social services. It can be seen that systems analysis techniques have been and are increasingly being applied to a variety of socioeconomic systems. However, although there is a very large annual expenditure in the United States on education, support for research in educational strategies has yet to find its way to practitioners of operations research or systems analysis on a large scale. Systems research in engineering education In one of a number of papers related to research in educa tion, Platt has made the following arguments for greater research effort in the field of education (71). 1. Education can trigger the multiplier-effect leading to higher achievements of individuals and of societies by liberating talent. 2. Although education accounts for from 2 to 8 per cent of the gross national product, practitioners of the science of decision-making devote far more effort f ' to phenomena having much less consequence in the scheme of things than to research in education. 3. There is a strong presumption of structural under investment in education,and therefore it is all the more important to find improved decision guides for the allocation of funds and resources to education so that it can more successfully compete with other investment allocations. 4. Education is a system and set of subsystems potentially susceptible of analysis, design, and perhaps eventually some optimization. As such, there should be more systematic ways of determin ing the size and mix of educational effort. Perhaps the experience of management scientists in pro gramming problems, in lead-time studies, in the techniques of relating an output to an input can be of help in improving educational effectiveness. A pioneering study of the potential of systems analysis in education was conducted at the Rand Corporation by Kershaw and McKean in 1959 (54). This was an exploratory study to assess the possibilities of making quantitative comparisons of educational sys tems. The study concluded that we should try to establish quantitative relationships between school characteristics and outputs, and then compare alternative educational systems explicitly. The ability to compare various policies and programs (inputs) in an educational institution depends to a very large extent upon the ability to measure performance (outputs). During the early 1960s the United States Office of Education sponsored a large-scale testing and statistical study intended to secure a national inventory of abilities of students of high school age, particularly the fifteen-year olds. Project Talent was carried out at the University of Pittsburgh by the American Institute for Research. Utilizing techniques sim ilar to those of operations research, the study resulted in the development of national norms based on a wide variety of aptitude and achievement variables, on the basis of an extremely large sample (close to 75,000 boys and girls), representing an entire age group. It was concluded that methods can be developed for extending the data analysis to get useful estim ates about the distributions of the variables under con sideration for age populations other than the age of fifteen. It is significant to note that to date, no sim ilar study of this type has been undertaken at the college-level, A comprehensive study of engineering and engineering educa tion was begun at the University of California at Los Angeles, Department of Engineering, in 1956. The program was accelerated in 1960 by a $1, 200,000 five-year grant from the Ford Foundation. The large-scale research program (Educational Development Program) created a rational procedure for curriculum synthesis, optimization, and maintenance (74). Central to the procedure was the careful definition of the projected needs (constraints) of the society in which the host institution was imbedded. The engineering faculty at U. C. L. A. assumed that the curriculum is a complex system and that any meaningful design can only be made in term s of the system limitations, param eters, variables, boundary conditions, constraints, inputs, outputs, resources, etc. The basic steps of any design process were formalized, the characteristics of the student input to the educational program were examined, the desired product or graduate of the program was described, and a mathematical curricu lum model for the optimum allocation of subject m atter was developed (61). It is noted at this point that in the context of this disserta tion, the use of the word "optimum" implies that a specific set of criteria have been completely satisfied. It is recognized that this is not a mathematical optimum. Although, as will be shown later, numerous specific tech niques have been developed for scheduling students, planning facilities, using teaching machines, investment policies, and other diverse areas applicable to higher education, no large-scale attempt has as yet been made to view an institution of higher learning as a social system and to apply systems analysis methods to it. The need for such a project is generally recognized but a serious attempt at implementation has not been made probably because of the complex ity of the problem, the lack of reliable input-output relationships, and the comparatively large resource allocation that would be required. Such an undertaking would require personnel with broad interdisci plinary backgrounds. There are a limited number of people who are fam iliar with systems analysis techniques, statistics, business administration, the structure and educational requirements of educational institutions, computer programming, and data process ing. In the next chapter, an initial attempt will be made to look at a school of higher learning as a system requiring improved techniques for the allocation of its resources. No attempt will be made to actually analyze such a complex system. Rather, a survey of available research techniques and programs will be made in the hope of delineating those areas in which potentially significant con tributions can be made. CHAPTER II THE SCHOOL AS A SYSTEM A conceptual model of engineering- typs systems The application of systems analysis to many different types of physical as well as socioeconomic systems has led to a host of conceptual frameworks or structural schematics to represent such systems. A large number of socioeconomic systems can be sub sumed under the general conceptualization shown in Figure 1. This is a generalized structural framework which involves populations, m aterials, capital equipment, orders, and information flows through transducing, policy, and control operations. With the aid of appropriate mathematical techniques and relationships like the conservation equations this framework can recognize feedback concepts, the inherent nonlinearities of the system, and it incorpo rates the idea that the multitude of variables that affect the overall dynamic behavior of a system cannot, in general, be given individual or isolated treatment. 9 Fig. 1. --A total systems concept. o k 11 Conceptually, a socioeconomic system can be thought of as cpnsisting of inputs, outputs, operations, sensor-controls, data processing, and decision rules. A simplified representation of these is shown in Figure 2. These components have different names in the literature. A summary of typical component symbols, their respective nomenclature in diverse fields, and their respective functions is given in Table 1. Some important characteristics of this approach are summarized as follows: 1. A complete system description is deducible from attributes of the inputs and outputs, initial conditions, and constraints. Every system can be broken down into 'sm aller subcomponents. The same methodology applies to each subsystem. The degree to which any system is decomposed into subsystems depends upon the questions being asked of the system. 2. Every system or subsystem includes operations, sensory-controls, data processing, and decision rules. The boundaries of socioeconomic systems are often definable only in term s of the interactive effects between the system and its surroundings. 3. In an actual system, the data processing system and its decision rules are usually distributed geographically 12 Inputs Decision Rules r* Data Processing i n g ^ --------- Sensor Control Operation(s) Sensor Control Outputs Fig. 2. - -A conceptual model of any system. 13 TABLE 1 SYMBOLS, NOMENCLATURE, AND FUNCTIONS OF COMPONENTS OF A SYSTEM Component Symbol Component Nomenclature Component Functions Opera- tion(s) Sensor Control f Data \Processing> Decision Rules 7 System thermodynamics Activity (S. D. C .) Level (Forrester) Node (Reisman) Impedance ), . . Transducer)^ ^ Modulator (electrical) Controller (feedback theory) Valve (flow regulation) Instrumentation (prop erty measurement) Transm itter),. , . Receiver )<lnfo'> Information processing system Controllers Management Administration Decision functions Control equations Policy Obeys the conservation concept for all mass, momentum, and energy flows: Rate of Input + Generation = Output + Accum ulation. Examples: Source, sink, storage, any device, school, hospital, department, classroom, library, activity. Measures or senses flow prop erties; transm its property data to data processing; receives control signals from data pro cessing; controls flow rates, splits and directions. Links the m ass, momentum, and energy flows with information flows. Performs the following functions on information: Arranging, balancing, checking, coding, comparing, computing, con verting, copying, counting, document writing, duplicating, filing, listing, posting, print ing, proving, punching, read ing, searching, selecting, sorting, summarizing. Provides rules, procedures, equations, algorithms, poli cies, and methods which relate the states (as rep re sented by properties) in various parts of the system to each other and to the surround ings or environment. throughout the system. In a simulation model, the data processing and decision rules are conceptually separate from the system under analysis and connected to it only via information flow channels (see Figure 1). A simulation model must take into account appropriate pipeline and processing delays, queues, storage, generations, dissipations, and distortions of informa tion and m atter in the real system. Information in general, for the purpose of manage ment and control, is assumed to represent a status description of the attributes of entities. This is analogous to what happens in thermodynamics where we analyze or describe a system in term s of the properties of the working substance. Other factors influencing the system are defined as: (a) Inputs--those resources which are converted or modified by the system in question. (b) Outputs--that which is produced by the system and whose value is to be improved; in the limit, optimized. (c) C onstraints--all other elements to be accounted for in the design. The adoption of the foregoing ideas as the basis for an initial approach to the application of systems analysis concepts to educa tional institutions facilitates the design of a schematic representation of significant areas of research in education. The resource allocation problem in a school The search for a significant but manageable research problem in education has led to the development of the schematic representation shown in Figure 3. This diagram incorporates the ideas about systems analysis which were presented in the last chapter and makes it possible to systematically explore the various areas of research in education. The school which is shown in the diagram as a black box represents a highly complex socioeconomic system. Students are the major input (raw materials) of the system. Alumni, school dropouts, and transfer students are the output (product). Other inputs to the school are the faculty and staff, m aterials, orders (purchas ing), facilities, and information. The exchange of information between the system and the environment is not shown on the diagram for the sake of clarity but it does exist. It is this exchange of information with the society in which the school is imbedded that 16 CURRICULUM SYNTHESES IMPLICATIONS TEACHING METHODS WHICH ITEMS CRITERION FUNCTION CURRICULUM CONTENT TRANS MISSION CURRICULUM CONTENT ALLOCATION TOTAL TIME PER ITEM ITERATION L O O P ADAPTIVE ’ DECISION RULES) TIM E DISTRIBUTION P E R ITEM UTILITY FUNCTION CONSERVATION CONCEPTS P O L I C Y ALLOCATIONS ALGORITHMS PERSONNEL RELATIONS DECISION RULES COMPUTER PROGRAMS PUBLIC RELATIONS DATA PROCESSING FINANCIAL FACULTY STAFF CAMPUS OPERATIONS MATERIALS PLANNING DEVELOPMENT R E S O U R C E MIX ORDERS FACILITIES STUDENTS (INPUT) ALUMNI (OUTPUT) SCHOOL Fig. 3. --A schematic representation for Systems Analysis Research. 17 makes the operations of the system meaningful. The larger systems that surround the school provide general educational objectives, curriculum constraints, limitations on funds, and other important inputs. The properties of all the inputs and outputs are constantly monitored by the sensor-controls. The sensor-controls not only regulate the flows into and out of the system but also measure the properties, convert them into signals that are sent to the data-processing system and receive new signals which in turn determine new flow rates. The way in which the data are processed is determined by a large number of decision rules. These decision rules are, in the case of a school, determined by the faculty and school adm inistrators. Ideally, the operations of an educational institution are determined by the educational objectives. From a managerial stand point, the major objective of a school is to allocate available resources in such a way that the difference in educational potential between entering and leaving students is maximized. In this dissertation, educational potential is defined as the level of m astery of given subject m atter. Students enter a school system with a given level of educational potential and hopefully, leave with a much higher level. The central task of adm inistrators is to allocate faculty, staff, m aterial, facilities, and information at such times and places 18 and such proportions that the objectives of the school will be achieved in a most efficient manner. During the past few years System Development Corporation has been developing a set of simulation models for five different types of high schools. The objective has been to develop a general simulation vehicle that permits a designer to construct on a computer a detailed dynamic model of real or proposed high school organiza tions. The vehicle is a simulation and list-processing system consisting of a comprehensive set of procedures written in the JOVIAL language. It is constructed in modular form so that the models can be built up by assembling the modular parts (activities, procedures, packages, modules, and total systems) into a particular configuration. An extensive series of flow charts delineate all school functions operationally. Individual or batch flows can be accommodated by what amounts to an elaborate bookkeeping system maintained with the aid of a large digital computer (50). The foregoing is a data-handling approach which is relatively unconcerned with a specific learning theory or method of evaluating educational system outputs. It assumes that our knowledge about teaching-learning processes can be greatly accelerated largely on an experimental basis, namely, by deducing new relationships by studying large quantities of data. Extensive sampling and use of 19 pseudorandom population flows give data from which cause-effect relationships are relatively hard to define (66). Research in this area is still in its infancy; the need for creative research contribu tions here is apparent. Michael Lackner, who has made a number of significant contributions to the development of simulation languages has stated that "realization of the great potential of digital simulation seems to await the development of a language capable of facile expression of a wide variety of systems" (56:1). The application of system s"^ analysis to educational institutions and ultimately to the simulation of these institutions on computers has caused a proliferation of simulation languages in recent years. A concise summary of basic definitions used in digital simulation and of the basic references in the field is contained in an eight-page report entitled SHARE Digital Simulation Glossary (55). A brief description of three simulation languages (GPSS, SIMISCRIPT, and SIMPAC), together with a discussion of their relative advantages and disadvantages, is given in reference (67). The technology for processing the vast amount of data generated in a school system is well developed. There is a very large body of literature which is expanding at an ever-increasing rate. Space does not permit a detailed discussion of the implications 20 of digital computer relationships with man and social systems, but the reader is referred to the Educational Data Processing Newsletter and to an extensive annotated bibliography entitled "The Administra tor and the Computer" (31). The bibliography deals with such diverse data processing subjects as optimal scheduling of students, college registration, self-instructional devices in counseling, construction of school simulation vehicles, administration of an automated school, computer simulation of human thinking, and sources of information on educational media. The development of high speed, large memory, data proces sing equipment is already enabling school adm inistrators to cope with the tremendous amount of information or data that must be considered in the management of-a- school. The ability to use all of these data is a strong function of the decision rules that are employed. Inputs for decision rules As can be seen from Figure 3, the decision rules in an educational institution are derived from three major sources: the rules that are expressed explicitly by mathematical formulations, the administrative policies and procedures that are usually known to the faculty and staff in an implicit or verbal manner, and the curriculum. 21 The explicit decision rules governing educational institutions are not as yet fully developed. The interrelationships between the various subsystems of a large social system like a school cannot as yet be described mathematically by the simple application of the conversa tion concepts, namely, the laws of conservation of m ass, momentum, and energy. - Taft and Reisman (75) have shown that when these conserva tion laws are formulated in general mathematical equations the basic equations for many fields can be derived from them as special cases. An examination of the basic equations (the Navier-Stokes equations) has shown that they are limited to such areas of physical science as aerodynamics, solid mechanics, thermodynamics, strength of m aterials, fluid mechanics, statics, dynamics, heat transfer, and electric circuits. Using an expanded form of Kirkhoff's laws for electric circuits, Reisman (36) has developed a mathematical fram e work which consists of K node or junction equations, M potential or branch equations, N constraint or auxiliary equations, and a methodology for their application. These equations provide for non linear and transient behavior of any socioeconomic system as well as physical system and give recognition to the value of system simulation techniques. Although these equations offer useful descriptions of the flow 22 of such diverse quantities as m aterials, money, energy, people, and information, they incur numerous practical problems in connection with the storage, retrieval, listing, and processing of myriad of data of information generated in a school. Additional research is needed to determine the units of measurement for information; to discern, m easure, and control impedances and potential differences in a school system; to develop techniques for handling mixed and coupled flows at a node; to generalize the equations to include mutual inductance; and to develop rational procedures for converting information which is now available regarding schools into a form that can be used in the model. The development of allocation rules, particularly those concerned with the allocation of funds and facilities, has been a highly decentralized process in the United States. Although a sub stantial number of allocation methodologies have been in use by government and industry in recent years, only a few of these have been adopted by colleges and universities. Techniques that appear to have potential usefulness for growing and expanding educational institutions include some of the following: 1. CRAFT, Computerized Relative Allocation of Facilities Technique, could enable college admin istrators to quickly and economically evaluate many possible school facilities layouts. This technique could help to determine the optimum location on a campus of the library, the cafeteria, the computer center, and the administrative offices (24). PERT and other critical-path techniques could be used in the planning and controlling of the work force and financial requirements of large school construction projects (65). CERBS, a general financial model, reduces to their present worth all disbursements and receipts involved in the possession and operation of capabilities to perform services and/or produce goods (37). This model could be used in colleges to systematically compare and evaluate a variety of policy decisions concerned with the purchase and replacement of laboratory equipment, maintenance tools, office and computer equipment, and large capital outlays. Operations Research could be applied in higher education for facilities utilization studies; economics of automating a library, registration, and student health services; statistical patterns of demand for college courses; and for the development of models 24 to describe real-system behavior (30). r In contrast with the lack of integrated basic research in the areas of decision-making described above, a large amount of work has been carried on in the areas of educational data processing of student records and the scheduling of students into classes. Through cooperative efforts between industrial organizations like International Business Machines and educational institutions like Stanford University and Massachusetts Institute of Technology, a number of comprehensive computer programs have been developed and made operational. These computer programs are being used to supply a variety of data at any time; to rapidly and efficiently carry on the multitude of activities related to school registration procedures, record keeping, grade reporting, and budget forecasting; and to produce m aster schedules for assigning courses, faculty, facilities, and students. A brief list of typical programs that are currently in operation together with their appropriate references follows: 1. GASP, Generalized Academic Simulation Programs, M .I.T. (59) 2. SSSS, Stanford School Scheduling System, Stanford University (69) 3. CLASS, Class Loading and Student Scheduling, IBM (68) 4. FDS, Flexible Daily Scheduling, Brookhurst Junior High School 5. SDPS, Student Data Processing System at the University of Illinois (47) 6. A Computer Program for Budget Forecasting, Harvey Mudd College (70) These programs are introduced here to give the reader an indication of the scope of the programs that are already available as potential inputs to decision-making in higher education. Another large class of inputs to the decision rules of an educational institution consists of the administrative policies of the school. These are policies related to such items as personnel relations, public relations, finances, campus maintenance and operations, planning, and development. These policies vary from school to school, department to department, administrator to adm inistrator. Often, policies are formalized in faculty handbooks, administrative codes, and committee minutes; but usually, they are contained in the minds of the people who are doing the work. As yet, few formalized procedures have been developed for systematically and economically gathering and compiling policy information so that it can be readily used in systems analyses and computer simulation of the educational system. The most significant input to the decision 26 rules in an educational system, however, is the curriculum. The curriculum: a major input to decision-making The curriculum, educational program, or program of study reflects the purposes and educational objectives of the school and the constantly changing needs of the society served by the school. It delineates in what ways the student population is to be transformed while passing through the educational system. Hence, it is concerned with the educational process; what is to be taught, how much, when, where, and how subject m atter is to be transmitted. Curriculum synthesis implications constitute prim ary inputs to the decision rules which control the data processing and ultimately the operations of the entire educational system. Decisions regarding the allocation of faculty, staff, facilities, equipment, and services flow directly from a knowledge of the requirements of the curriculum. These decisions in turn feed back to the curriculum analysis and synthesis area and thus form an iterative loop. In order to consider some of the research that has recently been carried on in the area of curriculum, this area will be divided conceptually into two parts: the curriculum content allocation and the curriculum content transm ission. These two parts form an iterative loop as shown in Figure 3. Studies are made to determine what and 27 when subject m atter should be taught, then the teaching methods are studied, then original assumptions regarding time allocations to the subject m atter are reexamined and revised, then the teaching methods are improved, etc. At any instant in time, the data regard ing the current status of our knowledge of the curriculum can be tapped off from the iterative loop and supplied to the decision-making component of the system. A comprehensive review of the literature on curriculum planning and development during the period of time between June 1960 and June 1963 is presented in the Review of Educational Research (38). Most of the literature reviewed was written by persons who are engaged in research in education, psychology, and related fields. . The following section will indicate some of the recent research in the area of curriculum which has been conducted by persons with mathe matical and/or engineering orientations. Curriculum content transmission and allocation The emphasis in curriculum content transm ission has shifted in recent years from research on conventional teaching methods, through programmed textbooks and simple mechanical teaching machines, to computer-based instruction. John E. Coulson, at System Development Corporation, has pointed out that a few con 28 trolled experiments with computer-based teaching systems, while encouraging, have not yet demonstrated clear-cut superiority of this method of instruction over simpler, more orthodox teaching methods (48:1). In addition to contributing to research and develop ment of learning laboratories, special mechanical teaching and communication devices, and time-sharing computer systems, programmed learning has led to basic research in learning theory. A considerable amount of attention has been focused upon optimum methods of presenting instructional items to the student (49). Programmed instruction has dealt largely with linear (fixed sequence) programs and with branching programs (scramble texts) which offer the student alternative paths or item sequences through the lesson (78). James E. Matheson, at Stanford University (58), has studied the teaching of a list of paired-associate items in a fixed number of presentations. He assumed the validity of the simple learning model of Atkinson and Estes (1), formulated a reward structure in order to measure the effectiveness of teaching, and in term s of the reward structure and the learning model he derived optimum teaching procedures by applying dynamic programming techniques to Markov processes. It is usually assumed that the state of the Markov process is directly observable at each step in the process. It is then possible to base all decisions about the process 29 upon the state of the Markov process without regard to the past history of the process. But the state of the Markov learning model is not directly observable and those observations that are available depend upon the state of the model in a probabilistic manner. Matheson derived an equivalent Markov process in the observable states of history and then treated this new process by conventional means in order to optimize it. In his engineering doctoral dissertation, Arnold Roe (60) developed an analytical adaptive decision structure for educational systems. His decision structure rested upon four cornerstones: a plan for gathering and using data; an explicit criterion function; a set of decision rules for achieving the criterion; and a utility function which relates system inputs and system outputs to a value scale outside of the system. The utility function defines the output of an educational system as the increment in life-cycle productive output attributable to the educational experience for all individuals who have been p art of the system . It provides a means of converting such available m easures as student grades, student learning time, teacher inputs, school capital and maintenance costs, and so forth, into a net value of the transformation effected by the system. The suggested criterion function which must be maximized is the sum of the net utility of all students' outputs. Roe also developed decision rules which tend to maximize the criterion function under different conditions of a priori information. This research led to the development of a computational backwards-induction solution for the multistage or continuous sampling procedure from k normal populations. The objective here is not to discuss the m erits, assump tions, or implications of the foregoing research efforts in the field of education by people who have mathematical and systems-analytic orientations. The two references just cited contain extensive bibliographies of other recent research projects in this field. The curriculum content allocation problem has been studied intensively by the Engineering Department at U. C. L. A. during the past few years. The Educational Development Program has produced some meaningful and systematic procedures for determining the content of the curriculum and the amount of time that is to be devoted to each item that is to be taught. The proposed procedure for curricular synthesis involves the application of three criteria to the subject m atter of the curriculum. The amount of time allocated to each instructional item, topic, or course depends upon its relevance to the aims of the curriculum (the criterion of relevance); the degree to which an item helps or reinforces other items (the criterion of 31 generality); and to a lesser degree, the use that a given item makes of other subjects (the criterion of articulation). This procedure maximizes the relevance of the whole curriculum to the aims of engineering design. The procedure can be generalized to fields other than engineering by defining different categories of subject matter and by adding other criteria as deemed necessary (62). This procedure, which has been utilized in curriculum synthesis studies at the School of Engineering at Dartmouth College, provides a systematic and relatively objective methodology for determining which items should be taught and how much time to spend on each item. It does not directly consider the problem of how to distribute the presentation of each item in time, i. e . , when should each item be taught. The problem of when to teach an instructional item, or more generally, the problem of optimum scheduling of subject m atter is one that has been largely ignored by researchers in education. It is a problem that must include a consideration of learning theory, combinatorial analysis, teaching methods, curriculum constraints like prerequisites for courses, and so forth. Solution of this problem will help to bridge the gap between curriculum theory and its practical implementation in the classroom. In the final analysis, the success of a given educational program hinges upon the ability of the faculty 32 and administration of a school to schedule each student into sequence of courses which will maximize his educational potential on the day of graduation and achieve the educational objectives of the school. Part II of this dissertation is devoted to a study of this problem. The significance of Part I of this dissertation may be summarized in term s of the following contributions: 1. An overview of systems analysis and some of its poten tial implications for education has been presented. 2. It was shown that when a school is viewed with an engineering-type of approach; namely, as a system, diverse areas of research in education can be viewed in a unified manner and research areas having high payoff potential can be more easily discerned. 3. The curriculum was shown to be of central im port ance in the decision-making activities of a school. It was also shown that powerful tools from other fields (linear programming, Markov processes, systems analysis, statistical induction, decision theory . . .) can and are being used in the analysis and synthesis of curricula. PART II OPTIMUM SCHEDULING OF SUBJECT MATTER CHAPTER III STATEMENT OF THE PROBLEM In the analysis and synthesis of a curriculum, methods are currently available for determining what subject m atter shall be taught, and approximately how much time should be devoted to each item in the curriculum. These decisions are based upon the educational objectives of the school, upon the faculty’s estimate of priorities that should be allocated to each item, and upon the faculty's previous experience in teaching the subject m atter. How ever, it is not only important to know the amount of total time that is to be allocated to each topic, it is equally necessary to develop a method for distributing that total time over the entire duration when the student is in school. For a given topic, shall we teach all of it during the first sem ester, the last sem ester, or shall we spread the teaching hours out over the entire time that the student is in school? The implications of this question can, perhaps, be better appreciated if we approach it from the point of view of a school counselor or a faculty advisor. 33 34 At the present time when a student plans a program of study with his counselor he is usually referred to the school catalog. In the catalog he will find a recommended sequence of courses that he should take in order to complete the requirements for graduation in his particular department or subject area of specialization. The student might take four or five courses each sem ester for approxi mately eight sem esters and thus at graduation will have completed about forty courses. As long as he does not violate the prerequisites nor the maximum number of hours allowed in classes per week he . r may take the courses in any sequence that he likes. If we ignore the prerequisite constraints and the maximum hours per week constraint, there are forty courses taken five at a time (^qC , _ = 40!/(40-5)!5!) or 658,008 combinations of courses. This large number is consider ably reduced when we take the constraints into account. In most schools at the present time, the student may follow any one of the large number of course sequences that remain after the two con straints have been imposed. If we impose only the two constraints mentioned above, any of the remaining courses of study (henceforth referred to as schedules) will be equally acceptable. However, if we consider the degree to which a student m asters all of the subject m atter in a given schedule, it is clear that some of the remaining schedules will be more acceptable than others. 35 In order to define and develop the additional constraint related to .the m astery and retention of subject matter by the student, it shall be necessary to consider our present knowledge about the mechanisms of learning and forgetting, teaching methods, various types of subject m atter, and various types of students. Thus, we shall be involved in educational psychology, combinatorial analysis (the area of mathematics concerned with the combinations and permu tations of large sets of items), sequencing theory, and finally in computer programming. It is contended here that the synthesis of new curricula involves not only the imposition of logical and time constraints, but also the consideration of the impact or the effect of the program of learning upon the students themselves. The succeed ing work represents an initial attempt to explicitly take into account some of the factors and principles, that in the past, have only implicitly been considered in the development of improved programs of study. The problem before us is to develop a mathematical model which will take into account learning theory and which will provide a method of sequencing the subject m atter of a curriculum in such a way as to maximize its benefit to the student. Such a model will enable us to evaluate one schedule relative to another. Then a systematic procedure and a comprehensive computer program will be 36 developed which will enable us to systematically find better and better schedules. Finally, some illustrative examples of the utility of the model and the computer program will be given and some suggestions for continued research in this area will be presented. We now turn our attention to the development of the sequencing model. CHAPTER IV A MATHEMATICAL MODEL FOR ESTIMATING STUDENT MASTERY OF SUBJECT MATTER It is assumed for the purpose of this work that the content of the curriculum, namely, what it is that shall be taught, is already known. In addition, it has also been decided how much time shall be devoted to each item or topic or course in the curriculum. What are the major items which will affect "when" we shall teach each item? We can imagine a student studying a course, such as differen tial calculus. He enters this course at the beginning of his second sem ester in school and as we expose him to more and more class hours of instruction, his educational potential in this subject area increases. What does educational potential mean? In the present context, the educational potential represents the level of m astery of the subject m atter of the student. Does this mean how well he has memorized this m aterial? Does it mean how well he can use it in new situations? Does it mean that he can see its relationships to other subjects? Does it imply that he has developed new attitudes 37 38 and skills in this subject area? The answer to these questions is that, depending upon the course content, some or all of these criteria will apply. Every person who teaches a subject has, implicitly or explicitly, developed a set of criteria with which to evaluate that subject. In addition,we are constantly giving students tests to determine what is the level of m astery of the m aterial that has been presented. Our tests may not be very valid but we are making decisions based upon our testing, as bad or as good as the testing may be. The point here is that we do have criteria and we do test. Even though improvements can be made in both of these areas, in the kinds of criteria that we have set up and in the testing, we are in a position where we are forced to make decisions as to whether a student passes a course or fails. Since the decision-making process is unavoidable, it is suggested here that improved criteria and testing procedures are both necessary and desirable. As we teach the student, hour by hour, his level of m astery in the given subject increases; but it does not increase linearly. It shall be shown in a later section that a considerable body of empirical data from experimental work in psychology already exists and that these data are directly related to the problems of learning. What emerges from this large body of data is that, in general, a person 39 follows an "S-shaped" learning curve. Initially, when a student is introduced to new m aterial, the slope of the curve is very close to zero. This implies that if we plot level m astery (P) versus time (t) we find that for every unit of time that we expose the student to the subject m atter, his level of m astery increases very little. But as the number of hours of learning increases, the slope increases rapidly until m astery is directly and linearly proportional to learning time. Then as the learning time is increased still further, the curve gradually levels off until it becomes asymptotic to some maxi mum level of mastery. This maximum level represents the level of m astery that the student would reach if he were to m aster not only the m aterial presented in the course in question, but also in all courses offered by the school which repeat and reinforce the m aterial in the given course. Thus, any student who was to achieve the maximum level of m astery in a given course would have obtained the largest amount of "overlearning" that is theoretically possible in that course and in that curriculum. By "overlearning" we mean learning beyond that of the given course. It can be seen that as the top of the curve is approached, each additional unit of learning time produces less and less increase in level of m astery; there is what may be thought of as a point of diminishing returns. Now we consider what happens when a student has completed the course and he is not using the m aterial from it in another course. The completed course has, in a sense, been an investment, by analogy to economic problems. When the student finishes this course and does not use it in other courses, there is a decay or depreciation in his level of m astery. In other words, if an equivalent final examination for the course were given to the student some time after completion of the course, he would obtain a lower test score than previously. We have found from psychological test data, class room experiments, and industrial studies over a wide variety of subject m atter areas, people, teaching methods, and so forth, that the decay or forgetting function may be represented by a negative exponential type of curve. In general, the student's level of m astery decreases with disuse as time goes by. However, as soon as he starts to use the subject m atter in question in another course, then his level of m astery begins to rise again. It shall be assumed that it rises from the point of the learning curve where it stopped the last time but it has been shifted downwards because of the decay. A more precise description of the learning and forgetting curves will be undertaken in succeeding sections after a survey of research in this field has been presented. At this point, it should be noted that in describing the student's m astery of a given subject we require a function or 41 functions which produce a series of curves that rise, then fall, then rise again, fall again, and so forth, until graduation day or term ina tion of the formal program of study. On the day that a student graduates or terminates he will have some level of m astery of the subject m atter in the given course. The final level of m astery depends upon how the subject m atter has been distributed over the total time of the program. If all of the hours available in the curriculum had been taught to the student in the first sem ester, he might have reached close to the top of the learning curve. During the remaining seven sem esters in school his level of m astery would continuously decay until, on the last day, he would remember very little about the course. However, we know that the student will probably experience some repetitions of the m aterial in other courses and probably in different contexts. There fore, the learning-forgetting curve will rise a little, fall a little, rise a little, and so on, until at graduation the student's final level of m astery in this subject will be somewhat higher than if there had been no repetitions at all. It can be shown that there are some time distributions of the subject m atter that produce high levels of m astery on the last day that the student is in school. Some schedules of subject m atter are better than others. Thus, when we consider the distribution of every 42 course, topic, or item in the curriculum on the day of graduation, we can aim to maximize the student's m astery over the entire curriculum. This means that if we added together the student's levels of m astery in all courses on the last day in school, the total would be a measure of m astery of the entire curriculum. Different schedules or distributions of subject m atter will yield different total levels of m astery on the last day of the course of study. It is a major objective of this work to develop a methodology for finding the schedule or schedules that have the highest overall level of m astery relative to all the others. Figure 4 illustrates these ideas by showing that the change in location of courses A and C in the initial schedule, produces a change in the overall level of mastery, P, from 22.9 to 32. 2. It is not sufficient m erely to maximize m astery on gradua tion day; it is also important to maximize the student's retention of the m aterial after graduation. The rate of forgetting decreases with the increase in the number of repetitions of the subject m atter, the teaching methods used, the cumulative amount of time taught, and other factors. It will be shown that, fortunately, schedules that are optimal with respect to total m astery are also optimal with respect to retention. It can be reasonably contended that the problem posed here 43 Course Course Course Course INITIAL SCHEDULE >> 10 time (weeks) pi = I p = PA + Pb + pC + pD = 22- 9 Course Course Course Course OPTIMUM SCHEDULE D / time (weeks) Pq £ p + Pg + p£ + pp 32. 2 Fig. 4. - -Increase in level of m astery due to sequence changes of subject m atter. 44 implies knowledge of factors beyond the present state of the art. In order to determine the learning curve and the decay curve, a large number of psychological factors must be considered. This contention can be answered by stating that it is precisely what must be done now if we are to obtain better criteria for the development of improved curricula. It must be done even though our knowledge in these areas is still quite sketchy and not very rigorously defined. However, a considerable body of research in the area of learning theory is already available. The next section is devoted to a brief survey of current knowledge in those areas of psychology, education, and business management related to learning and forget - ting. CHAPTER V BACKGROUND IN LEARNING THEORY Curve-fitting: empirical and rational As far back as 1885, Ebbinghaus (4) began the first quanti tative studies of learning by introducing mathematical formulations for empirical data. The procedure that he used has since been called em pirical curve-fitting. It must be emphasized that in empirical curve-fitting the function is selected solely on the basis of fit, and not on the basis of any theory. At a time when knowledge about the mechanism of learning was still relatively undeveloped, the curve-fitting approach of Ebbinghaus offered at least a quantitative estimate when learning conditions approached those of available experimental data. There has been a growing trend away from mere em pirical curve-fitting during the last seventy-five years and an increasing emphasis upon - rational curve-fitting, modeling, simulation, statistical studies, and ’’flight from the laboratory. ” In a series of papers and books published between 1947 and 1961 (41, 42, 43), the experimental 45 psychologist, B. F. Skinner, has forcefully pointed out that there is as yet no adequate theory of human behavior, nor a comprehensive theory of learning' in particular (16). Skinner's major contention is that intensive experimental work leads eventually to a theoretical formulation which has enormous technical potential; first get the data and then try to generalize from it. He argues that the psycholo gists have passed from a descriptive interest in learning to an almost immediate preoccupation with some theoretical explanation of what is occurring within the organism. The early studies of learning-from the nonsense syllables of Ebbinghaus, through the problem boxes of Thorndike and the mazes of Watson, to the discrimination apparat uses of Yerkes and Lashley--always yielded learning curves of disturbing irregularity. But each investigator and every school of thought in psychology assumed that learning was an orderly process which could be described by their theories and mathematical models. Skinner has consistently pointed out that no degree of deviation in the various learning theories from the experimental data has led to any restriction on the elegance and complexity of the mathematical treatment. The strength and validity of the experimental, curve-fitting type of approach has been verified in numerous industrial studies of learning. In an industrial setting, the learning curve is often called a time reduction curve. It has been found that for widely differing companies, production system s, numbers and training of personnel, and products when the time required to produce one unit of a product is plotted against the number of units produced, an exponentially decaying curve is the result. Thue, Maynard, and Shappell, management personnel at Douglas Aircraft, have shown that such curves occur and have the same basic shape whether a few chairs are reupholstered in a home workshop, a multimillion dollar transport plane is assembled, the m ass production of over 400,000 units of a common industrial product, or a student has traced out a "stylus maze" several times during a repetitive test in psychology (76). Figure 5 shows the common shape of the curves. Considerable liberty has been taken with the vertical scale in order to show all the examples on one set of coordinates. The trend followed by all the curves may be represented by an equation of the form Y = aX , where "a" is the Y-intercept, and "b" is the slope of the curve when it is plotted as a straight line on log-log paper. The fact that the learning or development of skill on the part of the plant personnel can be represented by a straight line (on a log-log paper) makes it possible to extrapolate the time per unit for a very large number of units (Figure 6). All that is required is the data for two points on the line, some confidence in this type of function resulting from 48 G 3 U 0 £ ! P bo G U l> £ •rl U > C d i“ H a £ 3 C 12 10 8 6 4 2 0 t " — Reupholstering chairs \ V ^-Stylus maze(X 60) A ircraft production (X1/30 o o o ) /-M ass production indiicjtfv /Y 1 O T P i 0 20 40 60 80 100120 140 160180 200 Number of units produced Fig. 5 .--Tim e reduction curves, arithm etic scale. G 3 U a 0 £ S P bfl rt U G ! > • t H - i - J cd * ■ < E 3 u 111 M i l I J - X I 1 1 Reupholstering chairs Stylus maze - Aircraft - Production Mass production industry 1 2 3 10 20 100 200 Number of units produced Fig. 6 .--Tim e reduction curves, logarithmic scales. (Reference: Redrawn from Thue, H. W ., Maynard, B. I . , and Shappell, N. H. Time reduction curves. Proceedings 5th Annual Industrial Engineering Institute, University of California, Los Angeles, January 30-31, 1953, pp. 28, 29, 35.) 49 experience obtained in sim ilar situations, and sufficient flexibility to make changes in the function if new data warrants it. John G. Carlson (26) has called the time reduction curve "the improvement phenomenon"; Frank J. Andress (23) has called it "the learning curve." As people learn more and more about their jobs, no m atter how complex their tasks may be, the time it takes for them to complete it decreases steadily and at a decreasing rate. This is another way of stating that the level of m astery increases with time at a decreasing rate. Figure 7 illustrates the use of the cost reduction curves in the production of an airplane at Douglas A ircraft Company. The dashed lines represent the estimated cumulative average hours per unit and the individual hours per unit based upon a knowledge of the production time of the first and tenth units. Although the actual times for individual airplanes varies widely due to changing production conditions, the cumulative average time follows the prediction line with fair accuracy. If there were no changes in the production routine with time, the curves would be quite smooth. However, since in the real system there are usually changes in the jigs, the individual compon ents, the number and caliber of workers, the production rates,'" the management decision rules, and so forth, there occurs the phenome non of forgetting which is accompanied by a rise in the time reduction M a n hours per unit 50 80,000 Cumulative average hours per unit 20,000 Hours per unit 10,000 3,000 2,000 1,000 2 3 1 0 20 30 100 200 1 Number of units produced Estimated cost curves from actual data on first ten units Actual cost curves for first 200 units Fig. 7. - -Estimated and actual time reduction curves used in long-range planning and production of a ir planes at Douglas Aircraft Company. 51 curve. This is followed by drop in the curve due to learning or r e learning. no detailed understanding of the mechanism of learning in large industrial systems, there is considerable evidence that learning and forgetting can be characterized on a macroscopic level by rather simple mathematical functions. These functions are sufficiently reliable to enable management to use them for such jobs as pricing, planning, scheduling, and cost control. The functions are used as ■ managerial tools that are sharpened or refined as experience is gained with them. The learning curve in the shape of an S has been represented by a number of functions. The monomolecular autocatalytic reaction was first proposed for learning by Robertson in 1908 (39). His formulation was rewritten in a form appropriate to learning by Gulliksen in 1934 (32). When the equation is solved for x and names are given to the constants, the resulting expression is: The point that is emphasized here is that although there is be' At where: x is a m easure of learning t is a measure of practice A is a constant for the learner x = At c+e and the task b is a limit of attainment c is an empirical constant e is the base for natural logarithms 52 This mathematical expression implies that the gain per trial is proportional to the amount already learned and the amount rem ain ing to be learned before the lim it of learning is reached. A judicious choice of constants will yield a curve of increasing gains at first, and then the curve will show decreasing gains. Culler and Girden tried to apply this equation and the equation for a normal ogive to a large body of learning data (28). An example of em pirical curve-fitting is, given in Figure 8, which is taken from Hilgard's comprehensive book entitled Theories of Learning (7:371). Culler and Girden found that a number of mathematical functions will fit the data quite well. They concluded that the choice of a learning function must ultimately be made upon the basis of some rational learning theory. Finally, an illustration of a general formula that can be made to fit a wide range of data was presented by Woodrow in 1942 (7:372). His formula, I 2 (1 2 ~ 7 , ot -KL2 y = a + ^p +k -(1 -r ) contained five param etric constants but no underlying psychological theory. This formula did not survive as a general psychological law mainly because it lacked a theoretiqal foundation. The development of rational learning curves was pioneered by Thurstone in 1930 (45, 46). Thurstone's equations represented an attempt to provide a psychological rationale for the experimental 100 53 o 0 0 o sO O o C M cm o C M sO cm oo o C M 00 o sO C M 00 j§ .S o w tn |! - ' ■ S -3 m H J 2 3 g - “ ■ § H « Percentage of retention Fig. 8. - -The fit of an ogive to learning data for the learning of three-phase symbols obtained by Culler (7:371). 54 data. Learning was conceived to be a process of sampling from a population of acts, some correct and some incorrect, for any given task; the successful acts being retained and the unsuccessful ones having some fixed probability of elimination on each trial. In Thurstone’s equation, n a +bn In this equation, e^ represents cumulative erro rs through trial n, and a and b are constants. The param eter b is assumed to vary inversely with the total number of available acts, and therefore with the difficulty of the task. With these rather simple psychological assumptions, Thurstone was able to make some predictions from his equation in addition to curve-fitting. When he coupled these assum p tions with a small number of others using the differential calculus, there was a satisfactory correlation between his predictions and the existing data. The relationship between learning time and the length of task was obtained by letting s represent successful acts, and e represent erroneous ones. Then probability of success, p, is simply the proportion of successful acts: g p = ■ ■ ■ . and the probability of failure, q = 1 -p. S t 6 55 The learning rate can be described as the increase in the probability of success with tim e. Thus, the learning rate is pro portional to p, and the rate of elimination of erro rs is proportional to the "negative of q. The differential equations are: ds , ks , de , -ke T iT = kP = and - r - = -kq = dt F s -f-e1 dt H s +e * Thurstone also showed that the product of the erro rs and successes available at any one time is a constant, m. The difficulty or com plexity of a given task is directly proportional to m, where m = (s)(e). Direct integration of the differential equations yielded the following expression for the learning curves 2p -1 _ kt +z where: p = probability of success/tim e k = constant reflecting learning / p - p 2 { m ability m = constant reflecting task complexity t = time in the units over which the probability is measured z = constant of integration Thurstone’s theory was generalized to include the initial strengths of correct and incorrect responses, a constant for rewarding a correct response, and a constant for the strength subtracted by repeating and punishing an incorrect response. The work of generalizing was carried on by Gulliksen, and later by Gulliksen and Wolfle (32). 56 Another major contributor to rational learning curves was Clark L. Hull. In his book, Principles of Behavior (9), he ventured to make guesses as to the precise nature of a learning function which involved a large number of variables. Since the function was so complicated that he was unable to conceive of it directly, Hull broke it down into successive sets of simpler component functions. For the component functions, new intervening constructs were defined in term s of the independent variables. Additional intervening variables were then introduced by stating them as functions of the first sets of intervening constructs, until finally the dependent behavior variable was postulated to be a function of one or more of the intervening variables. his theoretical constructs, each of which could be experimentally manipulated, is shown by the following set of equations: t. t = time between response and m* * = Me reinforcement m '' = upper limit of m* sHr = M(1 -e"kw)e~;ite~u1:,a -e-jN) 1 = time between conditioned K and unconditioned stimuli M, k, j, u, i = experimental constants An example of the specific manner in which Hull introduced sHr = m(l -e where: sHr = habit strength N = number of reinforcements m = upper limit of various indices of sHr w = amount of reward or reinforcement m' = upper limit of m 57 It can be seen from the final composite equation that Hull's theory assumed that the habit strength (a measure of the degree of learning) rises with the number of reinforcements (repetitions) and the amount of reward (motivation); it decreases exponentially with both the time between response and reinforcement, and the time between conditioned and unconditioned stimuli. Based upon these assumptions and their corollaries, Dollard and Miller (10) have elaborated and extended learning theory based upon the Hullian model. This approach has been attacked and defended at length and constitutes the most widely discussed version of stimulus-response or associa- tionist learning theory. Its significance lies in the fact that it has led to a great deal of experimentation on the part of its advocates and critics alike. The resulting experimental data have yielded a generally accepted confirmation of the shape of the upper part of the S-shaped learning curve. In education, it is not sufficient merely to learn given m aterial; it is also important to retain the m aterial after direct learning stops. To a large extent, forgetting may be caused by interference from subsequent activity. The amount of forgetting (or retention, the converse of forgetting) may be measured by recall, by noting the amount saved upon relearning, by having the student re construct a scrambled list, or by testing his ability to recognize 58 studied m aterial. Figure 9 shows that different methods of testing or teaching objectives give different results. An illustration of the difference between methods is found in a study by Clark (27) in which 468 subjects, mostly college students, were presented with matched pairs of faces and names. Later they were asked to recall certain names and faces, and to recognize others. Recognition scores were much higher than recall scores for both names and feces. In general, different degrees of retention are measured by recall, recognition, and relearning. When a learner is asked to recall what he has learned, he is expected to rehearse exactly what was presented to him, i. e ., to recite memorized poetry or write definitions or reproduce a laboratory procedure. This is a far more stringent requirement than being asked to reconstruct a process whose steps are given to the learner. The use of multiple-choice tests is an example of measurement of recognition; the learner is given a list of possible right answers to a question and asked to recognize o r select the right ones. If he is not asked for the answers directly, but is given a chance to relearn the m aterial and his original learning is measured by how much time he saves the second time over the original learning, this method is known as relearning. It is important to note that most of the experimental studies 59 100 Recognition Relearning c o Reconstruction Written . reproduction 40 4 _) c < D o < u On Anticipation CO CD O —( ^ r-H C N C M Time Fig. 9. - -Forgetting as related to the method of measurement. ( (Reference: Luh, C. W. "The Conditions of Retention," Psychol. Monogr. 1923, 31, No. 142.) have dealt with the retention of very specific types of m aterial, such as nonsense syllables, poetry, manual skills, arithm etic facts, and language usage. In the classroom, over a time period of a sem ester, the student is expected to learn and retain a variety of skills, attitudes, concepts, precepts, definitions, facts, and so forth. In addition, these educational experiences are not repeated over and over again in an identical manner each time. Instead, they are usually presented at higher levels of complexity; with more compli cated examples and broader generalizations. Fortunately, the generalizations and extrapolations that have been made from the data obtained from highly specialized experimental studies to conditions in the classroom, have been largely substantiated by the data obtained from macroscopic long- range studies concerned with the retention of complex subject m atter contained in courses. A typical example of a comparative study of the retention of entire courses for periods of time up to twenty-eight months after the courses were taught is given in Figure 10. The general shape of the forgetting curves for all the courses shown is the same as that, for retention of lists of spelling words, nonsense syllables, or arithm etic facts. All the curves show an exponential type of decay. M aterial that has more structure and form (verbal m aterial) and that which 61 100 90 Grade School History High School Chemistry 40 Psychology College Zoology 20 College Botany 16 20 0 4 8 12 24 28 Months Fig. 10. - -Retention of one elementary school subject, one high school subject, and three college subjects from six teen to eighteen months after the end of the course. (Reference: Crow, Lester D ., and Crow, Alice. Educational Psychology, New Revised Edition, p. 304. Copy right 1963, by American Book Company.) 62 invites the learner to form his own concepts (which has higher levels of abstraction) shows greater permanence of learning than that which is meaningless or requires rote-memorization. In her book entitled, Teacher's Guide to the Learning Process, May V. Seagoe gives an extensive bibliography of the experimental work specifically related to the principles of learning in the classroom. She presents 237 references related to forgetting alone. These references indicate that, among other factors, learning and forgetting curves are related to "intelligence, age, sex, scholarship, rapidity of presentation of the learning material, type of material, serial position, sensory mode of presentation, guidance, number of repetitions, spacing of practice, reminiscence, and retro active inhibitions" (13:135). Although there exists a considerable body of experimental data which indicates the general effects that each of the aforementioned factors have upon learning and forgetting, no mathematical formulation exists which takes all of these factors into account even on a macroscopic scale. A summary of some of the implications and contributions of these factors (based upon a survey of the literature) will be presented at the end of this chapter. Practice carried on after the learner has reached m astery of the m aterial is of great value to retention. This additional practice is called overlearning. Krueger had a group of subjects learn lists of nouns until they could repeat the lists without any mistakes. He then tested the subjects for retention after 1, 2, 4, 7, 14, and 28 days. This condition is known as "no overlearning” and is shown by the lowest curve in Figure 11. The next higher line shows the percentage of retention when the subjects studied the lists for an additional 50 per cent of the time required to achieve the level of no mistakes. The top curve shows the results after 100 per cent over learning. The graphs show that retention is considerably increased by overlearning but that there is a point of diminishing returns. From a practical standpoint in the classroom, about 50 per cent overlearning seems profitable for retention. More than 50 per cent overlearning is usually uneconomical for most m aterials and most time intervals. An economic evaluation of overlearning is required to assess the trade-off between time (resource allocations) and level of mastery. This dissertation would permit a refinement of such an evaluation. Overlearning of lists of nouns has been extrapolated to complex aggregations of subject m atter such as courses o r topics. Students of educational psychology (19:429), particularly teachers, have generalized limited amounts of specialized test data for use in the classroom situation. At least quantitatively, such generalizations have been confirmed by classroom experiences. Thus, educationists Percentage o f original learning retained 64 100 80 60 40 20 0 0 1 2 4 7 14 21 28 Days elapsed since last practice Fig. 11. —Forgetting after different degrees of overlearning. (Reference: Krueger, W .C .F. "The Effects of Overlearn ing on Retention,” J :_^xper;_Ps^cholegy, 1929, 12:71-78.) - -L / / — 100% ove / — 50% ove jrlearning irlearning / V / — No ovei•learning II 1 I 1 65 make the generalizations that retention increases with the meaning fulness of the m aterial, increases with the intelligence and motiva tion of the learner, and increases as learning becomes more and more complete (as the top of the learning curve is approached). Learning theory models from other fields With the advent, in recent years, of greater mathematical sophistication in most scientific fields, greater communication between disciplines, and a growing awareness of the limitations of highly specialized knowledge in the solution of real complex problems, attempts have been made to apply scientific theories from one field to the solution of problems in another field. This trend has been carried on by the use of models or analogies. Hilgard (7:375) has stated that a model represents a series of relationships--mathematical, physical, conceptual--which appear to be appropriate for the understanding of some realm of data. There must be some kind of fit between the data and the model. The adequacy of the model can be judged by its success in ordering the data, and in making verifiable predictions from the data. A number of models have been introduced into the field of learning, most of them without substantial experimental programs to back them up. Some of the potentially useful models are: The stochastic models. - - A stochastic model is based upon probability mathematics rather than the differential calculus or the assumption of a particular form of the learning curve. It breaks down the learning process into a series of discrete events, each of which has a definite probability of occurrence for a given sequence. Research in this area is called mathematical learning theory and is concerned with deriving a body of theorems and formulas that follow from the assumption that learning on any one trial is described by a simple linear transformation of the response probabilities. Estes (5:139) has pointed out that these linear models are prim arily descriptive but may also serve as explanatory functions which account for the course of a process over a period of time by showing it to be a result of simple effects of variables operating on single trials. Pioneering studies in this type of approach have been made by Bush and M osteller, 1955; Burke and Estes, 1957; and Bush and Estes, 1959, Part II (3, 25, 2). Continuous research to improve learning theory with the aid of mathematical learning models has proceeded from the linear models and one-element stimulus sampling model of Atkinson and Estes (1), which describe the item-by-item behavior in the learning process; through the development of intuitive decision structures for the optimization of the selection of intuitive decision structures for the optimization of the selection of large blocks of subject m atter by Smallwood (19); to the application of dynamic programming to Markov 67 processes in order to develop optimum teaching procedures by Matheson (58). Although this type of research offers a rational approach to the learning of small, discrete pieces of information o r simple skills and is already useful in the development of programs of study designed for teaching machines, it is not as yet possible to use it to obtain optimum sequences for large blocks of subject m atter such as courses. The feedback model. - - As was mentioned earlier, in industrial and social systems there are cause-and-effect information - feedback loops that link decisions to action to resulting information changes and to new decisions. F orrester has stated that "an information-feedback system exists whenever the environment leads to a decision that results in action which affects the environment and thereby influences future decisions. " This definition is one that encompasses every conscious and unconscious decision made by people as well as those mechanical responses (decisions) made by servomechanisms (6:14). When the concept of feedback is applied to the learning process it is possible to conceive of learning as more than a trial - and-error process. It becomes an adaptive process in which there is a stimulus (input), a response (output), feedback of the response information in addition to any new stimuli as the new input, and so forth. Since the analytical techniques for the design, control, and testing of feedback systems are quite developed in the fields of cybernetics, control system s, and industrial management, it is possible that researchers with appropriate interdisciplinary back- grounds could make some significant contributions to learning theory. Thus far, with the exception of some input-output formulations of Ellson in 1949 (29), and a study of the effects of feedback on communication by Leavitt and Mueller in 1951 (33), very little work has been undertaken in this area. The information-theory model. - -This model was originated by Shannon and Weaver (14) in 1949 as a result of their studies in communication. Their theory suggests that psychological events can be understood through an analogy with the events that occur when a m essage is transm itted through an electronic transm ission system such as a long-distance teletype system. A communication system was considered by Shannon to consist of a source of the message, an encoder, a communication channel, a decoder, a destination of the message, and noise (static) that may enter the message during the process of transmission. In the analogous psychological model, the organism is conceived to be emitting signals which are to be received or inter preted by the observing psychologist. The m ajor areas in which information-theory has been helpful in the development of learning theory has been in quantifying the amount of data transm itted during learning; evaluating redundancy in messages which insures under- standability; and suggesting that the amount of information that is unavailable at the end of the transm ission system due to noise is analogous to the loss of available energy in thermodynamics. In addition, M iller expects that information-theory will find useful applications in learning with regard to processes oTdiscrimination, variability, and interferences between stimulus and response (35). This brief survey of contemporary model-building in relation to the development of learning theory is not intended to be comprehensive nor complete. Important models such as von Neumann and M orgenstern's Theory of Games and Rashevsky’s Mathematical Biology of Social Behavior have been omitted. Hilgard has pointed out that there is a family resemblance among all these models. They all deal with sequential behavior, uncertainty, and decision-making. While they are all influencing current psychological theorizing about learning, their contribution to classroom learning, optimum scheduling of topics or courses, and curriculum synthesis is still quite minimal. 70 A summary of relevant principles of learning The following summary of relevant principles of learning is based upon the references presented thus far, and in particular, upon the extensive bibliographies which may be found in the volumes of Seagoe and Hilgard, respectively. 1. Learning conforms rather universally to some part of the S-curve. 2. When there is little transfer from previous experi ence, the learning curve starts with zero slope and the slope gradually increases. 3. The rapidity of initial rise of the learning curve is a function of the previous experience of the learner in the given subject m atter, the type of learner (dull, average, bright), the type of subject m atter (from rote, meaningless, concrete m aterial to highly structured, meaningful, abstract m aterial). 4. In school situations there is a relatively large amount of transfer from previous experience and hence, there is a rapid rise and a slow leveling off of the learning curve. This implies that the inflection point occurs less than half-way up to the top of the curve. 71 5. Use or repetition of given subject m atter in other courses or learning situations is accompanied by ever-diminishing increments in the levels of m astery along the learning curve. 6. The level of m astery to which the learning curve becomes asymptotic (namely, the top of the curve) is directly proportional to the maximum number of time units allocated to the given subject m atter, and directly proportional to improvement in the type of teaching method used. 7. Plateaus or irregularities in the learning curve usually represent failure of motivation, failure to measure learning in fine enough units to show what is occurring, or new conditions that have been introduced into the learning situation. 8. Some forgetting may always be expected, but total forgetting never occurs. 9. The characteristic forgetting curve shows a rapid initial drop, followed by a gradual leveling off. 10. An exponential decay function can be used to represent most of the available data related to forgetting. 72 11. The average "half life" of the student’s m astery of many college subjects is approximately four months or sixteen weeks, provided that there are no repetitions or reinforcements after completion of the given course. 12. The slope of the forgetting curve decreases (or retention increases) as the cumulative time taught increases, with improved teaching methods, with superior learners, and as the number of repetitions of the subject m atter increases. 13. Reminiscence, a tendency for memory to persist or improve immediately after learning or practice has ceased, is a relatively transient phenomenon. Since it has been observed to last only minutes or hours after cessation of study rather than over a period of weeks o r months, it can be neglected when considering gross learning experiences from a scheduling or curriculum synthesis standpoint. Figures 12 and 13 illustrate these ideas and are based on the proposed fundamental equation which is given in the next section. In the following chapter an initial attempt will be made to Level o f m astery, .2 .0 .8 6 .4 M = 1 2 0 - S = 1.002 - S = 1. oor M = 1 0 20 40 0 20 40 Time (weeks) Time (weeks) "M=l. 2 - M = l ■ M=.75 S = 1.003 0 40 20 Time (weeks) Fig. 12. --Typical learning curve trends for variations in the type of learner, L; type of subject m atter, S; and type of teaching method, M. < i co Level o f mastery Level o f mastery Level o f mastery 74 2 .0 R = 10 .8 . 6 .4 .2 0 Learning Forgetting Rate of forgetting decreases with reinforcements .2 .0 .8 6 ■ .4 .2 0 Increase of mastery decreases rate of forgetting .2 .0 ■Massed learning .8 .6 .4 Distributed learning .2 0 time (weeks) Distributed learning yields higher final p Fig. 13. - -Final level of m astery as a result of learning- forgetting interactions. utilize the foregoing ideas in the development of a suboptimum sequencing model for subject m atter. CHAPTER VI DEVELOPMENT OF A GENERAL MATHEMATICAL LEARNING FUNCTION Specification of variables and aggregate param eters The general trends predicted by the learning theorists with respect to learning and forgetting will be utilized in the formulation of a generalized learning function. In view of the large number of interacting variables connected with learning, and in view of the lack of any general theory of learning which is acceptable to most of the practitioners in the field, the present attempt at a mathematical model will involve the grouping of many of the variables into a small number of lumped (aggregated) param eters. The degree of m astery of a given course, topic, or subject m atter item, will be expressed as a function of the lumped param eters and a number of other pertinent variables. As a very gross initial approximation we can state that the degree of m astery (henceforth known as the educational potential, p) is a function of the type of subject m atter, S; the type of learner, L; the type of teaching method, M; the cumulative learning 76 77 time, tj^; the forgetting or decay time, t^; and the number of repetitions of the subject m atter, R. Thus, p = p(S, L, M, tL, tD, R) Since these are the fundamental param eters to be used in this model, a more detailed explanation of their meaning is now given. p - - The educational potential represents that level of m astery that the student has achieved at a given time t, relative to an initial base p. The base p is usually taken to be zero for the purposes of this model unless sufficient em pirical data regarding the student's initial knowledge of the subject area in question is available. It is reasonable to assume as a first approximation that when a new subject is introduced to the student, his level of m astery is very low and approaches zero in the limit. For a given schedule, the educational potential is directly but nonlinearly proportional to the cumulative learn ing time. We m easure educational potential in the same units as we use for time; namely, weeks or hours. In trying to determine how much a student has learned in a given course we normally say that "he has had four weeks of Algebra. " This does not imply that the student has mastered four weeks' worth of Algebra. Rather it means that if we know the learning time we can estimate the level of m astery by means of the appropriate learning function which relates p to t. - The type of subject m atter can be broken down into any number of arbitrary categories. For instance, the Engineering Science Curriculum at Dartmouth College (77) has been broken down into the following categories: analytical technique, concept, definition, engineering device, experimental technique, factual data, instrument, law, precept, principle, and special illustration. Another method of categoriza tion breaks the subject m atter into various levels of complexity; from extremely simple, concrete, illustrations to highly abstract and generalized m aterial. From a curriculum synthesis viewpoint, the subject m atter can be characterized by the importance that the faculty places upon it. The percentage of time 79 in the entire curriculum devoted to a given item by the faculty is an explicit measure of a number of implicit factors. It includes a faculty consensus regarding the total time that should be allocated to this subject m atter commensurate with the educa tional objectives of the school, the available re sources, and the time that students are available in the classroom. It also includes the faculty's estimate of the direct amount of time required to teach the subject under average conditions and the indirect time to be devoted to the repetition of the subject m atter in other courses or school activities. Thus, the type of subject m atter, S, is a direct function"of the total time devoted to it by the faculty. This total amount of time shall henceforth be designated by the letter, H. L - - The type of learner can initially be categorized into three groups: a fast learner, an average learner, and a slow learner. Students may be placed into these general categories by using a composite score derived from such data as I. Q. tests, College Entrance Examinations (SAT), cumulative grade 80 point averages, general ability profiles, counselors' recommendations, and so forth. For the purpose of this model a standard learner is defined as that student whose composite score is equal to the average composite score of all of the students in the educa tional institution. In this model the value assigned to a standard learner is given by L = 1. All other values of L are assigned relative to this value. M -- The methods of teaching can be characterized in many different ways. The following approach is presented here m erely as an illustration of the complexity of this area. Arnold Roe has stated (73) that the corpus of knowledge on teaching methods can be viewed in three dimensions: (A) relation to the structure of the specified course content, (B) relation of the structuring to the students and teachers, and (C) relation to the behavioral aspects of the students. Some of the possible elements along each dimension are illustrated in Figure 14. In private discussions with Stephen Abrahamson it was concluded that a fourth dimension may be added; namely, the behavioral aspects of the teacher. Individual study !_______ • _ Tutorial I------------ Lecture, small class----------- - j Lecture, large class!------------1 ! Discussion group j---------- | Instructor less group;-----------I- "' (A) Structuring of students and teachers — Feedback — Active participation — Motivation — Guidance — Satisfaction — Practice — Social aspects (C) Behavioral aspects of students y [ _ _ ............ * -----i Discursive ! ! Case method Socratic ----------- Problem oriented (B) Structuring of course content 0 0 82 Theoretically interactions between elements in one dimension with elements in other dimensions should lead to the filling of each celi with a number rep re senting a utility for the particular intersection of elements. Utility numbers are, in general, difficult to ascertain. In the experimental literature on teaching methodology statements are usually made regarding the elements in one of the dimensions only. Dimension (C), which is related to the behav ioral aspects of the students, has been summarized by McKeachie (34) and Tyler (44). The other dimensions have been discussed by Roe in the reference cited earlier. Gage's Handbook of Research on Teaching, a basic reference on teach ing methodology, cites other models and approaches to this complex area. It generally confirms the need for improved quantitative analyses of teaching methods. S' For the purpose of this model the teaching param eter, M, will be defined operationally in terms of its effect upon the level of m astery of the subject m atter at a given time relative to the level that would be achieved under standard conditions. Standard conditions are defined as those conditions that prevail at the time that this model is implemented in the school. In this sense, M represents an index which is equal to one (1) under standard conditions and is greater or less than one (1) depending upon whether the teach ing methods in general are better or worse than those considered under standard conditions. This approach is not as unprecise as it may seem at the outset, Although absolute values to describe teach ing methods cannot be obtained at the present time, relative values can be estimated and verifications of these estimates can be made with a minimum expend iture of time and resources. For the purpose of distinguishing between different schedules or sequences of subject m atter, it is sufficient to use values of M like 0. 8, 1.0, and 1. 25. These numbers could represent substandard, standard, and superior teaching methods, respectively. - -The variable t^ represents the cumulative amount of time that a given subject has been studied in the class room. It includes the estimated time that the subject 84 m atter has been reinforced or used in other courses or subject areas. In keeping within the precision obtainable from this model and the precision of the available data, time shall be measured in weeks, rather than in hours or some shorter time unit of measurement. R - - The variable R represents the total number of times that a given subject has been taught or reinforced up until the time under consideration. The total number of times includes the initial time that the course was introduced and each succeeding time that the course was used in other courses. In order to be considered a repetition a course may not be taught or used for less than one week, t^ - -The decay (or forgetting) time occurs when the given subject m atter is not being taught nor used. It is assumed that no forgetting occurs while the m aterial is being taught or used: forgetting occurs at all other times. A general learning function and its properties A general function which satisfies all of the conditions given 85 at the end of the previous section, which is dependent upon the param eters stated above and which reflects data from areas other than engineering (since none was available for engineering), is now presented. -Lit -At ^ " ^ L p = MH(1 -S )S acD where: A = ^L R B = an empirically derived constant related to initial forgetting t£= 0 during learning or relearning tj-j> 0 during forgetting. Ambiguities with regard to the independent variable t can be eliminated by writing one equation for learning and another for forgetting: -Lt^ -At PL = MH(1 -S ^ ) and pD = pLS The t used in the learning equation is the same as t^; while the t used in the decay equation is the same as t^. These equations satisfy the previously stated conditions of learning. Some of the conditions can easily be stated mathematically and immediately verified. Others can be recognized intuitively or by substituting numbers into the equations. The slope of the learning curve at time t = 0 is zero. lim dp _ n t-*0 dt The slope of the learning curve approaches zero as t-* o o . lim ^ dp _ n t - * - o°dt The inflection point occurs at less than half the maximum ordinate on the learning curve. It occurs where the second derivative is zero. The value for this case is the constant, 0. 40. This result implies that the inflection point will always occur at approxi mately 40 per cent of the maximum ordinate for this function. This value is reasonable and will be accepted until experimental data indicates that it should be different. The learning curve becomes asymptotic to some maximum value of p under standard conditions. We can determine the relationship between the param eters S and H by asking the following question: For an average learner (L = 1), and for the standard teaching method (M = 1), what is the time required to reach 99 per cent of the maximum p? What is the corre- sponding S? The maximum level of m astery p at standard conditions can be achieved by the student at the completion of all the weeks of learning and r e learning that are available in the given schedule for the given course. The maximum available weeks is represented by H. -Lt 2 p = MH(1 -S ) at t = H, p = . 99H 2 . 99H = MH(1 -S "LH ) c -LH2 _ . .99 b T v T But at standard conditions, M = L = 1. Therefore, -H 2 S n = 1 - . 99 = . 01 H2 S = 100_ This result shows that in this model the subject m atter param eter, S, is a function only of the maximum weeks available, H. Once the faculty has fixed the total hours that are to be allocated to a given subject, the value of S is determined. After the model has been tested in a number of school situations and experimental data obtained, it will be possible to inquire into validity of the faculty's time allocation to each subject with a higher level of confidence. The quantity inside the parentheses in the learning equation represents a function which varies from zero to one (0 to 1) as learning time varies from zero to infinity (0 to oo). This function gives the desired S-shaped curve. The maximum ordinate, under standard conditions, is obtained for p by multiplying this function by the maximum weeks available, H. If an improved teaching method is used, the level of m astery at any given time t will be higher than under standard conditions. The param eter M serves as an index of improvement; when it is greater than one (1) it serves to stretch out the learning curve and the maximum ordinate becomes higher. The curvature of the learning curve, K, at the start of learning (t = 0) is a measure of how rapidly the educational potential will rise. When the curve has a large curvature it rises rapidly; less curvature indicates a slower rise in educational potential. The curvature (which is the inverse of the radius of curvature, R) can be found with the help of the following formula from the differential calculus: The first and second derivatives at t = 0 are: ijE = 0 and ^ = 2LMH(lnS). Substituting these values into the equation for K, we find that , where p = MH(1 -S -Lt 2 3/2 K = 2LMH(lnS) = 2LMH(- InlO C X _ 9. 22LM H2 H 1 since S = 100 This result shows that the learning curve rises more rapidly at the start of learning if we have a better 90 learner, L; an improved teaching method, M; and a type of subject m atter that is deemed initially by the faculty (and later from test results) to require less exposure or learning time. This last conclusion results from the fact that S varies inversely as H. 7. Total forgetting occurs only after an infinite amount of time after learning of a particular subject has ceased. PD = PLS ~At = PL(100) ' At/H 2 As t-» o o , p -^ 0 . 8. The slope of the decay curve depends upon S and the exponential coefficient A. It can be seen that as S r decreases (or as the cumulative learning time allocated by the faculty increases), the rate of decay decreases. This means that the longer it takes to learn an item, the longer it takes to forget it. The major factor that influences the rate of forgetting is represented by A. As the cumulative time that an item has been studied approaches the maximum time available, H, the value of A (and hence the rate of forgetting) decreases. Forgetting occurs more slowly for better learners, improved teaching methods, and an increasing number of repetitions of the m aterial. The param eter, B, is introduced to enable us to adjust the points generated by the decay function to the data points obtained by testing forgetting in an actual school. This idea can perhaps be clarified by an example. In a school which uses a system of two sem esters per year, three hours per week for lecture and laboratory courses, the average half life of the educational potential for a given course might be found by testing to be sixteen (16) weeks. The value of B could be calculated as follows: H -BtT _ ^ £ nnnN H^MLR D p = pLS D = PL(100) at tp = 0, p = p^, and at t^ = 16, p = . 5p^. For a sixteen-week sem ester, one repetition, and standard conditions, 92 t L = H = 16 and R = M = L = 1 Substituting these values and solving for B, we get: B = . 8495 = .85 (0 < B < 1). In another school, where a trim ester or quarterly system is in operation, the value of B would probably be different. It should be noted that B may take on any value between zero and one; the limiting values are specifically excluded. 10. The composite equation shows that after a given course is completed and overlearning occurs due to its use in other courses, the top of the learning curve is approached with diminishing increments in the level of mastery. However, the major contri bution of overlearning or repetitions of the m aterial lies in the fact that the m aterial is retained at a higher level over a longer period of time. The equation shows that as the cumulative learning time increases, together with the number of repetitions, the slope of the forgetting curve decreases. 11. Additional assumptions, - -(a) When the m aterial from a given course is consciously used in another course, 93 the educational potential rise s along the learning curve of the given course as though the given course were of a longer duration. If there has been an intervening time of forgetting, the total drop in potential due to forgetting is subtracted from each point on the extended portion of the learning curve. This assumption is used in the model for the following reasons: (1) During the time period when a given course is originally taught, there is very little over learning of individual items of subject matter. Individual items are overlearned when they are presented from new perspectives or in different contexts in later courses. Such reinforcements have less and less effect upon the student's educational potential in the given course as the cumulative learning time (including reinforce ments) increases. (2) When learning of a given course is resumed in other courses after a period of forgetting, the item s that are relearned or over learned usually represent a small percentage of the items in the 94 given course. Recall of these items is almost immediate, of an order of magnitude of seconds (at most, minutes). Since time increments in the model are measured in weeks, any initial flat portion of the new section of the learning curve can be ignored for all practical purposes. If all of the m aterial in the given course was to be relearned in another course, the time required for recall might be long enough to be significant. In general, this is not the case. (b) When the m aterial from a given course is reinforced simultaneously in more than one other course, the educational potential is incre mented by the product of the increase in p for one course and the number of reinforcing courses. Thus, for example, if after course A was com pleted, courses B , C, and D each used or illustrated m aterial from course A for a period of one week, the educational potential of course A would be increased by three times the incre ment for just one of the reinforcing courses. The reasoning behind this assumption follows. As the number of reinforcements of a given course or topic increases from one to about five or six during the same time period, it might be expected that individual contributions of the rein forcements to the total educational potential of the given course would decrease. Each additional reinforce ment would carry less and less weight. On the other hand, the additional rein forcements tend to give the learner new perspectives, broader experience, greater confidence, and higher motiva tion with respect to the given course m aterial or topic. This process tends to cancel out the effects of point 1. In the absence of any empirical data to the contrary, the assumption of equal weight for simultaneous reinforcements-is reasonable. If a comprehensive testing program shows that this assumption should be modified, this can be accomplished with 96 little injury to the overall integrity of the model. The interaction factor, I The composite equation for the educational potential presented thus far represents an "additive m odel." This implies that the param eters M, L, and S are independent of each other. It is, however, more reasonable to expect that in a process as complex as learning these param eters are functionally dependent upon each other. We would expect some mutual interactions. A simple example will be used to illustrate the problem. Let S, the type of subject m atter, have two categories: and S2 L, the type of learner, have: L^, L ^ , and Lp M, the type of teaching method, have: Mp M^, and If the educational potentials of the students were to be actually measured at the end of a course of study, the experimental data due to mutual interactions might result in graphs sim ilar to those shown in Figure 15. 97 M M M T M , M . M T Teaching Method Teaching Method Fig. 15.--Interactions between param eters S, M, L. For subject m atter a low learner L^, taught by the method of teaching machines M^, might yield a high educational potential p and much lower p's for the method of independent study Mj or a lecture method M^. On the other hand, an entirely different graph might be obtained for a high learner Lpj. In order to compensate for the effects of interactions for which we do not as yet have an acceptable theory or explanation, we shall multiply the equation of our model by an interaction factor, 1 . In a manner entirely analogous to that used in engineering to define efficiencies, flow coefficients, friction and safety factors, we define the interaction factor as the ratio of the empirically-determined p and the p which is calculated from the model. 98 1 = 1 (M, L, S) = Interaction Factor = p ^ ^ e l ) , _ Educational potential from experimental data with interactions Educational potential from the model without interactions The composite equation for the educational potential for one particular course now becomes: 2 p = IMH(1 -S “LtL )S "AtD The model can be generalized for m courses in the j-th schedule. m m - « . p a - P. = p. = i .m .H.(1 -S. “V d S. i D ] i l r i ' i i=l i=l If there are n possible schedules, the next task is to find the schedule with the highest P. Thus, we m ust maximize P over schedules j = 1 through j = n. We turn our attention to this problem in the next chapter. CHAPTER VII A HEURISTIC METHOD FOR SUBOPTIMAL SCHEDULING The combinatorial problem The problem of developing optimum or suboptimum patterns of courses (which we call schedules) hinges upon the fact that for a relatively small number of courses, there are an extraordinary number of possible arrangements. For n courses there are C = n!/(n-r)!r! combinations of courses taken r at a time. Thus, n r ' ' ' if there were only forty (40) courses, there would be 658,008 possible schedules. However, not all of these schedules are really different from each other and many others are not acceptable because they do not satisfy constraints such as those for prerequisites and maximum hours of classes per week allowable. Even if symmetry and the constraints were to reduce the number of schedules by a factor of 10, the problem would not be appreciably simplified because the total number of schedules to be investigated would still be large. It is not possible even with the most rapid data-processing 99 100 equipment available to investigate and compare all the allowable schedules that are possible and to find the one which maximizes the objective function, namely, the educational potential P. What is needed is a technique which can generate and evaluate a relatively large number of possible schedules without a tremendous expenditure of time and money. Review of other methodologies A search of the literature for heuristic methods resulted in the conclusion that although numerous "rules of thumb" are used in the solution of large complex problems, no general approach to heuristics is as yet available. Donald W. Taylor has presented a comprehensive overview of theoretical problems in the realm of "thinking. " He included under that term such diverse processes as problem-solving, decision-making, and creativity (19). His paper discusses computer programs that have been written in recent years employing heuristic processes and simulating important kinds of human thinking. In particular, the work of Newell, Shaw, and Simon in developing programs like the Logical Theorist and the General Problem Solver, are cited. Fred Tonge has described a heuristic program for the balancing of assembly lines in 1961 (20). In some of his published work related to industrial management problems, Alan J. Rowe has discussed the necessity of a heuristic approach to 101 the problem of scheduling (40). In general, the usual approach to the combinatorial problem in scheduling involves the imposition of successive sets of realistic (and ofttimes unrealistic) constraints which reduce the number of possible solutions to one of manageable proportions. Then, using intuitive judgement based upon experience with sim ilar problems, a likely candidate for the solution is selected from among the remaining schedules or solutions. Solutions obtained by this procedure are usually workable but not necessarily optimal or even suboptimal. In order to solve the scheduling problem at hand, we seek an algorithm. An algorithm is a process which has the very valuable property that if the problem has a solution, the process will, sooner or later, produce it. Furtherm ore, we seek an optimum or at least a close-to-optimum solution. An algorithm which would solve the scheduling problem is one that requires the investigation of every possible schedule, the calculation of a total educational potential for every allowable schedule, and then the selection of the schedule having the highest value of P. This algorithm would work but if a high speed computer were to calculate one million P's per second for eight hours per day for a schedule with only twenty courses, it would take the computer about 250,0.00 years to complete the problem since it would have to investigate twenty factorial (20!) schedules. 102 Therefore, we also seek a heuristic algorithm; namely, a process for solving the problem which, if followed, may aid in obtaining an optimum solution but offers no guarantee of doing so. Fortunately, a heuristic algorithm of this type is available. In 1962, Gordon Armour presented a new methodology for determining suboptimum relative location patterns for physical facilities. This "Computerized Relative Allocation of Facilities Technique" (CRAFT) enables management to minimize incremental costs which vary with changes in location patterns of departments in a manufacturing plant (24). In a production system which uses a process controlled, functional, or "job lot" layout, equipment of common generic type (lathes, milling machines, welders, e tc .) are grouped together. Parts and products flowing through the system then proceed along routes dictated by the sequence of operations to be performed. These routes vary considerably for different parts or products so that no one choice of relative location of departments is best for all parts. Just as in the case of the scheduling of courses, the enumeration of all possible layouts is a practical impossibility. The CRAFT algorithm produces suboptimum solutions to the layout problem by successively exchanging each department in location with every other department, evaluating the incremental costs resulting from each exchange, and retaining only those layouts 103 that incur lower incremental costs. A solution has been reached when further exchanges of departments do not result in any reduction in the incremental costs. This heuristic algorithm which is imple mented by means of a large computer program produces layouts that are far superior to those produced by other methods, even though they are not necessarily optimal. A heuristic algorithm r The central idea upon which the solution to the scheduling problem shall be based is taken from KRAFT. By exchanging course locations rather than department locations, it is possible to generate a progression of different schedules and to systematically converge on one of a number of suboptimum solutions. These solutions are not as surely the best ones as the answers to linear programming are, but they do represent solutions which are far better than those currently available and solutions which cannot easily be improved upon. The algorithm that will be used in this model follows. 1. Compute the educational potential, P., for the first allowable schedule, j. (This is usually a schedule recommended in the school catalog.) 2. Calculate the value of obtained by exchanging the location in the schedule of each course with every other course in a non-redundant manner. Compute Pj values only for those courses that satisfy all of the school constraints. After each exchange, compare the value of obtained with the previous one. Retain the larger value. If there are n courses and they are exchanged r at a time, there will be " N i p T r J l "1){" ' r ! ' ' ' ^ " +1> exchai« e s- After all exchanges have been completed, print out the schedule having the highest P ^ (namely, the last schedule retained) and associated data. Compare the latest value, P. with the initial value, P .. If P. is J 1 J higher than P^, go back to step 2 and begin the next iteration. If P. is equal to P^ (there has been no increase in P.), continue at step 4. Compute an ideal value of P ^ for schedule J and an efficiency based upon the best schedule. Print the final schedule and associated data. Stop. A suboptimum schedule has been reached. The computer program The heuristic algorithm has been formalized in a computer program which is briefly presented here. The program contains eighteen subroutines and over 700 cards in the source deck. The program is written in Fortran IV and has been run on the Honeywell 800 computer. At the present time the program is capable of scheduling over 1 0 0 different courses and handling more than 2 2 0 time increments. The sm allest time increment is usually taken over a period of one week. This implies that the program can be used to schedule courses during every single week of a four-year curriculum. The essential logic of the program is shown in broad outline in the flow diagram, Figure 16. A brief description of each of the subroutines that are numbered in the flow diagram will aid in inter preting the entire computer program. 1. The entire input to the program is read from individual sets of data cards that can be stacked one after the other following the source deck. The input cards contain data regarding the type of format to be used in printing the schedules; the names, starting times, durations, and hours per week of all the courses; estimated reinforcing hours for each course; data regarding which Print errors in data (n) if any; then read data (n-fl) — - p s ------ Validation routine for input data and initial sched. ~ n ............. Build array for locating courses in the schedule F Develop initial schedule and its copy f 2 Initialization Read data (n) Dimensions Print: Input table and initial schedule ~ r ~ Store highest P; update appropriate 14 schedules T Non-redundant selection of courses I&J for exchange Last /ex ch . this ^ iteration ompute- print P(j) for this schedule ■ * ~ L Print: Schedule (j), all P(j)’s for this iteration Pre requisites violat ? 'es Check constraint: Prerequisited for I&J 13 Undo the exchange of I&J in the schedule * Max. yhrs. perv Check constraint: Maximum hours of classes per week for I&J ' week 12 ok \ y ^-------------y No 11 i 1 Initial checks if courses I&J may be exchanged r Exchange locations of courses I&J in schedule Compute Ideal P and efficiency for final schedule .Fig. 16. - -Flow diagram of the computer program. o ON courses shall remain in a fixed location; course param eters; prerequisite and corequisite infor mation. The program will then utilize the starting time and duration of each course to develop an initial schedule which will be printed out. For the purpose of keeping track of each of the courses in the schedule and to reduce the computing time, a subroutine has been developed which locates each course by its row and column in the schedule. The initial input data and first schedule is inspected to determine whether there are any major discrepan cies in the input. This routine will determine whether the prerequisites for each course in the initial schedule are met. It will also determine whether the maximum number of hours per week allowable have been exceeded during any time interval in the initial schedule. If any discrepancies in the initial schedule or input data arefound, the exact e rro r will be printed out and the program will return to the beginning and read the next set of data. If there were no discrepancies in the original data set, then a table of the initial input and a copy of the initial schedule will be printed. This subroutine computes a total P or educational potential for the given schedule by: (a) Dividing the schedule into small time incre ments such as one, two, or four weeks. (b) During each time increment an increment in P is calculated for each course that exists during that time period. The sum of all the P increments for each course is obtaiffed by adding the increment in P to the previous total P at the end of each time interval. During those time intervals when forgetting occurs, the increments in P are negative numbers and are subtracted from the current total P for the given course under consideration. (c) The total P for the initial schedule is compared with the previous total P which at this point is zero. The higher value is stored by subroutine 14 and the program then proceeds to item 8 . 8 . This subroutine contains a triangular matrix which provides for the non-redundant selection of two courses to be exchanged in location in the given schedule. Since there are n courses taken two at a time, it can be seen that by setting r equal to 2 in the previously given formula, the total number of possible exchanges for one iteration is n(n - l ) / 2. Thus for twenty courses there would be 20(20 -l)/2 or 190 individual exchanges. If the exchange under consideration is not the last one of the iteration, the program proceeds to item 9. 9. In this subroutine preliminary checks are made to see whether an actual exchange is advisable. Two courses may not be exchanged if their starting times are equal or if their durations are unequal. If these two conditions are not satisfied, then the program returns to item 8 and two new courses are selected for possible exchange. 10. If the two courses pass the initial checks, then the exchange routine physically exchanges the two courses in location in the schedule and also exchanges their starting times and locations in the locating array. 11. If a course that is given for three hours per week is exchanged with a course that is given one hour per week, it is possible that in its new location the three-hour per week course may cause the maximum number of hours per week that are allowable to be exceeded. If this occurs, the program goes to subroutine 1 2 . 12. This subroutine reverses the operations that were carried out in subroutine 1 0 ; it returns the two courses to their form er position in the schedule and then the program proceeds to subroutine 8 where two new courses are selected. 13. If the maximum hours per week constraint is not violated, the program continues to subroutine 13 where the prerequisites of the two courses in question are scrutinized. If there have been no violations of the prerequisites or corequisites of either of the two courses that have been exchanged, then the program goes to subroutine 7 where a P for this schedule is calculated. However, if the prerequisites are violated, the program proceeds to 12 and ultimately to 8 where new courses are I l l selected for exchange. 14. After a total P has been calculated for a given schedule, it is compared with the previous total P and the higher value is stored. In this routine appropriate tables and schedules are updated, depending upon which P must be remembered. 15. If the last exchange of this iteration has been completed, the program goes to subroutine 15. The latest schedule is printed, together with all of the P’s for this iteration. 16. The last P that was calculated is compared with the first one of this iteration and if there has been no improvement, namely, they are both equal, the program proceeds to item 18. If on the other hand there has been a change in P, the program returns to subroutine 8 and the triangular matrix is initiated from the beginning. The program will then carry out another entire iteration and continue in this way until there is no improvement in P during an entire iteration. 17. In this routine an ideal educational potential is computed for the final schedule. If the 112 reinforcements for a given course could be scheduled at any time that we wished, we would attempt to schedule them as soon as possible after the given course has been completed. This would mean that the student would learn all of the course ' m aterial continuously without any intervening periods of forgetting. Under these conditions, when all of the reinforcements have been completed, the rate of forgetting will be the minimum possible rate due to the fact that we have scheduled the maximum number of reinforcements (repetitions) and forgetting occurs only after there has been the greatest possible amount of overlearning. This calculation leads to a possible but highly improb able total educational potential which we classify as the ideal. This routine also carries out the calculation of an efficiency which is defined as follows: F ffir io n r v - P< final) 'P(initial) Efficiency - p^ideal) -P(initial) This efficiency represents one possible standard by which to judge the relative superiority of the 113 schedule calculated by the model and the schedule taken from a school catalog. 18. This subroutine inquires whether there are any new data sets that have not as yet been processed. If there are no new data sets left,-the program stops. The program will also stop if there are any contra dictions in the final set of data. Additional computer program capabilities 1. By changing a single "dimension" statement in the source deck, it is possible to increase the number of scheduled courses from 1 0 0 to 2 0 0 , or to increase the number of time intervals from 220 to 440 without requiring the use of a larger computer. 2. Any number of courses can be kept in fixed locations in the schedule; they would never be exchanged in location with any other courses. This feature is very helpful in those cases where it is decided to give certain courses at particular times. 3. The program can rapidly compute a value for the educational potential for individual schedules that are to be compared. For instance, it might be desirable to know whether a course in Statistics should be taught to engineering students in the freshman year or the junior year. Both types of schedules can be evaluated in less than a minute of computer time. 4. Schedules with courses of different durations can _ be handled by the program with no special instruc tions or changes in the source deck. 5. Changes in the function for calculating P, which would result from future research and field testing, can be made in a few minutes by retyping a few cards in the source program deck. Such changes in the. objective function will not affect the opera tion of the rest of the program. ^ The complete computer program is available at the University of Southern California-Honeywell Computer Center. In the next chapter, we will consider statistical methods for validating the model. CHAPTER VIII STATISTICAL EXPERIMENTAL DESIGN In order to test predictions that can be derived from the sequencing theory and computer program, a criterion measure of P, the education potential or degree of mastery, is needed. A number of measures that could be used to evaluate P are already available and are now being used as a basis for graduate school admission (i. e . , _the Graduate Record Examination). For evaluation of professional and educational m astery in the legal and medical professions, there are standardized tests developed by the Educational Testing Service. Several parallel forms of such a test, appropriate to the student's field of specialization, are needed so that students upon entry into the program and after each sem ester can be tested. This procedure is necessary so that the prediction of the model regarding the effects of sequencing on P can be evaluated. Specifically, students who are scheduled by the optimal sequencing program should show significantly higher P's than students scheduled by the school catalog. Repeated measurements of the students, utilizing parallel forms of the 115 116 m astery test, will provide data to evaluate specific predictions of the sequencing model regarding the trend of the learning o r m astery curve obtained by plotting P through time. In addition to the sequencing factor, the model mathemati cally relates a number of additional educational factors to the effects of sequencing such as ability of the learner, type of subject m atter, etc. (see Table 2). As was mentioned in an earlier chapter, one is led to expect a number of important interactions between these factors and the effects of sequencing. This expectation is anticipated by provision in the model for the effects of interaction (interaction param eter I) and should be evaluated statistically. Stated in statistical term s, research into the implications of the proposed model requires a multivariate experiment utilizing an analysis of variance repeat m easures design (21:298). This will provide statistical tests for the main effects (the significance of the differences between the levels of any one factor), interaction effects (the significance of the difference of the students' criterion scores based on groupings involving two or more factors) as well as an analysis of the trends that may be manifested in the repeated measures factor. The data regarding interactions can provide clues to estimate the values of the interaction param eter in the sequencing model. In addition, the trends in the learning curve for the repeated 117 TABLE 2 FACTORS CONSIDERED IN THE SEQUENCING MODEL F ac tor Description Levels Possible Experimental Identifications A Sequencing a^ optimal a2 suboptimal catalog Based on sequencing model,ideal Based on sequencing model, expected Based on college catalog a sub, sub- n ’ B Character- b^ fast istics of the learner, L b^ medium b^ slow Based on sequence actually fol lowed by the student(s) College entrance score: 1.) SCAT 2.) SAT I. Q. 1.) P. M. A. 2.) BINET 3.) WAIS 4.) College Entrance Boards C Teaching n c^ independent study C 2 lecture c^ laboratory D Subject m atter c^ teaching machines d^ mathematical Algebra, Calculus d9 applied Laboratory problems d3 E Subject m atter m astery over time n n Repeated measurements in par ticular subject m atter areas with parallel forms of a stand ardized subject m atter achieve ment test--end of sem ester 118 measurements can indicate what changes are needed in the model regarding the mathematical properties of the learning curve. Evalu ation of the degree of fit of the predicted learning curve and the empirical data can also be made using appropriate "goodness of fit" tests such as chi-square. This proposed design can be extended or shortened to include as many factors as time and money permit. In addition, the design can incorporate analysis of covariance (21:578). This statistical tool provides for a statistical control of differences between subjects that are not amenable to experimental control for some reason. For example, students may differ significantly in their level of knowledge in their m ajor subject of study (i. e . , engineering) when they enter the college program. For practical reasons there may be no way to form experimental groups that are homogenous with respect to knowledge in the area, and groups that differ significantly in their knowledge. An analysis of covariance design provides an analysis of the data corrected for differences in some factor that could not be> - experimentally controlled, provided that there is some measure of the factor. CHAPTER IX AN APPLICATION OF THE COMPUTERIZED SCHEDULING MODEL TO THE LAST TWO YEARS OF COURSES IN AN ENGINEERING SCHOOL The computerized sequencing model has been applied to the problem of finding improved schedules for the last two years of the engineering curriculum for a mechanical engineering student at the California State College at Los Angeles. A typical schedule which is recommended in the college catalog was used as the initial schedule in the computer program. This schedule is shown in Figure 17. Every line in the schedule represents the courses studied by the student for a period of one week. Each sem ester is sixteen weeks long; there are two weeks between sem esters; the summer vacation lasts for eighteen weeks; and graduation is assumed to occur two weeks after the last final examination is given. Since only engineering and mathematics courses were considered for scheduling, it was assumed that a maximum of thirteen hours per week of these technical courses are allowable. In 119 I N I T I A L S CHE DULE .TIM E INCREMENTS o f 1 WEEKS E LE CTR IC AL c i r c i d y n a m i c s STR. OF MATERIAL A D V . ENGRG. HATH e l e c t . C IR C , LAB e l e c t r i c a l C IRC I DYNAMICS s t r . OF m a t e r i a l a d v . ENGRG. HATH E LE C T . C IR C , LAB c l e c t r i c a l C IR C I d y n a m i c s S TR. OF MATERIAL A D V . ENGRG. HATH E LE C T. C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s S TR . OF MATERIAL A D V . ENGRG. MATH E LE C T . C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG. MATH E LE C T . C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG. MATH E LE C T , C IR C , LAB E LE CTR IC AL C IR C I d y n a m i c s S TR. OF MATERIAL A D V . ENGRG. MATH E LE C T. C IR C , LAB E LE CTR IC AL C IR C I d y n a h i c s STR- OF MATERIAL A D V . ENGRG. HATH E LE C T . C IR C , LAB C LECTRICAL C IR C I d y n a m i c s S TR. OF MATERIAL A D V . ENGRG. HATH E LE C T . C IR C . LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL AOV, ENGRG. MATH e l e c t . C IR C . LAB c l e c t r i c a l C IR C I d y n a m i c s S TR. OF MATERIAL A D V . ENGRG, m a t h E LE C T , C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG. MATH E LE C T . C IR C , LAB c l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG, m a t h E LE C T . C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG, MATH E LE C T . C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL AD V . ENGRG. HATH E LE C T , C IR C , LAB e l e c t r i c a l C IR C I d y n a m i c s STR. OF MATERIAL A D V . ENGRG. m a t h E LE C T . C IR C . LAB g WEEK SEMESTER BREAK THERMODYNAMICS E LE C T IC Sy s t e m s ENGRG. ANALYSIS S T R , OF HAIR LAB EHGRG D1G.C0HPTR THERMODYNAMICS e l e c t i c Sy STEHS ENGRG. A NA LY S IS S T R . o r HATR LAB ENGRG DIG.COHPTR t k e r h o o y n a m i c s ELECTIC s y s t e m s ENGRG. A NA LY S IS S TR . o r HATR LAB ENGRG d i g . c o h p t r t h e r m o d y n a m i c s E LE C T IC 5Y5TEH5 ENGRG. AN ALY5IS S T R , OF HATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s ENGRG. ANALYSIS S TR . OF HATR LAB ENGRG D IG .CO HPTR t h e r m o d y n a m i c s E LE CTIC s y s t e m s e h g r g . ANALYSIS S TR . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s e l e c t i c s y s t e m s ENGRG. ANALYSIS S TR , OF HATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELE CTIC 5YSTEM5 ENGRG. ANALYSIS S TR , OF MATR LAB ENGRG DIG.COHPTR t m e r m o d y n a h i c s ELE CTIC SYSTEMS ENGRG. ANALYSIS S T R . OF HATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s e l e c t i c SYSTEMS ENGRG. A NA LY S IS S TR . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s ENGRG. ANALYSIS S T R . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s ENGRG. ANALYSIS S TR . OF HATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s e l e c t i c s y s t e m s e n g Rg . ANALYSIS S TR . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s e l e c t i c S y STEHS ENGRG. ANALYSIS S TR . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s e l e c t i c s y s t e m s ENGRG. ANALYSIS S TR . OF MATR LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s ENGRG, ANALYSIS s t r . OF MATR LAB ENGRG d i g . c o h p t r SEM ESTER VACATION f l u i d m e c h a n i c s TECH. REPT. WRTG HEAT POWER LAB [HDEPENDEHTSTUOY INTERMED. THERMO FL U ID MECHANICS TECH. r e p t . WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED. THERMO FL U ID MECHANICS TECH. REPT. WRTG HEAT POWER LAB INDCPEHDENTSTUOY INTERNED, THERMO FL U ID MECHANICS TECH. REPT. WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED. THERMO f l u i d MECHANICS TECH. REPT. WRTG HEAT POWER LAD INDEPENDENTSTUDY INTERNED. THERMO T L U ID MECHANICS TECH. RE P T. WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERHED. THERMO f l u i d m e c h a n i c s TECH. REPT. WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED, THERMO t l u i d h e c h a n i c s TECH. RFPT ■ WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED. THERMO F L U ID m e c h a n i c s TCCh . r e p t . WRTG h e a t POWER LAB INDEPENDENTSTUDY INTERHED, THERMO F L U ID MECHANICS TECh . r e p t . WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED. THERMO f l u i d MECHANICS TECh . RE P T. WRTG HEAT POWER LAB i h d e p e n d e n t s t u o y INTERHED. THERMO F L U ID HECHANICS TECH. r e p t . WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERNED. THERMO f l u i d HECHANICS TECH. r e p t . WRTG HEAT PONER LAB INDEPENDENTSTUDY INTERNED. THERMO F L U ID MECHANICS TECH. r e p t . WRTG HEAT POWER LAB INDEPENDENTSTUDY INTE RHE D. THERMO f l u i d MECHANICS TECH. Re p t . WRTG HEAT POWER LAB INDEPENDENTSTUDY INTERHED. THERMO f l u i d MECHANICS TECH. R FP T. WRTG HEAT PONER LAB INDEPENDENTSTUDY INTERNED. THERMO f l u i d MECH. LAB f l u i d MECH. LAB FLUID MECH. LAB f l u i d MECH. LAB FLU ID MECH. LAB t l u i d MECH. LAB t l u i d MECH. LAB FLUID MECH, LAB f l u i d HECH. LAB f l u i d MECH. LAB f l u i d HECH. LAB f l u i d MECH. LAB F L U ID MECH. LAB f l u i d HECH. LAB f l u i d HECH. LAB FL U ID HECH. LAB TIME 1 NTERVAL* NO.OF COLUMNS* 2 WEEK SEMESTER BREAK 1NDU5T.HEAT INDUST.HEAT INO UST.HEAT INO u ST .H EA T IN D U S T.H E A T INDUS T.HE A T i n d u s t . h e a t i n d u s t . h e a t INDUS T.HE A T INDUS T.HE A T INDUS T.HE A T IN D u S T . h EAT IN D U 5T .H E A T INDUST.HEAT INDUST.HEAT INDUST.HEAT TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR t u r d o m a c h i n c r t t u r b o h a c h i n e r t TURBOHACHINERT t u r b o h a c h i n e r t TURBOHACHINERT t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t TURBOHACHINERT t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t AERODYNAMICS a e r o d y n a m i c s a e r o d y n a m i c s AERODYNAMICS a e r o d y n a m i c s AERODYNAMICS a e r o d y n a m ic s AERODYNAMICS AERODYNAMICS a e r o d y n a m i c s a e r o d y n a m i c s a e r o d y n a m i c s a e r o d y n a m i c s AERODYNAMICS AERODYNAMICS a e r o d y n a m i c s PONER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER PLANT5 PLANTS PLANTS PLANTS PLANTS p l a n t s PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS 1WEEKS t NO. NO OF INTERVALS IN SCHEDULES SB , OF SUBJECTS IN SCHEDULE* 20 TOTAL WEEKS* TOTAL ITE M S * SB, 2 6 . Fig. 17. - -Computer printout of initial schedule, two years of mechanical engineering at CSCLA. Last 121 addition, all of the prerequisites given in the 1965 college catalog were introduced. Finally, an estimate (based upon personal experience of the author as a member of the engineering faculty for five years) was mads of the number of weeks that each course reinforces every other course in the initial schedule. An estimate of this type would normally be made by a consensus of the engineering faculty. The computer program produced a printout of the level of m astery for each course for every week in the initial schedule (eighty-eight weeks total). These values have been plotted for four typical courses in Figure 18. The graphs show, among other things, that forgetting occurs between sem esters; forgetting decreases with increase in reinforcements and with proximity to total mastery; and that the maximum height of the learning curve varies with the subject matter. An improved schedule of courses produced by the computer program is shown in Figure 19. It appears that this final schedule is not much different from the initial schedule. Although 650 different schedules were attempted by the computer, only a comparatively small number of these were allowable. This is directly attributable to the large number of scheduling constraints that are imposed in this curriculum. Substantially greater improvements could be 50 P L , > > u 0 4 - > CO rt 'S 40 30 20 10 0 0 Junior Year Senior Year Fall Vacation Spring Summer Electrical circuits — Spring — "hermodynamics / I Z _ L Fluid mechanics / / _ i I I / Aerodynamics 10 20 30 40 50 Time, t (weeks) 60 70 80 90 Fig. 18. --Levels of mastery as function of time for four courses in Engineering at California State College at Los Angeles. (Computer computations were based upon typical catalog schedule and current college requirements and constraints.) 123 CDHPllTED SCHEDULE. TIME INCREMENTS OF 1 . WEEKS ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG, MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I DYNAMICS STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I Dy n a m i c s STR. OF MATERIAL ADV, ENGRG. MATH e l e c t r i c I C IR C I DYNAMICS STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG. HATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG, MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG, HATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV. ENGRG. MATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL a d v . ENGRG. h a t h ELECTRIC L C IR C I d y n a m i c s STR. OF HATER 1AL ADV, ENGRG. h a t h ELECTRIC L C IR C I Dy n a m i c s STR. OF MATERIAL ADV. ENGRG. HATH ELECTRIC L C IR C I d y n a m i c s STR. OF MATERIAL ADV, ENGRG, MATH ELECTRIC L C IR C I d y n a m i c s . STR. OF MATCRIAL ADV. ENGRG. MATH 2 WEEK SEMESTER BREAK t h e r h o d y h a h i c s ELECTIC s y s t e m s STR. OF HATR LAB E LE C T, C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s E LECTIC s y s t e m s STR. OF MATR LAB E LE CT. C IR C . LAB CNGRG d i g . c o h p t r THERMODYNAMICS FLEC TIC s y s t e m s STR. OF HATR LAB ELECT, C IR C , LAB ENGRG DIG.COHPTR THERMODYNAMICS ELECTIC s y s t e m s STR. OF MATR LAB E LE CT. C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s STR. OF HATR l a b ELECT. C IR C . l a b ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s STR. OF MATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s E LE CTIC s y s t e m s STR. OF MATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s E LE CTIC SYSTEMS STR. OF MATR LAB ELECT. C IR C . LAB CNGRG DIG.COHPTR t h e r m o d y n a m i c s E LE CTIC s y s t e m s STR. OF MATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR THERMODYNAMICS E LE CTIC s y s t e m s STR. OF MATR LAB E LE CT. C IR C . LAB ENGRG DIG.COHPTR THERMODYNAMICS ELECTIC s y s t e m s STR. OF MATR LAB ELECT. C IR C , LAS ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC s y s t e m s STR. OF MATR LAB ELECT, C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC SYSTEMS STR. OF MATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR THERMODYNAMICS ELECTIC s y s t e m s STR. OF MATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC 5Y5TEM5 STR. OF HATR LAB ELECT, C IR C . LAB CNGRG DIG.COHPTR t h e r m o d y n a m i c s ELECTIC SYSTEMS STR. OF HATR LAB ELECT. C IR C . LAB ENGRG DIG.COHPTR INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDT INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDT INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY INDEPENDENTSTUDY SEM ESTER VACATION F L U ID F L U ID F L U ID F L U ID F L U ID F L U ID f l u i d f l u i d F L U ID F L U ID f l u i d f l u i d f l u i d T L U ID f l u i d F LU IO MECHANICS MECHANICS MECHANICS MECHANICS MECHANICS HECHANICS MECHANICS mec hanics MECHANICS MECHANICS HECHANICS HECHANICS HECHANICS HECHANIC5 MECHANICS MECHANICS F L U IO F L U ID f l u i d F L U ID F L U ID F L U ID F L U ID f l u i d f l u i d F L U ID F L U ID F L U ID f l u i d f l u i d F L U ID FLUID MECH. LAB MECH. LAN HECH, LAB MECH. LAB HECH, LAB MECH, I AB MECH. LAB HECH, LAB MECH, LAB MECH, LAB HECH. LAB HECH. LAB HECH, LAB HECH. LAB HECH. LAB MECH. LAB TECH. TECH. TECH. TECH, TECh . TECH, TECH. TECh . TECH. TECH. TECH. TECH. TECH. TECH, TECH. TECH. i n d u s t . h e a t i n d u s t . h e a t i n d u s t . h e a t i n d u s t . h e a t i n d u s t . h e a t INDUST.HEAT i n d u s t . h e a t i n d u s t . h e a t I n Du S T . h E a T INDUST.HEAT INDUST.HEAT IN D U 5 T . h e a t i n d u s t . h e a t i n d u s t . h e a t INDUST.HEAT INDUST.HEAT REPT. WRTG HEAT POWER LAB ENGRG. ANA LY S IS INTERMED. THERMO RFPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED. THERMO R F P T . WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED, THERMO RE P T. WRTG HEAT POWER LAB CNGRG. a n a l y s i s INTERMED. THERMO REPT. WRTG HEAT PONER LAB ENGRG. ANALYSIS INTERHED, THERMO REPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED. THERMO REPT. WRTG HEAT POWER LAB l h GRG. ANALYSIS INTERHED. t h e r m o REPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED. t h e r m o REPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED, t h e r m o REPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERMED, t h e r m o REPT. WRTG HEAT PONER LAB ENGRG. a n a l y s i s INTERHED, THERMO RFPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERMED. THERMO REPT. WRTG HEAT POWER LAB ENGRG. ANALYSIS INTERHED, THERMO REPT. WRTG HEAT POWER LAB ENGRG, ANALYSIS i n t e r n e d . THERMO RFPT. WRTG HEAT POWER LAB ENGRG. a n a l y s i s INTERHED, THERMO REPT. WRTG HEAT POWER LAB ENGRG. ANA LY S IS INTERHED, THERMO 2 WEEK SEMESTER BREAK TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TRFR TURBOHACHINERT TURBOMACHINERY TURBOHACHINERY TURBOHACHINERT TURBOHACHINERT t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o h a c h i n e r t t u r b o m a c h i n e r y TURBOHACHINERT TURBOHACHINERT t u r b o h a c h i n e r y t u r b o h a c h i n e r y t u r b o h a c h i n e r t t u r b o h a c h i n e r t a e r o d y n a m ic s AERODYNAMICS a e r o d y n a m ic s a e r d o t n a h i c s AERODYNAMICS AERODYNAMICS AERODYNAMICS a e r o d y n a m i c s a e r o d y n a m ic s AERODYNAMICS AERODYNAMICS a e r o d y n a m i c s AERODYNAMICS a e r o d y n a m i c s a e r o d y n a m i c s a e r o d y n a m i c s POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER POWER PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS PLANTS f o r i t e r a t i o n 2 , m a ». n o . o f e x c h a n g e s * I n i t i a l P* 7 5 2 ,3fl F IN A L P* 259.*6 3 2 5 NO. OF ALLOWABLE EXCHANGES* ( P I F I H A D - P I I N I T I A L ) ! * T . o e 22. - Fig. 19. - -Computer printout of an improved schedule for last two years of mechanical engineering at CSCLA. 124 obtained by'reducing the number of prerequisites, increasing the number of courses by changing over to a "quarter" system or a continuous progress system, and by considering the interactions between the technical and liberal arts courses in the design of new schedules. A table of param eters used in the calculation of P is printed toward the end of the computer program output and is reproduced in Figure 20. The table also includes an ideal P and a final P for each of the twenty courses considered in the schedule. Figure 21 also shows the final table of data that is printed at the end of the computer program. It should be noted that the six courses at the end of this table are "blank" courses which are introduced into the program m erely to facilitate the exchange of courses in the schedule. The blank courses are not considered in the calculation of P. A series of sensitivity studies were carried out to determine the effects of small changes in the learner, teaching method, and the starting schedule used in the program. It was found that relatively small changes in these param eters yielded the same suboptimum schedule. This result indicates the relative stability of the solutions resulting from the use of the heuristic algorithm and this computer program in particular. Large changes in M and L resulted in significant changes in the suboptimum schedule. 125 TABLE OF PARAHETCRS used IN the c a l c u l a t io n OF P COURSE S I H L a PI IDEAL) P (FINAL) 1 1.00569 1.00 1 .0 0 28 00 1 .2 0 20.7 9 11*66 2 1.01156 1.00 1 .0 0 20 00 1 .2 0 6 .3 6 5*20 3 1.00664 1.00 1 .0 0 26 00 1 ,2 0 16.59 11*61 4 1.0095& 1.00 1 .0 0 22 00 1 ,2 0 17.64 15*06 5 I . 00664 1 . 00 1 .0 0 26 00 1 .2 0 16,60 9*50 6 1.00356 1,00 1 .0 0 36 00 1*20 13.7 6 9*49 7 1.00356 1.00 1*00 36 00 1,20 2 6 ,9 7 25(47 S 1.00956 1.00 1 .0 0 22 00 1.20 13,6 6 12*49 9 1.00664 1.00 1.0 0 26 00 1 .2 0 15.97 6 ,6 6 10 1.00603 1.00 1.0 0 24 00 1 .2 0 6*13 6*13 11 1.00403 1.00 1.0 0 24 00 1 .2 0 6*13 6*13 12 1.00574 1.00 1.0 0 23 00 1 ,2 0 46*79 16*79 13 1,00603 1.00 1 .0 0 24 00 1 .2 0 21.9 4 16*09 14 1.00451 1.00 u o o 32 00 1*20 6*66 6 ,6 6 IS 1,0 0 95 6 1.00 1 .0 0 22 00 1*20 15,7 0 14*26 16 1,01606 1.00 1 .0 0 17 00 1*20 15*47 15*47 17 1.01156 1.00 1 .0 0 20 00 1 .2 0 17*26 17*26 IB 1.01615 1.00 1 .0 0 16 00 1*20 14*66 14*86 19 1.00451 1.00 1 .0 0 32 00 1.20 13*23 9*91 20 1.01431 1.00 1 .0 0 16 00 1.20 16*06 16*06 TOTAL IDEAL o r * 311*06 TOTAL IDEAL P« 311 i f ir s t TOTAL P* 2 55 ,4 LAST TOTAL P* HAT • CKCHAttOCS* 325 EFFICIENCY* T FEU CE»T Fig. 20. - -Computer printout of param eters used in the calculatiomof P. &300 ] 1. 1* • 1 . 3* 6 2, 7* 1 10. 4, , 6 6 , * 20* , 26 ELECTRICAL CIRCI £301 2 1. 2 . * 1 , 3* 1 7 . 1. » , , 71. * 17* * 20 DYNAMICS £302 3 1. 3 , • 1 , 3* 1 1. 4, 19! 1 • , a 6 6 . • 20* * 26 STR* o r m a t e r ia l £303 4 53, 1. 52* 1 ■ 3 7 , 1, 12. 2 13* 1, 1. 16. • 16* 1 22 FLUID HECHANICS E305 5 19. 1. 16. 1 . 3 2 0 . 1, 13* 2 17. 1. 2 . 5 2 . « 18* • 26 THERMODYNAMICS E307 6 19. 2. 18. t . 3 7 . 4* , . 50, * 20* * *6 e l e c t i c SYSTEMS £309 7 53. 4, 52* 1 16. 4. i s ! 4 l o ! 4 . • 16* ■ 20 * * 36 CNCR6, ANALYSIS L306 8 53* 2, 52* 1 12. 1. 13* 1 14. 2* 19, • IT * *• 22 TECH, REFT* HRT6 H302 9 1. 4, 7 . 4 * 19* 2 . . . 66* * 2 0 * • 26 ADV* C tftR t* MATH £310 10 19. 4. 16* 1 , ■ , • 54* * 16* * 24 ELECT* CIRC* LAB £312 11 19. 3. 1 6 . 1 * • . , * 54* • 16* , 24 STR* OF HATR LAB £313 12 71. 1. 70* 1 16* 3* is! 2 , , « 2* » 16, , 23 FLUID MECH* LAB £315 13 5 3. 3 . 5 2. 1 . 1 16* 3* 20, 1 12, 1, 2* 17* * 19* * 24 HEAT PONER LSS £499 14 19. 6 . 16. 1 « 1 . * • 54* , 1 6 , • 32 INDCPCNDCNTSTUDr E406 15 5 3. 5 , 52. 1 13* 2* i t ! 1 20*' 1. » 19, , 17* , 22 INTERHED* THERMO E405 16 7 1. 2 . 70, l * 3 • , * * 2. . 1 4, • IT IN0U3T*NEAT TRFR £40* IT 7 1 . 3 . 7 0 , 1 • 3 » * • , 2 , , 1 6 , • 20 t u r b o h a c h in e r t £443 18 71. 4 . 70. 1 * 3 • • • . 2 , , 16* , 16 a e r o d y n a m ic s E490 19 19. 5 . l e . i « ' u ! 4. u ! 4 * * , SO. « 20* • 32 EN6R6 DI6.C0MPTR E420 20 7 1. 5* 7 0 . l • * . 2 . « 1 6 . , IB POWER PLANTS 6LK1 21 1* 6 • « , , * , • * * * BLK2 22 71* 6* 7 0 . l 4 , v • , * * * • BLK3 23 53* 6 . 52* 1 , • » 6L44 ?4 1* 5 . , a * . * 0LK5 25 7 1. 7* 70* 1 , * . , , * * * * * BLK6 26 1. 7 . * 1 , • ' « • • • * 1 * FOR DATA SET 3 , THESE HER 2 ITERATIONS AND 46 SCHEDULES HERE ALLOWABLE AND CONFUTED fOR P , A TOTAL OF 690 5CHEDUL WERE ATTEMPTED. Fig. 21. - -Computer printout of final table_of information showing input data for the program. 126 Although the sensitivity studies produced one suboptimum schedule, it also yielded a different value of total level of m astery (P) for every combination of M, L, and schedule Sc* It is of consider able importance to know whether these values of P are statistically significant, i. e . , whether the variations in P are due to chance or due to specific interactions between the param eters. To accomplish this, level of m astery scores were generated by computer simulation for two levels of teaching methods, three levels of learner ability, and two levels of scheduling (catalog and suboptimum). The following statistical analysis was conducted with these predicted m astery scores to provide insight on what may be expected with actual empirical data, assuming the essential validity of the model, and to provide an illustration of but one experimental design that would be appropriate for testing the model. The m astery scores generated by the computer simulation model for various combinations of teaching methods, levels of learner, and scheduling were treated as if they were means derived from an empirical study and appropriate for a three - way fixed effects analysis of variance. The data were analyzed with the further assumption that there were 1 2 0 subjects, ten in each treatment combination. The data obtained from simulation (sensitivity) studies on the computer are presented in Table 3. The analysis, based on TABLE 3 DATA FOR THREE-FACTOR ANALYSIS OF VARIANCE FOR LEVEL OF MASTERY, P L ■ M • Sc Interactions Teaching method Mx = 1 . 0 M2 = 1 . 2 Type of learner L ^ = .75 L 2 = 1 . 0 L3 - 1 . 2 L^ = . 75 L 2 = 1.0 l 3 - 1 . 2 Schedule of courses Catalog Sci Model Sc2 311.6 371.1 375.0 445.8 415.1 489.0 390.0 469.2 466.0 556.6 517.2 608.6 L . M Interactions L • Sc Interactions Mi M 2 Sc^ S c 2 Li 341.3 428.0 L 1 350.8 418.5 L 2 410.9 514.9 L 2 422.6 501.2 L3 452.0 562.9 L3 466.0 548.8 127 128 Weiner, 1962 (22:248), is summarized in Table 4 and graphically in Figures 22 and 23. Since the data are hypothetical, total sums of squares and e rro r term s were not available. An e rro r term was selected such that the sequencing or scheduling main effect (Sc) would be significant at the . 05 level. Had an experiment been conducted and these results obtained, the following interpretations would be appropriate, based on the analysis of variance: 1. The main effects for the learner are significant at better than the . 01 level. Examination of the means for the three levels of learners indicates that m astery level increases as a function of increased learner ability. This result is consistent with educational research findings reported earlier and with experience in the classroom. 2 . The main effects for scheduling are significant at the . 05 level. Analysis of the means for the two schedul ing methods indicates that the optimal scheduling method produces greater m astery than catalog scheduling. This supports the major hypothesis of this research. TABLE 4 SUMMARY OF ANALYSIS OF VARIANCE FOR COMPUTER EXPERIMENT Source of variation SS Sum of Squares df Degrees of Freedom MS Mean Square p Significance Level M (teaching method) 174,803.4 1 174,803.4 2.3 L (type of learner) 2,160,957.4 2 1,080,478.7 144.5 * * Sc (type of schedule) 299,000o 4 1 299,000.4 4.0 * ML 547,895.3 2 273,947.6 3.7 MSc 2,219.2 1 2,219.2 LSc 28,187,801.6 2 14,093,900.8 188.5 ** MLSc 1,852,683.8 2 926,341.9 12.4 ** Within cell (experimental error) * * * 108 74,750.1 Total * * * 119 * Significant at the . 05 level. Significant at better than the . 01 level. * * * to Total sums of squares and erro r term s were not available in this computer experiment. ^ The erro r term was selected such that the scheduling main effect (Sc) would be significant at th e . 05 level. Level o f Mastery, 130 600 L • Sc Interactions 550 500 450 400 350 300 0.7 0.8 0.9 1.0 1.1 1.2 L . M Interactions 110 M=l. 2 M=1 0.7 0.8 0.9 1.0 1.1 1.2 Type of Learner, L Type of Learner, L Fig. 22, --Graphic presentation of interactions between learner, teaching method, and schedule. Level o f Mastery, L * M • Sc Interactions 131 650 600 Teaching Method 550 500 450 S o 400 Sc 350 300 0.7 0.8 0.9 1.0 1.1 1.2 Type of Learner, L Sc2 Sc Teaching Method 0.7 0.8 0.9 1.0 1.1 1.2 Type of Learner, L Fig. 23. --Graphic presentation of interactions between learner, teaching method, and schedule. 132 3. The "learner-scheduling” interaction was significant at better than the . 01 level. Examination of the graphical results (Figure 2 2) indicates that the greater the learner's ability, the greater the effective ness of the optimal scheduling compared to the catalog schedule. (Notice that there is greater divergence between and S2 as ability increases, i. e . , as we go from to L ^.) 4. The three-way interaction was significant at better than the . 01 level. Examination of the graphical results (Figure 23) indicates that improvement of teaching method (M^ to M2) results in greater magni tude of improvement in m astery level for learners as ability increases when the scheduling is optimal (SC2) as compared with the catalog (Sc^). Finally, it must be noted that while the present formulation of the model does not specifically indicate that we can predict inter actions, it does explicitly provide for such a contingency. The model contains an interaction factor (1) which can be evaluated and interpreted after empirical data has been analyzed. The statistical analysis of the aforementioned simulation data showed that inter actions do occur and they are significant. These results could not have been predicted by considering the factors individually or by visual analysis of the model. The use of the analysis of variance in conjunction with the model's simulation data has made it possible not only to predict interactions that seem consistent with empirical findings in educational research but also to estimate which kinds of interactions are significant and the expected relative degree of significance between factors. This finding is very encouraging. CHAPTER X SOME IMPLICATIONS OF THE DISSERTATION Immediate applications of the computer program In its present state of development, the sequencing model in conjunction with the digital computer program provides a flexible tool for the solution of a number of rather complex problems in the area of curriculum management. 1. Given a particular set of subject m atter (i. e . , items, topics, or courses), it is possible to rapidly and systematically compare a large number of proposed presentation schedules of the subject m atter and to select a relatively "best" schedule. The "best” schedule is the one that, without varying any of the factors other than the sequencing of the subject m atter, will give the student greater m astery and retention of all the m aterial than any other schedule. 2. Given the current curriculum of a particular school, subject area, or course, it is possible to estimate 134 the relative value of the present schedule to that of an ’’ideal” rearrangem ent of the subject m atter. The word "ideal" in this context denotes, not the ultimate or very best possible schedule, but rather a schedule which can be considered signifi cantly better than current ones because some of the present constraints which are not absolutely necessary have been relaxed. In addition, a sub optimum schedule which is subject to at least all of the constraints of the present schedule can readily be computed. Usually, all possible arrange ments of the subject m atter cannot be investigated even with the help of a large digital computer. However, it is possible to estimate how close any suboptimum schedule is from an ideal or less constrained schedule by comparing computed ideal and suboptimum values. In the development of a curriculum to achieve a given set of educational objectives, it is difficult to quantitatively estimate the relative advantages . . . ^ ' and disadvantages of specialized versus generalized sequences of courses. For example, engineering students at U. C. L. A. are all required to take the same program of general courses as part of a '’unified curriculum"; whereas, students at Cal Poly are required to take programs which contain specialized courses within the scope of various engineering disciplines. These two curricula cannot be compared directly because their educa tional objectives, types of students and faculty, and so forth, are quite different. However, with the judicious application of the computerized sequencing model, it is possible to estimate the relative value to the student (in term s of his m astery of the entire curriculum) of various specialized or generalized courses of study in each school. Thus, for a specific school, sequences of specialized courses may be desirable in term s of some educa tional objectives but there is usually a point of diminishing returns. Beyond this point, additional courses in a particular subject area produce rela tively sm aller and sm aller increases in m astery of the m aterial and these small increases are achieved at the expense of relatively larger increments in m astery in other jsubject areas. The computerized sequencing model makes it possible not only to explore the effect of various course sequences upon the students' m astery of the entire curriculum, but also to estimate the optimal degree of specializa tion. In most schools, faculty members are mainly interested and knowledgeable in their own fields of specialization. By and large, they may not be intimately familiar with the interrelationships between all subject areas in the curriculum and the objectives of the school, division, or depart ment. The model developed in this dissertation facilitates the economical construction of individual ized suboptimal schedules which can be used by the faculty in counseling students. Each faculty mem ber can have at his disposal a manual of recom mended schedules which are appropriate for individual students, students with common educational objectives, and students enrolled in special programs such as honors programs. In this way, each counselor or advisor can have at his fingertips the latest and best thinking of the entire faculty. As increasing numbers of students make greater demands upon our educational resources, a pro portionate number of school systems have begun to turn their attention to the possibility of contin uous, year-around operation of their programs and facilities. Many schools are in the process of converting from two sem esters per year to three or four. This implies that the curriculum content must be redistributed, in general, into a greater number of discrete subject m atter pack ages which are presented over shorter time intervals. If this is done, there is greater flex ibility available in the design of new schedules. The sequencing model can materially improve our capability to rearrange the existing curriculum content into improved and allowable schedules. It thus becomes possible to investigate a very large number of combinations of topics to form new courses and new course sequences. Given the subject m atter to be covered in the present curriculum and an estimate of the time to be spent on each topic, it is possible (with the aid of the sequencing program) to develop radically new courses of study which will satisfy all of the present curriculum constraints and in addition, give the student a greater degree of m astery of all the subject m atter on the day of graduation. In the overwhelming majority of American schools, the curriculum is divided into segments of subject m atter which are presented in equal intervals of time, namely, sem esters. All students proceed through their courses of study in lock step; they begin each course at the same time and end it at the same time. If a student drops out of a course at the beginning of a sem ester, for whatever reason, he must wait until the start of the next sem ester before he can enroll in another course. Similarly, if a bright student is capable enough to complete the requirements of a given course in less time than an average student, he must essentially "mark time" until the sem ester is over. Conversely, slower students fall increasingly 140 further behind in their m astery of the overall curriculum content with every passing year. The major justification for the sem ester system, especially in schools where there are thousands of students involved, is that the problem of school administration and data^processing is kept within manageable proportions. However, an increasing number of educators have pointed out that the prim ary function of any school is the maximization of educational benefits to the students; the reduction of management problems is of secondary importance. As was mentioned earlier, the continuous-progress school seems to offer one possible solution to this problem. A considerable amount of research in this area is now in progress at the high school level. Most colleges and universities in other parts of the world have never used any other kind of system. It is not intended to discuss the relative m erits of both educational approaches in this dissertation. Rather, it is introduced here in order to point out that the computerized sequencing program is flexible enough to handle schedules for any continuous progress school. 141 Thus, units of subject m atter may be of various time durations, students may start ancf finish courses or topics at any time, and the maximum number of hours of classroom work allowed per week for each student may vary as a function of time. The computer program is capable of handl ing a number of different types of schedules for different types of schools. The number of topics or courses that can be handled by the program is limited only by the size of the computer that is available to carry out the calculations. Potential research projects The significance of a piece of research can often be judged by the number of fruitful areas of research that it opens up. The following items represent some of the research areas that have a high payoff potential for educational institutions, and which may reasonably be undertaken at the present time with the hope of achiev ing positive results. 1. In the development of the computerized sequencing model, it was assumed that the time distribution and sequencing of the students' exposure to subject 142 matter significantly influences the overall degree of m astery of an entire topic, course, or curriculum. This hypothesis was supported by reference to generalizations drawn from the fields of educational psychology, curriculum management, teaching methods, and industrial studies in learning theory. In addition, the hypothesis appears to be substantiated qualitatively by our experiences in and out of the classroom as well as by logical considerations. However, in his paper, "Men, Machines, and M odels," Alphonse Chapanis made a crucial statement: "I will gladly exchange 100 well-informed guesses at any time for the results of one carefully executed experiment. " In the context of his paper, Chapanis implied the desirability of experiments which are repeatable, controlled, and analytically precise. But in the study of adaptive (learning) systems, particularly those in which man is the prim ary subject of investigation, we find that repeatability is almost impossible; adequate control is usually difficult to achieve and often quite expensive; and analytical precision is not attainable at the present time because 143 comprehensive and quantitative functional relation ships which describe m an's behavior are not yet available. Mathematical analysis is not powerful enough to yield general analytical solutions to situations as complex as are encountered in social systems like schools. It is obvious that, even though no analytical solutions to problems in the area of curriculum exist, decisions regarding content allocation, sequencing, teaching methods, and so forth, are being made every day. As was pointed out earlier, it is desirable to develop a decision structure which systematically takes into account as many of the relevant factors and data as possible. The sequencing model represents an initial attempt to do this. Although no crucial experiment can presently be devised which would conclusively prove the validity of the model, there are some things that can be undertaken which could increase our level of confidence in it. (a) Develop a comprehensive battery of tests which cover the entire curriculum and which would be given to students upon graduation. These tests would be designed to measure the degree of student m astery of the skills, attitudes, and subject m atter in the entire curriculum as well as in the student's area of specialization. The actual schedule followed by each student could be used as the basis for predicting overall m astery of the entire cur riculum by that student with the help of the computer program. In this way, the predicted results from the sequencing model for each student can be compared with actual test results. Predicted and test results for individual students can be statistically combined to form average results for various types of schedules and various groups of students. These sets of data can then be analyzed to determine whether or not significant correlations exist between the predic tions of the model and the testing program results. In this initial type of experiment, it is expected that the sequencing model will be able to discrim inate between various schedules only in a gross sort of way; one schedule is either better or 145 worse than another from the point of view of what the student learns during his entire stay in school. (b) During the four-or five-year period of time that the aforementioned experiment is con ducted, a series of considerably shorter experiments can be carried on. These experi ments involve the investigation of improved sequences of items or topics in individual courses. The sequencing of the subject matter in specific courses provides the possibility of greater control of relevant factors such as students, teachers, teaching methods, repetitions, and so forth. Parallel sections of a given course could be offered; some of the sections could serve as control groups for carefully designed statistical experiments. Analyses of variance and covariance of model predictions and the em pirical test results could be made as was mentioned earlier. The reliability of these results could be considerably improved by repeat measures experiments for a number of successive sem esters. It is also possible in this way to compare the relative effects of different teachers, teaching methods, students, and sem ester programs of study upon the students' overall m aster of the subject matter. As the faculty in a given school gains confidence and familiarity with its own total curriculum, its general and specific educational objectives, and with the theory and assumptions of the sequencing model, the functional relationships and empirical factors will undergo revision and refinement. In fact, completely different functions may emerge which represent the empirical data more closely. This would require only minor changes in the computer program since the bulk of the program is con cerned with the general methodology. In developing the foregoing model, it was assumed that optimal sequences of subject matter yield not only maximum m astery but also maximum retention by the student after graduation. The "information . explosion, ” especially in the areas of science and technology, has sharpened the competiton between topics for time in the curriculum, has stimulated the process of continuously redesigning the cur riculum, and has served to focus attention upon the need for curricula that are not readily forgotten by the students shortly after graduation. Although the sequencing model provides a preliminary theoretical framework for studies concerned with the retention of skills, attitudes, and subject m atter before and after graduation, a considerable amount of follow- up research in this area is essential. Some research projects with high immediate payoff potential are: (a) Experimental studies to determine more p re cisely the degree to which such factors as the teaching methods, the teacher, the learner, the type of subject m atter, the cumulative time of exposure, and the number of repetitions, determine the extent and type of retention of specific learning experiences before graduation. (b) Similar retention studies involving alumni at various times after graduation. These studies would also take into account such factors as occupation requirements, social status, time after graduation, post-graduation schooling, and so forth. A large area for research is opened up when the following question is asked: Given the faculty's estimate of the relative importance of each item (or topic, or course) in the curriculum and an appreciation of the implications of learning theory for the sequencing and the distribution of the sub ject m atter, what is the total amount of time that should be allocated to each item in the curriculum? This is essentially the curriculum allocation problem considered by the faculty of the Engineering Department at U. C. L. A. but with the additional complication of taking into account the actual learn ing process in the design of the educational program. It is expected that as experience is gained in the application of the computerized sequencing program, increasing amounts of research will be put into the total time allocation problem. From an administrative or managerial viewpoint, productive research is the type that: results in more efficient allocation of resources, reduction of operating costs, or lower total budgets. In general, it is desired that research concepts and innovations shall produce tangible dollar and cents benefits in the operation of the enterprise. When dollar values are assigned to educational resources such as faculty manhours, facilities, audio-visual aids, teaching machines, and laboratory equipment, the sequencing model can become a powerful administrative research tool. When combined with standard economic analysis techniques, it can provide a vehicle for rapidly esti mating the costs versus the potential benefits to the students of a number of proposed or existing experi mental educational programs. CHAPTER XI CONCLUDING REMARKS It has been shown in this dissertation that the application of currently available systems analysis techniques to problems of management and curriculum synthesis in higher education can lead to productive results. The systems approach presented herein has delineated some of the areas in which considerable progress has already been made and has indicated areas in which significant research contributions are incipient. The conceptualization of an institution of higher learning as a system having sensor-controls, data-processing, and decision rules provides a systematic approach for the determination of areas of research having high payoff potential in higher education. The introduction of such techniques as linear programming, Markov processes, Lagrangian multipliers, combinatorial analysis, and digital computer simulation into the field of curriculum analysis and synthesis has led to increasing quantification of available data and concepts. The computerized sequencing model developed herein 150 151 illustrates the process of converting psychological, educational, and administrative principles from a set of largely qualitative form ula tions into a quantitative format. From this perspective, the model represents a significant contribution because it integrates a number of useful principles, facilitates the development of improved educa tional programs, provides increased capability to predict the out comes of the complex interaction between scheduling variables, and indicates specific areas for potentially fruitful research. The computerized sequencing model has already been used as the basis of a proposal to the Office of Economic Opportunities to provide individualized schedules of courses and activities for young women between the ages of sixteen and twenty-one who are trainees in Job Corp training centers. Personal conversations between the author and key personnel at various educational institutions in the metropolitan Los Angeles area indicated an interest in applying the model to a number of different curriculum problems in the very near future. It is expected that the model will be used at U. C. L. A. in the School of Education to determine in which order a student should take certain education courses. The School of Business and Economics at the University of Southern California is considering use of the model to examine the entire undergraduate and graduate curriculum. The Alhambra School D istrict is considering the experimental verification 152 of the model in the optimal sequencing of topics in individual courses given at the junior high school level. Finally, it is hoped that the author’s own Division of Engineering at the California State College at Los Angeles will initiate a comprehensive, long-range program of curriculum synthesis utilizing the scheduling model in the process of converting to the "quarter" system, the design of a new graduate program in engineering, and in the development of an interdisci plinary undergraduate core curriculum. In the final analysis, how ever, productive extension of the work initiated in this dissertation will depend upon the interest generated and the support obtained. BIBLIOGRAPHY BIBLIOGRAPHY Books 1. Atkinson, R. C ., and Estes, W. K. "Stimulus Sampling Theory," Handbook of Mathematical Psychology. Vol. II. New York: John Wiley, 1963, Chapter 10. 2. Bush, R. R ., and Estes, W. K. Studies in Mathematical Learning Theory. Stanford, California: Stanford University Press, 1959. 3. Bush, R. R ., and Mostelle, F. Stockhastic Models for Learning. New York: Wiley, 1955. 4. Ebbenghaus, H. (1885). Memory Translation by H. A. Ruger C. E. Busenius. New York: Teachers College, 1913. 5. Estes, W. K. Current Trends in Psychological Theory. Pittsburgh: University of Pittsburgh Press, 196l, 134-151. 6. F orrester, Jay. Industrial Dynamics. Cambridge: The M .I.T . Press, 1961. 7. Hilgard, E. R. Theories of Learning. New York: Appleton- Century-Crofts, 1956. 8. Howard, R. A. Dynamic Programming and Markov Processes. New York: The Technology Press of the Massachusetts Institute of Technology and John Wiley & Sons, 1960. 9. Hull, C. L. Principles of Behavior. New York: D. Appleton- Century, 1943. 10. Koch, Sigmund (ed.). Psychology: A Study of A Science. New York: McGraw-Hill, 1959. 154 155 11. Johnson, Cast, and Rosenzweig. Systems Analysis. McGraw-Hill, 1963. 12. Miller, N ., and Spence, K. W. Behavior Theory and Conditioning. New Haven: Yale University Press, 1956. 13. Seagoe, M. V. A T each ers Guide to the Learning Process. Iowa: William C. Brown, Second Edition, 1961. 14. Shannon, C. E ., and Weaver, W. The Mathematical Theory of Communication. Urbana, Illinois: University of Illinois Press, 1949, 377, 405. 15. Skinner, B. F. Science and Human Behavior. New York: Macmillan, 1953. 16. . "The Flight from the Laboratory," Theories in Contemporary Psychology. Editor, Melvin H. Marx. New York: Macmillan C o., 1963, 323-338. 17. Smallwood, R. D. A Decision Structure for Teaching Machines. Cambridge, Massachusetts: M .I.T. Press, 19W . 18. Stephens, J. M. Educational Psychology. New York: Henry Holt and Company, Revised Edition, 1956, 417-453. 19. Taylor, Donald W. "Thinking," Theories in Contemporary Psychology, Editor, Melvin H. Marx. New York: Macmillan C o., 1963, 475-493. 20. Tonge, F. M. Heuristic Program for Assembly Line Balancing. Englewood Cliff, New Jersey: Prentice-Hall, 1961. 21. Von Court, Hare, Jr. "Systems Analysis," Progress in Operations Research. Editor, David B. Hertz and Roger T. Eddison. New York: John Wiley and Sons, 1964, II, 123-158. 22. Winer, B . J. Statistical-Principles in Experimental Design. New York: McGraw-Hill Book Company, 1962. 156 Periodicals 23. Andress, Frank J. ”The Learning Curve as a Production Tool, " Harvard Business Review (January-February, 1954), 87. 24. Buffa, E. S . , Armour, G. C ., and Vollman, T. E. "Allocating Facilities with CRAFT, " Harvard Business Review, XLII, No. 2 (March-April, 1964), 136-147. 25. Burke, C. J ., and Estes, W. K. "A Component Model for Stimulus Variables in Discriminating Learning," Psychometrika, III (1957), 133-145. 26. Carlson, John G. "How Management Can Use the Improve ment Phenomenon, " California Management Review, III, No. 2 (Winter, 1961), 83-94. 27. Clark, H. M. "Recall and Recognition for Faces and Names, " Journal of Applied Psychology, XVIII (19 34), 757-7 63. 28. Culler, E ., and Girden, E. "The Learning Curve in Relation to Other Psychometric Functions," American Journal of Psychology, LXIV (1951), 327-349, 370-371. 29. Ellson, D. G. "Application of Operational Analysis to Human Motor Behavior," Psychological Review, LVI, 9-17. 30. Flagle, C. D. "Operations Research in the Health Service, " Operations Research, No. 5 (1962), 10. 31. Gerker, Thomas L. "The Administrator and the Computer: An Annotated Bibliography, " Educational Data Processing Newsletter, III, No. 11 (December, 1964), 22-29. 32. Gulliksen, H ., and Wolfle, D. L. "A Theory of Learning and T ransfer," Psychometrika, III, 127-149, 225-261. 33. Leavitt, H. J ., and Mueller, R. A. H. "Some Effects of Feedback on Communication, " Human Relations, IV, 376, 401-410. 157 34. McKeachie, W. J. "Understanding the Learning P rocess," journal of Engineering Education, L (I960), 863-865. 35. Miller, G. A. "What is Information Measurement?" American Psychologist, VIII (1953), 3-11. 36. Reisman, A ., and Buffa, E. S. "A General Model for Production and Operations System s," Management Science, XI (September, 1964), 64-79. 37. * "A General Model for Investment Policy," Management Science, VIII (April, 1962), 304-310. 38. Review of Educational Research, XXXIII, No. 3 (June, 1963), National Education Association, Washington, D. C. 39. Robertson, T. B . "Sur la dynamique chimique de system nerveux central," Arch. Int. de Physiol., VI (1908), 388-454. 40. Rowe, A. J. "Toward a Theory of Scheduling," Journal of Industrial Engineering, XI, No. 2 (March-ApriL 1960). 41. Skinner, B. F. "Experimental Psychology," Current Trends in Psychology. Pittsburgh: University of Pittsburgh Press, 1947. 42. . "Are Theories of Learning Necessary?" Psychological Review, LVII (1950), 193-216. 43. ____________ . "The Science of Learning and the Art of Teaching," Current Trends in Psychology and the Behavioral Sciences. Pittsburgh: University of Pittsburgh Press, 1954. 44. Tyler, R. W. "The Evaluation of Teaching, " Journal of Engineering Education, L (1960), 863-865. 45. Thurstone, L. L. "The Relation Between Learning Time and Length of Task," Psychological Review, XXXVTI (1930), 44-53, 372-73. 158 46. Thurstone, L. L. "The Learning Function," Journal of General Psychology, III (1939), 372-389 , 469-493. Reports 47. Chaney, John F. Student Data Processing System at the Uni versity of Illinois. A report prepared for and published by the International Business Machines Corporation. 48. Coulson, John E. Present Status and Future Prospects of Computer-Based Instruction^ Presented at the" 1964 Annual Convention of the National Society for Programmed Instruction, San Antonio, Texas. System Development Corporation document, SP-1629, 1964. 49._ _ ______________ . Computers in Programmed Instruction and Educational Data Processing. System Development Corporation document, SP-950, January, 1963. 50. Egbert, Robert L ., and Cogswell, John F, System Design for a Continuous Progress School. System Development Corporation document, TM-1493/103/000, 1964. 51. . System Analysis and Design in Schools. System Development Corporation document, SP-1141, March, 1963. 52. Hopkins, R. C. A Systematic Approach to Systems Develop ment. System Development Corporation document, FN-4176, August, 1960. 53. Kagdis, J ., and Lackner, M. R. Introduction to Management Control Systems Research. System Development Corporation document, TM-708/100/00, October, 1962. 54. Kershaw, J. A ., and McKean, R. N. Systems Analysis and Education. The Rand Corporation document, RM-2473-FF, October, 1959. 55. Kribs, Patricia. SHARE Digital Simulation Glossary. System Development Corporation document, SP-1562, February, 1964. 159 56. Lackner, Michael R. The Dependency of a Simulation Language on a Theory of Systems. System Development Corporation document, TM-602/400/00, January, 1963. 57. Le Grande, E. Development of a Factory Simulation System Using Actual Operating Data. Los Angeles: Hughes Aircraft Company, March, 1962. 58. Matheson, James E. Optimum Teaching Procedures Derived from Mathematical Learning Models. Report CCS-2 prepared for the Institute in Engineering, Economic Systems, Stanford University, August, 1964. 59. Murphy, Judith, and Sutter, Robert. School Scheduling by Computer—The Story, of GASP. New York: Educational Facilities Laboratories, Inc., 1964. 60. Roe, Arnold. An Adaptive Decision Structure for Educational Systems. Report No. 63-63, Department of Engineering, University of California, December, 1963. 61. Rosenthal, D ., Rosenstein, A. B., and Tribus, M. How Can the Objectives of Engineering Education be Best Achieved? A report of the Educational Development Program prepared by the University of California, Department of Engineering, Los Angeles. 62. Rosenthal, D ., Rosenstein, A. B., and Wiseman, G. Curricular Synthesis. Educational Development Program Report No. 3-63, prepared by the University of California, Department of Engineering, Los Angeles, May, 1964. 63. Rowe, A. J. Computer Simulation Applied to Job Shop Scheduling. A Report of Simulation Symposium, Co sponsored by the AIIEr. ORSA, TIMS, February, 1958. 64. Shaycroft, Dailey, O rr, Neyman J r ., and Sherman. The Identification, Development, and Utilization of Human Talents. Technical report to the United States Office of Education, Cooperative Research Project No. 566, Pittsburgh, 1963. 160 65. Voress, H. E ., Hauser, E. A ., and Marsh, F. E. Critical Path Scheduling—A Preliminary Literature Search. United States Atomic Energy Commission, Reference TID-3568, October, 1961. 66. Yett, Frank A. Simulating Populations School Simulation Vehicle Instruction. System Development Corporation document, TM-1493/301/000, May, 1964. 67. Young, Karen. A User*s Experience with Three Simulation Languages (GPSS, SIMSCRIPT, and SIMP AC), System Development Corporation document, TM-1755/000/00, February, 1964. Unpublished Material 68. Akin, G. "Class Scheduling on the IBM-1401 -7040." Report prepared for IBM Corporation, 76 Adams Avenue, West, Detroit, Michigan, October 15, 1963. (Mimeographed.) 69. Delay, Don. "Stanford School Scheduling System—School Manual." Draft document, Version No. 1, Stanford University, School of Education, February 22, 1964. (Mimeographed.) 70. Little, Edward F. "A Computer Program for Budget Fore casting. " Harvey Mudd College, Claremont, California, March 11, 1964. (Mimeographed.) 71. Platt, W. J. "Educational Policy for Economic Growth." Paper read before the 8th Annual International Meeting of the Institute of Management Sciences, Brussels, Belgium, August, 1961. * ' 72. Reisman, A. "A General Model for Production System s." Unpublished Ph. D. dissertation, Department of Engineering, University of California, 1963. _/• 161 73. Roe, Arnold. "Preliminary Draft for the Panel on Teaching Methods." Appendix VII, Report of the Conference on Honors Programs in Engineering, Boulder Colorado: University of Colorado (June 10-14, 1963), 163. 74. Rosenstein, Allen B. "Design of a Unified Engineering Curriculum ." A paper presented at the Annual Meeting of the American Society for Engineering Education, University of Maine, Orono, Maine, June, 1964. 75. Taft, M artin I . , and Reisman, Arnold. "The Conservation Equations: A Systematic Look." Unpublished paper accepted for publication in Journal of Hydrology, 1965. (Mimeographed.) 76. Thue, H. J . , Maynard, B. I ., and Shappell, N. H. "Time Reduction C urves." Proceedings 5th Annual Industrial Engineering Institute, University of California, January 30-31, 1953, 27-34. 77. Trent, Horace M ., and Trent, Eva Mae. "The Engineering Science Program at Dartmouth College Contents of the Core Curriculum ." Thayer School of Engineering, Dartmouth College, 1962-63. (Mimeographed.) 78. United States Army Ordnance Center and School, Aberdeen Proving Ground, Maryland. "Programmed Learning," April, 1963.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

The Study Of Continuum Normal Shock Structure By Perturbation Methods

PDF

A Monte Carlo Evaluation Of Interactive Multidimensional Scaling

PDF

Swim Bladder Dynamics And Acoustical Radiation

PDF

Generalized impact stability relations for a distributed elastic body in a conservative potential field

PDF

An Approximation To The Optimal Control Law For Stationary Systems Via Liapunov'S Direct Method

PDF

The Automation Of Workplace Design: Problem Formulation And Solution Under Several Algorithmic And Heuristic Approaches

PDF

Nonlinear Response Of A Circular Membrane To Acoustic Excitation

PDF

Jet Mixing Of A Two-Phase Stream With A Surrounding Fluid Stream

PDF

The Flutter Of A Triangular Panel

PDF

Nonlinear Large Deflection Vibrations Of A Cantilever Beam

PDF

The Natural Modes And Frequencies Of A Uniform Curved Beam Vibrating In The Plane Of The Beam

PDF

A Dynamic Study Of A Beam On The Pasternak And Tensionless Elastic Foundations

PDF

On The Derivation And An Illustrative Use Of Passive Element Electrical Analogs For Elastic Structures

PDF

Vapor-Liquid Equilibria For The Benzene - N-Octane System Near The Regionof The Critical-Locus

PDF

Stationary Radial Source Flow Of Liquid Particles Into Vacuum

PDF

Cost-Effectiveness--A Criterion For Component-Subsystem Design: A Case Study Applied To Military Airlift

PDF

Ultrasonic Testing Techniques To Study Water Pollution And Wound Healing

PDF

Situational Analysis: A Methodology For The Study And Practice Of Administrative Change

PDF

Measurement Of True Angle Of Attack And True Angle Of Sideslip By Sensing The Motions Of An Aircraft

PDF

An Eigenvalue Study Of An Euler-Bernoulli Beam Supported At Two Arbitrarypoints

Asset Metadata

Creator Taft, Martin Israel (author)

Core Title Toward A Systems Analysis Approach To Engineering Education; A Heuristic Model For The Scheduling Of Subject Matter

Degree Doctor of Philosophy

Degree Program Engineering

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Engineering, General,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Advisor Choudhury, P. Roy (committee chair), Abrahamson, Stephen (committee member), Freberg, Carl Roger (committee member), Georgiades, William (committee member), Rowe, Alan J. (committee member)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c18-195568

Unique identifier UC11360720

Identifier 6605497.pdf (filename),usctheses-c18-195568 (legacy record id)

Legacy Identifier 6605497.pdf

Dmrecord 195568

Document Type Dissertation

Rights Taft, Martin Israel

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA