00001.tif

PDF

Download

Open document

Transcript (if available)

Content INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. A Bell & Howell Information Com pany 300 North Z eeb Road. Ann Arbor. Ml 48106-1346 USA 313/761-4700 800/521-0600 EDUCATION AND TRAINING of the DEPARTMENT OF DEFENSE ACQUISITION WORKFORCE under PUBLIC LAW 101 - 510 U.S. CODE, TITLE 10 VOLUME I by Jay William Gould 1 1 1 A Dissertation Presented To The Faculty o f the School o f Public Administration o f the University o f Southern California In Partial Fulfillment o f the Requirements for the Degree o f Doctorate in Public Administration August 1995 ©1995 Jay William Gould Iil UMI Number: 9617100 UMI Microform 9617100 Copyright 1996, by UMI Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. UMI 300 North Zeeb Road Ann Arbor, MI 48103 UNIVERSITY OF SOUTHERN CALIFORNIA SCHOOL OF PUBLIC ADMINISTRATION UNIVERSITY PARK LOS ANGELES, CALIFORNIA 90089 This dissertation, w ritten By Jay. W illiam . Gipuld,.. I l l ........................ under the direction o f h .i.?... ‘ Dissertation Committee, a n d approved By a C C its mem- Bers, has Been presented to a n d accepted By the fa cu fty o f the School c f TuBBic Skhninistration, in pa rtia l fuC fitfm ent o f requirements o f the degree c f D O CTO R OF PUBLIC ADM INISTRATION DISSERTATION COMMITTEE DEDICATION In DEMING: The Way We Knew Him (Voehl et al. 1995), I stated three things came into my life which have changed things forever. My acceptance into the University of Southern California's Masters in Systems Management Program and later the Public Administration Doctoral Program. • Becoming a protege of the late Dr. W. Edwards Deming Meeting and immediately marrying the most gracious lady I have ever know, May-lum. From USC's most gifted of professors, as a self-directed learner, I gained knowledge and an understanding of the value of a personal empowering vision . Dr. Deming infused me with a passion for excellence. May-lum taught the essential ingredient in understanding cultural diversity is trust. These three elements are my foundation: Empowering Vision • Passion for Excellence Trust To the Professors of USC, Dr. W. Edwards Deming and May-lum Gould, I dedicate this dissertation. You have changed my life forever. ACKNOWLEDGEMENTS A dissertation is defined as being the product of a an individual's research and contribution to the discipline. But in reality, a dissertation is the product of a team of people woven intricately together. Chet Newland, who on my first day in class spoke of the personal will I must have to overcome cancer. Warren Schmidt, whose warm heart and love of mankind permeated my soul and being. • Carl Bryant, whose brilliant mind defined the task before I knew the relevancy of it all. John Snoderly, a buddy full of support, who went along with idea we would complete the doctorate program together. To my College, Defense Systems Management College and the Defense Acquisition Workforce Improvement Act that made all of this financially possible. The Ladies of the Library; Marry Ellen Tipper, Jane Cohen, Mila Gonzales, and Thelma Jackson. My eyes, ears, and gatherers of data - true professionals. The USC / WPAC Class o f '92 peer group that rocked my world and let me understand others see things differently than I. Eddie Moses, author, practical realist, "proof reader," and deep friend. May-lum, whose love brings great meaning into our lives. But most of all, I thank our God, who has watched over me all these years, guarding my body and mind, and loving me. iii TABLE OF CONTENTS VOLUME I: CHAPTER I ............................................................................................................. 1 Introduction: Cold War Afterm ath.............................................................. 1 Background of the Defense Acquisition Problem .................................... 7 Key Points of the Acquisition Workforce B i l l ...........................................9 Statement of the Acquisition Management Problem Situation ............. 11 Purpose of the S tu d y ....................................................................................12 Areas to be Investigated ............................................................................. 15 Defining the Acquisition Culture .............................................................. 17 Myers-Briggs Type Indicators® (M B T I® )...................................19 Perry's Learning Environment Preference (L E P )..........................21 PR O FILO R ® ................................................................................... 22 Changing a C ulture...........................................................................24 Delineation of the Research Problem ........................................................24 Statement of the Research Q uestion..........................................................27 Importance of the Study .............................................................................30 Definition of Terms ................................................................................... 32 Delimitations of the Study .........................................................................33 Outline of the Remainder of the Dissertation 34 CHAPTER I I ........................................................................................................... 36 Review of Related Literature .....................................................................36 Organization......................................................................................36 Section A: Congressional, DoD and Related Literature: D A W IA ............................................................................................ 37 Special Studies Buying Aircraft: Material Procurement for the Army Air F o rc e s.................................................................. 40 Hearings: Defense Acquisition Process, Before the Committee on Armed Services, United States Senate: 1988 (S. Hrg. 100-963) .......................... 45 The Quality and Professionalism of the Acquisition Workforce: Report of the Investigations Subcommittee of the Committee on Armed Services, House of Representatives - 101 Congress, 2nd Session, May 8, 1990 .................................................................... 55 Remarks by Mavroules (D-MA) on H.R. 5211: Creating an Acquisition Work Force, June 28, 1990 Press Release: Key Points of Acquisition Workforce Bill Nicholas Mavroules: Floor Debate - Acquisition Workforce Nicholas Mavroules: 1991 DoD Procurement Conference - Williamsburg, VA ...............................................74 v Life Is Too Short: A Review of the Brief Periods that Managers of Major Defense Acquisition Programs Stay on the Job; Report of the Investigations Subcommittee of the Committee on Armed Services, House of Representatives: 1990..................................................................... 78 National Defense Authorization Act for Fiscal Year 1991: Report of the Committee on Armed Services, House of Representatives on H.R. 4739 National Defense Authorization Act for Fiscal Year 1991: Conference Report to accompany H.R. 4739 10 U.S. Code Chapter 87 - Defense Acquisition W orkforce as of March 25, 1994................................................ 83 A History of the Defense Systems Management College: David P. Acker, 1986...................................................................... 87 The Defense Management Challenge: Weapons Acquisition: Dr. J. Ronald Fox and Mr. James L. Field (1 9 8 8 )............................................................ 93 Obstacles to Improving the Defense Acquisition Process: .... 98 Critical Issues in the Defense Acquisition Culture: Government and Industry Views from the Trenches: Dr. J. Ronald Fox and Mr. George Krikorian.............................. 102 Department of Defense Instruction (DODI) 5000.52-M: Reporting Management Information on DoD Military and Civilian Acquisition Personnel and Position (1991)........... 113 Department of Defense Manual DoD 5000.52-M: Career Development Program for Acquisition Personnel (1991) ...........................................................................114 VI Department of Defense ADS-93-01-GD: Acquisition Career Management, Mandatory Course Fulfillment Program and Competency Standards (1993)........................................................................... 119 Incentives of the Program Manager: Unpublished Student Report (1994) ........................................ 120 The Role of the Government Program Manager Unpublished Student Report (1994) ........................................ 126 Section B: Jungian / Myers-Briggs Type Indicator (MBTI) .............134 MBTI's Scientific Establishm ent...............................................138 Type Development ..........................................................153 Teaching Styles ................................................................155 Section C: Perry Learning Environment Preferences (LEP) / Adult Learning Concepts................................................................................... 169 Matching Teaching & Learning Styles...................................... 183 Theories of Learning ..................................................................184 B ehaviorists...................................................................... 184 C ognitive...........................................................................186 Humanist ...........................................................................187 Social Learning ................................................................188 Current Theory and Practices..................................................... 189 English Adult Learning Approach ............................................... 191 Androgogy ................................................................................... 192 Leaming-To-Leam...................................................................... 198 Conference on "Learner Driven Learning"................................201 Adult Memory .............................................................................207 Experts vs. Novices .................................................................... 209 Educational Effects on Learning ...............................................211 vii Section D: Multi-rater - 360 Degree Rating Metrics / PROFILOR . . 213 Instrument Comparative Analysis .............................................214 360 Degree Agglomerations Techniques - Team Evaluations .218 Trade and Media A rticles............................................................221 PROFILOR....................................................................................228 PROFILOR User's Conference S em in ar.................................. 234 Section E: Implementation and Guidance from Department of Defense (DoD), Defense Acquisition University (DAU), Acquisition Management Functional Boards, and O th e rs............................................................................. 241 PMC's Competency List: Beginnings and E volutions.............244 Student Assessment Methods ....................................................247 USAF Air University Approach................................................. 251 CHAPTER III: M ETHODOLOGY................................................................ 256 Subjects of Study......................................................................................256 Data C ollection........................................................................................257 Instructor Data Set .......................................................................258 Statistical B a s e ............................................................................. 258 Research Approach and D esigns............................................................260 Pilot Project in Adult E ducation............................................................260 Contract Learning Background................................................... 261 The Pilot Study P ro je c t..................................................... 262 Hypotheses...........................................................................263 Experimental D esign ..........................................................263 Methods of Gathering Data ..........................................................264 Questionnaire.......................................................................264 Data Collection .................................................................. 265 Scoring Procedure ..............................................................265 Method of A nalysis............................................................266 Control Group Interactions............................................................267 Section C's Project Kaizen Products........................................................268 Longitudinal Study - Pilot Project Adult Learning PMC 94-1, Section C .................................................................... 268 Qualitative R esponses....................................................... 269 Data Processing and A nalysis.................................................................. 273 Comparative Analysis: FSAM / ISAC / PMC Perry LEPs:....273 H ypothesis...........................................................................274 Experimental D esig n ..........................................................274 Methods of Gathering D a ta ...............................................275 R e su lts................................................................................. 275 Integrated Analysis MBTI and Perry L E P .................................. 276 PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs ...........................................................................278 PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 93-1 ................................................... 278 PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 93-2 ................................................... 280 PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 94-1 ................................................... 282 PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 94-2 ................................................... 283 Sum m ary.....................................................................................................285 CHAPTER IV: FINDINGS .........................................................................287 Statistical Quantitative Data Findings..................................................... 287 A Pilot Project in Adult Education using Nonequivalent Control Group Design / PMC 94-1 (Project Kaizen) Interpretation of Results and D iscussion.........................................................................288 Longitudinal Study - Pilot Project Adult Learning: PMC 94-1, Section C ............................................................................... 290 Comparative Data Analysis: FSAM / IS AC / PMC - Perry LEPs ....................................................................................291 PMC Integrated Data Analysis: MBTIs and Perry LEP Integrated Analysis MBTI and Perry L E P .................................. 292 Temperament Differences ............................................................294 PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs for PMC 93-1, 93-2, 94-1, 94-2 and 95-1 ................................ 295 Perry LEP Aggregated and Agglomerated Distribution vs Blooms' Curriculum Level for PMC 93-1 - PMC 95-1 ..................... 297 Initial Summary.......................................................................................... 300 Qualitative Data Findings...................................................................... 301 Gould's Open Systems Generic Model .......................................301 External Environm ent........................................................301 Stakeholders.........................................................................303 Congress Career an Functional Boards Sixteen DAU Consortium Members DAU Trends ..................................................................................304 Threat Down Sizing Budget Societal ............................................................................... 305 Organizational Purpose ......................................................305 Transformational Process Technical and Social Defense Systems Management College Acquisition Reform Communications Center XI Curriculum Faculty In p u t..................................................................................... 308 Supplier (DACMs) Student Body Feedback from the PMC Course to the Student Custom er..............................................................................310 Output Measurement Evaluation.....................................310 DAU Student Assessment Process Guide Feedback from the Student to DSM C-PM C Feedback Output To Input.................................................311 Partial Summary ........................................................................................312 CHAPTER V: SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS......................................................313 Sum m ary.................................................................................................... 313 Professional Military E ducation..............................................................317 Reframing and Reformatting to Fit the Environm ent............................319 Behavioral Change / T ea m s..........................................................319 Guided Self-Directed Learning................................................................322 Conclusions................................................................................................323 Recommendations................................: ...................................................324 Paradigmidic Cultural Change ......................................................325 Research Question......................................................................................326 Bibliograhy................................................................................................. 328 Definitions...................................................................................................350 APPENDIXES: Appendix 1. Competencies & Bloom's Level Required for Certification in the Program Management Career Field -- Consortium Baseline........................................ 361 Appendix 2. DoD Acquisition Career Management: Mandatory Course Fulfillment Program and Competency Standards, January 1993................................................393 Appendix 3. Perry Learning Environment Preferences Instrument........................................................................... 410 Appendix 4. Perry LEP and CCI Statistics Total PMC 93-1 through 95-1...................................................416 Histograms: PMC 94-1 / Sections A through N ......................420 Appendix 5. MBTI Frequency Distribution Total PMC 93-1 through 95-1...................................................448 Frequency Tables APMC and PMC 95-1 Sections A through N .................................................................450 Appendix 6. Student Led Acquisition Management Activities Questionnaire..................................................................464 Appendix 7. Cross Tabs Tables and Bar Charts Perry LEP / CCI Distributions w/ MBTI 467 LIST OF TABLES 1. Comparison of Temperaments: National Norms vs. PMC........................ 20 2. Tenure Requirements........................................................................................ 81 3. Three Career Levels........................................................................................117 4. Comparison by Type: PMC and US Surveyed Population.......................137 5. Self-Select Ratios (SSR).................................................................................160 6. Asset Predictors................................................................................................165 7. Liability Predictors............................................................................................165 8. Grant's MBTI Age Development Theory.................................................... 167 9. Moore's Nomenclature.....................................................................................176 10. Key Learning Theories...............................................................................189 11. Fonnal, Open, and Social Domains.......................................................... 203 12. USA TODAY Top Management Skills................................................... 222 13. Statistical Base of Study.............................................................................258 14. Courses, Careers, and Bloom's Levels........................................................273 15. Perry LEP Results: Course, Careers, and Location Studies......... 275/276 16. Agglomeration Course Levels LEP Means.................................................276 17. Satisfaction Index: Intrinsic, Extrinsic, and General - Pilot Study. . .289/290 18. Longitudinal Study: Pilot Project Adult Learning..................................290 19. Course, Bloom's Teaching Level, and Perry LEP Means.........................291 20. MBTI Significant Difference Between Types - Tukey H S D .................. 293 21. Temperament Significant Difference - Tukey HSD...............................294 22. Comparison of Bloom's Levels, Perry's LEP and Moore's CCI.............. 297 23. Student Statistics: Learning Levels, Frequency, and % Population. . . . 298 24. Comparison: Bloom's Instructional Levels, Student's Population and Curricula Percentages......................................................................... 299 FIGURES 1. The Functions....................................................................................................140 xiv LIST OF CHARTS 1. Civilians and Officers: Acquisition Workforce.............................................. 58 2. Milestone.............................................................................................................88 3. Experiential Learning......................................................................................... 90 4. The Theory: Dominant and Auxiliary Functions for Each Type................. 144 5. Keirsey - Bates Temperaments........................................................................158 6. Temperament: Strengths and Pitfalls..............................................................159 7. Learner Characteristics Implied by the Perry Scheme.................................. 182 8. Mocker and Spear: Learner vs. Institution Model........................................ 196 9. Personal Responsibility Orientation (PRO) Model....................................... 197 10. Relationship Between Autodidaxy and Learner Control of Instruction . . .198 11. PROFILOR Wheel..........................................................................................231 12. DSMC's Description: Guided Self-Directed Learning................................255 13. PROFILOR Agglomerated Group Report: Strengths vs. Developmental (Training) Needs............................................296 LIST OF GRAPHS 1 . Z Solution..........................................................................................................154 2. Gould's Generic Open Systems Improvement Model...................................302 xv ABSTRACT Congress passed the Defense Acquisition Workforce Improvement Act (DAWIA) in 1990 mandating a cultural change in the Department of Defense (DoD) acquisition workforce. This descriptive study is centered on the education and training of the acquisition program managers conducted by the Defense Systems Management College. Acquisition workforce personnel are uniformed officers and civil servants who self-select the Program Management career. The DSMC/ Program Management Course (PMC) curriculum is competency based using Bloom's Taxonomy Levels 3-6. The purpose of the study was to determine if the DoD competency skill set produces desired changes in the acquisition workforce culture. Norm-referenced student data were gathered on Myers-Briggs Type Indicator® (MBTI®) and Perry Learning Environment Preferences / Cognitive Complexity Index (Perry LEP/CCI) and also the 360-degree multi rater PROFILOR®. Sample sizes were MBTI - 5,463; Perry LEP/CCI - 1,922; PROFILOR - 1,908 (15,980 total observations). The study covers the period from July 1985 to July 1995. An experimental adult learning pilot project with a control group was accomplished and is reported. The result of xvi a longitudinal study indicates initial pilot experimental group continued behavioral change in new assignments after graduation. Frequency distribution studies of MBTI correlated with Perry LEP/CCI and Bloom's Instructional Levels were accomplished. Perry LEP/CCI range is 200 to 500, mean-363.9, standard deviation - 49.1. Perry LEP/CCI 200 corresponds to learning duality - teacher driven; 500 corresponds to multiplicity and autodidaxy. Cross tabulations of MBTIs and Perry LEP/CCIs indicates hierarchical invariant nature of adult learning in conflict with Bloom's Instructional Level, creating student learning difficulty and frustration. Findings indicate Perry LEP/CCI Level 2 students (1.5%) have difficulty with Bloom's Teaching Level 4 and up, 57.9% curriculum. Level 3 students (21.1 %) have difficulty with 24.8% of curriculum. Level 4 students (33.1%) have difficulty with 9.1% of curriculum. Anova (Tukey HSD) indicates PERRY LEP/CCI means for temperaments NF and NT significantly different from SJ. PROFILOR training needs analysis of Program Management population indicates developmental needs in team training, processes and operations. xvii General conclusion: Short-term curriculum strategy experiences difficulty achieving Congressionally mandated behavioral change. CHAPTER I Introduction: Cold W ar Aftermath "When people live in encapsulated worlds, it becomes difficult fo r them to grasp the realities o f worlds with which they have little experience" (Hermstein and Murray, 1994). The Acquisition Workforce of the Department of Defense under the Congressional mandate of the 1990 Defense Acquisition Workforce Improvement Act (DAWI A) is comprised of both uniformed military and civil service personnel who, under provisions of the Act, self select their career field. During the Cold War with the Soviet Union, this highly dedicated group were charged with the development and fielding of the modem weapons of defense that established and maintained a superior technical and war fighting capability favoring the United States. In retrospect, they accomplished their task and, in doing so, they defined a unique culture for the acquisition process. It is this culture, and the laws, rules and regulations under which it operates, that forms the basis of this descriptive study. When significantly large sums of Government money are authorized for the accomplishment of any governmental program, one frequently finds fraud, waste, and abuse. This carry-over from the Jackson spoils system has found its way into most every Administration. Vice President Agnew in the Nixon Administration was forced to resign, HUD and Pentagon " 1 1 1 Wind" scandals of the Reagan/Bush Administrations, and the "Irangate" pardons of the Bush administration, define a federal system that at times is ethically compromised. Throughout American history, the procurement of war materials is sprinkled with time delays, cost increases, and some corruption (SASC & HASC hearings and investigations, 1988-1990). From the Navy's 1800 circa purchase of the frigates Constitution and Constellation to the massive procurement for the Civil War and World War I, the "Iron Triangle" between Congress, industrial suppliers, and military buyers has generated conflicts of interests, political constituency support, and profiteering (Holley, 1962) (Wilson, 1989). Article I, Section 8 of the Constitution requires Congress: To raise and support Armies, but no Appropriation o f Money to that Use shad be for a longer Term than two years. To provide and maintain a Navy. To make Rules fo r the Government and Regulation o f the land and naval forces. and goes on to say, To exercise exclusive Legislation in all Cases whatsoever, over such District (not exceeding ten Miles square) as may, by Cession of particular States, and the Acceptance of Congress, become the Seat o f the Government o f the United States, and to exercise tike Authority over all places purchased by the Consent o f the Legislature o f the State in which the Same shall be, fo r the Erection o f Forts, Magazines, Arsenals, dock-Yards, and other needful Buildings; - And To make all laws which shall be necessary and proper fo r the carrying into Execution the foregoing Powers, and all other Powers vested by this Constitution in the Government o f the United States, or in any Department or Officer thereof. The Constitution gives to Congress the responsibility to make all laws defining the methods and procedures by which war materials are acquired. These Congressionally mandated acquisition laws are "faithfully executed" by the Executive Branch by the establishment of rules and regulations that are administratively implemented by the Department of Defense. As recorded by Holley in his official history of military procurement, as late as the Korean War, well into the Cold War, major procurement decisions were relatively decentralized. Military officers with the rank of Major or Lieutenant Commander (0-4) made major procurement decisions, without the Pentagon's review and approval (Holley, 1962). In the Kennedy / Johnson Administration, Department of Defense Secretary McNamara's announced practice, implemented by regulation and directive, was to extend the Pentagon's centralized control over program management. The primary mechanism of control of procurement decisions 3 was accomplished by expanding the decision making data base with the inception of the Program Planning and Budgeting System (PBBS). This DoD system was structured to provide comprehensive administrative procedures and processes for the orderly management of authorized programs and monies required to operate the Defense Department for the next year. The implementation methods used to administer and to accomplish the PPBS provides increasing data and knowledge. As Landau and Stout found in 1979, when management gains more knowledge of how a system works, that knowledge then provides a basis for the extension of control and authority. Defense contracts are a source of large sums of money in a Congressional district. Constituent political pressures tend to entice Congress to add to the acquisition overhead costs, as a method of protecting their districts' vested interests. The Department of Defense in compliance to these new Congressional mandates not only conforms by the issuance of appropriate DoD regulations and directives but then adds an administrative regulation creep to insure compliance, causing an avalanche of administrative requirements affecting implementation of acquisition programs (Technical Report TR-3, Project Kaizen, 1994). 4 As noted by Dr. J. Ronald Fox, when the Honorable David Packard, former Secretary of Defense in the Nixon Administration, examined the circumstances under which uniformed officers were required by law, regulation, and directive to administer the acquisition programs under their control, he believed that an understanding of business principles and management and administrative procedures would mitigate, if not alleviate, their identified lack of knowledge. In a study of the DoD college established by Packard, Dr. Acker described him as a noted pioneer in the electronics industry who was known for his business expertise and technical knowledge. In 1971, Packard established the Defense Systems Management School (DSMS) for the education and training of uniformed officers and their civil service counterparts in the business of acquiring war materials, weapons and services. The military cultural imperative is, "only military combat command personnel, 'a user,' has the experience and knowledge to define and manage the acquisition of a weapon system development and accomplish its fielding." The Pentagon's cultural imperative was so ingrained, despite Congressional mandates that the separate Armed services (Army, Navy, Air Force, and Marines) independently decided that attendance at DSMS was not a 5 mandatory prerequisite to becoming an Acquisition Program Manager. The Defense Systems Management School name was changed to the Defense Systems Management College (DSMC) in 1976 (Acker, 1986). Fox noted that during the Reagan/Bush Administrations' defense build up years, public scandal erupted over portions of acquisition contracts primarily associated with the gaining of insiders' data via the use of Washington consultants. Some scandal centered on the violation of ethical principles by either personnel in the Pentagon, Washington consultants, Congressmen, or defense contractors. "Operation 1 1 1 Wind" was instituted in the Pentagon to overcome these problems. In other instances, such as the Navy's A -12 program, there appeared to be no deliberate intent to subvert ethical checks and balances. This was true of the Navy's A -12 fighter- bomber aircraft development program, where over optimism, quibbling and a less than frank status and information reporting system produced a climate of distrust among the Secretary of Defense, the Navy's Program Managers, the defense contractors, and Congress. This unfortunate situation finally resulted in the Secretary of Defense's cancellation of the A-12 Program, firing of the Naval Program Managers, and the resignation of the Deputy Secretary of Defense for Acquisition (Fox, 1994). Congress took judicious notice that 6 neither of the principal A -12 Program Admirals was a graduate of DSMC and actually had little, if any, acquisition experience. According to the former Congressman Nicholas Mavroules (Massachusetts), these Admirals, relieved of their commands, did not have a full understanding of the intricate administrative processes defined by law, regulation, directive, and procedure. Background of the Defense Acquisition Problem In 1990, the Subcommittee of Investigations, House Armed Services Committee, chaired by the Mavroules completed a year-long study of the difficulties faced in the orderly acquisition of war materials. Speaking for the subcommittee, comprised of Congressmen Larry Hopkins (Kentucky), Dennis Hertal (Michigan), John Tanner (Tennessee), Frank McCloskey (Indiana), and Andy Ireland (Florida), Mavroules noted: The 1980's seemed to have brought one scandal after another. In response, Congress enacted major procurement legislation every year from 1982 through 1986. Yet, Defense Acquisition horror stories go on. Something had to address these problems. In the past, reform initiatives focused on changing either the process o f acquisition, such as the rules and procedures, or the structure o f acquisition, that is the offices and agencies that do the buying. These are the first two legs o f the stool in procurement process. These are obviously critical elements and may need additional focus in the future. But they failed to take into account the most vital and critical area: the quality and capabilities o f the people who must work within the structure. People, I believe, are the 7 critical third leg of the stool, the one that makes it stand (added emphasis) (Mavroules, 1990). Contained within the National Defense Authorization Act, H.R. 4739, P.L. 101-510, Title XII., was the legislation that provided the necessary framework for a professional acquisition workforce — The Defense Acquisition Workforce Improvement Act (DAWIA). The legislation was structured to address and resolve many long standing issues. As Mavroules stated in his extended remarks in the Congressional Record of June 28, 1990, chief among these issues was the necessity to bring a cultural change to DoD. He stated in part, "I recognize that bringing about a cultural change can be a long, arduous task. What is needed — and I believe the time is ripe -- is a candid and open discussion about root causes -- to wit, what is wrong, what should be fixed, and how can we go about doing that?” (added emphasis) Later in these remarks Mavroules answered his own question when he said, "We recognize that character often is far more important than organization, procedures, or individual technical skills. But there is no way Congress can legislate standards of character for admission to the acquisition 8 corps. So we will concentrate on their skills, their experience, their education, and their training." The press release from the Subcommittee on Investigations listed many ameliorating refinements directed by the Act. Key Points of the Acquisition Workforce Bill • Establish an "Acquisition Workforce" with its own career management system within each military department. • At the top of the Acquisition Workforce, create an "Acquisition Corps" in each service and the defense agencies, comprised of both military and civilian personnel — the senior 10-to-15 percent of the workforce -- and reporting to the civilian Service Acquisition Executives (SAEs). • Direct that all senior or critical acquisition jobs may be held only by Acquisition Corps members. • Direct the Pentagon to create a Defense Acquisition Management University structure that could serve as the center for all training and research geared to developing a more professional Corps. • Provide that the Pentagon's Under Secretary of Defense for Acquisition will oversee the Acquisition Workforce and Corps policies to assure uniformity. 9 • Professionalize acquisition by raising the qualification standard for acquisition positions and authorizing a number of focused education and training programs. • Assure that Program Managers and other senior acquisition officials be kept on the job longer than in the past to provide greater continuity and management and personal accountability. • Direct that civilians get a better deal in competing for top acquisition workforce jobs and reducing or eliminating any built-in structural bias favoring the military (Mavroules, 1990). Mavroules' announced purpose and intent of the DAWIA was to change DoD's culture. He further explained, "I want to stop for all time the practice of assigning great pilots to manage billion-dollar aircraft procurement programs, when they've only had a minimal experience in acquisition. You'd never send a great pilot to perform heart surgery or defend your case in court. Why on earth do we send them to handle immensely intricate acquisition programs?" It appears his veiled reference was to the previously mentioned failure of the Navy to assign acquisition trained officers to the Navy's A -12 program. When the program came under extreme scrutiny and pressure during budget hearings, the ineptitude of the Admirals, caused by their lack 10 of education and experience in acquisition, not only led to the cancellation of the program and the resignation of the Under Secretary of Defense for Acquisition but also to disciplinary actions for many lower level personnel (Mavroules, 1991). Statement of the Acquisition Management Problem Situation To change a culture is to change behavior. In an ordered military hierarchy, this is neither an easy nor short time-cycle task. Mavroules observed: During the past 25 years, Pentagon officials have assumed that the acquisition process could be managed effectively at any level by generalists, technology specialists, and by military officers whose primary training and experience was in combat operations. . . . As such, many key acquisition leadership positions were held by people with little practical training or experience in industrial management, and only one or two acquisition assignments. . . . Managers in the top jobs shouldn't be getting OJT. They should have already earned their spurs. They should be appointed to acquisition leadership positions because o f their capabilities and outstanding performance in the field o f acquisition — not because they were successful battalion commanders, squadron leaders or ship skippers. Fox, a Professor of Management at Harvard, chaired the Curricula Committee in the establishment of DSMS in 1971. At that time, Fox was an Assistant Secretary of the Army. In 1974 he w rote,". . . most far-reaching reform would be the establishment of a clearly defined procurement career 11 field within the military, with senior procurement managers controlling assignments and promotions. Anything short of this will not resolve the continuing crisis in procurement management." In 1988 he again w rote,". . . efforts to establish military career programs for program managers have been resisted by the services." In 1990, Fox testified before the House Armed Services Committee's (HASC) Subcommittee on Investigations to the effect, ". . . reform efforts usually fade once the initiator moves on, and within a year or two the same set of problems resurface, indicating that little has changed" (Fox, 1986, 1988, 1994). Purpose of the Study In a discipline as complex as war material acquisition, there are many specialty domains or career fields. These career fields, defined in the law are: 1. Program Management 2. Systems Planning, Research Development, Engineering, and Test 3. Procurement (Contracting) 4. Industrial Property Management 5. Logistics 12 6. Quality Control and Assurance 7. Manufacturing and Production 8. Business, Cost Estimating, Financial Management, Auditing 9. Construction 10. Joint Development and Production with other Government Agencies and Foreign Countries (US Code) The Defense Acquisition University (DAU), advised by the Directors of Acquisition Management (DACM), the Career and Functional Boards of the Army, Navy, Air Force, and Marines, the sixteen DAU consortium member colleges and schools including , by law, the Defense Systems Management College, defines accession requirements and establishes the framework for educational curricula for all of these career fields. The Program Manager is charged with complete responsibility for the satisfactory accomplishment of the assigned program and all other career fields report to the Program Manager. This study is limited to an evaluation of the curricula of the accession courses identified within the Program Management career field: * Level I, Fundamentals of Systems Acquisition Management (FSAM) 13 • Level II, Intermediate Systems Acquisition Course (ISAC) • Level III, Program Management Course (PMC) The methodology implemented by DAU in establishing course structures was to ask the Career and Functional Boards to develop a list of desired competency criteria for curriculum incorporation. DSMC produced a listing of acquisition competencies developed over the years in an ad hoc manner. As noted above, DAU adopted a pattern of three-level training, shown in Appendix 1. The course structure was the same for all three levels, and the principal differences between the levels were course length and depth/complexity of the subject matter, as determined by the Defense Acquisition University's (DAU) adoption of Bloom's Taxonomy for the setting of educational objectives within each course (DAU, 1994). As illustrated in Appendix 1, Level I (FSAM), a 9-day course, is taught primarily at Bloom's 1. Level II (ISAC), a 4-week course is taught primarily at Bloom's 2. and Level III (PMC), a 14-week course starting in 1995, is taught primarily at Bloom's 3 - 6 (DAU, 1994). The purpose of this study is to explore and examine the academic environment, structure and content of these three Program Management 14 courses in the context of the adult learning needs and preferences of the students, recognizing that the Congressional intent of the Defense Acquisition Workforce Improvement Act (DAWIA) is to change the long standing DoD culture. Areas to be Investigated Many questions must be addressed and numerous areas must be investigated in examining the problem of how best to modify or change the cultural composition of DoD's acquisition workforce. Some of these questions remained unanswered. As envisioned by the Packard in 1971, the Defense Systems Management School was established to provide an answer to this question: Are the students receiving an adequate education in preparation for program management careers? In 1990 Mavroules envisioned: Defense Acquisition University as the intellectual centerpiece o f the entire acquisition system — a place where seminars and thought- provoking meetings — intellectual ferment — can help to change the mindset, to bring about a lasting cultural change, throughout the acquisition system (Mavroules, 1991). 15 Dr. I. B. Holley, Major General, USAF, Retired and Professor Emeritus, Duke University and Major General United States Air Force Retired posited: Educating acquisition types is a good idea, but I very much fear the 'education' will turn into 'training.' Some o f each, o f course, will be necessary, but we won't get the broad gauge men and women of vision we need unless the corps stresses true education. This — requires a recruitment and selection process which is able to find and attract individuals with imagination, initiative, and intellectual energy as well as character — meaning o f course those who don't need regulations to tell them what a conflict o f interest is (Congressional Record, 1990). Fox, Professor of Management, Harvard University, testified: It is obvious that a program manager cannot be assigned as a wing commander without years o f carefully programmed flight training and experience. By the same token, a pilot cannot manage effectively a complex industrial program without extensive experience and carefully programmed assignments in acquisition process and in industrial cost control (Congressional Record, 1990). Norm Augustine, CEO Martin-Marietta, succinctly summarized: As we learn how to manage peace, the need for a highly qualified and professional acquisition workforce has never been greater (Congressional Record, 1990). In the aftermath of the Cold War, the collapse of the former Soviet Union has deterred the dominant military war concept of mutual mass destruction via intercontinental ballistic missiles, submarine launched ballistic missiles, land and sea attacks by Cruise missiles, and air attacks by stealth 16 fighters and bombers, armed witli nuclear warheads. In the Army's view, this nightmare scenario included massive tank and artillery assaults on the European continent, with Soviet mechanized forces pouring through the terrain feature called the Fulda Gap followed by a rapid crossing of the Rhine River barrier. Military weapons are designed and created as a result of a Mission Needs Statement. These formal DoD documents define the needs or requirements to create and arm the combat forces necessary to conduct successful military operations against a potential armed aggressor with a defined threat. These military requirements, generated by 40 years of Cold War, are now no longer present. What now drives the Mission Needs Statement? Defining the Acquisition Culture Part of Webster's Ninth New Collegiate Dictionary definitions of culture is: the integrated pattern o f human knowledge, belief and behavior that depends upon man's capacity for team ing. . . the customary beliefs, social forms, and material traits of a social group. 17 Following this definition, the acquisition workforce is more clearly defined as an "acquisition culture" with its own myths, metaphors, and exemplars. The data gathered by the Investigations Subcommittee of the House Armed Services Committee led the Committee to determine that the "acquisition culture" was the root cause of the problems present in the execution of the policies, procedures, and processes in acquiring war materials. Congress viewed the "acquisition culture" as "organized anarchies" caused by the lack of definable accountability, special interest politics, and a lack of working knowledge of the system. This should not be a surprise, since the military literature is replete with phraseology such as the "military mind" and the "marionette mentality" (Holley, 1962). As an example, the military, under its Universal Code o f Military Justice requires courts martials to consider in their deliberations the uncodified "traditions and customs" of the service {Manual fo r Courts-Martial, United States, 1951). One goal of this study is to define the "acquisition culture" by the examination of specific self-report instruments completed by members of the acquisition corps. Triangulation methodology is utilized. The three instruments that are utilized in the triangulation are (1) the Jungian, Myers- Briggs Type Indicator (MBTI), (2) Dr. William Perry's Learning Environment 18 Preferences (LEP), adapted from Dr. Perry's Scheme of Ethical and Moral Development; and (3) Personnel Decisions Incorporated's, PROFILOR, a multi-rater instrument for gathering constructive feedback on management skills from superiors, peers, and subordinates. These instruments and their corresponding data, are discussed in significant detail in other chapters. Myers-Briggs Type Indicators® (MBTI®) As described in Isabel Briggs Myers's and Mary H. McCaulley's manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator. The purpose o f the Myers-Briggs Type Indicator is to make the theory o f psychological types described by C. G. Jumg (1921/1971) understandable and useful in people's lives. The essence o f the theory is that much seemingly random variation in behavior is actually quite orderly and consistent, being due to basic differences in the way individuals prefer to use their perception and judgement. In their publication, A Description of Psychological Type at the Defense Systems Management College. (Gould et al., 1994) presents a complete breakdown of MBTI data gathered over the past ten years from students attending the Program Management Course (PMC) at DSMC. It is hypothesized that the cumulative MBTI percentages of the aggregate PMC students define the distribution of "type preferences" or "temperaments" 19 representative of the acquisition corps. If this distribution were the same as the norms of the national population surveyed, the ratios would be the same. However, if the ratios are significantly different, it would indicate that people of specific MBTI type or temperament tend to choose, or tend to avoid, the Acquisition Corps' career fields. People who select the career field of Program Management in the Acquisition Workforce illustrate self-selection. A comparison of the National norms of the four Keirsey-Bates Temperament Types, Intuiter-Feeler (NF), Intuiter-Thinker (NT), Senser-Perceiver (SP) and Senser-Judger (SJ) as computed from the MBTI Atlas of Type Tables by LTC. William H. Cummings and the DSMC data published by Gould is illustrated in Table 1. COMPARISON OF TEMPERAMENTS: NATIONAL NORMS VS. PMC TEMP. NF NT SP SJ NATION 22.0 22.0 12.0 44.0 DSMC 6.0 34.1 9.3 50.6 (Cummings, 1990) (Gould, 1995) Table 1 20 Perry's Learning Environment Preference (LEP) Dr. William G. Perry, Jr. admits that it took him and his Harvard colleagues 20 years to discover the coherent interpretive frameworks through which students give meaning to their educational experiences. The faculty would laugh when they read, "I expected an organized effort to improve my grades. This has been the most sloppy, disorganized course I've taken. O f course, I have made some improvements, but this has been due entirely to my own efforts" (added emphasis). Perry concluded that comments such as these were structurings of meaning, which students revise in an orderly sequence from relatively simple to the more complex. They shape the students' ways of learning and color their motives for engagement and disengagement in the whole educational enterprise. Discovering the students' "Pilgrim's Progress" or ways of knowing, complete with "Sloughs of Despondency," he assigned each stage an interpretative number (Chickering, 1981). A paper and pencil derivative from the original student responses to Perry's oral prompts, developed by Doctors William Moore and certified in a National trial, became the base of this study. This instrument assesses the students' cognitive and ethical growth, the understanding of meaning from a 21 Perry LEP on a scale from 1 to 5, or from basic dualism through the search for synthesis (Moore, 1982). Cognitive growth of individuals is a function of their individual genetically based preferences, cultural environment, and educational opportunities. One or any combination of these elements can either help or impede individuals in reaching their full potential. The training, education and development of DoD personnel is driven by the needs of the Service. Gadeken (1989), defines DoD training as teaching assigned personnel the duties and function of the position or office so that they will be able to immediately accomplish the job or task upon the completion. Education is defined as job related instruction in preparation for a future assignment. Development is defined as non job related instruction. PROFILOR® The PROFILOR is a multi-rater instrument developed by Personnel Decisions, INC, Minneapolis, Minnesota and is designed to give managers feedback about their behavior which their supervisors, peers, and subordinates observe in their everyday life. The primary purpose of the feedback is to give an insight as to what a manager can do differently in 22 order to increase effectiveness. The focus of the profiles is on skills and behavior, not management style. The feedback provides a tool for growth and development by providing direction about strengths and developmental needs. The profiles are oriented around eight core factors critical to management success, cover twenty-four (24) skills matrix of one hundred thirty five (135) items, and provide each individual with an overall performance rating with norm group comparison. This research utilizes the PROFILOR integrated group report for the cultural evaluation basis. Modem weapons are so complex that their development requires the integration of a number of extremely technical disciplines. No single individual has that degree of technical knowledge. Modem weapons must, therefore, be developed by technical teams that are not adversarial in nature or disposition, that work well together, and that strive for win - win environments. To be competent in the career field of Program Management, an individual must understand and know how to manage teams. A 360- degree evaluation of an individual by superiors, peers, and subordinates, defines that person's managerial strengths and developmental needs. 23 Changing a Culture Since culture is defined by nonnal group and individual behavior, the changing of a group culture requires the changing of behavior or redefinition of group norms. Behavior changes are accomplished by unfreezing, and changes to behavior of a highly knit group typically require a significant emotional event to accomplish the unfreezing step (Lewin, 1947). Delineation of the Research Problem In the Second Session of the 101st Congress, the Investigations Subcommittee of the Committee on the Anned Services, House of Representatives published a report entitled, "The Quality and Professionalism of the Acquisition Workforce." The "Executive Summary, Chapter 1 - Introduction" states: There are three distinct elements within the Department o f Defense Acquisition System: 1. the policies, procedures, and processes which govern the operation o f the acquisition system. 2. the organization o f the resources (people, management structure, capital, and facilities) that execute the policies and procedures. 3. the people within the organization that make the system work. Although it has been recognized in studies and commissions (including the First and Second Hoover Commissions in 1949 and 1955, the Fitzhugh Commission in 1970, the Commission on 24 Government Procurement in 1972, and the Packard Commission in 1986) for over thirty years that the quality and professionalism o f the defense acquisition workforce should be improved, the great majority o f reform efforts have focused on changes in policies and procedures, or organization. The report goes on to say, Before considering the adoption o f any o f the myriad proposals for improvement o f the acquisition workforce, the Committee on .Armed Services believed it critical to conduct an in-depth analysis o f the state o f the acquisition workforce - including an analysis o f the qualifications and professionalism o f the acquisition personnel, as well as a review o f various DoD efforts o f the department to establish and manage the career development o f the acquisition workforce. The report attempts to answer four major questions: 1. Are the Services appointing Program Managers, Deputy Program Managers, and Contracting Officers with the experience, education and training required by law and regulation; and are Program Managers being retained in their positions the mandatory four-year minimum? 2. Is there a career program structure to develop qualified and professional contracting and program management personnel - both civilian and military? 3. Is there an appropriate mix o f military and civilian personnel within the workforce? 4. What impediments exist that must be overcome in order to develop a quality, professional workforce, and how can that be accomplished. The Committee took official note that the data utilized by the report was in many cases raw statistical data which was incomplete and unstructured or classified. The report reads, "Conclusions drawn from the 25 data must be viewed with the knowledge that there are problems with the data bases, either because the information is not pure because of lack of agreement on definitions used, or the data system does not track the information gleaned in the form utilized." The report was based on faulted data. Despite this condition, the Committee generally concluded (answers have been paraphrased for brevity): 1. Officers appointed to the position o f Program Managers and Deputy Program M anagers are not experienced, educated nor trained as required by law or regulation, nor are they retained in their position for a four-year minimum. 2. There is no career program structure in place to develop qualified personnel, military or civilian. 3. There is not an appropriate mix o f military and civilian personnel. 4. The root cause that must be overcome in order to develop a quality, professional workforce is DoD's culture. As a result of these findings, the Committee Chairman of the Investigations Subcommittee speaking for himself and the members of the Committee introduced on June 28, 1990, HR 5211, "Creating an Acquisition Work Force," specifying in his floor remarks, ". . . the only way to change the culture, a long and arduous task, is to grapple with the root causes by concentrating on the Acquisition Corps members' skill, experience, education, and training." 26 Statement of the Research Question Following the codification of the DAWIA in Chapter 87 of Title 10, United States Code, DoD published its directive 5200.52M dated 25 October 1991, entitled, "Career Development Program for Acquisition Personnel." The directive defined the eleven (11) career fields of acquisition and also specified the education, training, and experience requirements for each of the three levels within the career fields. Each DoD component (Army, Navy, Air Force, Marine Corps) certifies each member of their acquisition workforce as to the career field and level within each career field. The DoD service component (Army, Navy, Air Force, Marine Corps) certifies each member of their acquisition workforce as to the career field and level within that career field. The level certification is based on the mandatory standards (education, training, and experience) established for the specific acquisition career field published in the directive. The procedural aspects of the certification process were left to each service component. Following this directive, Congress incorporated in the "National Defense Authorization Act for Fiscal Year 1993," Section 812(c) of Public Law 102-484, requiring the Secretary of Defense to develop a fulfillment plan enabling acquisition workforce members to satisfy their mandatory 27 training requirements based upon previous experience, education, and/or alternative training programs. A guide was issued by DoD in January 1993 listing the required competencies associated with each mandatory Defense Acquisition University course required in the various functional career field paths. In December 1993, the Defense Systems Acquisition University (DAU), Consortium Schools, and the Career and Functional Boards held a conference to develop the core curriculum competencies for the career field of Program Management; Levels I, II, and III. The minutes of the meeting state, "The group agreed that the Level I Core course should be a knowledge based course which equates to a Bloom's level 1 or 2. The general consensus was the Level II Core course would be at Bloom's level 2 or 3 and be based mostly on comprehension and very little analysis." The competencies nor Bloom's level of the Level III Core course was not addressed, but the Bloom's level from 3 to 6 was acknowledged to be of the correct domain. In July 1993, a Career and Function Board for Program Managers was convened to establish a Competency list for the Program Management Career Field. The list of validated Program Management competencies 28 requiring formal schooling for in-residence formal educational institutions was published. Citing the DoD publication, DoDI 5000.52M, Career Development Program for Acquisition Personnel, the document described the entry requirements, position descriptions, and the set of duties which are appropriate for the assignment to personnel in the program management career field to be used in the management, organization, and development of training and education of Program Managers. Two years after the passage of the DAWIA, the fonnal competency education and training requirements to be used as the base for Program Management curricula development were established. With the accomplishment of this milestone, the research question is: The Congressionally stated purpose of the DAWIA was to change the culture of the acquisition workforce " by concentrating on their skills, experience, education and training." Will the competency based skill set defined for the Program Management Career Field provide the education and training to alter or change the acquisition workforce culture? 29 Importance of the Study Different views of reality or perceptions have a significant bearing on opinion and behavior. In a non scientific opinion survey of 80 government and industry managers, Critical Issues in the Defense Acquisition Culture: Government and Industry Views from the Trenches. Fox et al. found a commonality of opinions and views (perceptions) across industry and government. Three pervasive topical issues were identified that the authors believed to be central to the challenges and problems of DoD's acquisition process were identified.. 1. Ambiguities and conflicts in the roles of program advocate and program manager. 2. Burdensome oversight and review activities imposed on contractors, government program offices and DPROs by the DCAA, the military’ service hierarchy, OSD, the DoD Inspector General, the General Accounting Office, and the Congress. 3. Deficiencies in acquisition qualifications and training, including lack of a structured mechanism for capturing and conveying lessons learned from prior acquisition programs. The authors posit, "The study addresses the characteristic of DoD’ s acquisition process and culture that are viewed by government and industry practitioner alike as inhibiting the proper exercise of responsibility and that 30 undermine DoD's mission to deliver satisfactory weapon systems on a timely and cost-effective basis." The importance of the understanding of the stated goal of Congress to change the culture of DoD's acquisition community is powerfully and poignantly defined by the Fox study: Participants also acknowledge, however, that major acquisition programs cost hundreds of millions of dollars more than necessary and take years longer to conduct than is reasonable. It is only the immense economic strength o f the United States and its several hundred-billion-dollar defense budgets that have permitted the development and production o f impressive advanced- technology programs while absorbing cost over runs typically in the range o f 20 to 40 percent and more. Now that the defense budget has declined and is programmed to decline further, it is vital that acquisition process become more efficient and require shorter periods o f time. There are delineations of importance from other perspectives. With the collapse of the Soviet Union, the technology for the production of weapons of mass destruction (chemical, biological, nuclear) and even the weapons themselves, may be available to any nation economically able and willing to "upgrade" its military potential. Nations such as Libya, North Korea, Syria, India, Pakistan, China, Iran, Iraq, Serbia, and Republics of the Old Soviet Union have the potential of presenting a defined and real threat. In some respects, the World has become more dangerous and unpredictable 31 since the Soviet collapse. Massive U.S. nuclear strike forces are of little value in regional clashes. In this context, the importance of the study indicates not only a need to change the acquisition workforce culture but also to shift the emphasis from weapons of massive deterrence to weapons supporting limited regional conflicts, terrorist containment, non lethal engagements, and even "weaponless" peacekeeping. Definition of Terms A Competency Model of Program Managers in the DoD Acquisition Process (1990) states: In 1972, McClelland published the article, "Testing for Competence Rather Than for Intelligence," a broadside attack on the so-called general aptitude and intelligence tests. He stated that most o f these tests do not predict who will do well in a particular job, or career, and proposed instead the use o f proven measures of job-related abilities . . . For common understanding, we define competency as any attribute o f a person that underlies effective performance; a job competency is simply an attribute related to doing a job effectively (Klemp, 1979). An attribute in this context can take many forms- specialized knowledge, ability interest, trait, motivation, even a self-concept-but the only way such an attribute becomes a competency is in the relationship to performance . . . (Gadeken, 1989). 32 This general definition is acceptable, coupled with the caveat: the competencies are in a list provided by the Career and Functional Board for Program Managers to the Defense Acquisition University contained in Appendix 2. Delimitations of the Study The scope of study of this inquiry is limited to the examination of those adult educational processes implemented by DoD's Defense System Management College in the education, training and development of the Acquisition Corps' Program Managers enrolled in the Program Management Courses; ISAC (Level I), FSAM (Level II), and PMC (Level III). The principal focus is placed on the PMC, Level III Course. The overall purpose of the study is to evaluate where the curricula imposed by the Defense Acquisition University and the Career and Functional Boards will accomplish the intent of Congress when it passed the Defense Acquisition Workforce Improvement Act (DAWIA). That intent was to change the culture of the Acquisition Workforce and to overcome the difficulties experienced in war material acquisition. 33 Outline of the Remainder of the Dissertation Chapter II: Review of the Literature The literature review focuses on Congressional records and other materials scrutinized by Congress to obtain the legislative consensus for the passage of DAWIA. It also includes a review of the domains of Jungian Psychology as interpreted by Myers-Briggs Type Indicators, adult learning theory and learning preference measurement, 360 degree multi-rater instruments defining management strengths and developmental (training) needs. It notes a narrowed field of the directives and memorandums emanating from within DoD in implementation of the Congressional mandate. Finally, it looks at the products of cultural change developed by PMC students during the course of a pilot model adult learning experiment. Chapter III: Methodology By bringing separate data elements together, such as, MBTI, Perry Learning Environment Preferences and PROFILOR individual and group reports, the triangulation matrix formed provides definition to individual as well as organization behavior. Using these elements as a defining base, adult learning theories were replicated in experimental and control groups to determine student education satisfaction. Comparison of individual and 34 grouped data to formal competency lists exhibits areas of nonconformity and problematic definition. Chapter IV: Findings Employing the Statistical Package for Social Sciences (SPSS) to define, manipulate, combine, contrast, and analyze a large student population provides meaningful metrics of who, individually or grouped, the students attending the three DSMC Program Management Courses (FSAM / ISAC / PMC) are and the nature of their behavior (MBTI), cognitive positioning (Perry LEP), and environmental posture (PROFILOR). Chapter V: Summary, Conclusions and Recommendations Based on the findings, an answer is provided to the Research Question, "Will the competency based skill set defined for the Program Management Career Field provide the education and training to alter or change the acquisition workforce culture." 35 CHAPTER II Review of Related Literature Organization According to Congress, the "root causes of the problem" in acquisition of war materials is the culture of the acquisition workforce. The literature review will be accomplished by examining the separate domains of inquiry: Section A: The Congressional, DoD, and Related Literature: Defense Acquisition Workforce Improvement Act (DAWIA). Section B: Jungian/Myers - Briggs Type Indicator Section C: Cognitive Development / Perry Learning Environment Preferences Section D: Multi-rater 360 / Superior - Peer - Subordinate Evaluations Section E: Communications from DoD and DAU 36 Section A: Congressional. DoD and Related Literature: DAWIA Developments that lie behind 1990s education and training challenges in defense acquisition are most clearly disclosed in congressional hearings and reports, Department of Defense sources, and closely related publications by a few principal authorities in acquisition. This chapter selectively reviews that literature, focusing particularly on dynamics behind adoption in 1990 of the Defense Acquisition Workforce Improvement Act (DAWIA). It accomplishes this through a largely chronological review and with several crucial excerpts from key documents, because this information is basic and it is otherwise fugitive for most readers. For analytical clarity, perspectives of key authorities are discussed non-chronologically in part. This literature review discloses high, sustained congressional involvement in these developments over three decades. Through its oversight activities. Congress has extensively involved many organizations and leaders in deliberations. Besides incumbent and former DoD officials, it has called upon key business and university authorities as well as the U.S. General Accounting Office (GAO), the Congressional Budget Office (CBO), the Office of Management and Budget (OMB), and Inspectors General. 37 This analysis first examines the official history of aircraft procurement during World War Two. It draws briefly on a 1962 study by Dr. Irving Brinton Holley, Jr., an emeritus professor at Duke University. Holley's research-based historical observations and submitted written testimony became the cornerstone used by the House Armed Services Committee to justify passage in 1990 of the DAWIA. Second to be examined are key Senate hearings and House investigative reports conducted on the subject of acquisition during 1988 and 1990, culminating in the passage of the bill into law in 1991. Third is a brief recounting of the history of the Defense Systems Management College which has played a significant role in the education and training of the Program Managers in Acquisition. Incorporated within this review are the collected works of Dr. J. Ronald Fox, Harvard, who has played an ever-increasing role in how DSMC conducts its education and training of Program Managers and what curricula is taught. Fourth is an examination of those policies, regulations, and procedures implemented by DoD in compliance with the Congressionally mandated DAWIA. 38 The fifth and final portion of this section analyzes duties of a Program Manager as investigated and reported by DSMC Program Management students in an academic perspective. This section of the literature review is of critical importance to the understanding of the Congressional determination that cultural change of the acquisition workforce is the primary motivation behind passage of the DAWIA. 39 Special Studies Buying Aircraft: Material Procurement for the A rm y A ir Forces. (Hollet, 1964) Irving Brinton Holley Jr. is a Professor Emeritus at Duke University. He is also a retired Major General, United States Air Force Reserve. During World War II he was intimately involved in the planning and acquisition of aircraft. He subsequently wrote a history of the acquisition of aircraft and material by the United States Army Air Force. In 1962, The Office of the Chief of Military History, Department of the Army, published the book under the title, Special Studies Buying_Aircraft: Material Procurement fo r the Army Air Forces. In the Preface, Holley points out: Above all, the author wishes to make clear that he did not undertake this book as an exercise in fulsome praise. He may have leaned in the opposite direction, emphasizing unduly the failures while neglecting the successes. But if this kind o f history is to be useful and meaningful, it cannot afford to devote its limited number o f pages in adding to the paeans o f praise already in print. If the nation is to escape or even minimize the blunders o f the past, it cannot neglect to study its mistakes. In his magisterial 643 page volume, Holley brings life and character to the tumultuous time prior to W. W. II when the United States was attempting to build a capable Air Force. In a recent telephone conversation, Holley observed, "In reading the book you will see the kind of problems we faced prior to W.W. II are nearly the same as we are facing now." An examination 40 of the text confirms his assertion - arguments between the Air Force and the Navy over roles and missions; arguments within the Air Force itself on the type of plane to build, fighter vs. bomber; the news media questioning excessive profits; and official military reports to Congress providing disingenuous status. Fie identified how Congress was bewildered and may have well wondered what to believe, when asked to vote on questions vital to the Army Air Force. Subject to criticism by the General Accounting Office (GAO), the Army Air Force contracting officers countered with their own accusation of "pettifogging and penny pinching during perilous times." O f course, the aircraft industry in the 1930s was not an economic giant. On the industrial hierarchy list of all types of production, the aircraft production industry ranked about 14 or 15. The aircraft industry' in W.W. II was faced with its own technical evolution, such as propellers, first the development of wooden, followed by metal, and finally hollow metal variable pitch propellers. Anny Air Force procurement officers were not trained or educated in acquisition, and there was no understanding of corporate deficit financing. The overextension of the industry to permit expanded production, with possibly no government sustaining help, placed the industry in a precarious situation. Industry executives bore some of the responsibilities as Holley explains: Before signing a contract, manufactures were willing to promise the moon itself; after signing a contract they asked for change orders increasing the contract price for every little item not included when the original fixed price was under consideration. In fairness, it must be recognized that many such change orders requests were fully justified. The requests may have been justified, but the Chief o f the Material Division, as an officer directly responsible, began to have qualms. "Someday," he said, "there has to be an accounting." The cold shadow o f a Leavenworth cell block fell across the future. In summary, Holley goes on to say: . . . the procurement staff seemed to be fighting against utter collapse from fatigue. . . . The main problem o f purchasing was not one o f honesty but talent. The need of good people, educated and trained in acquisition, was emphasized by the notorious Detroit Tool Scandal. Contributing to this scandal were a hostile media, a well meaning public spirited outsider, unclear orders and communication breakdown between echelons of command, and a young officer trying to get a messy job done. Holley describes the conditions and the aftermath: Lacking trained tool men, the warehouse staff made slow progress in identifying and segregating the mass o f material. The job became no easier when the warehouse staff sent their only set o f blueprints to the W ar Production Board in Washington to get advice on disposal o f the items. The W ar Production Board suggested selling the special tooling as scrap. The Material Command then ordered disposal as salvage. 42 Apparently there was a misunderstanding o f the telephone calls between Wright Field and Detroit. Some where along the line the recommendation o f WPB relating only to the special tools was assumed to apply to all o f the items. Down at the other end o f a long chain o f command a Second Lieutenant began to dispose o f the tools in the Detroit warehouse. . . A Detroit manufacture who had come to the sale in search o f standard tools, horrified by what he saw, reported the whole affair to WPB. Soon the local newspaper was howling for scalps, and the story went out nation wide. A local editorial writer raged at "maladministration, muddleheadedness, and apparent venality" and called for "appropriate punishment" o f those responsible. The Air Force reaction to this violent attack was just what might have been expected. Several o f the Officers concerned were punished after investigation. A Lieutenant Colonel was permitted to resign, one lieutenant was reclassified and another was dismissed from the Army only to be drafted into service again as an enlisted man. . . At a distance, o f course, censure and defense are relatively meaningless. The real significance o f the Detroit tool case lies in the influence it had thereafter on .Air Force administration. Property disposal thereafter became a highly unpopular assignment among Air Force officers. Boldness in the exercise o f discretion might be the philosophy o f the Under Secretary, but the officers who read the headlines, "Procurement Chief To Be Replaced," would think twice before they pursued a common-sense course, if it involved any relaxation o f conventional regulations. It was safer to keep the record clear, even if it cost the taxpayers more. This ''play it safe" scenario of yesterday had its counterpart in 1993. After the controversial Inspector's General report on the C-17, the Chief-of- Staff of the Air Force declared, "Take no risk," when charges were leveled at senior Air Force acquisition officers. In the acquisition environment it does not seem to make any difference whether or not the acquisition officer is 43 educated and trained. Cheaper cost to the American taxpayer appears to be overridden by strict compliance with outdated regulations. As will be noted later, prior to Secretary of Defense McNamarra, entire weapon systems or major portions were acquired at minimum dollar cost under the direction and control of Lieutenant Colonels or Commanders (0-5s). Such programs included the Atlas, Titan I, Titan II, Minuteman and the Polaris Intercontinental Ballistic Missile (Packard Testimony, SASC, 1988). 44 H EARING S: D E F E N SE A C Q U ISITIO N PROCESS, Before the Committee on A rm ed Services, United States Senate: 1988 (S. Hrg. 1 G O - 963) Congress is often proactive in seeking ways to improve the acquisition system. Testimony at hearings conducted by either the Senate or the House is an excellent source of expert opinion data.. The Senate Armed Services Committee (SASC), Chaired by Senator Sam Nunn, conducted a hearing entitled, Defense Acquisition Process on July 11, 12, 27; August 4, 1988. The chronological list of witnesses included: The Honorable Charles A. Bowsher, Comptroller General The Honorable Frank C. Carlucci, Secretary of Defense The Honorable Robert B. Costello, Under Secretary of Defense - Acquisition The Honorable David Packard, former Deputy Secretary of Defense The Honorable James R. Woolsey, former Under Secretary of the Navy Dr. J. Ronald Fox, Professor of Harvard Business School Dr. Lawrence J. Korb, Brookings Institute Dr. Richard A. Stubbings, Duke University 45 Each hearing was introduced with opening statements by the Committee Chairman, Senator Nunn and other Senators who have a specific point to make for the record. Senator Nunn's statement included the point, "Two years ago the Packard Commission diagnosed the acquisition system as 'fundamentally ill'." Many points of acquisition's 'illness' were presented and discussed in the course of this important four-day hearing, but attention here is limited to the testimony concerning the education and training of those involved in the acquisition of war materials. Bowsher, Comptroller General, expressed dissatisfaction with the level of experience, education, and training present in the program offices: The defense acquisition process is extremely complex and requires skills and knowledge that can only be developed over an extended period o f time. Yet, frequently key acquisition positions are filled with people who are often unprepared for their assignments. While I believe that this is true at all levels, it is especially true at the program office level . . . As I see it, we need to increase the professionalism o f the procurement work force and in so doing, impart greater prestige to the individuals and compensate them accordingly. Prior to the hearing, Senator Nunn submitted a number of topical questions to those scheduled to testify that included but were not limited to: Centralized Acquisition Agency, Qualified Acquisition Personnel, Academic Preparation, Need for Prior Acquisition Experience, and Qualification for 46 Senior Acquisition Executives. In response to these questions, Bowsher prepared written answers: The Federal Government should take the lead in providing adequate on-the-job training, specialized classroom instruction, and career opportunities for the college graduates it hires as acquisition specialists . . . W ithout question, the demands placed on DoD's top procurement officials are such that management skills and prior experience with the Government's acquisition process, obtained either in-house or with industry, are important. . . . Acquisition is a complex process, issues and problems must be dealt with in a daily and continuing basis. Therefore, it is important that individuals selected for these key positions possess the knowledge, skills, and abilities to manage the acquisition process . . . The Service Secretaries should identify types o f program manager positions for which civilians should be considered. On the second day of the hearings, Nunn's opening statement summarized Bowsher's testimony, "... the defense acquisition system is plagued by serious management deficiencies." The written testimony to Nunn's questions submitted by Secretary Carlucci stated: We have been working to increase the professionalism o f the acquisition workforce for several years with many concrete accomplishments in this area, including: (a) establishing experience and training requirements for military and civilian acquisition personnel; (b) establishing an improved training structure overseen by the Defense Systems Management College (DSMC); and, (c) working with the Office o f Personnel Management to implement the Packard recommendation o f requiring 24 semester hours o f business for entry into the contracting field. Carlucci reflected the Pentagon culture in his answer to Nunn's question regarding a centralized acquisition agency. He avoided the specific 47 question and framed an answer to affirm the Pentagon's position, "Separation of these functions (requirement determinations: what systems, what capabilities are necessary to overcome specific military problems) could result in procurement actions being taken without the expertise of the military professionals who understand the problems." He added additional words to the effect," ... central procurement policy development remains in the Office of the Secretary of Defense, while operational execution of the procurement functions must remain within the Military Departments (services)." On the third day of the hearing, Nunn's opening statement reflected on the report of the 1986 President's Blue Ribbon Commission (referred to as the Packard Commission), "Chances for meaningful improvement will come not from more regulations, but only with major institutional change." The type of institutional change in acquisition referenced by Nunn and recommended by the Packard Commission was structural, procedural, and behavioral. Without using the words, the institutional change required is a change in culture. Packard presented several recommendations for the Senate's consideration. One such recommendation attacked the Congressional side of the iron triangle, suggesting that the practice of appropriating funds for unneeded military equipment and bases should be discontinued. The impact 48 of this "Congressional Perk " on Program Managers as discussed later. Packard's testimony centered on the people who accomplished defense acquisition: . . . they must be officers who have opted for a career in procurement. This Commission has observed that it has become quite clear that DoD acquisition process has become too complex to be managed by military non-careerist who will be rotated to other unrelated assignments as often as every 2 years. No amount o f intensive training will equip such individuals, no matter how bright and well-intentioned, to cope with this process. Packard drew specific attention to the excessive number of people in DoD involved in acquisition, and he recommended that Congress mandate a 20% reduction. He believed that a better job could be done with fewer people. His basis for making such a recommendation was his own personal experience with the Minuteman and Polaris Intercontinental Ballistic Missile Programs of the late 1950s. Now, the other lesson from these two early examples o f good acquisition practice is that they were done with small teams o f highly competent people. These people were put in charge. They were given the responsibility and authority to do the job, and they were left alone. That's why those programs were successful. Packard stressed in the Question and Answer portion of his testimony the importance of the civilians working in the acquisition process: I think it is important to try to do with the civilians something parallel to what has been done with the military over the course o f the last 10 years, in improving the quality and background and training. And here I would particularly commend this committee as 49 a whole, especially Senator Bingaman, and the work that all o f you have done in trying to improve the quality o f the civilians involved in the Department o f Defense in acquisition. On the fourth and last day of the hearings, Nunn expressed an anxiousness to proceed immediately with the testimony. However, in his prepared statement contained within the minutes, Nunn made biting commentary about the Pentagon's acquisition culture: Over the past 2 years, we have consistently urged the Department (of Defense) to submit legislative recommendations to remove impediments to innovation - yet all we received was a single proposal with several minor changes that even Under Secretary Costello dismissed as "pablum." The absence o f quality efforts by the Department in these areas underscores the fact that complaints from the executive branch about congressional "micromanagement" are largely attempts to divert attention from DoD's own management failures - failures illustrated by David Packard's comment that the Department had "flunked out" on implementing the Commission's recommendations from streamlining the acquisition chain o f command. The first witness to present testimony was Dr. J. Ronald Fox, Professor of Business Administration at the Harvard Business School. Fox, as a former Assistant Secretary of the Army, chaired the committee investigating the curricula for Packard in the establishment of DSMC in 1971. Since that time, he has also served as Deputy Assistant Secretary of the Air Force. In his testimony, Packard honored Fox by acknowledging his life time career study of DoD's acquisition of war materials. Fox's testimony was based in part on a 50 4-year study of the defense acquisition process entitled, "The Defense Management Challenge," which is discussed later in this section. Fox first enumerated the different nature of defense acquisition failures over the past few years and said: When an organization (business sic) undertakes a major program, it must assign persons experienced in industrial management, give them the authority needed to accomplish their jobs, compensate them well, and hold them accountable. . . Yet the Defense department has attempted to manage multi-billion dollar programs by placing military officers and civil servants in impossible situations, asking them to control work performed by giant industrial firms, but failing to provide them with tough negotiating skills, street smarts, and stable assignments required to deal with contractors on an equal footing. After first defining the different roles of a "liaison manager" versus an "active manager" which is summarized as promoting versus vigorously managing all elements of a program, Fox emphatically added: If the program manager job is limited to promoting a program, preparing progress reports, and performing technical liaison, then experience as a pilot, tank commander, ship captain or engineer, and possibly 14 to 20 weeks at the Defense System Management College, may be sufficient. . . But for those who hold the active manager view, as I do, military experience and 20 weeks o f training, or even 2 or 3 years o f training and experience are insufficient for a manager o f a large complex program, (added emphasis) Although the twenty-week Defense Systems Management College course is an important first step in training program managers and their staffs, much more needs to be done to train them to an acceptable level o f performance. The current policy o f limiting this training to 138 days, because o f regulations concerning moving- expense reimbursement, must be changed. Compared with the 51 multibillion dollars involved in the acquisition process and the opportunities for saving, moving expenses are trivial. Further, program managers should be required to complete a minimum o f one full year o f formal training in which they study hundreds o f examples o f the dilemmas both they and contracting officers have encountered. . . The emphasis should be on making analyses and decisions, using simulation exercises, case studies, role playing, and other techniques. An internship in a program management office would ideally precede and follow the one-year practical training program; carefully chosen program managers would serve as supervisors. Applicants (Acquisition Corps, sic) would face an aggressive screening program to remove those who do not meet the high standards o f the service. Advancement would be based strictly on management ability and performance. Only senior acquisition officers, . . . no interference from combat arms officers, would control assignments and promotions. Fox discussed various acquisition issues, stating, "Encouragement is not enough; improvements in management require significant changes in the system. . ." It is his fundamental belief that unless changes are made in the management of personnel, current "abusive" practices will yield unsatisfactory results: — Unless changes are made in the current military personnel system that makes short-term assignments necessary for military officers to acquire the number and variety o f assignments required for promotion, any significant reduction in personnel turnover in defense program offices is unlikely. — Unless changes are made in the current DoD practice o f waiving training requirements and offering only short training courses, which limit coverage to introductory rather than in-depth treatment of important subjects, it is unrealistic to expect improved training for acquisition managers. 52 He concluded his remarks by noting a comment made by Gilbert W. Fitzhugh, Chairman of the President's 1970 Blue Ribbon Panel. In summarizing, Fitzhugh felt: When the studies are completed and the committee members departed, those who remain to assess and implement recommendations are those whose 'toes have been stepped on' in the findings. There is, not unexpectedly, a noticeable lack o f enthusiastic support. Those responsible for implementation need an advocate remaining on the scene to lend support to the findings with understanding conviction. It was Fox's belief that the same lack of enthusiasm was present in the implementation of the 1986 Packard Commission report. The anecdotal information from the Pentagon indicated that in early 1987, an official said, The official word is, the Packard Commission recommendations have already been implemented. Have they been completed? No. It will take as long as the next reorganization to implement them. The testimony offered in this hearing offers a picture of a persistent and possibly pernicious culture within the Pentagon, whose self defined goal and mission remains unchanged despite the findings that might be presented by any Executive Branch Blue Ribbon Commission conducted under any administration, Democratic or Republican. It would also seem that this same behavioral attitude has been immunized to ward-off any Congressionally mandated changes that tended to mediate these behavioral tendencies. It is believed that cultural considerations such as these allowed Fox to conclude, 53 It is clear that the military and civilian leadership will need to be unambiguous in declaring the steps to be taken to improve management o f the acquisition process and persistent in the follow- up to ensure that the changes take place The last witness to appear before the SASC Hearing on the Defense Acquisition process was Dr. Richard A. Stubbing from Duke University. He opened his testimony by quoting Secretary C ostello," ... inefficiencies claim up to one-third (1/3) of all defense purchases." He then added, "This amounts to about $50 billion each year ..." He believed there were several underlying problems and presented them all. In attacking the iron triangle between the Congress (who purchases unneeded weapons) and the military acquisition community (who manages inattentively), he voiced special condemnation for the Pentagon war fighters who deliberately overstate the enemy threat, in order to steer the annual defense budget. Stubbing expressed his belief, "Congress, too, through political interference and an overemphasis on micromanagement, must also bear a share of the blame." 54 The Quality and Professionalism o f the Acquisition Workforce. Report of the Investigations Subcommittee of the Committee on Armed Services, House of Representatives -101 Congress, 2nd Session, May 8, 1990 Congressman Mavroules presented the findings of the Investigations Subcommittee of the House Armed Services Committee (HASC), a report over a year in preparation and 776 pages in length, as the empirical basis for the presentation of a bill "to grapple with the root causes," the third element of the acquisition system, "the people themselves" with the defined goal of "changing the acquisition culture through education and training"(Mavroules, 1990). In the Executive Summary of the report on the quality and professionalism of the acquisition workforce, the members of the subcommittee in unanimous unison said: Before considering the adoption o f any o f the myriad proposals for improvement o f the acquisition workforce, the Committee on the Armed Services believed it crucial to conduct an in-depth analysis o f the qualifications and professionalism o f acquisition personnel as well as a review o f various DoD efforts o f the department to establish and manage the career development o f the acquisition workforce . It is hoped that through this effort a better assessment o f cause and effect can be made - providing greater assurance that changes adopted will in fact bring about the desired result. The questions this important investigation attempted to answer were: (1) Are the Services appointing Program Managers, Deputy Program Managers, and Contracting Officers with the experience, education and training required by law and regulation; and are the 55 Program Managers being retained in their positions the mandatory four-year minimum? (2) Is there a career program structure to develop qualified and professional contracting and program management personnel - both civilian and military? (3) Is there an appropriate mix o f military and civilian personnel within the workforce? (4) What impediments exit that must be overcome in order to develop a quality, professional workforce, and how can that be accomplished? The report defined the distinctions between the military and civilian personnel management in the acquisition workforce. These distinctions are of significance because it forms an element in the cultural base Congress has declared must be changed. The report asserts: To understand the role o f the Program M anager and contracting officer, and to evaluate the challenges confronting the Department in the development and management o f a professional acquisition workforce, requires a through understanding o f the personnel management organizations, policies, and systems which collectively impact the development and operation o f this workforce, as well as the underlying distinctions between military and civilian personnel. M ILITARY AND CIVILIAN PERSONNEL M ANAGEM ENT DISTINCTIONS Military officers exist to organize, equip, train, and direct military forces. They first and foremost are members o f the military officer corps charged with carrying out their service's military mission. Civil servants, on the other hand, perform specific technical functions for the government similar to those found in the private sector. Civilians hold no personnel rank or distinction, since their authority resides only in the position they occupy . . . As a result o f these basic differences, the personnel management systems differ. While both systems have the same career phases - recruitment or accession, training, career management, and 56 retirement or separation, there are pervasive differences throughout each phase. Recruitment for the military is done centrally, for civilians locally. Continuous training and professional development are ingrained in the military system while civilian system these are often difficult to obtain. Career management is centralized in the military, whereas civilian hiring is typically done at the local level and civilians are almost solely reliant on individual initiative in managing their careers. Retirement for the military is based on the "up or out" policy . . . Civil service retirement is based on the precept o f inability to continue to perform the job satisfactorily. The report goes on to cite the specific differences between the management of military and civilian personnel as conducted by the separate Services (Army, Navy, Air Force) and the Office of the Secretary of Defense. These differences between Services also define what may be tenned as "acquisition sub-cultures" within each Service's workforce. As an example, a civilian whose career and experience has been gained in the acquisition of nuclear submarines is neither technically nor culturally the same as a civilian whose career and experience has been gained in the acquisition of land-based intercontinental ballistic missiles. The total numbers of civilians and military officers broken down by Service in the acquisition workforce is contained in the following graph: 57 CIVILIANS & OFFICERS ACQUISITION WORKFORCE 80.000 NAVY g 60,000 1 ARMY 20.000 0 --------------- CIVILIANS OFFICERS TOTAL Workforce Army Navy Air Force DLA Officers 2,042 (4%) 3,441 (4%) 8,812(18% ) 620 (2%) Civilians 49,007 (96%) 82,574 (96%) 40,143 (82%) 26,429 (98%) Total 51,049 86,015 48,955 27,049 Chart 1 58 Viewing this data from the perspective of information that has already been presented defines a most interesting dichotomy. The report states: The similarities and differences between military and civilian personnel are rooted in the unique historical developments o f each. Military Officer Corps The officer corps is fundamentally different in its origins, purpose, and organization than the civil service. This difference is best explained by Samuel P. Huntington in The Soldier and the State: The officer corps is both a bureaucratic profession and a bureaucratic organization. Within the profession, levels o f com petence are distinguished by a hierarchy o f ranks; within organizations, duties are distinguished by a hierarchy o f office. . . . the professional character o f the officer corps rests upon the priority o f rank over the hierarchy o f the office. Officers develop a special military competence, which distinguishes them from civilians. This special competence is what Harold Lasswell called, "the management o f violence." Officers require a high order of expertise and must be prepared and trained physically, intellectually, and psychologically to fight (added emphasis). This later statement by Lasswell becomes significant when Congress states that the purpose of its legislation is to "change the culture". To change the culture of the officer corps through education and training, may be counter productive to the training and education, including psychological conditioning, they have already had to "manage violence" and to fight. The 59 report turns to discuss the civil servant's most essential characteristic, the concept of merit: Civil Service A merit system is a fair and orderly process for hiring, paying, developing, promoting, retaining, disciplining, and retiring people on the basis o f ability and performance . . . a merit system is based on merit principles; it is designed to produce a competent, stable workforce to carry on the business o f government. The personnel distinction between the two, officers and civil servants is defined by the report as: Officers, first and foremost, exist to organize, equip and train military forces, plan their activities, and direct operations in and out of combat. As a result military officers are members o f the officer corps first and, secondarily, technical experts . . . However, within the Department o f Defense civil servants are charged with carrying out non-combatant missions . . . Secondly, military rank resides in the individual, whereas for civil servants, the grade resides in the position they occupy (added emphasis). Another distinction is accession. For military officers it is centralized in terms of identifying types and numbers needed. In civil service this function is decentralized and done locally. This point has significance when the requirement of the "Acquisition Corps" are considered. If more of one type of acquisition functional specialists is needed than another, it might follow that the "through-put" becomes the controlling factor. Under a "through-put" concept, training would tend to dominate over education. 60 Again, there is a significant difference between military officers and civil servants. An implicit concept of training is that "it will provide individuals the basic technical and managerial skills to perform at an acceptable level of competence through out a period of service." The concept o f a military career for officers is long-established. This entails professional development, in part, through completion o f Professional Military Education (P M E ). . . Both the management o f PME and graduate education tend to be centralized with technical training less so. There is no comparable system o f professional education for civilians nor is there a recognized emphasis on graduate programs in the individual's technical speciality. Career Management, according to the report, "involves the full utilization of personnel in current positions and their career development or enhancement for progression to more demanding and higher graded positions." The distinction between officers and civilians is the difference between "highly centralized utilization, assignment, and promotion for officers while civilians must "remain solely reliant on individual initiative in managing their careers." Assignments for officers vary from a few months to not more than four (4) years in duration, whereas civilians know that they are in their positions until they are promoted, transferred, reassigned or separated (resignation, retirement or removal). The impact of these different processes for military officers and civil servants leads to different motivational and career approaches. The report states, "It is not an exaggeration to say that the two comprise different cultures" (added emphasis). Military officers view the military as a career requiring geographic and organizational mobility. Their general education is not directed toward a specific technical speciality, nor is their advancement dependent on the development of technical expertise. The civil servant is hired based upon a presumption of his/her qualification to perform the job. Here too, career advancement is not dependent upon specific additional education. In summary this section of the report concludes: These differences in orientation are evident in analyzing the characteristics used to judge the quality and professionalism o f the acquisition workforce, and must be taken into account in any effort to improve the quality and professionalism o f the workforce - be it military or civilian. Program Management Personnel Role O f all the functional disciplines in the acquisition career field, the position of Program Manager is the highest, a hierarchial elite post that is referred to as a "super-functional" discipline. Most, if not all, Program 62 Managers have a technical background with operational or combat-related experience. The role of the Program Manager and Deputy are: . . . the single executive focal point responsible for the successful management o f the program and the accomplishment o f the cost, schedule, and technical performance objectives established for the successful execution o f the program. The Program Manager, operating through the staff of the Program Office, has broad discretionary authority over planning and resources. In some cases, billions are at stake, based upon the decision of the Program Manager, such as in the case of the Nav>’ A -12 and Air Force C-17 programs. The five (5) basic functions of the Program Office as observed by Dr. J. Ronald Fox in Arming America: How the U.S. Buys Weapons (1974). Program Control Program Management. This function directs the overall system program planning, programming, cost and schedule, data collection, financial management and preparation o f reports. Configuration Management. It establishes and implements policies and procedures for managing system, subsystem, and item configuration to include specification and engineering change control. Procurement and Production. This activity is responsible for managing the contracting, development and production efforts. Engineering. This function manages the total system engineering effort, including the integration o f engineering systems and subsystems and the system's technical performance. Product Assurance or Test and Deployment. This office plans and coordinates the test program for the weapon system. 63 Attributes As stated in Chapter I, the military cultural imperative is, "only military combat command personnel, 'a user', has the experience and knowledge to define and manage the development and acquisition of a weapon system and to accomplish its fielding." Operational experience allows the Program Manager to communicate with "users" and logisticians supporting "users" in terms of their common understanding and cultural base. However, since the Program Manager leads and integrates a team of functional-disciplined experts, most would agree that technical knowledge or knowledge of basic business procedures is a prerequisite. Since the integration of functional teams into a cohesive Program Office precedes success at any stage of program maturity, the HASC's Investigations Subcommittee placed a, "premium on good management skills." With billions at stake, the ideal Program Manager's attributes profile depends on the program's stage of maturity within the acquisition life cycle. If the stage of maturity is research and development the demand would be for a technically oriented program manager. If the stage of maturity is production and delivery, a business oriented program manager would be more appropriate. 64 Qualifications Various Presidential Commissions, DoD Secretaries, Services, and Congress have expressed their "expert" opinions on the qualifications of Program Managers and the establishment of a qualifying career field for both officers and civil servants. The Second Hoover Commission (1955) recommended that the Secretary o f Defense establish a personnel system for support activities which provides comparable standards for selection, training, promotions, and compensation for both civilian and military managers and technical personnel. In 1963, Secretary o f Defense Robert McNamara set forth the basics for managing weapon system programs in a DoD Conference on Program Management. McNamara called for upgrading the status, authority and quality o f Program Managers, concluding that Program Managers "hold key positions in our Military Department. Such positions demand the best managerial talents, on which the Department o f Defense places full reliance for its future weapon inventories." The Fitzhugh Commission in 1970 recommended that the effectiveness of Program Management could be improved by establishing a career speciality code for Program Managers in each Military Department and by developing selection and training criteria that "will insure the availability o f an adequate number o f qualified officers. The criteria should emphasize achieving a balance between knov/ledge o f operational requirements and experience in management." Deputy Secretary o f Defense Packard established the Defense Systems Management School (now the Defense Systems Management College) on July 1, 1971 to provide a professional education in Program Management and defense acquisition management. 65 The Acquisition Advisory Group, established by Deputy Secretary o f Defense Clements in April 1975, noted the failings in the "Military Departments and program offices in the detailed execution o f weapon systems acquisition programs." In 1979, Donald B. Rice emphasized the necessity for the personnel systems to develop Program Managers. "The existing incentives for effective acquisition management at the program office level are among the weakest elements in an otherwise adequately structured system, and should receive priority attention. It is recommended that DoD undertake to design, test, refine, and install: A specialized Program Manager selection process (perhaps as a part o f the management training program) A special performance evaluation system (built around criteria relevant to system s acquisition) for Program M anagers that will recognize and reward effective leadership o f acquisition program and, equally important, will clearly identify less-than-adequate performance. After extensive Congressional hearings highlighted a lack of progress in DoD, Congress in November 1985 adopted the Defense Procurement Improvement Act (P.L. 99-145), requiring that the Secretary o f each Military Department "prescribe regulations establishing requirements for the education, training, and experience o f any person assigned to duty as the Program Manager o f a major defense acquisition program." It further established the following minimum experience and training criteria: the completion o f the Program Management Course (PMC) at the Defense Systems Management College (DSMC), or a comparable Program Management Course at another institution; and, a minimum o f eight years' experience in acquisition, support, and maintenance o f weapon systems, two years o f which must be in a procurement command, allowing credit for time spent pursuing a postgraduate degree in a technical or management field or attending the Program Management Course. These statutory requirements, which became effective on July 1, 1989, were extended by DoD Directive 5000.23 to include education requirements as well as the necessity for Professional 66 Military Education (PME). A baccalaureate or advanced degree in a technical, scientific, or managerial field was made mandatory with the proviso that "advanced technical education o f a long-term nature in service schools" could be substituted, and a master's degree was desirable. These findings of the inadequacy of Program Managers by a number of dissimilar agencies over the course of thirty (30) years indicate a definite lack of understanding by DoD personnel. Military officers do not report for duty at the Pentagon each day with an agenda of avoiding task fulfillment. It would further appear that what DoD considers task fulfillment, is not viewed that way by others outside DoD's cultural grouping. It is this non-compliant perceptive view held by Congress in both the HASC and SASC which precipitated the passage of the DAWIA to "change the culture of DoD acquisition by education and training." Authority The question of a Program Manager's authority developed over a number of years, as the acquisition of war materials to "fight the cold war" politicized guidance from Congress and the Pentagon. By 1965, DoD Directive 5010.14 required that a Program Manager and staff, "should have sufficient rank/grade and organizational stature to meet the needs of functional parity." 67 In 1969, the Defense Science Board Task Force Final Report on Systems Acquisition concluded that a "major increase in the recognition, the status, and the opportunities in Program Management mat be necessary to attract and retain a larger share o f the most capable career officers." In 1970 the Fitzhugh Commission found Program Management to be a key weakness in the defense acquisition system. Packard's initiation of the "5000 series" instructions established a base that gave Program Managers a clear written charter and established their authority. However, in the 1986 Packard Commission report, the need to re establish clear lines of authority required the following note: Authority for executing acquisition programs . . . and accountability for their results - has become vastly diluted . . . it is fundamental that we establish unambiguous authority for overall acquisition policy, clear accountability for acquisition execution, and plain lines o f command for those with Program Management responsibility. Nunn made specific note of the Pentagon's failure to implement this Packard Commission's recommendation, recalling David Packard's testimony that DoD had "flunked out" by failing to streamline the acquisition chain-of- command. Tenure O f all the culturally defined issues associated with Program Management, the issue most critical to the actual conduct of the program and 68 its relative success or failure as measured by the program's cost, technical achievement, and schedule is the issue of tenure. According to an October 1979 Rand Corporation study frequent changes in Program Managers can lead to "unnecessary shifts in program emphasis . . . [and] loss o f direction while the newly assigned Program Managers settle in and learned their jobs." {Acquisition Policy Effectiveness: Department o f Defense Experience in the 1970s). A short tenure tends to focus emphasis on the near-term and does not lead to a long-term commitment to the program. Program results are sub-optimized because immediate decisions often have a significant impact in the long-term, but the individual making the decision will no longer be accountable.. . . . The Rand study noted that even experienced Program Managers transferred to new programs must then learn the background o f the new program, master a great deal o f technical information, and establish a network o f functional and organizational contacts before becoming fully effective. This issue is long-standing and was first observed in the Second Hoover Commission 1955 Report. In 1981, Deputy Secretary of Defense for Acquisition Carlucci delineated a tenure requirement as part of his "32 Carlucci Initiatives." The Defense Procurement Reform Act of 1984 (P.L. 98-525) required: "the tour of duty of an officer of the anned forces as a Program Manager of a major defense acquisition program shall be (A) not less than four years, or (B) until completion of a major program milestone." Professionalism of the Acquisition Workforce All of the Congressional hearings and reports, the President's Blue Ribbon Commissions, DoD internal studies, and interested authors focus on 69 the improvement of the "professionalism of the acquisition workforce." The term means different things to different people. The HASC report, "explores the elements of professionalism and discusses what is or may be required to establish the acquisition career field as a profession." The Concept of Professionalism Definition of Terms Any consideration o f value-laden terms such as professionalism require careful and precise delineation o f their definition and usage. Thus, this section describes the terms profession, professional, professionalism, and professionalization and describes their interrelationship and characteristics. . . Profession A profession is commonly defined as a body o f qualified persons o f one specific occupation or field . . . Those who profess it are obligated to put integrity above personal gain in the skills that they exercise . . . The offer is made by the profession in terms o f its unique skills and inherent responsibility to convey something o f value to society. Professional A professional is engaged in activities suitable o f a profession, normally requiring great skill or experience in a particular field or discipline . . . Experience is acquired only after prolonged education and training. And throughout their careers professionals must keep abreast o f the changes in their occupation via specialized training and participation in activities o f professional associations. Professionalism Professionalism is derived from the term profession - an occupation in which one professes to be skilled. Professionalization The term professionalization is defined as the "acquisition o f professional characteristics by the occupation, without an increase in skill." The group involved thus crosses the threshold from an occupation to a profession. 70 Characteristics of a Profession A profession has seven qualifying characteristics: 1. a specialized body o f knowledge 2. requirements for education and training 3. professional organizations or societies 4. certification or licensing 5. a code o f ethics 6. social unity 7. education Body of Knowledge A profession has an established body o f knowledge, developed and maintained in academia. This body o f knowledge is inextricably tied to an academic curriculum . . . The body o f knowledge is attained through scholastic advancement through this curriculum. Inherently, this body o f knowledge is dynamic and expanding. Thus the evolution o f this knowledge is explicitly tied to on-going research. Education and Training Because the body o f knowledge in the profession is dynamic, the members o f the profession are ever-learning. This on-going education, training, and self-development should continue after entry into the profession . . . Professional Organizations Associations are the guidons o f the profession. They seek to further the interests o f the community . . . Certification or Licensing Professional associations can be perceived as a quality control mechanism o f the profession, ensuring quality education and membership competence. Certification encourages individual development, establishes standards o f achievement, measures knowledge, and recognizes the professional competence o f personnel. Code of Ethics Codes o f ethics and standards o f conduct are based on a concern and responsibility . . . The professional is in a position o f trust, which is shared by all members o f the particular discipline or community. 71 Social Unity The last evolutionary step or phase required to move from occupation to profession depends on the social utility provided by the practioners o f the profession. There has to be a desire on the part o f the society to see the need for the profession. Education Education is a prerequisite for a professional workforce. In modern post-industrial society, education is the lubricant for the machine of progress. For many years, outside experts and commissions have decried the absence o f educational requirements, including a college degree for federal civilian procurement workforce. This concept of a profession is derived from such historic professions as the clergy, doctors and lawyers. Military personnel acknowledge themselves as being members of the "oldest profession on earth." What Congress seems to be conveying with these definitions is an intent that the acquisition workforce must develop its own "subculture" within the military’ s professional institution in order to demonstrate its own "professionalism." However, as the HASC report states, "The measure of "professionalism" is difficult because professionalism embodies attitudes, values, and motivations which are difficult concepts to grasp and measure." It would appear the HASC use of "professionalism" is a surrogate for another term used by Mavroules in his press releases, floor speeches, and outside appearances. The word he used is "culture." In his outspoken 72 remarks he expresses the opinion that it is the "culture" of the defense acquisition workforce that requires change. 73 Remarks by MAVROULES (D-MA) on H.R. 5211: CREATING AN ACQUISITION WORK FORCE, June 28, 1990 Press Release: Key Points of Acquisition Workforce Bill Nicholas Mavroules: Floor Debate - Acquisition Workforce Nicholas Mavroules: 1991 DoD Procurement Conference - Williamsburg, VA These four documents originating from Congressman Mavroules are grouped for the purpose of presenting a single review. In essence, they all reflect the same information, each slightly reformatted to suit the intended audience or situation. As the Chairman of the Investigative Subcommittee of the House Armed Services Committee (HASC), Mavroules sought to eliminate the Pentagon procurement scandals of the 1980s by the enactment of a major piece of historic legislation. In his Williamsburg speech, Mavroules said: In the past, reform initiatives focused on changing either the process o f acquisition, such as the rules and procedures, or the structure o f acquisition, that is, the offices and agencies that do the buying. These are the first tw o legs o f the stool in the procurement process. These are obviously critical elements and may need additional focus in the future. But they failed to take into account the most vital and critical area: the quality and capabilities o f the people who must work within the structure. People, I believe, are the most critical third leg o f the stool, the one that makes it stand . . To accomplish this, we mandated the creation o f a professional acquisition workforce and corps within each o f the military services and the defense agencies . . . The Defense Acquisition 74 W orkforce Improvement Act (DAWIA) — was made part o f the House passed fiscal year 1991 National Defense Authorization Act, H R. 4739, P L. 101-510, Title XII. (added emphasis) Speaking for the Investigation Subcommittee of the HASC, Mavroules inserted his remarks in the Congressional Record - Extensions - Part II defining the reasons and purpose of the act. So, instead we will try to grapple with the root causes . . . Those o f us who have been working acquisition issues recognize that the many approaches (process and structure, sic) o f the 1980s failed to achieve the cultural changes that are essential for making acquisition reform work. I recognize that bringing about a cultural change can be a long, arduous task. What is needed — and I believe the time is now ripe — is a candid and open discussion about root causes — to wit, what is wrong, what should be fixed, and how can we go about doing that? So we will concentrate on their skills, their experience, their education, and their training. Mavroules methodology to accomplish this task of changing the culture through education was the establishment of the Defense Acquisition University. W e will create a Defense Acquisition University to be the center for all acquisition education, training and research, a focal point for acquisition excellence that will oversee instruction in the whys and wherefores o f acquisition as well as promote research leading to improvements in acquisition management. To help get at the training problem, we would establish a Defense Acquisition University that would be responsible for all acquisition 75 courses required for acquisition personnel. The University would provide centralized direction, operation, control, and accountability o f the Defense Department's education and training program for all acquisition personnel. Depending on how it is shaped by the Secretary o f Defense, the University could be the intellectual centerpiece o f the entire acquisition system — a place where seminars and thought- provoking meetings — intellectual ferment — can help to change the mindset, to bring about a lasting cultural change, throughout the acquisition system (added emphasis). Education — one key element o f professionalism — is o f crucial importance in developing a quality acquisition workforce. According to Mavroules, some of the cultural changes DAWIA was designed by the Investigative Subcommittee of the HASC to bring about were: Inadequate training o f Program Management and Procurement personnel has frequently been the cause o f costly acquisition deficiencies. The course (PMC, sic) designed to training management officials graduates five times as many people as will ever fill Program Manager slots. Yet, all three Services have failed to fill Program Manager positions with graduates o f this course, even though it is required by law. For example, less than a third o f those assigned as Program Managers o f major Navy systems have ever attended the course. While the mix o f civilians and military is varied among the Services, no Service has been in compliance with longstanding policies to appoint civilians to positions not requiring military officers. Compared to its industry counterparts, this workforce is undertrained, underpaid, and inexperienced. Whatever other changes may be made, it is vitally important to enhance the quality o f defense acquisition workforce — both attracting qualified new 76 personnel and by improving the training and motivation o f current personnel. At the senior levels o f the acquisition workforce, we would create an elite acquisition corps, comprised o f both military and civilian personnel — the senior 10 percent to 15 percent o f the workforce — those individuals who by virtue o f their expertise have achieved a certain rank or grade and who should be recognized as professionals in their own field. Today, a significant number (o f acquisition personnel, sic) remain untrained. How many? No one knows because currently the services are incapable o f tracking the training records o f their personnel. We have put teeth into the requirement that Program Mangers serve longer. We would require them to serve until a major milestone AND at least 3 years. The military has long been biased against civilian personnel because civilians are perceived as less receptive to change. In this respect, we can counter the bias against civilians by encouraging a policy o f mobility for civilians. The Undersecretary o f Defense for Acquisition, under the direction o f the Secretary o f Defense, must ensure that comprehensive career programs are established for the acquisition workforce — both civilian and military — within the Army, Navy, Marine Corps, Air Force, Defense Logistics Agency, and other defense agencies. These career programs should include the elements o f accession, education, training, experience, assignment, promotion and retention. In the Army, it is said that Division Commanders have the authority to make it rain. Program Managers also need the power to call up a rain shower now and then (added emphasis). 77 LIFE IS TOO SHORT: A Review of the Brief Periods Managers of Major Defense Acquisition Programs Stay on the Job; Report of the Investigations Subcommittee of the Committee on Armed Services, House of Representatives: 1990 The history of the tenure issue is like an open festering wound that never heals and is always there. It is a hard-rock chasm of un-spanable width and indeterminate depth between Congress and the Pentagon. It is the difference between the culture of the traditions and customs of the Services and the advice and consent of Congress. The HASC's Investigations Subcommittee reached findings that contain some rather harsh and damning words: The Subcommittee found that, for all practical purposes, the Services have simply flaunted the law - in only 11 percent o f all cases are the services in even technical compliance. Furthermore, instead o f improving, the average tenure o f Program Managers actually dropped after passage o f the law mandating longer tours. . . The Subcommittee finds this is a most egregious example o f the flouting o f statutory requirements. Regardless o f how the numbers are parsed, the statute has simply been flouted by all the Services. The waiver provision, however, was intended for exceptional circumstances. It was not intended for daily use . . . It is intended that the PMs become the men and women who can make it rain. But it is very hard to make it rain when Program Managers are raced through their assignments at breakneck speed. The primary achievement o f a Program Manager ought not be learning the location o f the executive washroom. 78 Subcommittee finds only one benefit from this: it supports the military m indset. . . But this personnel system should emphatically not be pursued to the detriment o f major equipment programs. The HASC Investigations Subcommittee concluded its report by promising, "The Subcommittee expects shortly to markup legislation that will address this issue - again. However, it intends not only to legislate a tenure policy. It also wishes to put the Services on notice that it will monitor compliance closely and expects a dramatic change in the sorry record since enactment of the 1984 law." Congress's attack is based upon what is perceived as non-compliance to recommendations first attributed to Robert Schaifer's research on aircraft engine development in the early part of the century. Presidential Commissions starting with the Second Hoover, Pentagon internal reports from Deputy Secretary of Defense Cyrus Vance, Association reports from the National Security Industrial Association, and research organizations reports from Rand Corporation were of a common philosophy that "frequent changes in Program Managers can lead to unnecessary shifts in program emphasis . . . [and] loss of direction while the newly assigned Program Manager settled in and learned their jobs." 79 Congress holds the view, "that the primary task of the PM is to run the programs so that the military services get the working equipment at a reasonable price within a reasonable amount of time." Based on the data researched by the Subcommittee, it would appear that the lack of tenure is the result of three Service practices. First, Program Managers were almost entirely military personnel. Civilians were rarely named to these key positions, although they frequently occupied the Number Two slot o f the Deputy Program Manager. Second, The officers chosen for these major positions were, quite reasonably, identified as "comers" who were rated as general officer material. In keeping with standard military philosophy, they were cycled in and out o f a variety o f positions in order to give them the kind o f broad and rounded experience that the military believes a general officer ought to have - the practice commonly referred to as "ticket-punching." Third, the Program Manager slots were reserved for a specific rank. When the incumbent won promotion, a position rated as appropriate for the new rank was found for him or her, again, in keeping with standard personnel procedures in the Service. In addition to this very generalized snap-shot of Service culture, other popular abuses were found to be Service dependent. The Army's chief abuse was the use of acting or temporary PMs. The Navy's abuse was the simple reassignment of officers as a matter of routine, ignoring the law outright. The Air Force abuse was the use of the Program Managers' slots as short stop overs prior to retirement. 80 The Subcommittee acknowledged that longer tours would not solve all the attendant problems of acquisition, but with billions of dollars involved it was their belief that: . . . a short tenure tends to focus the Program Manager's attention on near-term issues to the detriment o f long term concerns. Program results are thus sub-optimized because immediate decisions frequently have a significant impact on program's outcome in the long term. But the individual making the decision today will not be around to be held accountable. It should be noted that trade offs decisions commonly involved playing a short-term issue off against a long-term cost or benefit. . . But is difficult to find anyone who has studied the system who does not believe longer tenure - with concomitant increased accountability not to mention increased time spent actually doing rather than learning the job - can only help improve the management o f multi billion dollar systems. The passage of the DAWIA was in-part attributed to this report. How bad was the non-compliance of the Services? The Subcommittee's statistical chart presented this picture on the compliance with the four-year tenure requirement: RATIO NAL Number Percent Com pletion o f 48 or m ore months 5 5 Achiev ement o f ma jor m ilestone 1 1 W aiver by Service Secretary 4 4 N on-com pliance with statute 84 89 Total 94 100 Table 2 81 When Mavroules spoke of changing the culture of the acquisition workforce, tenure was one of the targets illuminated for change. 82 National Defense Authorization Act for Fiscal Year 1991: Report of the Committee on Armed Services, House of Representatives on H.R. 4739 National Defense Authorization Act for Fiscal Year 1991: Conference Report to accompany H.R. 4739 10 U.S. Code Chapter 87 — Defense Acquisition Workforce as of March 25, 1994 These two pieces of legislation and the updated and amended law are reviewed as one. The passage of the Defense Acquisition Workforce Improvement Act (DAWIA) created an Acquisition Corps within each Service for military and civilian personnel. The passage was based upon the differences between the legislative intent and concepts of Congress versus the implementation theory of the Pentagon. As Mavroules stated, in words to the effect, "The time was ripe to pass laws creating the Acquisition Corps within each Service that will in time hopefully change the culture." It is extremely doubtful if an act of this nature would have successfully withstood floor debate prior to the demise of the Soviet Union and the end of the Cold War. Without a massive defined threat, the Pentagon could not marshall public opinion to oppose the change. After years of research, hearings and voluminous reports, the introduction of a bill to essentially reorganize the organizational structure for the acquisition of war materials became a reality. 83 However, in the passage of the bill, the harshness of the HASC's Investigations Subcommittee's report language and the chief sponsor remarks in the Congressional Record were not carried forward in the textual language of either the HASC Authorization Act Report or the HASC / SASC Conference Report. Report language is a technique used by Congress to "get-a-message-across" and "gain compliance" without passage of a law. The historical record, documented in the SASC and HASC hearings and reports, indicates the lack of common understanding between Congress and the Pentagon culture, referred to as the "military mind," on the management of the acquisition of weapon systems . This "continuousness" goes back to the turn of the Century. Neither Congress nor the Pentagon would expect to see a cultural change in under 10 to 12 years. Congress has promised to keep its "micromanagement" eye on the Pentagon's progress, and its mandated reports allow it to do so. Certainly, if progress is not in the direction required or timely in implementation, Congress entertains remedial and corrective supplementary legislation. However, attempting to structure a corrective remedy, Congress may have introduced some flaws. 84 By the creation of the Defense Acquisition University (DAU), Congress interposed another level of administrative controls and oversight in the training of the elite Program Manager's functional discipline. However, DAU does not have sole command and control. This is shared with the services, the Acquisition Career Program Boards. P 1706. Acquisition Career Program Boards (a) Establishment. - The Secretary o f each military department, acting through the service acquisition executive, shall establish an acquisition career program board to advise the service acquisition executive in managing the accession, training, education, and career development o f the military and civilian personnel in the acquisition workforce and in selecting individuals for an Acquisition Corps. Under these provisions, the Career Boards in all the functional disciplines have significant control over the curricula of Level I, II or III courses. The purpose of the education and training of the Acquisition Corps is to create a more integrated acquisition workforce, rather than the "stove- piping" of dominant functional career disciplines. All Program Offices have been directed by DoD to be more team structured and mission oriented. However, without what is referred to as "unity of command," conflict may exist between the Career and Functional Board, DAU, and DSMC, in each attempt to determine the PMC course curricula. This part of the law may prove to be unworkable and may require revision. 85 The emotional tenure question of Program Manager and Deputy Program Manager may, however, have been resolved: (c) M ajor Milestone Regulations. — (1) The Secretary o f Defense shall issue regulations defining what constitutes major milestones for purposes o f this section. The service acquisition executive o f each military department shall establish major milestones at the beginning o f a major defense acquisition program consistent with such regulations and shall use such milestones to determine the assignment period for program managers and deputy program managers Acquisition Programs have a generic description of major program milestones, but each program's milestones are separately defined. It is not uncommon dining the course of a long term program (5 to 17 years) to question and change what constitutes completion of the milestone or "exit criteria." The Service Acquisition Executive has the primary influence and determination over a program's transition from one milestone to the next. The expressed intent of Congress is to retain a Program Manager until a particular milestone is complete and to use the state or condition of the program at the completion of that milestone as the basis for Program Manager evaluations and selections. 86 A H istory o f the D efense System s M anagem ent College: David P. Acker, 1986 Adverse, half-truth stories editorialized by the media bring unsubstantiated questions of personal ethics to the American public and causes Congress in a reactive mode to demand changes. Packard witnessed all of the problems associated with war material acquisition during W.W. II. When he became the Deputy Secretary of Defense in former President Nixon's first administration, he attacked the problem by establishing what was to become known as the Defense Systems Management College (DSMC). The history of DSMC is narrated in A History o f The Defense Systems Management College, written by the late David D. Acker, for whom the 'Acker Library' at the DSMC is named. Acker's recorded history of DSMC parallels many military organizations. As each new Commandant assumed command, the immediate mission was to implement a personal agenda by placing an individual mark on the organization. In every case from the first Commandant to the last, there was an attempt to change the academic system to make it more "realistic" by matching the current requirements of the "real world at the Pentagon." 87 MILESTONE PRE CONCEPT DEMONSTRATION ENGINEERING & PRODUCTION & OPERATIONS & MILESTONE EXPLORATION & & VALIDATION MANUFACTURING DEPLOYMENT SUPPORT 0 DEFINITION DEVELOPMENT * M ission * Evaluate f easibility o f * D esign System (s) * Mature and * Support Fielded System Area Assessm ent Alternative Concepts Finalize Selected * Demonstrate D esign * Monitor System * Determine warfighting * Determ ine Most Critical Processes Performance deficiencies/ Promising and Technologies * Validate technological Concept(s)/Solution(s) (Early Prototypes) Manufacturing & * Identify Improvement opportunities Production Processes Opportunities * A ssess nonmaterial * M odify/Upgrade as alternatives-, if material * Test & Evaluate Required solution required, System prepare M ission N eed Statement M S 0 MS M S 1 1 M S III M S IV Concept Studies Concept Demonstration Developm ent Production Major M odification Approval Approval Approval Approval Approval (as Required) A A A A A DISPO SAL * Approval to * Approval to I inter * Approval to Enter * Approve to Enter * Approval to M odify System Enter Phase 0 Phase I Phase II Phase III Still in Production * Exit Criteria * Program Initiation * LR1P Quantities * Production Baseline for Phase 0 * Concept Baselin Approved Approved Approved Approved * Developm ent B aseline * Exit Criteria for Approved Phase 1 Approved * Exit Criteria for Phase II Approved Chart 2 oo 00 The teaching vehicle of choice is the historical case problem designed around the current administrative theories of Pentagon practice in the development of a weapon system concept (Pre Milestone 0) through all the succeeding milestones (Concept, Exploration & Definition - Milestone 0; Demonstration & Validation - Milestone I; Engineering & Manufacturing Development - Milestone II; Production & Deployment - Milestone III; and Operations & Support - Milestone IV - Chart 2). This process for the actual fielding of a modem day weapon system has a real life time frame of five (5) to eighteen (18) years. Because the administrative structure of the Pentagon is driven by Congress, without a stable base of laws for war materials acquisition, it follows that many lessons learned in the classrooms of DSMC may be invalidated by events and time after students graduate and are assigned to acquisition programs. O f even greater importance in Acker's history is the following theory on "Experiential Learning" followed at DSMC as shown on Chart 3. 89 PMC STUDENT PERSONAL CHARACTERISTICS FACTS PERCEPTIONS IDEAS HABITS PERSONALITY INTELLIGENCE MOTIVATION BLOCKS TO | LEARNING jPROCESS EFFECTIVE FUNCTIONING "BLACK AND WHITE" NORMATIVE VIEW OF THE WORLD DESISION MAKING: AVOIDANCE BY RELIANCE ON AUTHORITY FOR DECISION OTHER: DUE TO COMBINATIONS OF PERSONAL CHARACTERISTICS ENTRY TO SYSTEM X EXPERIENTAL LEARNING PRACTICE APPLY EXPERIENCE 1 \ PROBLEMS \ AND ') ENVOLVEMENT / ; / OBJECTIVES 1 / I'' BROADEN PERSPECTIVES DRIVE STUDENTS TO BE HOLISTIC ! DEMONSTRATE PM COMPLEXITY CHALLENGE STUDENT TO USE REFLECTIVE THINKING MAKE SENSIBLE DECISIONS ACCEPT RESPONSIBILITY EXERCISE LEADERSHIP GOAL OPEN-MINDEDNESS OUTPUT from understanding of SYSTEM CONCEPTS X SOUND DECISION ; MAKING BETTER LEADERSHIP In examining this linear diagram, a number of errors of omission and commission are noted. "Facts" are not a part of a person's personal characteristics. The "Blocks to Effective Functioning" may be more properly labeled "Blocks to Effective Learning." It is noted the three items defined are not all inclusive or definitive in nature. Individual student assessment or feedback blocks utilizing such instruments as MBTI, Perry LEP, or PROFILOR are not present. The need for change and the type of change has not been properly established or proven. The accomplishment of a classroom case problem, no matter how rigorous, will not change individual behavior. An examination of the outcome goals is indeed a worthy consideration, but it is far removed from the behavior change reality presented by Lewin's theory of behavior change (unfreezing - change - refreezing), and it does not include or compensate for the students' return to an unchanged environment with no form of behavior change re-enforcement. Acker's recorded history of DSMC is devoid of any mention of a rigorous research program that would redesign the curricula by looking to the future, based upon the experiences of the past, in order to define better ways to accomplish the task of war material acquisition. The academic disciplines of the various departmental functions appear to be reactive to exigencies, 91 rather than proactive and seeking ways to improve either the system, process or individual behavior. 92 The D efense M anagem ent Challenge: Weapons Acquisition: Dr. J. Ronald Fox And Mr. James L. Field (1988) In his 1988 book, co-authored with Professor James L. Field, Fox discussed the structure and operation of the Defense Systems Management College (DSMC). A large part of the initial curriculum at DSMC was a result of Fox’ s efforts. He was present when Packard established the institution and gave it three missions: 1. to prepare selected military officers and civilian personnel for assignment in program management career fields 2. to conduct research and special studies in program management and defense systems acquisition management 3. to assemble and disseminate information relative to program management and defense acquisition management Fox posits that DSMC was never intended to be an academic institution, but rather," ... its function was intended to offer practical courses to help students acquire knowledge and skills in the operation of defense program management and procurem ent. .." As a Professor of Business Administration at Harvard, oriented to the aspects of corporate business management, cash flow, contracts, facility infrastructure and amortization schedules, it is not surprising that Fox would be more oriented toward the "hard science of corporate management." As Chairman of the original curriculum committee, Fox and his committee saw, 93 A danger in the term college, leading the school to evolve into an academic organization in which the faculty would devote the short time available for classroom sessions to more theoretical topics such as theories o f motivation, personal assessment, typology, and interpersonal behavior. By the 1980s courses on each o f these topics had been suggested by the faculty - and even offered at times. Although such courses can have intrinsic value, they are necessarily o f low priority in a practical program manager course condensed into twenty weeks. In the prior discussion, Fox's testimony before the SASC defined the significant need for change. DSMC was established in 1971 and its students were taught under curricula designed by Fox's committee. Yet in 1988, some seventeen (17) years later, the same problem still exists. To improve or change requires a change in behavior. Changes in behavior cause a change in culture. Planned changes in behavior are unlikely if the students are never counseled as to problems and alternative behaviors. The first step to change behavior according to Lewin is a significant emotional event or an unfreezing. Krathowohl et al. posits that change occurs via the affective domain bringing about cognitive understanding precipitating or causing behavioral modification. In either event, individuals in self-select groups cannot change unless they are presented with the information and knowledge that their overall behavior is aberrant to the stated objective - in this case better acquisition management. In the 17-year period that DSMC graduates have 94 influenced the acquisition process, a cultural change could have occurred if there was an understanding of their errant behavior. According to the former DoD Acquisition Secretary, John Betti, The Total Quality Management theory being embraced by Ford took 12 years to produce desired changes in culture. However, as in his SASC testimony, Fox focuses attention no the restricted length of the Program Management Course at DSMC. It is his contention the defined length grew out of the 138-day limit imposed by DoD on the length of a temporary duty assignment. It is certain that this is part of it, but there have been other military administrative and command considerations. How officer efficiency reports are made out and the value of a command or troop leading over a school assignment report are important examples. There is also the "time" an acquisition organization believes that they are able to afford the absence of a key officer or civilian. Many times the students at DSMC complain, in private, that they are required to perform both duties at the same time, schooling plus the duties of acquisition in their home unit. Fox is correct when he stated DSMC is under great pressure to shorten the course length from 20 weeks to 14. "In testimony before the SASC in 95 1984, senior Army, Navy and Air Force officers expressed their satisfaction with the current arrangement" (Fox, 1988). As a counter to the way the Pentagon views and operates the position of Program Manager, as a reward for a certain number of months after an arduous assignment, Fox points to the defense industry and how they use the concept. The program manager concept works well in industry . . . 1. They have genuine decision-making authority regarding personnel assignments, promotions, technical matters, and budget. 2. They have years o f training and experience in the operations o f the industry relevant to the program. 3. They understand the roles performed and the tactics employed within government, within industry, and between government and industry. Lacking the training, experience, and stature o f the private-sector counterparts, DoD program managers encounter another serious obstacle to performing their job. Namely, they are required to respond to (indeed, often placate) many people capable o f influencing their career: people in the buying command, the using command, the service headquarters staff, the service secretariat, Office o f the Secretary o f Defense (OSD), and Congress. These management groups have voracious appetites for data from the program manager: 1. How much will it cost to reduce the aircraft by two hundred pounds? 2. What is the consequence o f reducing the budget by $300 million? 3. How much can we save by reducing the production rate by 50 percent? 4. What will be the impact o f reducing the radar? 5. What caused the delay? 6. What will it take to regain the lost schedule? 7. How much can we save by introducing competition for the next production contract? 96 Taking note of the general "run of the mill" questions that are an every day occurrence in every program office, a reflection on Fox's statement concerning "theoretical topics such as motivation, personal assessment, typology, and interpersonal behavior" is unwarranted. In the sense of modem technology, no program manager is capable of single-handedly answering this myriad of questions. The program manager must rely on others and in some cases even the contractor for answers. Since this is a true example, one primary responsibility of a program manager is the management of him/herself and other people. It is not a high level of technical expertise that sees a program manager through such dilemmas. It is an understanding and education in the management of people, an "academic subject" which according to Fox is, ". . . too theoretical to be taught at DSMC." 97 Obstacles To Improving The Defense Acquisition Process: Dr. J. Ronald Fox (1989) Fox presented a well researched study as a part of the National Security Program Discussion Paper Series. O f the four major problems facing the Secretary of Defense, three related to program managers: 1. Too few government managers, at all levels o f the acquisition process, have the necessary understanding, skills, and experience in business management and industrial cost controls to manage the acquisition process effectively. 2. Government managers often enter the acquisition field too late and leave too early in their careers for the Defense Department to have an institutional memory. Necessary career-enhancing assignments in acquisition are rare. 3. There are too few acquisition manager role models who achieve effective control o f cost. (By role models, I mean individuals who are singled out, rewarded, promoted, and retained in the acquisition field.) At question in this study is the true role of the program manager. The Office of the Secretary of Defense issued DoD Directive 5000.1 that outlines the management responsibilities of acquisition. However, as is true of many official regulations, the book definition and the actual job differ markedly. The official responsibilities are: 1. Establish firm and realistic system and equipment specification. 2. Define organizational relationships and responsibilities. 3. Identify high-risk areas. 4. Select the best technical approach. 5. Explore schedule, cost, and technical performance trade-off decisions; appraising the impact these decisions will have on the schedule, cost, and technical performance o f contractor operations. 6. Establish firm and realistic schedules and cost estimates. 98 7. Formulate realistic logistics support and operational concepts. 8. Lay the groundwork for adopting an appropriate contract type that will elicit the desired contractor performance, given the mix o f the contractor's business, the structure o f the contractor's overhead rates, and the contractor's investment in the business. General Henry A. Miley, USA (ret) describes the government program manager's role as . . . . . . the Department o f Defense does not develop or produce its weapon systems in-house. The management o f these functions is contracted for, with, or through prime contractors. Hence the principle functions o f the project manager and his staff are contracting for, monitoring, controlling, and evaluating the technical performance o f the contractor(s). . . In reality, he does not enjoy the authority for his project. He cannot, unilaterally, make any substantive changes in cost, schedule, or performance characteristics o f his system. . . Technical design decisions are made by the contractor. . . His fundamental responsibility is to continuously observe and evaluate the technical performance o f the contractor(s). . . The environment of the Pentagon is based upon the primary consideration that the military services exist for combat. As Fox points out, it is only natural to seek a youthful service by retiring out men in their forties. However, as he further illustrates with case studies of all the services, the added mission of acquisition is "one of great complexity, one requiring training, career development, duration of assignments, and length of career radically different from that required for combat" (Fox, 1989). He further clarifies in the report, "In a 1986 study of defense acquisition, the General Accounting Office found that many government program managers were 99 simply not equipped to tackle the intricate problems of weapon systems procurement. He also quoted Norm Augustine's (CEO, Martin-Marietta) response to Bingaman's 1984 SASC hearing. Asked if DoD personnel have the necessary training and qualifications, Augustine replied: I would say that in many cases they truthfully don’t ... It is pretty tough to be equipped to do that when one comes out o f a military operating force. It would be much the same as taking somebody like myself and putting them in charge o f an air wing; it would be a terrible mistake. Many industry leaders and retired military personnel believe as Augustine does. Fox posits, "Given little authority and few tools to manage their programs, program managers are often relegated to functioning as briefing specialists and marketing managers, spending much of their time seeking additional funds and continued support for their programs" (Fox, 1989). The literature examined to this point defines a military culture that views the acquisition of war materials as an additional function and duty that an officer may be asked to perform at some point is his career. Men who have worked closely inside the Pentagon in very prestigious positions like Packard have attempted to fix this problem with memoranda and findings of blue ribbon committees. Others, like Fox, have invested years of their lives in 100 the study of how defense acquisition really works in attempts to correct its illnesses. Others, such as the leaders in the industrial community, have openly suggested that military program managers lack both needed knowledge and experience. Some of the Pentagon's retired generals are of like mind. Certainly the investigating arm of Congress, the General Accounting Office, has significant evidence that military program managers and others assigned at all levels of acquisition management lack the skills, education, and training necessary to perform effectively and efficiently in the execution of their tasks. These environmental factors came into sharper focus in 1990, when the Investigations Subcommittee of the House Armed Services Committee filed its report. 101 Critical Issues in the Defense Acquisition Culture: Government and Industry Views from the Trenches: Dr. J. Ronald Fox and Mr. George Krikorian (1995) This study was requested by DoD's Program Integration Office of the Under Secretary of Defense for Acquisition (USDA). The study was conducted without restraints or qualifications on its findings and recommendations. However, the study cannot be framed as a scientific study, in that the participants were not randomly selected, but were recommended. As such, according to the theories expressed in O'Sullivan and Rassel, Issac and Michael and others, the study fails in "construct validity." However, the study represents the opinion of the group of experts chosen by Fox and Krikorian, DSMC Industrial Chair for American Defense Preparedness Association and National Security Industrial Association. The purpose of the study was defined in the Executive Summary: . . . this study addresses characteristics o f the defense acquisition process and culture that are viewed by government and industry practitioners alike as inhibiting the proper exercise o f responsibilities and that undermine DoD's mission to deliver satisfactory weapons on a timely and cost-effective basis. Personal interviews, coupled with a completed questionnaire (87% return) from 80 industrial and government defense industry managers, formed 102 the basis of the report's findings and recommendations. Three topical issues were identified: Issue #1 Ambiguities and conflicts in the roles of program advocate and program manager. Issue #2 Burdensome oversight and review activities imposed on contractors, government program offices and DPROs by the DCAA, military service hierarchy, OSD, the DoD Inspector General, the General Accounting Office, and the Congress. Issue #3 Deficiencies in acquisition qualifications and training, including lack of a structured mechanism for capturing and conveying lessons learned from prior acquisition programs. The primary question the study was directed to address by the Office of the Secretary of Defense was to determine, "overlaps in the roles and responsibilities of the government and industrial program management." Most experienced acquisition managers do not see overlap between government and industry as a major problem. What they see is a conflict between the reporting o f realistic program status, on the one hand, and doing what is necessary to keep the program funded and moving through the acquisition process, on the other. Program cancellation or reduction in scope is perceived by (government, sic) program managers and their service superiors as failures (added emphasis). As such, government managers often must decide whether the "success" o f a program , meaning its continuing existence and movement through the acquisition process, takes precedence over reporting realistic program status and problems. 103 The relative instability of the senior levels of the Pentagon is inexorably tied to the same instability of the Executive Branch and is compounded at the managerial levels in the Pentagon, as officers are rotated every two to four years. Further, the "up or out" syndrome causes acquisition officers to retire, "... when the officers are beginning to accumulate the extensive experience required to exercise good judgement in dealing with the dilemmas inherent in acquisition of large, complex, development programs." Regarding civil servants: Government civilians, by contrast tend to become proficient in a single discipline, but their career progression is restricted by the fact that they typically cannot aspire to being the "boss." This combination o f circumstances results in a set o f sub-cultures that hamper communication and make meaningful change difficult (added emphasis). Since promotions are controlled by Pentagon line officers, whose views on acquisition are frequently distorted and not based on the reality of sound industry management principles, acquisition officers are under extreme pressure to view their program in the most optimistic perspective, because only those who demonstrate progress are rewarded by the DoD system. The report cites a GAO report which states: 104 Acquisition reforms have had limited effectiveness because they have not changed the basic incentives or pressures that drive the behavior o f the participants in the process. Reforms have also suffered because of acquisition executives limited ability to effect cultural change. This last point is very salient. In the view of these authors, a change in culture can only occur if there is a change in environment. It is believed that this theory expresses a certain naivety. However, resolution theory is: RECOM M ENDATION tissue #11: The Secretary of Defense should direct the military services to transfer to the Under Secretary of Defense (A&T) operational control of their acquisition organizations and detach to the Office of the Under Secretary, the military and civilian billets and personnel associated with those organizations. Irrespective of whether the current service acquisition organizations remain separate from each other, they should be placed under the direct control of the Under Secretary (A& T), who will have both assignment authority and promotion authority for military and civilian personnel assigned to the acquisition corps. The changes should produce an unambiguous chain of command for assignm ent o f responsibility and accountability from the USD(A&T) to the service acquisition organizations and their program managers. Over the next several years the changes should also produce top to bottom management by a corps of trained and experienced professional acquisition managers, led by individuals at the top who have actually experienced life on the line. Issue #2 discussed in the report fails to concentrate on the cultural aspect of what is the driving force behind burdensome oversight and reviews that are a result of a number of DoD cultural norms. The first cultural norm is called, "briefing-up-the-chain-of command." Every military operation must 105 inspect its lower echelons prior to any military engagement. So it is with briefings. When they are scheduled or due as a matter of process or procedure, Program Managers must brief their status up-the-chain to obtain the approvals of "what" and "how" they are going to say. At the time of the initial Defense Management Review (DMR) in 1989-1990, a DoD edict came down-the-chain to "stop all up-the-chain briefings." That did not last very long. The second cultural norm is a consideration that Program Managers are overly optimistic and will seek ways to brief their program in a manner that exhibits it in the most positive light. This has significant bearing on "how" they are going to say things. Most military officers pride themselves on being ethical and honest. However, as noted in the work done by Dr. Donahue of Georgetown and Dr. Cook of Santa Clara, DoD personnel are hung up at levels 3 and 4 on the Kohlberg scale of 1 to 6. This is consistent with the findings of the GAO and the researchers of this report, . . one must keep in mind that they do not act irrationally or with bad intentions. Rather, they do what they believe is right, given the pressures they face.” The result is a history of very publicized programs that have been "briefed-up-the-chain,” in sequence from the Program Manager to Program 106 Executive Officer to Senior Acquisition Executive to Under Secretary of Defense for Acquisition to Secretary of Defense to Congress, that to some seemed at the outset to be disingenuous. As a result, Secretaries are forced to resign, officers are forced out- of-service, contracts are cancelled, civilians are fired or demoted, GAO and the Inspector General accomplish numerous investigations, DoD conducts its own internal investigations, lower echelon military careers are shortened, and law suits are filed. Oversight has become a big business in the GAO, IG, and DoD. However, the large number o f oversight managers between the level o f the service program manager and the Under Secretary o f Defense results from an effort to collect information to enable each higher level o f management to be prepared for any questions that may be asked about a program. RECOM M ENDATION tissue #21: The Secretary o f Defense should direct the DoD oversight organizations cited above to reduce their oversight in those situations where a contractor and the government program manager can demonstrate that they have established a reliable process for reporting program status and realistic estimates to complete. RECOM M ENDATION tissue #31: The Under Secretary of Defense (A&T) should direct that all government oversight contacts with a program office and its contractors be coordinated through a position to be designated within the Office of the Under Secretary o f Defense (A&T). RECOM M ENDATION tissue #41: The Under Secretary of Defense (A&T) should explore the causes of extensive GAO oversight audits and correct their causes wherever possible. RECOM M ENDATION tissue #5): The Secretary and Under Secretary o f Defense (A&T) should take strong action to defend 107 program managers from inappropriate interference by outside organizations and condemn the use of hindsight knowledge to criticize program management performance. The third issue discussed was deficiencies in acquisition qualifications and training. The researchers initiated the discussion by providing their definition of what culture is and how it comes about: Culture is the integration pattern o f behavior that reflects an institution's values: how the people in the organization think and how they behave. These values do not come from organization charts or policies, and they do not come from plans and strategies. They come from managers, from training programs, and from incentive systems. Understanding this concept is key to understanding the defense acquisition process and how it operates. This definition of culture and where it comes from is too simplistic in nature and is not anthropologically correct. According to Szilagyi and Wallace, (1987), "Adaptations to surroundings (environment, sic) constitute culture. Culture includes those ideas shared by groups, individuals and the language by which these ideas are communicated. In essence, culture is a system of learned behavior." In another sense, culture can be a behavior of self-selection, a knowledgeable deliberate choice made to select an environment that is suitable to an individuals beliefs, values and ethics. Schein (1988), posits that the elements of organizational culture are: 108 1. The organization's relationship to its environment. Reflecting even more basic assumptions about the relationship o f humanity to nature, one can assess whether the key members o f the organization view the relationship as one o f dominance, submission, harmonizing, finding and appropriate niche, and so on. 2. The nature o f reality and truth. Here are the linguistic and behavioral rules that define what is real and what is not, what is "fact," how truth is ultimately determined, and whether truth is "revealed" or "discovered"; basic concepts o f time as linear or cyclical, monochronic or polychronic; basic concepts such as space as limited or infinite and property as communal or individual; and so forth 3. The nature o f human nature. What does it mean to be "human," and what attributes are considered intrinsic or ultimate? Is human nature good, evil, or neutral" Are human beings perfectible or not? Which is better, Theory X or Theory Y? 4. The nature o f human activity. What is the "right" thing for human beings to do, on the basis o f the above assumptions about reality, the environment, and human nature: to be active, passive, self-developed, fatalistic, or what? What is work and what is play? 5. The nature o f human relationships. What is considered to be the "right" way for people to relate to each other, to distribute power and love? Is life cooperative or competitive: individualistic, group collaborative, or communal; based on traditional lineal authority, law, or charisma; or what? The researchers using a simplistic mode of cultural definition, then assumed that a change in culture to make the acquisition corps more professional, required that only the issues of management, training, and incentives be addressed in their solution set. Their conclusions are based upon that assumption. Four years after the enactment o f DAWIA, most government acquisition managers and their industrial counterparts report that 109 many government managers are not adequately prepared for acquisition assignments. As illustration, they cite lack o f familiarity with lessons learned on prior programs; unfamiliarity with industry practices; and insufficient skill in using the tools o f financial management. Government and industrial practioners perceive an urgent need for more carefully programmed career management and more practical training comparable to that now provided for combat arms officers and industry acquisition managers. Several industry managers expressed the view that military chiefs o f staff usually do not take acquisition as seriously as they take operations, or for that matter, as seriously as they take the legal corps or the medical corps in their service - even though the latter two corps, unlike the acquisition corps, come already equipped with trained and qualified forces. Training inodes using the "lecture and vugraph" approach to training come under sharp attack. The heavy emphasis on communicating governmental rules and regulations, wherein the research indicates business management develops and relies on judgment, is also attacked It is their belief that: Professional training requires the opportunity to question, discuss, and practice the skills one is expected to perform in a profession. It also requires that lessons learned from actual experience he collected systematically, communicated, and practiced as part of one's preparation for the profession. Reliance on case study problems as the preferred method of instruction, in consideration of the norms of this cultural self-select group, 110 could prove to be very unsatisfactory for a number of reasons. A few of these reasons and the lessons learned are discussed next. Case study problems in the military are patterned after the study of military battles, a case of winners and losers. There is an approved "school solution" to every tactical battle. The problem is that, when a case such as the A-l 2 or C-l 7 is presented, the case may be biased and not written from an honest brokers viewpoint. Also, examinations of current case studies may lead to the possible embarrassment of officers who are still in the service or who may have been tainted because they were assigned to the program of the case study. Possibly, better models exist than case histories. Such a model might be the study of anticipated or possible problems of either real current or imaginary programs. The researchers recommend: RECOM M ENDATION tissue MY. The Under Secretary of Defense (A&T) should exercise aggressive leadership in career management, creating a truly professional acquisition corps, requiring prospective program managers to have preparatory assignments in which they: - develop skills in translating military requirements into program specifications, in understanding the reasons for changes in specifications, schedules and funding, and in minimizing these changes; and - develop an understanding of industry incentives and practices, learning how to deal with situations in which 111 government and industry share the same goals, as well as situations in which they have differing goals. RECOM M ENDATION tissue #71: The Under Secretary of Defense (A&T) should exercise the authority currently vested in his office to: - establish incentives (e.g., attractive career development paths and promotion rates at least comparable to those of the combat arms to ensure that the acquisition corps attracts and retains capable, outstanding military officers and civilians, with experience commensurate with the technical complexity and major responsibilities associated with managing multi-billion dollar engineering development and production programs. - establish and enforce processes to vigorously implement the authorities identified in Paragraph E, DoDD 5134.1, "Under Secretary o f Defense for Acquisition (USD(A))," dated September 30, 1992, with particular emphasis on the assignment, reassignment, and training of key acquisition personnel. RECOM M ENDATION tissue #81: The Under Secretary of Defense (A&T) should direct the Defense Acquisition University (1) to collect lessons learned from prior and ongoing acquisition programs [describing successful approaches, problems, and failed approaches] and (2) to use these lessons learned in practical training programs for prospective program managers. Such programs should also be designed to convey a working knowledge o f the ways to conduct effective oversight of contractors without imposing excessively detailed reporting requirements. 112 Department of Defense Instruction (DODI) 5000.55: Reporting Management Information on DoD Military and Civilian Acquisition Personnel and Position (1991) Shortly after the passage of DAWIA, detailed, line by line, instructions were issued for the establishment of a DoD management information system that would be used by the Services for the registration of all personnel in acquisition positions. This system would be the first step in creating a DoD capability to monitor, report and track the composition, education, experience, and training of the acquisition workforce. These instructions also established methods for the validation of the educational and training data of each assigned individual. 113 Department of Defense Manual DoD 5000.52-M: Career Development Program for Acquisition Personnel (1991) As was noted in the HASC / SASC hearings and reports, there was a significant attempt to create a level playing field between military officers and civilians assigned as Program Managers and other critical positions. The law itself contains a provision for: the "limitation on preference for military personnel", "opportunities for civilians to quality" and under the "management of workforce:" The Secretary o f Defense shall ensure that the acquisition workforce is managed such that, for each fiscal year from October 1, 1991, through September 30, 1996, there is a substantial increase in the proportion o f civilians (as compared to armed forces personnel) serving in critical acquisition positions in general, in program manager positions, and in division head positions over the proportion o f civilians (as compared to armed forces personnel) in such positions on October 1, 1990. The manual's stated purposes are: Establishes the Department o f Defense Career Development Program for Acquisition Personnel. It includes procedures for effective career development o f all persons serving in acquisition positions in the DoD. . . The program establishes education, training, and experience standards for specific acquisition workforce position categories and career fields, provides for certification o f acquisition workforce members, and provides career paths for the acquisition workforce. Is designed to improve the management and professionalism o f the acquisition workforce by . . . Increasing the proficiency o f DoD acquisition personnel in their present position and providing guidance and 114 opportunity for broadening experiences and progression commensurate with abilities. Ensuring effective use o f training and education resources. As previously noted in the HASC report, The Quality and Professionalism o f the Acquisition Workforce, the military career for officers includes a long-established Professional Military Education which includes advanced military schools, colleges, and graduate level education. The DAW1A directed a comparable system of professional education for civilians and an emphasis on graduate programs. Planning for Career Development Career development plans shall be developed for military and civilian acquisition personnel. . . The generic career paths in the Appendices to this manual shall be used as a guide when developing career development plans. Military Personnel. Procedures for career development plans for military personnel are established by each Service. For military career fields that are in the acquisition workforce, the career development plans shall incorporate standards in this Manual. Civilian Personnel "Individual; Development Plan" or its equivalent, is a primary career planning tool for civilian personnel and goes hand in hand with performance management. An Individual Development Plan (IDP) shall be prepared for each civilian member o f the acquisition workforce, until certification at Level III is achieved, with the assistance, advice, and review o f his or her supervisor, personnel office, and career advisor. In the IDP, the employee and supervisor document the employee's short- and long-range career goals, development objectives, training and 115 development toward those goals and objectives, and desired future development activities or assignments. In the DAWIA, the expressed intent of Congress is to increase the presence and influence of civilians as opposed to military in the acquisition workforce. A part of the Congressional intent was also the leveling of the military - civilian playing field in accession, education, training and career management. In viewing the above implementation of the DAWIA by DoD, it appears that a slight leveling of the playing field has occurred, but the Officer Corps still retains the centralized corporate administration of professional military development, which now includes the separate service acquisition career development plans. The civilian, however, does not have this centralized corporate administration advantage and is still limited to making a best guess, with his local supervisor as the advisor. In addition, the question concerning the availability of education and training is in question. The number of civilians selected to attend professional military educational schools and colleges is minuscule. PME selects and sends officers for advanced degrees. Officers also have available Montgomery College Funds. The civilian does not have equal opportunity or funding availability, since it is provided locally out of assigned unit 116 educational allowances. It would appear that the intent of Congress to level the playing field between officers and civil servants has not been successfully addressed by DoD. Under the title, Career Development, specific standards for education, training, and experience for each position category and career field by career level for civilian and military personnel are specified. The career fields are divided into three career levels: LEVEL QUALIFY CIVILIANS MILITARY I BASIC OS 05 - 08 01 -03 II i n t e r m e d i a t e OS 0 9 - 12 03 - 04 III SENIOR OS 13 & > 04 & > ’able 3 Career levels are generically defined as: Level 1 Basic level training standards are designed to establish fundamental qualifications and expertise in the individual's job series or career field. Development at the basic level lays the foundation for career progression and is designed to prepare qualified, motivated personnel for positions o f increasing responsibility. At the basic level, trainees should be exposed to the functions o f acquisition and the roles o f its various specializations. Therefore, in addition to participation in education and training courses, it is extremely beneficial to develop new trainees through structured on- the-job rotational assignments between a variety o f functional offices. Level II At the intermediate level, specialization is emphasized. Development continues, including on-the-job rotation assignments, but the length o f time an individual spends in each position generally increases. While specialization is emphasized at the beginning o f this level, the individual should later begin to broaden his or her background toward a more general expertise in the overall processes of his or her career field. Development o f the generalist normally involves establishing a good foundation o f experience in the employee's primary speciality followed by lateral movement to a related speciality. Level 111 By the time an individual reaches the senior levels o f acquisition management, he or she should have completed all mandatory training and educational requirements to that level, and should have advanced through a career pattern that has given him or her depth o f knowledge in his or her functional area and breath o f knowledge across the entire acquisition process. At the senior level, advanced acquisition education and training become essential. Individuals at this level should seek acquisition related opportunities available through the DoD components, or pursue other formal educational opportunities. The manual states that certification is the process through which DoD determines that the individual meets the mandatory standards (education, training and experience) established for that acquisition career field and career level. It is further recommended that individuals be certified in more than one career field. This recommendation is most important as discussion on the Program Manager's career field is continued. 118 Department of Defense ADS-93-01-GD: Acquisition Career Management, Mandatory Course Fulfillment Program and Competency Standards (1993) When a new requirement is introduced into any workforce, standards need to be established for those individuals who have education and experience in the field. The Fulfillment Guide served that purpose for both the military and civilian acquisition workforce. The guide also specified the process and procedure to be followed for members of the acquisition workforce to be certified by providing proof of their prior accomplishments. All career fields, at all career levels, are defined in the guide. For Program Managers the competencies self-assessment for Level I, II, and III are contained in Attachment 2. 119 Incentives of the Program Manager: Unpublished Student Report (1994) This examination of the Program Management literature has to this point been limited to institutional type documents. However, the expression of Program Managers themselves, regarding their concepts and perceptions of the acquisition process and their operational environment is of particular interest. Incentives o f the Program Manager is a statement of Program Managers on how they see the environment within which they are required to operate. Their unpublished report says: The 1991 Defense Acquisition W orkforce Improvement Act (DAWIA) establishes a career acquisition workforce and required qualifications and training for both civilian and military personnel involved in acquisition. The Federal Acquisition Streamlining Act of 1994 simplified procedures for procurement under $100,000. While all these reforms have been well intentioned and have been aimed at improvement o f weapon systems acquisition, the prospects for the future are for more o f the same kind o f problems. The current system does not provide any incentive for realism in program planning or status reporting. To the contrary, the basic drives of the acquisition process encourage optimism, parochialism, and protectionism to the extent that program survival has become a more powerful incentive than program affordability (Added emphasis). GAO has concluded that the persistent problems may be the consequence o f an acquisition culture which is actually meeting the needs o f its individual participants. 120 This case study was prepared from existing data bases collected by Dr. Owen Gadeken, Chairman of the Educational Department at DSMC and Dr. J. Ronald Fox, Harvard Graduate School of Business, who with Mr. George Krikorian, Holder of the National Industrial Security Association / American Defense Preparedness Association Chair in the DSMC Executive Institute, completed interviews in 1994 with acquisition officials in both industry and DoD. O f significant importance, this study completed by Program Managers examining other existing data files yields an interpretation that is based upon their own perspectives. The related motivational literature they first examined was that of Maslow, Skinner and Rogers. The writers of this study first cite Maslow: A greater value is usually placed upon the higher need than upon the lower by those who have been gratified by both. Such people will sacrifice more for the higher satisfaction, and furthermore will more readily be able to withstand lower deprivation. For example, they will find it easier to live ascetic lives, to withstand danger for the sake o f principle, to give up money and prestige for the sake o f self-actualization. Their initial conclusion states, "The difficulty with higher level motivators is discovering that true relationship between an incentive and a 121 motivation. Likelihood of success, individual perceptions, and conflicting goals are among the many factors determining this relationship." The problem is one of integrity of leadership, when the leader's "person interests" are at stake. Turning to Skinner and Rogers for resolution they find: . . . that group approval for an individual's behavior to the group can be a more powerful incentive than avoiding hardship or martyrdom. Their final conclusion is that, "A leader's commitment to the organization could well lead him or her to actions opposite to the leader's personal interests." In studying the Congressional intent of changing the "acquisition culture," this conclusion of Program Managers reached from analyzing Program Management "natural intent," becomes vitally important. This thought seems to be a natural extension of Laswell's management o f violence. The high order of officer training required for this expertise, requires that the officer be prepared physically, intellectually, and psychologically. When such an officer who is interjected into a Program Office scenario, both the program and the people of the program office become for some officers more important than self or their ego. From this motivation conclusion, the study then logically cites Hudy: To make motivational efforts successful, management must focus on intrinsic factors - rewards that come directly from performing the task - instead o f extrinsic factors - rewards that are given for performing a task. 122 This, that we shall now refer to as a "program protection syndrome or construct," becomes the model of operation which guides or governs other behaviors, such as "avoiding actions that are perceived to hazard a program." To counter this environmental consideration the report posits: Effective leaders need for themselves, and realize that their teams need, worthy and enduring goals for organizational success. Within the framework o f current DoD program acquisition experiences, avoiding "problem finding" behavior is essential to the vitality o f the effort, because finding problems puts the program at risk - not because o f the problem itself - but because o f the effect the problems will have on external relationships. Just as "completing the mission" or the "reaching of the objective" is a guiding axiom of the Officer Corps (and would we in truth want it any other way), the "Program" becomes the mission or objective for the Program Manager. Recognizing this cultural perspective, the findings of the study are contextually understandable: 1. A program manager believes in the merit o f his program and in the need for its continuation. Because the program manager is motivated to work for continuation o f his program, he is naturally drawn into the role o f program advocate. Generally, the program manager will be criticized for excessive advocacy if his program falls out o f favor, but praised for visionary leadership if his program continues. The dividing line between excessive advocacy, which results in unrealistically optimistic program assessment to higher authority, and necessary leadership optimism in the face of program risks and uncertainties is difficult for the program manager to s e e .. . (added emphasis) 123 2. The adversarial budget process penalizes any perceived risk of failure to meet performance, schedule, or cost goals. This effort is a tremendous drain on the resources o f the program office, and severely limits the amount o f time the program manager has available to actually manage his problems. To ensure program survival, the program manager is motivated to pay more attention to continuing justification o f his program in the various stages o f the budget process than to the management details o f weapon system acquisition. 3. Bureaucracy and oversight prevent progress. It is clear that the current acquisition system is not designed to support the program manager, but rather establishes an adversarial process whereby the program manager is required to continually respond to inquiry and criticism, and where success is achieved in spite of the acquisition process. 4. Career goals o f a military officer and a program manager are often at odds. As long as the career goal o f the military program manager is to "punch the ticket" in program management in order to be promoted to higher level command assignments then this situation will continue. The acquisition career path for military program managers is not seen as rewarding successful program management with retention and promotion. Corrective measures required to mediate the findings of this study of Program Managers by Program Managers are institutional in nature and are in large measure is a part of their self-selected culture. Equation of a program to a mission or to the capture of a military objective is a cultural derivative. The number of internal Pentagon reviews is also culturally derived. An analogy is that each successive officer in the chain of command may want the 124 troops ready for "budget inspection" ten minutes earlier than the next officer in the chain-of-command. As to the budget process itself within the Pentagon, it is an axiom within the Pentagon that if one's Program spending plan falls behind for any reason, the comptroller will take away the unspent funds, disregarding Congressional intent. Promotions of military officers assigned to the Acquisition Corps are controlled by military officers. If military officers in these career fields are not receiving promotions, it is not only in violation of Congressional intent, but of law as well. There is a mandatory oversight Congressional report to "preclude" this abuse. However, the report provides realistic recommendations on cost reviews and oversight: The key, then, is to maintain sufficient oversight to recognize failed programs which should be terminated, and to find a means to eliminate the continuous threat o f loss o f program support at the slightest hint o f any risk or deviation from plan. A possible approach would be a joint process between DoD and Congress to identify high priority programs to be baselined in cost, schedule, and performance at the time o f major milestone review. The program manager's use o f management reserve should not be considered a sign o f fatal risk or difficulty and justification for budget cut, but rather a normal part o f the development o f new technology. The program manager's task must be changed from one o f continuous defense o f the program to one o f management o f a challenging effort with freedom to use management reserves o f cost, schedule, and performance, without the continuous threat o f loss o f support from the acquisition process. Until this happens, the 125 program manager will always be forced into the role o f program advocate and defender, to the determent of successful m anagement of weapons system acquisition (added emphasis). The Role of the Government Program Manager: Unpublished Student Report (1994) What is the role and duty of a Program Manager? What surprised the research investigators was a finding that the role was not formally defined. "One would surmise that formal roles and responsibilities o f a Program Manager would be clearly delineated and documented in law, Congressional language, and Defense Department directives instructions and manuals. . . this was not the case. The law and most o f the service and DoD regulations are unexpectedly vague with respect to the Program Manager's specific job description." Although significant numbers of guiding principles are identified in the 5000 series of DoD documents, the investigators f o u n d , . . they still lack sufficient guidance to fully define the assignment." As a result of their research using DSMC's Program Management data files, they identified 5 overlapping Program Manager generic roles: • The Program Manager as an Interface with the Bureaucracy • The Program Manager as an Interface with the Contractor • The Program Manager as a Social Advocate • The Program Manager as a Manager / Leader 126 Interface with Bureaucracy During interviews with Congressional staffs, it was found that most oversight agencies believe that "the program manager function is an honest broker role, providing frank, open and detailed information regarding the program’ s actual status." This perception of the Program Manager's role contrasts with the actual role of the Program Manager as program champion in the DoD culture. This Catch 22 is typically caused by over-optimistic PM status reporting, which, when determined to be "disingenuous" contributes to distrust and adversarial relationships with Congress. To correct this situation, it has been suggested that to save the Program Manager from dying on his own poniard, "a program sponsor be added with the following division of labor: . . . establish a team relationship with their sponsor wherein the sponsor was responsible for advocating the need for the system and working the budget issues while the Program Manager was responsible for implementing and articulating the acquisition process. This approach presents an excellent model for program management advocacy. The Program Manager must not be charged with demonstrating the need for his product; rather he should be responsible for articulating how the program meets the need, how the system trade-offs have been made to provide for the best solution for the budget dollar, and how technical issues are being addressed. 127 In this discussion, advocates of a program sponsor point out one other salient feature that occurs inside the Pentagon, the difficulty of dealing with organizations outside the chain-of-command that have their own agendas and constituencies. This cultural aspect of DoD has not been previously discussed. These organizations may represent technical, social, or other interests and will try to force their policy, program, or other agenda on the Program Manager. The Program Manager ignores these forces to his own hazard (or program hazard, sic). In an atmosphere of down sizing and realinement, coupled with budget shortfalls caused by unanticipated "peace keeping" missions, a program that is duly authorized and funded by Congress can suddenly awaken to find the Service Comptroller has zeroed out their funds. Interface with the Contractor A part of the mystic of the Program Office that is embedded in the culture of DoD, is that the Program Office, and specifically the Program Manager, is in technical control of the program. This does not match realities of the situation. The Program Manager often is limited to involvement in an indirect manner in so far as the actual designing, manufacturing, and assembling o f the system is concerned, although this will vary by service and product line. . . 128 . . . the Contractor must contend with actual performance o f the task and overcoming the technical and engineering hurdles encountered. The establishment o f an adversarial contractor relationship will only yield a program that will be under constant micromanagement review by all involved Governmental agencies (GAO & IG) and Congress. Since it is clear that the objective o f both parties are worthwhile and that their motives can be presumed to be honorable, it follows that a role and responsibility o f each is deserving o f respect. Given a "win - win" philosophy. . There is a need for mutual respect and long lasting commitment to mutual program goals. An often overlooked contractor interface issue is a consequence o f the relative frequency with which Program Managers are changed. The advice given to a new Program Manager is to review the, "preexisting agreements" of the old Program Manager and to conduct oneself in a manner, "avoiding any conflicts or confusion.” One of the several interface roles that become confused is the interface between the Program Manager and the Program Contracting Officer. Only the Program Contracting Officer has the warrant to execute contracts and to make amendments in the name of the United States. A Program Manager does not have such authority and this contracting consideration is an important part of the roles each must play in dealing with the contractor. 129 Social Advocate Congress has mandated that the expenditure of each defense dollar must satisfy two requirements. One is to build weapon systems to overcome real threats. The second is a socioeconomic consideration dealing with social agendas and the Program Manager, as a public servant. This second requirement takes the form of ensuring compliance with Congressionally mandated social-economic programs. Among these acts which are provided for small and disadvantaged business utilization, labor practices and relations, and protection o f the environment. Since the program is executed under government supervision by a contractor, who is required to observe certain laws and regulations with socioeconomic impacts, the Program Manager will acquire the role of enforcer of those requirements (added emphasis). Another concern is to improve the environmental quality through pollution control, energy conservation, identification o f hazardous material, and use o f recovered materials. . . No Federal employee immunity is provided with respect to environmental statutes. Program Managers are therefore potentially personally liable for violations o f environmental laws and regulations within their programs. Manager The responsibility of the Program Manager is to act as a centralized management authority in the attainment of the cost, schedule and performance goals that are defined by the Service and the Milestone Decision Authority. 130 To accomplish this task: Program Managers must be capable o f effectively dealing with problems in personnel, business, financial, and technical management and must be able to view these factors in the context o f the overall program. Staffing the office with competent, motivated people is especially important to program success and is a critical concern o f the Program Manager. M anagers must recruit the most highly qualified and best performing people available. He then needs to meld these people into a coherent team, ensure that they are properly trained and supervised, and that they receive the support required to do their job. This managerial responsibility also extends to an ongoing interface with the user. A program requirement meeting a threat is never a closed issue. Requirements creep, advances in technology, new threats, and unexpected costs will all impact the ability o f the Program Manager to achieve the(program objective, sic) baseline. Each of these tasks is a management task, separate and distinct from a leadership task. Most management tasks are formalized by DoD directives which govern and control in substantial detail what and when the Program Manager is to accomplish as a manager. The following distinctions are made: Leader The most important role o f a Program Manager is as a leader. Leadership differs from management in that it deals with the intangible practices which achieve synergy and create energy in the totality o f the program. Management, on the other hand, deals with the structure and practice o f planning and controlling the project. 131 Management may be flawlessly carried out and still not achieve a truly outstanding result. Leadership, conversely, can often overcome shortcomings in management techniques or programs by taking best advantage o f the skills and motivations o f all the participants and making them more in aggregate than would be implied by simply the sum o f the participants. Distinctions are made between leaders and managers. A manager is expected to understand the administrative mechanics of forming an Integrated Product Development Team; however, "the effective molding of the team into a cohesive entity which works toward a common set of goals and objectives, within a framework of mutual trust and confidence is a function of leadership." The study closes with this conclusion: The Program M anager is a leader, a mentor, a technical arbiter, an occasional referee, a voice on the phone, and a manager. He is expected to meld complex and divergent requirements and agendas into successful defense acquisition. No single description completely describes the Program Manager's role as the program evolves from one acquisition phase to the next. The one constant throughout is that Program Manager is the single individual who is ultimately responsible for achieving success for the program, however it is defined. In the DoD "culture," the Program Manager symbolizes the gist of the program. It could easily be said, S/he is the program." Given this perspective, it is most difficult to understand why congressional staffs and other governmental agencies cling to the perspective that the role of the 132 Program Manager is that of, "an honest broker, providing frank, open and detailed information regarding the program's actual status." The great difficulty of being an honest, frank, and open manager while the DoD "culture" defines the Program Manager as "The Program" at some milestone within the program time-frame is fairly clear. The DAWIA requires the Program Manager to remain with the Program until the next milestone has been achieved. The theory being the responsibilities of decision making reside with the Program Manager who made them. In other words the Program Manager has the duty to achieve the "military objective" e.g. achieve the milestone. The differences in the Congressional and DoD Program Manager perspective creates a Catch 22 scenario. 133 Section B: Jungian / Mvers-Briggs Type Indicator (MBTI) Why do people and organizations seem to behave in certain ways? Since behaviors are external to the individual or organization, they are observable. To some observers, behavior appears to be random variation and to others it is orderly, even, consistent, and measurable. By gathering empirical data, it can be determined why individual preferences and organizational environment significantly affect the way judgments are made and perceptions are drawn. Over fifty years ago, Isabel Myers, driven by the horror of World War II began her work to create an instrument that would help people understand why they made the judgments they did and how they were guided by their own perceptions. By doing so, it was her belief she would help people make better decisions. (McCaulley, 1993) Through an instrumented structure of preference self-reports, people are able to identify their reactions, in a polar sense, defining perceptions and judgments and the attitudes in which they are used. This is why the first overhead chart Colonel David B. Porter, Department Chairman of Behavioral Sciences and Leadership/United States Air Force Academy, shows when opening his lecture on Myers-Briggs Type Indicator (MBTI), reads as follows: 134 PREFERENCE • Reflects Comfort and Choice Not Capability • Self-awareness & Self-control Not Selection The literature to be reviewed is that collection of work surrounding Isabel Myers's accomplishment of her self-directed task. . . to make the theory of psychological type described by C. G. Jung understandable and useful in people's lives" (Myers and MCCaully, 1992). According to Jung's theory, preferences influence not only what people consider in any given situation, but also the manner in which they make decisions about what they perceive. Additionally, as the old cliche phrases it, "Birds of a feather flock together." The Atlas of Type Table published by Psychological Press compiles a listing of various professions and the MBTI's of those who are engaged in that profession. The study of these tables would indicates that people with similar MBTI characteristics tend to self-select certain career fields, while avoiding others. The career fields listed in this Atlas have organizational and functional attributes that seem to provide a belonging or matching with the dominant MBTI profile in that particular profession. It is the hypothesis of this study that the MBTI profiles of those who have self-selected the profession of DoD Acquisition are so significantly 135 dominated by certain of the 16 MBTI types that the behavioral characteristics of this "critical mass" defines a cultural imperative or paradigm. An analogy from Nature serves as an illustrative example. As the pine needles drop and cover the forest floor their decomposition creates an environment which primarily supports the growth of only trees of the same species. It is postulated that when the environment of the workplace is dominated by certain MBTI profiles, far in excess of normal population distribution, that domination tends to establish and control the cultural norms of the environment. Students attending the Program Management Course of the Defense Systems Management College at Fort Belvoir have taken the MBTI instrument since 1985. The cumulative results are defined in a comparison table of the surveyed U.S. population norms and the actuals of the PMC students. This data first published in Bryant's A Description of Psychological Type at the Defense Systems Management College in 1991 has been maintained and updated for each DSMC / PMC class. In the conclusions of his 1991 report, Bryant raises concerns about the acquisition workforce resistance to change, being a result of the numerical domination of certain MBTI types, while other MBTI's in lesser numbers develop a "siege mentality." 136 COMPARISON BY TYPE: Program Management Course / US SURVEYED POPULATION PMC: N = 5463 (7/'85 to 3/’ 95) ISTJ PMC 1 6 2 7 /2 9 .8 % US 5% ISFJ PMC 1 2 4 /2 .3 % U S 5% INFJ PMC- 85 / 1.6% U S 2% INTJ PMC 6 3 3 /1 1 .6 % U S 2% ISTP ISFP INFP INTP PMC PMC PMC- PMC- 2 7 9 /5 .1 % 4 0 / .7% 80 /1 .5 % 3 7 3 / 6.8% US U S U S US 5% 5% 2% 2% ESTP ESFP ENFP ENTP PMC PMC PMC- PMC 1 6 0 /2 .9 % 31 / .6% 1 0 5 / 1.9% 3 2 0 /5 .9 % US U S US US 14% 14% 5% 5% ESTJ ESFJ ENFJ ENTJ PMC PMC- PMC- PMC- 9 22 /1 6 .9 % 94 / 1.7% 63 / 1.0% 537 / 9.8% U S US 1 1 5 US 14% 14% 5% 5% (Gould et al. 1995) Table 4 Since the first publication of this report in 1991, a shift has occurred from the SJ side of the table toward the NT side as the numbers of uniformed officers attending the PMC have decreased and a more diverse group of civil servants have taken their place. As an example, ISTJ percentage in 1991 was 31.5% (N = 2818). In 1995, ISTJ percentage is now 29.8% (N = 5463). 137 MBTI’s Scientific Establishment At the outset of Dr. Jung's Analytical Psychology: Its Theory and Practice, the Editor notes: In 1935, C.G. Jung was invited to deliver a series o f five lectures to members o f the medical profession. The stenographic record o f the lectures was taken and privately distributed by the Analytical Psychology Club o f London. These became known as the Tavistock Lectures. Passages from a similar lecture given in 1934 in Basel, Switzerland were published in French in 1944. Adding a lecture given in Zurich, they all were published under the Collected Works o f C.G. Jung. In Lecture 1, Professor Jung, as he is referred to in the text, after first laying the foundation and definition of consciousness, proceeds to discuss the two main topics, "structure o f the unconscious mind and its contents." Expressing his reasoning as being opposite that of Freud, Jung posit, ". . . the unconscious and that consciousness really arise from an unconscious condition." Basing his concept on a very complex concept of ego, Jung described his theory of the four "functions." Ectopsyche is a system o f relationship between the contents o f the consciousness and facts and data coming in from the environment. It is a system o f orientation which concerns my dealing with the external facts given to me by the function o f my senses. The endopsyche, on the other hand, is a system o f relationship between the contents o f the consciousness and postulated processes in the unconscious. 138 The first function described is sensation, the sense function. Sensation tells one that something is, nothing more or less, not even what it is. The second function is thinking. Thinking tells one what a thing is. Jung states emphatically, "Thinking is perception and judgment." The third function is feeling. He describes feelings as " . . . informing you through its feeling-tones of the values of things. Feeling tells you for instance whether a thing is acceptable or agreeable or not." The fourth function is described," ... you do not know what shares will do after a while, but you get the hunch that they will rise. That is what is called, intuition, a sort of divination, a sort of miraculous faculty." Jung goes on to state, "Intuition is a function by which you see round comers, which you really cannot do; yet the fellow will do it for you and you trust him." Jung develops the aspect of the polar opposites in the discussion of the dominant function. The dominating function gives each individual his particular kind o f psychology. For example, when a man uses chiefly his intellect, he will be o f an unmistakable type, and you can deduce from that fact the condition o f his feeling. When thinking is the dominant or superior function, feeling is necessarily in an inferior condition. The same rule applies to the other three functions. Jung then draws the defining diagram of the polar opposites of the four functions. 139 You can make the so called cross o f the functions (Figure 1). In the center is the ego (E), which has a certain amount o f energy at its disposal, and energy is the will-power. In case o f the thinking type, the will-power can be directed to thinking (T). Then we must put feeling (F) down below, because it is, in this case, the inferior function. That comes from the fact that when you think you must exclude feeling, just as when you feel you must exclude thinking. If you are thinking, leave feeling and feeling-values alone, because feeling is most upsetting to your thoughts. . . . People have sometimes assured me that their thinking was just as differentiated as their feelings, but I could not believe it, because an individual cannot have the two opposites in the same degree o f perfection at the same time. The same is the case with sensation (S) and intuition (I). T S F Figure 1 The Functions Jung's theory of inferior function are the opposite of the differentiated or dominate functions. He concludes, "The inferior function is always associated with the archaic personality in ourselves; in the inferior functions we are all primitives." Jung states most convincingly, "... never force a man into his feelings when he is an intellectual. He controls it with an iron hand because it is very dangerous. . . .but 140 in their feelings they can be influenced, they can be caught, they can be cheated, and they know it." The remaining lectures on day two through day four deal with the applications of Jungian theory to the art of criminology, the collective unconscious, arch types, anima, and other considerations which are interesting in and of themselves but are distant from the focus of this study. Otto Kroger and Janet Thuesen acknowledge in Type Talk: The 16 Personality Types that Detennine How We Live. Love and Work. Jung's work would not appeal to the lay reader, and were it not for the brilliant efforts of Katharine Briggs and her daughter Isabel Briggs Myers, a system to measure the differences in Jungian psychological type would never have been perfected. A text, Manual: A guide to the Development and Use of the Mvers- Briggs Type Indicator, acknowledges the foundational work accomplished by Katharine C. Briggs and Isabel Briggs Myers. The influences of these two permeates this volume via summaries, written records, and supplemental notes resultant from their hundreds of hours of discussion. The work contained in this volume reflects," ... Myers six decades of interest in Jung's 141 theory of psychological types and four decades of research to develop a way of putting Jung's theory to practical use." In addition to the four functions (Sensing, Intuition, Thinking and Feeling) most adequately described by Jung at the Tavistock lecture, the MBTI is comprised of two attitudes, extroversion and introversion (El), described by Jung as . mutually complementary attitudes whose differences generate the tension that both the individual and society need for the maintenance of life." Extroverts are oriented primarily toward the outer world; thus they tend to focus their perception and judgement on people and objects, Introverts are oriented primarily toward the inner world; they tend to focus their perception and judgement upon concepts and ideas. The remaining two elements of the MBTI are Judgment and Perception. Although Jung did not specifically define their position, Briggs and Myers believed that he indicated their application in the overall context. Accordingly, they define Judgment and Perception (JP) attitudes as; The JP index is designed to describe the process a person uses primarily in dealing with the outer world, that is, with the extroverted part o f life. A person who prefers judgement (J) has reported a preference for using a judgement process (either thinking or feeling) for dealing with the outer world. A person who prefers perception (P) has reported a preference for using a perceptive process (either sensing or intuition) for dealing with the outer world. 142 Each of the four indices are comprised of polar opposites; attitudes (El), temperaments (SN) and (TF) and (JP) attitude index. (It is noted here were Jung used the letter I to indicate intuition, Briggs and Myers used the letter N so as to eliminate confusion with the I for introvert.)The result is a possible combination of 16 different types. Jungian theory as expressed by Myers-Briggs postulates," ... specific dynamic relationships between preferences." The dynamic interrelationship between these polar indices was described by Jung,"... but did not go into a great detail concerning, the need for an auxiliary function that was in every respect different from the nature o f the primary function.'' The MBTI assumes: 1. For each type, one function will lead or be dominant. 2. Members o f each type will mainly use their first function in the favorite attitude. 3. In addition to the first or dominant function, a second or auxiliary function will be developed to provide balance. 4. The second function provides balance between extroversion and introversion. 5. The second function also provides balance between perception and judgement. 6. The JP preference points to the function used in extraverted attitude, for both extroverts and introverts. 7. If the dominant function is typically extroverted, the other three functions will be typically introverted. If the dominant function is 143 typically introverted, the other three functions are typically extraverted. 8. The functions opposed to the dominant is typically the least developed or inferior. 9. The function opposite to the auxiliary is the third function. The Theory: Dominant and Auxiliary Functions for Each Type ISTJ ISFJ INFP INTJ Sensing is dominant and Sensing is dominant and Intuition is dominant Intuition is dominant introverted introverted and introverted and introverted Thinking is auxiliary Feeling is auxiliary and Felling is auxiliary and Thinking is auxiliary and extroverted extroverted extroverted and extroverted ISTP ISFP INFP INTP Thinking is dominant Feeling is dominant and Feeling is dominant and Thinking is dominant and introverted introverted introverted and introverted Sensing is auxiliary and Sensing is auxiliary and Intuition is auxiliary and Intuition is auxiliary and extroverted extroverted extroverted extroverted ESTP ESFP ENFP ENTP Sensing is dominant and Sensing is dominant and Intuition is dominant Intuition is dominant Extroverted F.xlroverted and Extroverted and Extroverted Thinking is auxiliary Feeling is auxiliary and Feeling is auxiliary and Thinking is auxiliary and introverted introverted introverted and introverted ESTJ ESFJ ENFJ ENTJ Thinking is dominant Feeling is dominant and Feeling is dominant and Thinking is dominant and Extroverted Extroverted Extroverted and extroverted Sensing is auxiliary and Sensing is auxiliary' and Intuition is auxiliary and Intuition is auxiliary and introverted introverted introverted introverted Chart 4 144 The MBTI theory developed by Myers-Briggs from a Jungian concept goes on to postulate: Together, these functions and orientations influence how a person perceives a situation and decides on a course o f action. Each o f these choices is like a fork in the road o f human development, offering different paths that lead toward different kinds o f excellence. How far different individuals will go, how much excellence they will actually achieve, depends, in part, upon their energy and aspirations. The kind o f excellence toward which they are headed is determined, according to type theory, by the inborn preferences that direct them at each fork o f the road. As Table 4, Comparison of Temperaments: PMC / US Surveyed Population indicates, PMC students are typed as 51.% being SJ, 33.4% - NT, 9.3% - SP and 6.1% - NF. The SJs are in such an overwhelming number that they tend to dominate the cultural norms. Myers-Briggs theory states that SJs tend to be realistic decision makers. SJs use judgement in their outer behavior, but their inner world focuses on facts o f immediate experience. SJs seek order in their environment. They are organized, dependable, and conservative. They tend to solve problems by reliance on past experiences, and they dislike ambiguity. As noted on Table 4, 30.0% or by statistical inference more than 1/4 of the total Acquisition Corps are ISTJ. The type table description state: ISTJ - Introverted Sensing with Thinking People with ISTJ preferences are extremely dependable and have a complete, realistic, and practical respect for facts. They absorb, remember, and use any number o f facts and are careful about their accuracy. When they see that something needs to be done, they 145 accept the responsibility, often beyond the call o f duty. They like everything stated clearly. Their private reactions, which seldom show in their faces, are often vivid and intense. Even when dealing with a crisis they look calm and composed. Not until you know them very well do you discover that behind their outer calm they are viewing the situation from an intensely individual angle. When ISTJs are "on duty" and dealing with the world, however, their behavior is sound and sensible. ISTJs often choose careers where their talents for organization and accuracy are rewarded. . . . They often move into supervisory and management work. If they are in charge o f something, their practical judgement and valuing o f procedure makes them consistent and conservative, assembling the necessary facts to support their evaluations and decisions. They look for solutions to present problems in the success o f the past. With time they become masters o f even the smallest elements o f their work, but don't give themselves any special credit for this knowledge. They may encounter problems if they expect everyone to be as logical and analytical as they are. They then run into danger o f inappropriately passing judgement on others or overriding less forceful people. A useful rule is for them to use their thinking to make decisions about inanimate objects or their own behavior, and use their perceptions to understand others. If they use their senses to see what really matters to others, so that it becomes a fact to be respected, they may go to generous lengths to help. Another problem may arise if the ISTJs thinking remains underdeveloped. They may retreat, becoming absorbed with their inner reactions to sense-impressions, with nothing o f value being produced. They may also tend to be somewhat suspicious o f imagination and intuition, and not take it seriously enough. The other SJ type comprising a significant percentage (17%) of the Acquisition Corps is the ESTJ. ESTJ - Extraverted Thinking with Sensing ESTJ people use their thinking to run as much o f the world as may be theirs to run. They like to organize projects and then act to get things done. Reliance on thinking makes them logical, analytical, objectively critical, and not likely to be convinced by anything but reasoning. They tend to focus on the job, not on the people behind the job. The like to organize facts, situations, and operations related to a project, and make systematic effort to reach their objectives on schedule. They have little patience with confusion or inefficiency, and can be tough when the situation calls for toughness. They think conduct should be ruled by logic, and govern their own behavior accordingly. They live by a definite set o f rules that embody their basic judgements about the world. Any change in their ways requires a deliberate change in their rules. They like jobs where the results o f their work are immediate, visible, and tangible. They have a natural bent for business, industry, production, and construction. They enjoy administration where they can set goals, make decisions, and give necessary orders. Getting things done is their strong suit. Like the other decisive types, ESTJs run the risk o f deciding too quickly before they have fully examined the situation. They need to stop and listen to the other's viewpoint, especially with people who are not in a position to talk back. This is seldom easy for them, but if they do not take time to understand, they may judge too quickly, without enough facts or enough regard for what other people think or feel. ESTJs may need to work at taking feeling values into account. They may rely so much on their logical approach that they overlook feeling values - what they care about and what other people care about. If feeling values are ignored too much, they may build up pressure and find expression in inappropriate ways. Although ESTJs are naturally good at seeing what is illogical and inconsistent, they 147 may need to develop the art o f appreciation. One positive way to exercise their feelings is to appreciate other people's merits and ideas. ESTJs who make it a rule to mention what they like, not merely what needs correcting, find the results worthwhile both in their work and in their private lives. The other major MBTI concentration in the Acquisition Corps are the NTs. The Acquisition Corps is comprised of 11.5% - INTJ, 9.8% - ENTJ, 6.8% - INTP, and .1% ISTP. Each type has its strengths and developmental needs. However, it can be stated there appears to be a natural tension between SJs and NTs which can have a deleterious effect upon an organization or program. Myers-Briggs theory states NTs tend to be logical and ingenious. NT people prefer intuition for purposes o f perception, but they prefer the objectivity o f thinking for the purpose o f judgement. They too focus on possibilities, theoretical relationships, and abstract patterns, but they judge these with an impersonal analysis. Often the possibility they pursue is a technical, scientific, theoretical, or executive one, with the human element subordinated. NTs tend to be logical and ingenious. They are best in solving problems within their field o f special interest, whether scientific research, mathematics, the more intricate aspects o f finance, or any sort o f development or pioneering in technical or administrative areas. The detailed type table descriptions INTJ, ENTJ, INTP and ENTP are: INTJ - Introverted Thinking with Intuition People with INTJ preferences are relentless innovators in thought as well as action. They trust their intuitive insights into the true relationships and meaning o f things, regardless o f established 148 authority or popularly accepted beliefs. Their faith in their inner vision can move mountains. Problems only stimulate them - the impossible takes a little longer, but not much. They are the most independent o f all types, sometimes to the point o f being stubborn. They place a high value on competence - their own and others. Being sure o f the worth o f their inspiration, INTJs want to see them worked out in practice, applied and accepted by the rest o f the world; they are willing to spend time and effort to that end. They have determination, perseverance, and will drive others almost as hard as they drive themselves. Although their preference is for intuition, they can, when necessary, focus on details o f a project to realize their vision. INTJs often value and use confidently their intuitive insights in fields such as science, engineering, invention, politics, or philosophy. The boldness o f their intuition may be o f immense value in any field, and should not be smothered in a routine job. Some problems may arise from INTJs single-minded concentration on goals. They may see the end so clearly that they fail to look for other things which might conflict with the goal. Therefore they need to actively seek the viewpoints o f others. INTJs may neglect their feeling values to the point o f ignoring other people's values and feelings. If they do, they may be surprised by the bitterness o f their opposition. An INTJ's own feeling values have to be reckoned with also, for if too much suppressed, they may build up pressure and find expression in inappropriate ways. Their feeling needs to be used constructively, such as through appreciation o f other people. Given their talent for analysis, appreciation may be hard for INTJs, but they will find it helpful on the job as well as in personal relationships. To be effective, INTJs must develop their thinking to supply needed judgement. If their judgement is underdeveloped, they will be unable to criticize their own inner vision, and will not listen to the opinions o f others. They will therefore be unable to shape their inspirations into effective actions. ENTJ - Extraverted Thinking with Intuition 149 ENTJ people use their thinking to run as much o f the world as may be theirs to run. They enjoy executive action and long-range planning. Reliance on thinking makes them logical, analytical, objectively critical, and not likely to be convinced by anything but reasoning. They tend to focus on the ideas, not the person behind the ideas. They like to think ahead, organize plans, situations, and operations related to a project, and make a systematic effort to reach their objective on schedule. They have little patience with confusion or inefficiency, and can be tough when the situation calls for toughness. They think conduct should be ruled by logic, and govern their own behavior accordingly. They live by a definite set o f rules that embody their basic judgem ents about the world. Any change in their ways requires a deliberate change in the rules. They are mainly interested in seeing the possibilities beyond what is present, obvious, or known. Intuition heightens their intellectual interest, curiosity for new ideas, tolerance for theory, and taste for complex problems. ENTJs are seldom content in their jobs that make no demand on their intuition. They are stimulated by problems and are often found in executive jobs where they can find and implement new solutions. Because their interest is in the big picture, they may overlook the importance o f certain details. Since ENTJs tend to team up with like-minded intuitives who may also under estimate the realities o f a solution, they usually need a person around with good common sense to bring up overlooked facts and take care o f important details. Like the other decisive types, ENTJs run the risk o f deciding too quickly before they have fully examined the situation. They need to stop and listen to the other person's viewpoint, especially with people who are not in a position to talk back. This is seldom easy for them, but if they do not take the time to understand, they may judge too quickly, without enough facts or enough regard for what other people think or feel. ENTJs may need to work at taking feeling values into account. Relying so much on their logical approach, they may overlook feeling values - what they care about and what other people care 150 about. If feeling values are ignored too much, they may build up pressure and find expression in inappropriate ways. Although ENTJs are naturally good at seeing what is illogical and inconsistent, they may need to develop the art o f appreciation. One positive way to exercise their feeling is through appreciation o f other people's merits and ideas. ENTJs who learn to make it a rule to mention what they like, not merely what needs correcting, find the results worthwhile both in their work and in their private lives. INTP - Introverted Thinking with Intuition People with INTP preferences use their thinking to find principles underlying whatever ideas come into their awareness. They rely o f thinking to develop these principles and to anticipate consequences. As a result, they are logical, analytical, and objectively critical. They are likely to focus more on ideas than the person behind the ideas. They organize ideas and knowledge rather than situations or people, unless they must for the sake o f their work. In the field o f ideas, they are intensely curious. Socially, they tend to have a small circle o f close friends, and like being with others who enjoy discussing ideas. They can become so absorbed with an idea that they can ignore or lose track o f external circumstances. INTPs are somewhat quiet and reserved, although they can be quite talkative on a subject to which they have given a lot o f thought. They are quite adaptable so long as their ruling principles are not violated, at which point they stop adapting. Their main interest lies in seeing possibilities beyond what is present, obvious, or known. They are quick to understand and their intuition heightens their insight, ingenuity, and intellectual curiosity. Depending on their interests, INTPs are good at pure science, research, mathematics, or engineering; they may become scholars, teachers, or abstract thinkers in fields such as economics, philosophy, or psychology. They are more interested in the challenge o f reaching solutions to problems than o f seeing the solutions put to, practical use. Unless INTPs develop their perception, they are in danger o f gaining too little knowledge and experience o f the world. Then their thinking is done in a vacuum and nothing will come o f their ideas. Lack o f contact with the external world may also lead to problems in 151 making themselves understood. They want to state the exact truth, but often make it so complicated that not everyone can follow them. If they can learn to simplify their arguments, their ideas will be more widely understood and accepted. INTPs may rely so much on logical thinking that they overlook what other people care about and what they themselves care about. They may decide that something is not important, just because it isn't logical to care about it. //IN T P s always let their logic suppress their feeling values, their feeling may build up pressure until it is expressed in inappropriate ways. Although they excel at analyzing what is wrong with and ides, it is harder for INTPs to express appreciation. But if they try, they will find it helpful on the job as well as in personal relationships. ENTP - Extraverted Intuition with Thinking People with ENTP preferences are ingenious innovators who always see new possibilities and new ways o f doing things. They have a lot o f imagination and initiative for starting projects and a lot of impulsive energy for carrying them out. They are sure o f the worth o f their inspirations and tireless with the problems involved. They are stimulated by difficulties and most ingenious in solving them. They enjoy feeling competent in a variety o f areas and value this in others as well. They are extremely perceptive about the attitudes o f other people, and can use this knowledge to win support for their projects. They aim to understand rather than judge people. Their energy comes form a succession o f new interests and their world is full o f possible projects. They may be interested in so many different things that they have difficulty focusing. Their thinking can be help them select projects by supplying some analysis and constructive criticism of their inspirations, and thus add depth to the insights supplied by their intuition. Their use o f thinking also makes ENTPs rather objective in their approach to current project and to the people in their lives. ENTPs are not likely to stay in any occupation that does not provide many new challenges. With talent, they can be inventors, scientists, 152 journalists, troubleshooters, marketers, promoters, computer analysts, or almost anything that it interests them to be. A difficulty for people with ENTP preferences is that they hate uninspired routine and find it remarkably hard to apply themselves to the sometimes necessary detail unconnected with ant major interest. Worse yet, they may get bored with their own projects as soon as the major problems have been solved or the initial challenge has been met. They need to learn to follow through, but are happiest and most effective in jobs that permit one project after another, with someone else taking over as soon as the situation is well in hand. Because ENTPs are always being drawn to the exciting challenges of new possibilities, it is essential that they develop their judgement. //th e ir judgement is undeveloped, they may commit themselves to ill-chosen projects, fail to finish anything, and squander their inspirations on incompleted tasks. The primary implication of the self-reporting instrument used to measure MBTI preferences is the importance of behavior patterns. The MBTI is not a "tesf'but rather an "indicator." Each type has its own strengths and developmental needs. The numerical scores do not express maturity or excellence, but they do indicate strength of preference of one element over the polar opposite. Type Development Jung and Isabel Myers expressed the belief that type development is a life-long process. Type development is the use of a well differentiated dominant process coupled with the full development of the auxiliary process. "It is the job of the auxiliary process to do the things that the dominate 153 process cannot and/or does not want to do." Isabel Myers goes on to clarify, "The auxiliary process supplies judgement if the dominant process is perceptive. . . . Finally, full type development involves learning to use one’ s two less-favored, less-developed processes when these process are needed." Whenever people have a problem, a decision to make, or a situation to deal with, they exercise the Z solution. Graph 1 Use your sensing (S) for facing the facts; being realistic; finding exactly what the situation is what you are doing, and what other people are doing. Use your intuition (N) to discover all the possibilities, that is, all the ways in which you might change a situation, your handling o f it, or other people's attitudes toward it. Use your thinking - judgement (T) in an impersonal analysis o f cause and effect. Include all the consequences o f alternate solutions, pleasant and unpleasant. 154 Use your feeling -judgement (F) to weigh how deeply you care about the things that will be gained or lost by alternative solutions. M ake a fresh appraisal; try not to let the temporary outweigh the permanent, how-ever agreeable or disagreeable the immediate prospect may be. It is Isabel Myers's conclusion that, by following this exercise, the solution reached will appeal to the favorite process. . . but the decision will be made on a sounder bias because you have considered facts, possibilities, consequences, and human values." Isabel Myers explains: What makes the hard steps hard is that they call for the strengths o f opposite types. When your problem is important, you may be wise to consult someone to whom these strengths come naturally. It is starting to see how differently a given situation can look to a person with the opposite type. Consulting with others o f opposite type will help you understand and use the neglected opposite side o f your self. Teaching Styles DeNovellis and Lawrence (1983) reported interviews with seventy-two voluntary teachers. Their observations allowed them to classify the teachers into representative descriptions as ST, SF, NF, and NT. For our purposes, the ST teacher incorporates the SJ and SP temperaments. The condensed results are as follows: I he role o f the teacher is to: ST: Set the example for students, be a role model, and share knowledge and experiences. N T : Encourage, inspire, help students develop as citizens and persons 155 Ideas for teaching come from: ST: State and local curriculum guides, textbooks, and experience N T : Concepts from subject area, knowledge o f students' needs and development, synthesis o f ideas from many sources. Teaching is planned by: ST: Making complete, detailed plans in advance for year and term with specific objectives N T : Making a plan according to an overall yearly structure; organizing by concepts or themes; determining details by student levels. Typical method o f teaching is described as: ST: Following daily routine, directing activities. N T : Having a flexible daily routine that depends on topics and student need, with interaction based on expectations for order and learning. Students' work are evaluated by: ST: Using points and percentages in a systematic way. NT: Using number as a factor The teacher feels successful if: ST: Students grades and behavior improve. N T : Students have increased involvement with learning. The importance of this last item cannot be understated. One's preference in learning becomes a preference in teaching. As seen in the various MBTI tables, the Acquisition Workforce is dominated by SJs. The Congressional mandate is to change the behavior of the Acquisition Workforce via education and experience. What is taught is as important as to how it is taught. Otto Kroger Associates conducts a MBTI qualifying workshop for those desiring to become qualified in the administration and facilitation of 156 MBTI. This five-day workshop is designed to instruct participants in the administration of the MBTI and to understand the basic tenets of its use. The primary goals of the course are based upon the student being responsible and accountable for his/her own learning. The teaching methodology is an appropriate mix of cognitive and affective techniques. Upon course completion and successful passage of the end-of-course certifying examination, the student will be competent, responsible, and an ethical user of the MBTI. The course opens with each student completing the MBTI instrument. Special emphasis is place on the mental frame of reference the student must place him/herself into in order to obtain a valid result. If the student should answer the instrument's questions from the frame of reference of what s/he believes their job requires them to answer, or that the job position requires a certain type of answer, the results will be invalid. The instrument must be taken "with your shoes off - relaxed - trusting in who you are - no pretence." Following the instrument is a self assessment test on MBTI concepts, for self validation on learning, to be accomplished during the course of instruction. 157 The course work conducts a step-by-step analysis and discussion of the eight preferences (E/I, S/N, T/F, J/P), 16 personality types, theory of functions (ST, SF, NF, NT), theory of temperament developed by David Keirsey and Marilyn Bates (NF, NT, SJ, SP), and the concept of "full development of type." The Keirsev-Bates Temperament Chart TYPES TEM P Q U EST STYLE ACHILLES HEEL ENF.UNF.I ENFPJN FP NF Identity Catalyst Guilt HNT.I, INT.! ENTPTNTP NT Competency Visionary Incompetence EST.I, ISTJ ESF.I, ISFJ SJ Belonging to a M eaningful Institution Stabilizer Traditional Disarray Disorganized ESTP, ISTP ESFP, ISFP SP Action Trouble Shooter Negotiator Routine Inactivity Chart 5 A comparison of this characteristic of this chart with Table 1, Chapter 1 will identify the environmental and cultural dominance of the SJ temperament (50.6%). As pointed out by Bryant (1991), "If the demographic changes which are predicted by Johnston and Packer (1987) occur, far greater emphasis will have to be placed on team building. . . . The question o f the resistance to change by this population is evidenced by the statements o f many PMC students. Recent PMC students have reported a great deal o f resistance to the implementation o f Total Quality 158 Management. . . .If this resistance is, in fact, a characteristic o f the acquisition environment, successful work groups may become more and more rare." Bryant's point is brought home by student comments to an elective, Dr. Deming's 14 Points in 1995. In a class of 36 students, six classified Deming's 14 Points as teaching "socialistic and communistic philosophies." The relative strengths and developmental needs (pitfalls) of each temperament are: t y p e STRENGTI IS PITFALLS NF People M otivators Empathic Aware o f Other's Feelings Persuaders Authority in Persons R escue Everybody Guilt Ridden A void Conflict Grudge Carriers Flounder without Heros NT Conceptualizers System s Planner Competent & Consistent Finn Minded & Fair Authority is in being Competent Mental Gymnasts M iss Immediate Com plex & Theoretical N o Sim ple A nswers Impersonal & A lo o f They define Competency SJ Administrators Precise Take Charge 1 lold Subordinates Accountable Authority is in the System Organization Nit Pickers Rigid D o the W rong Thing Only Critical o f Wrong "The boss made m e do it" See the N egative - not the Positive SP Problem Solvers Practical Immediate & Resourceful Quick Starters Authority in M oment Create Problem s when none to Solve I ,ovv Interest Beyond Practical Get Bored Easily Low Follow' Through Vague Idea o f Authority Chart 6 159 Other strengths and developmental needs (pitfalls) defined are those associated with Leadership, Teaching, and Learning Styles. To gain a better understanding of the numerical dominance of certain MBTI types in the Acquisition Workforce the Self-Selection Ratio (SSR) is a comparison type table that is: . . . based on the hypothesis that if type doesn't make a difference, there should be approximately the same proportion of type preferences represented in a specific group as in the total base group. It indicates whether people of different types tend to choose or tend to avoid certain situations as occupations. Finding the SSR involves comparing the percentage of a specific group with the percentage of the overall base. The formula for this calculation is: SSR = observed % frequency of specific group actual % frequency of total group Values above 1.00 show positive self-selection of that choice Values below 1.00 show some degree of avoidance of that choice. SSR ACQUISITIO M CORPS ISTJ ISFJ INFJ INTJ 6 .46 .8 5.75 ISTP ISFP INFP INTP 1.02 .14 . 7 3.4 ESTP ESFP ENFP ENTP .21 .04 .38 1.16 ESTJ ESFJ ENFJ ENTJ 1.2 .12 .2 1.96 Table 5 160 As illustrated by the SSR index, the Acquisition Corps is significantly over-represented by ISTJ, INTJ, INTP, and ENTJ types. While the ESTJ, ISTP, and ENTP types appear to be normal in their representation, the ESFP, ESFJ, ESTP, ISFP, ENFJ, ENFP, ISFJ, INFP, and INFJ are most significantly under represented. The observation made by Bryant in his 1991 report concerning the issue of diversity appears to be correct. Do those that do not fit the mold (SFs, sic) perceive themselves as being, "forced to operate in what could be called a 'siege mentality' and how does the Program Manager get this group (NFs, sic) to 'buy into' his or her program." Barr's and Barr's Leadership Development: Maturity and Power (1994) poses the natural tension between SJs and NTs as a more fundamental issue. It is these two temperaments that statistically account for more than 80 % of the personnel in a program office. Nominally, there are three SJs for every two NTs in most program offices. The Barr list includes numerous perceptive differences between SJs and NTs, such as: A Sensor hears what was said and treats words as facts; the Intuitor hears what was said and treats perceived intentions as facts. Sensors say, "Let me see it with my own eyes and verify it with my senses," Intuitors say, "I see it in my mind's eye." Sensors verify reality by sensorily / physically seeing, while Intuitors verify reality as soon as they have intuitive flash o f insight. 161 Barr and Barr posit, "... organizational culture is the way people behave - the norms, rules, traditions, tolerances, intolerance, customs, habits, and patterns of interaction." They go on to state, "Although most people tend to resist change, ISTJs have a particularly difficult time embracing change readily." As noted in Table 2, Comparison by Type: PMC US Surveyed Population, ISTJs comprise 29.8% of the Acquisition Workforce. It can be hypothesized that if the Intuitors (NTs) are the initiator of change and ISTJs (SJs) are culturally resistant to change, the natural tension in Acquisition Workforce Program Offices must contain a considerable managerial control dynamic. The stated purpose of the DAWIA is to change the culture of the Acquisition Workforce via education, training, and experience. Gordon Lawrence provides an insight to the educational approaches to be utilized in consideration of the different educational preferences of the 16 MBTI types in his book, People Types & T iger Stripes. Third Edition. As Lawrence describes it, the purpose of the book is to: . . . present a way to recognize the basic ingredients in behavior that make up a motivation pattern, and to then make an educated, shortcut prediction about what strategy will work b e s t. . . with the wide variety o f people we teach and supervise. 162 Lawrence recognizes there is no best type and that each individual's satisfaction is derived through preferred type strengths, recognizing that individuals of the same MBTI type are different. No two persons of the same MBTI are exactly alike, they are similar but different. The true discovery of an individual type comes through observation of self. Lawrence provides a check list to help people understand self-typing. He furnishes key words to orient reflective thinking and self observation. An understanding of nature is an essential element in validating your own type and in typewatching others. His greatest contribution is in his explanation of how various MBTI types learn, teach, and change. Lawrence holds that the instructors preferred way of learning is the most preferred way of teaching. Pointing out his own NT preference, Lawrence acknowledges that he had to learn quickly and dramatically that SJs had a difficult time with his NT classroom approach. However, recognizing that achievement comes through the use of strengths, he had to find an accommodating style that would address the learning needs and preferences of the students in his class. Where the NTs need a global or "big picture" of the contextual context, the STs need concrete experiences that are "linear" and well ordered. By re-examining the Z pattern for problem solving, Lawrence develops the differences between the SJs and the 163 NTs analysis of facts and their determination of position. Lawrence's position on the tension between NTs and SJs reemphasizes Barrs' postulate. The effectiveness and efficiency of a Program Office is dependent on the leadership of the Program Manager. Mary H. McCaulley, Center for the Application of Psychological Type, in her keynote address to, The Myers- Briggs Type Indicator and Leadership: An International Research Conference discussed the predilections of leaders who neglected to properly develop their attitudinal or functional preferences. In her address, McCaulley suggested that leaders who are ISTJ are likely to have subordinates who are similarly disposed. Otto Kroger echoes these sentiments in his lectures proposing that, "You clone your own." As for NTs who are also a minority but play a significant part in Acquisition Program Management, McCaulley stresses their leadership in breaking new ground and long-range planning. McCaully's predictions Ss vs. Ns. 164 Asset Predictors o f well developed functions: SENSORS INTUITORS Know what the facts are, respect them, use them expertly See the possibilities and implications o f subtle clues Create efficient systems and procedures for smooth operations Forecasts are long-range and strategic often non-linear patterns Make incremental changes that causes less disruption M ake changes by bold leaps into the future Communicate the how-to and why in language people understand Generate alternatives, new strategies Respect the lessons o f experience and act to conserve them Vision often seen globally, abstractly or metaphorically Table 6 Liability Predictors o f underdeveloped functions: SENSORS INTUITORS Focus on short-term may blind- side them to changes Change for change sake; risk for excitement o f challenge Discount recommendations that do not accord with experience If facts don't fit the theory, ignore the facts Too conservative and fail to take risks required by the situation May be unwilling to spell out vision in realistic steps for action May reject proposals because they don't fit operational standards May avoid the labor o f thorough evaluation o f their ideas Distrust proposals that aren't based upon hard data Give more weight to data that support their models Tab e 7 165 As is readily identifiable, the assets and labilities of the SJs and NTs are capable of counterbalancing each other. The concept of team work in an Acquisition Program Office is vital to its success. Sandra Hirsh defines her concept of MBTI "lenses" as a method for strengthening team development. The "lens" are: the 16 types, functional (ST, SF, NF, NT), quadrants (IS, ES, IN, EN), temperaments (SJ, SP, SJ, NF, NT) and dynamics (F, T, N, S). It is her contention the team operational profile can be forecasted and defined by a detailed examination of each lens as a function of the individual MBTIs of all team members reflective of their relative positions in the organization. Both the assets and liabilities or strengths and needs of the group can be defined by filling in the formatted charts of the five lenses. The key is the balance and understanding of MBTI by all team members. In the 1993 Special Topic Symposium on "Type and Leadership," Catherine Fitzgerald's presentation centered on Harold Grant's theory of type development. Grant's theory holds that the functions (sensing, intuition, thinking, and feeling) develop in certain age ranges and follow a specific sequence: 166 0 - 6 Undifferentiated 6 - 12 Development o f Dominant 12-20 Development o f Auxiliary 2 0 - 35 Development o f Tertiary 35 - 50 Development o f Inferior Table 8 Fitzgerald's experience is that prior to midlife (late thirties) the tertiary is rarely well-developed. In her observation of managers in their late thirties, "INTJs and ISTJs do not seem to have better-developed feeling judgement than ESTJs and ENTJs, even though feeling is the tertiary of the former and the inferior of the latter." Her observation has specific relevance considering the dominance of these four types in the Program Management career field and in specific the ISTJ. Her specific point is the strong possibility of career derailment precipitated by lack of full MBTI development. "I've worked so hard to create my life and myself; why all of a sudden is it not good enough?" In any Program Office, the concept of teamwork and team cohesion is pervasive. The goal of "completing the project" is a way of life. As defined at the outset, certain MBTI types are naturally attracted toward the Acquisition Workforce. This collection of individuals is significantly different from most norms. Jung (1971) warns: Any serious check to individuality is an artificial stunting . . .a social group consisting o f stunted individuals cannot be a healthy 167 and viable institution; only a society that can preserve the internal cohesion and collective values, while at the same time granting the individual the greatest freedom, has any prospect o f enduring vitality. 168 Section C: Perry Learning Environment Preferences fLEP) / Adult Learning Concepts Contained within each PMC student's preregistration package is a Perry Learning Environment Preference Instrument (Appendix 3). The form is completed by the student prior to attendance and forwarded to DSMC for computer evaluation. The Perry LEP instrument is the product of William S. Moore and was initially developed and validated in 1987 as a portion of his Doctoral Dissertation, "The Learning Environment Preferences: Establishing Preliminary Reliability and Validity for an Objective Measure of the Perry Scheme of Intellectual Development." In his dissertation, Moore examines the goal of being able to measure the intellectual development or cognitive complexity assumed to be the outcome of a college education by use of the Perry Scheme developed by William Perry at Harvard and Radcliffe in the late 1960's. Moore's contribution is a simple paper and pencil instrument replacing the significantly higher costs associated with the training of facilitators analyzing and interpreting students' video tape responses to the facilitators' questions. Moore's approach is an attempt to simulate the openness of Perry's original probes such as, "Tell me what stands out about your experiences of 169 the past year." Recognizing the considerable research work accomplished by Knefelkamp and Widick in their adaptation of the Perry Scheme, "Measure of Intellectual Development (MID)" (1975), Moore accomplished further research and adapted and modified their approach to yield a more reductionistic and simple instrument. The LEP admittedly capitalizes on the MID by incorporating five major content domains: • View of Knowledge and Course Content • Role of Instructor • Role of Student and Peers in Classroom • Classroom Atmosphere • Role of Evaluation The major scoring index, the Cognitive Complexity Index (CCI) incorporates all of the individual responses and is a global composite. The CCI is the composite indicator used by this study to accomplish comparison analysis. Perry's work is delineated from Piaget's by Merriam and Caffarella (1991), who first acknowledge their similarities, wherein cognitive development is hierarchical, sequential, invariant, and under stress individuals tend to retrogress, but then define Perry's equal emphasis on transitions of the stages themselves as being a major difference. 170 Perry is described by Rybash, Hoyer, and Roodin (1986) as being the first researcher in "relativistic thinking." In their concept, Perry is "postformal," as opposed to Piaget's "formal thinking." Perry himself, commented on the subject in 1986 by defining formal thinkers as being bounded by, "... uses of pure logic, rational analysis to provide the one correct solution to a problem, regardless of the domain within which the problem is embedded." In summary of Perry (1986), Labouvie-Vief (1984), and Rybash, Hoyer and Roodin (1986), offer their critique of formal operational thinking: (Take note of the similarity to the MBTI SJ approach) 1. Overemphasis on possibilities and abstractions. 2. Overemphasis on problems within the realm o f "physically reality". 3. Contemplation o f "closed system" problems with knowable variables and a specific outcome. 4. Over emphasis on "separating and analyzing" the relationships among variables within closed system. 5. Place greater emphasis on problem solving than on problem finding. 6. Over emphasis on understanding a single system and underempnasizes the relationship between multiple systems o f reference. 7. The inability o f formal operations (thinking, sic) to "know itself' as a system o f thought. 171 Pepper (1942), and Reese & Overton (1970) concurred in these views stating that formal operational thinking is, "Viewing reality as if it were a machine (which, sic) leads the formal thinker to conceptualize reality as a collection of stable, permanent, complex and knowable entities." As viewed by Rybash, Hoyer and Roodin (1986), fonnal thinkers believe, "Scientific research will uncover a set of universal, abstract, invariant, and absolute rules, laws, and principles that govern the behaviors of reality's elements and entities." Perry's "postformal" style of adult thinking was described by Kramer (1983), as having three basic characteristics: (Notice MBTI - NT similarity) 1. Possess an understanding o f relative, nonabsolute nature o f knowledge. Knowledge and reality are viewed as temporarily true (or real) rather than universally fixed 2. Accept contradiction as a basic aspect o f reality. 3. Possess an ability to synthesize contradictory thoughts, emotions, and experiences into more coherent, all-encompassing wholes. To these three characteristics, Rybash, Hoyer and Roodin (1986) would add a conceptualization that reality is an "open system" and "changing circumstances of life have no absolutes or universal principles that apply across all contexts and circumstances." 172 The postformalists that have conducted research in the area have concluded there are four aspects of post formal thinking: (a) the relativistic nature of knowledge and the limits of pure logic; (b) the development of metasystematic thinking; (c) a change in emphasis from problem solving to problem finding; and (d) the development of dialectic thinking. A large amount of their data hinges on Perry's seminal work. Perry's "Scheme o f Cognitive and Ethical Development." Position 1 Authorities know, and if we work hard, read every word, and learn Right Answers, all will be well. Transition But what about those Others I head about? And different opinions? And Uncertainties? Some o f our own Authorities disagree with each other or don't seem to know, and some give us problems instead o f answers. Position 2 True Authorities must be Right, the others are fraud. We remain Right. Others must be different and Wrong. Good Authorities give us problems so we can learn to find the Right Answers by our own independent thought. Transition But even Good Authorities admit they don't know all the Right Answers yet! Position 3 The some uncertainties and different opinions are real and legitimate temporarily, even for Authorities. They're working on them to get to the truth. Transition But there are so many things they don't know the Answers to! And they won't for a long time. 173 Position 4a Transition ( a n d /o r ) Position 4b Transitions Position 5 Transition Position 6 Transition Position 7 Transition Position 8 W here Authorities don't know the Right Answers, everyone has a right to his own opinion; no one is wrong! But some o f my friends ask me to support my opinions with facts and reasons In certain courses Authorities are not asking for the Right Answers; They want us to think about things in a certain way, supporting opinions with data. That's what they grade us on. But this "way" seems to work in most courses, and even outside them. Then all thinking must be like this, even for Them. Everything is relative but not equally valid. You have to understand how each context works. Theories are not Truth but metaphors to interpret data with. You have to think about your thinking. But if everything is relative, am I relative too? How can I know I am making the Right Choice? I see I am going to have to make my own decisions in an uncertain world with no one to tell me if I'm Right. I'm lost if I don't. When I decide on my career (or marriage or values) everything will straighten out. Well, I've made my first commitment. Why didn't that settle everything? I've made several commitments. I've got to balance them - how many, how deep? How certain, how tentative? 174 Transition Things are getting contradictory. I can't make logical sense out o f life's dilemmas. Position 9 This is how life will be. I must be wholehearted while tentative, fight for my values yet respect others, believe my deepest values right yet be ready to learn. I see that I shall be retracing this whole journey over and over - but, I hope, more wisely. Schemes, as opposed to theories, are described by Glaser (1985, 1987) as: "... represent knowledge that we experience, such as interrelationships between objects, situations, and events that occur. Schemata are prototypes in memory of frequently experienced situations that individuals use to integrate and interpret instances of related knowledge." The cognitive growth of individuals according to Perry's Scheme is summarized and described for positions 1 - 5: Position 1 Basic Duality Position 2 Multiplicity Prelegitimate Position 3 Multiplicity Legitimate but Subordinate Position 4a Multiplicity (Diversity and Uncertain) Coordinate Position 4b Relativism Subordinate Position 5 Relativism Moore's LEP measures Positions 2 through 5 inclusive. Position 1 is ignored as being too hypothetical for college graduates. Positions 6 through 9 175 bear on the ethical side of development and are more discemable through qualitative interviews. In a similar manner, position 4a (Multiplicity), 4b (Relativism Subordinate), and position 2's and 3's, escape or denial o f growth via encapsulation or dissociation, retreat via negativism, dogmatism, or reactionism are indeterminable from the LEP's indicator table. These subsets are only discemable through qualitative interviews. To distinguish between the Perry whole number levels and in recognition of the importance of the transition, Moore's CCI numerical structure is based on the hundreds scale as follows: Perry Level Moore's Nomenclature CCI Index 2 Basic Dualism 200 - 240 transition 241 -285 3 Multiple Perspective 286 - 300 transition New Truth Dualism 301 -327 transition 328 - 373 4 Multiple Context 374 - 400 transition Truth in Different Contexts 401 -416 transition Multiple Commitments understanding 417-461 5 Search for synthesis 461 -500 Table 9 176 Moore's inability to distinguish the psychological subsets within Perry's Scheme does not render Moore's data useless. Rather it allows the researcher to better understand the learner's style or preferences. According to Smith (1982), for learning to take place three elements are involved: "learner’ s needs, learner's learning style, and training or curriculum." From a DoD perspective, the learner's needs are defined as those Level III competencies to be mastered as a Program Manager. The DoD training / curricula is the technique / instructional method used to impart "competency knowledge." The learner's learning style and needs can be ascertained in a number of ways; however, the three instruments available at DSMC are the MBTI, Perry LEP and PROFILOR. The individual data on each student is "Personal Data" falling under the Privacy Act of 1974 (PL 93-579) and is protected by the privacy act. Specific PMC Class or Section composites are not classified, because group or composite data is not identifiable to a specific individual. MBTI and Perry LEP composites have been furnished to the instructor teams since Claxon and Murrell (1988), under contract with the Office of Educational Research and Improvement (ED), reported: M ore effective learning can be promoted through (1) professional development activities on the application o f the concept o f learning styles, (2) encouragement o f classroom research, especially using the 177 concept o f learning styles, (3) developing curricular experiences that help students learn to learn, and (4) in hiring faculty, considering candidate' understanding o f teaching-learning practices that take into account individual differences. As an aid to this effort, the results of Culver's and Fitch's (1988) workshop, "Rational Curriculum Design", which translated each Perry level 2 through 5, into learner characteristics is also furnished to each instructor. The domains of course structure, written reports, technical projects, homework assignments, oral presentations, lecture, discussions, and examinations provide the instructor with information to match the classroom's level of intellectual maturity with the appropriate level of challenge. Course Instructions Perry Level 2 Highly structured schedule, giving dates, lectures, readings and homework assignments, examinations, projects, detailed grading policy, single text with supporting notes. 3 Similar to 200, but with extra care in design o f grading scheme, faster pace, multiple reading sources, group assignments OK. 4 Schedule should be more open, more variety in tasks, multiple texts, group projects OK. 5 Structured schedule OK but probably not needed to be used as a guide in priortizing time, text supplemented with technical reports, exams as appropriate. 178 W ritten Reports 2 Can write a report which is descriptive, beginning to compare alternative concepts, but only where the validity o f both is clearly established. 3 Can write about procedures as well as concrete subjects, use supportive evidence; will seek model for appropriate writing style from instructor, "How many pages?" 4 Can write critical analysis o f subject or procedure, using positive and negative can describe the abstract concepts and evaluate them, using supporting evidence. 5 Can describe and evaluate abstract concept in context or related issues using multiple informational sources; can set boundaries on scope o f report for self. Technical Projects 2 Can handle simple, linear project, with a single solution; wants instructor to define what is to be found and how to find it. 3 Can handle a project with multiple variables; can select best solution on basis o f quality; can work with peers in group, but has difficulty evaluating them. 4 Can handle complex open-ended project, doing both qualitative and quantitative evaluation o f alternate solutions, will work as a member o f team, but may not listen to others. 5 Can deal effectively with projects which involve non technical implications, evaluating alternative solutions in context. Looks at possible solutions from different view may use multiple sources o f expertise. Homework Assignments 2 Comfortable with problems which require routine solutions, procedures; wants answers available problems similar to lecture example, closed form. 179 3 Wants clearly defined homework assignments, with credit for number worked; likes problems which develop procedures, derivations, iterative solutions. 4 Likes problems that require judgement, synthesis of multiple concepts; open-ended problems particularly if they bring in new ideas, concepts. 5 Will attempt complex, open-ended problems requiring evaluations o f alternatives; interest goes beyond traditional theories, applications, to implications o f subject; seeks problems which put subject in context. Oral Presentations 2 Limited to short reports, answers to previously worked problems 3 Formal presentation o f solutions, report on projects, answer class questions, means o f grading must be clearly defined. 4 Critical evaluation o f alternatives if there is time for preparation, will enjoy debates, particularly with instructor. 5 Capable o f open discussion on complex issues in informal and formal situations, can provide balanced critical evaluations using qualitative judgement. Lecture 2 Clearly expository lecture, definitions, formulas, applications, no derivations; primary form o f instruction lots o f illustrations , tie lecture closely to text. 3 Clear expository lecture, derivations, analytical procedures, application type problems for practice, historical perspective valid. 4 Varied lecture format, informal probably best, historical development o f interest, challenge students through lecture, derivations, puzzles, paradoxes-inaccuracy in various mathematical mode use alternatives, new developments in subject. 180 5 informal lecture format with student participation used sparingly for introduction o f new material, related concepts to others previously discussed, demonstrate complex analytical procedures. Discussions 2 Used in small groups for tutorials, instructor leads, taking with each student in turn, supplementary to lecture; help problem areas. 3 Occasionally in Lecture to solicit feedback, connect with class; tutorials in which all students can contribute but instructor leads. 4 Major source o f instruction, with instructor modeling 5 Primary mode o f instruction with students sharing leadership with instructors; ideas, models openly proposed challenged by all. Examinations 2 Can handle questions which ask for definitions, following standard procedures, simple analysis, must be right answers, need sample exam problem, no essays. 3 Procedures derivations, instructions, descriptive essays, OK; evaluation is very important, so test be carefully designed to be fair to all. 4 Complex derivations, critical evaluation o f alternative solutions; enjoys non-traditional evaluation tasks, particularly if they pit student against instructor. 5 Enjoys challenge o f open ended problems with alternative answers, requiring judgement; expects and desires critical evaluation. This information coupled with the Comfeld and Knefelkamp (1979) "Learning Characteristics Implied by the Perry Scheme" allow instructors to challenge and support the students in the class. 181 CHART 7: LEARNER CHARACTERISTICS IMPLIED BY THE PERRY SCHEME 1 PERRY PREFERENCE #2 | PERRY PREFERENCE §3 | PERRY PREFERENCE 94 | PERRY PREFERENCE 94 | VIEW OF KNOWLEDGE A U KNOWLEDGE IS KNOWN There is a certainty that RIGHT and WRONG ANSWERS exist for everything. Knowledge it collection of information. MOST KNOW LEDGE B K N OW N. ALL KNOW LEDGE tt KNOW ABLE (first view of laaminoea a process that students can learn). Certainty that there exists a RIGHT WAY to find Right Answers. Realize that some areas of knmvtedge are “fuzzy" W HILE M SOME AREAS WE H A V E CERTAINTY ABOUT KNOW LEDGE, M M OST W E DON'T KNOW FOR SURE. Certainty that there is NO CERTAINTY. Hence, "do your own thing"-one opinion's as vaM as an others. ALL KNOW LEDGE IS CON TEX TU A L. Al Knowledge is disconnected from'Absolute Truth.” Right 4 Wrong adequate 4 inadequate, can exist within a specific context and are judged by 'rules of adequacy.” VIEW OF THE ROLE OF THE INSTRUCTOR Source of Knowledge. Rote is to give the knowledge to student. Good instructor equals Absolute Authority and Knowerof Truth. Source of Right Way to find knowledge, of how to learn. Role is to model "the way” or process. Source of the Process of Thinking-modeling the use of supportive evidence-modeling "the way they want us to think.” Instructor can also be completely discounted. Source of Expertise, role of expert/gukJe/consuftant within the framework of "rules of adequacy" and within context. One earns authority through havmq expertise. I VIEW OF THE | ROLE OF 1 t h e H STUDENT Rote is to receive the infor mation or knowledge and to demonstrate having learned the riqht answers. Role is to learn how to learn, how to do the processes called for, to apply oneself, and to work hard. Role is to learn to think for oneself and to use supportive evidence. Independence of thouqht is valued. Role Is to use intellect, to shift from context to context, and to apply rules of adequacy to information. I VIEW OF 1 PEERS 1 PLANNING | PROCESS Peers are not a legitimate source of knowledge or teaming. Peers are now more legitimate. Value views of peers, but still see the Instructor as the Final Authority. Peers opinions are legitimate, but may not be listened to. One's opinion is just as good/bad as everyone etse'e. Peers are legitimate sources of learning if they use contextual presentation of perspectives. EVALUATION ISSUES Evaluation directly related to sense of self. Bad/wrong •nswer« bad/wrong person. Real concern if teacher, content, and/or evaluation format is fuzzy. Evaluation is THE PRIMARY ISSUE Often related to amount of time, work, "style", and QUANTITY focus. FAIRNESS is a major issue...in judging, assigrwnents, amount of work. "New Truth"-lndependent thought should get good grades. Are learning to accept qualitative criteria as legitimate in evaluation. Evaluation of work can be separated from evaluation of self. Understand that a good critique has positives 4 negatives. See evaluation as legitimate process/part of leamina. PRIMARY INTELLEC TUAL TASKS Learning basic information definitions of words, concepts, and to identify parts of the whole. Beginning to compare and contrast things. Can do compare & contrast tasks. Can see multiples- perspectives, parts, opinions, evaluations, difference between process and content. Use supportive evidence. Good at analysis. Can do some synthesis. Critiques with positives & negatives. Use supportive evidence wed. Relates learning to other issues, team s to think in abstractions. Relate learning in one context to another. Look for relationships. See complexity. Can evaluate, conclude, support mm analysts. Can synthesize. Can adapt, modify, and expand concepts. SOURCES OF CHALLENGE Ambiguity, multiple perspectives, uncertainty- especially by an Authority: any disagreement between two respected Authorities; concept of independent thouoht. Complexity—initially seen as quantity, not qualrty. Evaluation causes great concern, teaming processes as opposed to facts. Trying to determine which of the multi Dies is realty rioht. Demand to use evidence to support opinion. Accepting responsibility in learning. For some, learning to listen to authority sgain. For others, learning to think independently. Requirement of choice or commitment. How to choose between equally good alternatives. Highly challenged to intellectual excefience. SOURCES OF SUPPORT High degree of structure. Concrete examples. Carefti sequencing, timing of cfiversty. Environment where people are treated kindly. Modeling on pari of instructor. Chance to practice skOs. Clarity of evaluation procedures 4 assignment instructions. Peers are big source of support. Comfort that we know the right process and that the right answer is out there. Enjoy diversity. Tendtobstkat structure...seek independence. Comfort with different formats, aRhough may dearly prefer one. Can play the intellectual "game" fairly wefl. Feel comfortable moving across contexts-have the Intellectual tools to do so. Feeling of MeRectual mastery. Comfortable seeking aid of appropriate authority/exper 182 Matching Teaching & Learning Styles Claxton's and Murrell's (1987), study for the US Department of Education holds that, "Knowledge of learning style can thus help the faculty design experiences appropriate for students in terms of matching or mismatching and enable them to do so thoughtfully and systematically." Merriman and Caffarella suggest that"... cognitive-style instruments have proved useful in helping both learners and instructors alike become aware of their personal learning styles and their strengths and weaknesses as learners and teachers." If then, the statistical data for each PMC Student Section listing the Perry LEP and MBTI distributions are given to the instructor teams for that section, it allows the instructors to have an understanding of the overall cognitive level of the class and provides a base for curricula adaptation. This application of the learning preference data complies with Hiemstra and Sisco (1990), who conclude,". . . learning style instruments are best used as tools to create awareness that learners differ and as starting points for individual learners' continued investigation of themselves as learners." However, once this understanding becomes the certain knowledge of the instructor, McCarthy (1980), Kolb(1984), and Claxton and Murrell (1988), amongst 183 many agree the instructor must make some fonn of individual accommodations for the individual variances. Theories of Learning Academic case problems tend to be highly definitive and structured wherein real world issues are normally full of ambiguity, ill defined and very contextualized. Culver (1984) reports his first year of teaching was to act as a mentor and facilitator for graduate engineering students, who as undergraduates had not learned how to . . handle open-ended problems, organize an unstructured body of data, or assume the risk of dealing with a problem for which there was no answer in the back of the book." It then must follow if the Congressional mandate is to be observed and the Acquisition Workforce's behavior is to be changed through education and training, an educational method which imparts learning is required. Merriam and Caffarella cite four key adult learning theories; Behaviorist, Cognitive, Humanist, and Social. Behaviorists The behaviorist or stimulus - response is dominated by Edward L. Thorndike who discovered connections between sensory impression (stimuli) 184 and subsequent behavior (response). These observations allowed Thorndike to formulate his "three laws of learning." Law o f Effect: Learners will acquire and remember responses that lead to satisfying aftermaths. Law o f Exercise: repetition o f a meaningful connection results in substantial learning. Law o f Readiness: If the learner is ready for the connection, learning is enhanced; if not, learning is inhibited. The behaviorist theory assumes certain control over the "process of learning." Observable behavior rather than internal thought processes is the focus o f the study. Environment shapes one's behavior; what one learns is determined by the elements in the environment. Principles o f contiguity (how close in time two events must be for a bond to be formed) and reinforcement ( any means o f increasing the likelihood that an event will be repeated) are central to explaining the learning process. Notables in the field of behaviorist learning theory are Pavlov (conditioned response) and Skinner (operant conditioning). Pavlov and Skinner both utilized reward systems as a method for achieving learned behavior by arranging reinforcements in the learners environment. Skinner believed, "The teacher’ s role is to design an environment that elicits desired behavior toward meeting goals (survival of the human species, societies, and individuals - sic) and to extinguish behavior that is not desirable." Merriam 185 and Caffarella believe that several educational practices are traceable to this view: The systematic design o f instruction, notions o f the instructor's accountability, programmed instruction, computer-assisted instruction, competency-based education and so on are solidly grounded in behavioral learning theory. It must be noted here that considerable amount of military training is predicated on stimulus - response e.g.. the reactive /defensive mode military combat strategies are dominated by attempts to achieve total control of the environment. The course structure of the PMC is competency based which is also a stimulus - response strategy. Cognitive The behaviorist school of learning was strongly criticized by Bode. He suggests looking at total patterns rather than single isolated events. The Gestalt view expressed that perception, insight and meaning are the key ingredients to the learning process. The dividing issue is the exercise of control. For the behaviorist, control resided in the environment / teacher; while those of cognitive orientation believed control resided within the student. Subsequently, Piaget (1966) developed a modified theory defining internal cognitive structures changed as a result of maturation, interaction 186 with the environment, and iife experiences. Other contributors to the Cognitive learning theory are: 1. Asubel's (1967) rote vs. meaningful learning wherein rote fails to become linked to cognitive structure and is easily forgotten. 2. Gagne's and Briggs' (1979) eight types of knowledge. 3. Bruner's principle of discovery. 4. Lewin's sensitivity and field theory. Humanist The chief proponents of a "person being all s/he can be" are Maslow and Rogers. Maslow's "Need Hierarchy" lists self-actualization as being the principal of his needs structure. His construct defines learning as being intrinsically driven to fill the need to know and to obtain growth, achievement and advancement. (Szilagyi & Wallace, 1987) Rogers' "rational- emotive" approach teaches that the student's ineffective beliefs or thought patterns can be corrected by a facilitator aiding the student to realize the nature of personal thought processes and thereby enable the student to correct faulty logic (Ivey, 1980). According to Knowles (1990), who incorporates much of Maslow's and Rogers' rational into his 187 andragogy theory, "Rogers' student centered approach to education is based on five basic hypotheses." We cannot teach another person directly: we can only facilitate his learning. A person learns significantly only those things which he perceives as being involved in the maintenance of, or enhancement of, the structure itself. Experience which, if assimilated, would involve a change in the organization o f self tends to be resisted through denial or distortion o f symbolization. The structure and organization o f self appear to become more rigid under threat; to relax its boundaries when completely free from threat. Experience which is perceived as inconsistent with the self can only be assimilated if the current organization o f self is relaxed and expanded to include it. The educational situation which most effectively promotes significant learning is one in which (a) threat to the self o f the learner is reduced to a minimum, and (b) differentiated perception o f the field is facilitated. Social Learning The theory is based upon a concept that people learn from observing others. The theory is similar to the leadership theory expressed by Stogill in the 40s and 50s. To become a leader, the 50s' concept was to observe the "natural bom" leaders and mimic their traits. The primary problem was that there was no outside observer evaluating the competency of the mimicry. Bandura (1976) contended that mere observation does not cause a lesson to 188 be learned. He further expressed the consideration that learning was environmentally influenced. Rotter further developed Bandura's theory, defining the interaction of the student and the environment as being a consideration of what he termed "locus of control." Success or failure is attributed to either external or internal causes. Merriam and Caffarella summary table of key learning theories: Theory Behaviorist Cognitivist Humanist Social V iew o f learning Change behavior Internal mental process Fulfill potential Observation o f others I .oeus o f learning External environment Internal cognitive structure Affective and cognitive needs Interaction person, environment and behavior Purpose o f Education Produce behavioral change D evelop capacity and skill Self- actualization autonomous M odel new roles and behavior Teacher's role Arrange environment Structure content D evelop w hole person M odels new behavior Manifest in adult learning Com petency based - skill developm ent Cognitive developm ent Leaning how to learn Andragogy' Self-directed learning Socialization Locus o f control Social roles Table 10 Current Theory and Practices In 1984, the Army Research Institute requested the National Academy of Sciences to examine the potential value of certain techniques that had been proposed to enhance human performance. In the light of ever pressing claims 189 of non-traditional methods of producing enhanced performance, coupled with Anderson's and Van Atta's accusation in the Washington Post (July 17,1985) that the U.S. Military was losing the "mind race" to the Soviets, the Academy was pressed into service to validate and authenticate the claims associated with accelerated learning, sleep learning, guided imagery, split- brain effects, stress management, biofeedback, influence strategies, group cohesion, and parapsychology. The Academy's Enhancing Human Performance: Issues. Theories and Techniques, a voluminous report, detailed the examination of the Army's expressed "... interest in techniques that help people acquire, maintain, or improve such skills as classroom learning, communication and influence, creativity, and accuracy in execution of tasks requiring motor skills." However, as an "enhanced education and training by-product" the Army hoped to ". . . attain certain ideals, such as fearlessness, cunning, courage, one-shot effectiveness, fatigue reversal, and nighttime fighting capabilities." In the findings and conclusions, the Academy found possibilities in sleep learning, stress reduction, organizational cultural, and integration of mental and physical instructional elements. On the negative side, no evidence was found to support visual enhancement training, split-brain or neurolinguistic programming, parapsychology, and accelerated 190 learning packages. O f greater significance was the potential of negative effects resultant from group cohesion. These negative consequences were defined as; ineffective handling of deviance, groupthink, negative norms, and intergroup conflict. While the Army, in its effort to respond to the media assertions and to find increased effectiveness as a moral imperative, "if we are to commit our soldiers to fight outnumbered and win", searched for new, unexplored methods of educating and training its adult population. Others in academia performed experiments to find a better way. English Adult Learning Approach At the University of Salford, England, the concept of adult learning theory merged with management development theories. Reg Revens chronicled his experiments in England and Belgium in Developing Effective Managers (1971). Working with great Britain's National Coal Board, Revens experimented with techniques to anticipate and solve real problems in real companies in real time. His key questions establishing the base to what he termed as "action learning" are: 1. What am I (or what is my firm) really trying to do, first and last? 2. What is stopping me (or my firm) from doing it? 191 3. What can I (or my firm) contrive to do about it? 4. Who knows what the line o f action that we are trying to implement is? 5. Who cares about getting this line o f action really implemented? 6. Who can actually contribute anything towards getting it implemented? Matching the "right fellows" into a "set" (team, sic), Revens acts not as an expert, but rather as a facilitator evaluating and providing feedback to this group of self learners. The "set's" (teams, sic) goal is to plan and implement a solution to a real time problem with higher management's support. Dr. David Botham (1994) reports that the Doctoral Degree currently awarded by Salford is based upon a "real world" curricula structure. Additionally, each student is required to authenticate their own new learning in their dissertation. The Doctorate will not be awarded unless the applicant is able to authenticate what and how new learning occurred. Androgogy Malcom Knowles' Andragogical Theory (1973) is based upon several assumptions derived from the pedagogical model: 1. The need to know. Adults need to know why they need to learn something before undertaking to learn it. 2. The learner's self-concept. Adults have a self-concept o f being responsible for their own decisions, for their own lives. 192 3. The role o f the learner's experience. Adults come into an educational activity with both a great volume and different quality o f experience from youth. 4. Readiness to learn. Adults become ready to learn those things they need to know and be able to do in order to cope effectively with real-life situations. 5. Orientation to learning. In contrast to children's and youth's subject-centered orientation to learning, adults are life-centered (or task-centered or problem-centered) in their orientation to learning. 6. Motivation. While adults are responsive to some external motivators, the most potent motivators are internal pressures. Allen Tough (1971) preceded Knowles with his discovery that adults are intrinsically motivated and highly self-directed when they go about learning something for themselves. It is his belief that in a rapidly changing world, learning projects are necessary to help the individual adjust to changes that affect him/her on the job and elsewhere. The learning project produces a episode which implicitly contains the intent is to gain knowledge and skill. Tough's definition of knowledge and skill is: . . includes any positive or desired changes or improvement in a person's knowledge, understanding, awareness, comprehension, beliefs, ability to apply, ability to analyze and synthesize, ability to evaluate, judgement, perceptual; skills, physical skills, competence or performance, response tendencies, habits, attitudes, emotional reactions, recall, sensitivity, insight, confidence, patience, and self- control, and/or some other personality characteristic, inner behavior or overt behavior. 193 This rather all inclusive list seems to include all those attributes Congress defined as being the required Acquisition Corps cultural change. To achieve the goal of a learning episode that would achieve the results of knowledge and skill, he established the conceptual base for Knowles' Theory of Learning Contracts. The purpose of the learning contract is to obtain a negotiated balance between the learner and those stakeholders influencing what is to be learned. More often than not, the administrative overburden of a structured curricula defining the Corporate competency matrix fails to keep pace in a dynamic rapidly changing environment. The construction of the Knowles' Learning Contract (1973) is a student eight step process: Step 1. Define the gap between where you are now and the competencies required to excellently perform your assigned tasks. Step 2. Each learning need specified in Step 1 should be translated into a learning objective. Step 3. Describe how you propose to accomplish each learning objective identifying resources to be used and the strategy employed. Step 4. A description o f the evidence to be acquired to indicate the degree to which you have achieved each objective. S tep s. Specify how the evidence collected in Step 4 is to be validated. A salient feature in self-directed learning is the wisdom displayed in the selection o f validators and validation techniques. Step 6. Review the contract with facilitator. Gain feedback as to the clarity and adequacy o f your self-analysis. 194 Step 7. Carry out the contract. Do it. Step 8. After completion ask the question, "Did I accomplish my intent and learned what I ought to have learned?" Consult your evidence and validation data to determine their adequacy. Knowles' Andragogy theory has been the center of controversy sense its inception. Knowles' 1984 redefinition that pedagogy-andragogy represents a continuum dependent on time and situation, causes Merriam and Caffarella (1991) to qualify it as a technique and not a theory. Brookfield (1986) defines andragogy as being extravagant: The drive toward self-directedness has been too readily accepted as an innate characteristic o f adulthood. Research on the characteristics o f successful self-directed learners has shown that some important adult learning is most emphatically not problem centered; rather it is often a free- flowing exploration o f an area o f knowledge undertaken for the innate fascination o f that activity. The idea that adults always seek immediate application o f learning within competency development categories is also doubtful. Pratt's (1984) view was also critical but from a different context. Acknowledging that learning objectives are usually pre-planned, his criticism centered on Knowles' institutional control in the planning, execution, and evaluation of learning activities. The significant question is the differences between the learners' own needs and the institutional-set objectives. 195 This question, institutional control, was also introduced by Mocker and Spear (1982), who identify four categories of life long learning. LEARNER CONTROL OF MEANS INFORMAL FORMAL SELF DIRECTED NONFORMAL INSTITUTION CONTROL OF OBJECTIVES Chart 8 When the institution is in complete control of both learning objectives and means, the student has no options. If however, the institution empowers the student to be self-directed, the institution must relinquish control of both learning objectives and means. A corresponding structure was offered by Bockett and Hiemstra (1991), by their "Personal Responsibility Orientation" (PRO) model. In this environmentally controlled model, the individuals "personal responsibility" has two paths. The first path tempered by the characteristics of the teaching- learning transaction develops the "self-directed learning" The second path 196 tempered by the characteristics of the learner develops the "learner self- direction. These two developments, under the egress of the environmental social context, merge into the "self-direction in learning." The ’’ Persona! Responsibility Orientation” (PRO) Model CARACTERISTICS OF THE TEACHING-LEARNING TRANSACTION CHARACTERISTICS OF THE LEARNER LEARNER SELF-DIRECTION SELF-DIRECTED LEARNING SELF-DIRECITON IN LEARNING JACTORS WITHIN THE SOCL -CONTEXT. Chart 9 197 Learning-T o-Learn The dichotomy between learner control vs. teacher direction has been illustrated by Candy (1988) and reprinted in Huey Long and Associates, Advances in Research and Practice in Self-Directed Learning (1990). Adapting Schmidt's and Tannenbaum's continuum, Candy illustrates that, as teacher direction diminishes, the learner assumes more and more control. Hie Relationship Between Autodidaxy and Learner - Control of Instruction Autodidactic Domain c o N T R 0 L Teacher Direction Autodidaxy Instructional Domain Chart 10 198 Introducing the thought of autodidaxy he further modifies the learner / teacher continuum by adding another layer of assisted autodidaxy / autodidaxy to the continuum. Citing Perry (1970) and Brookfield (1881), Candy acknowledges that as learners become more mature they tend to see knowledge differently. However it is his belief that epistemological sophistication in one area or domain does not constitute a corporate view of knowledge. Although he defines his thoughts as research recommendations, they could just as well be described as teacher guidelines. 1. Researchers should examine how learners construe different subject areas which they are attempting to learn. 2. Researchers should investigate with learners their orientation to new learning tasks, their initial choice o f a surface or deep approach and the criteria used in arriving at the choice; their personal definitions of whether learning is essentially reproductive or a transformational process; and how their views change as they engage further in the learning process. 3. Researchers could inquire into the changing nature o f the learning tasks as learners engage more fully, and enter more deeply into the material, and into the changing nature o f their motivation towards learning endeavors, both in instructional and autodidactic situations. 4. Researchers should use devices such as cognitive maps, repertory grids, learning journals and the like, to find out what learners already know about existing domains o f knowledge, and how the attempt to reconcile new learning with existing views o f knowledge. 5. Instead o f striving only for general laws concerning assistance with learning projects, researchers could profitably direct their efforts towards exploring with learners; how the construe various forms o f assistance; what criteria the use to decide between competing alternatives when attainment o f goals requires some 199 relative loss o f autonomy; what criteria they use to distinguish direction from assistance; and what renders one helper acceptable, while another is seen as threatening or inadequate. 6. Researchers should attempt to explore with learners how they view various learning resources, and in what way(s) some resources are seen as more helpful or more appropriate than others. 7. Researchers should examine learner's concepts o f themselves as learners. 8. Researchers should undertake long term studies o f individual learners with a view to discovering the components o f their continuing search for personal autonomy. 9. Researchers should study how teachers construe learner autonomy. 10. Researchers should examine issues such as how the helpers view being approached for assistance, how the construe the learner's need for help, the sort o f strategies they employ in trying to meet the learner's needs, and how they sense the learner is nearing independence and hence no longer needs their help. Cavaliere's discussion, "The Wright Brothers as Self-Directed Learners: The Role and Revelation of Goal Setting, Feedback and Motivation During the Process of their Self-Directed Learning Project," contained in Long and Associates text (1990) is an example of significant accomplishment in the autodidactic domain. This discussion is of special interest in consideration of the state of the art technical advancement undertaken by the Acquisition Workforce in all developmental contracts. Her article, "The behavioral approach exhibited by the Wright Brothers," brings 200 full definition to Acquisition Program Management tasked to produce a new weapon-system technology. The Wright Brothers, working as a team, were compatible in purpose, thought and action. They worked on this self-directed learning project as if they were a single entity; their individual strengths complemented and enhanced each other's efforts. The behaviors exhibited by the Wrights in their learning project were repetitive forming series, groups and stages o f action. These learning behaviors were non-linear, demonstrating increased proficiency as the learners moved through their project. The productive outcomes o f their collaborative teamwork and synergy were driven by the cyclical forces o f goal setting, feedback and motivation. Their success founded a new era of transportation and military warfare. Their accomplishment is remarkable in the absence of a significant emotional event or triggering phenomenon which usually precedes the decision to engage in a learning project as defined by Brookfield (1986). Conference on "Learner Driven Learning" The questions of learner-driven learning was the subject of a rather unique conference held in 1986 when 14 notables representing 6 nations explored the nature and implications of "leaming-how-to-leam." The principals present were: Phillip Candy, University o f New England, Australia; Ronald Cervero, University o f Georgia; Mark Cheren, Sangamon State University; K. Patricia Cross, Harvard University; Ravindra H. Dave, UNESCO Institute fo r Education, Hamburg; Maurice 201 Gibbons, Simon Fraser University; Virginia Griffin, Ontario Institute fo r Studies o f Education; Dai Hounsell, University o f Edinburgh; Malcolm Knowles, North Carolina State University; Alan Knox, University o f Wisconsin-Madison; Dana Maxfield, Northern Illinois University; Jean Moon, University o f Wisconsin-Mi/waukee; and Robert Smith, Northern Illinois University. Smith (1988) as Editor and Chairman attempted to maintain copious notes of the discussion and the various comments. As presenters made certain points of their individual postulates, counterpoints were offered by others in attendance. One such issue was "control." Gibbons asserted, "The locus of control over decision making about the [learning-related] event should rest in the hands of the learner. Smith countered, "If you're going to learn Sumo wrestling in Japan and you say, 'I want to develop my own program,1 you aren't going to last veiy long. . . .What you have to recognize are requirements of the educational and appropriate behavior - which includes the amount of control to relinquish." The group attempted to build a new theory by the establishment of a paradigm chart with five domains with two dimensions on "leaming-to-leam." This initial effort was not well accepted, and, after long ruminations four 202 domains and five dimensions were defined. O f these domains, formal, open, and social are of interest in this discussion. D O M A IN S Dim ension form al Open Social Nature o f K nowledge Critical acceptance External Critical engagem ent D ialogue Learning A ctivities M aking m ost o f options Negotiate options Understand roles o f collaborators Critieal skills o f H valuation 1 Effective use o f evaluations received N egotiate evaluation process and standards D evelop evaluation standards Thinking Select best m ode Divergent Creative Not applicable Self-assessm ent Not applicable Sense o f direction Self management Not applicable Table 11 In assessing this framework, a rather strong point emerged. Candy posits that content or subject matter must have strong framing and strong classification so that the boundaries are clearly delineated. He goes on to conclude, "that French is not physics whereby academic boundaries are defined." However, a counterpoint was made by Cheren concerning the organization of physicians education. In an open course at Southern Illinois, 203 the student and faculty invent a "new course" based on "... going through a process that is identical to what they will go through later on." This concept of being trained today for today's competencies as opposed to tomorrow's educational needs was strongly attacked by Knowles. Citing a book by J. W. Botkin et al., No Limits to Learning from Pergamon Press, 1979, Knowles stated: The main proposition is that the world is accelerating change, as we have entered, it is not functional for educational systems to be turning out people who only have the competencies to deal with the world as it is now. They become obsolete rapidly. They call our present system maintenance education - providing people with equipment to maintain things as they are now. (emphasis added) Knowles went on, "And they propose that in a world that is already changing we must reorganize our whole education system on the basis of 'innovative education' or 'anticipatory learning." Continuing his quote of Botkin et al. Knowles added, "The primary mission of formal education should be to develop the skills of learning." The issues which would preclude this approach to education are the current school reform movement. Cross explained,". . .the movement's direction seems to be toward students learning how to give quick, easy 204 answers on answer sheets that are easily scorable." Another barrier she cited was, . . . a researcher's discovery that the predominant mode o f teaching in colleges was what was called "bitting." Bitting is when you provide isolated bits o f information with no overall framework. The researcher discovered that to be the way the teachers responded to the perceived low ability o f their students. . . . Teachers were teaching down, making low cognitive demands, by presenting bits o f information. Students responded by trying their best to recall bits. During the course of the meeting, consensus agreement was obtained on the following paraphrased points: • Learning-to-learn is a life time commitment for all. • Case studies encourage people to reflect on past instances in which they have been engaged to critique, analyze, and figure out what went well, what went poorly, what they've learned that is transferable. • The entire first week o f a nominal sixteen week technical course must be devoted to learning-to- learn. • In ideal learning environments, student learning preferences or styles should be matched with the instructors teaching style. On this latter point of teacher teaching styles, four college teaching prototypes were outlined by Cross: 205 Professor A's goal is to cover the subject matter in a rather systematic and orderly fashion, presenting the facts and principles and then testing students on their comprehension and their understanding o f those principles and facts o f the discipline. He regards knowledge as a product rather than a process: his style o f teaching is ordinarily to use lectures, but occasionally he uses discussions, primarily to determine whether the students comprehend the material. He works very hard at "covering the subject matter in his class, and his evaluation approach usually consists o f short answer tests to which he assigns numerical scores because most questions can be graded as right answers. Motto: "I teach what I know." Professor B is one o f the most colorful figures on campus, and his goal in teaching is to demonstrate how the educated mind works. His style is to lecture, and his purpose is to provide a model for students, to give students a demonstration o f how a person who is really competent in a discipline attacks that discipline. He makes no attempt to cover the subject matter because he regards learning as process rather than product. His evaluation tends to be asking the students to demonstrate that they are educated persons in the use o f their minds. Motto: "I teach what I am." Professor C has a goal to promote cognitive development in students. He will ordinarily concentrate on the higher levels o f the Bloom taxonomy o f cognitive skills, focusing more on asking students to analyze, to synthesize, and to evaluate rather than simply recall and recognize and comprehend. He would ordinarily in teaching style use a discussion method or Socratic method or perhaps the case study, primarily to see how people are developing in terms o f cognition. The evaluation would ordinarily consist o f essay exams or problem solving stressing analysis. Motto: "I develop minds." Professor D is student centered and his goal is to develop the whole person. His teaching style would ordinarily be informal, would quite frequently serve as facilitator o f the learning group, would have a lot o f off-campus and on-campus interactions with students, would know all students by name, and would ordinarily know a good deal o f their backgrounds and their current 206 concerns. He would steer clear o f hard-nosed evaluation and instead would diagnose for learning development. Motto: "I develop people." After doing so, she posits a series of conclusions based upon several tacit facts, the final stating a concern about the doubling of technical and scientific information every five and one half years. With this consideration, she then asks a series of trenchant questions centered on lack of teaching expertise in colleges, teaching easily measured facts, change in the criteria for excellence from faculty to the quality of student learning, and that leaming-to- leam is the only education worth pursuing for students, for educators, and for society. Adult Memory A life time dedicated to learning raises the issue of cognitive functioning over a life course. The concepts of "fluid" and "crystallized" memory coupled with the considerations of novice vs. expert are the subject of Cognitive Functioning and Social Structure Over a Life Course edited by Carmi Schooler and K. Warner Scaie. The statistical data of the students attending the Program Managers Course at DSMC identifies that the majority of the students are in the mid to late thirties. Certain aspects of this debate 207 over the level and nature of intelligence as related to age must be addressed in order to arrive at the most productive method/s of curriculum delivery. It is tacit knowledge that test-relevant abilities digress with age while life-relevant abilities may or may not. According to Schooler,". . .the complexity of an individual's environment is defined by its stimulus and demand characteristics." Under this theory the more diverse the stimuli, decisions required, number of intervening factors, and ill defined contradictory contingencies, individuals should be motivated to develop intellectual capacities dependent on environmental rewards. Schooler goes on to say, "Consequently, continued exposure to relatively simple environments may result in a decrement in cognitive functioning, particularly intellectual flexibility, and a change to values, orientations, and behaviors in keeping with the level of environmental demand." Schooler cites work by Blalock, Goldberg, and Kohn & Schooler as establishing a casual relationship between complex work and intellectual flexibility and self-directedness. However, Schooler also found that when the environmental conditions would not allow intellectual flexibility and required a conformist orientation the lack of occupational self-directedness affected the individual's cognition, values and orientations. Schooler postulated the greater the diversity and complexity of 208 the environmental stimuli and decision faced on the job, the more self directed their orientation. However in counter position, Streufert & Streufert (1978), found, "Experience with too many unfamiliar events in too short a period of time is likely to produce overload due to too much incongruity." According to the Streufert's theory individuals will exercise self-direction if the environment permits and if not, the individual will suffer decrements in their functioning. Experts vs. Novices The mandate imposed by Congress to change the behavior of the acquisition workforce via education and training bears on the memory issue as related to experts vs. novices. As discussed in the preceding paragraphs, knowledge-rich task domains illustrate a possible casual inference, resultant from an interaction between environment and individual cognitive processes. Glaser's Thoughts on Expertise identify that if the knowledge of the domain is not organized by the individual for efficient utilization, expertise cannot exist. In addition, when the individual's knowledge classifies problems only at the surface level and not in terms of the principles involved, the individual is a novice and not an expert. Glaser states, "Experts are principle oriented, 209 whereas the representations of novices are object oriented." It is his belief with novices," ... general problem solving heuristics are limited in the novices inability to infer further knowledge from the literal clues of the problem." Experts metacognition and general skills," . . . show that the generalizable and transferable expertise lies in an ability to use familiar domains of knowledge for analogical and metaphorical thinking about new domains." Glaser's conclusions about experts are: 1. There seems to be a continuous development o f competence. 2. Expertise seems to be very specific. 3. Experts develop the ability to perceive large meaningful patterns. 4. The knowledge o f experts is highly procedural and goal oriented. 5. The fast-access recognition and representational capability o f experts facilitate problem perception in a way that leads to the reduction o f the role o f memory search and general processing. 6. Individuals who acquire expertise in several domains may develop generalized thinking and problem-solving skills. 7. The experience o f experts enables them to develop skilled self-regulatory processes. 8. The precision o f expert performance results from specialized schemata which drive their performance. 9. Although experts display specific domain intelligence, they do not necessarily achieve high scores on measures o f general intelligence. 210 10. The development o f expertise is influenced by task demands encountered in the course o f experience and by the conditions o f working situations. The importance of Glaser's conclusions is defined in his final remarks. It seems evident that expertise is acquired when people continually try to confront new situations in terms o f what they know. . . . When teaching beginners, this might be accomplished by assessing and using relevant prior knowledge, or y providing obvious organizational schemes or temporary models as scaffolds for new information. . . . In addition, instruction should focus on the development o f procedurally oriented knowledge and structures that incorporate knowledge with conditions o f its use and applicability. Educational Effects on Learning The Rand Corporation completed a composite study of the results of 19 cost studies to determine the cost effectiveness of various combinations of environmental factors. According to Walenberg (1987), the Rand conclusion was, "Research has not identified a variant of existing systems that is consistently related to students' educational outcome." From a systems perspective this would indicate the existing system has absolutely no bearing on student outcome. This being the case, either the system itself is suspect or the methodology used to accomplish the research is suggested. O f the two, the methodology would appear to be the most correct answer. Schools and colleges measure collectively, as opposed to individually. It is Walenberg's theory that psychological climate of the classroom, peer group pressures, and 211 out of school stimulations influence learning. He believes that environmental influences also affect classroom learning, because students are naturally exposed in the environment and have learned from it. The sum and substance of what is learned is a function of prior exposure in the environment. This has significant implications in setting the environment for the acquisition workforce and defining who is eligible to join the Acquisition Corps to become a Program Manager. 212 Section D: Multi-rater - 360 Degree Rating Metrics / PROFILOR In the 50s, peer ratings were referred to as "buddy sheets" and in the corporate world its was called "sack meetings for pecking order." Peer ratings seemed to be based on personal likes and dislikes and were accomplished without adequately educating the participants . Follow-up consultations were for the most part nonexistent and there appeared to be no supporting base of scientific research. The annual employee rating by the immediate superior that determined raises and promotions normally created more dissatisfaction than job improvement. Unless an individual had a corporate sponsor or mentor there was little individual advice on developmental needs to maximize strengths and to make needed improvements. At that time, many of the ratings were based on Stogdil's trait theory and not on either behavior or job accomplishment. The ratings were individually based and gave little consideration to team contributions. Because of the inadequacy of this rating system, the commercial development of rating instruments has enjoyed tremendous growth in recent years. With this growth, research and evaluation techniques have established both reliability and validity in a number of trusted well-designed multi-rater systems. Human factor research efforts have identified effects such as 213 "Hawthorne," "Matthew," "John Henry," "Placebo," "Post Hoc," "Misplaced Precision," "Typical Case Studies," and "The Law of the Instruments." David Campbell, currently with the Colorado Springs office of the Center for Creative Leadership (CCL), has been instrumental in the development of many individual and corporate systems evaluation tools. Instrument Comparative Analysis In 1978, Morrison, McCall, and DeVries, from the Center for Creative Leadership, published their seminal work Feedback to Managers: A Comprehensive Review of Twenty-four Instruments. With this publication, for the first time the commercial world had a documented comparison on most of the multi-perspective instruments available. Following this publication, the Center for Creative Leadership published a restructured edition in 1991, Feedback To Manager: Volume I and II. Volume I is a subject matter guide for evaluating multi-rater feedback instruments, while Volume II is a comprehensive review and comparison of the sixteen most popular multi-rater instruments. All instrument comparisons are standardized and consist of, but are not limited to, such elements as: the statement or purpose of the instrument, target audience, feedback scales, reliability, inter rater agreement, validity, construct validity, concurrent validity, international 214 use, feedback characteristics, support materials, and cost. Such a comparative analysis is invaluable to the human resource personnel, who must evaluate the different types of instruments available on the market for corporate use. The monograph, Multi-rater assessment and Feedback: Applications. Implementations and Implications by Howard, Byham, and Hauenstein from Development Dimensions International (DDI) provides a critical insight on the importance of multi-rater assessments to the individual and corporation. Placing the focus on the how and what of performance, individuals are rated by their superiors, peers, subordinates, self and even customers and are able to obtain a complete picture as to how their behaviors are perceived by those with whom they interact. The sponsoring corporation, as well as the individual, gains invaluable insights. The individual gains useful knowledge in individual performance development by the identification of relative strengths and weaknesses. The assessment also sharpens the focus on career planning, and provides a metric linkage with performance by the measurement of self-achievement towards a definable goal. London and Beaty agreed in their article "360 Degree Feedback as a Competitive Advantage" published in Human Resource Management (Summer & Fall 215 1993). This issue of Human Resource Management (T993I is a special publication discussing varied views and positions on 360 degree feedback by many of the notables in the field. The issues lacks a unified position on 360 degree feedback. Dunnette, who was charged with authoring the summary findings, stated, "In these papers, I see a hodgepodge of techniques, testimonials, cautions, methodological difficulties, some axes being ground, and a rather confusing lack of cohesion." However he did go on to state: The net impact o f these articles is significant, showing clearly that 360 degree feedback methodology possesses distinct promise for improved success in both individualized and group programs o f personal development in the bureaucratic and organizational settings o f the future. Howard, et al. hold that Human Resources Planning can aggregate individual data to determine accession information, in addition to pointing out corporate strengths and weaknesses relative to job dimensions. If the customer is incorporated into the multi-rating system, determinations can be made on how well the corporation is meeting customer service goals. Of major importance is the concept that aggregated data is able to define a perspective corporate behaviors. If corporate change is desired, time series multi-rater aggregates will delineate if progress has been made toward the desired cultural shift. This methodology is of vital importance considering 216 the Congressional DAWIA mandate to change the culture of the Acquisition Workforce. Moses, Hollenbeck, and Sorcher (1993) say that 360 degree feedback is severely limited by many factors including: 1. The reliance on generalized traits 2. A limited non-existent frame o f reference for making rater / observer judgements 3. A memory based, often incomplete description o f past performance 4. The inability o f the observer to interpret behaviors 5. The reliance on the instrument designer's scoring system, factor analysis, or data collection methods to interpret the information for the participant It is their view that raters use "expectations" as the foundation of their ratings. Using this as a defined fact, their proposal is to separate expectations from observations by the use of two different instruments. Dunnette concurs, in part, accepting Moses' et al. caution and challenge to stay close to behavior avoiding traits admitting that the 5 "problems" have merit and deserve research. However, Dunnette finds that the details of their proposed instruments are lacking in detail, and, he suggests that they are probably subject to the same limitations present in 360 degree formats. 360 Degree Agglomerations Techniques - Team Evaluations By separating the aggregate data into teams or groups, evaluation metrics on the validation of team education and training is possible, as well as the generation of the companion "team training needs assessment." It is Howard's et al. view that all team members should rate one another and the aggregate results then be openly discussed," . . . a discussion at team level can inspire team diagnostics insights; that is, the team might discover that common problems can be attributed to team functioning." The salient issues raised in this monograph is whether the individual receiving the rating believes the accuracy of the assessment. This question identifies factors which may adversely effect the ratings accuracy. Howard et al. suggest that such factors are inflated supervisor ratings, years of peer acquaintance, and over-rated or under-rated self ratings. Citing published reports from Harris & Schaubroeck and London & Wohlers, the monograph concludes that the lowest agreement between raters occurs when self is one of the rating sources. Veslor, Taylor and Leslie (1993) agree with these findings adding a caveat that gender differences also exist wherein females tend to significantly underrate themselves. However, Nilsen and Campbell (1993) utilized a different basis for analysis and found that self-perception is 218 a stable individual difference. All agree, the greatest inter-rater agreement occurs between supervisor, peers, and subordinates. To partially eliminate some of these rating anomalies, Howard et al. recommend the use of an assessment center to mediate rating shortcomings through education and training. They acknowledge that corporate multi rating occurs in a natural environment, where an assessment center is more of a laboratory nature. However, an assessment center can accomplish an integration of assessment techniques, such as an in basket exercise coupled with a multi-rater assessment, which is difficult to accomplished in a corporate multi-rater natural environment. Hazucha, Hezlett, and Schneider (1993) and Howard et al (1994) report on the progression of multi-rater instruments from their use as a developmental tool to their use as a promotional and extrinsic reward system. Other researchers have reported both positive and negative results when so utilized. The use of upward appraisals has also attracted detractors, who raise concerns over corporate integrity. In specific, they cite Bemardin (1986), who expresses concern that: 219 • The supervisor focuses on pleasing subordinates to get a high appraisal. • The practice undermines the authority o f the manager. • Subordinates are reluctant to be candid about their supervisors for fear o f repercussions. • Subordinates inflate their ratings to gain favor. In their concluding remarks, Howard et al. identify that if the provision of non-attribution is violated or otherwise compromised the multi-rater instruments has a significant downside. However, the benefits include: • Communication o f the corporate values • Enhance two-way communication • Uncover unresolved conflicts • Build more effective working relationships • Demonstrate respect for employee opinions • Increase opportunities for employee involvement • Create greater value for customers • Foster a participative climate Acknowledging the danger accompanying the improper use of privacy data, all authors agree that the advantages outweigh the disadvantages so long as the pitfalls are defined and avoided. General guidelines proposed by Howard et al. are: 220 1. Choose applications wisely: Never use the power plays such as pay or layoff. 2. Implement the process respectfully. Consciously observe confidentiality. 3. Select well-developed systems with appropriate options. Flexible - with strong aggregate and analytical capabilities. 4. Confront rater accuracy. Error introduction occurs at the outset. M onitor output for inter-rater agreement. If possible train raters. 5. Don't overuse the system. Validation o f change and training needs maybe invalidated without inter-rater agreement. Trade and Media Articles The trade magazines and popular media publications contain feature articles on the use of 360 Feedback or Multi-rater Assessments as do the more conservative education and training journals. It is recognized that some of these sources many times fail in the rigor required of sound and strong argument. However, they reflect the current paradigm shift that is taking place in industry and is being discussed and taught in colleges and universities. As such, coverage of the paradigm shift is important to the understand of the use of multi-rater feedback at DSMC. A users conference of multi-rater 360 degree systems; USA TODAY and various government agencies such as the Central Intelligence Agency, industry consultants, associations and private businesses. In the 221 agglomerated managerial survey conducted by USA TODAY in August 1989, companies using a multi-rater methodology defined these skills (traits, sic) as necessary to get to the top: Communication Skills 89% Problem Solving / Decision Making Skills 85% Understanding Organization and its People 81% Personal Power - Poise, Serenity, Inner resources 78% Cooperating as Part o f a Team 75% Knowing how to Motivate People 71% Savvy in Organizational Politics 63% Delegation o f Supervisory Skills 63% M entor 54% Charisma, Charm, Social Skills 53% Table 12 It is recognized that the popular press is not a reliable source, but the traits defined by the press are those measured by most of the multi-rater assessment tools. A like article in FORTUNE authored by O'Reilly (1994) makes a strong point, "What is most interesting about feedback isn't the pain it causes, the mechanics of its operation, or its growing popularity. It is in the huge variety of unpredictable comments - and potential learning - that it 222 delivers." Interviewing personnel from the Center for Creative Leadership, Personnel Decisions Incorporated, Teams Incorporated, O'Reilly cites the advantages claimed by personnel from major US corporate and educational entities such as Raychem, AT&T, Nestle, General Motors, Pitney Bowes, Chase Manhattan Bank, DuPont and Vanderbilt University. All cite the vital information gained via multi-rater assessment, without which career derailment most likely would have been a certainty. The boss learns of facts never known before such as: "Oversized ego - the higher they go the less feedback they get, so the view of themselves gets distorted," "For some control freaks, it takes massive doses of feedback before the light finally comes," or the discovery that subordinates "... were frustrated because he sometimes made them follow prescribed processes without finding out whether there were better ways of doing things." In another FORTUNE 1989 article, "When Subordinates Evaluate the Boss", Walter Kiechel III writes that the feedback report business has been "mushrooming" since 1984. Part of this is due to the fact that the feedback report takes on the burden of providing the "bad news" that no junior manager wants to communicate and in the opposite sense removes from the senior manager the same onus. As the article states,W hatever the findings, 223 managers and consultants should then work together to devise a plan that will build on strengths and address developmental needs." However, there is a down side if nothing comes about as a result. Professor Black of Dartmouth states in the article, "You have raised expectations, by doing nothing you dash those expectations. Better for employees to think the boss doesn't care for them than to think the that he betrayed them." Even the Wall Street Journal feature article, "Firms Ask Workers to Rate Their Bosses" written by Larry Reibstein (1988), quotes college professors as an authenticating and reliable source, claiming that "subordinate feedback is probably more valid than superiors." But not all bosses are willing to accept such feedback if it is negative. Reibstein cites the case of the rather hard bitten executive who believed that feedback was a "bunch of b.s., seeing the whole thing as an attack on him, justifying his actions by insisting subordinates don’ t understand the demands placed on him." This view is very reflective of some former and current uniformed officers in the Acquisition Corps. It is a reflection of status quo and not of cultural change. Cris Lee's article, "Talking Back To The Boss," in Training. (1990) speaks to the aspect that multi-rater feedback can be useful in 224 instituting cultural change. However, certain of the consultants she quotes warn against comparative norms. One, J. Robert Carlton with Vanguard Group in San Francisco contends, "To compare the behavior for effective supervision of these groups (bachelor degree with five year vs. Ph.D.s with tenure and 20 year, sic) are the same is ludicrous. In an analogous sense the same could be said of the comparison of a military combat command commander vs. a civil servant supervisor of a military acquisition command. Carlson goes on, "Here is how you are perceived, here are your strengths and weaknesses. Now think, of your business, your people and strategic plans. Are those strengths and weaknesses really strengths and weaknesses in this situation?" If they are, he recommends doing something about it. If not, he recommends that weaknesses that have no impact should not be attended to - it would be a waste of time and money with dubious results. Other articles in Training and Training & Development carry a similar theme. "When Workers Rate the Boss," Training by Robert McGarvey and Scott Smith (1993) reflect that one third of the bosses see 360 degree ratings as "Intrusive and really want nothing to do with it." They have found in their interviews with firms that use multi-rater feedback, that many of the bosses 225 "... go into shock when they first read their results." It is what they call the "SARAH" reaction, shock, anger, rejection, acceptance, and help. The article does add an element that has not been dealt with before, the use of outside consultants. Emotional distance is offered by outside consultants which according to the authors," . . . may help the employees open up and provide more honest answers." The big drawback is,"... an outsider will not fully understand a client company's culture, and in fact may be limited to his own consulting firm's model on how the universe is supposed to work." The broad warning offered in this article about the conduct and use of 360 degree assessments has significant consideration to the Acquisition Corps, namely that, "Organizations being downsized or any place with minimal communication up and down should not bother, because trying to do so will just feed paranoia." An open multiple feedback rich environment is not a cultural attribute of the military. The application of multi-rater instruments in the Acquisition Corp probably should be limited to school situations. The Human Resources and Training and Development trade journals offer a more academic examination of the potential offered by multi-rater feedback instruments. Many of the articles are corporately identified like the 226 article that appeared in the Human Resources Professional. Winter 1990, "Employee Feedback and Development of Managers at Honeywell" by Rick Clevette and Brian Davis. The key point in this article relates to peer ratings. "Because peers are pitted in competition with each other for resources, budgets, promotions, and perks, the assumption has been that they are more apt to be critical of one another." The article goes on to point out, "... a resulting concern is that peers should perhaps work more closely together and, in effect, cooperate for the greater productivity of the work team." This aspect of "teaming" is not a simple accomplishment. It represents a cultural shift and cognitive growth that is well defined by the MBTI and Perry LEP charts, as it relates to acceptance of peers and their ideas. With the onset of the DoD order directing the use of Integrated Product Teams (IPD), peer review has taken on an increased importance. Gloria Bader and Audrey Bloom (1992) discuss, "How to do Peer Review" in Training & Development. Their expressed concern correlates to the options chosen during the planning phase of the review, including: Range o f Performance to be evaluated Participation Options 227 Procedures Training and Orientation Tools Their view is emphasized by Martin Ramsey and Howard Lehto (1994) in the article, "The Power of Peer Review", Training & Development. However, they add the consideration that the key element necessary to obtain good team peer reviews is the help of a team facilitator. "A facilitator would provide training to teams on conducting peer reviews, guide team members through the appraisal process, and facilitate the actual reviews." PROFILOR Other multi-rater feedback systems were evaluated by DSMC, including, the Center for Creative Leadership's Benchmarks®, however instruments offered by Personnel Decisions Incorporated (PDI) have been used at DSMC since 1991. The PDI instrument of current usage is the PROFILOR. This instrument has been in use since January 1993 commencing with PMC 93-1. Other than the individual feedback report, the key literature items are: The Development Guide Successful Managers Handbook 228 PROFILOR Certification Workshop Manual Making a World of Difference Manual. These proprietary PDI documents are team authored. The Development Guide is provided to the students at the time of receipt of their individual PROFILOR reports. The guide's focus is an explanation on how to read, interpret, and use the PROFILOR report to its lull advantage. Detailed information is provided on each phase or section of the report: Importance Summary, Skills Overview, Perspective Comparisons, Specific Question Results, Overall Composite, Highest Ratings, Focus for Development, and Development Suggestions to include a Focus Chart on Strengths and Developmental Needs requiring immediate address. A sample Development Plan illustrating the methodology used to address Strengths and Developmental Needs is also included as are the respective blank work sheets. The PROFILOR report is explained as being a document whose feedback is: Anonymous from direct reports and peers / colleagues, and thus more likely to be frank and honest that face-to-face feedback from those sources. Comprehensive, in that it comes from direct reports and peers / colleagues as well as the boss. 229 Specific and completely job related, thereby providing a clear understanding o f strengths and development needs. It is PDI's belief that, "This kind of feedback can lead to significant, positive change." However, PDI does acknowledge the,"... usefulness of this information depends on you." The strongest bit of advice PDI gives is, "Remember that people behave according to what they believe is true. Changing perceptions may be just as important as improving your skills." This latter point of improving skills and perceptions is also indirectly addressed by the participant's individual report. To answer the question, "Ok, now what do I do about it", the Developmental Guide provides a step by step guide for the preparation of a development plan that will address both strengths and needs. Additionally, the individual report provides a ready reference to another document, Successful Manager's Handbook: Developmental Suggestions for Today’ s Managers, which goes into detail on what specific actions should be considered. This last point is an important consideration. It may well be that Congress' impression of the Acquisition Workforce, upon which they acted in the passage of the DAWIA needs to be corrected! The Successful Manager's Handbook is oriented around the eight core factors PDI has found to be critical to management success: 230 Administrative Skills Communication Skills Interpersonal Skills Leadership Skills Motivation Skills Organizational Strategy Skills Self-Management Skills Thinking Skills The Skills which are the subset of these nine core factors are illustrated on Chart 10. The skill assessment sets of each individual are defined and prioritized in personalized report received by each participating student. Handbook page reference numbers of each skill item listed for that individual are listed in the margin of the report for handy reference. ( ‘PROFilQR' Wheel from Personnel Decisions, Inc. F«dor \ F acb Chart 10 A 231 The handbook is designed with two thoughts in mind, first development of self, and second the development of others. The handbook causes the reader to refer back to an individual skills assessment or to accomplish one, if this step has not been accomplished. What follows is a step-wise plan though all of the skills areas defining a methodology to achieve the developmental goals, to include outside readings, seminars, and workshops. The handbook is very detailed and a sound management reference manual. The proper use of a 360 degree instrument is for growth and development of the participant. To facilitate a participant through the PROFILOR growth and developmental process, which may be emotionally stressful, requires certification. A PROFILOR Certification Workshop is conducted by PDI personnel for those individuals who are licensed and authorized to use their proprietary instrument/s. O f course, the first step in the process is for the individual being certified to start by having a 360 degree instrument filled out on him/herself. The process begins with the completing a self-report and then requesting that your supervisor, four to five peers and four to five subordinates also fill out a report on you. All reports are accomplished on a non-attribution basis with the exception of your own and 232 your supervisor. After processing, a completed individual report, as outlined in the Developmental Guide, is furnished to the trainee. The PDI's Profile Family consists of the PROFILOR, Executive Success Profile, Management Skills Profile, Individual Development Profile and Consulting Skills Profile. The faculty at DSMC is trained and licensed to administrate the Management Skills Profile and the PROFILOR. The instrument of current usage upon which this report is based, is the PROFILOR. The certification consists of understanding the technical aspect of the measurement system, how it was developed, and a general briefing on the research statistics. The concepts of how to communicate feedback to your students and what reactions to expect from what might be overwhelming data full of surprises, either positive or negative, is first experienced from your own report and, secondly, formally discussed in an academic construct. The roles and responsibilities of the PROFILOR facilitator are many: administrator, presenter, interpreter, counselor, and resource person. As such, for the benefit of the students, the facilitator must follow certain guidelines: • Create a safe environment in which to learn • Outline the benefits o f the program 233 • Be organized and efficient • Invite participants' contributions • Encourage participants to interact • Use examples • Model behavior • Provide feedback PROFILOR User's Conference Seminar The theme of the PROFILOR 1995 Conference, Making a World of Difference, was the an informal discussion in the continuous development of 360 degree feedback in the human resources arena. PDI presented a logic argue centering on its use as an upward appraisal tool. The theory presented was that with strategic intent the corporation could use it in the furtherance of the development of critical personal skills. This requires line management's support to address real corporate business and organizational needs. An area which PDI specifically pin-pointed was the cross cultural standardization required to yield a more efficient global corporate structure. Corporate data gathered from all sources using the PDI instrument reported the 1994 top five Importance Ratings as being: Act with Integrity 2 3 4 Focus on the Customer Use Sound Judgement M anage Execution Foster Teamwork On this subject, PDI emphasized the use of grouped data to identify cultural norms and as a methodology for tracking behavioral change. PDI shared its corporate data from all available sources on the nature of the business community cultural norms. In the PDI Norm Group the highest rated skills were: Drive for Results Act With Integrity Analyze Issues Use Sound Judgement Focus on Customer Needs Foster Team Work The most admired companies had a slightly different top five: Drive for Results Act With Integrity Focus on Customer Needs Analyze Issues Think Strategically Use Sound Judgement The composite PDI listing for individual development were: Champion change Develop and motivate people Manage quality and work progress Manage profitability skills Global Focus Influence 235 This latter listing is extremely interesting, as it bears on the issue of fostering work and contains other behavioral considerations relevant to the culture of the Acquisition Workforce. The primary question raised by Congress in the passage of the DAWIA was the culture and competency of the workforce. It is PDI's contention th a t,". . .competencies are the knowledge, skills, abilities, and attitudes required for successfiil performance." Therefore factors such as leadership are measurable via a 360 degree feedback on skills such as coach and develop others, as measured by the behavior defined by peers and subordinates. Kaplan and Palus from the Center for Creative Leadership have authored a pamphlet utilizing this technique for the personal counseling of client managers. The pamphlet entitled, "Enhancing 360-Degree Feedback for Senior Executives: How to Maximize the Benefits and Minimize the Risks", suggests using data much deeper that can readily be defined in a normal 360 degree genera. To achieve this level of intimacy they obtain: Observations from family members and friends Psychometric measures o f personality and motivation Historical biographical data A personal coaching relationship is established with one or more of the assigned professional staff. The rationale supporting this rather powerful, 236 intensive and intrusive intervention is based on the research of Kaplan, Drath & Kofodimos (1991), defining a broader range of challenges faced by the modem executive. In addition CCI holds that the psychological makeup of people who rise to high-level positions is . . extremely achievement oriented, very focused, and highly demanding." As such, CCI and other senior manager consultants believe that standardized feedback does little to deal with the skepticism of senior executives. An additional conundrum is the organization itself and its belief that executives are unable to change at the psychological level. According to the research of Kaplan, Drath, & Kofodimos (1985), successful senior managers, by virtue of their high position, are normally treated as though they were exceptional. They see no need to change and are,"... reluctant to tamper with what got them to where they are." However, these same senior executives can underperform and as CCI sees i t ,". . .get outstanding results for the short run by sacrificing the organization's ability to maintain high performance in the long run." This in depth view of an enhanced 360 degree feedback defines high risk and strict standards of ethical conduct, because its power is so potentially harmful. However, a close examination of the environment defining the military culture of the Acquisition Workforce produces some striking 237 parallels. All Program Managers and the majority of their direct subordinates (civil servants or uniformed officers) are of sufficient rank and stature so as to be treated differentially. They too are reluctant to change or tamper with what has brought them success. The normal military tenure in a PM position is less than 18 months. Through the long history of Acquisition Program Operations, the civil servant workforce knows that the probability of witnessing a military officer change at the psychological level is nil or non- existant. This aspect is well identified by Dr. McAleer (1993), who states, "... The military officer who has made it successfully to the 15 to 20 year point in his career may think he is an effective leader." McAleer goes on to state that at the Industrial College of the Anned Forces (ICAF) they have the formidable task to point out that the continued road to success may not be following the same path. McAleer postulates: What we (ICAF, sic) attempt to do is this: Convince these military officers that the behaviors that made them successful up to this point in their career will probably not be the ones that will serve them best as a strategic decision-makers in the future. This notion is usually received with both doubt and trepidation. They neither believe it - nor at first anyway - nor do they relish the idea o f changing their modus operandi 238 O f those senior officers attending ICAF, only one in five will attain the rank of general in the next five years. The reasons for the career derailment of the other four is not stated. Possibly the issue of feedback, as it relates to self- development, was not understood by the ICAF student, or for some other rationality was ignored. Holt's (1994) dissertation, "Understanding Managerial Feedback and Seif-Development: Does Readiness Make a Difference?", investigates part of this enigma. Holt found, "Older employees perceived less opportunity for advancement, and were less interested in self-development than younger employees and those having high organizational commitment with equivalent managerial aspirations. There was no relationship between attitudes toward self-development and any self-reported training needs." In her literature research, Holt cites: Noe (1990) who proposed that "locus o f control" is a determinant o f self-efficacy Earley, Northcrafl, Lee & Lituchy (1990) identified self-confidence as a mediating variable between feedback and performance. Ashford (1986) stated, . .those lacking confidence will likely see negative feedback as particularly threatening to their self-esteem." From this and other elements, Holt posits the relevance of feedback is a composite of the latent variable effects of three predictors. 239 Self-confidence: The demonstrated belief in one's own competence to handle a variety o f situations successfully, including taking charge when necessary, making timely decisions, and persisting even in ambiguous or stressful conditions. Feedback-seeking Orientation: The tendency to engage in activities such as monitoring and inquiry to elicit information about oneself or one's environment (including the assessment o f strengths, weaknesses, and / or potential impact o f skills, behaviors, goals, or plans). Learning Orientation: The tendency to set learning goals for the purpose o f developing / increasing competence or mastery in various attributes, behaviors, or areas o f performance. Holt also found that participants experiencing multi-rater feedback indicated their bosses most often do not provide developmental information on a periodic basis. H o w e v e r ,. .fifty one percent (51%) reported that the feedback experience has led them to see feedback from others more often than before." It is common knowledge the annual officer and civil servant rating systems are gamed and not a viable or reliable source of developmental information. As such, commercial off-the-shelf 360 degree feedback systems offer to the Acquisition Workforce the most viable alternative to gaining insight to developmental strengths and needs. At question is the environment where such instruments are used and the readiness of the civil servants and military officers to acknowledge the truth of their ratings. 240 Section E: Implementation and Guidance from Department of Defense (DoD), Defense Acquisition University (DAU), Acquisition Management Functional Boards, and Others The niunber of documents produced by the Department of Defense in any one year is beyond imagination. In like manner, the number of documents produced by the Defense Acquisition University in any one year is also amazing. Certain of these documents produced by DoD and DAU have specific consequences, reverberations, and repercussions upon the operational environment of the Defense Systems Management College and the required career course offerings, such as the Program Management Series; Fundamentals of Systems Acquisition (FSAM); Intermediate Systems Acquisition Course (ISAC); and the Program Management Course (PMC). Of these course offerings a select number bear on this research. There is a constant search in DoD to find a better way. To some (the cynical), whether it be the changing of command or the changing of the political power, structure the posture of these changes seem to have a Baskin- Robins Flavor of the Month. Other DoD or DAU initiatives are a repackaging of old techniques under a new set of labels. Also, in a hierarchical structure there are initiatives fostered by those in executive power positions, who are always "just going to do it their way." An 241 examination of a few of these documents will provide a frame of reference of the environment currently defined by DoD's Corporate reaction to the Congressional mandate of the DAWIA. The Secretary of Defense's memorandum, "Use of Integrated Product and Process Development and Integrated Product Teams in DoD Acquisition" (May, 1995), ordered what is described,. . as a fundamental change in the way DoD acquires goods and services." The principal change in methodology is the use of Integrated Product Teams (IPT) in performing Integrated Process and Product Development (IPPD) work. The IP P D ,". . . simultaneously integrates all essential activities through the use of multi disciplinary teams to optimize design, manufacturing, and supportability processes." The greatest change, however, was the shift of DoD's oversight staffs from checking on a program six months prior to a milestone to a continuous participation facilitating and assisting the acquisition process through all milestones. In the past, roadblocks would suddenly appear when a program was due to obtain a milestone review and approval. Attached to this DoD Secretary of Defense directive is an accompanying implementation memo, "The Use of Integrated Product Teams (IPT) in DoD Acquisition." The IPT team is described therein as being 242 comprised of "program management, engineering, manufacturing, test, logistics, financial management, procurement, contract management, to include 'customers' and suppliers." The essential difference between an IPT and a normal Program Office is simply the empowerment of the team to make decision, which otherwise were always bucked or referenced to higher authority. The Under secretary of Defense (Acquisition & Technology) is quoted: . .move away from a pattern o f hierarchical decision-making to a process where decisions are made across organizational structures by integrated product teams. It means we are breaking down institutional barriers. It also means that our senior acquisition staffs are in a receive mode - not just a transmit mode. The objective is to be receptive to ideas from the field to obtain buy-in and lasting change." To ". . .move away from a pattern of hierarchical decision making" is a cultural revolution to a military organization. For centuries, the US Military has been educated and trained on a doctrine of delegation of authority while retaining ultimate responsibility. There is a significant cultural difference between delegation and empowerment. The objective of empowerment is to place decision making at the lowest possible level. Note the difference, the lowest possible level is significantly different than the lowest command level. 243 Because of its revolutionary nature, this order will not be readily accepted nor wholeheartedly support. As is normally the case in situations of this nature, Attachment 2, to the memorandum, "Integrated Product and Process Development (IPPD) Tenants," contains a caveat. "The team should be given the authority, responsibility, and resources to manage their product and its risk commensurate with the team’s capabilities." (added emphasis) A dramatic change such as this requires not only education and training but also metrics to validate implementation over time. In subsequent messages from lower tier offices, DSMC was immediately directed to modify its curriculum to incorporate team training while also being held responsible for the previously allocated curriculum and course competencies, all of this to be accomplished within the same time restrictions. PMC's Competency List: Beginnings and Evolutions Tracking the evolution and development of these required competencies is defined in a DAU directive, "Acquisition Workforce Core Course Developments" (January, 1994). This directive validated a memorandum written by the Executive Secretary of the Acquisition Management Functional Board, "Competency List for the Program 2 4 4 Management Career Field" (July, 1993). The antecedent to this list was contained in a document, "Competencies" published by DSMC's Program Management Education Division. This more interesting list of competencies, according to one of the authors, was generated in an ad hoc fashion by three faculty members who brain-stormed the question, "What should a DSMC graduate know when they leave DSMC?" Years later, as a result of the influence of Dr. Myron Tribus and Mr. David Langford, Total Quality Learning Seminar, Level I (FASM); II (ISAC); III (PMC), curriculum training levels were catagorized and differentiated by the addition of Bloom's Taxonomy levels of application to the respective and the corresponding educational level of the program managers course (FSAM, ISAC, PMC). From the forgoing, it is readily seen that the basis for the establishment of a set of competencies to define a curriculum at the different career levels is a lengthy process definably and is not based upon research or needs analysis. It is tme that the original list was validated by a properly constituted career and functional board, but here too there appeared to be no accomplishment of any curriculum research, rather a general acceptance of the majority of the list that by their experience and judgement should be correct. 245 However, arrived at this set of competencies is now changed by addition of a set of new competencies directive by the Secretary of Defense. This is not the only mandatory curriculum revision or change. DAU in discussions with the Army, Navy and Air Force jointly concluded that the 20 week PMC course was too long. The PMC time frame, as announced by the DSMC annual catalog on course offerings, was cut to fourteen weeks. What followed was a curriculum adjustment accomplished by an arbitrary distribution of academic hours to the various departments. Again, each academic department is charged to teach the same set of competencies previously scheduled in a 20 week course. Reflecting on this and other aspects of the DAWIA, Rhodes' article "Is DAWIA Worth It?", Acquisition Review Quarterly (SP, 1995) outlined a methodology to obtain,"... qualitative and quantitative feedback that DoD executives need to make informed decisions." He recommends: 1. Define study objectives 2. Define problem domain and boundaries 3. Identify measures o f effectiveness (MOEs) 4. Develop models 5. Identify data to be collected and sources o f data 6. Collect data 7. Analyze and interpret data 8. Report 246 It is Rhodes's belief that such questions as, "Does the Intermediate Systems Acquisition Course, a required course for Level II certification in the career field of Program Management and Communications - Computers, provide effective training on cost control measures?" Student Assessment Methods Part of Rhodes’ s answer may rest in DAU's "Student Assessment Process Guide."(1995) The announced goal in the forward to the guide states, "The goal of the Defense Acquisition University (DAU) is to provide quality instruction and educational products to our acquisition workforce." The forw ard further states, "It is the result of a through literature search conducted by the DAU Student Assessment Process Committee, which saw the need to publish a referenced document to aid DAU faculty. The specific purpose of the guide is,"... to better assist instructors in the development, revision, and evaluation of course products." A statement of "Responsibilities" defines the duties of the various offices affecting the content or administration for any Acquisition course. 1. The office o f the President, Defense Acquisition University will establish the policy, guidelines, and standards for student 247 assessment. The Student Assessment Process Committee will recommend policy, guidelines and standards to the Office o f the President. 2. The Functional Boards determine the critical competencies, task, and skills that will be taught and assessed. 3. The DAU Course Sponsor in concurrence with the applicable Functional Board, will identify courses appropriate for credit-by- examination establishment or discontinuance to the President DAU. The Student Assessment Process Committee will issue guidance for the appropriateness and development o f credit-by-examination equivalency instruments. 4. The DAU Course Sponsor will be responsible for developing the assessment instruments, ensuring the critical competencies are measured at the prescribed level-of-knowledge, It will also provide assessment instruments to its certified offerors. 5. The Course Sponsors and Course Offers will have the same evaluative process. 6. The DAU Course Sponsors and Course Offers will have appropriate test control and administration procedures for assessment. 7. Designated DAU curriculum review teams will review the procedures, instruments, and analyses o f the student assessment process during their periodic visits to the DAU consortium schools. The review team will apply the guidelines set forth in this document while performing its reviews. 8. Course Directors should use the Student Assessment Process Guide for appropriate guidance in the development o f student assessment. The seven sections of the guide are divided into Introduction, Objectives, Student Assessment, Learning Methods— Media— Modes, Curriculum Evaluation and Documentation, Remediation Techniques for Competency-based Education and Training, and Equivalency Certification / Credit-by-Examination. The Introduction Section is most interesting is a statement concerning the use of literature as f o llo w s,. .the committee did 248 not invent schema; all concepts are found (in the literature)." Reviewing the bibliographic references of each Section, it was found the vast majority of references were institutional within the military educational system. The popular authors cited were from the mid to late 70s with a few scattered in the 80s. The only major authors cited are Bloom, Gagne, and Mager. The guide defines, "A competency based educational program is one in which desired learning outcomes are specified in advance" whereby the performance statements are written into the learning objectives. "The performance statements are designed to the extent that they specify what the learner must be able to do or perform when he or she will be asked to demonstrate m astery of an object." By this means, "Competencies represent the specific instructional intents of the program and behavioral outcomes." Blooms' Taxonomy is a test methodology students to illustrate or prove their achievement of successive hierarchical learning goals in a specific subject matter. Test methodology is limited to: Multiple-choice True / False Matching Items Completion / Fill-In Essay Problem Solving Role Play / Simulations 249 In a like manner teach methods identified by a series of charts are: Reading assignments Lecture Demonstrations Field Trips Discussion Breakout Groups Recall Exercises Practical Exercises Case Studies / Critical Instance Incident Process Role Play Simulations Performance Test In-basket Exercise Student Presentations Charts on student learning modes define the advantages vs, disadvantages for: Classroom Instructor lead Tutoring Lecture Recitation Discussion Homework On-the-Job Training Self-Instruction The guide is silent on such matters as student cognitive assessment, individual learning preferences, team education and training, multi-rater evaluations, training needs assessment, educational research, course design research, and other educational assessment considerations. 2 5 0 USAF Air University Approach A DAU consortium member, the Air University (AU) at Maxwell Air Force Base (AFB), Alabama also examined assessment techniques in a paper, "Distance Learning (DL): Curriculum Analysis 7 Media Selection." Their defined purpose was to conduct resident student research to determine if, "... a course or a significant portion of it can be converted into a DL format." and to ". . . see which media are most likely candidates for exporting the course." In addition, the Course Director provided research data on Course Objectives, Cognitive Level (Blooms' Taxonomy), Affective Level (Krathwohl's Taxonomy), and current instructional methodology. The DL teaching methodology to be considered for use in this research proposal were extensive: Computer Hypertext Computer Simulation and Gaming Intelligent Computer-Assisted Instructed Multimedia (Combinations o f various media techniques) CD-Rom Digital Video Interactive (DVI) Soft Audio & Soft Video Audio Conferencing T eleconferencing Computer Mediated Conferencing (CMC) The question concerning distance learning arose because of two reasons. The first, under the DAWIA, certain formal Acquisition Workforce 251 courses became mandatory Professional Military Education. Second, the number of uniformed officers and civil servants now required to certified under Congressional mandate as being competent in their career fields overwhelms the numbers of classes that the methodology of the current system. Faced with the impasse of too many students, too few qualified instructors, and inadequate classroom space, DAU immediate resolution was to consider exporting the classes via distance learning to achieve the necessary multiplying effect. DAU's primary concern became the immediate through-put of large numbers of students. The Summary Report, "DAU Education Media Selection Guide" (1995) "The media selection guide was developed with the overall objective of identifying those educational technologies which offer the most potential for improving delivery of Defense Acquisition University (DAU) training courses." The instruction for Fundamentals of System Acquisition Management (2 weeks) was evaluated to best be accomplished by computer-based studies, followed by intelligent tutoring systems and finally classroom instruction. The Intermediate Systems Acquisition Course (4 weeks) was evaluated to be best accomplished by classroom instruction, followed by video conferencing, and real-time text- based discussion groups. The Program Management Course (14 weeks) was 252 evaluated to be best accomplished by classroom instruction followed by computer based studies, and video conferencing. The report concludes" Although Classroom Instruction was frequently ranked first or second, the results clearly illustrate that other educational technologies offer significant potential benefits in delivery o f DAU courses. . . . In general, there is strong evidence that many interactive and engaging educational technologies other than traditional classroom instruction can positively benefit both DAU students and instructors. . . . Clearly, there are important issues that have not yet been addressed in this media selection guide. These include quantitative cost factors o f the various media alternatives, existing capabilities, and other course characteristics such as enrollment and frequency o f delivery. The report contains no research data on student learning under any of the systems presented. "DSMC Electronic Campus Project (ECP) and Digital Teleaming Network (DTN) Implementation Plans" (1995) identified details for implementing a program of placing DSMC courses into DL constructs defined by the Air University memorandum, however, no provision for the conduct of research to determine course applicability to DL distribution as delineated in the Air University plan is presented. DSMC's identified in its Corporate Plan for 1995 the premise that: W e will move toward the educational philosophy that emphasizes that: (a) learners are in charge o f their learning; and (b) learners must be afforded opportunities to learn within a situation relevant to them. 253 It could be deduced that this direction was adopted following the adult education successes achieved in PMC 95-1, Section C's Project Kaizen. Following a Provost directive previously issued in November 1994, certain staff and faculty first coined the name. "Guided Self-Directed Learning (GSDL)" to best describe the DSMC codified version of adult education. Table 12 contains the of description DSMC's / GSDL. To achieve this goal, certain staff members were directed to accomplish a series of 2 to 3 hours faculty departmental briefings to "get everyone on board." The guidance memorandum states, "The presence o f any o f these characteristics indicates some degree o f GSDL is being used. The DSMC goal is to design, develop, and execute all DSMC courses so that most o f these characteristics are present. The purpose o f GSDL is to help each DSMC student become a more competent Defense Acquisition M anager." Summary: As is readily visible in this series of memos, directives, reports, and military corporate goals, the Congressional mandate of "changing the behavior of the Acquisition Corps" has experienced extreme difficulty in the transitions from the language of the bill through the downward hierarchical implementation by DoD (Pentagon), DAU, Functional Boards, DSMC, FSAM, ISAC, PMC (courses) and finally to the student. 254 255 ENVIRONMENT LLARNKR FACILITATOR COMPETENCIES ASSESSMENT • A lot of prep lime is devoted to providing the "right" environment and guidance for the learner. • A personal learning plan, often called a learning contract, is negotiated between learner and faculty member. • Learning is "situated" (issue-driven), learning situations arc selected for relevancy to the learner • Faculty approach changed from "tell them" to help them assess, and then to plan learning activities, and then to conduct learning activities. Faculty leam along with students. Flexibility is evident in all. • Learning is process-focused (in reference to content & context) • Learning takes place in many different activities, in and out of classroom. Learners learn from many resources, including each other. • learners decides which "classes" to attend. • learners have "offices" or "home Rooms" from which they operate while on campus. • Involvement with real world issues and problems involves a variety of strategies, e.g., small groups, cooperative student projects, simulations, and case studies. • Faculty teams become even more important. May not have mandatory' classrooms. Teaching teams (IPT) offices, ideally, are located together and in close proximity to their assigned students. • Teleleaming. and other educational technology' is available to learners wherever and whenever it might be useful to them. • Learning is enhanced to the extent that threats, coercion, manipulation, direction, and control by others do not characterize the learning situation(s). • Learning is enhanced to the extent that openness, equality, respect and constructive dialogue do characterize the learning situation • The learner is in charge within the competency expectations set by the applicable functional board(s); is responsible for own learning; directly participates in course planning and execution. • Faculty "mentors" (with appropriate facilitator and coaching skills) assist the learner with structuring and carrying out a personalized plan. • I .earner "I/earns how to Learn." Takes active role. • Students leam more; work smarter - devote more time on their own. • Faculty role is facilitating -consulting coaching role. • Faculty have responsibility to guide learners in the preparation of their personal learning plans. • It is the responsibility of the faculty member and DSMC to "create" an env ironment in which the material to be learned is most readily learned by the learner. • Faculty may also participate as a resource. • Faculty gives intro to GSDL, including providing a clear understanding of learner’ s responsibilities, and than begins implementing. (Some students will not understand until they experience it. So move from explaining to doing should be expeditious) • Competencies are useful as standards of what "the field" thinks are tilings the learners need to know. They serve as a starting point for course design and learner self-assessment of learning needs. • Core set of competencies is recognized (must be addressed - not necessarily "taught" - in some fashion. • Competencies should be understood in the learner’ s context and may be supplemented by "what else." The learner will drive the process to provide in order to make the learning meaningful. • Learner needs rather than instructional needs arc paramount. • Assessment is critical. Pre-assessment diagnostic extremely important. Continuous assessment during execution is required. • learner does "front-end" self-assessment (guided by faculty using designed process) • Assessment of success of a learning activity is determined by designing in a closure reflection discussion period (few minutes) at the end of each activity. Don't wait until the end of a module and give a test, that won't tell us what we really need to know. .Also, ask the students whether the readings assigned were useful to them. • Traditional tests for tradition purposes (i.e., evaluating the student) are not used. • Evaluations of process effectiveness and accumulation of evidence, by the learner, of his/her accomplishments in learning are mandatory. • Testing is a useful "tool" to help decide what the learner and the faculty member are going to do next. • The learning process is evaluated rather than the learner. • Learning mastery is continuous (e.g., learning journal: self-assessment; group assessment. DESCRIPTION OF GUIDED SELF-DIRECTED LEARNING ( Chart 12) CHAPTER HI: METHODOLOGY Subjects of Study: The historical biographical data of students attending the DSMC's PMC course indicates that the plurality of the students are white male, late thirties, with fifteen years of service, and have an advanced technical degree. However, with the advent of the DAWIA the MBTI data base is changing as the numbers of military officer students decreases and the student population becomes much more inclusive and diverse. The representation of civil servants, women, Blacks, Asians, Hispanic, and other ethnic ancestry has increased significantly, and the student-body is experienced in the customs, tradition and environment of military acquisition. The protocol for attendance in DSMC / Program Management Courses/s (FSAM / ISAC / PMC), as defined in DSMC's catalog, is nomination by the candidate's Service (Army, Navy, Marine Corps, Coast Guard, Defense Logistic Agency, Office of the Secretary of Defense, and others). By Congressional mandate, ten percent of each class is initially reserved for the Military Complex Industry students. 256 Data Collection: At the time of nomination to PMC, the Office of the Registrar transmits to the candidate an informational packet on the PMC course and personal data forms for completion and return by the candidate. Included in these requests for information are biographical data sheets, the PROFILOR blank 360-degree assessment instrument for distribution to superior/s, peers and subordinates, and the Perry Learning Environment Preferences (LEP) instrument. The PROFILOR and Perry are returned in self-addressed envelopes. Upon receipt of the completed instruments, the data sheets are scanned and/or processed via computers. The Myers-Briggs instrument is completed on the first day of academic duty. The information resulting from these instruments and questionnaires are agglomerated into PMC Section data. Appendix 4 provides a composite summary of PMC 93-1 through PMC 95-1, Perry LEP - CCI and all sections (A-N) of PMC 94-1. Upon examination, notice the scales and zero points differ with the nature of the section. Appendix 5 contains MBTI Type Table data for PMC 93-1 through 95-1 and all sections of PMC 95-1. 257 Instructor Data Set Composite data consisting of Perry LEP and MBTI Type Tables for each section (thirty students) is provided each faculty member by the Department's Academic Department Chairman. As previously discussed, the indicated by the MBTI and Perry Section data each section presents a different and unique problem for the assigned instructors . As an example, referring to Appendix 4, Section F, PMC 94-1, the instructor has differing needs from two different groups. Unfortunately, the curriculum that has been developed in the form of competencies is outcome driven, treating the student as a dependent variable, who at the end of the course of instruction are to retain all the competencies required by DAU. Statistical Base: This study is based on the following statistical data: Instrument Date Commenced Number o f Observations MBTI (norm referenced) July 1985 5463 Pern' LEP (norm referenced) January 1993 1922 PROFILOR (norm referenced) January 1993 1908 / 14,072 (15,980 total observations) Table 13 258 These instruments, MBTI, Perry, and PROFILOR are analyzed for each student and are later agglomerated in various combinations to provide significant cognitive and behavioral data, which yields suggested approaches to a more beneficial educational process. As an example, if the LEP data is coupled with the MBTI Type Tables, an insight is gained into the character of the section and the best approach by the instructor to accomplish the learning task. Unfortunately, individual student data concerning specific MBTI or Perry LEP results can not be distributed because personal computerized data on an individual is controlled by the Privacy Act. Program Management Course, Class or Section, data is not controlled. Students who were unwilling to participate, for any reason, in the completion of any instrument were excused. In some cases, such as the PROFILOR, the student may not receive sufficient 360 degree responses to compute a PROFILOR report. The total possible population for MBTI participation since 1985 is not known but since 1993 the student MBTI response rate has been 100%. The total possible population for Perry LEP and PROFILOR is 2106 and their respective response rates are 91.3% and 90.6%. 259 Research Approach and Designs: In this descriptive study, a number of methods have been utilized in the examination of the research question, "Will the competency based skill set defined for the Program Management Career Field provide the education and training to alter or change the acquisition workforce culture?" The methods exercised in this study are: • A Pilot Project in Adult Education using Nonequivalent Control Group Design / PMC 94-1. • Comparative Data Analysis: FSAM / ISAC / PMC - Perry LEPs • PMC Integrated Data Analysis: MBTIs and Perry LEPs • PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs for PMC 93-1, 93-2, 94-1, 94-2. Pilot Project in Adult Education: The first of these pilot experiments was conducted with a control group and is documented under an unpublished report, "Project Kaizen" (Gould and Fujii, 1994). 260 Contract Learning Background Section C of PMC 94-1 provided for the conduct of a contract learning pilot study. The full report of this pilot study is contained in Gould's and Fujii's 1994, unpublished "Project Kaizen: A Pilot Study of Adult Learning Contracts - A Study of Causal Inference" Portions of that report are extracted for the purpose of this study. Each PMC class is currently comprised of 14 sections (A - N) of 30 students each. Sections C and D have been designated "senior sections" for PMC 94-1 and are made up of senior uniformed personnel (0-6s), senior civil servants (GS / GM - 15s and SESs), and senior industry managers. Sections C and D are similar in terms of their Myers-Briggs Type Indicator (MBTI) preference scores and have a modal (most frequently occurring) ISTJ (Introvert, Sensing, Thinking, Judging) type. Sections C and D are also similar in terms of their Perry LEP mean scores (Section C = 370.7; Section D = 364.6). A "t-test" for independent samples was conducted and yielded a "t" of 0.48, with 48 degrees of freedom. This value is insignificant at the .05 level. Therefore, the null hypothesis 3 70.7 equals 364.6 could not be rejected, and both means are statistically equal. Based upon all comparative elements, Sections C and D are considered equal. 261 The Pilot Study Project A major portion of the PMC curriculum is the Integrative Program Management Course comprised of Experiential Learning (EL), Integrated Subjects (IS), and Grand Slam (GS). It is the supposition that IS and GS classes offer the student the opportunity to review and apply lessons learned on a large acquisition program involving a wide range of programmatic, political, and management issues. When the students in Section C received their initial introduction to IS, a few, who had considerable Program Management Office experience, expressed a desire to try an alternative learning approach based on the Malcolm Knowles concept of learning contracts. The issue was, "How can a learning contract be framed so that the students will achieve the critical acquisition process competencies that would have been covered in IP?" Section C presented a briefing to the Dean, Program Management Education, addressing this issue. The student proposal was accepted and student research prepared learning contracts were negotiated with various members of the faculty department. Certain Managerial Department classes on team building, group dynamics, and research methodology were modified to support the student led learning research. Assigned faculty from the 262 Managerial Department provided additional facilitation and resource support for students challenged by the ambiguity of their learning goals. Additionally, the Managerial Development Department faculty developed a questionnaire to measure resultant effects due to the learning contract approach. Hypotheses: Null Hypotheses: There is no difference in satisfaction between Section C and Section D students due to different approaches in conducting the IS and GS exercise. Alternate Hypotheses: Section C will have a different level of satisfaction from that of Section D due to a different approach in conducting the IS and GS exercise. Experimental Design: Because Sections C and D are similar with respect to the manner in which the students were assigned to each section, i.e. identical ISTJ modal types and similar Perry LEP, both sections are considered similar. Collectively, these characteristics are equivalent to a series of pretests conducted to determine the degree of similarity between the groups. The Nonequivalent Control Group Design was selected to test the null hypotheses (the hypotheses of no difference). 263 Experimental Group (Section C) O, X 0 2 Control Group (Section D) 0 3 0 A Section C is the experimental group and Section D is the control group. The learning contract approach is the experimental treatment (X). Section D, the control group, followed the current IS and GS approaches. Rejection of the null hypotheses will be interpreted as supporting the alternative hypotheses. Methods of Gathering Data: Questionnaire: The Minnesota Satisfaction Questionnaire (MSQ), the Job Description Index (JDI), the Semantic Differential Scale, and Bullock's Scale were examined as potential instruments to measure student satisfaction. Because of the general wording used in these instruments, they were considered inappropriate for the pilot study. As a result, The Managerial Department Chairman, Dr. Donald Fujii, elected to reword the MSQ short form to fit the IS and GS vs Learning Contract situation. During the modification process, every effort was made to preserve the parallelism between the MSQ wording and the modified wording, in order to maintain the original context. The modified questionnaire was reviewed by the Dean, Program Management Education Division, who concurred in the tailored changes. The revised 264 questionnaire, "Student Led Acquisition Management Activities Questionnaire" (Appendix 6 ). yields three measures of satisfaction. Intrinsic Satisfaction is defined as a person's attitude toward an activity or task that is influenced by the activity or task itself Extrinsic Satisfaction is defined as a person's attitude toward an activity or task that is influenced by sources other than the activity or task. General Satisfaction is defined as a person's attitude toward the task or activity in general, and is the composite score of all 20 items on the questionnaire. Data Collection: The principal investigator, Gould, visited each section's class room to administer and gather completed questionnaires. Seven sections were visited (including of Sections C and D). Section (X) was selected at random for inclusion for statistical purposes. Scoring Procedure: The scores from the individual questionnaires were entered on an individual scoring sheet. The scores from 12 of the 20 questions were totaled to yield the Intrinsic Satisfaction score. The scores from 6 of the 20 questions were totaled to yield the Extrinsic Satisfaction score. The scores form all 20 questions were then totaled to yield the General Satisfaction score. The 265 maximum values for Intrinsic, Extrinsic and General Satisfaction are 60, 30 and 100, respectively. Method of Analysis: The raw data was entered into the Statistical Package for the Social Sciences (SPSS) software program and manipulated to produce the relevant statistical data. The satisfaction data from each section was then analyzed with the Kolmogorov- Smirmov Goodness of Fit Test to determine the extent to which each section’ s distribution of scores approximated a normal distribution. The results indicated that all distributions were within the normal range. This normality of distribution must be established before the statistical test - Tukey HSD (Honestly Significant Difference) can be used to test for the significance between the means of Sections C, D, and X. The .05 level of significance is used for the Tukey HSD test. For each section, the mean, mode, kurtosis, SE skew, maximum score, standard error, standard deviation, SE kurtosis, range, median, variance skewness and minimum score of the intrinsic, extrinsic and general satisfaction scores were calculated. 266 In addition to the Tukey HSD test, the raw scores were converted to normalized (percentile) scores, in order to determine the distribution between high, average and low satisfaction scores. Control Group Interactions During this time frame, Section D, the control group, recorded two unusual happenings. For the first time a Senior Section competed against all other sections in the design and building of a model car powered by rat trap springs. This competitive event is called the "Mouse Trap Exercise." The car was designed, fabricated and assembled with the "full" benefit of DoD's acquisition specification and contracting rules and regulations. The object is to allow participating students to see what a contractor encounters when trying to meet government specifications, contract pricing, and control requirements. To most everyone's amazement, Section D won the competition. The second circumstance was a bit different. A student from Section D, observing Section C s adult learning program, determined that he too would embark on his own learning program, but without obtaining proper authority and approval. At graduation he received a letter of reprimand with his diploma. 267 Section C’s Project Kaizen Products After the completion of the satisfaction survey, Section C conducted many briefings for senior DoD personnel on their "adult learning experience" and presented the results (products) of their research. Section C's accomplishments in the form of printed products were: 1. "Project KAIZEN: Review of the Oversight by Congress of DoD Acquisition Programs With the Aim of Continuous Improvement" Technical Report TR 3 - 94, DSMC Press, Fort Belvoir, VA., 1994 2. "RDT & E Budget Exhibit Handbook: Preparation / Review’ Guidelines" DSMC Press, Fort Belvoir, VA., 1994 3. "Better Management with Fewer People", Program Manager Magazine. DSMC Press, Fort Belvoir, VA., September - October 1994 4. "Project Kaizen Looks at Congressional Oversight of Defense Acquisition Programs" Acquisition Review Quarterly. Volume II, Number 1, Defense Acquisition University, Alexandria, VA., Winter '95 Longitudinal Study - Pilot Project Adult Learning PMC 94-1, Section C: One year later, a follow-up questionnaire was sent to the 31 students of Section C, PMC 94-1. Each student was requested to rate five questions on a Likert Scale from 0 to 5 (0 = don't know, 1 = strongly disagree, 5 — strongly agree). In addition, each student was requested to provide a written responses to the following five questions. The questions were: 268 1. The Project Kaizen experience was a value-added experience (5-0). my? W hat, if anything did it add? 2. Specifically, I learned new behaviors as a result o f Project Kaizen (5-0). W hat behaviors did you leam? Have you applied any o f the learned behaviors to your subsequent assignment? 3. I learned new ideas (5-0). How have you applied your new learning to your current assignment? 4. M y attitudes about the acquisition system were changed as a result o f Project Kaizen (5-0). W hat change took place? .5 . I would recom m end Project Kaizen ( 5-0). W hat benefit did you derive? W as the benefit applicable to your next assignment? Qualitative Responses: The qualitative responses for each question have been reduced to the key words and phrases voluntarily expressed by the respondents. Although some words and phrases were mentioned more than once by respondents, the responses are listed only once. 269 Question 1 a. Why was Project Kaizen a value added experience? Team building Peer bonding Established assertiveness baseline Independent research Congressional insight Changed course instruction for the better Real life problems Personal investigations Comprehensive knowledge o f Acquisition Reform principles Empowerment Research o f compelling interest Current issue Question lb. What did Project Kaizen add? Congressional insight Team skills Application o f m anagem ent principles Open discussion with OSD staff Awareness Knowledge o f Acquisition Law reform Understanding o f process o f change Opportunity to suggest researched solutions Question 2a. What new behaviors did you leam? Peer relationships Handling ambiguity Empowerment o f teams Research m ethods Handling diversity (people o f different MBTI types) Contacting Congressional personnel Question 2b. Have you applied learned behaviors in your new assignment? Handling ambiguity Teaming Group development 270 Research m ethods MBTI differences in people Patience Question 3. What new ideas have you applied? Team and group dynamics Increased delegation Increased training o f personnel Acquisition reform dynamics M anagement o f people Keep it Simple Modification o f Acquisition Strategy Understanding o f Congressional oversight Interpersonal relations Process reform Open communications M anagement principles Question 4. What change in attitudes took place? Appreciation o f complex views Appreciation o f Congressional role Understanding o f change Understanding o f system is non-responsiveness Understanding o f DoD obstinacy Reduce m y oversight needs Power o f teams People can make a difference DoD policy is not always correct DoD / Congress need open communications Question 5. What benefit did you derive from Project Kaizen? Knowledge o f Acquisition process Knowledge o f the m anagem ent o f employees Understanding participation o f everyone is essential Knowledge o f the need for streamlining Learned teaming and group dynamics Importance o f academic rigor Understanding o f Congressional / DoD interaction 271 Personal growth Understanding ambiguous situations M ethods o f Self-direction Question 5. W as the benefit applicable to your next assignment? Yes Affected m ethods in part o f m y duties Focused team experience Absolutely Changed the mold Not sure Doing it every day No how to be a team m em ber Definitely Communications break through These unsolicited key word phrases from students indicate a significant cognitive development advancement coupled with a behavioral shift. 272 Data Processing and Analysis: Comparative Analysis: FSAM / ISAC / PMC Perry LEPs: A descriptive pilot study was undertaken to develop a statistic/s useful in the discussion of the application of Blooms' Taxonomy levels as required by the DAU competency matrix (Appendix 1) in the methodology used in the construction of the academic curriculum attendant to the Acquisition Workforce Career Levels I, II and III. COURSE CAREER LEVEL BLOOMS’ LEVEL FSAM I 1 ISAC n 2 PMC m 3-6 Table 14 The level of instruction is set at a specific Blooms’ level by DAU for the teaching of the various Program Management competencies. This outcome-based educational theory is structured around the stimulus response theory of learning. The cognitive level of the students attending these courses is not an element of consideration. A pilot study was initiated to determine the cognitive levels of the students slated to attend specific education and training at all three career levels of education and training, FSAM / ISAC / PMC. 273 Hypothesis: Null Hypothesis: There is no difference in the Perry LEP levels, Blooms' levels, and Career levels for students attending their respective Program Management Courses. Alternate Hypothesis: PMC Level III students will have Perry levels not hierarchically suitable to the Bloom's level of instruction. Alternate Hypothesis: ISAC Level II students will have Perry levels not hierarchically suitable to the Bloom's level of instruction. Alternate Hypothesis: FSAM Level I students will have Perry levels not hierarchically suitable to the Blooms' level of instruction. Experimental Design: Students assigned to attend the FSAM and ISAC Courses were randomly selected by their Service. In a like manner, students assigned to attend the PMC Course were randomly selected by their Service. No post test was accomplished and a control group was not used. Experimental Group IFSAC / ISAC / PMCI X O Methods of Gathering Data: Prior to any course's class, each group was given the Perry LEP instrument. Since an existing data base for students from PMC 93-1, 93-2, 94-1, and 94-2 contained PMC Perry LEP indicators, a random sample lot of three sections (nominal 90 students) was drawn from each PMC. Students enrolled in ISAC 95-1 and 95-2 and FSAM students at Hanscom Air Force Base, Massachusetts; Huntsville, Alabama; and St. Louis, Missouri were given the Perry LEP instrument. Results: The detailed statistical results are contained in Appendix 5. Treated on an individual student basis, the results indicates non-uniformity in the assignment of students to all Program Management Courses based upon their individual LEP, when compared to the Bloom's Taxonomy Level at which the class is being conducted. Course Bloom's TL Location LEP Min LEP Max LEP M ean FSAM 1 St. Louis 217 417 324.4 FSAM 1 Huntsville 242 392 297.0 FSAM 1 Hanscom 217 408 314.6 ISAC 95-1 9 Ft. Belvoir 225 444 346.4 ISAC 95-2 2 Ft. Belvoir 217 440 340.9 275 PMC 93-1 3 - 6 Ft. Belvoir 225 444 363.7 PMC 93-2 3 - 6 Ft. Belvoir 260 450 362.8 PMC 94-1 3 - 6 Ft. Belvoir 217 458 368.3 PMC 94-2 3 - 6 Ft. Belvoir 250 460 365.0 Table 15 Further agglomeration of all students into their respective courses yields. Course Bloom's Tax. Lvl. LEP Mean FSAM 1 312.5 ISAC 2 343.5 PMC 3 - 6 364.7 Table 16 Integrated Analysis MBTI and Perry LEP The instructional strategies defined by Lawrence (1993) distinguish between linear (SJ's) and global learners (NT's). Perry (1970) also distinguishes learning preferences between those who are basic dualism and authority oriented (LEP 2) and those who are self-directed and search for synthesis (LEP 5). The SPSS analytical tools allow a splitting of the Perry LEPs CCI scores into the 9 Transitional levels. Once accomplished, these levels may be statistically analyzed for their respective MBTI preferences content. As an example, all MBTI Type Indicators present at Perry 276 Transition Level 8 are identified. Appendix 7 illustrates this example for all nine Perry LEP CCI Transition Levels and their corresponding MBTI content for 1923 students. Accomplishing this step, a SPSS cross tab report can be generated, which defines on the vertical axis the nine Perry LEP CCI Transition Levels and on the horizontal axis the 16 MBTI Type Indicators. This technique will aid in determining if a single MBTI Type Indicator is dominant at any one Perry LEP CCI Transition Level. As is illustrated by the charts in Appendix 7, the ISTJ's dominate all Perry LEP -CCI transitions levels except level 9. The dominant learning preference of this ISTJ group is teacher directed, linear learning, with defined goals and no ambiguity. In a similar manner, the recode capability of SPSS can be utilized to convert the sixteen MBTI Type Indicators into the four Temperaments (SJ, SP, NF, and NT). The Perry LEP indicators of these defined different indicators are statistically analyzed to determine their means and other statistically relevant data. 277 PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs: When individual PROFILOR reports have been generated for all the students in a PMC course, a group report is then printed. This group report is a composite of the ten highest rated questions compared to the norm (Managerial Strengths) and the ten lowest rated questions compared to the norm (Developmental or Training Needs). The PROFILOR protocol expresses the consideration that Managers succeed because of their managerial strengths and therefore these should be fully developed. In the opposite sense, Managers "derail" for lack of attention to their developmental needs and therefore should be defined as the learning tasks. PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 93-1 The report was generated from 373 self, 640 superiors, 477 direct and 911 peer reports for a total of 2401 observations compared to national norms. Strengths: 1. Leam new information quickly 2. Apply accurate logic in solving problems 3. Confront problems early, before they get out of hand 278 4. Influence and shape the decisions of upper management 5. Speak clearly and concisely 6. Protect confidential information 7. Work constructively under stress and pressure 8. Accept responsibility for own mistakes 9. Pursue learning and self-development 10. Bring cross-disciplinary knowledge to bear on issues and opportunities Developmental (Training) Needs: 1. Delegate enough of own work to others 2. Create an environment where people work their best 3. Develop effective working relationships with direct reports 4. Take people's feelings and preferences into account when making decisions 5. Clarify what people say to ensure understanding 6. Seek out new work challenges 7. Encourage discussion of ethical considerations before decisions are made 8. Deal constructively with own failures and mistakes 279 9. Seek feedback to enhance performance 10. Stay informed about industry practices and new developments PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 93-2 The report was generated from 334 self, 582 superiors, 648 direct and 1496 peer reports for a total of 3060 observations compared to national norms. Strengths: 1. Leam new information quickly 2. Coordinate work with other groups 3. Confront problems early, before they get out of hand 4. Influence and shape the decisions of upper management 5. Live up to commitments 6. Protect confidential information 7. Work constructively under stress and pressure 8. Accept responsibility for own mistakes 9. Pursue learning and self-development 10. Bring cross-disciplinary knowledge to bear on issues and opportunities 280 Developmental (Training) Needs: 1. Delegate enough of own work to others 2. Create an environment where people work their best 3. Let people know when results are not up to expectations 4. Develop effective working relationships with direct reports 5. Tale people's feelings and preferences into account when making decisions 6. Clarify what people say to ensure understanding 7. Seek out new work challenges 8. Encourage discussions of ethical considerations before decisions are made 9. Deal constructively with own failures and mistakes 10. Seek feedback to enhance performance 281 PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 94-1 The report was generated from 361 self, 515 superiors, 807 direct and 1585 peer reports for a total of 3268 observations compared to national norms. Strengths 1. Learn new information quickly 2. Confront problems early, before they get out of hand 3. Influence and shape the decisions of upper management 4. Speak clearly and concisely 5. Protect Confidential Information 6. Project an appropriate degree of self-confidence 7. Work constructively under stress and pressure 8. Accept responsibility for own mistakes 9. Pursue learning and self-development 10. Bring cross-disciplinary knowledge to bear on issues and opportunities Developmental (Training) Needs: 1. Delegate enough of own work to others 2. Create an environment where people work their best 282 3. Develop effective working relationships with direct reports 4. Take people's feelings and preferences into account when making decisions 5. Clarify what people say to ensure understanding 6. Seek out new work challenges 7. Encourage discussions of ethical considerations before decisions are made 8. Deal constructively with own failures and mistakes 9. Seek feedback to enhance performance 10. Stay informed about industry practices and new developments PROFILOR Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs - PMC 94-2 The report was generated from 334 self, 582 superiors, 648 direct and 1496 peer reports for a total of 3060 observations compared to national norms. Strengths 1. Learn new information quickly 2. Coordinate work with other groups 3. Confront problems early, before they get out of hand 283 4. Influence and shape the decisions of upper management 5. Live up to commitments 6. Protect confidential information 7. Work constructively under stress and pressure 8. Accept responsibility for own mistakes 9. Pursue learning and self-development 10. Bring cross-disciplinary knowledge to bear on issues and opportunities Developmental (Training) Needs 1. Delegate enough of own work to others 2. Create an environment where people work their best 3. Let people know when results are not up to expectations 4. Develop effective working relationships with direct reports 5. Take people's feelings and preferences into account when making decisions 6. Clarify what people say to ensure understanding 7. Seek out new work challenges 8. Encourage discussions of ethical considerations before decisions are made 284 9. Deal constructively with own failures and mistakes 10. Seek feedback to enhance performance Summary: The Congressionally directed purpose of the DAWIA was to change the culture of the acquisition workforce "by concentrating on their skills, experiences, education, and training." To accomplish this purpose, DAU has implemented an objective approach: 1. DAU and the Program Management Functional Board have stated clearly and specifically the competencies and knowledge each student must have attained when they have completed their obligatory education and training at the appropriate level. 2. DAU and the Program Management Functional Board have planned the course length and certified the curriculum for each course, FSAM / ISAC / PMC, designed to obtained these objectives. 3. DSMC has selected the measures, Guided Self-Directed Learning, by which the attainment of these objectives are to be achieved. OBJECTIVES -* MEANS -* MEASURES The student attending required education and training has been viewed as an independent variable. However, if system approach were selected, 285 INPUT -* PROCESSING -> OUTPUT 4- FEEDBACK 4- it may have been recognized that the students cognitive, behavioral, and environmental conditioning actually defines the nature of the output obtained. The educational purpose is to improve. The methodology implemented in this descriptive study has been to define the student as an individual cognitively (Perry LEP), behaviorially (MBTI), and environmentally (PROFILOR) and by the means of agglomeration and / or cross tabulations define additional considerations which have significant bearing on educational theory and classroom practice. To validate in part the student models thusly described, a pilot study was accomplished to measure the results of implementing a modified and facilitated learning contract with an experimental PMC section. 286 CHAPTER IV: FINDINGS The findings are presented in two parts. Part One is based upon the statistical quantitative data derived from MBTI, Perry LEP-CCI, PROFILOR, Career Levels, and Bloom's Taxonomy as applied to course curriculum. Part Two is based upon a qualitative analysis utilizing the Gould's Generic Open System Improvement Model as an evaluative diagnostic tool. Statistical Quantitative Data Findings The statistical data packages utilized are: a. Perry Learning Environment Preferences b. Myers-Briggs Type Indicator c. PROFILOR d. Project Kaizen Casual Inference and Follow-up Study e. FSAM / ISAC / PMC Comparative Analysis: Student Perry Learning Environment Preferences f. Competency List for the Program Management Career Field, July 1993. Through the statistical manipulations of the SPSS software the data elements may be combined into relevant statistical definitions or characterizations providing clarity and rationale. 287 The order for presentation is the same established in Chapter 3: Methodology: • A Pilot Project in Adult Education using Nonequivalent Control Group Design / PMC 94-1. • Comparative Data Analysis: FSAM / ISAC / PMC - Perry's LEPs • PMC: Integrated Data Analysis: MBTI's and Perry LEPs. • PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs for PMC 93-1,93-2, 94-1,94-2. • Perry LEP Aggregated and Agglomerated Distribution vs Blooms' Curriculum Level for PMC 93-1 - PMC 95-1. A Pilot Project in Adult Education using Nonequivalent Control Group Design / PMC 94-1 (Project Kaizen) Interpretation of Results and Discussion: To determine the difference between means, the results of the one-way analysis of variance utilizing Tukey's HSD test with a level of significance of .050 indicated that Section C's intrinsic, extrinsic and general satisfaction scores were significantly different from the intrinsic, extrinsic, and general satisfaction scores of both Section D and Section X. The 288 contract learning approach resulted in significantly different and higher satisfaction scores for Section C. The probability of this happening on the basis of chance is 5 out of 100 cases. Therefore, the null hypotheses, which predicted no difference between Sections C and D can be rejected, and the alternative hypotheses, which predicts a difference between Sections C and D, is supported. A possible explanation for the significantly positive satisfaction scores of the students in Section C is found in adult learning theory by Malcolm Knowles and other authors. The results of the normalized (percentile) scores for intrinsic, extrinsic and general satisfaction are presented in Table 17. The students in Section C had a higher percentage of high satisfaction scores (percentile scores of 75 or higher) for intrinsic, extrinsic, and general satisfaction than the students in Section D. The average satisfaction scores (percentile scores of 26 to 74) and low satisfaction scores (percentile scores of 25 or lower) are also presented. Intrinsic Satisfaction Index Section C Section D High Satisfaction 25 (89.3%) 11 (36.7%) Average Satisfaction 3 (10.7%) 17 (56.7%) Low Satisfaction 0 2 (6.6%) 289 Extrinsic Satisfaction Index Section C Section D High Satisfaction 17 (60.7%) 8 (26.7%) Average Satisfaction 11 (39.3%) 21 (70%) Low Satisfaction 0 1 G eneral Satisfaction Index Section C Section D High Satisfaction 21 (75%) 9 (30%) Average Satisfaction 7 (25%) 21 (70%) Low Satisfaction 0 0 n = 28 n = 30 Table 17 Longitudinal Study - Pilot Project Adult Learning PMC 94-1, Section C Twenty of the thirty-one students (64.5%) responded to the questionnaire (5-0). Question Mean 1. The Project Kaizen w as a value-added experience 4.7 2. Specifically, 1 learned new behaviors as a result o f Project Kaizen 4.05 3. I learned new ideas 4.05 4. My attitudes about acquisition system s w ere changed as a result o f Project Kaizen 4.05 5. I w ould recommend Project Kaizen 4.8 Table 18 290 Comparative Data Analysis: FSAM / ISAC / PMC - Perry LEPs To determine the difference between means, a one way analysis of variance applying Tukey - HSD (.05 level of significance) was accomplished. Using the location and course level as the distinguishing criteria, it was found that students attending ISAC in 95-1 and 95-2 or PMC 93-1, 93-2, 94-1 and 94-2 are significantly different from those attending the FSAM courses at St Louis, Huntsville, and Hanscom AFB. Also, ISAC 95-1 and 95-2 students are not significantly different from PMC 93-1, 93-2, 94-1, and 94-2 students, with the exception of PMC 94-1 whose students are significantly different from ISAC 95-1 students. Further agglomeration of all students into their respective courses yields: Course Bloom's TL LEP Mean FSAM 1 312.5 ISAC 2 343.5 PMC 3 - 6 364.7 Table 19 To determine the difference between means, a one-way analysis of variance applying Tukey - HSD (.05 level of significance) was accomplished. 291 The means of ISAC and PMC students are defined as being statistically significantly different from FSAM students, and in a like manner PMC students are significantly different from ISAC students. PMC Integrated Data Analysis: MBTIs and Perry LEPs Integrated Analysis MBTI and Perry LEP The instructional strategies defined by Lawrence (1993) distinguish between linear (SJ's) and global (NT's) learners. Perry (1969) also defines the differences between learning preferences for those who are basic dualism and authority oriented (LEP 2) from those who are self-directed and search for synthesis (LEP 5). The SPSS analytical tools allow a splitting of the Perry LEPs CCI scores into the nine (9) Transitional levels. These levels may then be statistically analyzed for their respective MBTI preferences content. As an example, all MBTI Type Indicators present at Perry Transition Level 8 would be identified along with the specific numbers of each type. Appendix 6 illustrates this example for all nine Perry LEP CCI Transition Levels and their corresponding MBTI content. Accomplishing this step, a SPSS cross tab report can be generated which defines on the vertical axis the nine Perry LEP CCI Transition Levels 292 and on the horizontal axis the 16 MBTI Type Indicators. This technique aids in determining if any single MBTI Type Indicator is dominant at any one Perry LEP CCI Transition Level. As is illustrated by the charts in Appendix 7, the ISTJs dominate by numbers of students at all Perry LEP - CCI Transition Levels except Level nine (9). The dominant learning preference of this ISTJ group is linear learning, teacher directed, defined goals and no ambiguity. However, to determine the difference between means, a one way analysis of variance applying Tukey - HSD (.05 level of significance) was accomplished. SPSS statistically generates a table of differences. The table has been shortened to eliminate non-relevant data. MBTI ISFJ(342.14) ISTJ(357.52) ESTJ(357.80) INTJ (372.98) A A A INTP(376.32) A A A ENTP(3 77.95) A A A ENFP(381.92) A Table 20 The (*) indicates significant differences in Perry LEP-CCI means between MBTI types INTJs, INTPs and ENTPs and ISFJs, ISTJs, and ESTJs. Further ENFPs are different from ISFJs. The total population 293 standard deviation is 49.10. The difference between ISFJ (342.14) and ENFP (381.92) is more than 80% of one standard deviation. Temperament Differences In a similar manner, the recode capability of SPSS can be utilized to convert the sixteen MBTI Type Indicators into the four Temperaments (SJ, SP, NF, and NT). To determine the difference between means, a one way analysis of variance applying Tukey - HSD (.05 level of significance) was accomplished: Temperament SJ SP NT NF SJ (356.6) SP (360.4) NF (370.9) * NT (373.8) * * (*) Indicate significant differences Table 21 To determine the difference between means, a one way analysis of variance was conducted using Tukey-HSD test a significance level of .05. As illustrated, the group Perry LEP - CCI mean of NT's and NF's are significantly different from SJs Additionally, NT's are significantly different from SP's. 294 PROFILOR Summary Group Statistical Assessment of Managerial Strengths and Developmental (Training) Needs for PMC 93-1, 93-2, 94-1, 94-2 and 95-1. An analysis of the PROFILOR Group Statistical Assessment of Managerial Strengths for each individual PMC class (93-1, 93-2, 94-1, 94-2, and 95-1) essentially identifies the same basic strengths and developmental (training) needs. However, when all the PMC classes are agglomerated and re-analyzed, a different pattern emerges. Eight core factors have been found by PDI to be critical to management success. The analysis of the PMC total Group Composite of Strengths and Developmental (Training) Needs displays this distribution. The sample size is 1908 self, 2899 superiors (bosses), 3621 direct and 7552 peers reports for a total of 15,980 observations. 295 PROFILOR Agglomerated Group Report PDI SIJCCHSS FACTORS STRENGTH DEVELO PM ENTAL (TRA INING ) N E E D Administration • Coordinate with Others • D elegate Own Work • Establish H igh Standards Communication • M ake sure people have no "sunrises" Interpersonal Leadership •Confront Problem s Early • Influence and Shape D ecisions • M otivate Others • Coach and D evelop • Champion Change • Are Assertive • Stimulate Others to make Changes and Improvements Motivation • Let people know when Results are not up to Expectations • Display a High Energy Level • Readily put in extra time and effort Organizational Strategy • Bring cross-disciplinary knowledge to bear on issues and opportunities • Stay informed about Industry Practices and N ew Developm ents Self-M anagem ent • Act with Integrity • Demonstrate Adaptability • D evelop O neself • Show consistency between Words and Actions • Work constructively under stress • A ccept responsibility for own mistakes • Pursue learning and self developm ent • Encourage D iscussions o f Ethical Considerations before decisions are made Thinking • Consider alternatives Chart 13 296 Perry LEP Aggregated and Agglomerated Distribution vs Bloom's Curriculum Level for PMC 93-1 - PMC 95-1. Perry (1970) Learning Environment Preferences are defined as being sequential, hierarchical, invariant and revertible under stress. The Perry scale for cognitive development is from 1 through 5. Moore (1987) defined the Cognitive Complexity Index (CCI) as being 100 through 500. Perry and Moore suggest the transition from one level to the next is more important than the level itself. Blooms' Taxonomy for the cognitive domain has six divisions. Bloom's Level 2, Perry's Level 2, and Moore's LEP - CCI 200 represent the lowest level of understanding and apprehension. Utilizing Moore's CCI scale, a recoding of Perry LEPs into the six Bloom's Taxonomy Levels can be accomplished: BLOOM'S NOM ENCLATURE PERRY LEVEL NOMENCLATURE MOORE's CCI - RANGE 1 - Knowledge 1 - Basic Dualism 2 - Comprehension 2 - Multiple-perspective 200 - 240 3 - Application 3 - Multiple-context 241 - 327 4 - Analysis 4 - Multiple-understand 3 2 8 -4 1 6 5 - Synthesis 5 - Search for Synthesis 4 1 7 -4 6 1 6 - Evaluation 462 - 500 Table 22 297 The student populations adjusted by the use of Moore's CCI Conversion Range values will yield a Bloom's / Perry's Transition Conversion Levels from 2 through 6. The total PMC population of 1922 students can be recoded into the Bloom's Learning Preference Level (BLPL): STUDENT'S DEMONSTRATED LEARNING LEVEL FREQUENCY % o f Student Population 2 28 1.5 405 21.1 4 1239 64.5 5 230 12.4 6 19 .6 Table 23 Competencies are taught at required Bloom's Levels 2 through 6. A list of competency requirements for all class elements of each functional academic area is published by the Executive Secretary of the Acquisition Management Functional Board. (Stewart, 1993) Reviewing all classes, competency level requirements for the PMC course yields: 298 BLOOM'S INSTRUCTIONAL LEVEL NUMBERS OF ACADEMIC CLASSES % OF 121 COMPETENCIES 2 19 15.7 3 32 26.4 4 40 33.1 5 19 15.7 6 11 9.1 Table 24 If the assumption is made that student demonstrated learning levels are correct and that learning levels are sequential, hierarchical, invariant and revertible under stress, student academic difficulty will occur. • Level 2 students (1.5%) have difficulty with all Bloom's instructional level courses conducted at teaching level 4 and up (57.9%). • Level 3 students (21.1 %) have difficulty with Bloom's instructional level courses conducted at teaching level 5 and up (24.8%) • Level 4 students (33.1 %) have difficulty with Bloom's instructional level conducted at level 6 (9.1%) 299 Initial Summary: The findings presented are for the most part the result of the statistical manipulations of various data elements: • Competency List as a function of Bloom's Taxonomy Levels Program • Management Course Levels (FSAM - 1, ISAC - II, and PMC - III) • Perry LEP as measured by Moore's Cognitive Complexity Index • Myers-Briggs Type Indicator and PROFILOR Group Report Data This data was augmented by additional data from the pilot study, Project Kaizen. Contained within the data base are other elements which could be used for the generation of descriptive evaluation information. Such elements of data include unifonned officer vs. civil servant; Military Branch of Service (Anny, Navy, Air Force, Marine Corps or Coast Guard); military rank or civil service designator, gender, length of total service, length of service in the Acquisition Workforce. These subsets of descriptive data could provide insight on the nature of other important relationships such as, cognitive development as a function of length of service, or variances of Myers-Briggs distributions as a function of military service and so forth. However, the 300 focus of this study was centered on the developing the best set of data to answer the research question. Qualitative Data Findings Gould's Open Systems Generic Model Gould's Open Systems Generic Model, Graph 2, is utilized as the model for the generation of qualitative findings. Under each generic model heading, the applicability to the DAWIA is established and the research documented postures and/or processes are presented. External Environment The Constitution requires Congress to raise and support the Annies and to provide and maintain a Navy. After lengthy research, investigations, and Congressional testimony, Congress developed a consensus finding that, ". . .the culture of the acquisition workforce must change. . . . But there is no way Congress can legislate standards of character for admission to the acquisition Corps. So we will concentrate on their skills, their experience, their education, and their training." Congress views the "acquisition culture of organized anarchies" to be caused by lack of definable accountability, special interests politics, and lack of working knowledge of the system. 301 302 Gould's Generic Open Systems Improvement Model EXTERNAL STAKEHOLDERS TRENDS SOCIETAL S U P P i L : INPUT I DETERMINATES E \ R ORGANIZATIONAL PU RPO SE ► TRANSFORMATION PROCESS TECHNICAL C U S \ T \ O M OUTPUT ) E EVALUATIONS R / ▲ w S 0 C I A L ^ ▲ w v BOUNDARIES F E E D B A C K \_ BOUNDARIES _ ENVIRONMENT FEEDBACK Graph 2 Stakeholders Congress The key points of the Congressional mandate were not only the changing of the acquisition workforce culture but also: Bring Civil Servants into the Program Managers position. Require that only Acquisition Corps members hold acquisition critical positions. Direct the establishment of DAU. Require the Undersecretary of Defense for Acquisition to oversee uniform policies in administration of Acquisition Workforce. Career and Functional Boards Maintain a clear authority as to the education and training outcome objectives. Sixteen DAU Consortium Members Define accession requirements for career fields DAU Appointed by Congress to fulfill the m ission,". . . to be the center for all acquisition education, training, and research, a focal point for acquisition excellence that will oversee instruction in the whys and wherefore 303 of acquisition as well as promote research leading to improvements in acquisition management. A place where seminars and thought - meetings - intellectual ferment - can help to change the mind set, to bring about a lasting culture change, throughout the acquisition system." (added emphasis) T rends Threat The definable security threat to the United States in the mid 1990s is primarily from Terrorist States such as South Korea, Libya, Iran and Iraq. Down Sizing With the lack of a definable massive threat, the US military will be drawn down to a smaller, more deployable, unified armed service. Budget The DoD expenditure dollar will continue to be reduced. 304 Societal Continued reduction in acquisition dollars will produce anxiety in the military products industrial complex causing more consolidations and efforts to find new or expanding markets. Acquisition Workforce personnel will suffer from an inability to understand and/or be able to do anything about Congressionally mandated changes. Organizational purpose Transformation Process Technical and Social Defense Systems Management College Establish a curriculum for an education and training program for Level I, II, and III Program Managers, in accordance with the time-frame restrictions imposed from an external source and in accordance with the competencies established by Program Management Career Board, that will accomplish the Congressional Mandate of changing the behavior of the acquisition workforce's skill, experience, education and training. 305 Acquisition Reform Communications Center "The ARCC goal is to CHANGE BEHAVIOR BY: * Communicating a common Acquisition Reform message to the people we need to engage for lasting success. * Ensuing consistency o f that message. * Promoting a rapid, effective communication process. * Focusing on learning not broadcasting * Promoting use o f success stories, lessons learned, incentives and recognition programs." (DAU, 1995) Curriculum Appendix 1, "Competencies & Bloom Level Required for Certification in the Program Management Career Field - Consortium Baseline" defines a goal oriented, stimulus response curriculum. However, the number of "academic" hours that should be devoted to functional subject matter areas is not defined. By fiat, the PMC course, commencing with PMC 95-2, has been cut from 20 to 14 weeks duration. The number of academic hours allocated to each functional department was determined by the career management function board for the Program Management Career field. These hours were further adjusted by the PMC Course Director to accommodate administrative activities. 306 Faculty The art of acquiring weapons of war is gained through technical applied knowledge coupled with experience. The design and production of a tank is significantly different from a car or truck. The faculty at DSMC is highly skilled in the individual functional discipline for which s/he has been trained and spent the majority of their career. The majority of the DSMC faculty, either uniformed officer or civil servant, are selected to teach in a specific academic functional area because of their functional discipline, practitioner experience, and technical knowledge. In Program Offices, briefings conducted for decision-making purposes are frequent and are the normal method of operation. The faculty members are very adequately trained and experienced in conducting a "decision brief," and they tend to teach their classes in the same manner. There is little comprehensive knowledge of teaching theory and mechanics of the classroom. The faculty is representative of the Acquisition Workforce and follows the same MBTI and Perry LEP distribution patterns as exhibited by the Acquisition Workforce in general. 307 Input Supplier Nominated by their respective Armed Services Directors of Acquisition Career Management (DACMs), the students attending any Program Management Course will by DAU mandate be authenticated by the use of a single source for managing training requirements, quotas, and class reservations. This administrative process will also provide a means for maintaining course schedules, monitoring quota utilization, and managing and reviewing reporting requirements. (Crean, 1995) DAU relies on the DACMs to send "qualified" students to DAU classes. Since no class pre-assessment occurs prior to class attendance, instructors have been encouraging some students to leave the course of instruction. Students will not be directly encouraged by instructors to leave a course of instruction. Instructors may recommend but only the school head has decision authority. (Crean, 1995) Student Body The student body is characterized by the Myers-Briggs Type Indicator, Perry LEP - CCI Index, and PROFILOR. As pointed out by Issac and Michael (1981), there is a significant risk using conclusions based upon 308 group data as it relates to laws o f behavior and an individual However, in this descriptive study the individuals have segregated themselves by self report into their self-selected preferences. Each of these preferences, whether it be MBTI or Perry, has a defined set of research guiding principles associated with instructional methodology. Lawrence (1993), and Barr and Barr (1994) have provided instruction strategies for each MBTI type. Perry (1968), Knefelkamp (1974,1981), and Moore (1987) have furnished challenge and support techniques for each cognitive level. The PROFILOR has defined the managerial strengths and developmental (training) needs by individual and group reports. Feedback from the PMC Course to the Student Only the Level III PMC course has a comprehensive feedback system developed through the Managerial Development department. The feedback each student receives are class discussions on MBTI and PROFILOR. Individual counseling is also available for each student. As a student selective item, a comprehensive three year career plan may be accomplished. 309 Customer The DACMS who nominate the student, the Service from where the student came, and the Command in which the student serve all are customers of the "transformational process" performed by DSMC in the education and training of the Program Management students in FSAM / ISAC / PMC. Output Measurement Evaluation DAU Student Assessment Process Guide A criterion based structure based upon Bloom's Taxonomy and its associated action verb/s used to: 1. Gather via testing quantitative information about the degree to which competency or ability is present 2. Make judgments regarding appropriateness. Render evaluative decisions either quantitatively or qualitatively is answer to, "Has this student demonstrated the skills necessary to become a Program Manager?" (DAU, 1995) Feedback from the Student to DSMC / PMC PMC 95-1 was requested to complete a survey to help DSMC identify key items to improve classroom teaching skills. O f the 420 students queried, only 109 (26%) responded, severely limiting the usefulness of this feedback. 310 The top ten assessment items identified by students to improve the classroom, in precedent order, are: 1. Correct Information 2. Time distribution logical and realistic 3. Adequate time provided for each element 4. Create, arouse, and maintain student interest and enthusiasm 5. Logically organized into an integrated whole 6. Recent changes in regulation and/or practices 7. Enthusiastic, energetic, and dynamic 8. High quality current (Fujii, 1995) 9. Build on experience / expertise of class participants 10. Real life references 11. Varied and stimulating. Feedback Output To Input When students graduate from PMC, they are expected to perform to task in the Program Office. If they are incapable of doing so, DSMC is criticized for non-performance of the task of "educating and training the acquisition workforce." If the numbers of PMC graduates in any course (FSAM / ISAC / PMC) do not equal the number of personnel the DACMs require, DSMC is criticized for non-performance in making "through-put quotas." Such criticism is reflected into the External Environment and to DAU specifically. 311 Partial Summary DAU / DSMC has implemented a set of actions to accomplish the DAWIA task mandated by Congress. Viewing the findings from a qualitative and quantitative open system perspective raises the research question? That Question is: The Congressionally stated purpose of the DAWIA was to change the culture of the acquisition workforce "by concentrating on their skills, experience, education and training." Will the competency based skill set defined for the Program Management Career Field provide the education and training to alter or change the acquisition workforce culture? 312 CHAPTER V: SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS SUMMARY: To some, the length of the Cold War with the former Soviet Union would be measured from the Berlin Blockade in the 1950s to the collapse of the Berlin Wall some 40 years later. To others viewing the threats of the prospect of nuclear proliferation and other methods of mass destruction now available, the World has become a more dangerous place than it was in the Cold War. During this time frame, the American public has seen its confidence in the Armed Services go through many peaks and valleys. In the late 1940's, the expressed public sentiment was pride in the "citizen soldier." However, as the Cold War developed and the conflicts of Korea and Vietnam shaped perceptions of the military, the concept of a warrior class protecting the United States from the Soviets and eminent mutual mass destruction was bom. The overriding philosophy of a potential massive Soviet nuclear strike drove the United States to react, especially during the Reagan Administration, and to arm its military with the best technical equipment available. A resultant mind set soon developed: "If enough money is thrown at it, any 313 weapon system technical problem can be solved, military superiority will occur, and battle-field supremacy will be maintained." Congress played its role with the passage of the military appropriation and authorization bills until the mid 1980s, when the appearance ($500.00 hammer) and actual occurrence (Operation 1 1 1 Wind) of military procurement scandals started to surface in the media. An embarrassed Congress started the long and arduous process of building an environmental consensus of all the stakeholders in order to revise long standing problems in military procurement. Drawing on expertise from academia, military industrial complex leaders, GAO, The Controller General, Inspector Generals, the Pentagon, and their own Staff/s, Congress set about to discover the essence of the issues governing and the controlling resolution of problems in the military acquisition conundrum, problems that had eluded them for many years. As related in the literature review of Chapter II, Congress came to the realization that, to attack the root cause, it would be necessary to change the environmental culture in which military acquisition had embedded itself. To that end, Congress passed the DAWIA with the stated intent, "... to change 314 the culture by concentrating on their skills, their experience, their education and their training. "(Mavroules, 1990) The Congressional view of the "Acquisition Culture" was that of "organized anarchies," caused by the lack of definable accountability, special interest politics, and a lack of knowledge of the system. On a positive note, the majority of those uniformed officers and civil servants who are attracted to the acquisition workforce and self-select the occupation are noted for their desire to belong to a meaningful institution. They are stable and traditionalists in their work practices and ethics, and they dislike disorganization and disarray (Kroger, 1981). By updating historical Myers- Briggs Type Indicator data available in the files from July 1985, a composite MBTI picture was published. (Bryant et al., 1991). This data base is updated semi-annually as each new PMC class arrives. The literature review of Chapter 2 is that body of knowledge surrounding the Jungian theory and how it relates to individual and group behavior. Lawrence's (1993) discussion on MBTI's influence on learning as a function of type, provides a clarifying depiction defining the dichotomy between those who, on one hand, are global theorists and those who, on the other hand, prefer linear direct experience tactics. 315 If the "acquisition culture" is to be changed, it must first be known where it is, a benchmark of known dimension and measurement. Perry's (1968) effort at Harvard and Radcliffe as expanded by Knefelkamp (1974, 1981) and as further refined by Moore (1987), provided an instrument having significant bearing on the question of learning preferences and cognitive development. The literature review in this area concentrated on individuals and group learning dynamics, noting the classroom differences between those who could only see and accept "black or white", "wrong or right" and "this not that" statements from their instructors, versus those who, at the other end of the classroom spectrum manifest the DSMC motto, "It All Depends." Perry provided the jumping off place to delve into the theories of adult learning, examining the question of true self-direction versus directed learning. It was found that a measurable portion of students are truly not self directed and would be unable or would fail to respond to a facilitated curriculum as opposed to a curriculum that is a directed, stimulus response, outcome-defined, and goal oriented. With individual and group cognitive (Perry LEP) and behavioral (MBTI) benchmarks, the remaining side of the triangulation is the environmental factor measured and normalized by the PROFILOR. This 316 method of developing 360 degree multi-rater data is not an element of the professional military or civil servant preview. The data base itself, with a nominal 16,000 observations of 2,000 mid-career level acquisition personnel, can be used as a significant statistical base to define training needs for strengths and development. But more importantly, it represents the first time in the career of most acquisition workforce professionals, that they have a solid definable base upon which to develop a three-year improvement plan and to monitor their progress. The literature search of this domain reflects the emerging nature of 360 degree multi-rater systems and their ever expanding application into accession, team structuring and corporate placement. Professional Military Education Professional Military Education (PME) services supports every military officer. These are specialized programs designed to guide, if not at times force, career military officers into the right school/college at the right time. Such a personal service does not exist for the civilians in the acquisition workforce. The Congressional findings that military officers were not experienced, educated or trained, as required by law before the DAWIA, could occur 317 again. It comes to the question as to how the officer's position description is written. It answers the question, "When is a Program Manager not a Program Manager?" Congress determined that the mix between programs managed by uniformed versus civil servants did not have proper balance. Again, based on the absence of professional PME counseling service for civil servants, this is still true today. Despite the Congressionally mandated goals, no office has been established to insure that civil servants get "their fair share of Program Managers assignments." In fact, in all services, there is no "PME" type office for civil servants nor one that would integrate them with the established military PME offices. (Drinkwater et al., 1994) 318 Reframing and Reformatting to Fit the Environment Congress declared the task of changing the culture to be "a long and arduous task." There is every appearance that DoD / DAU / DSMC have refrained and reformatted the Congressional intent of the DAWIA by treating hierarchical problems and by centralizing power in DAU where it could have been decentralized by empowering DSMC. By the flavor of the very directives and policies it has issued, it would be difficult to consider DAU as a, " . . intellectual centerpiece of the entire acquisition system - a place where seminars and thought-provoking meetings - intellectual; ferment - can help to change the mindset, to bring about a lasting cultural change, throughout the acquisition system." Behavioral Change / Teams? If DoD / DAU / DSMC accepted and understood the Congressional tasking of changing the behavior of the Acquisition Workforce, the reorganization of the 20 week PMC course into a 14 week format as well as the reconstituted ISAC and FSAM courses, would have a significantly different curriculum and course syllabus. In 1991 Bryant found in his DSMC / PMC research that teaming was a most important and neglected issue. 319 Utilizing the Johnston and Packer data, Bryant predicted a need for a far greater emphasis on team building in the Program Mangers Office. Examining the empirical quantitative and qualitative data gathered by this research, Bryant's prediction was not only on target but also correct in estimating the resistance to change in the acquisition environment. Kroger's MBTI argument is, "We clone our own." Therefore if ISTJs dominate a Program Office, those who advance into the positions of influence will be ISTJs and if not ISTJs certainly ESTJs. It can be inferred from McAleer's posture on promotions after ICAF, that he would concur with Kroger's findings. The Perry LEP - CCI self-select distribution reviewed with Knefelkamp's Learner Characteristics, as implied by the Perry Scheme, defines that those in Perry LEP CCI 200 to 300 view peers as not being a legitimate source of knowledge or information. Until peers are legitimately trusted (Perry LEP CCI 363 and up), can a true integrated team be built? Covey's and Bennis's stated predilection is: "The foundation of any team is trust!" If the technical challenge, reduced funding, indeterminate time differentials and Pentagon down sizing influence Secretary of Defense Perry 320 to issue his directive on Integrated Product Teams, why were those academic hours dedicated to achieving a cultural change by teaching the empowerment of teams deleted from the Managerial Development curricula of the 14 week PMC course? The Conference of Learners on "leaming-how-to learn" conducted in 1986, found that in a fifteen week course, the students first full week should be spent in understanding the concept of "leaming-how— to-leam. "(Smith, 1988) The successful pilot program conducted by Gould in PMC 94-1, 94-2, and 95-1 replicated this concept. The longitudinal data gathered from PMC 94-1 identified "teaming" as a behavioral change carried from the PMC classroom into the new assignments. This change when viewed against the PROFILOR developmental (training) needs, listed in the group report demonstrates a dramatic behavioral, cognitive, and cultural paradigm shift. The four needs are: • Motivate Others • Coach and Develop • Champion Change • Stimulate Others to make Changes and Improvements Following the success of the adult learning theory replicated in PMC 94-1, hierarchical pressure was exerted within DSMC to extend and repeat 321 the effort in other areas of the college. However, in this instance as well as in many others, the organizational structure was placed in charge. In an analogous sense, Mavroules spoke out against Program Managers who were appointed because of their outstanding performance as a pilot or battalion commander. He demanded a cultural change by appointing Program Managers who were experienced in industrial management, further setting a specific goal of a greater civil servant participation in critical Program Office position. If Mavroules' proclivities were extended to the hierarchical structure of DAU / DSMC, experienced educators would be appointed to the critical positions, as opposed to the uniformed officers now filling these positions. Having attained the military rank of full colonel or general does not automatically endow one with the knowledge to be a great course director or educator. Guided Self-Directed Learning A specific example of this approach is illustrated in the DSMC endorsed, "Description of Guided Self-directed Learning (GSDL)." The plan developed by an ad-hoc committee, chaired by a retired colonel, is a good attempt. If compared to the relevant literature in the field authored by the 322 subject matter experts, the plan is lacking in its understanding of fundamental key issues. An element not considered is that approximately one third of the student population, based upon their stated learning preferences, would be unable to be beneficially responsive, either cognitively or behaviorally. Another element is the faculty itself. Faculty selection is based on expert functional knowledge in the discipline of the department. Selection is not based upon the ability to teach. GSDL requires an instructor who is educated, trained, and experienced in the instructor-facilitation art. There is no facilitator educational program for faculty at DSMC. CONCLUSIONS: Based upon Fox's and Krikorian's "Critical Issues in the Defense Acquisition Culture," the staffs of Senator Roth and Congressman Kasich prepared a bill introduced in 1995 for the dissolution of DAU / DSMC in favor of privatization. The driving force behind the bill appears to be short term economic impact with no regard for the circumstances of the passage of the DAWIA. The rather unique responsibilities of the Program Office in the development and production of war materials places it in a class not amenable to the formalities of a college academic environment, nor is it amenable to the 323 vicissitudes of a consultant speciality educator group. To be current in the teaching of DoD procurement policy and budget, not to mention understanding lessons learned from on-going programs, a strong informational bond must exist between DSMC's functional departments, their Service counterparts in the Pentagon, and in the field. Because this information is acquisition sensitive, such a communications bond would probably be illegal if an outside contractor became responsible for delivery of the PMC courses. RECOMMENDATIONS The primary purposes of the DAWIA is to improve the professionalism of the Acquisition Workforce by changing the culture, eliminating management duty assignment abuses, expanding civil servants' roles into critical positions and conducting focused educational and training programs. The pilot model experiment occurring in PMC 94-1, 94-2, and 94-1 replicates academically proven approaches to adult learning. The data developed from these series provides every indication that having the first week as a fixed unalterable time set aside for teaching "leaming-how-to learn" provides the challenge and support necessary to motivate and move the class as a whole 324 (30 students) into the non-ambiguity mode necessary to expand cognitive and behavioral awareness. One-on-one individual feedback and counseling utilizing PROFILOR allows the instructor-facilitator to bring to the student's attention a method for planning an individualized improvement program. Prior to this step, each faculty member would be required to enroll in facilitator-instructors education and training course. Because of their own individual behavioral and cognitive preferences, it is recognized that one third of the instructors would not desire to be student facilitator-instructors. However, to understand the nature of the education and training program, all instructors would be required to understand the methodology to be employed. The program recommended would closely approximate, "The Metfessel- Michael Paradigm Involving Multiple Criterion Measures for Evaluation of the Effectiveness of School Programs" (Issac & Michael, 1990) Paradigmidic Cultural Change Mavroules stated, "the changing of the Acquisition Workforce culture is a long and arduous task." Lewin described change as an "unfreezing, change, and refreezing." If the Acquisition Workforce is to undergo cultural change via education and training, the DAU / DSMC curricula and 325 competencies must be restructured to accomplish this mandate. The environment of the group controls the culture of the group. To change group culture requires that individual members of the group be initiated into the change cycle. From the quantitative data presented, it would appear that one third of the students will readily adapt and embrace change and another third will assimilate the change after they have had a first-hand non-threatening success in an academically controlled environment. The final one third will not achieve behavioral or cognitive growth and may never change due to propensities and proclivities of their self-select preferences. Restructuring DAU / DSMC in the words of M avroules," ... by reducing or eliminating any built-in structural bias favoring unifonned or retired military" is a vital consideration, because a cultural change cannot be accomplished if reenforcement of the existing cultural norms persist. RESEARCH QUESTION The purpose of this study is to explore and examine the academic environment, structure, and content of these three Program Management courses in the context of the adult learning needs and preferences of the students, recognizing that the Congressional intent of the Defense Acquisition 326 Workforce Improvement Act (DAWIA) is to change the long standing DoD culture. It is the conclusion of this descriptive study that, based upon the DAU / DSMC issued directives and established curricula, the cultural change mandated by Congress will not be achieved. If significant restructuring taking into account the basic intent and of the passage of the DAWIA were accomplished, a more favorable short-term outcome could be accomplished. As Congress stated, "Culture change is a long and arduous task." 327 BIBLIOGRAPHY Acker, David D. (1986). A history of the Defense Systems Management College. Fort Belvoir, VA. Adams, Maurianne & Marchesani, Linda S. (1992). Curricula innovations: Social diversity as course content. In Adams, Maurianne (Ed.), Promoting diversity in college classrooms: Innovative responses for the curriculum, faculty, and institutions (pp. 85-97). San Francisco: Jossey-Bass. Adams, Scott. (1995). Gadfly of the high-tech workplace. Technology Review, pp. 22-29. Argyris, C., Putnam, R., & Smith, D. M. (1990). Action science. San Francisco: Jossey-Bass. Argyris, Chris. (1991, May-June). Teaching smart people how to leam. Harvard Business Review, pp. 99-109. Armbruster, VickR. (1994, September-October). Better management with fewer people. Program Manager, pp. 23-28. Ashmawy, Saad. (1972). Consortium revans. Journal of European Training. 1, 53-56. Bader, Gloria E., & Bloom, Audrey E. (1992, June). How to do peer review. Training & Development, pp. 61-66. Barker, Joel. (1994). Paradigm hunting: How to explore the future more effectively. Washington, DC: The Quality Learning Services Division of The U.S. Chamber of Commerce. Barrett, Gerald V., & Depinet, Robert L. (1991, October). A reconsideration of testing for competence rather than for intelligence. American Psychologist. 46(10). 1012-1024. 328 Baumgartner, Jerry. (1994, June). Give it to me straight. Training & Development, pp. 49-51. Baxter Magolda, Marcia B. (1987, July). A rater-training program for assessing intellectual development on the Perry scheme. Journal of College Student Personnel. 28(4), 356-364. Bennett, Roger. (1990). Effective set advising in action learning programmes. Journal of European Industrial Training. 14 (7). Bennett, Milton J. (1994). Towards ethnorelativism: A developmental model of intercultural sensitivity. Education for the Intercultural Experience. Bloom, Benjamin S., Editor. (1956). Taxonomy of educational objectives: Cognitive domain. New York: Longman. Bloom, Benjamin S., Krathwohl, David R., & Masia, Bertram B. (1956). Taxonomy of educational objectives: Affective domain. New York: Longman. Boddy, David. (1981). Putting action learning into action. Journal of European Industrial Training. 5(5), 39-52. Bowman, James S. (1994). At last, an alternative to performance appraisal: Total quality management. Public Administration Review. 54(2), 129- 135. Bracken, David W. (1992, June). Benchmarking employee attitudes. Training & Development. Brookfield, Stephen D. (1986). Understanding and facilitating adult learning. San Francisco: Jossey-Bass. Brookfield, Stephen D. (1987). Developing critical thinkers: Challenging adults to explore alternative wavs of thinking and acting. San Francisco: Jossey-Bass. 329 Brookfield, Stephen D. (Speaker). (1987). Creating dynamic adult learning experiences: Stephen D. Brookfield interviews Alan B. Knox and Leonard Nadler (Vol. 1 & 2). San Francisco: Jossey-Bass. Biyant, C., O'Callaghan, K. W., Buron, R. J., Lorenzen, G. L., & Roffey, A. E. (1989). The cognitive development of officer candidates. Colorado Springs, CO: United States Air Force Academy. Bryant, C., & et al. (1991). A description of psychological type at the Defense Systems Management College. Fort Belvoir, VA: DSMC/FD-MD. Brockett, R. G., & Hiemstra, R. (1991). Self direction in adult learning: Perspectives on theory, research, and practice. London & New York: Routledge & Kegan Paul. Brown, Dianne C. (1994). Subgroup norming: Legitimate testing practice or reverse discrimination? American Psychologist. 49(11), 927-928. Budman, Matthew, & Rice, Berkeley. (1994, February). The rating game. Across the Board, pp. 35-38. Candy, PC. (1989, February). The transition from leamer-control to autodidaxy: More than meets the eye. Paper presented at the 3rd North American Symposium on Adult self-directed learning. Carr, David. (1992, Spring). Cultural institutions as structures for cognitive change. New Directions For Adult And Continuing Education, pp. 21- 35. Carskadon Tom & McCarley. (1993). Research. Bulletin of Psychological Type, pp. 18. Chan, Heng Chee. (1994, November 24). Paradigm shift, paradigm found? Far Eastern Economic Review, pp. 36. 330 Chapla, Daniel B., & et al. (1994). A description of psychological type at the Defense Systems Management College. Fort Belvoir, VA: DSMC/FD- MD. Charles, Keith. (1994, September). Reducing cycle time. Washington, DC: Department of the Army. Chickering, Arthur W. (1984). Integrating liberal education, preparation for work and human development: Implications for teacher education. Memphis, Tennessee: Memphis State University. (ERIC Document Reproduction Service No. ED 241 483) Chickering, Arthur W. (1990). The Modem American College. San Francisco, CA: Jossey-Bass. Claxton, C. S., & Murrell, P. H. (1987). Learning styles: Implications for improved educational practices. Washington, DC: Association for the Study of Higher Education. (SAHE-ERIC Higher Education Reports No. 4). Clevette, Rick & Davis, Brian. (1990). Employee feedback and the development of managers at Honeywell. Human Resources Professional. 2(2), 18-20. Cowles, H. D. (1993). Acquisition career management program. Washington, DC: Department of Defense. Congress. (1990). Title XII— Defense acquisition workforce. Washington, DC: U.S. Government Printing Office. Congress. (1995). To amend title 10. United States Code, to modernize Department of Defense acquisition procedures, and for other purposes. Washington, DC: U.S. Government Printing Office. Congressional Record - Extensions - Part II Section., Remarks by Mavroules (D-MA), June 28, 1990 331 Congressional Report. (1990). 101st Congress 2d Session. Investigations Subcommittee of the Committee on the Armed Services. House of Representatives. “The Quality and Professionalism of the Acquisition Workforce". Washington, DC: U.S. Government Printing Office. Congressional Report. (1990). Life is too Short: A Review of the Brief Periods Managers of Major Defense Acquisition Programs Stay on the Job. Washington, DC: U.S. Government Printing Office. Covey, Stephen R. (1989). The 7 habits of highly effective people: Powerful lessons in personal change. New York: Fireside. Crean, Thomas M. (1995). Student Assessment Process: Guide. Alexandria, VA: Defense Acquisition University. Crean, Thomas M. (1995, May). Release of students from courses. Alexandria, VA: Defense Acquisition University. Crean, Thomas M. (1995, May). Class reservations/rosters administrative policy. Alexandria, VA: Defense Acquisition University. Cummings, William H. (1990). Age Group Differences and Estimated Frequencies of the Mvers-Briggs Type Indicator Preferences. Davis, B. L., Skube, C. J., Hellervik, L. W., Gebelein, S. H., & Sheard, J. L. (1992). Successful manager's handbook: Development suggestions for today's managers. New York: Personnel Decisions, Inc. DeCoster, David A. & Mable, Phyllis. (1981). Understanding today’ s students. San Francisco: Jossey-Bass. Defense Acquisition University. (1993). Competency List for the Program Management Career Field. Washington, DC. Defense Acquisition University. (1994). Minutes of the Defense Acquisition University Consortium: Core Curriculum Working Group Meeting. Washington, DC. 332 zs\ Defense Acquisition University. (1995). Software Engineering Institute - Report on distance learning technologies. Washington, DC. Defense Acquisition University. (1994). Title V - Acquisition management. Washington, DC. Defense Acquisition University education media selection guide: Summary report. (1995). Alexandria, VA: Defense Acquisition University. Deming, Edwards W. (1993). The new economics: For industry. government, education. Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study. Department of Army. (1994). Reducing cycle times. Washington, DC. Department of Defense. (1991). Career Development Program for Acquisition Personnel. DoD 5000.52-M. Washington, DC. Department of Defense. (1991). Reporting Management Information on DoD Military and Civilian Acquisition Personnel and Position. DoDI 5000.55. Washington, DC. Department of Defense. (1993). Acquisition Career Management: Mandatory Course Fulfillment Program and Competency Standards. ADS- 93-01-GD. Washington, DC. Department of Defense. (1993). Acquisition Career Management Program. DoD 5000.58-R. Washington, DC. Department of Defense. (1994). Minutes of the Defense Acquisition University consortium core curriculum working group meeting. Washington, DC. Department of Defense. (1994). Acquisition workforce core course developments. Washington, DC. 333 Department of Defense. (1995). Re-engineering the acquisition oversight and review process. Washington, DC. Department of Defense. (1995). DSMC 95 implementation plan. Fort Belvoir, VA. Department of Defense. (1995). Use of integrated product and process development and integrated product teams in DoD acquisition. Washington, DC. Department of Defense. (1995). Guided self-directed learning. Washington, DC. Diamond, Robert M. (1989). Qualities of the liberally educated person. Designing and improving courses and curricula in higher education (pp.233-246). Distance learning curriculum analysis & media selection. (1994). AL: Air University Maxwell. Dunnette, Marvin D. (1993). My hammer or your hammer. Human Resource Management. 32, 373-383. Drinkwater, Thomas H. E., & et al. (1994). An analysis of the Department of Defense Acquisition workforce level III certification process. Fort Belvoir, VA: Defense Systems Management College. Drucker, Peter F. (1994, November). The age of social transformation. The Atlantic Monthly, pp. 53-80. Edwards, M. O. (1985). Doubling idea power: A program in creative/innovative problem-solving and decision-making. Palo Alto, CA: Idea Development Associates. Fitch, Peggy & Gulver Richard S. (1984). Educational activities to stimulate intellectual development in Perry's scheme. Proceedings of the ASEE Annual Conference. 1-6. 334 Fitch, Peggy & Kansas, Lawrence. (1988). Workshop: Rational curriculum design. Proceedings of the ASEE Annual Conference. 563-568. Fitzgerald, Catherine R. (1993, March). The Myers-Briggs type indicator and leadership development: Reflections on levels of interpretation. Psychological Press. Fitzgerald, Catherine R. (1994). The Myers-Briggs type indicator and leadership: An international research conference. Conference Proceedings. MD: Center of Adult Education , University of Maryland. Fox, J. Ronald. (1989). Obstacles to improving the defense acquisition process. Cambridge, MA: Harvard Business School Press. Fox, J. Ronald, & Field, James, L. (1988). The defense management challenge. Cambridge, MA: Harvard Business School Press. Fox, J. R., Hirsch, E., Krikorian, G., & Schumacher, M. (1994). Critical issues in the defense acquisition culture: Government and industry views from the trenches. Fort Belvoir, VA: DSMC Executive Institute. Freud, Sigmund. (1961). Civilization and its discontents. New York: W. W. Norton & Company. Fry, Prem S. (1992). A consideration of cognitive factors in the learning and education of older adults. International Review of Education. 38(4), 303-325. Fujii, Donald S. (1993). Results of PROFILOR survey. Fort Belvoir, VA: Defense Systems Management College. Fujii, Donald S. (1994). Ideas on teaching adults. Fort Belvoir, VA: Defense Systems Management College. 335 Fujii, Donald S. (1995). Faculty assessment system (FAS1 survey results. Fort Belvoir, VA: Defense Systems Management College. Funke, Gail., & Wayson, Billy. (1991). Strategy, power and influence. The Federal Executive Institute. Gadeken, Owen C., & Cullen, Bernard. (1990). A competency model of program managers in the DoD acquisition process. Fort Belvoir, VA. Gargiulo, Martin. (1993). Two-step leverage: Managing constraint in organizational politics. Administrative Science Quarterly. 38. 1-19. Garrison, D. R. (1991). Critical thinking and adult education: A conceptual model for developing critical thinking in adult learners. International Journal of Lifelong Education. 10(41. 287-303. Garvin, David A. (1993). Building a learning organization. Harvard Business Review, pp. 78-91. Gebelein, Susan H. (1994, June). Multi-rater performance appraisal: The promise, the pitfalls, and the steps. Society for Human Resource Management Conference. St. Louis, MO. Gladis, Stephen D. (1993). Write type: Personality types & writing styles. Amherst, MA: Human Resource Development Press. Glaser, Robert. (1987). The nature of expertise. Columbus, OH: The National Center for Research in Vocational Education. Glasser, William M.D. (1984). Control theory: A new explanation of how we control our lives. New York: Harper & Row. Glasser, William M.D. (1990). The quality school: Managing students without coercion. New York: Harper & Row. Glasser, William M.D. (1991, July). What is a quality school? The quality school training program, pp. 70-73. 336 Golden, Daniel. (1994, M y). Building a better brain. Life Magazine, pp. 63-69. GofFm, Richard D., & Jackson, Douglas N. (1992). Analysis of multitrait- multirater performance appraisal data: Composite direct product method versus confirmatory factor analysis. Multivariate Behavioral Research. 27(3), 363-385. Gottfredson, Linda S. (1994). The science and politics of race-norming. American Psychologist. 49(11), 955-963. Gould, Jay W. III. (1994). A description of psychological type at the Defense Systems Management College: 1994 Spring Edition. Fort Belvoir, VA. Gould, Jay W. I ll, & Fujii, Donald S. (1994). Contract learning pilot study: Project kaizen. Unpublished paper, Defense Systems Management College, Fort Belvoir, VA. Grow, G. (1991). Teaching learners to be self-directed: A stage approach. Adult Education Quarterly. Harris, John W., & Baggett, J. Mark. (Eds.). (1992). Quality quest in the academic process. AL: Samford University. Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993). The impact of 360- degree feedback on management skills development. Human Resource Management. 32, 325-351. Hirsch, Edward. (1994). Mission requirements. Fort Belvoir. VA: Defense Systems Management College. Hirsch, Edward. (1995). Establishment of the acquisition reform communications center fARCCY Fort Belvoir, VA: Defense Systems Management College. 337 Hirsch, Edward. (1995). DSMC electronic campus project IECP1 and digital teleleaming network IDTN) implementation plans. Fort Belvoir, VA: Defense Systems Management College. Hirsch, Edward. (1995). guided self-directed learning. Fort Belvoir, VA: Defense Systems Management College. Hirsch, Edward. (1995). Use of integrated product and process development and integrated product teams in DoD acquisition. Fort Belvoir, VA: Defense Systems Management College. Hirsch, Edward. (1995). Training. Fort Belvoir, VA: Defense Systems Management College. Hirsh, Sandra Krebs. (1994). MBTI Team building program: Team member's guide. MBTI Training Symposium. FL: Consulting Psychologist Press. Hogan, R., Curphy, G. J., & Hogan, J. (1994, June). What we know about leadership. American Psychologist. 49(6), 493-504. Holley, Irving Brinton, Jr. (1964). Buying aircraft: Material procurement for the army air forces. Washington, DC: Office of the Chief of Military History Department of the Army. Holt, Katherine Elizabeth. (1994, August). Understanding managerial feedback and self-development: Does readiness make a difference? Unpublished doctoral dissertation, University of Minnesota. House of Representatives. (1990). National Defense Authorization Act for Fiscal Year 1991. HR 4739. Washington, DC: U.S. Government Printing Office. House of Representatives. (1990). National Defense Authorization Act for Fiscal Year 1991: Conference Report HR 4739. Washington, DC: U.S. Government Printing Office. 338 Howard, A., Byham, W. C., & Hauenstein, P. (1994). Multirater assessment and feedback. NJ: Development Dimensions International. Isaac, Stephen., & Michael, William B. (1990). Handbook in research and evaluation. San Diego, CA: Edits Publishing. Isaksen, Scott G., Puccio, Gerard J., & Treffinger, Donald J. (1993). An ecological approach to creativity research: Profiling for creative problem solving. The Journal of Creative Behavior. 27(3), 149-169. Isaksen, Scott G., & Treffinger, Donald J. (1985). Creative problem solving: The basic course. Buffalo, N Y : Bearly. Jackson, Conrad N. (1987). Organizational development annual: Vol. 1. Entry: Beginning the OD consultation process. VA: American Society For Training And Development. Jackson, Conrad N. (1988). Organizational development annual: Vol. 2. Contracting for organization development consultation. VA: American Society For Training And Development. Jackson, Conrad N. & Manning Michael R. (1990). Organizational development annual: Vol. 3. Diagnosing client organizations. VA: American Society For Training And Development. Jackson, Conrad N. & Manning Michael R. (1992). Organizational development annual: Vol. 4. Intervening in client organizations. VA: American Society For Training And Development. Jacob, Rahul. (1995, April). The struggle to create an organization. Fortune Magazine, pp.90-99. Jaques, Elliott. (1989). Requisite organization: The CEO's guide to creative structure and leadership. Cason Hall & Co., Publishers. Johnson, Collie J. (1995). Secretary Preston underscores dramatic changes in DoD’ s acquisition arena. Program Manager, pp. 35-40. 339 John-Steiner, V. (1985). Notebooks of the mind: Explorations of thinking. Albuquerque, NM. Jones, Merrick L. (1990). Action learning as a new idea. Journal of Management Development. 9(5). Johnson, Collie J. (1995, March-April). Secretary Preston underscores dramatic changes in DoD's acquisition arena. Program Manager. pp. 35-40. Jung, C. G. (1968). Analytical psychology: Its theory & practice. New York: Vintage Books. Kandel, Eric R., & Hawkins Robert D. (1992, September). The biological basis of learning and individuality. Scientific American, pp. 78-86. Kasworm, Carol E. (1983). An examination of self-directed contract learning as an instructional strategy. Innovative Higher Education. Kasworm, Carol E. (1983). Towards a paradigm of developmental levels of self-directed learning. Austin, Texas: University of Texas, Department of Curriculum & Instruction. (ERIC Document Reproduction Service No. ED 230 705) Keirsey, David & Bates, Marilyn. (1978). Please understand me: Character & temperament types. CA: Prometheus Nemesis Book. Kiechel, Walter III. (1989). When subordinates evaluate the boss. Fortune Magazine. Knefelkamp, L. (1974). Developmental instruction: Fostering intellectual and personal growth in college students. Unpublished doctoral dissertation. University of Minnesota, MN. Knefelkamp, L., Widick, C., & Parker C. A. (1978). Applying new developmental findings. San Francisco, Jossey-Bass Inc., Publisher. 340 Knefelkamp, L. (1981). Developmental instruction: A process model for instructional design. Unpublished paper, Center for Application Development Instruction, Farmville, VA. Knox, Alan B. (1995). Learning and change in the adult years: A developmental perspective. San Francisco: Jossey-Bass. Kolb, David A. (1984). Experiential learning: Experience as the source of learning and development. NJ: Prentice-Hall. Krackhardt, David & Hanson Jeffrey R. (1993). Informal networks: The company behind the chart. Harvard Business Review, pp. 104-111. Kroeger, Otto & Thuesen, Janet M. (1988). Type talk: The 16 personality types that determine how we live, love, and work. New York: Dell Publishing. Lado, Augustine A. & Wilson, Mary C. (1994, October). Human resource systems and sustained competitive advantage: A competency-based perspective. Academy of Management Review Langford, David P. & Cleary, Barbara A. (1995). Orchestrating learning with quality. WI: ASQC. Lawrence, Gordon. (1982). People tvpes & tiger stripes (3rd ed.). Gainesville, FL: Center for Applications of Psychological Type. Lee, Chris. (1990, April). Talking back to the boss. Training. Lee, Elizabeth. (1995, March 8). Fairfax park takes turn as alien planet. The Fairfax Journal, pp. A l, A8. Levinson, D. J., Darrow, C. N., Klein, E. B., Levinson, M. H., & McKee, B. (1978). The seasons of a man's life. New York: Ballantine Books. 341 Long, Huey B., & Associates. (1990). Advances in research and practice in self-directed learning. OK: Oklahoma Research Center for Continuing Professional and Higher Education of the University of Oklahoma. Making a world of difference. (1995). Minneapolis, MN: Personnel Decisions. Margerison, Charles. (1987). Action learning for managers. Journal of Management Development. 18(3), 57-63. Mavroules, Nicholas. (1990). Press release - Key points of acquisition workforce bill. Washington, DC. Mavroules, Nicholas. (1990). Floor statement - general debate: Acquisition workforce. Washington, DC. Mavroules, Nicholas. (1991). 1991 DoD Procurement Conference. Williamsburg, VA. Maynard, Micheline. (1994, August 3). Evaluations evolve from bottom up. USA Today. McAleer, George R. (1993, March). Leadership in the military. Psychological Press. McCall, M. W. Jr., Lombardo, M. M., & Morrison, A. M. (1988). The lessons of experience: How successful executives develop on the job. New York: Lexington Books. McCarthy, Bemice. (1980). The 4 mat system: Teaching to learning styles with right/left mode techniques. Oak Brook, IL: EXCEL McClelland, David C. (1973). Testing for competence rather than for "intelligence". American Psychologist. 28UL 1-14. 342 McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do self attributed and implicit motives differ? Psychological Review. 96(4), 690-702. McGarvey, Robert & Smith, Scott. (1993, March). When workers rate the boss. Training. Messick, S. (1984). The nature of cognitive styles: Problems and promise in educational practice. Educational Psychologist. Meyers, Chet & Jones, Thomas B. (1993). Promoting active learning: Strategies for the college classroom. San Francisco: Jossey-Bass. Mills, Daniel Quinn., & Friesen, Bruce. (1992). The learning organization. European Management Journal. 10(21 146-156. Mitchell, Bryant & Abdul-Rahman, Khalil. (19 ). Organizational learning and the management process. Unpublished report. Mocker, Donald W., & Spear, George E. (1982). Lifelong learning: Formal, nonformal. informal, and self-directed. Columbus, OH: National Center Publications. (ERIC Document Reproduction Service No. ED 220 723). Moore, William S. (1994). Perry scheme of intellectual & ethical development. Olympia, WA: Perry Network. Moore, William S. (1986). Forms of intellectual and ethical development in the college years: A scheme. Cambridge, MA. Moore, William S. (1987). The learning environment preferences: Establishing preliminary reliability and validity for an objective measure of the Perrv scheme of intellectual development. Unpublished doctoral dissertation, University of Maryland, Maryland. 343 Moore, William S. (1988). The learning environment preferences: An instrument manual. Olympia, WA: Center for the Study of Intellectual Development. Moore, William S. (1990). Tuning in to student voices: Assessment & the Perry scheme of intellectual & ethical development. Workshop material prepared for the Fifth AAHE Assessment Forum. Washington, DC. Moore, William S. (1993). Perceptions of learning environment. Olympia, WA: Center for the Study of Intellectual Development. Morrison, Ann M. (1992). The new leaders: Guidelines on leadership diversity in America. San Francisco: Jossey-Bass. Moses, J., Hollenbeck, G. P., & Sorcher, M. Other people's expectation. Human Resource Management. 32. pp. 283-297. Motorola University. (1995). Catalog and schedule of training and education programs. Schaumburg, IL: Galvin Center for Continuing Education. Mumford, Alan. (1985). A review of action learning. Management Bibliographies & Reviews. 11(2), 3-15. Mumford, Alan. (1984). Effectiveness in management development. Journal of Management Development. 3(2), 29-37. Neimark, Edith D. (1987). Toward a cross-cultural examination of adult thought. Rutgers University. (ERIC document Reproduction Service No. ED 291 869) Nowack, Kenneth M. (1993, January). 360-degree feedback: The whole story. Training & Development, pp. 69-72. O'Reilly, Brian. (1994, October). 360 feedback can change your life. Fortune. 344 Pardee, Ronald L. (1990). Motivation theories of Maslow. Herzberg. McGregor & McClelland. (ERIC Document Reproduction No. ED 316 767) Paradigm hunting: How to explore the future more effectively. (1994). The Quality Learning Services Division of the U.S. Chamber of Commerce. Pavett, Cynthia M., & Lau, Alan W. (1983, March). Managerial work: The influence of hierarchical level and functional specialty. Academy of Management Journal 26(1), 170-177. Penderghast, Thomas. (1993). Management and organization development. Bulletin of Psychological Type, pp. 17-18. Perryr , Jr. William G. (1968). Forms of intellectual and ethical development in the college years. Cambridge, MA: Personnel Decisions, INC., PROFILOR®. Minneapolis, MN, 1993 Posner, Barry Z. (1987, March). What it takes to be a good project manager. Project Manager Journal. 18(1). 51-53. Process improvement: The DSMC approach. (1994). Fort Belvoir, VA: Defense Systems Management College. Program management course: 94-2 competencies. (1994). Fort Belvoir, VA: Defense Systems Management College. Provost, Judith A., & Anchors, Scott. (Eds.). (1987). Applications of the Myers-Briggs type indicator in higher education. CA: Consulting Psychologist Press. Pun, Aaron. (1990). Action learning for trainers' development: A design for postgraduate studies. Journal of European Industrial Training. 14(9). Quenk, Naomi L. (1993). Personality types or personality traits: What difference does it make? Bulletin of Psychological Type, pp. 9-13. 345 Ramos, John. (1994, March). Producing change that lasts. Across the Board,pp. 29-33. Ramsay, Martin L., & Lehto, Howard. (1994, July). The power of peer review. Training & Development, pp. 38-41. Reed, Richard H. (1994, November). Adult learning. Fort Belvoir, VA: Defense Systems Management College. Reed, Richard H. (1994, November). Ideas on teaching adults. Fort Belvoir, VA: Defense Systems Management College. Reibstein, Larry. (1988, June 13). Firms ask workers to rate their bosses. The Wall Street Journal. Reveans, R. W. (1982). What is action learning? Journal of Management Development. 1 (3), 21-27. Rimer, Edward. (1993). Organization theory and Frederick Taylor. Public Administration Review. 53(3), 270-272. Roodin, P. A., Rybash, J. M., & Hoyer, W. J. (1986). Qualitative dimensions of social cognition in adulthood. Educational Gerontology. pp. 301-311. Rybash, J. M., Hoyer, W. J., & Roodin P. A. (1986). Adult cognition and aging. New York: Pergamon. Sackett, Paul R., & Wilk, Steffainie L. (1994). Within group norming and other forms of score adjustment in pre-employment testing. American Psychologist. 49,(11), 929-954. Scanlon, John. (1993, September). Leadertypes, part 1: The NF. The Type Reporter, pp. 1-7. Scanlon, Susan. (1994, December). Older and wiser, part 2. The Type Reporter, pp. 1-4. 346 Scanlon, Susan & Scanlon, John. (1994, February). Leadertypes, part 2: The NT. The Type Reporter, pp. 1 -7. Scanlon, Susan & Scanlon, John. (1994, April). Leadertypes, part 3: The SP. The Type Reporter, pp. 1-7. Scanlon, Susan & Scanlon, John. (1994, July). Leadertypes, part 4: The SJ. The Type Reporter, pp. 1-5. Schein, Edgar H. Process consultation: Its role in organization development (2nd ed.) (Vol. 1). Reading, MA: Addison-Wesley Publishing Company. Schein, Edgar H. Process consultation: Lessons for managers and consultants (Vol. 2). Reading, MA: Addison-Wesley Publishing Company. Shapira, Zur & Zevulun, Eli. (1989, April). Performance ratings in organizations: A facet analysis interpretation of the multitrait- multirater approach. Multivariate Behavioral Research. 24(2), 209- 232. Shapiro, Nancy S. (1985). Rhetorical maturity and Perry's scheme of cognitive complexity. MD: University of Maryland, English Department. (ERIC Document Reproduction Service No. ED 255 935) Simesk, Hasan & Louis, Karen S. (November/December, 1994). Organizational change as paradigm shift. Journal of Higher Education. Vol 65, No. 6, Ohio State University Press. Simons, Robert. (1995). Control in an age of empowerment. Harvard Business Review, pp. 80-88. Skube, Carol. (1993). Profile certification workshop. Minneapolis, MN: Personnel Decisions. 347 Smith, Kenwyn K. (1989). The movement of conflict in organizations: The joint dynamics of splitting and triangulation. Administrative Science Quarterly. (34), 1-20. Smith, R. M. (1982). Learning how to leam: Applied learning theory for adults. Chicago: Follett. Smith, R. M. (Ed.). (1987). Theory building for learning how to leam. Chicago: Educational Studies Press. Stewart, Jesse. (1993, June). Competency list for the program management career field. Fort Belvoir, VA: Defense Systems Management College. Stogdill, Ralph M., (1959). Individual Behavior and Group Achievement. New York, NY., Oxford University Press Summers, Lynn. (1991, July). Upside-down performance appraisals. Training & Development, pp. 15-22. Sylwester, David L., & Harris, John W. (1992). Quality improvement in higher education: An overview (RR 91-14). Tennessee, Knoxville: Center for Assessment Research and Development. Szilagyi, Andrew D. Jr., & Wallace, Marc J. Jr. (1987). Organizational behavior and performance (4th ed.). Glenview, IL: Scott, Foresman and Company. Taylor, Kathe., & Moore, Bill. (1984). The Perry scheme & its implications for higher education. Olympia, WA. Taylor, Kathe. (1993). The dilemma of difference: The relationship of intellectual development, self-esteem, and racial identity to tolerance for diversity. The rise of the knowledge worker. (1995). Civilization, pp. 19. 348 Tomow, Walter W. (1993). Introduction to special issue on 360-degree feedback. Human Resource Management. 32. pp.211-219. Tough, A. (2nd. ed.). (1979). The adult's learning projects: A fresh approach to theory and proactive in adult learning. Toronto: Ontario Institute for Studies in Education. Tribus, Myron. (19 ). TOM in schools of business and of engineering. Hayward, CA: Exergy, Inc. United States Army, Manual for Courts-Martial. Washington, D.C. 1954 United States Code 10 - Chapter 87, "Defense Acquisition Workforce", Washington, D.C. 1994 Vamey, Glenn H. (1989). Building productive teams. San Francisco: Jossey-Bass Publishers. Velsor, Ellen Van & Leslie, Jean Brittain. (1991). Feedback to managers: A guide to evaluating multi-rater feedback instruments (Wol. 1). Greensboro, NC: Center for Creative Leadership. Velsor, Ellen Van & Leslie, Jean Brittain. (1991). Feedback to managers: A review and comparison of sixteen multi-rater feedback instruments (Vol. 2). Greensboro, NC: Center for Creative Leadership. Villela-Minnerly, Lucia. (1986). To think well, be aware and not beware of contradictions. Chicago: Chicago State University. (ERIC Document Reproduction Service No. ED 290 857) What does it take to get to the top? (1989, August). USA Today. Yockey, Don. (1993). Acquisition career management: Mandatory course fulfillment program and competency standards. Washington, DC: The Under Secretary of Defense. 349 DEFINITIONS 1. Acquisition. The planning, design, development, testing, contracting, production, introduction, acquisition logistics support, and disposal of systems, equipment, facilities, supplies, or services that are intended for use in, or support of, military missions. 2. Acquisition Career Program Board tACPBV Board that advises DoD Component acquisition executives in managing the accession, training, education, and career development of military and civilian personnel in the acquisition workforce, in selecting individuals for an Acquisition Corps, and in waiving certain requirements pennitted by law or regulation. 3. Acquisition Corps. A subset of DoD Component's acquisition workforce, composed of selected military and civilian personnel in grades of Lieutenant Commander, Major, General Schedule and/or General Manager (GS/GM) 13 and above, who are acquisition professionals. There is one Acquisition Corps for each Military Department and one for all the other DoD Components (including the OSD and the Defense Agencies). 4. Acquisition Experience. Experience gained while assigned to an acquisition position. Also includes intern, exchange, education or training with industry, and other acquisition developmental assignments. Includes 350 experience in DoD acquisition positions and in comparable positions outside the Department of Defense. (No more than 1 year of a period of time spent pursuing a program of academic training or education may be counted as acquisition experience). 5. Acquisition Logistics. The process of systematically identifying, developing, assessing, purchasing, and upgrading logistics requirements through the acquisition process. 6. Acquisition Organization. An organization, including its subordinate elements, whose mission includes planning, managing and/or executing acquisition programs which are governed by DoD Directive 5000.1, (reference (a)), DoD Instruction 5000.2, (reference (b)), and related issuances. Specifically: Office of the Under Secretary of Defense (Acquisition). 7. Acquisition Positions. Civilian positions and military billets that are in the DoD acquisition system, have acquisition duties, and fall in an acquisition position category established by the USD(A). While most frequently located in organizations having an acquisition mission, acquisition positions are also 351 located in management headquarters organizations, management headquarters support organizations, and other organizations. 8. Acquisition Position Categories. Functional subsets of acquisition positions. There are fourteen acquisition position categories: Program management; Program management oversight; Communication-computer systems; Contracting (to include contracting); Industrial property management; Business, Cost estimating & financial management; Auditing; Quality assurance; Manufacturing & production; Acquisition logistics; Systems planning, Research, development & engineering; Test & evaluation engineering; Education, training & career development. 9. Acquisition Program. A directed, funded effort that is designed to provide a new or improved material capability in response to a validated need. (See DoD Directive 5000.1, reference (a)). Acquisition Programs are managed in accordance with reference (a), DoD Instruction 5000.2, reference (b), and related DoD issuances. 10. Acquisition Workforce. The personnel component of the acquisition system. The acquisition workforce includes permanent civilian employees and military members who occupy acquisition positions, who are members of an Acquisition Corps, or who are in acquisition development programs. 352 11. Career Advisors. Persons designated by individual commands or organizations who are typically responsible for maintaining a current knowledge of career development programs, the sources of and availability of mandatory training, and administrative procedures for obtaining funding and training. 12. Career Development. The professional development of employee potential by integrating the capabilities, needs, interests, and aptitudes of employees participating in a career program through a planned, organized, and systematic method of training and development designed to meet organizational objectives. It is accomplished through the combination of work assignments, job rotation, training, education, and self-development programs. 13. Career Field. One or more occupations that require similar knowledge and skills. There are twelve acquisition Career Fields: Program management; Communications - computer systems; Contracting (to include contracting for construction); Purchasing (to include procurement assistant); Industrial property management; Business, cost estimating & financial management; Auditing; Quality assurance; Manufacturing & production; 353 Acquisition logistics; Systems planning, research, development & engineering; Test & evaluation engineering. 14. Career Levels. Groupings of education, training, and experience standards that provide the framework for progression within a career field. There are three career levels: (I) entry or basic; (II) intermediate; (III) senior. 15. Career Paths. The range of opportunities at each career level and the optimum pathways for vertical and horizontal movement within a career field. 16. Certification. A process through which it is determined that an individual meets all the education, training, experience standards established for his or her acquisition career field or position or for membership in an Acquisition Corps. 17. Certified School. A training organization, certified by the USD(A) or designee, that has the capacity to carry out various functions related to acquisition education and training, including developing and revising curriculum, training instructors, maintaining a student database, and delivering mandatory courses. 18. Certified Offeror. An organization or activity that has been approved by the USD(A) or designee as having the capability, appropriate resources (trained instructors, facilities, course materials, etc.), and quality standards to 354 conduct a given mandatory course(s). Certified offerors utilize materials developed by sponsoring schools and are not typically schools themselves. 19. Continuing Education and Training. A mandatory education or training standard established by a Functional Advisor or Functional Board, which is determined to be essential for maintaining currency in a career field and must be accomplished by members of the acquisition workforce in the career field for which the standard is established regardless of the individual's certification level. 20. Contracting Officer. A person with the authority to enter into, administer, and/or terminate contracts and make related determinations and findings. 21. Critical Acquisition Position. Those senior positions carrying significant responsibility, primarily involving supervisory or management duties, in the DoD acquisition system. Those positions are designated by the Secretary of Defense, based on the recommendations of the DoD Component Acquisition Executives, and include any acquisition position required to be filled by an employee in the grade of GS/GM 14 or above, or military grade 0-5, or above. Also specifically includes all the Program Executive Officers (PEOs), 355 the Deputy PEOs, the PMs and the Deputy PMs for major defense acquisition programs, and the PMs of significant non-major programs. 22. Deputy Program Manager fPMY The person who has continuing authority to act on behalf of the PM in his or her absence. 23. Director of Acquisition Career Management (DACM). The official appointed to assist a Component acquisition executive in the performance of his or her duties as they relate to the training, education, and career development of the acquisition workforce of that Component. 24. DoD Component Acquisition Executive fCAEY A single official within a DoD Component who is responsible for all acquisition functions within that Component. This includes Service Acquisition Executives for the Military Departments and Acquisition Executives in other DoD Components who have acquisition management responsibilities. 25. DoD Functional Boards. Chartered by the Under Secretary of Defense (Acquisition) and will be chaired by the functional chief/advisor to the USD(A) for that career program. Membership will consist of senior functional advisors from the Components. The functional boards will recommend mandatory training for the applicable function including required 356 curricula/course content to meet the mission needs of that functional community. 26. Employee. DoD civilian and military personnel. The terms "member," "personnel," and collectively "acquisition workforce" as used in the context of this Manual are synonymous with the term "employee." 27. Equivalent Course or Program. A course or program that has been determined to contain the appropriate level of knowledge and that would enable individuals who take the course or program to perform as if they had completed a designated, mandatory course under the Defense Acquisition Education and Training Program. 28. Functional Board or Advisor. A designated group of senior executives in a functional area that acts in an advisory capacity on issues of career development for its functional area. Functional Boards can exist at the Component level and at the DoD level. At the DoD level, acquisition functional boards are subordinate boards to the ACPB and advise the USD(A). In the absence of a chartered Board, a senior DoD official may perform these duties acting as a Functional Advisor. 29. Individual Development Plan flDP). A document used to lay out an individual's education, training, and experience needs for orderly progression within the career field and to plan for the fulfillment of these needs. It is a 357 combined effort of the civilian employee, supervisor, career advisor, and employee development specialist, as appropriate. 30. Major Defense Acquisition Program IM DAPf An acquisition program that is not a highly sensitive classified program. 31. Major Milestone. Milestone that marks the completion of a significant phase in an MDAP. Such milestones must be the same as the milestones contained in the baseline description established for the program. 32. Mandatory DoD Acquisition Course. A course of study that has been identified by the USD(A) as meeting an established DoD education and training requirement. These courses provide a common, non-component- specific foundation of knowledge for each acquisition function. Each of the following courses is mandatory to qualify for certain assignments, or both: a. Career-development Mandatory Course. A course that must be taken for an employee to be certified at Level I, II or III within one of the career fields. Example: Government Contract Law is mandatory for Level II of the Contracting and the Manufacturing & Production career fields. b. Qualification Mandatory Course. A course that must be completed for an employee to be eligible to perform certain duties or to be given certain assignments. 358 33. Non-Major Defense Acquisition Program (ACAT IIP. Programs not meeting the criteria for Category I or II that have been designated Category III by the Component Acquisition Executive. 34. Non-Major Defense Acquisition Program fACAT IVV All other acquisition programs for which the milestone decision authority should be delegated to a level below that required for category III. 35. Program Executive Officer (PEOY A military or civilian official who has primary responsibility for directing several acquisition category I programs and for assigned acquisition category II, III, and IV programs. A PEO has no other command or staff responsibilities within the Component, and only reports to and receives guidance and direction from the DoD CAE. 36. Program Manager (PM ). A military or civilian official who is responsible for managing an acquisition program (reference (b)). With respect to a major or significant non-major defense acquisition program, the term "program manager" means the member of an Acquisition Corps responsible for managing the program, regardless of the title given the member. 37. Program Office. An acquisition office with the mission to plan, manage, or executer and acquisition program. 359 38. Senior Contracting Official. A director of contracting, or a principal deputy to a director of contracting, serving in the office of the Secretary of a Military Department, the headquarters of a Military Department, the headquarters of a Defense Agency, a subordinate command headquarters, or in a major systems or logistics contracting activity in the Department of Defense. 39. Service Acquisition Executive (SAE). The civilian official in a Military Department who is designated as the SAE for DoD Regulations and Procedures for a SAE for that Military Department. 40. Sponsoring School. A training source with overall responsibility for a specific course of instruction that is part of the curricula of the Defense Acquisition University structure. Responsibilities include development and maintenance of course materials, course reviews, and assessment of core competencies associated with mandatory courses. Sponsoring schools are designated by the USD(A) with the recommendation of the appropriate functional board or advisor. 360 EDUCATION AND TRAINING of the DEPARTMENT OF DEFENSE ACQUISITION WORKFORCE under PUBLIC LAW 101 - 510 U.S. CODE, TITLE 10 APPENDIXES by Jay William Gould III A Dissertation Presented To The Faculty o f the School o f Public Administration o f the University o f Southern California In Partial Fulfillment o f the Requirements for the Degree o f Doctorate in Public Administration August 1995 ©1995 Jay William Gould III TABLE OF CONTENTS APPENDIXES: Appendix 1. Competencies & Bloom's Level Required for Certification in the Program Management Career Field -- Consortium Baseline.............................................361 Appendix 2. DoD Acquisition Career Management: Mandatory Course Fulfillment Program and Competency Standards, January 1993.....................................................393 Appendix 3. Perry Learning Environment Preferences Instrument......................................................... 410 Appendix 4. Perry LEP and CCI Statistics Total PMC 93-1 through 95-1...........................................416 Histograms: PMC 94-1 / Sections A through N .............420 Appendix 5. MBTI Frequency Distribution Total PMC 93-1 through 95-1...........................................448 Frequency Tables APMC and PMC 95-1 Sections A through N.........................................................450 Appendix 6. Student Led Acquisition Management Activities Questionnaire...................................................................... 464 Appendix 7. Cross Tabs Tables and Bar Charts Perry LEP / CCI Distributions w/ MBTI 467 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Acquisition Policy Level I Competence Level I I Competence Level III Competence □ Ac RELATE OTHER SERVICES APPROACHES TO ACQUISITION MANAGEMENT TO THAT OF OWN SERV ICE D escribe other services ap p roach es to acquisition m anagem ent to th at of own service. Bloom 1 S u b S e t S chool H i § S . C om pare oth er services a p p ro ach es to acquisition m an ag em en t to that of own service. Bloom ■ p « r 2 "S u E S e T School E g § § . R elate o th er services ap p ro a ch e s to acquisition m an ag em en t to th at of own service. □ Ac DIFFERENTIATE AMONG THE PERSPECTIVES O F SENIOR LEADERS IN BOTH GOVERNMENT AND INDUSTRY CONCERNING POUCY AND STRATEGY DEVELOPMENT. Bloom PMFlel S u b S e t Explain the p ersp ectiv es of sen io r lead ers in both governm ent a n d industry concerning policy a n d strateg y developm ent. Bloom T « r 2 • S u b s e t Differentiate am ong the p ersp ectiv es of sen io r le a d e rs in both governm ent and Industry concerning policy a n d strateg y d evelopm ent. S chool H i § S . scnool fjjjl S3. 0 Ac SYNTHESIZE AND IMPLEMENT SOLUTIONS TO MANAGEMENT ISSUES ASSOCIATED WITH JOINT/INTERNATIONAL CONSIDERATIONS IN DEFENSE PROGRAMS. D escribe solutions to m anagem ent issu e s asso ciated with joint/intem ational considerations In d efen se program s. Bloom S u b S e t 1 S cho o l Bd. SS. Explain solutions to m an ag em en t issu e s asso ciated with Joint/intemational considerations In d efen se program s. Bloom - P » "S u E S eT School f id SS D evelop a n d im plem ent solutions to m an ag em en t issu e s a sso ciated with Joint/intem ational consid eratio n s in d efen se program s. [SI L Ac SYNTHESIZE AND IMPLEMENT SOLUTIONS TO MANAGEMENT ISSUES ASSOCIATED WITH ENVIRONMENTAL CONSIDERATIONS IN DEFENSE PROGRAMS. D escribe solutions to m anagem ent issu e s asso ciated with environm ental considerations in d efen se program s. Bloom PM l-iek 1 S u b S e t Explain solutions to m an ag em en t issu e s asso ciated with environm ental considerations in d efen se program s. Bloom T f » 2 s u b s e t Design a n d im plem ent solutions to m an ag em en t issu e s a sso ciated with environm ental considerations In d efen se program s. School Bd. § 2 . 1 scnool fjjjl S3. COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Love| (| Level I I I Competency Competence Competence Competence m EXPLAIN THE FLOW OF REGULATORY AUTHORITY FOR D escribe th e flow of regulatory authority from th e Executive Branch to the Program M anager. Bloom 1 A iraf W i e l d 1 D escribe th e (low of regulatory authority from the Executive B ranch to th e Program M anager. Bloom Explain th e flow of regulatoiy authority from th e Executive Branch to th e Program M anager. Bloom i . Ac ACQUISITION MANAGEMENT. p M n w d 1 P m field 2 S u b S e t S u b s e t S u b S e t School E ll §§. 1 S cho o l ^ SS. S cnool £M § £ . 2 2 9 EVALUATE THE PRINCIPAL REGULATIONS GOVERNING D escribe the applicable DoD acquisition regulations for any m ajor DoD system acquisition even t/m ilesto n e. Bloom s J r f f f lk 1 Distinguish th e applicable DoD acquisition regulations for any m ajor DoD system acquisition event/m ilestone. Bloom Justify th e applicable DoD acquisition regulations (or any m ajor DoD system acquisition even t/m ilesto n e. Bloom Ac SYSTEM ACQUISITION WITHIN THE DOD. P m Field 3 P M Held 6 S u b S e t s u b s e t S u b S e t S chool E H SS 1 sc h o o l Fid SS T “ sc h o o l fid S S 6 So ANALYZE THE RELATIONSHIP AMONG THE ACQUISITION R ecognize th e acquisition categ o ries a n d th e functions of th e respective m ilestone decision authorities and th e relationship Bloom -> |J| fUelcl 1 Explain the acquisition categ o ries a n d th e functions of the respective m ilestone decision authorities and the Bloom Differentiate am ong the acquisition categories a n d the functions of th e respective m ilestone decision authorities Bloom Ac CATEGORIES AND MILESTONE DECISION AUTHORITIES. PM Held 2 PM field 4 betw een th e two. S u b S e t relationship betw een th e two. S u b s e t an d th e relationship betw een th e two. S u b S e t S chool E H § S S cnool SS. S cnool f id § 2 . 4 31 SYNTHESIZE THE KEY ACTIVITIES FOR EACH PHASE OF THE ACQUISITION CYCLE INCLUDING DOCUMENTATION Identify th e critical m ilestone even ts, roles a n d responsibilities, a n d required docum entation for eac h p h a se ol the life cycle. Bloom Explain th e critical m ilestone even ts, roles and responsibilities, a n d required docum entation for e ac h Bloom R a n th e critical m ilestone even ts, roles a n d responsibilities, a n d required docum entation for e ac h p h a se Bloom Ac 3m Field 1 PM Field 2 PM fie ld s REQUIRED. S u b S e t p h a se of the life cycle. S u b s e t of th e life cycle. S u b s e t S chool E H SS 1 S cnool Fid SS T — s c n o o l Fid §S_ 5 COMPETENCIES & BLOOM LEVELS REQUIRED FOR IN THE PROGRAM MANAGEMENT CAREER FIELD - BASELINE CERTIFICATION CONSORTIUM Competency Level I Competence Level I I Competence J3 5 A C Level I I I Competence SYNTHESIZE THE ST E PS, PURPOSE AND KEY DECISION CRITERIA IN THE MILESTONE REVIEW PRO CESS, INCLUDING A DESCRIPTION AND EXAMPLE(S) OF MULTIFUNCTION EXIT CRITERIA. Identify th e s te p s, p u rp ose a n d key decision criteria In the m ilestone review p ro cess, including a description a n d exam ple(s) of multifunction exit criteria. Bloom 1 S u b S e t School EH . §S Explain the s te p s , p u rp ose a n d key decision criteria in the m ilestone review p ro c ess, Including a description a n d exam ple(s) of multifunction exit criteria. Bloom 3 s u b s e t S chool ££i SS . D esign th e s te p s , purpose a n d key decision criteria In the m ilestone review p ro cess, including a description a n d exam ple(s) of multifunction exit criteria. J33 Ac SYNTHESIZE THE PURPOSE AND PRINCIPAL ELEMENTS OF A PROGRAM ACQ UISITION STRATEGY. SPECIFICALLY ADDRESSING THE ACQUISITION PROGRAM BASELINE. RISK MANAGEMENT. TAILORING AND CONCURRENCY. D escribe th e p u rp ose and principal elem ents of a program acquisition strategy including the acquisition program baseline, risk m an ag em en t, tailoring a nd c oncurrency. Bloom are -telt 1 S u b S e t School E li SS. 1 Explain th e p u rp ose a n d principal elem ents of a program acquisition strategy Including the acquisition program b aseline, risk m anagem ent, tailoring a n d concurrency. Bloom 3 Subset s c n o o l f id S S T C reate th e p u rp ose and principal elem en ts of a program acquisition strategy including th e acquisition program baseline, risk m an ag em en t, tailoring a nd c oncurrency. 0 Ac EXPLAIN HOW CONGRESSIONAL ACTIVITIES IMPACT ACQUISITION MANAGEMENT. Bloom D escribe how C ongressional activities im pact the acquisition p ro cess. Bloom 1 Explain how congressional activities im pact acquisition m anagem enL S u b S e t S u b S e t School E ll § £ . School §2. APPLY PROCEDURES FOR PROGRAM INITIATION AND ELEMENTS O F KEY PLANNING AND CONTROL DOCUMENTS. Identify procedures for program initiation a n d elem ents of key planning a n d control docum ents. Bloom 1 S u b S e t Explain p ro ced u res for program initiation and elem ents of key planning and control d ocum ents. Bloom T J » 2 su b set U se pro ced u res for program Initiation a n d elem ents of key planning a n d control d o cu m e n ts. School E ll SS. 1 S cnool ^ S S COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD > CONSORTIUM Level I BASELINE Leve)1, Level I I I Competency Competence Competence Competence 16 0 EXPLAIN THE DAW1A IMPLEMENTATION AND ITS IMPLICATION FOR THE ACQUISITION W ORKFORCE. D escribe DAWIA implem entation a n d its Implications tor the acquisition workforce. Sloom Sum m arize DAWIA im plem entation a n d its Implications (or th e acquisition w orkforce. Bloom Explain DAWIA im plem entation a n d its im plications for th e acquisition workforce. Bloom A c D M Field 1 ™ ¥ S 3 d 2 f*M ?illd 2 S u b S e t S u b S e t S ubfeet S chool E li § S . S cho o l fjja s s . sc n o o l E l § § . 2 1 2 RESOLVE ISSUES RELATED TO AN ACQUISITION STRATEGY FOR A SPECIAL ACCESS PROGRAM. Bloom sfrwsfc Sum m arize issu e s related to a n acquisition strategy (or a special a c c e ss program . Bloom Resolve Issues related to an acquisition strategy for a special a c c e ss program . Bloom L-L- A c PM Retd F*Mfteld S u b S e t s u b s e t 2 S u b S e t 3 S chool E li SS s c n o o l Fid SS “ T S chool Fid S S . 3 t EXPLAIN THE ROLE OF THE PRIVATE SECTOR IN DEFENSE ACQUISITION. Explain th e role ol th e d efen se industrial b ase and th e private sec to r o( th e econom y in defen se acquisition. Bloom Bloom Bloom i—— A c PMField 2 PM R eid PM Held S u b S e t S u b s e t S u b S e t S chool E li § S 2 s c n o o l Ed § § . s c n o o l E l §£. 3 0 ? UNDERSTAND HOW TO DEVELOP AN ACQUISITION STRATEGY FOR EFFECTIVE MANAGEMENT O F COST, D escribe how to develop an acquisition strategy lor effective m an ag em en t ol cost, sch edu le, a nd perform ance risk which is In Bloom f anal Field 1 Explain how to develop an acquisition strategy (or effective m an ag em en t ol cost, sch edu le, and Bloom Bloom A c PM R eid 2 m Reid SCHEDULE, AND PERFORMANCE RISK AND IS IN com pliance with DOD series 8120 Instructions. S u b S e t perform ance risk which Is In com pliance with DOD series S u b s e t S u b S e t COMPLIANCE WITH DOD SERIES 8120 INSTRUCTIONS. S chool E li S S 1 8120 Instructions. s c n o o l Fid S S T “ S cnool E H §s_ u> O n 4^ COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE L evem Level I I I Competency Competence Competence Competence Contract Management c EXPLAIN THE EFFECTS OF SMALL BUSINESS. SMALL DISADVANTAGED BUSINESS. AND OTHER SOCIO-ECONOMIC Bloom Describe the effects of small business, small disadvantaged business, and other socio-economic Issues Bloom Explain the effects of small business, small disadvantaged business, and other socio-economic Issues on the Bloom _ C 3MFieid PM Field 1 PMf%3d 2 ISSUES ON THE CONTRACTING PROCESS. SubSet on the contracting process. Subset contracting process. Subset School E li 22. Scnool E d §3. scnool E l 52. 2 t ANALYZE THE APPROVED METHODS OF CONTRACTING AND TYPES OF CONTRACTS. Deline the approved methods ol contracting and types ol contracts. Bloom Explain the approved methods of contracting and types ol contracts. Bloom Illustrate the approved methods of contracting and type of contracts. Bloom c 3M Field 1 Pm Field 2 Pm Held 4 SubSet su b set SubSet School Eld 22. 1 scnool E ji s s scnool E l 22 . 4 ti RELATE THE SHARED ORGANIZATIONAL RESPONSIBILITIES FOR THE POST AWARD PHASE OF A Define the shared organizational responsibilities lor the post award phase ol the contract. Bloom Explain the shared organizational responsibilities the post award phase of a contract. Bloom Relate the shared organizational responsibilities for the post award phase of a contract Bloom l _ c 3M Field 1 PM Field 2 PM Reid 3 CONTRACT. SubSet Subset SubSet School Eld SS 1 scnool Fid SS T ~ scnool Ed SS 3 48 ANALYZE THE CONCEPTS AND TECHNIQUES INVOLVED IN THE Define the concepts and techniques Involved In the Bloom 1 SubSet School Eld 2 2 Explain the concepts and techniques involved In the Bloom TlJlHk" 2 Subset scnool i | l 32. Illustrate the concepts and techniques Involved In the Bloom p M r 4 SubSet scnool E l 22 . 4 c ACQUISITION OF DATA AND DATA RIGHTS. acquisition ol data and data rights. acquisition ol data and data rights. acquisition of data and data rights. u > c\ U l COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Leven| Level I I I Competency Competence Competence Competence 4 9 PREPARE A PLAN FOR AND MAINTAIN COMPETITION THROUGHOUTTHE ACQUISITION LIFE CYCLE. D escribe how to plan lor and m aintain com petition throughout th e acquisition life cycle. Bloom Explain how to plan lor and m aintain com petition throughout th e acquisition life cycle. Bloom P repare plan for and maintain com petition throughout th e acquisition Gfe cycle. Bloom L_ C PMFwld 1 PM Field 2 P m field 3 S u b S e t s u b s e t S u b S e t S cho o l Bd S3. s c h o o l 4 S S . S cnool fa ss. 3 5 0 DESIGN A D R A ff D escribe the elem ents/ relationships o l a draft RFP. Bloom 1 P repare a draft RFP. Bloom Design a draft R equest lor p roposal. Bloom c PM Field 3 P m fie ld 5 S u b S e t s u b s e t S u b S e t S chool E H S S . s c h o o l Fid S S T “ S chool R d § S . S I 5 1 ANALYZE THE FORMAL SOURCE SELECTION PROCESS. INCLUDING PLANNING, PARTICIPANTS. AND D escribe th e formal source selection p ro cess. Including planning, participants, and e v e n ts . Bloom D em onstrate th e formal sou rce selection p ro c ess, Including planning, participants, and even ts. Bloom Illustrate the formal source selection p ro c ess, Including planning, participants, a nd e v e n ts. Bloom c 3M Field 1 P m Field 3 P m fie ld 4 EVENTS. S u b S e t s u b s e t S u b S e t S cho o l Bd S3 s c h o o l fjj3 §3. sc h o o l fa 52. 4 52 ANALYZE THE MISSION AND RESPONSIBILITIES O F A CONTRACT ADMINISTRATION OFFICE (CAO). D escribe th e m ission and responsibilities ol a CAO. Bloom Explain the m ission and responsibilities ol a CAO. Bloom Illustrate th e mission and responsibilities ol a CAO. Bloom c PMField 1 I W-r © a . P m fie ld 4 S u b S e t S u b S e t 3 S u b S e t S cho o l Bd S S 1 sc n o o l £M SS S cho o l Fid S S 4 u> Os Os 367 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence n Level III Competence EVALUATE PROPOSALS RECEIVED IN RESPO N SE TO A REQ U EST TOR PROPOSAL (RFP) R ecognize how to evaluate proposals received In re sp o n se to a RFP. p i t o * Bloom S u b S e t 2 S chool E li § 5 . D em onstrate how to evaluate p ro p osals received In re sp o n se to a RFP. Bloom T i t o * s u b s e t s c n o o l f j j j i §3. A ppraise proposals received In resp o n se to a R FP. Bloom T i t o * 6 S u b S e t s c n o o l §3. 6 H ANALYZE THE MANAGEMENT ISSUES ASSOCIATED WITH THE PREPARATION FOR AND CONDUCT OF CONTRACT NEGOTIATIONS TO INCLUDE PR O PO S-A L REVIEW. DATA ANALYSIS. STRATEGY DEVELOPMENT, TACTICS AND D escribe th e m an ag em en t issu e s asso ciated with th e preparation (or and conduct o( contract negotiations to include proposal review, data analysis, strategy developm ent, tactics and com m unication p ro c e sse s. Bloom ie1d 1 S u b S e t S chool E ll SS P redict the m anagem ent issu e s a sso ciated with the preparation lo r a n d conduct of contract negotiations to include proposal review, data analysis, strategy developm ent, tactics and com m unication p ro c e sse s. Bloom T it o * 3 Subset s c n o o l £ jd S S Illustrate th e m anagem ent Issu es asso ciated with th e preparation for a n d conduct of contract negotiations to include proposal review, data analysis, strategy developm ent, tactics and com m unication p ro c e sse s. Bloom T it o * 4 S u b S e t s c n o o l Fid S S . 4 DEVELOP A SOLICITATION THAT EFFECTIVELY COMMUNICATES THE GOVERNM ENTS REQUIREMENTS, ACQUISITION STRATEGY, AND FACTORS FOR AWARD. D escribe a solicitation that effectively com m unicates the G overnm ent's requirem ents, acquisition strategy, a n d factors for aw ard. Bloom 1 S u b S e t P rep are a solicitation that effectively com m unicates th e G overnm ent's requirem ents, acquisition strategy, and factors for award. Bloom Subset Design a solicitation that effectively com m unicates the G overnm ent's requirem ents, acquisition strategy, a n d factors for award. Bloom T i t o * 5 SubSet S chool ELL SS S cno o l S S . Scnool E l S3. 5 57 c EVALUATE THE TECHNIQUES OF PRICING, FACT-FINDING, EVALUATING COST AND TECHNICAL PROPOSALS, AND NEGOTIATING FROM BOTH A GOVERNMENTAND CONTRACTOR PERSPECTIVE. D escribe th e techn iqu es of pricing, fact-finding, evaluating c o st a n d technical proposals, and negotiating from both a G overnm ent and contractor p ersp ectiv e. Bloom S u b S e t 1 S chool E L L S 3 D em onstrate th e techniques of pricing, fact-finding, evaluating cost a n d technical pro p osals, a n d negotiating from both a G overnm ent end contractor perspective. Bloom T i t o * Subbet s c n o o l Fid S S T “ C om pare the techniques of pricing, fact-finding, evaluating c o st a n d technical proposals, a n d negotiating from both a G overnm ent a n d contractor p ersp ectiv e. Bloom pito* 6 S u b S e t sc n o o l E l S§. 6 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Contractor Finance Level I Competence Level I I Competence Level III Competence '[EXPLAIN THE FINANCIAL ISYSTEMS, PR O C E SSE S, AND |PRACT1CES USED BY DEF. CONTRACTORS TO MANAGE WEAPON SYSTEM ACQUISITIONS, INCLUDING THE MOTIVATIONS AND CONSTRAINTS IN THEIR hum n in iT iT iffti D escribe the financial system , p ro c esse s, a n d p ractices u se d by d efen se contractors to m an ag e w eap o n system acquisitions, including the m otivations and constraints in their Im plem entation. Bloom S u b S e t S chool Eld. §3. Sum m arize th e financial sy stem s, p ro c esse s, and practices u se d by d efen se contractors to m an ag e w eap o n system acquisitions, including th e motivations and constraints in their im plem entation. Bloom ■ prirH lr S u b S e t School fid 53 Explain th e financial system , p ro c e sse s, a n d p ractices u se d by d efen se contractors to m an ag e w eap o n system acquisitions, including th e m otivations a n d constraints in their Im plem entation. ! EXPLAIN THE BASIC TOOLS O F FINANCIAL ANALYSIS TO EVALUATE CONTRACTOR FINANCIAL HEALTH AND VIABILITY AS PRESENTED IN FINANCIAL STATEMENTS. Bloom S u b S e t Explain the basic tools of financial analysis to evaluate contractor financial health and viability a s p resen ted in financial statem en ts. Bloom S u b S e t Explain th e b asic tools of financial analysis to evaluate contractor financial health and viability a s presen ted In financial statem en ts. S chool Eld. SS. School fk j SS. IidlE X PL A IN THE P R O C E SSE S JlNVOLVED IN DEVELOPING AND C DEVALUATING SHORT-TERM AND LONG-TERM SALES FORECASTS. D escribe th e p ro c e sse s involved In developing and evaluating short-term a n d long-term sa le s fo recasts. Bloom P i » 1 S u b S e t Sum m arize the p ro c esse s involved in developing and evaluating short-term and long-term s a le s forecasts. Bloom ■ p « r "SuU SeT Explain th e p ro c esse s Involved In developing and evaluating short-term and long-term sa le s forecasts. S cho o l Eld. SS. 1 school ^ S S |40]D EM ONSTRA TE HOW DIRECT/INDIRECT C O STS AND "C |O V E R H E A D RATES ARE DEVELOPED AND USED IN COST/MANAGERIAL ACCOUNTING. D escribe how direct/indirect costs and overhead rates are developed and u se d In cost/m anagerial accounting. Bloom I auol PM Field 1 S u b S e t Explain how direct/indirect co sts and overhead rate s a re dev elo p ed a n d u se d in cost/m anagerial accounting. Bloom T * * Subset D em onstrate how direct/indirect c o sts and o v erhead ra te s a re d eveloped an d u se d in cost/m anagerial accounting. S cho o l Eld. 53. t School £ j£ § 3 . COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Levem Level I I I Competency Competence Competence Competence EXPLAIN THE FINANCIAL MANAGEMENT ISSUES ASSOCIATED WITH THE C OST CONCEPTS O F Identify the financial m anagem ent Issu es asso ciated with th e c o st con cep ts of re a so n ab len ess, aDocability a n d allowability. Bloom Sum m arize the financial m an ag em en t Issu es a sso ciated with th e cost con cep ts of re a so n a b le n e ss. Bloom Explain th e financial m an ag em en t issu e s a sso ciated with th e cost c o n cep ts of re a so n ab len ess, Bloom L — c 3M h e ld 1 pM FitSd 2 Pt3i¥¥3d 2 REASONABLENESS, ALLOCABILfTY AND S u b S e t aDocability a n d aDowability. S u b S e t allocabfflty a n d aDowability. S u b s e t ALLOWABILITY. S chool E li § § . 1 s c h o o l Eg § 3 . S chool E l 33. 2 [42 ESTIMATE THE IMPACT ON PRICING DECISIONS CAUSED BY THE DIFFERENCES BETWEEN FIXED AND VARIABLE C O STS, Bloom D escribe th e im pact on pricing decisions c a u s e d by th e differences b etw een fixed a n d variable co sts, Bloom E stim ate the Im pact on pricing decisions c a u s e d by the differences betw een fixed and v ariable c o sts, Including th e Bloom c P m R eid 1 Pm 153d 2 INCLUDING THE USE OF BREAK-EVEN ANALYSIS AND S u b S e t Including th e u se of break-even analysis a nd marginal pricing. S u b s e t u se of break-even analysis a n d m arginal pricing. S u b S e t MARGINAL PRICING. S chool E li SS . s c n o o l S S s c n o o l E l S S . 2 c c SUMMARIZE THE BASIC TOOLS O F CAPITAL INVESTMENT ANALYSIS AND EVALUATION. Bloom Explain the b asic tools of capital Investm ent analysis a n d evaluation. Bloom D em onstrate th e basic tools of capital Investm ent analysis a n d evaluation. Bloom PM Field PM R eid 2 PM Weld 3 S u b S e t S u b S e t S u b S e t School E li § S s c h o o l E £ 5 2 . s c n o o l E l § £ . 3 4 4 EXPLAIN THE ISSUES RELATED TO WORKING CAPITAL MANAGEMENT AND SOURCES O F CONTRACTOR FINANCING. Bloom Identify the issu e s related to working capital m an ag em en t a n d so u rces of contractor financing. Bloom Explain th e issu e s related to working capital m anagem ent a n d so u rce s of contractor financing. Bloom c PMField PM Rwd 1 PwlW Sd 2 S u b S e t S u b S e t S u b s e t School E li SS s c h o o l s c n o o l E l 2 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence 30 IJL C EXPLAIN THE BASIC CONCEPTS OF FINANCIAL MANAGEMENT AND FINANCIAL REPORTING OF GOVERNMENT CONTRACTORS. rtJrfS fc Bloom S u b S e t S chool Eld. 33. Explain th e basic con cep ts ol financial m an ag em en t and financial reporting ol g o vernm ent contractors. Bloom ■ n J rK S ftr 2 "SuESeT School S3. D em onstrate th e b asic c o n cep ts ol financial m an ag em en t a n d financial reporting o l governm ent contractors. 30 Li C OUTUNE THE BUSINESS PR O C ESS A GOVERNMENT CONTRACTOR U SES IN ESTABLISHING PRO PO SED PRICES. Bloom S u b S e t D em onstrate th e b u sin ess p ro c ess a governm ent contractor u s e s in establishing p ro p osed prices. Bloom SubSet Outline th e b u sin ess p ro c ess a governm ent contractor u se s In establishing p roposed p rices. S chool Eld. SS. s c h o o l R d S S T 30 LI c OUTLINE THE PR O C E SS A GOVERNMENT CONTRACTOR USES IN C OST ESTIMATING. Bloom D em onstrate the p ro cess a governm ent contractor u se s in co st estim ating. Bloom iL. - p k M Illustrate the p ro cess a governm ent contractor u se s In c o st estim ating. S u b S e t "SuESeT S chool Eld. 3 3 . scnool Cost/Schedule Control p t RELATE THE HISTORICAL DEVELOPMENT. PRINCIPLES. OBJECTIVES. AND CURRENT DOD POLICIES CONCERNING CONTRACT PERFORMANCE MEASUREMENT. Identity the historical developm ent, principles, objectives, a n d current DoD policies concerning contract perform ance m easurem ent. Bloom M r S u b S e t 1 School Eld. 3 3 . Sum m arize the historical developm ent, principles, objectives, a n d current DoD policies concerning contract perform ance m easu rem ent. Bloom T * T 2 "SuESeT 3 s c n o o l D em onstrate the historical developm ent, principles, objectives, a n d current DoD policies concerning contract perform ance m easurem enL COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence INTERPRET HOW THE EVALUATION O F A CONTRACTOR’ S MANAGEMENT CONTROL SYSTEM USING THE C/SCSC CRITERIA BENEFITS MANAGEMENT O F A PROGRAM. Describe e contractor’s m an ag em en t control system s using th e five C /SC SC criteria. Bloom S u b S e t 1 S cho o l Eli §§. Explain a contractor's m an ag em en t control syste m s using the five C /S C S C criteria. Bloom tM t "SuESeT 3 S cnool i | l §3. Appraise a contractor's m an ag em en t control sy ste m s using th e five C /S C S C criteria. |60 C RECOMMENDTHEKEY COST/SCHEDULE RELATED ELEMENTS WHICH SHOULD BE CONSIDERED WHEN DEVELOPING A REQUEST FOR PROPOSAL (RFP). Bloom S u b S e t Explain the key C ost/ S ched ule related elem en ts which should b e con sid ered w hen developing a R FP. Bloom 2 “SuESeT Support th e key C ost/S ch ed u le related elem en ts which should b e co n sid ered w h en developing a R FP. S chool Eli ss. sc n o o l m ANALYZE THE COST/SCHEDULE IMPLEMENTATION AND REVIEW PR O CESS. TO INCLUDE TAILORING THE APPLICATION AND IDENTIFYING THE TYPICAL INVOLVEMENT BY THE PMO. D escribe the C ost/S chedule im plem entation a n d review p ro cess, to Include tailoring the application a n d identifying the typical involvem ent by the PMO. Bloom ^ Hel< 1 S u b S e t S chool E ld §§. 1 Explain the C ost/S ch ed u le im plem entation a n d review p ro cess, to include tailoring th e application a n d identifying the typical Involvem ent by the PMO. Bloom i « r "SuESeT 4 S cno o l “ § § . Illustrate th e C ost/S chedule im plem entation a n d review p ro c ess, to Include tailoring the application a n d Identifying th e typical Involvem ent by th e PM O. p2 OUTLINE HOW TO DEVELOP. EVALUATE AND CHANGE AN INTEGRATED PERFORMANCE MANAGEMENT BASELINE. Bloom I ftMnl PM Fiald S u b S e t Explain how to develop, evaluate, and c h a n g e an Integrated P erform ance M easurem ent B aseline. Bloom tM t S chool E ld ss. “SuESeT 4 sc n o o l Fid S S T Illustrate how to develop, evaluate, a n d ch an g e an Integrated Perform ance M easurem ent Baseline. COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence ANALYZE THE ROLES, RESPONSIBILITIES AND COORDINATION REQUIRED POR C /SC SC SURVEILLANCE. D escribe th e roles, responsibilities and coordination required tor C /SC SC surveillance. Bloom 1 S u b S e t S cho o l Eli §3. Explain th e roles, responsibilities a n d coordination required for C /S C S C surveillance. Bloom T rirfS fr 2 S u b s e t S chool S3. Illustrate th e roles, responsibilities a n d coordination required for C /S C S C surveillance. |66 C DEMONSTRATE THE PU RPO SE, USE, AND GENERATION O F THE REQUIREMENTS FOR EXTERNAL REPORTING O F A PROGRAM'S COST/SCHEDULE STATUS. Bloom p M S u b S e t Explain the requirem ents for external reporting of a program 's cost/sch ed u le sta tu s. Bloom relt 2 SubSet D em onstrate th e requirem ents for external reporting of a program 's co st/sch ed u le sta tu s. S cho o l Eli SS. school f|! §3. 0 INTERPRET COST/SCHEDULE CONTRACT PERFORMANCE INFORMATION TO YIELD CURRENT PROGRAM STATUS, PROJECTED PERFORMANCE AND RECOMMENDATIONS. D escribe program cost, schedule, a n d perform ance progress/deliclencles using c o st perform ance reports to Identity problem a re as, a s s e s s basefine im pacts a n d develop recom m endations. Bloom * 1 S u b S e t S cho o l E l i 3 3 . Explain how cost/sch ed u le contract perform ance Information Is u se d to yield current program statu s, projected perform ance and recom m endations. Bloom "S u E S eT School f j j p 33. Interpret cost/schedule contract perform ance Information to determ ine sta tu s of th e program , project future perform ance, and identify appropriate actions. c PREPARE A LIFE CYCLE C OST (LCC) ANALYSIS REPORT. Describe a LCC analysis re p o rt Bloom 1 Explain a LCC analysis re p o rt S u b S e t Bloom t M 2 Subset P repare a LCC analysis rep o rt S cho o l E li s s . sc h o o l Fid S S COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Leve| M Level I I I Competency Competence Competence Competence Fiscal Management 0 Fi EVALUATE PROGRAM DOCUMENTATION TO ENSURE THAT THE NEEDS O F AN INDIVIDUAL PROGRAM ARE MET. P f » Bloom S u b S e t S chool Eli S3. P rep are program docum entation to e n su re th at the n e e d s of an Individual program are m et. Bloom 7 « T 3 “ SuESeT s c h o o l ^ S3. A ppraise program docum entation to e n su re that th e n e e d s of a n Individual program a re m e t pa Fi ANALYZE COST/TECHNICAL DATA AND ESTIMATES TO IDENTIFY SYSTEM COST DRIVERS FOR RISK AND TRADE-OFF ANALYSIS IN DECISION MAKING. Bloom _i flual- ( W i e l d S u b S e t Explain cost/technical data a n d estim ates to identity system cost drivers for risk and trade-off analysis in decision making. Bloom tM t "SuESeT Illustrate cost/technical data a n d estim ates to Identify system cost drivers for risk and trade-off analysis In decision m aking. S chool Eld S 3 . S cho o l ^ § 2 . U SE T H E C O E A A S APPROPRIATE IN PREPARING FOR A MILESTONE (M/S) REVIEW. Define the COEA a n d its u se . Bloom Explain and justify th e COEA p ro c ess. Bloom U se the COEA a s appropriate In preparing for a M/S review. 1 S u b S e t subset S chool Eld S S . S cho o l R d S S " 7 “ 0 F I DEMONSTRATE HOW COST ESTIMATES ARE PREPARED AND USED IN GOVERNMENT AND INDUSTRY. D escribe cost estim ating p ro c e sse s, techniques, principle: an d u se within governm ent and industry. Bloom it I fltrnl. PM Field 1 S u b S e t Explain c o st estim ating p ro c e s se s , techn iqu es, principles a n d u se within governm ent a n d Industry. Bloom p « r 2 subset D em onstrate how c o st estim ates a re prepared a n d u se d in governm ent Industry. S chool E ld S3. s c n o o l E k j S3. 374 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Competency Level I Competence BASELINE Level I I Competence 0 F I Level III Competence ANALYZE THE PLANNING, PROGRAMMING AND BUDGETING SYSTEM (PPBS) PRO CESS AND ITS RELATIONSHIP TO AND EFFECT] ON THE SYSTEMS ACQUISITION P R O C E S S . D escribe the PPB S p ro cess and Its relationship to a n d effect on sy stem s acquisition p ro cess. Bloom S u b S e t School B jt § 3 . Explain th e P P B S p ro cess a nd its relationship to and effect on sy ste m s acquisition p ro c ess. Bloom 2 Subset school £ H S 3 . Illustrate th e PPB S p ro cess a n d its relationship to and effect on th e system s acquisition p ro c ess. Bloom M a r, 4 Subset scnool E d S3. 4 I75IANA | CON F i ANALYZE THE CONGRESSIONAL ENACTMENT P R O C E S S . D escribe the C ongressional enactm ent p ro cess. Bloom srirK S fr i S um m arize the C ongressional enactm ent p ro c ess. Bloom T « T Illustrate th e C ongressional en a c tm e n t p ro c ess. S u b S e t Subset Bloom 4 S u b S e t S chool S i S S 1 scnool S S scnool B d S S . 4 0 F I ANALYZE THE BUDGET EXECUTION PRO CESS. D escribe th e budget execution p ro c e ss. Bloom I Aunt 3M Field 1 S um m arize th e budget execution p ro c ess. Bloom S u b S e t 7 « r 2 subset Illustrate th e bu dg et execution p ro c e ss. Bloom nfrfiSd 4 S u b S e t S chool B d. 33. scnool E£ 33 school E d S3. 4 77 Fi ANALYZE THE PPBS. ENACTMENT AND BUDGET EXECUTION AND THEIR RELATIONSHIP TO THE ACQUISITION PROCESS. D escribe th e PP B S , enactm ent a n d budget execution and their relationship to th e acquisition p ro c e ss. Bloom 3M Nell 1 S u b S e t Explain the P P B S , enactm ent a n d bu dg et execution and their relationship to the acquisition p ro cess. Bloom T jJitS Ik 2 Subset Illustrate th e PPB S, e n actm en t a n d budget execution a n d their relationship to th e acquisition p ro c e ss. Bloom T « T 4 S u b S e t S chool 0 1 SS . 1 scnool ss School E d S S . 4 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD • CONSORTIUM BASELINE 0 FI Competency Level I Competence D escribe how th e cost/budget reviews relate to the program review /decision p ro c ess. Level I I Competence Level III Com petence ANALYZE HOW THE COST/BUDGET REVIEWS RELATE TO THE PROGRAM REVIEW/DECISION PR O C E SS. Bloom 1 S u b S e t S chool E U . §2. Explain how th e cost/budget review s relate to th e program review /decision p ro c ess. Bloom 2 S ubset s c n o o l ^ §3. Illustrate how th e cost/b ud g et review s relate to th e program review /decision p ro c e ss. Logistics Support flop l_JF Lo ANALYZE THE ILS REQUIREMENTS. ELEMENTS AND ACTIVITIES ASSOCIATED WITH EACH PHASE OF THE LIFE CYCLE. Define th e ILS requirem ents, elem ents a n d activities associated with e a c h p h ase of the life cycle. Bloom S u b S e t S chool E L L 52. Sum m arize the ILS requirem ents, elem en ts a n d activities asso ciated with each p h a se of th e life cycle. Bloom ? M r 2 "SuESeT s c n o o l ^ 5 2 . Illustrate th e ILS requirem ents, elem en ts a n d activities asso ciated with e a c h p h a s e of th e life cycle. 0 Lo ANALYZE THE REQUIREMENTS FOR AND IMPLEMENTATION OF RELIABILITY, MAINTAINABILITY. AND AVAILABILITY (RMA), AS IT IMPACTS THE SYSTEMS ENGINEERING PRO CESS. Define th e requirem ents for a nd Im plem entation of reliability, maintainability, a n d availability (RMA), a s it im pacts th e system s engineering p ro c ess. Bloom P f » 1 S u b S e t School E 1 4 S S 1 Explain the requirem ents for a n d im plem entation of reliability, maintainability, and availability (RMA), a s it im pacts th e sy stem s engineering p ro c ess. Bloom i « r 2 "SuESeT Illustrate th e requirem ents for a n im plem entation of reliability, maintainabilty, a n d availability (RMA), a s it Impacts the sy stem s engineering p ro c ess. S cho o l fW S S Lo ANALYZE THE REQMTS FOR AND PU RPO SE O F AN INTEGRATED LOGISTICS SU PPO RT PLAN, ITS ROLE IN PLANNING AND INTEGRATION ACROSS ALL LOGISTIC ELEMENTS, DEPLOY-MENT/ FIELDING/FLEET INTRO. PLANS, . nAi.T Describe th e requirem ents for a nd p urpose of a n Integrated Logistics Support Plan(ILSP) and its role in planning a n d integration across all logistic elem ents, deptoym enV fieldlng/fleet introduction plans, a n d po st deploym ent support. Bloom 1 S u b S e t S chool E l d . SS. Explain the requirem ents for a n d purpose of a n Integrated Logistics Support Plan(ILSP) a n d Its role in planning and Integration acro ss all logistic ele m en ts, deploym ent/fielding/fleet Introduction plans, a n d po st Bloom I awal T*M Field "SuESeT School 5 5 . Illustrate th e requirem ents for a n d p u rp ose of a n Integrated Logistics Support Plan(ILSP) a n d Its role in planning a n d integration acro ss all logistic ele m en ts, deploym ent/fieldlng/fleet introduction plan s, a n d p o st i M i w ■■■ 376 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD • CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence p Lo ANALYZE THE ISSUES. MANAGEMENT TASKS AND THE INTEGRATION OF FUNCTIONAL SPECIALTIES APPLICABLE TO DEPLOYMENT/FIELDING/FLEET INTRODUCTION/SITE ACTIVATION. Identity the issues, management tasks and the Integration ol functional specialties applicable to Deployment/Fielding/Fleet Introduction/Site Activation. Bloom p iM i SubSet School Bd. §§. 1 Explain the issues, management tasks and the Integration ol functional specialties applicable to Deptoyment/Fielding/Fleet Introduction/Site Activation. Bloom T « r 2 Subset scnool ^ § 2 . illustrate the issues, management tasks and the integration ol functional specialties applicable to Deployment/Fielding/Fleet Introduction/Site Activation. Bloom 4 Subset scnool EH 22. 4 ANALYZE THE LOGISTIC SUPPORT ANALYSIS (LSA) PROCESS. ITS USES AND REQUIREMENTS AS IT APPLIES TO EACH PHASE OF THE ACQUISITION PROCESS. Define the Logistic Support Analysis (LSA) process, its uses and requirements. Bloom L Yeld 1 Explain the Logistic Support Analysis (LSA) process, Its uses and requirements. Bloom SubSet ■pjJrt* 2 Subset Illustrate the Logistic Support Analysis (LSA) process, Its uses and requirements. School Eld. SS. 1 scnool Ej£ S S Bloom 4 SubSet School £W SS. 4 ANALYZE THE CONTINUOUS ACQUISITION AND LIFECYCLE SUPPORT (CALS) PROCESS AND IMPLEMENT IT INTO PROGRAM ACQUISITION AT BOTH GOVERNMENT AND INDUSTRY DURING THE LIFE-CYCLE OF A PROGRAM. Define the Continuous Acquisition and Life-Cycle Support (CALS) process and its place in both government and industry. Bloom P i » 1 SubSet Define the Continuous Acqui sition and Life-Cycle Support (CALS) process and Its place In both government and Industry. Bloom 2 Subset School Eld. 22. 1 Scnool £ j£ 22 Illustrate the Continuous Acquisition and Life-Cycle Support (CALS) process and Implement it Into program acquisition at both government and Industry during the Cfe-cycte of a program. Bloom 4 SubSet school £ H 22. 4 Managerial Development M DEMONSTRATE CORRECT Bloom Explain correct usage of Bloom Demonstrate correct usage of Bloom USAGE OF SERVICE AND DOD P 5 » service and DoD policy In - m i r f f a i r service and DoD policy In POLICY IN DEALING WITH THE dealing with the media. 2 dealing with the media. MEDIA. 3 SubSet Subset SubSet School Scnool scnool Eld. 22. EH 2 2 EH 22. 3 3 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence rrr LJ M Level III Competence ILLUSTRATE THE PHILOSOPHY, TECHNIQUES AND TOOLS O F TOTAL QUALITY MANAGEMENT (TQM). Explain th e philosophy, techniques a n d tools of TQM. Bloom S u b S e t S cho o l Hd. § 3 . 2 Explain the philosophy and u se th e techn iqu es a n d tools of TQM. Bloom ■ p « r 3 subset School j|j S3. Illustrate th e philosophy, techn iqu es a n d tools of TQM. 10 _6_ M DEVELOP A PROACTIVE, PERSONAL, ETHICAL DECISION MAKING APPROACH TO DEAL WITH ETHICAL DILEMMAS IN ACQUISITION MANAGEMENT. Explain a proactive, personal, ethical decision making approach to deal with ethical dilem m as In acquisition m a n a g e m e n t Bloom PM h e k 2 S u b S e t D em onstrate a proactive, p ersonal, ethical decision making approach to d eal with ethical dilem m as In acquisition m a n ag em en t. Bloom tM subset C reate a proactive, personal, ethical decision m aking approach to deal with ethical dilem m as In acquisition m a n a g e m e n t S chool Eli SS 2 School Fid S S T 1 1 M GENERALIZE THE CAUSES, CONSEQUENCES AND TECHNIQUES FOR MANAGING STRESS IN THE ACQUISITION MANAGEMENT ENVIRONMENT. D escribe th e c a u ses, c o n se q u e n c e s a n d techn iqu es for m anaging stre ss In Ihe acquisition m an ag em en t e n v iro n m en t Bloom D M Fielc 1 S u b S e t Sum m arize the c a u se s, c o n se q u e n c e s a n d techn iqu es for m anaging stre ss in th e acquisition m an ag em en t en v iro n m en t Bloom tM t 2 Sublet G eneralize the c a u s e s , c o n se q u e n c e s a n d techn iqu es for m anaging stre s s in the acquisition m an ag em en t en v iro n m en t S chool E li 3 3 . 1 school R d 33 , 14 2 _ M EXPLAIN SPECIFIC TIME MANAGEMENT STRATEGIES TO MITIGATE/ELIMINATE TIME WASTERS IN A TYPICAL ACQUISITION MANAGEMENT ORGANIZATION. Identify specific time m anagem enl strateg ies to mitigate/eliminate time w asters in a typical acquisition m anagem ent organization. Bloom S u b S e t Sum m arize specific time m anagem ent strateg ies to mitigate/eliminate time w asters in a typical acquisition m an ag em en t organization. Bloom tM t SubSet Explain specific time m anagem ent strateg ies to m itigate/elm inate time w asters In a typical acquisition m anagem ent organization. S cho o l E ll S 3 . 1 School ^ S S COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD • CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence APPLY MANAGERIAL, INTERPERSONAL, ORGANIZATIONAL AND LEADERSHIP SKILLS TO DEVELOP AN EFFECTIVE PROGRAM MANAGEMENT ORGANIZATION. Explain m anagerial, interpersonal, organizational and leadership skills to develop a n effective program m anagem ent organization. Bloom S u b S e t S chool E li 5 2 . 2 D em onstrate m anagerial, interpersonal, organizational a n d leadership skills to develop an effective program m an ag em en t organization. Bloom ■ p d rfS fr 3 " S u b S e t' school E § i S 3. U se m anagerial, interpersonal, organizational a n d leadership skills to develop s n effective program m anagem ent organization. EXPLAIN HOW EFFECTIVE STAKEHOLDER MANAGEMENT CAN BUILD COALITIONS, OVERCOME OBSTACLES/RESISTANCE TO CHANGE AND ESTABLISH SUPPORT FOR A PARTICULAR POSITION. D escribe how effective 'stak eh o ld er m anagem ent" can build coalitions, overcom e ob stacles/resistance to chan ge and establish support for a particular position. Bloom ive tele 1 S u b S e t S chool E li SS G eneralize how effective "stakeholder m anagem ent* can build coalitions, overcom e o b stacles/ re sistan ce to ch an g e a n d establish support for a particular position. Bloom P f J f f le f r s u b s e t School R d S S T Explain how effective "stakeholder m anagem ent* can build coalitions, overcom e obstacles/resistance to c h a n g e a n d establish support for a particular position. FORMULATE TAILORED. HUMAN-SKILLS APPROACHES BASED ON INDIVIDUAL DIFFERENCES, THE INTERPERSONAL COMM PR O CESS. AND SMALL GROUP DYNAMICS TO MAXIMIZE INTERPERSONAL EFFECTIVE- Sum m arize tailored, human-skills ap p ro ach es b a se d on individual differences, th e Interpersonal communlction p ro cess, and small group dynam ics to maximize Interpersonal effectiveness In the pgm m gt environm ent. Bloom P i » S u b S e t S chool E ll ? § . 2 D em onstrate tailored, human-skills app roach es b a s e d on Individual differ e n c e s, th e interpersonal com m p ro cess, and small group dynam ics to maximize interpersonal effectiveness in th e pgm m gt environm enL Bloom TdfSftr "S u E S eT s c h o o l E j j i 33 C reate tailored, hum an- skills ap p ro ach es b a s e d on individual differences, the interpersonal com m p ro cess, and sm all group dynam ics to maximize interpersonal efT ness in the pgm mgt environm ent Manufacturing Management n EVALUATE AN INDUSTRIAL BASE ASSESSM ENT. Describe th e Industrial B ase participants a n d a sse ssm e n t ele m en ts. P 7 « 1 Bloom S u b S e t S chool E ll 2 2 . 1 Explain a n industrial b ase a ss e s s m e n t Bloom T f » 2 Subbet School Interpret a n industrial b ase a s s e s s m e n t COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level I I I Competence pa M A SSESS A PRODUCTION STRATEGY FOR TECHNICAL AND ECONOMICAL CONGRUENCE WITH THE PROGRAM ACQUISITION STRATEGY! D escribe th e technical and econom ical a sp e c ts o t production a s related to acquisition. Bloom 1 S u b S e t S cho o l ELL § S Explain how th e technical and econom ical a sp e c ts of production affect acquisition strategy. Bloom tM t 2 S u b s e t sc n o o i § 3 . A ppraise a production strategy for technical a n d econom ical con g ru ence with th e program acquisition strategy. c M APPLY PRODUCIBILITY ENGINEERING AND PLANNING (PEP) TO ENSURE A TIMELY TRANSITION FROM CONCEPT EXPLORATION TO LOW RISK ECONOMICAL PRODUCTION. Define th e m ajor elem ents of Producibility E ngineering and Planning (PE P). Bloom iva ieh 1 Explain th e elem en ts of P E P . S u b S e t Bloom t« t 2 s u b s e t U se P E P to ensu re a timely transition from concept exploration to low risk econom ical production. S cho o l E li S S . 1 s c n o o i R d SS T “ ANALYZE THE ROLES OF DPRO/CONTRACTOR FOR AN EFFECTIVE QUALITY/PRODUCT ASSURANCE PROGRAM IN SU PPO RT O F THE GOVERNMENT PM. Define th e roles of D PRO/ contractor to r a n effective quality/product a ssu ra n c e program in support of the governm ent PM. Bloom 1 S u b S e t Explain the roles of D PRO/ contractor for an effective quality/product a ssu ra n c e program in support of the governm ent PM. Bloom Trifair s u b s e t Differentiate betw een the roles ol th e D PRO and that ot the contractor relative to a n effective quality assu ra n c e program . S cho o l E li § § . s c n o o i 4* 0 M ANALYZE THE ISSUES AND MANAGEMENTTASKS APPLICABLE TO PRODUCTION PLANNING TO INCLUDE INTEGRATION WITH THE DESIGN PR O C E SS AND LONG LEAD ITEM PLANNING. D escribe production planning issu e s a n d a sso ciated program m an ag em en t actions. Bloom S u b S e t Explain and give exam ples of, production planning issu e s a nd related program m an ag em en t actions. Bloom 7 * r Illustrate production planning issu e s a n d th e relationship to program m an ag em en t actions. S u b s e t S chool E li SS. 1 S ch o o l 4 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD • CONSORTIUM BASELINE Competency Level I Competence Level I I Competence Level III Competence M ANALYZE TYPICAL PRODUCTION PROBLEMS AND ISSU ES AND DEVELOP PR O PO SE D SOLUTIONS. List so m e ol the typical production p roblem s/issues, and historical app roach es taken to a d d re ss them . Bloom P i » S u b S e t 2 S chool Eld. § S 1 2 Explain and give exam ples of typical production problem s/ Issues, a n d com m only u se d strateg ies to a d d re ss them . Bloom T « r Subset 4 School Illustrate typical production problem s a n d Issu e s and out-llne strateg ies and solutions to a d d re ss them . EXPLAIN CURRENT MANUFACTURING TECHNOLOGY AND TECHNIQUES EMPLOYED IN INDUSTRY. Bloom S u b S e t D escribe current m anufacturing technology and techniques em ployed In Industry. Bloom 1 Explain current m anufacturing technology a n d techn iqu es em ployed in Industiy. S u b S e l S chool E li SS School S S 1 0 I 0 M A SSE SS THE APPLICATION O F MANUFACTURING MANAGEMENT PRINCIPLES TO THE DESIGN PR O CESS AND EXECUTION OF MANUFACTURING AND TRANSITION TO PRODUCTION PLANS. D escribe th e m anufacturing m anagem ent principles that a fte d a program throughout its lite cycle. Bloom 1 S u b S e t Explain th e m anufacturing m an ag em en t principles that a ile d a program and give exam ples o l their elffe d s. Bloom ■ fM t 2 SubSet A ppraise a program 's production-related efforts during Its acquisition life. School E li § § . 1 school S3. Program Management Pr EXPLAIN THE MANAGEMENT METHODS AND GOALS OF THE DEFENSE INDUSTRY. D escribe th e m anagem ent m ethods a n d go als ol the d efen se industry. P i » Bloom 1 S u b S e t School E l i § § . 1 Sum m arize the m an ag em en t m ethods a n d goats of the d efen se Industry. Bloom Subset school E H Explain th e m an ag em en t m eth o ds a n d go als of the d e fe n se Industry. COMPETENCIES & BLOOM LEVELS REQUIRED FOR IN THE PROGRAM MANAGEMENT CAREER FIELD - BASELINE CERTIFICATION CONSORTIUM Competency Level I Competence Level I I Competence □ Pr Level III Com petence DIFFERENTIATE MAJOR ACQUISITION PROGRAM MANAGEMENT BY WORKING TYPICAL PROGRAM ISSUES. DEVELOPING PROGRAM DOCUMENTA- TION, HAVING DISCUSSIONS WITH PROGRAM MANAGERS. AND VISITING n n r u . l f » T U 1 » l r i f M i rrt,-r- Explain m ajor acquisition program m anagem ent by working typical program issu e s, developing program docum entation, having discussions with program m an ag ers a n d visiting production facilities. Bloom S u b S e t S chool Eld. 52. 2 Discover major acquisition program m an ag em en t by working typical program Issu e s, developing program d ocum entation, having discussions with program m an ag ers a nd visiting production facilities. Bloom 3 subset sc n o o i O p 5 2 . Differentiate m ajor acquisition program m an ag em en t by working typical program iss u e s , developing p rogram docum entation, having d iscussions with program m an ag ers a n d visiting production facilities. □ Pr DEVELOP AN ACQUISITION STRATEGY THAT PROVIDES FOR EFFECTIVE MANAGEMENT OF CO ST, SCHEDULE AND PERFORMANCE RISK AND IS IN COMPLIANCE WITH DOD 5000 SERIES INSTRUCTIONS. Describe a n acquisition strategy th at provides for effective m an ag em en t of cost, sch edu le a n d perform ance risk and is in com pliance with DoD 5000 series Instructions. Bloom PMFiek 1 S u b S e t School E li 5 2 . 1 Explain an acquisition strategy by that provides for effective m an ag em en t of cost, sch edu le and perform ance risk a nd is in com pliance with DoD 5000 serie s Instructions. Bloom Subset S cho o l Fjd SS Tf C o m p ose a n acquisition strateg y th at provides for effective m an ag em en t of cost, sch edu le a n d perform ance risk and is com pliance with DoD 5 0 00 serie s Instructions. 0 Pr ANALYZE AN ESCALATING THREAT RELATIVE TO AN EXISTING SYSTEM TO DETERMINE VIABLE C O U RSES OF ACTION (I.E.. SYSTEM IMPROVEMENT, NEW START.) Bloom P i » S u b S e t D escribe an escalating threat relative to an existing system to determ ine viable c o u rses of action (i.e., system im provem ent, new start). Bloom T « r 1 subset Illustrate an escalating threat relative to a n existing system to determ ine viable c o u rse s of action Q.e., system Im provem ent, new start). S chool Eld. 5 2 . s c h o o l fjja § 2 , DEVELOP, ANALYZE AND UPDATE PERFORMANCE, SCHEDULE, AND CO ST ACQUISITION PROGRAM BASELINES (APBS). D escribe perform ance, sch edu le, a n d c o st APBs. Bloom PM Field 1 P rep are perform ance, sch edu le, a n d c o st APBs. Bloom S u b S e t ■ n JifB fr 3 Subset C reate, analyze and u pdate p erform ance, sch edu le, a n d c o st APBs. School E li 5 2 . 1 S cho o l Fid SS T T COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Leve( (| Level I I I Competency Competence Competence Competence t A SSE SS A PROGRAM'S READINESS TO EXIT FROM A GIVEN PHASE O F THE ACQUISITION PR O C ESS. PA SS D escribe a program 's readin ess to exit from a given p h ase ol the acquisition p ro cess, p a s s the required review, a n d en ter into Bloom Explain a program 's readiness to exit from a given p h a se ol the acquisition p ro c ess, p a s s th e required review a n d e n ter Bloom A ppraise a program 's re a d in ess to exit from s given p h a se o l th e acquisition p ro c ess, p a s s th e required Bloom L — Pr 3M Fieid 1 Pm Field 2 PM ?ield 6 THE REQUIRED REVIEW. AND ENTER INTO THE NEXT PHASE. th e next p h ase. S u b S e t Into th e next p h a se . s u b s e t review e n d e n te r into th e next p h a s e . S u b S e t S chool E li 3 3 . 1 S cno o i S 3. S chool EM S 3 . 6 c ANALYZE AN ISSUE AND PRESENT A DECISION BRIEFING TO HIGHER AUTHORITY. G eneralize an issue lor a decision briefing to higher authority. Bloom skK Sfr 2 Analyze a n issu e a n d p resen t a decision briefing to higher authority. Bloom Analyze an Issue and p resen t a decision briefing to higher authority. Bloom Pr PM Field 4 PM^iffld 4 S u b S e t S u b S e t S u b S e t S chool E li SS 2 S chool Fid S I T sc n o o i f id S S 4 1 " APPLY VARIOUS TYPES OF ORGANIZATIONAL MODELS THROUGH THE PHASES OF THE LIFE CYCLE TO EXECUTE AN D escribe various ty p es ol organizational m odels through the p h a s e s ol the lile c y d e to execute an acquisition strategy. Bloom Explain various types ol organizational m odels through th e p h a s e s o l th e file cycle to execute an Bloom D em onstrate various ty p es ol organizational m odels through th e p h a s e s ol the file cycle to ex ecu te a n acquisition Bloom L— Pr Pm Field 1 PM Field 2 P fiffe ld 3 ACQUISITION STRATEGY. S u b S e t acquisition strategy. S u b s e t strategy. S u b S e t S chool E li SS. S cho o l E j$ i SS. s c h o o l E d 53 . 3 120 SYNTHESIZE A VALID SET OF REQUIREMENTS FOR PROGRAM INITIATIVES AND PLANNING. Bloom Bloom C reate a valid s e t ol requirem ents for program initiatives a n d planning. Bloom 1 __ Pr ?M Field PM Field PM fie ld s S u b S e t s u b s e t S u b S e t S chool E li SS s c n o o i R d S S s c h o o l Fid S S 5 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence 0 Pr Level I I Competence Level I I I Competence SYNTHESIZE THE REQUIREMENTS O F SEVERAL RELATED FUNCTIONAL AREAS INTO WELL INTEGRATED AND EFFECTIVE STRATEGIES, PLANS AND P R O C E SSE S WITH EMPHASIS ON RISK MANAGEMENT. Bloom S u b S e t S chool E ld §3 Bloom i « r "SuESeT sc n o o i E d S3 Com bine the requirem ents ot sev eral related functional a re a s Into well Integrated a nd effective strateg ies, plan s and p ro c e sse s with em phasis on risk m a n ag em en t p 2 Pr SYNTHESIZE REQUIRED PROGRAM SU PPO RT DOCUMENTATION (INTEGRATED PROGRAM SUMMARY. ACQUISITION PROGRAM BASELINE, RISK MANAGEMENT PLAN. BUDGET ESTIMATES). Identity required program support docum entation (Integrated Program Sum m ary. Acquisition Program Baseline, Risk M anagem ent P lan, B udget Estim ates). Bloom pdtfssfr 1 S u b S e t S chool E ld S S . 1 P rep are required program sup p o rt docum entation (Integrated Program Sum m ary, Acquisition Program B aseline, Risk M anagem ent P lan, B udget Estim ates). Bloom T i» 3 "SuESeT sc n o o i Ej^ S S D esign required program sup p o rt docum entation (Integrated Program Sum m ary, Acquisition Program Baseline, Risk M anagem ent Plan, B udget E stim ates). 0 Pr SYNTHESIZE CRITICAL MANAGEMENT ISSUES, DOCUMENTATION AND RELEVANT INFORMATION TO SU PPO R T A MILESTONE REVIEW P R O C ESS. Bloom S u b S e t Explain critical m anagem ent Issues, docum entation and relevant Information to support a m ilestone review p ro c ess. Bloom ■prirtsfr "SuESeT P rep are critical m anagem ent Issu es, docum entation a n d relevant Information to support a m ilestone review p ro cess. S chool E ld §3. s c h o o l E j J S3 . EXPLAIN THE STRUCTURE AND OPERATION OF A TYPICAL DEFENSE INDUSTRY CORPORATION EMPHASIZING PROGRAM MANAGEMENT TECHNIQUES. D escribe the structure and operation of a typical D efense Industry Corporation em phasizing program m anagem ent te c h n iq u e s. Bloom S u b S e t Sum m arize the structure and operation of a typical D efense Industry Corporation em phasizing program m an ag em en t techn iqu es. Bloom tM t 2 subset Explain th e structure and operation of a typical D efense Industry Corporation em phasizing program m an ag em en t techiquea. S chool E ld SS s c n o o i £ d SS COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD • CONSORTIUM Level I BASELINE Level„ Level I I I Competency Competence Competence Competence 9 6 DEVELOP MULTIFUNCTION / INTEGRATIVE PROGRAM EXIT CRITERIA FOR ALL MAJOR MILESTONES. Bloom Explain multifunction/ Integrative program exit criteria (or all major m ileston es. Bloom P rep are multifunction/ Integrative program exit criteria for a l m ajor m ilestones. Bloom 1 — Pr ^ Had PM F ield 2 f’M ^ e ld 3 S u b S e t s u b s e t S u b S e t S chool E li 3 2 . s c n o o i f k i § 5 . S chool E d § 2 . 3 E SYNTHESIZE A MULTIFUNCTION ALT INTEGRATIVE RISK MANAGEMENT P U N TO Explain a multifunctional/ integrative risk m anagem ent plan to support acquisition program planning a n d control. Bloom 2 P rep are a multifunctional/ integrative risk m anagem ent plan to support acquisition program planning a n d control. Bloom D esign a multifunctional/ Integrative risk m an ag em en t plan to sup p o rt acquisition program planning a n d co n tro l Bloom Pr P m R e id 3 PM Reid 5 SUPPORT ACQUISITION PROGRAM PLANNING AND S u b S e t S u b S e t S u b S e t CONTROL.. S chool E li SS 2 s c n o o i Fid S S T “ sc h o o l f id § § . 5 I 10 | 4 DEMONSTRATE THE ADVANTAGES AND CAPABILITIES O F APPROPRIATE PROGRAM D escribe th e various m ethods of program planning a n d control. Bloom Explain the various m ethods o l program planning and control. Bloom D em onstrate th e various m eth o ds of program plannlngand control. Bloom Pr Pm Field 1 PM Held 2 p jJrtfelS r 3 PLANNING AND CONTROL METHODS. S u b S e t s u b s e t S u b S e t School E li 33 . 1 S cho o l fj<J S3. S cnooi E d S 3 . 3 IIS ANALYZE QUANTITATIVE PROBLEM SOLVING METHODS D escribe quantitative problem Bloom D em onstrate quantitative Bloom Similar to Level II, but illustate Bloom solving m ethods such a s decision 1 prob. solving m ethods su c h T i t e i r 3 th e u se a n d appropriate p tf rttS lir 4 Pr SUCH AS DECISION ANALYSIS, MULTI-ATTRIBUTE UTILITY analysis, multi-attribute utility analysis, regression analysis, a s decision enatysis, multi attribute utility analysis. choice of quantitative problem solving m ethods su ch a s ANALYSIS. REGRESSION ANALYSIS, BASIC PROBABILITY THEORY AND RISK ANALYSIS IN ADDRESSING PGM MGT basic probability theory and risk analysis in addressing program m gt problem s and issues. S u b S e t S chool E li SS 1 regression analysis, basic probability theory a n d risk analysis in addressing pgm m gt problem s a n d Issues. S u b s e t s c n o o i B d § s decision analysis, e tc. S u b S e t Scnooi Fid S S 4 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Competency Level I Competence BASELINE Level I I Competence Level III Competence i° I 9 Pr APPLY QUALITATIVE PROB. SOLVING METHODS SUCH AS BRAINSTORMING. NOMINAL GROUP TECHNIQUE. KEPNER-TREGOE PROBLEM ANALYSIS AND DELPHI TECHNIQUES IN ADDRESSING PROGRAM MANAGEMENT n n /-'n i r-iif" m r . i p f i ir-r>______ D escribe qualitative problem solving m eth o ds su ch a s brainstorm ing, nominal group tec h lq u e , K epner-T regoe problem analysis and the Delphi techniques In add ressin g pgm m an ag em en t problem s and Issu e s. Bloom 1 S u b S e t S chool Bd. §§. Explain qualitative problem solving m ethods su c h a s brainstorm ing, nom inal group technique, K epner- T reg o e problem analysis a n d the Delphi techniques In addressing pgm mgt problem s a n d is s u e s . Bloom t « t 2 "SuESeT Scnooi §2. Similar to Level II, but d em on strate qualitative problem solving m ethods such a s brainstorming, etc. EXPLAIN THE LESSONS LEARNED AND CORRECTIVE ACTIONS TAKEN FROM THE HISTORY AND CIRCUMSTANCES SURROUNDING A MAJOR ACQUISITION PROGRAM. Bloom 5 « r S u b S e t D escribe th e le sso n s learn ed a n d corrective actions taken from th e history and circum stances surrounding a m ajor acquisition program . Bloom Jr “SuESeT S chool Bd. ss. school S S Explain th e history and circum stances surrounding a m ajor acquisition program and b e able to draw conclusions a s to le sso n s learn ed a nd corrective actions. Software Management EXPLAIN TYPICAL ACQUISITION RISKS FOR SYSTEMS. SELECT APPROPRIATE MITIGATION STRATEGIES AND ILLUSTRATE THEIR RELATIVE MERITS. Identity typical acquisition risks for sy ste m s. Bloom 1 S u b S e t S chool Bd. §£. 1 Explain typical acquisition risks tor system s, select appropriate mitigation strateg ies. Bloom T « r 3 "SuESeT scnooi ^ SS. Explain typical acquisition risks tor sy stem s, selec t appropriate mitigation strateg ies and. illustrate their relative merits. EXPLAIN DOD REGULATORY AND TECHNICAL FRAMEWORKS THAT APPLY FOR THE ACQUISITION O F MCCR, C3I AND A1S; SELECT AND DIFFERENTIATE TECHNIQUES APPROPRIATE TO MANAGE EACH CLASS OF Identity the DOD regulatory and technical frameworks that apply tor th e acquisition of MCCR, C3I a n d AIS. Bloom 1 S u b S e t Explain th e DOD regulatory and technical fram eworks that apply for the acquisition ol MCCR, C3I and AIS. Bloom 2 "SuESeT S chool Bd. §§. t Scnooi ^ §3. Explain th e DOD regulatory and technical frameworks that apply for the acquisition of MCCR, C3I and AIS; select a n d differentiate techniques to m an ag e e ach class of system . COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Leve,„ Level I I I Competency Competence Competence Competence h ? EXPLAIN THE SOFTWARE D E V aO PM E N T A N D INTEGRATION PR O C ESS AND THE SOFTWARE TECHNICAL D escribe th e softw are developm ent a n d Integration p ro cess a n d th e softw are technical lita cycle. Bloom Explain th e softw are developm ent a n d Integration p ro cess a n d th e softw are technical life cycle, and Bloom Explain th e softw are developm ent a n d Integration p ro c ess e n d th e softw are technical life cycle, and Bloom _ s Field 1 Pm Field 2 P M fte ld 4 LIFE CYCLE. AND ILLUSTRATE THEIR RELATIONSHIPS TO THE S u b S e t sum m arize their relationships to the overall system acquisition p ro cess. S u b s e t illustrate their relationships to th e overall system acquisition p ro c e ss. S u b S e t OVERALL SYSTEM ACQUISITION PR O C ESS. School EM. S S . s c n o o i s s . s c n o o i f i d S S . 4 [7 3 | 7 EXPLAIN SOFTWARE PROCUREMENT REQUIREMENTS; USE S/W SOURCE SELECTION "BEST Identity softw are procurem ent requirem ents. Bloom 1 Explain S/W procurem ent requirem ents; u se govt, a n d com m ercial softw are source selection "best p ractices'; Bloom Explain S/W procurem ent reqm ts; u s e govL a n d com m. S/W so u rce selec - tion "best practices*; Ulus- Irate proposal Bloom s p M R eld 3 PliAneld 4 PRACTICES"; ILLUSTRATE PROPOSAL EVALUATION S u b S e t distinguish proposal evalu ation criteria and docum en S u b s e t evaluation criteria and docum enta- tion relevant for S u b s e t CRITERIA AND DOCUMENTATION RELEVANT S chool EM. SS 1 tation (or acquisition of softw are system s. s c h o o l R d SS T — acquisl-tlon of SAN system s. s c h o o l f jd S S 4 12 R EXPLAIN, USE AND ILLUSTRATE TOOLS & TECHNIQUES AVAILABLE FOR PLANNING. MEASURING AND PREDICTING Identity tools a n d techniques available tor planning, m easuring a n d predictlong softw are d evelopm ent p ro g ress. Bloom Explain and u se tools and techniques available for planning, m easuring a n d predictlong softw are Bloom Explain, u se and illustrate tools a n d techniques available for planning, m easuring a nd predictlong softw are Bloom s 3M Field 1 P m R eid 3 P M field 4 SOFTW ARE DEVELOPMENT PR O G R E SS. S u b S e t d ev elopm ent p ro g ress. S u b s e t dev elo p m en t p ro g ress. S ubfeet S chool EM. SS. 1 s c h o o l ^ s s . s c n o o i m SS. 4 s EXPLAIN AND RELATE CURRENT "BEST PRACTICES" FOR SOFTWARE TEST PROGRAM PLANNING AND Describe current policies and ‘b est p ractices' (or softw are te st program planning, execution and te st sufficiency. Bloom Explain a n d u se current policies a n d "best practices* for softw are te st program planning, execution a n d te st Bloom Explain a n d relate current policies a n d "b est practiced* for softw are te st program planning a n d execution a n d Bloom s Fteld 1 FM Field' 3 P m field 4 EXECUTION. AND ILLUSTRATE SOFTWARE TEST SUFFICIENCY S u b S e t sufficiency. s u b s e t illustrate softw are te st sufficiency. T u B S e t ' S chool E M . SS 1 s c n o o i R d S S T “ s c h o o l fid SS. 4 Competency COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level! BASELINE Level), Level 1 1 1 Competence Competence Competence 1 1 S EXPLAIN AND ILLUSTRATE PROGRAM OFFICE AND CONTRACTOR PLANS AND STATUS DOCUMENTS FOR DEVELOPMENT. INTEGRATION, MANAGEMENT AND SU PPO RT OF MCCR, C3I AND AIS. D escribe key program office and contractor plans a n d statu s d o cu m ents for developm ent, integration, m an ag em en t and support of MCCR, C3I a nd AIS. Bloom pjrtsfr i S u b S e t S chool Bd §3. Explain program office and contractor plans a n d sta tu s d ocum ents for d evelopm ent, Integration, m an ag em en t and support of MCCR, C3I a nd AIS. Bloom 7 « r 2 “SuESeT s c h o o l E g 55. Explain a n d illustrate program office a n d contractor p lan s a n d sta tu s d ocum ents for developm ent. Integration, m an ag em en t a n d sup p o rt of MCCR, C3I and AIS. EXPLAIN AND ILLUSTRATE THE RESPECTIVE ROLES O F GOVERNMENT AND INDUSTRY IN SOFTWARE ACQUISITION MANAGEMENT ACTIVITIES. D escribe governm ent a n d Industry softw are acquisition m an ag em en t activities. Bloom 1 S u b S e t Explain th e respective roles of governm ent a n d Industry In softw are acquisition m anagem ent activities. Bloom - pM t "SuESeT Explain a n d illustrate the respective roles of governm ent a n d Industiy In softw are acquisition m an ag em en t activities. S cho o l O d S S . 1 S cho o l R d S S T — 0 S EXPLAIN AND RELATE THE ECONOMIC FACTORS OF SOFTWARE SYSTEM S, TO INCLUDE C O ST ESTIMATION, BUSINESS CASE ANALYSIS, MANAGEMENT OF O BSOLESCENCE AND COST/PERFORM -ANCE T i-r.. i.i/m Tnriinn Bloom Bloom S u b S e t subset S cho o l Bd 53. sc n o o i Ed 53. Explain and relate the econom ic factors of softw are sy stem s, Including co st estim ation, b u sin e ss c a s e analysis, m anagem ent of o b so le sce n c e a n d cost/perform ance technology tren d s. System s Engineering 0 SYNTHESIZE OPERATIONAL REQUIREMENTS INTO DESIGN REQUIREMENTS. M atch operational requirem ents a n d d esign requirem ents. Bloom Explain th e relationship betw een operational requirem ents a n d d esign requirem ents. Bloom T ranslate operational requirem ents Into d esign req u irem en ts. Bloom Field 1 PM Raid' 2 5 S u b S e t s u b s e t S u b S e t S cho o l Bd 55. 1 sc n o o i i | l 55. sc n o o i E d 55. s COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE Competency Level I Competence Level I I Competence 0 Level III Competence DEMONSTRATE THE TECHNICAL ADEQUACY O F THE EXISTING DESIGN TO MEET KNOWN TECHNICAL REQUIREMENTS. Bloom S u b S e t S chool Bd §3. E stim ate the technical ad eq u acy o l the existing d esign to m eet known technical requirem ents. Bloom ■ptair 2 S ubset s c n o o i ^ 52. D em onstrate th e technical ad eq u acy o l th e existing d esign to m e et known technical requirem ents. PREDICT THE TECHNICAL AND ECONOMIC IMPLICATIONS OF DESIGNS FOR PROGRAM AND TECHNICAL REVIEWS THROUGH USE OF MULTIFUNCTIONAL TEAMS. Define th e technical and econom ic implications o l d esig n s lor program a n d technical reviews through u s e o l multifunctional team s. Bloom p M 1 S u b S e t Explain th e technical and econom ic Implications ol d esig n s lor program and technical review s through u se ol multifunctional te a m s. Bloom prfrt* 2 Subset Predict th e technical a n d econom ic im plications ol desig n s lo r program a n d technical review s through u se o l multifunctional team s. S chool B d SS 1 s c n o o i fj£ ss 0 RELATES GOVERNMENT/ CONTRACTOR ROLES AND MANAGES ISSUES ASSOCIATED WITH TRADE-OFFS BETWEEN TECHNICAL AND BUSINESS CONCERNS. Define governm ent/contractor roles In S E a n d identifies issu e s related to technical and b u sin e ss trade-offs. Bloom s sS rfS fr 1 S u b S e t S chool Bd §S G eneralize gov't/contractor roles in S E a n d issu e s a sso ciated with th e inherent conflict b etw een sy stem s engineering a n d b u sin e ss areas. Bloom ■ M r su b set R elate governm ent/ contractor roles a n d m an ag es issues asso ciated with trade-offs b etw een technical a n d b u sin e ss c on cerns. iachooi S S 0 APPLY TECHNICAL PERFORMANCE MEASUREMENT (TPM) AS AN ELEMENT O F THE RISK MANAGEMENT PLAN. Define TPM a s an elem ent of the risk m anagem ent plan. Bloom 1 Explain TPM a s a n e lem ent ol the risk m an ag em en t plan. Bloom U se TPM a s an elem ent of the risk m anagem ent plan. S u b S e t 2 "SuESeT S chool E H SS 1 s c n o o i Fid S S T — COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Levem Level I I I Competency Competence Competence Competence APPLY CONCURRENT ENGINEERING TECHNIQUES AS PART O F INTEGRATED PRODUCT DEVELOPMENT AND TOTAL QUALITY MANAGEMENT/LEADERSHIP. Define concurrent engineering techn iqu es e s p ert ot integrated product developm ent a n d total quality m anagem ent/leadership. Bloom P » 1 S u b S e t S chool EM. § § . Interpret concurrent engineering techn iqu es a s p art o l integrated product developm ent a n d total quality m an ag em en t/lead ersh ip . Bloom ■ etc 2 Subset Scnooi fjp 52. U se concurrent engineering techniques as part ol integ rated product developm ent a n d total quality m an ag em en t/lead ersh ip . USE THE W BS AS A MANAGEMENTTOOL Explain a W BS. Bloom P rep are a WBS. Bloom S u b S e t T « r 3 subset U se the W BS as a m an ag em en t tooL S chool Eld. SS 2 Scnooi R d S S T ILLUSTRATE THE ROLE OF CONRGURATION MANAGEMENT IN THE SYSTEMS ENGINEERING P R O C E S S . D escribe the role o l configuration m an ag em en t in tho S y stem s E ngineering p ro c ess. Bloom ^ M F i& d 1 S u b S e t Explain the role ol configura tion m an ag em en t in the S y stem s E ngineering p ro c ess. Bloom T « r 2 “SuESeT Illustrate the role ol configuration m an ag em en t In th e S y stem s E ngineering p ro c e ss. S chool Eld. § § . Scnooi E g 22. ANALYZE SYSTEMS ENGINEERING AS A COMPREHENSIVE. ITERATIVE TECHNICAL MANAGEMENT PR O C ESS THAT INTEGRATES EFFORTS OF THE ENTIRE DESIGN TEAM. Define S y stem s E ngineering a s a com prehensive, iterative technical m an ag em en t p ro cess th a t integrates th e efforts o t th e entire design team . Bloom 1 S u b S e t S chool Eld. SS 1 “ Explain S ystem E ngineering a s a com prehensive, iterative technical m an ag em en t p ro c ess th at integrates th e efforts ol th e entire design team . Bloom M 2 S u b s e t Scnooi Illustrate S y stem s Engineering e s a com prehensive, iterative technical m anagem ent p ro c ess th at Integrates th e efforts o l th e entire design team . COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE Leve( (| j_evel I I I Competency Competence Competence Competence U l APPLY THE SYSTEMS ENGINEERING PRO CESS TO PREPARATION FOR AND EXECUTION O F A TECHNICAL D escribe th e epplication ol the sy ste m s engineering p ro c ess to preparation for a n d execution of a technical review. Bloom Explain th e application of the sy ste m s engineering p ro cess to preparation for and execution of a technical Bloom " R t o e f r " 2 U se th e S E p ro c ess In th e preparation for a n d execution of a technical review. Bloom L £ . s ^ f f iS d " 1 P m R o d 3 REVIEW. S u b S e t review. S u b s e t S u b S e t S cho o l EM. § 5 . s c n o o i 4 * s & sc n o o i E l § 3 . 3 l ± ILLUSTRATE. THROUGH THE S E PR O C ESS, THE RELATIONSHIP BETWEEN THE TECHICAL ASPECTS O F DESIGNS AND Identify, using the S E p ro cess, th e relationship betw een the technical a sp e c ts of d esig n s and th e co sts, sch edu le, a n d risk Bloom 1 Explain, using th e S E p ro cess, th e relationship betw een the technical a sp e c ts of d esig n s a n d the Bloom Illustrate, through th e S E p ro c ess, th e relationship betw een th e technical a sp e cts of d esig n s a n d th e costs, Bloom L J_ s PM Field 2 Pm field 4 THE COST, SCHEDULE, AND RISK ASSOCIATED WITH THOSE asso ciated with th o se d esigns. S u b S e t co sts, sch edu le, a n d risk a sso ciated with th o se S u b s e t sch edu le, a n d risk asso ciated with th o se d esig n s. S u b S e t DESIGNS. S chool EM. S S d esig n s. sc n o o i § s S chool B d S S 4 0 APPLY DESIGN-TO-COST (DTC) ANALYSIS FOR A MORE EFFICIENT DESIGN PRO CESS. Bloom Defend DTC analysis for a m ore efficient design p ro c ess. Bloom U se DTC analysis for a more efficient d esig n p ro cess. Bloom S 3MFi&d PM ¥ield 2 P M ^ e ld 3 S u b S e t s u b s e t S u b S e t S chool EM. § S . s c h o o l ^l & sc n o o i E l §S_ 3 [3 0 I f i RELATE THE MANAGEMENT OF TECHNOLOGY TO THE SYSTEMS ENGINEERING PR O CESS TO INCLUDE DOD Deline technology m an ag em en t a n d d escrib e the relationship betw een th e S E p ro cess and DOD S&T th rusts a n d technology Bloom 1 n»nf ? m r S d 1 Explain technology m an ag em en t a n d generalize the relationship of DOD technology th rusts a n d Bloom R elate th e m an ag em en t of technology a n d DOD S&T th rusts to th e S E p ro cess throughout a system Gfe c y d e . Bloom s PM field 2 PM field 3 S&T THRUSTS AND TECHNOLOGY TRANSITION transition relationships throughout a system life cycle. S u b S e t technology transition to the S E p ro cess throughout a system life c y d e . S u b s e t S u b S e t THROUGHOUT A SYSTEM LIFE CYCLE. S chool EM. S S 1 sc n o o i R d S S T “ sc n o o i E l S S 3 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM Level I BASELINE L eve| n |_ eve| m Competency Competence Competence Competence 3 1 RELATE APPROPRIATE D escribe specifications and Bloom Explains exam p les ol Bloom R elate appropriate Bloom M SPECIFICATIONS AND sta n d a rd s In relation to products appropriate specifications and T g jr f f J li r specifications a n d sta n d a rd s s STANDARDS TO PRODUCTS O F o l th e syste m s engineering stand ard s in relation to 2 to products o l th e S E p ro cess. THE SYSTEMS ENGINEERING p ro c e s s. products o l th e S E p ro c ess. 3 P R O C E S S . S u b S e t s u b s e t "S u b S e t School S cno o i sc n o o i Eld. SS. s s . fa. ss. 3 Test and Evaluation [1 WRITE AND JUSTIFY A TEST AND EVALUATION PROGRAM/ PLAN THAT SU PPO RTS THE SELECTED ACQUISITION Identity w here a te s t and evaluation program is d o c u m e n te d . Bloom Explain a te st and evaluation program /plan th at sup p o rts th e selec te d acquisition strategy. Bloom P rep are a test a n d evaluation program /plan for a selec te d acquisition strategy. Bloom L L Te D M Field 1 P M F w d 2 P m Held 3 STRATEGY. S u b S e t s u b s e t S u b S e t School E ld . 53. 1 s c n o o i 33. S cho o l f ! d S 3 . 3 r n h APPLY THE REQUIREMENTS OF FEDERAL LAW IMPOSED UPON DOD T EST AND EVALUATION. Bloom Explain th e requirem ents ol lederal law Im posed upon DoD T est a nd Evaluation. Bloom D em onstrate th e m an dated requirem ents Im posed upon DoD T e s t a n d Evaluation. Bloom Te aM F i5d Pm Field 2 P M lY eld 3 S u b S e t s u o s e t S u b S e t School E ld SS S cno o i Fid S S T “ s c h o o l f jd S S . 3 1 1 DEMONSTRATE AN UNDERSTANDING OF THE RELATIONSHIP O F TEST AND EVALUATION T O EACH Bloom Explain the relationship ol te st a n d evaluation to the sy ste m s acquisition p ro cess. Bloom D em onstrate th e relationship ol T est a n d Evaluation to the system s acquisition p ro cess. Bloom Te PM Field PM Field 2 P m H eld 3 DEVELOPMENT PHASE AND MILESTONE IN THE S u b S e t s u b s e t S u b S e t ACQUISITION PR O CESS. S cho o l E ld S3. S cno o i S3. sc n o o i B d S 3 . 3 COMPETENCIES & BLOOM LEVELS REQUIRED FOR CERTIFICATION IN THE PROGRAM MANAGEMENT CAREER FIELD - CONSORTIUM BASELINE H ^Ih e I 7 RC Te ' Competency Level I Competence Level I I Competence Level III Com petence RELATE THE RESPECTIVE ROLES O F GOVERNMENT AND INDUSTRY IN DOD T&E. Define th e respective roles ol G overnm ent a n d Industry In DoD T&E. Bloom 1 S u b S e t S chool E H § § . Explain th e respective roles of G overnm ent a n d industry In DoD T&E. Bloom tM t S u b S e t S cno o i f|| § 2 . R elate th e respective roles ol G overnm ent a n d Industry In DoD T&E. Bloom 3 S u b s e t s c n o o i E d 3 3 . 3 113IDISI I a TYF Te DISCRIMINATE BETWEEN THE TYPES AND PU R PO SE S OF DEVELOPMENTAL AND OPERATIONAL TESTING AS THEY RELATE TO WEAPONS AND AUTOMATED INFORMATION SYSTEMS. Define the ty p es a n d p u rp o ses ol developm ental a n d operational testing a s they relate to w eap o n s a n d autom ated Information system s. Bloom St 1 S u b S e t S chool Eld. SS 1 Explain th e types and p u rp o ses ol developm ental a n d operational testing a s they relate to w e ap o n s a nd autom ated information sy stem s. Bloom 2 "SuESeT s c h o o l fW S S Discriminate betw een the ty p es a n d p u rp o se s of developm ental and operational testing a s th ey relate to w eap o n s and auto m ated information sy stem s. Bloom 3 S u b S e t S cnooi Fid § § _ 3 APPLY MODELING AND SIMULATION PLANNING P R O C E SSE S IN ALL FUNCTIONAL AREAS. Bloom S u b S e t Explain modeling and sim ulation planning p ro c esse s In all functional a re as. Bloom TltecT U se m odeling a n d simulation planning p ro c e sse s in all functional a re a s. S u b S e T S chool Eld. § 5 . s c h o o l fjjp S3. Bloom 3 S u b S e t s c n o o i E H §§. 3 JESSE STEWART * 8:33 AM • 6/27/95 APPENDIX 2: Level I: System Acquisition Fundamentals (at least one year of acquisition experience) (A D S-93-01-G D : D oD Acquisition Career Management: Mandatory' Course Fulfillment Program and Competency Standards, January 1993) 1. Describe a basic Acquisition Strategy which demonstrates a fundamental knowledge of DoD's life cycle acquisition management system policies. Identify and incorporate technical and business components, within the Acquisition Strategy development process. 2. Describe the principle elements of resource allocation and the basic requirements for Milestone approval. 3. Discuss the challenges to DoD and Program Manager, including the cultural and organizational linkages that influence decisions during the acquisition of weapons systems. 393 Level II: Intermediate Systems Acquisition (at least two years of acquisition experience) (A D S-93-01-G D : D oD Acquisition Career Management: Mandatory Course Fulfillment Program and Com petency Standards, January 1993) 1. Develop an acquisition strategy that is the framework for integrating functional activities essential to fielding a defense acquisition program. 2. Discuss the financial system, process and practices used by defense contractors to manage systems acquisition, including the motivation and constraints in their implementation. 3. Discuss both the competitive and noncompetitive contract award process and their implications on contract management. 4. Identify and analyze contract cost, schedule, and performance progress/difficulties to determine contract and programmatic impacts and recommended actions. 5. Explain the process of Program Objective Memorandum (POM) and Budget Estimate Submission (BES) reviews and how they interrelate with the program review/decision process. 6. Describe the DoD logistic policy and related management tools, such as Logistic Support Analysis, the logistics planning and contracting process, and how these should be applied to program office and contractor logistics during acquisition cycle. 7. Comprehend how to formulate tailored, human-skills approaches based on individual differences, the interpersonal communications process and small group dynamics to maximize interpersonal effectiveness in the program management environment. 394 8. Explain the principles of manufacturing management and how these principles should be applied to influence the design, process, transition to production, and execute the manufacturing plan. 9. Review and evaluate program plans and documentation to ensure that the needs of an individual program are consistent with sound business practices, common sense, and the degree of risk involved. 10. Translate operational requirements into design requirements which are explicit functional quantitative ranges or point values of performance. 11. Analyze program office and contractor status and plans for Mission Critical Computer Resources (MCCR) development, integration, management, and support. 12. Comprehend how to develop a comprehensive test and evaluation (T&E) program and how to integrate T&E program into the systems engineering and system lifecycle processes. Additionally, comprehend the T&E facilities and resources required to execute a T&E program within the lifecycle framework. 13. Apply the philosophy, techniques, and tools of total quality management (TQM) and how it should be applied in program office and contractor organizations. 395 Level III: System Acquisition Fundamentals (at least 4 years of acquisition experiences) (2 years in a systems program office or similar organization; and, be qualified for the acquisition corps) (A D S-93-01-G D : D oD Acquisition Career Management: Mandatory Course Fulfillment Program and Com petency Standards, January 1993) 1. Experience in a joint "purple suit" educational or operational organization which provided understanding of other Services' approach to acquisition and management. 2. Experience working with or studying the defense industry. 3. Insight into the perspective of senior level DoD acquisition managers. 4. Understanding of major acquisition program management from experience working with or study of program issues and documents, discussions with government and industry PMs, and visiting contractor production facilities. 5. List Capitol Hill Programs or workshops you have completed, or other means by which you have had an opportunity to study, and visit Congress to understand congressional interrelationships on acquisition policy and programs by direct exchange with members and key staffers. 6. List key briefings on acquisition issues which you have prepared and delivered to senior officials for decisions. 7. Competence in dealing with the media on program issues. 8. Able to develop an acquisition strategy that is the framework for integrating functional activities essential to fielding a Defense Acquisition Program and is the basis for developing specific program management documents. 9. Understand how to translate operational requirements into design requirements which are explicit functional quantitative ranges or point values of performance and is able to do a trade-off analysis. 396 10. Understand how to analyze an escalating threat relative to an existing system to determine viable courses of action (i.e., major system improvement, new start). 11. Understand how to analyze the technical adequacy of the existing design to meet known technical requirements. 12. Able to review and evaluate program documentation to ensure that the needs of an individual program are consistent with sound business practices, common sense, and the degree of risk involved. 13. Able to develop, analyze, and update performance, schedule, cost, and program baselines. 14. Able to assess a program's readiness to exit from a given phase of the acquisition process, pass the required review, and enter into the next phase. 15. Understand the financial and technical systems, processes, and practices used by defense contractors to manage weapon system acquisitions, including the motivations and constraints in their implementation. 16. Know how to develop and implement solutions to management issues associated with international considerations in defense programs. 17. Able to apply the philosophy, techniques, and tools of total quality management in the program office and the contractors' organization. 18. Able to apply quantitative and qualitative methods to analyze an issue and present a decision briefing to higher authority. 19. Able to identify and plan for the personnel and functional management support required to execute the acquisition strategy of a defense acquisition program. 20. Able to analyze need and synthesize requirements information to develop a valid set of requirements for program initiatives and planning. 397 21. Able to synthesize the requirements of several related functional areas into well integrated and effective strategies, plans and processes to support acquisition programs. 22. Able to assemble and synthesize necessary programmatic, funding and technical information to create required program support documentation (Integrated Program Summary, Acquisition Program Baseline, Risk Management Plan, Budget Estimates). 23. Able to assemble and synthesize critical management issues, documentation and relevant information to support a major milestone review process. 24. Able to analyze changes that impact on an acquisition strategy and to develop responsive and effective changes in strategy. 25. Understand the structure and operation of a typical defense industry corporation with the emphasis on use of program management techniques. 26. Understand the impact of environmental and program specific issues on the execution of an ongoing DoD acquisition program from government and industry viewpoints. 27. Understand how to develop plans and review program status for deployment. 28. Know the flow of regulatory authority from the Executive Branch to the Program Manager. 29. Understand the principal regulations governing system acquisition within the DoD. 30. Know the different categories of acquisition programs, including the respective milestone decision authorities. 31. Understand key activities taking place during each of the phases of the acquisition life cycle, to include documentation required. 398 32. Understand the steps, purpose and key decision criteria in the milestone review process, including a description and example(s) of exit criteria. 33. Understand the purpose and principal elements of a program acquisition strategy which includes specifically addressing the acquisition program baseline, risk management, tailoring and concurrency. 34. Know key management considerations and areas of concern for managers of joint and international programs. 35. Understand the major environmental laws and policy documents which affect the systems acquisition process and their impact on the program manager. 36. Understand how congressional activities impact acquisition management. 37. Understand the structure and inter-relationships of the elements of a financial statement, such as the differences between the income statement, the balance sheet and cash flow statement. 38. Able to apply the basic tools of financial analysis to the information in a financial statement to evaluate contractor financial health and viability. 39. Understand the processes involved in developing and evaluating short term and long-range sales forecasts. 40. Understand the differences between direct and indirect costs, and the processes used in the development of overhead rates and the management of indirect costs. 41. Understand the financial management issues associated with the cost concepts of reasonableness, allocability and allowability. 42. Understand the impacts on pricing decisions caused by the behavioral differences between fixed and variable costs, including the use of break-even analysis and marginal pricing. 399 43. Understand and apply the basic tools of capital investment analysis and evaluation. 44. Understand the management issues associated with working capital and the unique sources of financing available to defense contractors. 45. Understand the approved methods of contracting and the effects of social-economic policies on the contracting process. 46. Understand the characteristics and appropriate use of various types of contracts. 47. Understand the shared organizational responsibilities and breadth of activities usually encountered during the post award phase of contracted work 48. Understand the concepts and techniques involved in the acquisition of data and data rights. 49. Understand the necessary planning for and applicability of various methods for maintaining competition throughout the acquisition life cycle. 50. Understand how a Request for Proposal is developed and who is involved. 51. Understand the formal source selection process, including planning, participants, and events. 52. Understand the mission and responsibilities of a Contract Administration Office (CAO). 53. Understand the fundamentals in preparing for a contract negotiation. 54. Able to evaluate cost and technical proposals received in response to a Request for Proposal. 400 55. Understand the management issues associated with the preparation for and conduct of contract negotiations to include proposal review, data analysis, strategy development, tactics and communication processes. 56. Understand how to prepare a solicitation that effectively communicates the Government's requirements, acquisition strategy, and evaluation criteria. 57. Understand the competitive award process and in the noncompetitive process be able to apply the techniques of pricing, fact-finding, evaluating cost and technical proposals, and negotiating from both a Government and contractor perspective. 58. Understand the historical development, principles, objectives, and current DoD policies concerning contract performance measurement. 59. Understand how the five Cost/Schedule Control System criteria areas evaluate a contractor's management control system. 60. Understand the key Cost/Schedule related elements which should be considered when developing a Request for Proposal. 61. Understand the Cost/Schedule implementation and review process, to include tailoring the application and identifying the typical involvement by the PMO. 62. Understand how a realistic and valid Performance Measurement Baseline is developed and evaluated, and describe the process used to effectively manage change to this baseline. 63. Understand how the principal financial reports used on major acquisition contracts are used to measure and manage the status of contractor performance. 64. Able to prepare a written analysis of the contractor's performance, to include an assessment of performance to date, the Estimate at Completion, and indirect cost performance by reviewing Cost Performance Reports and other pertinent data, considering relevant cost, schedule and technical factors. 401 65. Understand the roles of the personnel responsible for C/SCSC surveillance, how the PMO coordinates this effort to enhance effective contract oversight. 66. Understand the documents and processes a PMO must follow for external reporting of a program's cost/schedule status, to include typical contents and reporting levels. 67. Able to analyze program cost, schedule, and performance progress/deficiencies through use of Cost Performance Reports to identify problem areas, assess baseline impacts and develop recommendations. 68. Understand how cost analysis contributes to the program decision making process. 69. Understand the concepts of risk evaluation from a statistics, sensitivity and key assumptions (i.e., cost driver selection) perspective. 70. Understand the development of the cost estimate and the ensuing analysis from both the DoD and Contractor perspective. 71. Know the estimating techniques and principles. 72. Able to formulate viable and dependable cost estimates for acquisition programs. 73. Understand the relationship of the budgeting process to the weapon system acquisition process. 74. Understand the Planning, Programming and Budgeting System and its effect on systems acquisition. 75. Understand the Congressional enactment process. 76. Understand the budget execution process. 402 77. Understand the concept of the annual budgetary cycle and the relationship of budgetary processes (planning, programming, enactment and execution) to the systems acquisition process. 78. Understand the role or reporting and backup documentation (SAR, RDDS, P-Forms, R -l, P -l, FYDP, etc.). 79. Understand the processes of cost and budget reviews and how they interrelate with the program review/decision process and associated reporting. Able to prepare and analyze a program cost estimate and POM/budget input. 80. Understand Integrated Logistics Support (ILS) requirements and elements (10) and their relationship to the systems acquisition process. 81. Understand the requirements for an implementation of Reliability, Availability and Maintainability (RAM). 82. Understand the concept of Logistics Support Analysis (LSA) and its role in defining, identifying and establishing the logistics support structure for a system. 83. Understand how and where ILS requirements fit into the contract package. 84. Understand ILS objectives and activities and requirements associated with each phase of the system life cycle. 85. Understand the issues and management tasks applicable to Fielding/Fleet Introduction/Turnover (Site Activation) with emphasis on the integration of various flinctional specialties. 86. Understand the DoD logistics policy and related management tools, such as logistic support analysis (LSA), the logistic planning and contracting processes, and how these should be applied to program office and contractor logistic efforts during the acquisition life cycle. 87. Able to conduct and analyze an industrial base assessment. 403 88. Able to analyze a production strategy for technical and economical congruence with the program acquisition strategy. 89. Understand producibility/manufacturability design philosophies and design techniques. 90. Able to analyze a manufacturing plan for completeness to include technical and economic feasibility. 91. Able to plan and conduct a Production Readiness Review. 92. Understand the theory of variability and be able to manage the implementation and application of statistical process control. 93. Understand production risk areas and be able to recommend risk reduction actions. 94. Understand manufacturing issues at all major design reviews, with the associated technical and economic implications. 95. Understand the responsibilities and characteristics of effective quality assurance activities for both the contractor and the DPRO organizations, and how these organizations should interact with the PMO/SPO. 96. Able to develop and evaluate manufacturing related program milestone exit criteria/metrics. 97. Able to analyze the issues and management tasks applicable to production planning to include integration with the design process, long lead item planning and conduct of Production Readiness Reviews. 98. Able to analyze typical production problems and issues to include production capacity rate determinations, production tooling planning, make- buy determinations, choke points and competitive advantage to include development of proposed solutions. 404 99. Know current manufacturing technology and techniques as being employed in the defense industry. 100. Understand the principles of manufacturing management and how these principles should be applied to influence the design process, transition to production, and execution of the manufacturing plan. Able to analyze program office and contractor status/plans for transition to production. 101. Understand the risk management process as a program management tool and the role of risk management in acquisition program planning and control. 102. Understand the use of the acquisition program baseline as a control tool, to include an understanding of the difference between thresholds, objectives, and parameter values on maturity/growth curves. 103. Understand the use of exit criteria and their relationship to reduction of cost, schedule, and performance risk for a particular phase of a program. 104. Understand the advantages, disadvantages, capabilities, and appropriate usage for the following: Network Chart, Milestone Chart, SWAN Chart, Lead-time Chart, and Line of Balance (LOB) Chart. 105. Understand the procedures for program initiation, and the role which key planning and control documents (such as the ORD, COEA, APB, and IPS) play in the integration of the acquisition planning process during the CE/D phase. 106. Understand the impact that Deming's and Juran's Quality Management principles have had on the traditional principles and functions of management. 107. Understand the various types of organizational models and how to apply those models through the phases of the acquisition life cycle in support of defense systems acquisition. 108. Able to apply quantitative problem solving methods such as decision analysis, multi-attribute utility analysis, regression analysis, basic probability 405 theory, and risk analysis in addressing program management problems and issues. 109. Able to apply qualitative problem solving methods such as brainstorming, nominal group technique, Kepner-Tregoe problem analysis, and the Delphi techniques in addressing program management problems and issues. 110. Able to analyze the history and circumstances surrounding a controversial major acquisition program and be able draw conclusions as to lessons learned and corrective actions. 111. Able to develop an acquisition strategy that provides for effective management of cost, schedule, and performance risk, while meeting the policy requirements of DoD 5000 series instructions. 112. Understand the government role in Technique Reviews. 113. Understand the government role and industry role in Systems Engineering Management. 114. Able to apply Risk Management techniques. 115. Able to apply technical Performance Measurement. 116. Understand concurrent engineering. 117. Able to develop and apply a Work breakdown Structure in preparing a Statement of Work. 118. Understand configuration management and the role of systems engineering in product improvement. 119. Understand the essential elements of the Systems Engineering process. 406 120. Understand the general concepts and considerations involved in the application of computers and data bases to Systems Engineering Management. 121. Understand the content and application of both the Government SEMP and the Contractor SEMP. 122. Able to apply the planning processes associated with preparation and execution of a technical review in the context of a systems engineering process. 123. Able to analyze interfaces between the hardware and software development cycles and apply program management techniques to resolve scheduling and programmatic conflicts. 124. Understand the framework for the acquisition and management of Mission Critical Computer Resources (Software) as an integrated part of a weapon system. 125. Understand the DoD software development process and the software life-cycle to include their relationships to the overall System Acquisition Process. 126. Understand typical software acquisition risks and appropriate risk reduction strategies to include the advantages and disadvantages of each strategy. 127. Able to recognize and analyze unique software requirements for software contracting (solicitation), source selection issues and evaluation criteria and program documentation. 128. Understand the tools and techniques available to the PMO for measuring and predicting software development progress. 129. Understand current DoD policies and practices relevant to software test program planning, execution, and evaluation to ensure testing sufficiency. 407 130. Understand Software Development from the perspective of the contractor. 131. Understand how to analyze program office and contractor status/plans for mission critical computer resources (MCCR) development, integration, management and support. 132. Understand how to develop, document, and staff a test and evaluation program in DoD. 133. Know the legal requirements imposed upon DoD Test and Evaluation. 134. Understand the relationship of Test and Evaluation to the systems acquisition process. 135. Able to develop and manage the development of an acceptable Test and Evaluation Master Plan. 136. Know the opportunities for programs to benefit from T&E activities. 137. Know the respective roles of Government and industry in the management of DoD T&E. 138. Understand the types and purposes of Developmental and Operational Testing. 139. Able to integrate the various aspects of T&E with each other, and with other programmatic requirements of systems acquisition. 140. Able to analyze issues and critical criteria so as to develop a comprehensive and responsive test and evaluation plan. 141. Understand the cause, consequences and techniques for managing stress in acquisition management environment. 142. Able to recognize the major time wasters in acquisition management and develop individual strategies to manage the use of time. 408 143. Able to apply total quality management tools within a total quality paradigm. 144. Able to apply managerial, interpersonal, organizational and leadership skills to develop an effective and efficient program management organization. 145. Able to build coalitions with influential players, program sponsors, users and contractors and to orchestrate situations to overcome obstacles, obtain support and to accomplish tasks through the efforts of others. 146. Know the specific interests, motivations, strengths, and weaknesses of others to take and maintain positions despite anticipated resistance or opposition from influential others. 147. Able to evaluate performance in accomplishing goals or meeting specific standards. 148. Able to organize resources to analyze problems, develop solutions, and to implement new ways of meeting program requirements. 149. Able to formulate tailored, human-skills approaches based on individual differences, the interpersonal communication process, and small group dynamics to maximize interpersonal effectiveness in the program management environment. 409 APPENDIX 3: Dear Prospective Student, To help DSMC create a classroom environment which will best meet your needs, we have designed the Learning Environment Preference questionnaire. By answering all the questions in the manner indicated, you will be providing us with the necessary information for us to create for you an ideal classroom learning environment. GENERAL INSTRUCTIONS: 1. Please answer all survey questions. 2. Before answering the survey questions, please answer the following: a. Your name__________________________________ b. Your gender_________________________________ c. Your years of Acquisition Experience_________ d. Service______________________________________ e. Rank or GS grade____________________________ 3. This survey consists of five sections appropriately marked on the questionnaire. After you have rated all the items in each section, you will be asked to identify the item number of the three (3) most important items in each section at the bottom of the page. Mark the item number in the space indicated. 4. If you have any questions related to the survey refer them to Professor Jay W. Gould III. 410 COURSE CONTENT / VIEW OF LEARNING Rating Scale: 1 2 3 4 Not at all Somewhat Moderately Very Significant Significant Significant Significant MY IDEAL LEARNING ENVIRONMENT WOULD: 1. Emphasize basic facts and definitions. 2. Focus more on having the right answers than on discussing methods or how to solve problems. 3. Insure that I get all the course knowledge from the professor. 4. Provide me with an opportunity to leant methods and solve problems. 5. Allow me a chance to think and reason, apply facts to support my opinions. 6. Emphasize learning simply for the sake of learning or gaining new experience. 7. Let me decide for myself whether issues discussed in class are right or wrong, based on my own interpretations and ideas. 8. Stress the practical applications of the material. 9. Focus on the socio-psycho, cultural and historical. 10. Include content which serves as a catalyst for research and learning on my own, integrating the knowledge gained into my thinking. 11. Stress learning and thinking on my own, not being spoon-fed learning by the instructor. 12. Provide me with appropriate learning situations for thinking about and seeking personal truths. 13. Emphasize a good positive relationship among the students and between the students and teacher. THE THREE MOST SIGNIFICANT ITEMS ABOVE ARE: (14)____ (15)_____ (16)____ 411 ROLE OF INSTRUCTOR Rating Scale: 1 2 3 4 Not at all Somewhat Moderately Very Significant Significant Significant Significant IN MY IDEAL LEARNING ENVIRONMENT, THE TEACHER WOULD: 31. Teach me all the facts and information I am supposed to learn. 32. Use up-to-date textbooks and materials and teach from them, not ignore them. 33. Give clear directions and guidance for all course activities and assignments. 34. Have only a minimal role in the class, turning much of the control of the course content and class discussions over to the students 35. Be not just and instructor, but also an explainer, entertainer, and friend. 36. Recognize that learning is mutual - individual class members contribute fully to the teaching and learning in the class. 37. Provide a model for conceptualizing living and learning rather than problem solving. 38. Utilize his / her expertise to provide me with a critique of my work. 39. Demonstrate a way to think about the subject matter and then help me explore the issues and come to my own conclusions. 40. Offer extensive comments and reactions about my performance in class (papers, exams, exercises, etc.) 41. Challenge students to present their own ideas, argue with positions taken, and demand evidence for why they believe what they do. 42. Put a lot of effort into the class, making it interesting and worthwhile 43. Present arguments on course issues based on his / her expertise to stimulate active debate among class members. THE THREE MOST SIGNIFICANT ITEMS ABOVE ARE: (44)____ (45)____ (46). 412 ROLE OF STUDENTS / PEERS Rating Scale: 1 2 3 4 Not at all Somewhat Moderately Very Significant Significant Significant Significant IN MY IDEAL LEARNING ENVIRONMENT, AS A STUDENT I WOULD: 51. Study and memorize the subject matter - the teacher is there to teach it. 52. Take good notes on what's presented in class and reproduce that information on the tests / papers. 53. Enjoy having my friends in class but not expect them to add much to what I get from the class 54. Hope to develop my ability to reason and judge based on standards defined by the subject. 55. Prefer to do independent research allowing me to produce my own ideas and arguments. 56. Expect to be challenged to work hard in the class. 57. Prefer that my classmates be concerned with increasing their awareness of themselves and others in relation to the world. 58. Anticipate that my classmates would contribute significantly to the learning in the course through their own expertise in the content 59. Want opportunities to think on my own. making connections between the issues discussed in class and other areas I'm studying. 60. Take some leadership along with my classmates in deciding how the class will be run. 61. Participate actively with my peers in class discussions and ask as many questions as necessary to fully understand the topic. 62. Expect to take learning seriously and be personally motivated to learn the subject. 63. Want to learn methods and procedures related to the subject - learn how to learn. THE THREE MOST SIGNIFICANT ITEMS ABOVE ARE: (64)_____ (65)____ (66)____ 413 CLASSROOM ATMOSPHERE / ACTIVITIES Rating Scale: 1 2 3 4 Not at all Somewhat Moderately Very Significant Significant Significant Significant IN MY IDEAL LEARNING ENVIRONMENT, THE CLASSROOM ATMOSPHERE AND ACTIVITIES WOULD: 81. Be organized and well structured - there should be clear expectations set (like a structured syllabus that's followed). 82. Consist of lectures (with a chance to ask questions) because I can get all the facts I need to know more efficiently that way. 83. Include specific, detailed instructions for all activities and assignments. 84. Focus on step-by-step procedures so that if you did the procedure correctly each time, your answer would be correct. 85. Provide opportunities for me to pull together connections among various subject areas and then construct an adequate argument. 86. Be only loosely structured, with the students themselves taking most of the responsibility for what structure there is. 87. Include research papers since they demand that I consult sources and then offer my own interpretation and thinking. 88. Have enough variety in content areas and learning experiences to keep me interested. 89. Be routine and structured, but be balanced by group experimentation, intuition, and imagination. 90. Consist of a seminar fonnat, providing an exchange of ideas so that I can critique my own perspectives on the subject matter. 91. Emphasize discussions of personal views based on relevant evidence rather than just right and wrong answers. 92. Be an intellectual dialogue and debate among small group of peers motivated to leam for the sake of learning. 93. Include lots of projects and assignments with practical everyday applications. THE THREE MOST SIGNIFICANT ITEMS ABOVE ARE (94)____ (95)____ (96)____ 414 GRADING / EVALUATION PROCEDURE Rating Scale: 1 2 3 4 Not at all Somewhat Moderately Very Significant Significant Significant Significant EVALUATION PROCEDURES IN MY IDEAL LEARNING ENVIRONM ENT WOULD: 101. Include straightforward, not "tricky" tests or papers, only what has been taught and nothing else. 102. Be up to the teacher since s/he knows the material best. 103. Consists of objectives - style tests because they have clear-cut right or wrong answers. 104. Be based on how much students have improved in the class and how hard they have worked in class. 105. Provide an opportunity for me to judge my own work along with the teacher and learn from the critique at the same time. 106. Not include grades since there aren't really any objective standards teachers can use to evaluate student's thinking. 107. Include grading by a prearranged point system (participation, reading or other homework, papers, tests, etc.), since I think it seems the most fair. 108. Represent a synthesis of classroom and outside opportunities for judgement and learning aimed at enhancing the quality of the class. 109. Consist of thoughtful criticism of my work by someone with appropriate expertise. 110. Emphasize essay exams, papers, etc. rather than objective style tests so that I can show how much I've learned. 111. Allow students to demonstrate that they can think on their own and make connections not made in class. 112. Include judgements of the quality of my oral and written work as a way to enhance my learning in the class. 113. Emphasize independent thinking by each student but include some focus on the quality of one's argument and evidence. THE THREE M OST SIGNIFICANT ITEMS ABOVE ARE NOTE: LAST TWO DIGITS ONLY (114)____ (115)____ (116)____ 415 APPENDIX 4: CCI Perry Frequency LEP / CCI Percent V alid Percent Cum Percent 209 1 .1 .1 .1 217 3 .2 .2 .2 218 1 .1 .1 .3 225 /I -2 o .2 .5 227 3 .2 .2 . 6 230 J. .1 .1 .7 233 10 .5 .5 1.2 240 5 .3 .3 1.5 242 7 .4 .4 1.8 245 2 .1 .1 1.9 250 15 .8 .8 2.7 255 5 . 3 .3 3.0 256 1 . 1 . 1 3.0 258 12 . 6 . 6 3.6 260 3 .2 .2 3.8 264 2 .1 .1 3.9 267 19 1.0 1.0 4 . 9 270 2 . 1 .1 5.0 272 1 . 1 . 1 5.0 273 4 . 2 .2 5.3 275 14 .7 .7 6.0 278 2 .1 .1 6.1 280 10 .5 .5 6.6 282 2 .1 .1 6.7 283 17 . 9 .9 7.6 289 1 . 1 . 1 7.6 290 5 .3 .3 7.9 291 13 .7 .7 8.6 292 30 1 . 6 1.6 10.1 300 57 3.0 3.0 13.1 307 1 .1 .1 13.2 303 32 1.7 1.7 14 .8 309 10 .5 .5 15.3 310 14 .7 .7 16.1 311 1 1 .6 . 6 16.6 317 32 1.7 1.7 18.3 313 1 .1 .1 18.4 320 9 .5 .5 18.8 322 15 .3 .8 19.6 325 45 2.3 2.3 22.0 327 11 . 6 . 6 22.5 330 18 . 9 .9 23.5 333 64 3.3 3.3 26.8 336 9 .5 C 27.3 340 20 1.0 1.0 28.3 342 52 2.7 2.7 31.0 343 1 . 1 .1 31.1 416 Vadid Cura CCI Frequency P ercent P ercent P ercent 344 18 . 9 .9 32.0 345 7 .4 .4 32.4 346 1 .1 .1 32.4 350 72 3.7 3.7 36.2 355 13 .7 .7 36.8 356 38 2.0 2.0 38.8 357 1 .1 .1 38.9 358 71 3.7 3.7 42 . 6 360 16 .8 .8 43.4 363 1 .1 .1 43.4 364 14 .7 .7 44.2 367 93 4.8 4.8 49.0 370 19 1.0 1.0 50.0 372 1 .1 .1 50.1 373 19 1.0 1.0 51.0 375 86 4.5 4.5 55.5 377 1 . 1 . 1 55.6 378 38 2.0 2.0 57.5 380 16 .8 .8 58.4 382 12 . 6 . 6 59.0 383 63 3.3 3.3 62.3 386 1 .1 .1 62.3 388 5 .3 .3 62.6 389 28 1.5 1.5 64.0 390 38 2.0 2.0 66.0 391 15 .8 .8 66.8 392 81 4.2 4.2 71.0 393 1 .1 .1 71.1 394 1 .1 .1 71.1 400 155 8.1 8.1 79.2 407 1 .1 .1 79.2 408 84 4.4 4.4 83.6 409 14 .7 .7 84 .3 410 31 1.6 1.6 86.0 411 17 .9 .9 86.8 413 2 . 1 .1 86.9 414 1 .1 .1 87.0 417 49 2.5 2.5 89.5 418 2 .1 .1 89.6 420 19 1.0 1.0 90.6 422 11 . 6 .6 91.2 425 46 2.4 2.4 93.6 427 5 .3 .3 93.9 428 1 .1 .1 93.9 429 2 .1 .1 94 .0 430 6 .3 .3 94.3 433 28 1.5 1.5 95.8 436 3 .2 .2 95.9 438 1 .1 .1 96.0 440 4 .2 .2 96.2 442 19 1.0 1.0 97 .2 417 CCI 443 444 445 450 456 458 460 467 475 478 483 486 488 500 >tal Frequency Percent 1 .1 6 .3 1 .1 21 1.1 3 .2 7 .4 3 .2 3 .2 2 . 1 3 .2 1 .1 1 .1 1 .1 1 . 1 1922 100.0 Vadid Cum Percent Percent .1 97.2 .3 97.6 .1 97.6 1.1 98.7 .2 98.9 .4 99.2 .2 99.4 .2 99.5 .1 99.6 .2 99.8 .1 99.8 .1 99.9 .1 99.9 .1 100.0 100.0 cases 1922 Missing cases TOTAL PMC 93-1 TO 95-1 PERRY LEP STATISTICS Cognitive Complexity Index (CCI) 500 400 3 0 0 i 200 - 100 - Std. Dev = 49.10 Mean = 363.9 N = 1922.00 200.0 240.0 280.0 320.0 360.0 400.0 440.0 480.0 220.0 260.0 300.0 340.0 380.0 420.0 460.0 500.0 Perry Level 419 PMC 9 4 -1 : SECTION A V alid Cum CCI Frequency Percent Percent Percei 218 1 3.6 3.6 3.6 225 1 3.6 3.6 7.1 283 1 3.6 3.6 10.7 300 1 3.6 3.6 14.3 317 2 7.1 7 .1 21.4 327 1 3.6 3.6 25.0 333 1 3.6 3.6 28 . 6 340 2 7.1 7.1 35.7 350 2 7.1 7 .1 42.9 356 3 10.7 10.7 53. 6 358 2 7.1 7.1 60.7 360 1 3 . 6 3 . 6 64 .3 375 3 10.7 10.7 75.0 391 1 3.6 3.6 78.6 392 1 3.6 3.6 82.1 400 1 3.6 3.6 85.7 409 1 3.6 3.6 89.3 417 1 3 . 6 3.6 92.9 425 1 3.6 3.6 96.4 450 1 3.6 3.6 100.0 Total 28 100 .0 100.0 420 PMC 94-1 SECTION: A 225.0 250.0 275.0 300.0 325.0 350.0 375.0 400.0 425.0 450.0 PERRY1 Std. Dev = 52.66 Mean = 351.9 N = 28.00 421 PMC 9 4 - 1 : SECTION B Valid Cum CCI Frequency Percent Percent Percei 292 1 4.2 4.2 4.2 300 1 4.2 4.2 8.3 308 1 4.2 4.2 12.5 322 1 4.2 4.2 16.7 325 2 8.3 8.3 25.0 342 3 12.5 12 .5 37.5 350 1 4.2 4.2 41.7 358 1 4.2 4.2 45.8 367 4 16.7 16.7 62.5 390 1 4.2 4.2 66.7 3 91 .1 4.2 4.2 70.8 400 2 8.3 8.3 79.2 408 o 12.5 12.5 91.7 410 1 4.2 4.2 95.8 433 1 4.2 4.2 100.0 Total 24 100.0 100.0 422 PMC 94-1 SECTION: B 8n 300.0 320.0 340.0 360.0 380.0 400.0 420.0 440.0 PERRY1 423 PMC 9 4 - 1 : SECTION C Valid Cum CCI Frequency Percent Percent Percei 250 1 4.2 4.2 4.2 300 1 4.2 4.2 8.3 317 1 4.2 4.2 12.5 320 1 4.2 4.2 16.7 330 1 4.2 4.2 20.8 350 1 4.2 4.2 25.0 356 1 4.2 4.2 29.2 358 2 8.3 8.3 37.5 367 1 4.2 4.2 41.7 375 2 8.3 8.3 50.0 378 1 4.2 4.2 54.2 380 1 4.2 4.2 58 . 3 382 1 4.2 4.2 62.5 383 2 8.3 8.3 70.8 389 1 4.2 4.2 75.0 400 2 8.3 8.3 83.3 409 1 4.2 4.2 87 . 5 417 1 4.2 4.2 91.7 420 1 4.2 4.2 95.8 500 Total 1 24 4.2 100.0 4.2 100.0 100.0 424 PMC 94-1 SECTION: C Std. Dev = 48.04 M ean = 370.7 N = 24.00 250.0 275.0 300.0 325.0 350.0 375.0 400.0 425.0 450.0 475.0 500.0 PERRY1 4 2 5 PMC 9 4 - 1 : SECTION D Valid Cum CCI Frequency Percent Percent Perce) 280 2 3.8 3.8 3.8 282 1 3.8 3.8 7.7 309 1 3.8 3.8 11.5 310 1 3.8 3.8 15.4 311 1 3.8 3.8 19.2 317 1 3.8 3.8 23.1 325 1 3.8 3.8 26.9 327 2 3.8 3.8 30.8 358 1 3.8 3.8 34 . 6 367 2 7.7 7.7 42.3 378 1 3.8 3.8 46.2 382 1 3.3 3.8 50.0 383 1 3.8 3.8 53.8 389 2 7.7 7.7 61. 5 391 1 3.8 3.8 65.4 392 2 7.7 7.7 73.1 400 4 15.4 15.4 88.5 408 1 3.8 3.8 92.3 411 2 7.7 7.7 100.0 Total 26 10C.0 100.0 426 PMC 94-1 SECTION: D 280.0 300.0 320.0 340.0 360.0 380.0 400.0 420.0 PERRY1 Std. Dev = 41.78 Mean = 364.6 N = 26.00 427 PMC 9 4 -1 : SECTION E V alid Cum ccx Frequency Percent Percent Perce 300 1 4.3 4.3 4.3 325 2 8.7 8.7 13.0 333 1 4 . 3 4.3 17.4 342 1 4.3 4.3 21.7 358 2 8.7 8.7 30.4 367 1 4.3 4.3 34 .8 375 2 8.7 3.7 43.5 383 1 4.3 4.3 47.8 383 1 4.3 4 . 3 52 .2 390 3 13.0 13.0 65.2 392 1 4.3 4 . 3 69.6 4 00 8.7 8.7 78.3 403 3 4.3 4.3 82.6 411 1 4.3 4.3 87.0 414 1 4.3 4.3 91.3 417 2 8.7 8.7 100.0 Total 23 100.0 100.0 428 PMC 94-1 SECTION: E 8i Std. Dev = 33.01 Mean = 376.5 N = 23.00 300.0 320.0 340.0 360.0 380.0 400.0 420.0 PERRY1 429 PMC 9 4 -1 : SECTION F CCI Frequency Percent Valid Percent Cum Percent 242 1 4.2 4.2 4.2 250 1 4.2 4.2 8.3 258 2 8.3 8.3 16.7 300 2 8.3 8.3 25.0 307 1 4.2 4.2 29.2 325 1 4.2 4.2 33.3 333 1 4.2 4.2 37.5 364 1 4.2 4.2 41.7 367 1 4.2 4.2 45.8 375 1 4.2 4.2 50.0 378 1 4.2 4.2 54 .2 383 2 8.3 8.3 62.5 385 1 4.2 4.2 66.7 400 i_ 4.2 4.2 70.8 40S 2 8.3 8.3 79.2 417 1 4.2 4.2 83.3 420 T _ 4.2 4.2 87.5 422 1 4.2 4.2 91.7 425 2 8.3 8.3 100.0 Total 24 100.0 100.0 430 PMC 94-1 SECTION: F 250.0 275.0 300.0 PERRY1 325.0 350.0 375.0 400.0 Std. Dev = 61.23 Mean = 355.7 N = 24.00 425.0 431 PMC 94-1 SECTION G CCI 310 311 322 340 345 350 356 383 390 392 400 409 417 425 442 450 Total V a l i d Cum F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t 1 4.5 4.5 4.5 2 9.1 9.1 13.6 1 4.5 4.5 18 .2 1 4.5 4.5 22.7 1 4.5 4.5 27 .3 2 9.1 9.1 36.4 2 9.1 9.1 45.5 2 9.1 9.1 54 .5 1 4.5 4.5 59.1 1 4.5 4.5 63.6 1 4.5 4.5 68 .2 1 4.5 4.5 72.7 1 4.5 4.5 77.3 3 13.6 13.6 90.9 1 4.5 4.5 95.5 1 4.5 4.5 100.0 22 100.0 100.0 432 PMC 94-1 SECTION: G Std. Dev = 44.21 Mean = 376.9 N = 22.00 PERRY1 433 CCI 267 325 340 350 355 360 367 370 375 383 400 408 430 438 478 PMC 94-1 SECTION H V a l i d Cum F re q u e n c y P e r c e n t P e r c e n t P e r c e n t 1 5.3 5.3 5.3 2 10.5 10.5 15.8 1 5.3 5.3 21.1 1 5.3 5.3 26.3 1 5.3 5.3 31.6 2 10.5 10.5 42.1 1 5.3 5.3 47.4 1 5.3 5.3 52.6 1 5.3 5.3 57.9 1 5.3 5.3 63.2 2 10.5 10.5 73.7 2 10.5 10.5 84.2 1 5.3 5.3 89.5 1 5.3 5.3 94.7 1 5.3 5.3 100.0 19 100.0 100.0 PMC 94-1 SECTION: H Std. Dev = 47.46 Mean = 375.7 N = 19.00 275.0 300.0 325.0 350.0 375.0 400.0 425.0 450.0 475.0 PERRY1 435 PMC 9 4 -1 : SECTION I CCI Frequency Percent Valid Percent Cum Percei 273 1 5.6 5.6 5 . 6 300 1 5.6 5.6 11.1 325 2 11.1 11.1 22.2 355 1 5.6 5.6 27.8 358 2 11.1 11.1 38.9 367 1 5.6 5.6 44.4 37 0 1 5.6 5.6 50.0 375 ± 5. 6 5 . 6 55.6 378 1 5.6 5.6 61.1 380 1 5.6 5.6 66.7 383 1 5.6 5. 6 72.2 390 1 5.6 5.6 77.8 400 2 11.1 11.1 88.9 408 2 11.1 11.1 100.0 Total 18 100.0 100.0 436 PMC 94-1 SECTION: I 280.0 300.0 320.0 340.0 360.0 380.0 400.0 PERRY1 437 CCI 258 275 300 308 336 340 342 350 356 358 367 375 390 400 408 410 411 417 420 450 PMC 9 4 - 1 : SECTION J V alid Cum Frequency Percent Percent Percent 1 4.0 4.0 4.0 1 4.0 4.0 8.0 1 4.0 4.0 12.0 1 4.0 4.0 16.0 1 4.0 4.0 20.0 x 4.0 4.0 24.0 1 4.0 4.0 28.0 1 4.0 4.0 32.0 1 4.0 4.0 36.0 1 J_ 4.0 4.0 40.0 3 12 .0 12.0 5 2 . C 2 8.0 8.0 60.0 1 4.0 4.0 64.0 1 4.0 4.0 68.0 1 4.0 4.0 72.0 1 4.0 4.0 76.0 2 8.0 8.0 84.0 2 8.0 8.0 92 .0 1 4.0 4.0 96.0 1 4.0 4.0 100.0 25 10 0 .0 1 0 0 .0 PMC 94-1 SECTION: J Std. Dev = 47.84 Mean = 368.3 N = 25.00 250.0 275.0 300.0 325.0 350.0 375.0 400.0 425.0 450.0 PERRY1 439 CCI 217 283 310 317 322 325 330 350 356 364 367 3 7 0 375 383 350 392 417 442 458 PMC 9 4 -1 : SECTION K V a l i d Cum F r e q u e n c y P e r c e n t P e r c e n t P e r c e n t 1 4.3 4.3 4.3 1 4.3 4.3 8.7 1 4.3 4.3 13.0 1 4.3 4.3 17.4 1 4.3 4.3 21.7 1 4.3 4.3 26.1 1 4.3 4.3 30.4 2 8.7 8.7 39.1 1 4.3 4 . 3 43.5 1 4.3 4.3 47 .8 1 4.3 4.3 52 .2 1 4.3 4.3 56.5 2 8.7 8.7 65.2 1 4.3 4.3 69.6 1 4.3 4 . 3 73.9 2 8.7 8.7 82.6 2 8.7 8.7 91.3 x 4 . 3 4.3 95.7 1 4.3 4.3 100.0 23 1 0 0 .0 1 0 0 .0 PMC 94-1 SECTION: K Std. Dev = 52.83 Mean = 361.0 N = 23.00 225.0 250.0 275.0 300.0 325.0 350.0 375.0 400.0 425.0 450.0 PERRY1 441 PMC 9 4 -1 : SECTION L V alid Cum CCI Frequency Percent Percent Perce: 245 1 3.3 3.8 3.8 250 1 3.8 3.8 7.7 267 1 3.8 3.8 11.5 300 1 3.8 3.8 15.4 317 2 7.7 7.7 23.1 322 1 3.8 3.8 26.9 330 1 3.8 3.8 30.8 340 1 3.8 3.8 34.6 350 1 3.8 3.8 38.5 356 1 3.8 3.8 42.3 358 2 7.7 7.7 50.0 367 2 7.7 7.7 57.7 370 1 3.8 3.8 61.5 37 3 1 3.8 3.8 65.4 375 i 3.8 3.8 69.2 378 1 3.8 3.8 73.1 390 1 3.8 3.8 7 6.9 392 2 7.7 7.7 84.6 393 1 3.8 3.8 88.5 400 1 3.8 3.8 92.3 417 1 3.8 3.8 96.2 442 1 3.8 3.8 100.0 Total 26 100.0 100.0 442 PMC 94-1 SECTION: L Std. Dev = 48.67 Mean = 352.5 N = 26.00 240.0 260.0 280.0 300.0 320.0 340.0 360.0 380.0 400.0 420.0 440.0 PERRY1 443 PMC 9 4 -1 : SECTION M V alid Cum CCI Frequency Percent Percent Percei 233 1 4.8 4.8 4.8 292 2 9.5 9.5 14.3 300 1 4.8 4.8 19.0 327 1 4 . 8 4.8 23.8 336 2 9.5 9.5 33.3 350 1 4.8 4.8 38.1 367 1 4.8 4.8 42.9 375 1 4.8 4.8 47.6 382 1 4.8 4.8 52.4 383 1 4.8 4.8 57.1 392 1 4.8 4.8 61.9 400 3 14.3 14.3 76.2 410 1 4 . 8 4.8 81.0 417 1 4.8 4.8 85.7 425 1 4.8 4.8 90.5 430 J. 4.8 4.8 95.2 433 Total .1 21 4 . 6 100.0 4.8 1CO.O 100.0 444 PMC 94-1 SECTION: M Std. Dev = 53.83 Mean = 365.7 N = 21.00 225.0 250.0 275.0 300.0 325.0 350.0 375.0 400.0 425.0 PERRY1 445 PMC 9 4 -1 : SECTION N V alid Cum CCI Frequency Percent Percent Percei 240 1 4.2 4.2 4.2 292 1 4.2 4.2 8.3 322 1 4.2 4.2 12.5 330 1 4.2 4.2 16.7 344 1 4.2 4.2 20.8 350 1 4.2 4.2 25.0 356 1 4.2 4.2 29.2 358 2 8.3 8.3 37 . 5 367 1 4.2 4.2 41.7 373 1 4.2 4.2 45.8 375 2 8.3 8.3 54 .2 383 2 8.3 8.3 62.5 391 1 4.2 4.2 66.7 400 2 8.3 8.3 75.0 408 1 4.2 4.2 79.2 409 1 4.2 4.2 83.3 422 1 4.2 4.2 87.5 425 1 4.2 4.2 91.7 458 X 4.2 4.2 95.8 475 Total 1 24 4.2 100.0 4.2 100.0 100.0 446 PMC 94-1 SECTION: N PERRY1 447 APPENDIX 5: MYERS-BRIGGS TYPE INDICATOR T otal PMC S3-1 through 95-1 Type Frequency Percent Valid Percent Cum Percent ENFJ 21 1.1 1.1 1.1 ENFP 37 1.9 1. 9 3.0 ENT J 177 9.2 9.2 12.2 ENTP 114 5.9 5.9 18.2 ESFJ 36 1. 9 1.9 20.0 ESFP 14 . 7 .7 20.8 ESTJ 293 15.2 15.2 36.0 ESTP 52 2.7 2.7 38.7 INFJ 27 1.4 1.4 40.1 INFP 3C 1.6 1.6 41.7 INTJ 253 13.2 13.2 54 .8 INTP 134 7.0 7.0 61.8 ISFJ 42 2.2 2.2 64 .0 ISFP 11 . 6 . 6 64 . 6 ISTJ c " ■ > ? U 1 * 1 29 . S 29 . S 94.3 ISTP 109 5.7 5.7 100.0 Tota 1 1922 100.0 100.0 VALID CASES 1922 Missing cases 0 448 TOTAL PMC 94-1 TO 95-1 MBTI FREQUENCY ISTJ INFJ ISTP INFP ESTP ENFP ESTJ ENFJ ISFJ INTJ ISFP INTP ESFP ENTP ESFJ ENTJ MBTI TYPE 449 * SECTION PREFERENCES SCORES * APMC 95-1 SECTION: A 30 STUDENTS ISTJ = 5 ISFJ = 0 INFJ = 1 INTJ = 8 ISTP = 3 ISFP = 0 INFP = 0 INTP = 4 ESTP = 1 ESFP = 0 E N F P = 1 ENTP = 1 ESTJ = 2 ESFJ = 1 ENFJ = 1 ENTJ = 2 E = 9 / 30 % 1 = 2 1 /7 0 % G R O U P M O D E S = 12 / 40% N = 1 8 /6 0 % T = 26 / 87% F = 4 / 13% INTJ J = 2 0 67% P = 1 0 /3 3 % SJ = 8 / 27% SP = 4 / 13% NP = 6 / 20% N J = 1 2 /4 0 % IJ = 1 4 /4 7 % IP = 7 / 23% EP = 3 / 10% EJ = 6 / 20% ST = 11 / 37% SF = 1 / 3% NF = 3 / 10% NT = 1 5 /5 0 % TJ = 1 7 /5 7 % TP = 9 / 30% FP = 1 / 3% FJ = 3 / 10% IN = 13 /4 3 % IS = 8 / 27% EN = 5 / 17% ES = 4 / 13% 450 * SECTION PREFERENCES SCORES * APMC 95-1 SECTION: B 30 STUDENTS ISTJ = 3 ISFJ = 1 INFJ = 0 INTJ = 5 ISTP = 2 ISFP = 0 INFP = 1 1NTP = 0 ESTP = 0 ESFP = 1 ENFP = 2 ENTP = 4 ESTJ = 4 ESFJ = 1 ENFJ = 1 ENTJ = 5 E = 1 8 / 6 0 % 1 = 1 2 / 4 0 % G R O U P M O D E S = 1 2 /4 0 % N = 18 / 60% T = 23 / 77% F = 7 / 23% ENTJ J = 2 0 67% P = 1 0 /3 3 % SJ = 9 / 30% SP = 3 / 10% NP = 7 / 23% N J = 11 / 37% IJ = 9/ 30% IP = 3 / 10% EP = 7 / 23% E J = 11 / 37% ST = 9 / 30% SF = 3 / 10% NF = 4 / 13% NT = 1 4 /4 7 % TJ = 1 7 /5 7 % TP = 6 / 20% FP = 4 / 13% FJ = 3 / 10% IN = 6 / 2 0 % I S = 6 / 2 0 % EN = 1 2 / 4 0 % ES = 6 / 2 0 % 451 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: C 30 STUDENTS ISTJ = 11 ISFJ = 2 INFJ = 0 INTJ = 4 ISTP = 0 ISFP = 0 INFP = 0 INTP = 3 ESTP = 1 ESFP = 0 ENFP = 0 ENTP = 0 ESTJ = 4 ESFJ = 1 ENFJ = 1 ENTJ = 3 E = 10 / 33 % 1 = 2 0 /6 7 % G R O U P M O DE S = 1 9 /6 3 % N = 1 1 /3 7 % T = 26 / 87% F = 4 / 13% ISTJ J = 2 6 / 8 7 % P = 4 / 13% SJ = 18 /6 0 % SP = 1 / 3% NP = 3 / 10% NJ = 8 / 27% I J = 17/57% IP = 3 / 10% EP = 1 / 3% EJ = 9 / 30% ST = 16 /5 3 % SF = 3 / 10% NF = 1 / 3% N T = 10 /3 3 % TJ = 22 / 73% TP = 4 / 13% FP = 0 / 0% FJ = 4 / 13% IN = 7 /2 3 % IS = 1 3 /4 3 % EN = 4 / 13 % E S = 6 /2 0 % 452 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: D 30 STUDENTS ISTJ = 8 ISFJ = 0 INFJ = 2 INTJ = 2 ISTP = 2 ISFP = 0 INFP = 0 INTP = 2 ESTP = 0 ESFP = 1 ENFP = 3 ENTP = 3 ESTJ = 2 ESFJ = 1 ENFJ = 0 ENTJ = 4 E = 1 4 / 4 7 % 1 = 1 6 / 5 3 % G R O U P M O D E S = 1 4 /4 7 % N = 16 / 53% T = 23 / 77% F = 7 / 23% INTJ J = 1 9 / 6 3 % P = 1 1 /3 7 % SJ = 11 / 37% SP = 3 / 10% NP = 8 / 27% NJ = 8 / 27% I J = 12/40% IP = 4 / 13% EP = 7 / 23% EJ = 7 / 23% ST = 1 2 /4 0 % SF = 2 / 7% NF = 5 / 17% NT = 1 1 /3 7 % TJ = 1 6 /5 3 % TP = 7 / 23% FP = 4 / 13% FJ = 3 / 10% IN = 6 / 20% I S = 1 0 /3 3 % EN = 1 0 /3 3 % ES = 6 /2 0 % 453 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: E 30 STUDENTS ISTJ = 6 ISFJ = 1 INFJ = 0 INTJ = 3 ISTP = 2 ISFP = 0 INFP = 0 INTP = 3 ESTP = 0 ESFP = 0 ENFP = 0 ENTP = 2 ESTJ = 7 ESFJ = 1 ENFJ = 1 ENTJ = 4 E = 1 5 / 5 0 % 1 = 1 5 / 5 0 % G R O U P M O D E S = 1 7 /5 7 % N = 1 3 /4 3 % E O R I T = 27 / 90% F = 3 / 10% STJ J = 2 3 /77% P = 7 / 23% S.J = 15 / 50% SP = 2 / 7 % NP = 5 / 17% NJ = 8 / 27% IJ = 10/30% IP = 5 / 15% EP = 2 / 7 % EJ = 1 3 /4 3 % ST = 1 5 /5 0 % SF = 2 / 7% NF = 1 / 3% NT = 1 2 /4 0 % TJ = 20 / 67% TP = 7 / 23% FP = 0 / 0 % FJ = 3 / 10% IN = 6 / 20% I S = 9 / 3 0 % EN = 7 / 23 % ES = 8 /2 7 % 454 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: F 30 STUDENTS ISTJ = 9 ISFJ = 1 IN FJ = 1 IN TJ = 2 ISTP = 1 ISFP = 0 INFP = 3 INTP = 3 ESTP = 1 ESFP = 0 ENFP = 0 ENTP = 4 ESTJ = 8 ESFJ = 0 EN FJ = 0 E N T J =1 E = 10 / 33 % 1 = 20/67% GROU P MODE S = 16/53% N = 14 / 47% T = 25 / 83% F = 5 / 17% ISTJ J = 1 8 /6 0 % P = 12/40% S J = 14/47% S P = 2 /7 % NP = 10/33% NJ = 4 / 13% I J = 13/43% IP = 7 / 23% EP = 5 / 17% EJ = 5 / 17% ST = 15/50% SF = 1 / 3% NF = 4 / 13% NT = 10/33% T J = 16/53% TP = 9 / 30% FP = 3 / 10% FJ = 2 / 7% IN = 9 /3 0 % IS = 11 / 37% EN = 5 / 17% ES = 5 /1 7 % 455 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: G 30 STUDENTS ISTJ = 13 ISFJ = 3 INFJ = 0 IN TJ = 3 ISTP = 0 ISFP = 1 INFP = 0 INTP = 1 ESTP = 3 ESFP = 0 ENFP = 0 E N T P = 1 ESTJ = 3 E SFJ = 0 ENFJ = 1 E N T J =1 E = 9 / 30 % 1 = 2 1 /7 0 % G RO U P M ODE S = 23 / 77% N = 7 / 23% T = 25 / 83% F = 5 / 17% ISTJ J = 2 4 /8 0 % P = 6 / 20% S J = 19/63% S P = 4 /1 3 % NP = 2 / 7% N J = 5 / 17% I J = 19/63% IP = 2 / 7% EP = 4 / 13% EJ = 5 / 17% ST = 19/63% SF = 4 / 13% NF = 1 / 3% NT = 6 / 20% T J = 20 / 67% TP = 5 / 17% FP = 1 / 3% FJ = 4 / 13% IN = 4 /1 3 % IS = 17/57% EN = 3 / 10% ES = 6 /2 0 % 456 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: H 30 STUDENTS ISTJ = 5 ISFJ = 0 INFJ = 0 INTJ = 3 ISTP = 5 ISFP = 0 INFP = 0 INTP = 3 ESTP = 1 ESFP = 0 ENFP= 1 ENTP = 3 ESTJ = 7 ESFJ = 0 ENFJ = 0 ENTJ = 2 E = 14/47% 1 = 16/53% GROUP MODE S = 18/60% N = 12/40% T = 29 / 97% F = 1 / 3% ISTJ J = 17/57% P = 13/43% SJ= 12/40% SP = 6/20% NP = 7 / 23% NJ = 5 / 17% IJ = 8 / 27% IP = 8 / 27% EP = 5 / 17% EJ = 9/30% ST = 18/60% SF = 0 / 0% NF = 1 / 3% NT = 11 / 37% TJ= 17/57% TP = 12/40% FP = 1 / 3% FJ = 0 / 0% IN = 6/20% IS = 10/33% EN = 6 / 20 % ES = 8/27% 457 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION I 30 STUDENTS ISTJ = 10 ISFJ = 1 INFJ = 0 INTJ = 8 ISTP = 3 ISFP = 0 INFP = 1 INTP = 0 ESTP = 1 ESFP = 0 ENFP = 0 ENTP= 1 ESTJ = 2 ESFJ = 1 ENFJ = 0 ENTJ = 2 E = 7 / 23 % I = 23 / 77% GROUP MODE S = 18/60% N = 12 / 40% T = 27 / 90% F = 3 /10% ISTJ J = 24/80% P = 6 / 20% SJ= 14/47% SP= 4/13% NP = 2 / 7% NJ = 10/33% IJ =19 / 63% IP = 4 / 13% EP = 2/7% EJ = 5 / 17% ST = 16/53% SF = 2 / 7% NF = 1 / 3% NT = 11 / 37% TJ = 22 / 73% TP = 5 / 17% FP =1 / 3% FJ = 2 / 7% IN = 9/30% IS= 14/47% EN = 3 / 10% ES = 4/13% 458 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: J 30 STUDENTS IS T J = 7 IS F J = 1 IN F J = 0 IN T J = 5 IST P = 0 IS F P = 0 IN F P = 1 IN T P = 3 E S T P = 0 E S F P = 0 E N F P = 0 E N T P = 2 E S T J = 6 E S F J = 0 E N F J = 1 E N T J = 4 E = 13 / 43 % 1 = 17/57% GROUP MODE S = 14 / 47% N = 16 / 53% T = 27 / 90% F = 3 / 10% INTJ J = 24/80% P = 6 / 20% SJ= 14/47% SP= 0/0% NP = 6 / 20% N J= 10/33% IJ =13/43% IP = 4 / 13% EP = 2/7% E J= 11 / 37% ST = 13/43% SF = 1 / 3% NF = 2/7% NT = 14/47% TJ = 22 / 73% TP = 5 / 17% FP = 1 / 3% FJ = 2 / 7% IN = 9 / 30% IS = 8 / 27% EN = 7 / 23 % ES = 6/20% 459 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: K 30 STUDENTS ISTJ = 7 ISFJ = 0 INFJ = 0 INTJ = 4 ISTP = 0 ISFP = 1 INFP = 0 INTP = 3 ESTP = 1 ESFP = 0 ENFP = 0 ENTP = 3 ESTJ = 6 ESFJ = 1 ENFJ = 0 ENTJ= 4 E = 15/50% 1= 15/50% GROUP MODE S = 16/53% N = 14/47% E o r l T = 28 / 93% F = 2 / 7% STJ J = 22/73% P = 8 / 27% SJ= 14/47% SP= 2/7% NP = 6 / 20% NJ = 8/27% IJ =11 / 37% IP = 4 / 13% EP = 4 / 13% EJ = 11 / 37% ST = 14/47% SF = 2 / 7% NF = 0/0% NT = 14/47% TJ = 21 / 70% TP = 7 / 23% FP = 1 / 3% FJ = 1 / 3% IN = 7/23% IS = 8 / 27% EN = 7 / 23 % ES = 8/27% 460 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: L 30 STUDENTS ISTJ = 9 ISFJ = 0 INFJ = 1 INTJ = 3 ISTP = 6 ISFP = 0 INFP = 0 INTP = 1 ESTP = 0 ESFP = 0 ENFP = 0 ENTP = 3 ESTJ = 3 ESFJ = 0 ENFJ = 1 ENTJ = 3 E = 10 / 33 % 1 = 20/67% GROUP MODE S = 18/60% N = 12/40% T = 28 / 93% F = 2 / 7% ISTJ J = 20/67% P = 10/33% SJ= 12/40% SP= 6/20% NP = 4 / 13% NJ = 8/27% IJ = 13 / 43% IP = 7 / 23% EP = 3 / 10% EJ = 7 / 23% ST = 18/60% SF = 0/0% NF = 2/7% NT = 10/33% TJ = 18/60% TP = 10/33% FP = 0/0% FJ = 2 / 7% IN = 5/17% IS= 15/50% EN = 7 / 23 % ES = 3/10% 461 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: M 30 STUDENTS ISTJ = 8 ISFJ = 1 INFJ = 1 INTJ = 6 ISTP = 1 ISFP = 0 INFP = 0 INTP = 3 ESTP = 0 ESFP = 0 ENFP = 2 ENTP = 0 ESTJ = 4 ESFJ = 0 ENFJ = 0 ENTJ = 4 E = 10 / 33 % 1 = 20/67% GROUP MODE S = 14/47% N = 16/53% T = 26 / 87% F = 4 / 13% INTJ J = 24/80% P = 6 / 20% SJ= 13/43% SP = 1/ 3% NP = 5 / 17% NJ = 11/37% IJ =16 / 53% IP = 4 / 13% EP = 2/7% EJ = 8/27% ST = 13/43% SF = 1 / 3% NF = 3/10% NT = 13/43% TJ = 22 / 73% TP = 4 / 13% FP = 2 / 7% FJ = 2 / 7% IN = 10/33% IS= 10/33% EN = 6 / 20 % ES = 4/13% 462 * SECTION PREFERENCES SCORES * PMC 95-1 SECTION: N 30 STUDENTS ISTJ = 4 ISFJ = 0 INFJ = 0 INTJ = 1 ISTP = 5 ISFP = 2 INFP = 1 INTP = 3 ESTP = 1 ESFP = 0 ENFP= 1 ENTP = 2 ESTJ = 5 ESFJ = 1 ENFJ = 0 ENTJ = 4 E = 14/47% 1 = 16/53% GROUP MODE S = 18 / 60% N = 12 / 40% 1ST T = 25 / 83% F = 5 / 17% J or P J = 15/50% P = 15/50% SJ= 10/33% SP= 8/27% NP = 7 / 23% NJ = 5 / 17% IJ =5/17% IP = 11 / 37% EP = 4 / 13% EJ = 10/33% ST = 15/50% SF = 3 / 10% NF = 2/7% NT = 10/33% TJ = 14/47% TP = 11 / 37% FP = 4 / 13% FJ = 1 / 3% IN = 5 / 17% IS = 11 / 37% EN = 7 / 23% ES = 7/23% 463 APPENDIX 6: Student Led Acquisition Management Activities Questionnaire: (Adapted from MSQ*) Section Date___ The purpose o f this questionnaire is to give you a chance to tell how you feel about the Student Led Acquisition Management Activities, what things you are satisfied with and what things you are not satisfied with. On the basis o f your answers and those o f other sections, we hope to get a better understanding o f the things people like and dislike about being a student in the Student Led Acquisition Management Activities. On the back o f this sheet you will find a statement about your experiences in the Student Led Acquisition Management Activities. - Read each statement carefully. - Decide how satisfied you feel about the aspect o f your student status described by the statement. Keeping the statement in mind. - if you feel that your student activity gives you more than you expected, check (x) under "VS" (Very Satisfactory). - if you feel that your student activity gives you what you expected, check (X) under "S" (Satisfactory). - if you cannot make up your mind whether or not your student activity gives you what you expected, check (X) under "N" (Neither Satisfactory nor Unsatisfactory). - if you feel that your student led activities gives you less than you expected, check (X) under "DS" (Dissatisfied). - if you feel that you student activity gives you much less than you expected, check (X) under "VDS" (Very Dissatisfied). Remember: Keep the statement in mind when your deciding how satisfied you feel about that aspect o f the Student Led Acquisition Management Activities. Do this for all statements. Please answer every item. Be frank and honest 464 Ask yourself: I low w ell satisfied am I with this aspeet o f the Student Led M anagement Activities'? "VS" m eans 1 am very satisfied with this aspect. "S" m eans 1 am satisfied with this aspect. "N" m eans I cannot decide whether I am satisfied or not with this aspect. "DO" means I am dissatisfied with this aspect. "VDS" m eans I am very dissatisfied with this aspect. VDS DO N S VS 1. Being able to keep busy all the time___________________________________________ ______ ___ ________ _____ 2. The chance to work on my own in the exercise________________________________ ______ ___ ________ _____ 3. The chance to do different tasks from time to time ______ ___ ________ _____ 4. The chance to be "somebody" in the exercise ______ ___ ________ _____ 5. I he way the Instructor/s handle the students___________________________________ ______ ___ ________ _____ 6. The com petence o f our work group or team in making decisions ______ ___ ________ _____ 7. Being able to do things that don't go against my conscience ______ ___ ________ _____ S. The way the exercise prepared m e for job activities after PMC ______ ___ ________ _____ 9. The chance to help other people learn new things ______ ___ ________ _____ 10. The chance to tell people what to do ______ ___ ________ _____ 11. The chance to do things that makes use o f my abilities ______ ___ ________ _____ 12. I he way D oD procurement policies are put into practice ______ ___ ________ _____ 13. The extrinsic rewards and the amount o f work 1 do ______ ___ ________ _____ 14. The chance to leam som ething new ______ ___ __ 15. The chance to try my ow n methods o f doing the activities ______ ___ ________ _____ 16. The chance to try my own methods o f doing the activities ______ ___ ________ 17. The classroom conditions 18. The wav my teammates get along with each other ______ ___ ________ _____ 19. The Instructor/s praise I get for effective decision making ______ ___ ________ _____ 20. The amount o f learning I got out o f the Student Led Acquisition Management Activities.____________________________________________________________ ___ __ _ __ 465 SATISFACTION DEFINITIONS: 1. Intrinsic Satisfaction— A person's attitude toward an activity or task that is influenced by the activity or task itself. Examples o f intrinsically satisfying activities and tasks in a work environment include the following: a. The chance to do something that makes use o f my abilities. b. The feeling o f accomplishment I get from the job. c. Being able to keep busy all the time. d. The chances for advancement on this job. e. My pay and the amount o f work I do. f. The way my co-workers get along with each other. g. The chance to try my own methods o f doing the job. h. The chance to work alone on the job. i. Being able to do things that don't go against my conscience, j. The chance to do things for other people. k. The chance to be "somebody" in the community. 1 . The working conditions. 2. Extrinsic Satisfaction— A person's attitude toward an activity or task that is influenced by sources other than the activity or task. Examples o f extrinsically satisfying activities and tasks in a work environment include the following: a. The chance to tell other people what to do. b. The way company policies are put into practice. c. The praise I get for doing a good job. d. The freedom to use my own judgment. e. The way my job provides for steady employment. f. The chance to do different things from time to time. 3. General Satisfaction— A person's attitude toward the human relations and technical aspects o f supervision. Examples o f generally satisfying activities or tasks in a work environment include the following: a. The way my boss handles his men. b. The competence o f my supervisor in making decisions. *Source. Manual for the M innesota Satisfaction Questionnaire, Bulletin 45, October 1967. M innesota Studies in Vocational Rehabilitation: XXII. 466 APPENDIX 7: TRANSITION LEVELS by MBTI TYPE MBTI1 Count INTJ ! INTP ISFJ ISFP ISTJ 1.00 1 1 15 2.00 13 6 5 1 44 3.00 5 1~ 5 2 1 38 4.00 21 j 9 | 39 j 8 2 62 5.00 69 13 2 154 6.00 78 i i 35 i 7 4 159 7.00 25 i 8 ! 2 ______ 1 44 8.00 35 1 32 1 _ L . 4 54 9.00 6 : i [ | 2 Column 253 T o ta l 13.2 134 7.0 42 2.2 11 572 .6 29.8 Row T o ta l 28 1.5 118 6.1 106 5.5 181 9.4 548 28.5 541 28.1 150 7.8 238 12.4 12 . 6 1922 100.0 467 TRANSITION LEVELS b y MBTI TYPE MBTI1 Count 1.00 ! . 00 1.00 4 .00 5.00 6.00 7 .00 S . 00 9.00 Column T o ta l ESFP ESTJ IESTP I IN F J ROW INFP i T o ta l 28 1.5 118 6.1 106 5.5 181 9.4 5 4 8 28.5 541 28.1 150 7.8 238 12.4 12 .6 1922 1.6 100.0 468 TRANSITION LEVELS by MBTI TYPE MBTI1 Count : 1.00 2.00 2NFJ ENFP ENTJ ;ENTP ESFJ Row T o ta l 28 1.5 118 6.1 3.00 4 .00 ,.00 6.00 7.00 S . 00 5.00 Column T o ta l 11 14 14 45 52 17 35 12 20 21 1.1 1.9 177 9.2 114 i. 9 36 1.9 ; i o 6 | 5.5 1 [ 181 I 9.4 | 548 , 28.5 i 541 ! 28.1 i : 150 | 7.8 1 ! 238 12.4 i : 12 | .6 1922 100.0 469 TRANSITION LEVELS b y MBTI TYPE Count 1.00 2.00 3.00 4 .00 5.00 6.00 7.00 8.00 9.00 Column T o ta l MBTI1 i ISTP j _______ : i 7 i 35 31 16 109 5.7 Row T o ta l 28 1.5 118 6.1 106 5.5 181 9.4 548 28 . 5 541 23 .1 150 7.8 238 12.4 . 6 1922 100.0 Bar Chart TRANSITION LEVEL 1 CCI 200 - 240 ENTJ ENTP ESTJ INFP INTJ ISFJ ISTJ ISTP MBTI1 471 Bar Chart TRANSITION LEVEL 2 C C I241 - 285 EN FJ ENTJ E S F J E ST P INFP INTP ISFP ISTP EN FP ENTP E ST J INFJ INTJ ISFJ ISTJ MBTI1 472 Bar Chart TRANSITION LEVEL 3 CC I286 - 300 40- ENTJ ESFJ ESTJ INFJ INTJ ISFJ ISTJ ENTP ESFP ESTP INFP INTP ISFP ISTP MBTI1 473 Bar Chart TRANSITION LEVEL 4 CCI 301 - 327 ENFJ ENTJ ESFJ ESTJ INFJ INTJ ISFJ ISTJ ENFP ENTP ESFP ESTP INFP INTP ISFP ISTP MBTI1 474 Bar Chart TRANSITION LEVEL 5 CCI 328 - 373 2001 100 J ENFJ ENTJ ESFJ ESTJ INFJ INTJ ISFJ ISTJ ENFP ENTP ESFP ESTP INFP INTP ISFP ISTP MBTI1 475 Bar Chart TRANSITION LEVEL 6 CC! 374 - 400 2001 ENFJ ENTJ ESFJ ESTJ INFJ INTJ ISFJ ISTJ ENFP ENTP ESFP ESTP INFP INTP ISFP ISTP MBTI1 476 Bar Chart TRANSITION LEVEL 7 C C I401 -416 ENFJ ENTJ ESFJ ESTJ INFJ INTJ ISFJ ISTJ ENFP ENTP ESFP ESTP INFP INTP ISFP ISTP MBTI1 477 Bar Chart TRANSITION LEVEL 8 CCI 417-461 ENFP ENTJ ENTP ESFJ ESTJ ESTP INFJ INFP INTJ INTP ISFJ ISTJ ISTP MBTI1 478 Bar Chart TRANSITION LEVEL 9 CCI 461 - 500 ENTJ ENTP ESTJ ESTP INTJ ISTJ MBTI1 479

Abstract (if available)

Linked assets