Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
An Empirical Analysis Of The Relationship Between Risk Taking And Personal Probability Responding On Multiple-Choice Examinations
(USC Thesis Other)
An Empirical Analysis Of The Relationship Between Risk Taking And Personal Probability Responding On Multiple-Choice Examinations
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
AN EMPIRICAL ANALYSIS OF THE RELATIONSHIP BETWEEN RISK TAKING AND PERSONAL PROBABILITY RESPONDING ON MULTIPLE-CHOICE EXAMINATIONS by John Sherman M artois A D issertation P resented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In P artial Fulfillm ent of the Requirem ents for the D egree DOCTOR OF PHILOSOPHY (Education) June 1971 72-3790 MARTOIS, John Sherman, 1941- A N EMPIRICAL ANALYSIS O F TH E RELATIONSHIP B E T W E E N RISK TAKING A N D PERSONAL PROBABILITY RESPONDING O N MULTIPLE-CHOICE EXAMINATIONS. U n iv e rsity o f Southern C a lifo r n ia , P h .D ., 1971 E d u cation , psychology University Microfilms, A X E R O X Company, Ann Arbor, Michigan T H IS DISSERTATION HAS BEEN MICROFLIMED EXACTLY A S RECEIVED UNIVERSITY O F S O U T H E R N CA LIFO RN IA T H E G RA D U A TE S C H O O L U N IV ER SIT Y P A R K L O S A N G E L E S . C A L IF O R N IA 9 0 0 0 7 This dissertation, written by under the direction of his..... Dissertation Com mittee, and approved by all its members, has been presented to and accepted by The Gradu ate School, in partial fulfillment of require ments of the degree of JOHN SHEBMAN MAETOIS D O C T O R OF P H I L O S O P H Y Dean Date...... DISSERTATION COMMITTEE Chairman PLEASE NOTE: Some Pages have i n d i s t i n c t p r i n t . Filmed as re c e iv e d . UNIVERSITY MICROFILMS ACKNOWLEDGMENTS The author w ishes to express his appreciation to the Torrance Unified School D istrict and to the Educational Testing Service for their co operation and assistan ce in conducting this study. Most d irectly, the author is indebted to his wife Judy, and to his fam ily, without w hose m oral support this work could not have been accom plished. ii TABLE OF CONTENTS Page ACKNOWLEDGMENTS ............................................................................................ ii LIST OF TABLES ...................................................................................................... v LIST OF FIGURES ......................................................................................................... vi Chapter I. THE PROBLEM 1 The Purpose of the Study Significance of the Study R esearch Hypotheses Definition of Term s Organization of the Rem ainder of the D issertation II. REVIEW OF THE LITERATURE .............................................. 7 Organization of the P resen t Chapter Risk Taking on Objective Examinations Probability Response System s Summary III. PR O C E D U R E ........................................................................................ 29 Subjects Exam inees Instrumentation T est Responding Procedure Design of the Study Scoring of the Instruments Hypotheses Statistical Treatm ent of the Data IV. RESULTS .............................................................................................. 43 Organization of the P resent Chapter Prelim inary Data Statistical Hypotheses Summary D iscussion of R esults ___________________ i i i ______________ Chapter P age V. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 62 Summary Conclusions Recom m endations BIBLIO G R A PH Y ................................................................................................................. 67 APPENDIXES A. NONSENSE ITEMS ............................................................................ 74 B. ANSWER SHEET ............................................................................... 79 C. GENERAL DIRECTIONS ................................................................ 81 D. DIRECTIONS FOR PROBABILITY RESPONDING .............. 83 E. PROCEDURE FOR COMPARING R E L IA B IL IT IE S 87 F. NUMBER AND PER CENT OF SUBJECTS COMPLETING VARIOUS PERCENTAGES OF ITEMS ON EACH TEST 91 G. NUMBER AND PER CENT OF PROBABILITY RE SPONSES MADE BY EIGHTH GRADE SUBJECTS ACCORDING TO ITEM SCORES ......................................... 93 H. NUMBER AND PER CENT OF PROBABILITY RE SPONSES MADE BY TENTH GRADE SUBJECTS ACCORDING TO ITEM SCORES ......................................... 95 I. NUMBER AND PER CENT OF PROBABILITY RE SPONSES MADE BY TWELFTH GRADE SUBJECTS ACCORDING TO ITEM SCORES ......................................... 97 LIST OF TABLES The Continuous Scoring M e th o d ......................................... The F ive Star M ethod............................................................... The Truncated Logarithm ic System ................................ R esults of Romberg and Shepler’s Experim ent . . . . Scoring System for Probability R esponse T est . . . . A n alysis of V ariance D e s i g n ............................................... Eighth Grade T est C h a r a c te r is tic s ................................... Tenth Grade T est C h aracteristics ................................... Twelfth Grade T est C h a r a c te r is tic s ................................ A nalysis of V ariance Summary T a b l e ............................ The Means of the Two Responding P rocedures at the Three Grade L e v e ls ........................................................... F igure ! 1. LIST OF ILLUSTRATIONS Page A P lot o f the M eans of the Two R esponding P roced u res at the T hree Grade L ev els .......................................................... 52 vi CHAPTER I THE PROBLEM M ultiple-choice tests have developed as the dominant instrum ent for the a ssessm en t of achievem ent in m ost educational situations (Nunnally, 1967). A m ajor research effort to in crease the usefu ln ess of the instrum ent has been concerned with reducing the error variance attributable to guessing. The m odel of test behavior for this effort assum ed the existence of two possible exam inee response states, either the response made is due to perfect knowl edge or the subject se le c ts one of the options by chance. The recommended scoring procedure in this situation has been to subtract a percentage of the wrong responses from the total number of correctly marked resp on ses. The assum ption of random guessing basic to this scoring system has been ques tioned by psychom etricians in light of accumulating evidence which suggests that many individuals sele ct responses on the b asis of partial knowledge (Cureton, 1966; G lass and W iley, 1964; L ittle, 1962; Lord, 1963; Lord and Novick, 1968; Lyerly, 1951; Stanley and Wang, 1968; W illey, 1960). The concept of risk taking as a personality trait has raised additional questions concerning formula scoring. R esearch indicates that individuals differ to the degree they gu ess on an item when they know there is a penalty for an incorrect response. These studies have shown that individuals differing only in their propensity to take a risk^are penalized to the extent they respond 2 con servatively to an item (Sherriffs and B oom er, 1954; Slaker, 1967b). The u se of d ecisio n theory has been proposed to allow for partial knowledge and to elim in ate the confounding effects of varying le v e ls of risk taking among exam inees by stipulating resp o n ses in term s of person al proba b ilitie s. One assum ption of probability responding req u ires an individual to co rrectly report his p erson al p rob ab ilities concerning the options to an item . The values reported m u st correspond to the individual's sub jective probabili tie s . This assum ption is violated if an exam inee m akes a resp on se indicative of risk taking. De Finetti (1965) has recognized that risk taking may s till e x ist under p erson al probability responding, but if the test is sufficiently long, th is problem w ill disappear. Lord and N ovick (1968) is referrin g to the lengthening procedure indicated de Finetti may be c o rrect in theory but ". . . any such lengthening is se lf defeating sin ce the whole rationale of considering th e se new m easu rem en t p rocedures is that they extract m ore inform ation per item and hence require few er item s to attain a sp ecific p recision " (p. 320). They further indicated that a m od el based on the existen ce of p artial knowledge is m ore r e a listic than the traditional acceptance of random gu essin g. Cronbach (1970) in d iscu ssin g probability responding suggested that "guessing behavior d oes not have enough influence on te s t sc o r e s to justify such elaborate m achinery" (p. 59). The P urpose of the Study The purpose of this in vestigation was to explore the inherent contra- 3 diction in the statem en ts by Lord and N ovick and Cronbach: D oes the use of a probability resp on se m odel control for d ifferen tial propensity toward risk taking ? This study w as concerned with the reduction in erro r variance attribu table to risk talcing behavior at sev er a l educational le v e ls through the u se of two m u ltip le-ch o ice responding and scoring proced u res. Significance of the Study The elim ination of confounding v a ria b les from m easuring instrum ents is of p rim e im portance to psychom etrician s. One such variable which exam i nees p o s s e s s in different amounts is propensity toward risk taking. Conven tional m u ltip le-ch oice scoring p rocedures confounds total te st sc o r e s with this p erson ality trait. A procedure based on d ecisio n theory iso la te s this portion of erro r variance. R esponses with p erson al p rob ab ilities toward the item options have been shown to in crea se the reliab ility of a test (Shuford and M assen gill, 1968a; Rippey, 1968). The relation between reduced e rr o r variance and the n ecessa ry "elaborate m achinery" would provide further inform ation toward in crea sin g test reliab ility. R esearch H ypotheses A review of litera tu re concerning risk taking on objective exam inations and p erson al probability responding suggested the follow ing resea rch hypoth e se s: 1. The correlation between observed sc o r e s on a conventional m ul tip le-ch o ice te s t and those on a m easu re o f risk taking behavior in c r e a se s with 4 level of education. 2. The correlation between the observed scores on a personal proba bility response te st and those on a m easure of risk taking behavior is not re lated to level of education. 3. Subjects at the three educational lev els studied who are character ized as being risk takers w ill achieve higher sco res on a conventional m ultiple- choice test than on a personal probability response test, concomitantly, sub jects characterized as non-risk takers w ill achieve higher scores on a personal probability response test. 4. The personal probability response te st w ill be m ore reliable than the conventional m ultiple-choice te st at the three educational lev els studied. Definition of Term s Conventional method. — The selection of one correct option among the alternatives to a m ultiple-choice item . The total score on the test for an ex - W am inee is calculated by standard formula scoring where S equals the total sco re, R equals the number of correct resp on ses, W equals the num ber of wrong responses not including those omitted, and k equals the number of options to the item ) (Cronbach, 1970). D eg ree-o f-b elief sta te. —An individual’s expectation as to the probable occurrence of an event prior to placing it in quantitative term s (de Finetti, 1965). i F ive-S tar Method. — A pattern of points corresponding to a perm itted j " " 1 j i " i ■ response such that the perm itted probabilities are 0 .0 0 , 0 .2 0 , 0 .4 0 , 0 .6 0 , I ! 0. 80, 1. 00. Each star being used to indicate a unit of probability equal to j ! ! j 0 .2 0 (de Finetti, 1965). j | ' | ! j High risk taker. — An exam inee who achieves a sco re in the upper j ; ! j 27 per cent on a m easure of risk taking behavior. i ! Low risk tak er. — An exam inee who achieves a sc o r e in the lower 27 i | per cent on a m easure of risk taking behavior. j i ! j | i P artial knowledge. — Sufficient inform ation concerning an item to allow! the exam inee to elim inate one or m ore, but not all of the d istractors for that ! ; item . i I Probability point. — A d eg ree-o f-b elief state exp ressed in num erical i i term s to rep resen t the probable occurrence of any event (de Finetti, 1965). i Probability responding. — The assignm ent of a probability point to an l | option in a m u ltip le-ch oice item requiring the transference of a d eg ree-o f- ! b elief state into num erical term s. The only restriction being that the sum m a- I ! tion of the probability points equal unity (Shuford and M assengill, 1969b). i i I R isk taking on objective exam inations. — A response made on the j j b a sis of ". . . gu essin g when the exam inee is aware there is a penalty for in co rrect resp o n ses” (Slakter, 1967a, p. 33). 6 Subjective prob ab ility. — "An individual’s expectation a s to the occu r ren ce of an unknown event” (Toda, 1963, p. 1). O rganization o f the E em ainder of the D isserta tio n The liter a tu re is review ed in C hapter H. C hapter III ex p lica tes the d esig n and m ethods em ployed in the in vestigation . The resu lts a re p resen ted in C hapter IV. Chapter V: p rovid es a sum m ary, co n clu sio n s, and recom m en dations . CHAPTER II i i I i j REVIEW OF THE LITERATURE i ! Organization of the P resen t Chapter j Chapter II review s two general areas. The fir st part traces the con- I j ; cept of risk taking on objective exam inations. The second review s probability I i I response system s designed to m easu re partial knowledge and control for v a ri- ! ous risk taking strategies. i [ R isk Taking on Objective Exam inations I The concept of risk taking has been defined as ". . . guessing when ! | the exam inee is aware there is a penalty for incorrect resp o n ses'1 (Slakter, j 1967a, p. 33). The fir st reported attempt to investigate and m easure risk I taking was by Granich in 1931. Specific referen ce was made to an "index of I j i guessing" based on the number of resp on ses m ade by an exam inee to ". . . questions which involve very obscure facts or newly coined nam es" (p. 145). j Granich was unable to dem onstrate a relationship between risk taking and ! ! sc o r e s on an achievem ent exam ination. j ! W iley and Trim ble (1936) attempted to ascertain the feasib ility of i i | using objective achievem ent tests in m easuring risk taking by having subjects indicate the degree of confidence they held in their resp o n ses. R eliability coefficients calculated for the three response lev els of "certainty, " "doubt," 7 ! and M gu ess" w ere found to be higher than the reliab ility of the o v era ll ach ieve m ent m easu re. R isk taking as a person ality trait was studied by Swineford (1938). ! / i | | This in vestigation was concerned with the d ifferen tial tendency to gam ble when j i j j ; faced with an unknown situation. Her procedure, sim ila r to one developed by j ! W iley and T rim ble (1936), for calculating a risk taking sc o r e was 1 , Num ber of erro rs m arked with highest confidence Gambling = “------------------------- Total e r r o r s plus one-half the o m issio n s i The assum ption of the form ula w as "an individual m ay be said to gam ble on his j I : sco re to the extent that he asks for cred it for th ose item s which he does not ; i I I ' I know" (p. 298). The r isk taking sco re w as found to be uncorrelated with and i ! j m ore relia b le than the achievem ent sco re. A m odified form of the Granich (1931) procedure w as proposed by ; Swineford and M iller (1953). They suggested the in clu sion of nonsense item s | to m easu re risk taking. The reliab ility of this method of a s s e s s in g risk taking w as estim ated to be .9 2 (Slakter, 1967b). A nalysis of the resp o n ses to the nonsense item s indicated a tendency for the b etter students to attem pt th ese item s when all exam inees w ere instructed not to gu ess. ! Z iller (1957) detailed a procedure based on the ". . . ratio of the j number of item s upon which the subject gu essed to the total num ber of item s 1 i the subject did not know, but upon which he could have gu essed " (p. 290). The j sco rin g form ula was I j i | R = [n /(n -l)]w ' [n /(n -l)w + u : w h ere R is the r isk sc o r e , n is the num ber of a ltern a tiv es, w is the number ; I o f in co rrect r e sp o n se s, and u is the number o f item s om itted. Z iller believed! I I | I that sin ce the R sc o r e is a p erson ality c o r r e la te , it m u st add to the erro r i ! ; v a ria n ce on an ach ievem en t te s t and be con sid ered when new m u ltip le-ch o ice i ! | sco rin g and responding p roced u res are proposed. | R esea rch designed sp ecifica lly to d eterm in e what effects differen tial propensity toward r isk taking has on the ob served sc o r e s on a m u ltip le-ch o ice j t e s t was conducted by Votaw (1936). R isk taking w as defined a s R = (n -u )/n ; I : w here u is the num ber of unattem pted item s and n is the total number of j ! ite m s on the test. R esu lts of th is study sugg ested that ascendant students had I a definite advantage over th eir m ore su b m issiv e counterparts when told not to | g u e ss at unknown ite m s. T his finding held true when both groups had equal i l : ability to answ er the sam e num ber of om itted ite m s. T here w as evidence to i j ' indicate the better students profited m o re than the le s s able when required to an sw er a ll item s. The com bination of low ability and ascendant person ality i w as found to be m o st harm ful to an exam inee when tested under ”do not g u e s s ’* j in stru ctio n s. Votaw, in review ing th ese findings, concluded M . . . the p ro- I v isio n that item s m ay be om itted introduces into a tr u e -fa lse or a m u ltip le- ! ch oice te s t, in part at le a st, a m ea su re of p erson ality tra its: consequently j the p rovision red u ces the valid ity of such te sts if they purport to m easu re know ledge of sub ject m a tter” (p. 703). i I 10 1 I R esea rch concerned w ith identifying contam inating p erson ality fa cto rs j i i in ob served s c o r e s on a conventional m u ltip le-ch o ice te st w as undertaken by i ! I ; S h erriffs and Boom er (1954). They hypothesized ". . . students who lack self-j | t i j | con fid en ce, esp ecia lly confidence in th eir own judgm ent, who had d ifficu lties inj ! j ! m aking d ecisio n , and who w ere e a sily threatened by am biguous situ ation s, j I ■ ■ ! I would be penalized by the R-W te s t condition1 ' (p. 84). The determ ination as j I to what extent individuals p o sse sse d th e se c h a r a c te r istic s w as by m eans of the | A sc a le of the MMPI. This s c a le operationally m e a su r e s in troversion , ru m i- ' nation, and an undue con cern about the im p re ssio n one m ak es on oth ers. [ S h erriffs and B oom er (1954) scored an exam ination in three w ays, a ! right m inus wrong s c o r e , a num ber right sc o r e when requested to answ er all i ite m s, and a sco re w hich rep resen ted the d ifferen ce betw een the fir st two j s c o r e s . A n a ly sis indicated the m o re m aladjusted individuals, in term s of the i 1 ' A sc a le c rite r io n , om it m ore item s even when they are capable of co rrectly I an sw erin g them . Subjects knowledge of the co u rse content w as not a factor. The con clu sion of this study w as that form ula sco rin g penalized th ese students fo r th eir m aladjustm ent by causing them to be m ore co n serv a tiv e than norm al i i ! students when responding to th is type of instrum ent. | ! E vidence that varying am ounts of risk taking behavior d ifferen tially I j affect observed sc o r e s has been presented by Q uereshi (1960) and Ham m erton i i (1965). Q uereshi em ployed fa cto r analytic p roced u res to show the ex isten ce I t ! of a relation sh ip betw een te st s c o r e s and what he term ed "adaptive R isk - A cceptance. " The in clu sion of th is factor in te s t s c o r e s elim inated the p o ssi 11 bility of a pure m ea su re of achievem ent. It w as suggested that te st con stru c to rs determ in e the influence of this factor and w eight the sc o r e s proportionately to a rriv e at a m ore accurate estim a te of an exam inee's ability. A m ajor re sea r ch effort concerning risk taking behavior on objective te sts has been conducted by Slakter (1967a, 1967b, 1969; Slakter and K oehler 1969). This rese a r ch has focused on the developm ent of a reliab le and ea sily adm inistered index of risk taking behavior on objective exam inations (1967a, 1967b). Slakter (1967a) suggested the inclusion of a sm a ll number of n onsense item s into a te st is m o st satisfactory for m easuring this trait. The follow ing exam ple of a nonsense item is used in an educational psychology test: What is the ch ild 's age in months when the Lumdane Effect is usually fir st noticed ? A. 1 -2 B. 3 -4 C. 5 -6 D. 7-8 (1967a, p. 37) Item s of this type have neither c o rrect nor in co rrect altern atives. The risk taking sco re for a subject is m ea su re by the ratio betw een the num ber of non s e n se item s attem pted to the total number of such item s in the test. A r e lia b ility of . 95 for this m easu re of risk taking has been shown with as few as six nonsense item s. Slakter (1969) a lso provided evidence to indicate the gen erality of r isk taking m ea su res on variou s types of aptitude and achievem ent te sts. Subjects w ill sc o r e high on risk taking ir r e sp e c tiv e of content area. Slakter concluded the testin g situation used to m ea su re risk taking could be decided on the b asis of e a se and convenience of adm inistration. Item s of the synonym - 12 antonym type placed into a vocabulary te s t b est m e e t th ese c riterio n to m easure r isk taking. An additional study by Slakter (1967b) in vestigated the relationsh ip b etw een r isk taking s c o r e s based on n o n sen se item s and ob served s c o r e s on vocabulary and verb al reasoning te s ts . The co rrela tio n , as p red icted , w as p o sitiv e fo r both undergraduate students and graduate students w hen the te s ts w ere sco red in a conventional m anner. T his finding w as attributed to a p o s i tiv e expected valu e for r isk taking when responding in a conventional m anner to m u ltip le-ch o ice te sts w hich contain a penalty fo r g u essin g . T his expected value i s based on the b e lie f that risk tak ers w ill sco re higher on a conventional in stru m en t when com pared to m o re co n serv a tiv e sub jects o f equal ability. Ex am in ees operating under this r isk strategy are m o re lik ely to take a chance on the b a sis of p artial knowledge when they a re not com p letely su re of an answ er. The m o re co n serv a tiv e subject refrain s from m aking a ch o ice in th is situation. The co rrela tio n betw een risk taking and sc o r e s on an ach ievem en t exam ination w as in the hypothesized n egative d irection fo r undergraduate but not graduate students. This r e su lt occu rred when both w ere required to r e spond by c r o ssin g out a ll of the recogn izab le d istra cto rs to an ite m . The b a sis fo r the finding and the hypothesis was th e n egative expected valu e of risk taking when responding to an instrum ent designed to m ea su re d e g r ee s of p artial know ledge. In a subsequent in vestigation th e co rrela tio n betw een r isk taking and s c o r e s on a vocabulary te st u sin g conventional m u ltip le-ch o ice responding | 13 I f 'procedures was found to be negative for college freshm en (Slakter, 1967b). Sim ilar resu lts occurred between risk taking and observed sco res on language ' and m athem atics aptitude and achievem ent exam inations for eighth grade stu - | dents. The correlations in both ca ses based on the expected value of risk taking w ere predicted to be positive. Slakter advanced two tentative hypotheses I to explain the contrary findings among the eighth grade students; either a lack i o f testw isen ess among students at this grade lev el or students knowing sco res would not enter into c la ss grad es made no attem pt to choose the co rrect re sponse. An exam ination of sev era l scatter diagram s with the high risk takers elim inated still indicated negative correlation s. This was taken as substan tiating evidence for the fir st hypothesis sin ce M . . . the conjecture that the ; n egativeness was due to disin terested exam inees was considered le s s plaus ib le” (p. 30). I Evidence that subjects utilizing a high risk strategy achieved sig n ifi- i cantly higher sco res than subjects with a low risk strategy while operating under standard m u ltip le-ch oice testing conditions with d o-n ot-gu ess instruc tions w as developed by Slakter (1967b). He concluded "In general, it appears i that exam inees adopting a cautious RTOOE [risk taking on objective exam ina- i : tions] strategy w ill achieve low er test sc o r es than if they had followed a le s s ; cautious RTOOE strategy” (p. 40). Probability R esponse System s i j V arious m ethods for item responding and scoring procedures have I been proposed to u tilize partial knowledge. These m ethods did not provide for 14 r isk taking on ob jective exam inations (David and F ife r , 1959; D r e s s e l and Schm id, 1953; C oom bs, 1953; C oom bs, M ilholland, and W orner, 1956; M ichael. 1967; H errell, 1967). To sp e cific a lly control for a ll p h a ses of g u essin g be h avior, as w e ll as to provide fo r the m easu rin g o f varying am ounts of p artial know ledge, m ethods of responding based on the theory of su b jectiv e probability have been proposed (de F in etti, 1965; Shuford, A lbert, and M assen gill, 1966). De F in etti (1965) identified the b a sis of th ese new m ethods as ". . . the total know ledge and b elief of a given subject about a q uestion and its r p e r m itted a ltern a tiv e a n sw ers is ex p ressed and fully rep resen ted by the p robabili tie s p , p . . , p he attaches to each of the altern a tiv es 1, 2, . . . , r being 1 2 r the righ t an sw er (p. 90). The m odel proposed by de Finetti is based on th ese assu m ption s: (1) the sc o rin g m ethod and the m anner of responding is known to the exam inee, (2) the exam inee is d esiro u s of attaining a m axim um s c o r e , and (3) the exam i nee is able to tra n sla te h is own b e lief sta tes into n u m erical p ro b a b ilities. Two m ethods of responding w ith p erso n a l probability le v e ls to the options in a m u ltip le -ch o ic e te s t w ere developed. The Continuous Method T his m ethod p rovid es m axim um d iscrim in a tio n sin ce it u se s the exact id en tification of the p rob ab ilities held by an exam inee in regard to a ll the options in a rriv in g at a sc o r e . The proposed sco rin g form ula is S = 2 P, - - 2 15 w here S equals the item score, equals the probability attached to the cor rect response and P equals the probability assigned to an option. De Finetti (1965) stated that the b asis of this scoring function is "to oblige the subject to reveal his true b eliefs, because any falsification w ill turn out to be his disad vantage" (p. 91). The example in Table 1 illu strates this system where A is the correct choice. TABLE 1 THE CONTINUOUS SCORING METHOD Subjects A B Choice C D .... E Score 1 1 0 0 0 0 1.00 2 .7 .3 0 0 0 .82 3 .5 . 5 0 0 0 .50 4 .3 .7 0 0 0 .02 5 .2 . 7 .1 0 0 -. 14 6 .2 . 2 .2 .2 . 2 .20 7 0 1 0 0 0 -1 . 00 Subject one received maximum item value when the exam inee's total subjective probability was concentrated on the correct option and subject seven received the minimum value with a concentration on a single wrong alternative. Subjects two through six indicate that an item score is dependent on the probabilities assigned to the correct option and to the distractors. Subjects who respond on 16 the b asis of m is-in form ation are penalized to the extent they concentrate I probability points on the wrong option. j ! i ■ The Strict L east D istance Method 1 1 — I i ! T his system is a sim plification of the continuous response method. I | ! The exam inee is lim ited to a given pattern of probability points which m ust sumj I j ! to unity. Scoring is the sam e as in the continuous method. The Five Star i Method proposed by de Finetti (1965) is an exam ple of the method. Each star J I j corresponds to a probability of 0 .2 0 . De Finetti regarded the u se of sym bols ; ! I ! 1 instead of figures as ". . . a suitable d evice for a rough indication of probabil-] i i ity, which even rather naive people could u se with at le a st an intuitively c o r - ! i i | rect understanding" (0. 102). D isregarding perm utations, there are seven j | | ; ways of allotting five stars to an item of five or m ore options: 5, 4 -1 , 3 -2 , 3 - i i I 1 -1 , 2 -2 -1 , 2 -1 -1 -1 , l - l - l - l - l . Table 2 illu stra tes the sco res obtained when ■ varying lev els of probability are assigned to the co rrect option. The individu- i j | a lrs sco re is a function of how the probability points are distributed over the altern atives. A subject who distributes his allowable probabilities in the form ; . 80 - . 20 to two of the options in an item receiv es a sco re ranging from 4 to I I 24. An assignm ent of . 80 to the co rrect option would receive a sco re of 24. ! i An assignm ent of . 20 to this option would receiv e a sco re of 8. A sco re of 4 ! would resu lt if neither of the two le v els of probability w ere assigned to the i i correct option. A sim ila r interpretation holds for the rem aining distributions. A ll sco res in this table have been transform ed to a sca le ranging from 0 to 25 TABLE 2 THE FIVE STAR METHOD 17 Type D istribution of Corresponding Score Stars P robabilities 5 * * * * * 1. 0 25 0.0 0 4-1 * * * * 0. 8 24 * 0.2 8 0.0 4 3-2 * * * 0.6 21 * * 0.4 16 0. 0 6 3 -1 -1 * * * 0 .6 22 * 0 .2 12 * 0 .2 12 0. 0 7 2 -2 -1 * * 0. 4 18 * * 0 .4 18 * 0. 2 13 0. 0 8 2 -1 -1 -1 * * 0 .4 19 * 0 .2 14 * 0 .2 14 * 0. 2 14 0. 0 9 l - l - l - l - l * 0 .2 15 * 0 .2 15 * 0 .2 15 * 0. 2 15 * 0. 2 15 0. 0 10 I 18 j to rem ove negative num bers and fra ctio n s. De F in etti suggested exam inees should not be taught the co m p lex ities of nor even have a c c e s s to the table. , M The intuitive m eaning of the p o ssib ility of distributing the probability, e.g. , j 0. 6 + 0.4 o r 0.6 + 0. 2 + 0. 2, and an understanding of the idea of a better I a greem en t of one’s p referen ces with one d istrib u tion rather than another are [ th e only req u irem en ts presupposed ... ” (p. 105). T his m ethod elim in a tes, I accord in g to de F in etti, the gu essin g problem . The only suggested restrictio n J i is that exam in ees be inform ed of the ru les for a ssig n in g p rob ab ilities. Cronbach (1970) review ed th ese responding and sco rin g proced u res and questioned w hether the effects of g u essin g a re se rio u s enough to ju stify such an involved procedure. The one value that he saw in such a sy stem is as ! i an instructional d ev ice in ord er ” . . . to help the student fa ce and report his : u n certa in ties" (p. 59). Lord and N ovick (1968) con sid ered de F in etti's proposed item scorin g | and responding p roced u res as a ’’. . . v ery p rom isin g lin e of in vestigation into m ethods o f extractin g m o re inform ation per te s t item " (p. 320). In p articular a m odel based on the ex isten ce of p artial know ledge is m o re r e a listic than the I trad ition al acceptance in random g u essin g . Lord and N ovick suggested the i | F ive Star Method a s the m o st p rom isin g fo r em p irica l testin g , but any evalu a- i tion m u st w eigh the value of the additional inform ation gained against the added | c o sts of in creased tim e, m oney, and effort. j Scoring p roced u res have been proposed by Shuford, A lb ert and M a s- | sen g ill (1966) for u se with p erson al probability responding. They noted that j texts in the area of psychom etrics sum m arily d ism issed the question of g u ess- i ■ ing with the inference that it ”. . . p oses no real difficulty for applications of ; | objective testing” (p. 1). This inference was contrasted to the expectations of | i j their theoretical m odel. This m odel dem onstrated . . that the existence of | guessing reduced both test reliability and m easurem ent validity . . . "(Shuford, i i | 1967b, p. 8 ). Personal probability responses to the alternatives does not re- i i i quire a choice. The only requisite is that a subject accurately report his d e- j | I g ree-o f-b elief state concerning the correctn ess of the item options in term s of [ j I ! num erical probabilities. It was suggested the u se of such a system would elim inate the problem of guessing with a concomitant reduction in error variance, j ■ This would, in term s of the m odel, in crease reliability and validity. Shuford i and M assengill (1968) further hypothesized that increased exam inee experience i i | with probability responding w ill m ake it possible to approach perfect te st r e - ] | liability. Probability responding as a minimum w ill yield information equiva- | lent to that of a conventional test based on the assum ption that an exam inee t j ■ would choose the answ er under conventional testing procedures to which he assigned maximum probability. I The procedures developed by Shuford, A lbert, and M assengill (1966) I have been term ed "adm issible probability m easurem ent procedures. ” They ! | w ere ”. . . designed in a very sp ecial manner that any student, at whatever i lev el of knowledge or sk ill, can m axim ize his expected sco re if and only if hehon- I i estly reflects his d eg ree-o f-b elief probabilities” (p. 126). These scoring sy s tem s relate an exam inee's sco re to the lev el of probability assigned to the cor 20 rect answ er and the d istra cto rs. Shuford et al. (1966) developed a scoring sy stem for ea se of adm inistra tion with paper and pencil te sts. This procedure, the truncated logarithm ic scorin g sy stem , has the sa m e p rop erties as the above sy stem s for assigned p rob ab ilities ranging from . 027 to . 973. Any lo s s in accuracy attributable to truncation would be insignificant. The form ula for scoring is 1 + log rk fo r . 0 1 < r k < l GK(rk) = - 1 for 0 £ rk < . 0 1 w here rk is the probability weight assig n ed to the c o r rect choice. Table 3 illu stra tes this scoring system for various prob ab ilities a s - . signed to the c o r r ec t ch oice. (See p. 21.) This sy stem is dependent on the probability assign ed to the co rrect ch oice. The range of s c o r e s is -1 to +1. A subject who a ssig n s a 0 to the c o r r e c t choice through m isin form ation is penalized in a m anner sim ila r to right m inus wrong scorin g. Lord and N ovick (1968) recom m ended em p irical a n a ly sis of the truncated logarithm ic sco rin g sy stem . E m pirical testin g of personal probability responding was conducted by Shuford and M assen gill (1968a) at Lackland A ir F orce B ase. U sing the A ir m an Q ualifying Exam ination- 6 6 (AQ E-6 6 ), this study com pared sc o r e s d e term ined by conventional m u ltip le-ch oice testin g and the logarithm ic scoring sy stem . The in itia l an a ly sis of the logarithm ic scorin g procedure showed 21 TABLE 3 THE TRUNCATED LOGARITHMIC SYSTEM Subject Probability Score 1 0 - 1 . 0 0 0 2 . 1 . 0 0 0 3 .4 . 602 4 .7 . 845 5 1 . 0 0 1 . 0 0 0 Note: Adapted from Rom berg and Shepler, 1968, p. 5. j 22 i j "... a functional rela tio n to e x is t betw een the d egree of confidence assign ed ■ to an answ er and the rela tiv e frequency w ith which it is , in fa ct, the c o r r e c t ! ; answ er" (p. 4). I Although the AQE- 6 6 is a w ell-d evelop ed te s t with published reliab ility i co efficien ts ranging from . 84 to . 94, an a ly sis of the probability resp o n ses of I the su b jects showed they encountered gu essin g situations on approxim ately one- fourth of the ite m s. T his w as encouraging for the continued developm ent of i 1 probability responding sin ce g u essin g resp o n ses in conventional choice te sts in c r e a se e r r o r varian ce. C om parisons of the m ean s and standard deviations fo r both p roced u res indicated that conventional responding resu lted in lo w er j m ean s and in la rg er standard deviation s for the four su b -te sts of the A Q E-6 6 . j Shuford and M a ssen g ill (1968a) attributed th is to the te s t being ". . . too ea sy i ’ fo r optim al perform an ce as a confidence test" (p. 9). i ; The r elia b ility co efficien t for the conventional resp o n se m ethod w as i I 0. 954. The com parable co efficien t for the exp erim en tal procedure w as 0. 961. i : The sm a lle r varian ce of the exp erim en tal procedure w as expected to ca u se a d e c r e a se in the relia b ility co efficien t as would ord in arily happen to a c o r r e la - ' tion co efficien t with a reduction in varian ce. T his did not occu r. The in v e sti- i j ga to rs com m ented : although th ese d ifferen ces in relia b ility m ay appear to be triv ia l, they | a re not. The co rrela tio n co efficien t is not a lin ea r m ea su re of testin g efficien cy . As the co rrela tio n co efficien t approaches 1, sm a ller and ; sm a lle r d ifferen ces becom e m ore im portant, (p. 1 0 ) | The conventional m ethod required the addition of 121 item s to the 150 item testj I ! to ach iev e c om parable relia b ility . 23 Romberg and Shepler (1968) using scorin g system s proposed by Shuford et al. (1968) analyzed the effect of probability responding on the reliability of a m ultiple-choice test. Four scoring system s w ere applied to a te st designed to m easure insightful m athem atical thinking ability. The com parison group was scored for the total number of correct choices. The three remaining groups were scored by (1 ) summing the probability weights assigned to the correct choices, (2) the spherical scoring system ,'1 ' and (3) the truncated logarithm ic scoring system . This study hypothesized that probability m easuring procedures would yield higher reliability coefficients and higher mean sco res than the conven tional method. The instructional period relating to how one should respond to a test utilizing a probability m easuring procedure varied from fifteen minutes at the twelfth grade to forty m inutes at the eleventh grade. Table 4 sum m arizes the resu lts of th is study. ^The Spherical System where X. is the probability assigned to the Jth alternative and X is the correct choice. (Shuford, Albert, and M assengill, 1966) 24 TABLE 4 RESULTS OF ROMBERG AND SHEPLER'S EXPERIMENT Control Sum Sum Treatm ent Spherical Logarithm ic 12th Grade X = 6.75 X = 6 . 80 X = 8 .1 6 X = 3 .4 5 r = .63 r = 62 r = .5 1 r = .43 S2 = 8.25 S2 = 5. 63 S2 = 4 .8 1 S2 = 13. 77 N =32 N =32 N = 32 N =32 K =15 K =15 K = 15 K =15 11th Grade X = 4. 00 X = 4. 25 X = 6.33 X = -1 .7 8 r = .18 r = 1 0 r = - . 0 2 r = .5 6 S2 = 3.36 S2 = 1. 75 S2 = 1.56 S2 = 5 .6 0 N = 25 N = 25 N = 25 N = 25 K = 17 K = 17 K = 17 K = 17 — 2 Note: X = m eans, r = reliab ility (Hoyt), S = variance, N = sub jects, K = item s. (Romberg and Shepler, 1968, p. 6 .) i 25 J j T h ese r e su lts w ere contrary to the findings of Shuford and M a ssen g ill (1968a) j i ; w hich indicated in crea sed r elia b ility for a probability responding procedure- j ! The authors in d iscu ssin g th eir findings su ggested the te st in stru m en t w as not i | su itab le fo r probability responding. i ! . . . Students apparently attem pt to a rriv e at a n sw ers by m ath em atical ; tech n iq u es and are w illin g to bet that th eir answ er is c o r r e c t even j though the techniques used often lead to wrong a n sw ers. In addition, j ! if the an sw er found is not one of the fiv e a ltern a tiv es, the student r e so r ts to g u essin g . (Shuford and M a ssen g ill, 1968a, p. 7) A n a ly sis of item resp o n se patterns showed the tw elfth grade group ; responded w ith su b jective certain ty (0 ,1 ) 50 per cen t of the tim e and in a m an- i n er in d icative of no know ledge ( .2 , .2 , .2 , .2 , . 2) 14 per cen t of the tim e. The i eleven th grade group responded (0 ,1 ) 35 per cent of the tim e, and no know l- | ed ge 33 p er cen t of the tim e. The d ifferen ce in the resp o n se patterns fo r the t i | tw o g ra d es w as attributed to the added know ledge of the tw elfth grade su b jects, i E m p irica l testin g was conducted by Rippey (1968) to analyze . . the I j a sse r tio n of Shuford, A lbert,and M assen gill that in c r e a se s in r elia b ility as a 1 r e su lt of p rob ab ilistic adm inistration and sco rin g could be anticipated" (p. 2 1 1 ), A n a ly sis b ased on a 64 item te s t adm inistered to graduate psychology students | show ed in crea sed relia b ility fo r probability responding u sin g sp h erica l and i i | lo g a rith m ic sco rin g functions when com pared to conventional m u ltip le-ch o ice j testin g . The in creased r elia b ility w as achieved by a com parable in cr e a se in 1 I te stin g tim e equal to th e ad m in isterin g of a te st of one and o n e-h alf tim e s the i S o rig in a l length. It w as noted that further in c r e a se s in re lia b ility could be I achieved as a resu lt o f m o re appropriate item se le c tio n sin ce . . ite m s which 26 I ! had low o r negative correlation s with the total te st sco re w ere not the sam e item s when the te st was adm inistered in a conventional manner" {p. 2 1 2 ). / P ersonal probability responding with high school freshm an and fourth i ! I j grade underprivileged students to im prove te st reliability w ere not su ccessfu l j i _ | (Rippey, 1968). The results for the high school freshm an w ere erra tic and for j ; 1 : the fourth grade students responding " . . . the probabilistic fashion m et with j ! com plete d isa ster . . . " (p. 212). The resu lts were attributed to a lack of j m aturity and test naiveness among students at th ese a g e lev e ls. [ R esults (Rippey, 1968) based on second year m ed ical students w ere erratic with no one method of responding showing superiority. A nalysis of the j sco res showed the conventional testing procedure correlated higher with : sco res on an essa y exam ination covering the sam e topic than the probabilistic procedures. Rippey suggested The basic difference between the kind of inform ation contained in the two te st types is that the probabilistic te st contains inform ation about | how certain a student is about the item he is responding to. This would su ggest that conventional scored m u ltip le-ch oice te sts and essa y te sts are alike in the following way. N either contains inform ation about the d egree of certainty of the student. If the student is uncertain about a point on a m u ltip le-ch oice test, he probably w ill not w rite about it on an essa y test. Though additional careful work in te s t developm ent w ill be needed to establish whether one m ay expect con sisten t effects upon reliab ility, the m echanism of probabilistic scoring m ay have yet another type of u se. (p. 214) Rippey (1970) in considering the conflicting resu lts suggested they I could be attributed to a lack o f ". . . feedback prior to scoring with regard to the I payoffs resulting from the subject's choice of his distribution of b elief "(p. 1 ). | | Since im m ediate feedback is im practical in a testing situation, Rippey in v esti- I gated w hich form o f probability responding would r e su lt in the la r g e s t gains in I ' the ab sen ce of payoff know ledge. To exp lore this q u estion , five m ethods w ere j | com pared u sin g random ly selected high school students: | I 1. Probability assign ed to the co rrect resp o n se s I ! 2. L ogarithm ic 3. Spherical | 4. E uclidean 1 i I 5. Inferred ch oice In 10 of 14 adm inistrations o f 30 difficult ite m s taken from the ST E P j w riting te s t, L ev el 1, the fir s t m ethod, a sum m ing of the p rob ab ilities attached i j to the c o r r e c t ch o ice, resu lted in the highest te s t relia b ility . In a ll c a s e s , ! the in ferred s c o r e s , com parable to conventional responding, resu lted in the | lo w est relia b ility co efficien ts. Rippey concluded te sts should be scored u sin g the p ro b a b ilities a s - j signed to the c o r r e c t options and fo r com parison stu d ies, this sa m e sy stem i I should be used . . rath er than the obviously d eficien t standard of inferred ch oice" (p. 14). ■ Sum m ary ! C onsiderable experim entation has been expended over the past forty ! j y e a r s to develop a p ra ctica l method fo r elim ination of erro r v a ria n ce attrib u - j I table to g u essin g . C onventional m ethods have em ployed form ula scorin g. The - J E uclidean System S = 1 - .2 ( (r. - k.)^) ^ \ / ~ 2 w here r. equals the probability , 2.1 I bilify assign ed to the ith resp on se | a ssig n ed to the i th resp o n se a n^i kj^equals the cr ite r io n group m ean proba- 28 developing concepts of p artial knowledge and d ifferren tia l risk taking stra teg ies has questioned the validity of this convention. As a resu lt of this questioning new s y s t e m s of responding and scorin g have been proposed. Prom inent among the m ethods are those which require an exam inee to indicate his p erson al b e lie f of c o rr ec tn ess concerning the options to an item . This procedure would, in theory, elim in ate the gu essin g problem and thus provide a m o re accurate estim ate of an individual’s true sco re. Investigations in the area have been contradictory. Lord and Novick (1968) suggest further em p irica l studies b e fore the theory o f personal probability responding is used in testing. CHAPTER ID PROCEDURE M ethods em ployed to c o lle c t and analyze the data a re p resen ted in this chapter. Subjects Slakter (1967a, b) using eighth grade and co lleg e fresh m en a s su b jects, show ed contrary to expectation, a n eg a tiv e co rrela tio n fo r both groups be tw een risk taking and sc o r e s on a conventional m u ltip le-ch o ice te st. T his d is crepancy w as attributed to a lack of te stw ise n e s s am ong eighth grade students. No explanation w as offered fo r th e fresh m an su b jects. The test groups for th is in vestigation co n sisted of students at the eighth, tenth, and tw elfth grade le v e ls to provide additional inform ation a s to the p o ssib le e x iste n c e of a trend for r isk taking related to educational le v e l. E xam inees Four c la ssr o o m s of students at each of the th ree designated grade levels w ere se le c te d for the study. C riterio n for sele ctio n w a s c la ssr o o m groups w ith norm al academ ic ab ility, con ven ien ce, and w illin g n ess on the part of the tea ch ers to p articip ate in the study. The total num ber of su b jects at the beginning o f the exp erim en t was 29 30 366. There w ere 134 at the eighth grade, 111 at the tenth grade, and 121 at the twelfth grade. Fifteen of the eighth grade sam ple, 19 of the tenth grade sam ple, and 30 of the twelfth grade sam ple failed to com plete all phases of the testin g program due to their absence from school. Examination of school at tendance records indicated the absences w ere in no apparent way connected to the testing program . Eighth and tenth grade sam ples w ere each randomly r e duced to 91 subjects, the siz e of the twelfth grade group, for ease of com pari son. Subjects at the eighth grade level attended an interm ediate school characterized as m iddle cla ss. The intelligence lev els of the four c la sse s selected for inclusion in the study ranged from 90 to 110 as reported by the four teachers. Subjects at the tenth and twelfth grade attended a high school in a p re dominately m iddle c la ss area. The teachers indicated the intellectual ability of the eight cla ssro o m s ranged from 85 to 110. Instrumentation The design of this study required each student to be tested under two responding procedures to compare the observed sc o r e s, the correlation be tween risk taking and the observed sc o r es, and the reliability coefficien ts of the two m ethods. This required achievem ent tests with high com parable form reliab ility. The selected tests w ere the C ivics te st for the eighth grade sub je c ts, the World History te st for the tenth grade subjects, and the Problem s of 31 D em ocracy te st for the tw elfth grade subjects. T h ese te sts a re part of the C ooperative Social Studies T ests developed by the Educational T esting Service (1965). No published instrum ent designed to m ea su re risk taking on objective exam inations w as available at the tim e of this study. The u tilized procedure w as to randomly p lace ten nonsense ite m s among the leg itim a te item s of one form (A) of the achievem ent instrum ent (Appendix A). Subjects w ere cla ssified as high risk tak ers if they w ere in the top 27 p er cen t of the sco res on the risk taking m ea su re, low risk tak ers w ere the low 27 per cen t of the sc o r e s (K elley, 1939). T est Responding P rocedure The utilization of achievem ent te sts which contained four options r e quired a m odification of de F in etti's (1965) F ive Star Method. The exam inees w ere lim ited to blocks of 25 per cent. R esp on ses w ere recorded by subjects on answ er sh eets prepared by the in vestigator (Appendix B). D esign of the Study The in vestigator m et with a ll tea ch ers involved in the experim ent prior to the com m encem ent of the study. The purpose of th ese m eetin gs was to ex plain the rationale of the study and of p erson al probability responding. The developed te st instructions w ere used for the conventional re spon se procedure including a rem inder that scorin g w as based on a correction for g u essin g form ula (Appendix C). 32 The F iv e Star Method of probability responding is view ed by de Finetti (1965) as r ela tiv ely sim p le to u se and understand. Instructional m a teria l de tailin g th is proced u re w as prepared by the in v estig a to r a n d p resen ted to the students by the teacher. This m a teria l began with a b rief introduction to the concept of probability in fa m ilia r term s. T his was then related to p erson al probability responding. S everal exam p les of d ifferen t b e lie f sta te s in regard to a sam p le q uestion w ere d iscu sse d . The instru ction al m a teria l a lso illu str a ted the appropriate resp o n se pattern corresp on d in g to each b e lie f sta te. T his explanation p eriod took p la ce on the day p r io r to the adm inistration of the te st (Appendix D). A b rief review w a s provided by the teach ers ju st p rio r to the exam ination. The o rd er of p resen tation w as varied over the four c la s s e s at each grade le v e l to avoid potential confounding due to learning effec ts. A ll subjects com pleted the two testing p erio d s within th ree d ays. Subjects who m isse d any phase of the testin g program w ere elim in ated from the experim ent. Scoring of the Instrum ents A r isk taking s c o r e w as determ ined for each individual by sum m ing the num ber of r e sp o n ses to the ten n on sen se ite m s. S co res on the conventional resp o n se te st w ere determ ined by the fo l low ing co rrectio n for g u essin g form ula: w here S equals the s c o r e , R equals the num ber of c o r rect re sp o n se s, W equate 33 the number of wrong responses not including those omitted, and k equals the number of options to an item (Cronbach, 1970, p. 57). Scores on the probability response test w ere calculated by the de Finetti form ula (1965, p. 91) ^ 2 s = 2 p h - p j where S equals the item sco re, Ph equals the probability assigned to the cor rect option and Pj equals the probability assigned to an option. A transform a tion of the form S’ = (S + l) /2 was utilized to place sco res on a sca le ranging from 0 to 1 (de Finetii, 1965, p. 91). Table 5 illu strates the obtained scores when the p ossib le combinations of 25 per cent blocks are assigned to the cor rect option. Hypotheses This section presents research hypotheses and the resulting te st hy potheses. R esearch Hypothesis I. The correlation between the observed sco res on a conventional m u ltiple-choice test and those on a m easu re of risk taking behavior w ill in c rea se with level of education. T est Hypotheses 1. The correlation between the observed sco res on a conventional 34 TABLE 5 SCORING SYSTEM FOR PROBABILITY RESPONSE TEST TYpe D istribution of blocks of 25 per cent Corresponding P robabilities Score 4 xxxx 1 . 0 0 1 . 0 0 0 . 0 0 0 . 0 0 3-1 XXX .7 5 .9 4 X . 25 .4 4 0 . 0 0 . 19 2 - 2 X X . 50 .75 XX . 50 .7 5 0 . 0 0 0 . 0 0 2 - 1 - 1 XX .5 0 00 X .2 5 .5 6 X .2 5 .5 6 0 . 0 0 C O I — 1 l - l - l - l X .25 .6 2 X .25 .6 2 X .2 5 .62 X .2 5 .6 2 35 m u ltip le-ch o ice te s t and th o se on a m ea su re of r isk taking behavior w ill not d iffer sign ifican tly from z e ro at: a. the eighth grade le v e l. b. the tenth grad e lev el. c. the tw elfth grad e le v e l. 2. The co rrela tio n betw een the ob served sc o r e s of high r isk ta k ers on a conventional m u ltip le-ch o ice te s t and th eir sc o r e s on a m ea su re of r isk tak ing behavior w ill not d iffer sign ifican tly from z e r o at: a. the eighth grade le v e l. b. the tenth grade le v e l. c . the tw elfth g rad e le v e l. 3. The co rrela tio n between the ob served sc o r e s of low r isk ta k ers on a conventional m u ltip le-ch o ice t e s t and th e ir s c o r e s on a m ea su re of r isk taking w ill not d iffer sign ifican tly from z e r o at: a. the eighth grad e le v e l . b. the tenth grad e le v e l . c. the tw elfth g rad e l e v e l . R esea rch H ypothesis II. The co rrela tio n betw een the ob served sc o r e s on a p erso n a l proba b ility resp o n se te s t and th o se on a m ea su re of r isk taking behavior is not r e lated to le v e l of education. 36 T e s t H ypotheses 4. The co rrela tio n betw een the ob served s c o r e s on a probability r e sp o n se te s t and th ose on a m ea su re of r isk taking behavior w ill not differ s ig n ifican tly from z ero at: a. the eighth grade le v e l. b. the tenth grade le v e l- c. the tw elfth grade le v e l. 5. The co rrela tio n betw een the observed s c o r e s of high r isk tak ers on a probability resp o n se te st and th eir sc o r e s on a m ea su re of r isk taking b e h avior w ill not d iffer sign ifican tly from zero at: a. the eighth grade le v e l . b. the tenth grade le v e l • c. the tw elfth grade le v e l. 6 . The co rrela tio n betw een the ob served s c o r e s of low r isk tak ers on a probability resp o n se te st and th eir s c o r e s on a m ea su re of r isk taking be havior w ill not d iffer sign ifican tly from zero at: a. the eighth grade le v e l. b. the tenth grade l e v e l . c. the tw elfth grad e le v e l. R e se a r c h H ypothesis III. Subjects at th e th ree educational le v e ls studied who a re ch a ra cter iz e d as being r isk tak ers w ill a ch iev e higher s c o r e s on a conventional m u ltip le- 37 ch oice te s t than on a p erson al probability resp on se te st, concom itantly, non risk tak ers w ill achieve higher sc o r e s on a p erson al probability resp on se test. T est H ypotheses 7. There w ill not be a sign ifican t difference between the m eans on an achievem ent exam ination of high and low risk tak ers averaged a c r o ss grade le v e ls. 8 . There w ill not be a sign ifican t d ifferen ce between the m eans under a conventional m u ltip le-ch o ice te st and those under a probability resp on se te st averaged a cro ss grade le v e ls. 9. T here w ill not be a sign ifican t interaction betw een d eg ree of r isk taking and method of item responding averaged a cro ss grade le v e ls. 10. There w ill not be a sign ifican t interaction betw een grad e le v e l and method of item responding. 11. There w ill not be a sign ifican t interaction between amount of risk taking and grade le v e l. 12. T here w ill not b e a significant interaction between method of item responding, amount of risk taking, and grade lev el. B esea rch H ypothesis IV. The p erson al probability resp on se te st w ill be m ore reliab le than the conventional m u ltip le-ch o ice te st at the th ree educational le v e ls studied. 38 T est H ypotheses 13. T h ere w ill not be a sign ifican t d ifferen ce betw een the relia b ility co efficien ts of the conventional m u ltip le-ch o ice te st and the probability r e sponse te s t at: a. th e eighth grade le v e l. b. the tenth grade lev el. c. the tw elfth grade le v e l. 14. T h ere w ill not be a sign ifican t d ifferen ce betw een the relia b ility co efficien ts com puted fo r high r isk tak ers on the conventional m u ltip le-ch o ice te st and on the probability resp o n se te s t at: a. the eighth grade le v e l. b. the tenth grade le v e l. c. th e tw elfth grade le v e l- 15. T h ere w ill not be a sign ifican t d ifferen ce betw een the relia b ility co efficien ts com puted fo r low r isk tak ers on the conventional m u ltip le-ch o ice te st and on the prob ab ility resp o n se te st at: a. the eighth grade le v e l . b. th e tenth grade le v e l. c. th e tw elfth grade le v e l. S ta tistica l T reatm ent of the Data M eans, standard d eviation s, and in ternal co n sisten cy e stim a tes o f re lia b ility based on H oyt's (1941) p roced u re w ere com puted for each of the subjects three sc o r es. T h ese w ere determ ined for the follow ing m odes of cla ssifica tio n at each of the three grade lev els; (a) the total group, (b) the high risk tak ers, and (c) the low risk takers. T est sta tistic s for hypotheses one and four w ere based on the P earson Product Moment C orrelation for each pair of sc o r e s specified in the hypotheses The null and alternative hypotheses at each grade le v el w ere P 7 ^ 0 T est sta tistic s for hypotheses two, three, five and six w ere based on K endall's tau for each pair of sco re s specified in the hypotheses. Kendall's tau w as used in lieu of P earson 's r due to the large number of tie s within the subgroups on the m easure of risk taking behavior (Nie, Bent, and Hull, 1970, p. 153). The null and alternative hypotheses for the two subgroups at each grade lev el w ere H0 : T = 0 : T 1 0 T ests of hypotheses seven through tw elve w ere based on a three-w ay an alysis of variance design with repeated m ea su res on one factor (Winer, 1962, p. 337). This d esign is illu strated in Table 6 . The null and alternative hy potheses w ere 40 TABLE 6 ANALYSIS OF VARIANCE DESIGN C 1 C 2 B 1 X(ij)k X(ij)k A 1 B 2 X(ij)k X(ij)k B3 X(ij)k X(ij)k B 1 X(ij)k (D)k A 2 B 2 X(ij)k X(ij)k B3 (D)k X(ij)k Note: A l = A 2 “ B l = B 2 = B3 = High risk takers Low risk tak ers Eighth grade level Tenth grade level Twelfth grad e level (W iner, 1963, p. 337) Cl = Conventional responding procedure C 2 = Probability responding procedure 41 7. H. : a. = 0 fo r a ll i 0 l H„ : a. 7^ 0 fo r so m e i 1 l 8 . H : Y. = 0 for a ll k 0 k H„ : Y, ^ 0 fo r so m e k 1 k 9. EL : aY.. = 0 for a ll ik 0 ik H„ : a y ,. ^ 0 fo r so m e ik 1 ik 10. H : 8 y.! = 0 fo r a ll jk 0 jk Hf : 0 f ° r s o ^ e kj 1 1 IL : a 8 .. = 0 for all ij 0 ij H„ : o t 6 . ^ 0 fo r so m e ij 1 iJ 12. H Q : a ^ j k = ® f°r EL : a 0 Y.t ¥ ■ 0 for so m e ijk 1 13k The m ain e ffe ct of grade le v e l { g ) was not tested s in c e it w as co n sid ered ir relev a n t to the study. S co res on the dependent v a ria b le w ere tra n sfo rm ed to T s c o r e s based on the norm ing sa m p le's m ean and standard d eviation . A pro ced u re recom m ended by G reenhouse and G e isse r (1959) w as u sed to te s t the e ffe c t of the rep eated fa cto r sin ce the assu m ption of hom ogen eity of cova ria n ce w as q u estion ab le. T his p roced u re did not change th e c r itic a l v a lu e for any ra tio . A te st of the sim p le m ain e ffe c ts w as used to in c r e a s e know ledge con cern in g sig n ifica n t in teraction s. The te s t of sta tistic a l hyp oth eses th irteen through fifte e n w e re de term in ed by a p ro ced u re p roposed by K ristof (1964). T h is m ethod (Appendix 42 E) w as developed to ascertain w hether two te sts given to the sam e group of subjects are equally reliab le. The null and alternative hypotheses for each hypothesis w ere H : r - r = 0 0 tt^ tt2 H1 : r - r = * o 1 tt1 tt2 when rtt^ is the reliab ility of the conventional m u ltip le-ch oice te st and r ^ is the reliability of the probability resp on se test. CHAPTER IV RESULTS Organization of the P resen t Chapter D escriptive sta tistics relative to the adm inistration of the three achievem ent tests and the risk taking m easure a re fir st presented. Following this section , each sta tistica l hypothesis is considered. Prelim inary Data Tables 7, 8, and 9 present descriptive data relative to the adm inistra- tion of the two form s of the achievem ent test. A ll reliability coefficients w ere based on Hoyt's (1941) analysis of variance method. Inspection of Table 7 indicates eighth grade subjects had low er m eans and reliability coefficients under formula scoring when com pared to probability responding. The low est m eans and reliability coefficien ts w ere found for sub jects classified as high risk takers. A nalysis of the m ean and standard devi ation of the risk taking m easure indicate this trait was skewed in a negative direction for exam inees at this grade level. R esults for tenth grade students (Table 8) w ere com parable in overall pattern to the eighth grade sam ple. The highest m eans and reliability coeffi cients along with the sm a llest standard deviations occurred under probability responding. Low risk takers had higher m eans and sm aller standard devia- 44 TABLE 7 EIGHTH GRADE TEST CHARACTERISTICS Mean S .D . R el. Form ula Score Total Group 1 6 .2 1 11. 86 . 874 High R isk Group 12. 53 12. 56 . 832. Low R isk Group 18. 02 9 .6 7 .881 Probability R esponse Score Total Group 37 .3 7 9. 14 .911 High R isk Group 34. 25 8 . 89 . 896 Low R isk Group 3 6 .9 5 9. 12 . 915 N onsense Score Total Group 7. 08 2 .9 2 . 861 High R isk Group 10. 00 0 . 00 ------- Low R isk Group 3 .2 0 1. 35 . 074 45 TABLE 8 TENTH GRADE TEST CHARACTERISTICS M ean S. D. R el. Form ula Score Total Group 1 2 .1 0 7. 10 .7 5 9 High R isk Group 1 0 .4 8 5 .9 2 .4 1 8 Low R isk Group 1 2 .4 1 7 .4 8 . 839 P robability R esp on se Score Total Group 38. 15 5. 52 .7 7 4 High R isk Group 3 7 .3 5 4 .5 3 .6 7 5 Low R isk Group 38. 83 7 .2 3 . 880 N on sen se S core Total Group 5. 61 3 .4 2 . 885 High R isk Group 9. 68 0.5 2 - . 233 Low R isk Group 1. 36 0 .9 9 - . 078 46 tio n s when com pared to high r isk ta k e rs. The m ean and standard deviation of th e r is k taking m e a su r e rev ea l th is tr a it w as n orm ally d istrib u ted at th is grad e le v e l. Insp ection of T able 9 r e v e a ls t e s t c h a r a c te r is tic s c o n siste n t w ith the eigh th and tenth g rad e sa m p les. T w elfth grade stu d en ts obtained higher m ean s c o r e s and sm a lle r standard d eviation s when req u ired t o respond in t e r m s of p e r so n a l p ro b a b ilities. This responding p roced u re w a s a lso m o r e relia b le than form u la sc o rin g . Subjects c la s s ifie d as low r isk ta k ers r ec eiv e d higher m ea n s and had m o re re lia b le t e s t s c o r e s than th o se ch a ra c te r iz ed a s being in th e high r isk group. D e sc r ip tiv e s ta tis tic s in r e fe r e n c e to the m e a su r e of r is k taking in d icate th is tra it w as n ega tiv ely skew ed in th e sa m p le o f tw elfth grad e su b jects. Appendix F p r ese n ts d e sc r ip tiv e data con cern in g the num ber and p er cen t com p letin g v a rio u s p ercen ta g es o f ite m s on each te st. Appendix G, H, and I p ro v id e data re la tiv e to th e ex a m in ees d istrib u tio n of prob ab ility points in te r m s of the num ber and p er cen t of item s c o r e s . T h is in form ation is p ro vided fo r each g ra d e in te r m s o f the to ta l sam p le and the two subgroups. S ta tistic a l H ypotheses S ta tistic a l H ypothesis 1 The s ta tistic a l h yp oth eses for th e co rrela tio n betw een s c o r e s on a m e a su r e of r isk taking and s c o r e s on a conventional re sp o n se t e s t w ere Hq : P = 0 at ea ch grad e le v e l. 47 TABLE 9 TWELFTH GRADE TEST CHARACTERISTICS Mean S. D. R el. Form ula Score Total Group 16 .3 9 10.65 . 837 High R isk Group 13. 20 10.29 .817 Low R isk Group 17 .8 3 11. 14 . 816 Probability R esponse Score Total Group 35. 80 9. 92 . 853 High R isk Group 32. 13 7.0 2 . 856 Low R isk Group 36. 07 7.4 6 . 864 N onsense Item s Total Group 7 .4 8 3. 49 . 797 High R isk Group 9 .9 6 0. 20 . 000 Low R isk Group 3 .7 2 1.45 -.5 7 7 a. Eighth grade r = - . 16 n t > 0 b. Tenth grade r = - . 10 P = .3 1 c. T w elfth grade r = - . 24 P = . 02 The hypotheses w ere not r e je c te d . S ta tistica l H ypothesis 2 The sta tistic a l h ypotheses for the co rrela tio n betw een s c o r e s on a m ea su re of risk taking and s c o r e s on a conventional resp o n se te st w ere : t = 0 for high risk tak ers a t each grade le v e l. a. Eighth grade T ----- b. Tenth grade r = . 00 P = . 99 c. T w elfth grade t = . 08 P = . 56 A co rrela tio n w as not com puted for the eighth grade sam p le due to the lack of variab ility on the m ea su re of risk taking. The hypotheses w ere not rejected . S ta tistica l H ypothesis 3 The sta tistic a l hypotheses for the co rrela tio n betw een s c o r e s on a m ea su re of r isk taking and sc o r e s on a conventional resp o n se te st w ere H0 : r = 0 for low risk ta k ers at each grade lev el. a. E ighth grade r = - . 14 P = . 29 b. Tenth grade T = - . 20 P = . 15 c. T w elfth grade t = 12 P = . 36 The hypotheses w ere not rejected . S tatistical Hypothesis 4 The sta tistica l hypotheses for the correlation between sc o r e s on a m easu re of risk taking and s c o r e s on a person al probability response te3t w ere Hq : P = 0 at each grade lev el. a. Eighth grade r = - . 01 P = .43 b. Tenth grade r = -.0 7 P = .47 c. Twelfth grade r = .0 4 P = .6 7 The hypotheses w ere not rejected. Statistical Hypothesis 5 The sta tistica l hypotheses for the correlation between sco re s on a m easu re of risk taking and sc o r e s on a personal probability response test w ere : r = 0 for high r isk takers at each grade level. a. Eighth grade t ----- b. Tenth grade r = . 13 P = . 36 c. Twelfth grade t = 11 P = . 40 A correlation was not computed for the eighth grade sam ple due to the lack of variability on the m easure o f risk taking. The hypotheses w ere not rejected. Statistical Hypothesis 6 The sta tistica l hypotheses for the correlation between sco res on a m easu re of risk taking and sco res on a personal probability resp on se test w ere Hq : t = o for low risk tak ers at each grade lev el. a. Eighth grade ■ > = . 10 P = . 47 50 b. Tenth grad e T = 13 P = . 33 c. Tw elfth grade t = - . 04 P = . 75 The hypotheses w ere not rejected. S ta tistica l H ypotheses 7 through 12 T he an a ly sis of varian ce provided te sts of the sta tistica l hypotheses that the m ain and in tera ctiv e effects of degree of risk taking (A), grad e level (B), and responding procedure (C) w ere equal to zero. A sum m ary of the re sults is p resen ted in Table 10. ABC Interaction E ffect H : af5Y.„ — 0 for a ll ijk F = . 27 P > . 05 0 ijk The hypothesis w as not rejected. BC Interaction E ffect H0 : = ^ ^°r ^ F = 8 .1 8 P < . 05 The hypothesis w as rejected . A plot of the C m eans on B is shown in F igure 1. A te st of the sim p le m ain effects of C at the th ree le v e ls of B indicated sig n ifica n ce (P-^. 05) for a ll com p arison s with CQ g rea ter than C in each c a se A te st o f the sim p le m ain effects of B at C1 w as not significant. The effect of B at C„ w as sign ifican t (P < . 05). A com parison of the B m ean s at C in - 2 " dicated that BC22 w as significantly g rea ter than BC32 (Q = 3 - 9 5 » Q'q q = 2 * 80)- No other com parison at C2 w as significant. 51 TABLE 10 ANALYSIS OF VARIANCE SUMMARY TABLE Source of Variation SS df MS F Between Subjects 13391.82 149 A 777.07 1 777.07 9 .1 5 * B 214.25 2 107.12 1. 26 B at Cx 98.02 2 49. 01 . 86 B at C 582. 14 2 291.07 5 .1 4 * AB 173.25 2 86.62 1.02 Subjects Within Groups (Error [between]) Within Subjects 30370. 88 150 C 25780.66 1 25780.66 909.69 * C at B .^ 7974.49 1 7974.49 281.42 * C at B 12155.06 1 12155.06 421. 84 * C at B 6585.32 1 6585.32 232.36 * AC 29. 41 1 29.41 1.03 BC 464.14 2 232.07 8.1 8 * ABC 15.46 2 7.73 .27 C x Subjects Within Groups (Error [within]) 4081.21 144 28.34 Note: * P < . 05 A is degree of risk taking B is grade level C is responding procedure 52 Three Grade L ev els 55 50 45 40 35 30 25 Cg (Probability Responding (Conventional Responding 1 (Eighth Grade) (Tenth Grade) 3 (Twelfth Grade) Figure 1. — A P lot of the M eans of the Two Responding P ro ce dures at the Three Grade L evels. TABLE 11 THE MEANS OF THE TWO RESPONDING PROCEDURES AT THE THREE GRADE LEVELS Grade c i C2 B1 29. 70 47. 56 B2 27 .7 2 49. 77 B_ 28 . 72 44. 95 3 53 AC Interaction Effect : ay., = 0 for a ll ik F = 1. 03 P > . 05 0 'ik The hypothesis was not rejected. AB Interaction E ffect H„ : <*0 = 0 for a ll ij F = 1. 02 P > . 05 0 l] The hypothesis w as not rejected. A M ain E ffect IT : a . = 0 for a ll i F = 9 .1 2 P < . 05 0 l The hypothesis w as rejected. The m ean of the high risk tak ers was 36. 54. The com parable figure for the low r isk takers w as 39, 75. C Main E ffect H : y = 0 for all k F = 90 9 .6 9 P < . 05 v / K The hypothesis w as rejected. The m ean on the conventional resp on se te s t was 28. 87. The com parable figu re for the personal probability resp on se te st w as 4 7 .4 2 . S ta tistica l H ypothesis 13 The sta tistic a l hypothesis relating to the relia b ility co efficien ts of the two responding procedures w as H_ : r - r = 0 at each grade lev el. 0 tt^ tt^ a. Eighth grade Conventional responding rtt^ - ■ 86 54 P robab ility responding r = .9 1 2 2 X = 3 .4 5 P > . 05 b. Tenth grade C onventional responding r = . 69 ttj P rob ab ility responding r = . 73 2 X 2 = . 32 P > . 05 c. T w elfth grade It w as not p o ss ib le to sta tistic a lly com p are th e r e lia b ilitie s of the twelfth; g ra d e sa m p le due to a violation o f the a ssu m p tio n of equivalent v a r ia n c e s on th e tw o h alves of the conventional resp o n se te st. Although the m ea n s w ere 9. 56 and 8. 3 5 , the v a r ia n ce s w ere 7 0 .1 2 and 29. 86. The d iffer e n ce betw een th e m ea n s w as com p arab le w ith that found at th e other grad e le v e ls . No e v i d en ce w as found to in d ica te a sig n ifica n t d iffe r e n c e b etw een the r e lia b ilitie s of th e tw o p ro ced u res at the tw elfth grad e le v e l. P a r ts A and B o f sta tistic a l h yp oth esis 13 w ere not rejected . No sta tem en t w as m ad e w ith r e sp e c t to P a r t C. S ta tistic a l H ypothesis 14 The s ta tis tic a l h yp oth esis rela tin g to the r e lia b ility co e fficien ts o f th e tw o resp on din g p ro ced u res w as H : r - r = 0 fo r high r isk ta k ers at U tt_ X a each grade le v e l. a. Eighth grade C onventional responding r .. = . 84 1 55 Probability responding r ^ = . 85 X2 = . 05 P > . 05 b. Tenth grade Conventional responding r ^ = . 22 Probability responding r ^ = . 73 X2 = 3. 59 P > . 05 c. Twelfth grade Conventional responding r ^ = . 74 Probability responding r ^ “ • 90 X 2 = 2. 89 P > . 05 The sta tistica l hypotheses w ere not rejected. Statistical H ypothesis 15 The sta tistica l hypothesis relating to the reliab ility coefficien ts of the two responding proced u res was : r ^ - = 0 for low risk tak ers at each grade level. a. Eighth grade Conventional responding r^. = . 84 Probability responding r ^ = . 92 X2 = 1 .8 4 P > .0 5 b. Tenth grade Conventional responding r^. = . 88 Probability responding r ^ 2 = . 89 ___________________X 2 - . 03___________P > . 05__________________________________ 56 c. Tw elfth grade Conventional responding r ^ - . 74 Probability responding r ^ = . 90 X 2 = 2 .8 9 P > .0 5 The sta tistica l hypotheses w ere not rejected. S ta tistica l H ypothesis 15 The sta tistica l hypothesis relating to the relia b ility co efficien ts of the two responding p rocedures was HQ : rtt^ - r ^ = 0 for low risk tak ers at each grade level. a. Eighth grade Conventional responding r ^ =. 84 Probability responding r^. = . 92 X 2 = 1 ,8 4 P > .0 5 b. Tenth grade Conventional responding r ^ = . 88 Probability responding r ^ = .8 9 X 2 = .0 3 P > .0 5 c. Twelfth grade It w as not p o ssib le to sta tistica lly com pare th e r elia b ilitie s o f the tw elfth grade sam ple due to a violation of the assum ption of equivalent va ri an ces on the two halves of the probability resp on se te st. Although the m eans w ere 17 .3 2 and 11. 79, the d ifferen ce betw een the m ean s w as com parable with 57 th o se found at th e oth er g rad e le v e ls . N o e v id en ce w as found to in d icate a s ig n ifican t d iffer en ce betw een th e r e lia b ilitie s of th e tw o p roced u res at th e tw elfth g rad e le v e l. P a r ts A and B o f s ta tis tic a l h yp oth esis 15 w ere not rejected . No statem en t w as m ad e w ith r e s p e c t to P a rt C. Sum m ary 1. S c o r es on a m e a su r e of r isk taking b eh avior w ere not c o r re la te d w ith s c o r e s on eith e r of the tw o resp on din g p ro ced u res. T h is finding held for a ll g ra d es and c la ss ific a tio n s of su b jects. 2. T he m ean s c o r e fo r su b jects op eratin g under a low r isk strategy w as sig n ifica n tly higher than the com p arab le fig u r e for su b jects c la s s ifie d a s high r isk ta k ers ir r e s p e c tiv e o f responding p roced u re. 3. The m ean sc o r e for e x a m in e es req u ired to respond in te r m s of p e r sonal p ro b a b ilities w as sig n ifica n tly h igh er w hen com pared to conven tion al sco r in g p ro ced u res. 4. T h ere w as no sig n ifica n t d iffe r e n c e betw een the relia b ility co effi c ien ts of the two responding p r o ce d u r es. D isc u ssio n o f R e su lts R e se a r c h r e la tiv e to the con cep t of r is k taking on o b jectiv e exam ina tion s had in d icated a p o sitiv e exp ected v a lu e in te r m s of ob served s c o r e s for su b jects operating under a high r isk stra teg y . T h e resu lts of the p r e s e n t study did not support th is exp ectation . No s ta tis tic a l relationship w as d eveloped be tw een s c o r e s on the in stru m en t d esign ed to a s s e s s risk taking and s c o r e s on 58 the conventional m u ltip le-c h o ic e test. Slakter (1967b) encountered sim ila r findings am ong eighth grade stu d en ts and attributed the r e su lts to a lack of te s t w is e n e ss am ong exam in ees at th is grade le v e l. T his study did not substantiate S lak ter's theory sin ce com p arab le r e su lts w ere obtained fo r sub jects in the tenth and tw elfth grad e who a r e high in te s t soph istication . The d iscrep an cy betw een r esea rc h conducted by S h erriffs and B oom er (1954) and the p resen t in v estig a tio n m ay be explained in ter m s of two opera tion al defin ition s of r isk taking. The tra it m easu red by the A sc a le of the MMPI m ay not be the sa m e as that m easu red by the in clu sio n of ten n on sen se ite m s into an ach ievem en t test. A p o ssib le in terpretation of the con trad iction betw een th ese p a st find in g s and the p resen t r e se a r c h is that the p o sitiv e expected valu e of r isk taking on ob jective exam inations is not g rea t enough to o ffse t the penalty for gu essin g among students of a v era g e ability. Ind irect support for th is p o ssib ility w as provided by Votaw (1936) when he offered evid en ce to in d icate that the com bination of low ability and ascendant p erson ality r e su lts in low sc o r e s on an in stru m en t w here a p en alty for g u essin g is em ployed. It m ay be that fo r stu dents of a v era g e ab ility th e num ber of ite m s answ ered c o r re ctly due to risk taking is su rp a ssed by the num ber answ ered in co rrectly . T his could then re su lt in eith er no co rrela tio n or a sm a ll in sig n ifica n t co rrela tio n betw een risk taking and ach ievem en t te s t s c o r e s . C larification of th is hypothesis would r e q u ire additional r e se a r c h using in tellectu a l ab ility a s an independent variab le. 59 An additional hypothesis concerning the concept of risk taking w as that sc o r e s on a m ea su re of th is tra it would be u ncorrelated with sc o r e s on an achievem ent te s t requiring resp o n ses in te rm s of p erson al p rob ab ilities. A n a ly sis of the data in the p resen t study confirm s th is hypothesis. A sta tistic a l com p arison of the observed sc o r e s under the two m ethods indicated significantly higher s c o r e s for subjects required to respond in term s of p erson al p rob ab ilities. T his finding w as unexpected sin ce it has been hy p oth esized for subjects low in risk taking and not th ose defined to be high in this tra it. The hypothesis relatin g to low risk ta k ers w as based on evidence which indicated con servative subjects p o sse ssin g p artial knowledge fa il to r e spond to an item when faced w ith a penalty for an in co rrect resp on se. T here fore, subjects low in risk taking would respond and receiv e cred it for partial knowledge under this sy stem . The hypothesis relatin g to high r isk takers w as based on the p o sitiv e ex p ectiv e value of responding on the b a sis of p artial knowledge when faced w ith a penalty for an in co rrect resp on se. T his p o sitiv e expected value has been em p irica lly dem onstrated by Slakter (1967a). R e se a r ch has shown that high r isk tak ers p e r s is t in responding in term s of a m axim um risk strategy when faced w ith an instrum ent designed to a s s e s s vary ing am ounts of p artial know ledge. A resp on se se t of th is nature would v io la te a fundam ental assum ption of probability responding. This assum ption states that a subject can only m a x im ize his sc o r e if he accurately reports his d e - g r e e -o f-b e lie f state concerning the co r r e c tn e ss o f the options to an item . The findings of th is study did not support S lak ter's p osition . Subjects c la ssifie d a s | high risk tak ers responded in term s of personal p rob ab ilities rather than con- i sisten tly m axim izin g th eir probabilities on one option. The p ra ctica l value of the m easu rem en t of partial knowledge by proba- i b ility responding m u st a lso be considered. If test adm inistrators con sisten tly [ take the tim e to analyze su b jects' individual resp o n ses to item s, a rea s of co n - ! I , fusion and m isin terp retation within the te s t dom ain can be identified. Such r e - ; su its could be u sefu l in planning la ter instructional m aterial. The question r e - j . ; m ains w hether the con sid erab le in crea se in sco rin g tim e and in analyzing resp o n - i s e s is ju stified in term s of the inform ation received . If the a sse ssm e n t of partial knowledge can not be justified on p ractical grounds, another m ethod of responding m ay be appropriate. A v isu a l com parison of the m ea n s, standard d eviations, and te s t reliab ility co efficien ts d e - j 1 [ I rived from probability responding when com pared to the responding and s c o r - j ing m ethod em ployed by the norm ing sam ple revealed sligh t d ifferen ces. It j | ; appears that instructing exam inees to respond to a ll item s without penalty for j an in co rrect resp o n se resu lts in som ewhat higher m ean sc o r e s which are as ! relia b le as probability responding. This method would a lso control fo r any I confounding effects attributable to the differen tial propensity to take a risk on i ! an ob jective exam ination. This study had hypothesized an instrum ent which required item r e - : sp on ses in term s of person al p robabilities would be significantly m o re reliab le ; than a conventional m u ltip le-ch o ice te st. The rationale for th is prediction w as I the em p irica lly dem onstrated gain in te st sco re relia b ility for probability r e - I sponding. The determ ination of im proved relia b ility in the above rese a rc h w as i i by m eans o f the Spearm an-B row n Prophecy Form ula. No d irect sta tistic a l ! com p arison s w ere undertaken. A procedure developed by K ristof (1964) was i [ u tilized in the p resen t resea rch to sta tistica lly com pare the two co efficien ts. 61 ! Although none of the com parisons w ere sign ifican t, probability responding did i resu lt in higher observed relia b ility coefficien ts. A p o ssib le explanation for the lack of significant d ifferen ces between the two coefficien ts m ay be due to a violation of an underlying assum ption of I K ristof’s procedure. This assum ption stated that the sam ple s iz e be not too sm a ll. The number of subjects used in this resea rch m ay not have been su ffi- i cien t to allow for the proper determ ination of m axim um likelihood estim ations : of the te st varian ces, co -v a r ia n ce s, and relia b ility co efficien ts. Increased | sam p le s iz e m ay y et show probability responding to be sta tistica lly m ore re lia b le than conventional procedures. relia b ility as a resu lt of probability responding m ay be that item selectio n and | the p resen t resea rch w ere constructed and normed on sam p les of subjects who w ere instructed to attem pt all item s without a penalty for in co rrect resp on ses. P o ssib ly te sts m ust be constructed and norm ed on sam p les responding in ter | of personal probabilities before this method can be shown to con sisten tly r e - ■ suit in significant in c r ea se s in reliab ility. Evidence to support this hypothesis v/as provided by Eippey (1968) when he noticed a change in the pattern of item co rrela tio n s under probability responding com pared to those under conven- i ; tional procedures. Som e item s which had high correlation s with total test : sc o r e s under one method had low correlation s under the other. T est reliability : would be affected if changes of th is type occurred with resp ect to many item s. ! If the resu lts of subsequent resea rch support R ippey's ob servation s, em p irical ! investigations should be conducted with r e sp e c t to the construction of a test i j based on an orm in g sam ple required to respond in term s of personal probabilities. Another p o ssib ility for the failure to show sign ifican t in crea ses in test] te s t construction is sp ecific to the responding procedure. Instrum ents used in CHAPTER V SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS Summary The Problem It w as the purpose of this investigation to com pare two form s of m ul tip le-ch o ice responding and scoring procedures at three educational lev els with respect to the effects that various risk taking strategies have on the observed sco res and the test reliability coefficien ts. The two procedures compared w ere personal probability responding using fixed blocks of 25 per cent and con ventional responding with the te st scored by a correction for guessing formula. Subjects w ere selected at the eighth, tenth, and twelfth grade level. Procedure A sam ple of 273 students distributed equally over the three grade lev els was selected for inclusion in this study. Subjects received form s A and B of the C ivics test, the World History test, and the Problem s of Dem ocracy test at the eighth, tenth, and twelfth grades respectively. Ten nonsense item s w ere included in form A of each te st in order to provide an estim ate of risk taking behavior. Exam inees who scored in the top 27 per cent on the risk taking m easure w ere defined to be high in risk taking on objective examinations Those who scored in the bottom 27 per cent w ere defined to be low in risk 62 63 taking on objective exam inations. Subjects w ere instructed to respond in a conventional m anner to form A with the understanding that scoring would be accom plished by the follow ing correction fo r g u essin g form ula: w k-T w here S equals the sc o r e , R equals the num ber right, W equals the number w rong, and k equals the number of options. R esp on ses to form B w ere in term s of personal p robabilities concerning the co r r e c tn e ss of the options. E xam inees w ere lim ited to exp ressin g their p rob ab ilities in blocks of 25 per cent. Individual sc o r e s for the subjects w ere derived from _2 S = 2 P, ■h- £ p j = i J w here S is the item sc o r e , is the probability assign ed to the c o r r e c t op tion and Pj is the probability assign ed to an option. The ord er of presentation of the te s ts w as random ized over the four c la ssr o o m s at each grade lev el. T est m ea n s, reliab ility co efficien ts, and in ter co r relations betw een the th ree m ea su res for each subject w ere subjected to sta tistic a l com parisons at each grade le v el for the follow ing c la ssific a tio n s of su b jects; (a) the total group, (b) the high risk group, and (c) the low risk group. R esu lts C orrelations involving the risk taking s c o r e and the sc o r e on the con ven tion al m u ltip le-ch o ice te st w ere not sign ifican tly different from zero. 64 Statistical hypotheses 1, 2, and 3 w ere not rejected. A ll correlations relative to the risk taking sco re and the sco re on the personal probability response te st w ere also not significantly different from zero. Statistical hypotheses 4 , 5, and 6 w ere not rejected. The mean of the low risk tak ers, averaged over the three grade lev els, w as found to be significantly g rea ter than the corresponding mean of the high risk takers. Statistical hypothesis 7 was rejected. The overall mean for probability responding was determ ined to be significantly greater than the par a llel figure for the conventional responding procedure. Statistical hypothesis 8 w as rejected. No interaction w as found to ex ist relative to degree of risk taking and method of item responding. Statistical hypothesis 9 was not rejected. Statistical hypothesis 10 was rejected. There was a significant interaction b e tween grade level of the subject and method of item responding. A com parison of the sim ple main effects of grade level at each of the responding procedures revealed a significant difference with respect to probability responding. This effect, as indicated by a com parison of the joint m eans, was attributed to the significant difference between the tenth and tw elfth grade m eans under proba bility responding. A ll com parisons relating to the sim ple m ain effects of re sponding procedure at each grade level w ere significant indicating higher m ean sco res under probability responding. No interaction was found to exist be tween risk taking and grade lev el nor between risk taking, grade lev el, and method of item responding. Statistical hypotheses 11 and 12 w ere not rejected. Statistical com parisons to determ ine which of the two responding p ro- 65 ced u res w as the m o re relia b le w ere calculated in all but two in stan ces. A violation of one of the te st's assum ptions occurred in the c a se of the tw elfth grade total sam ple and the tw elfth grade low risk group. S tatistical hypotheses 13 A and B, 14, and 15 A and B w ere not rejected . No d ecisio n w as m ade in the c a se of hypotheses 13 C and 15 C. C onclusions The resu lts of this study led to th ese con clu sion s: 1. Conventional responding w ith a correction for gu essin g did not offer a d ifferen tial advantage or disadvantage to subjects c la ssifie d as eith er high or low risk tak ers on objective exam inations. 2. P erso n a l probability responding did not offer any advantage over conventional responding in controlling for a differen tial propensity to take a risk on an objective exam ination. 3. Subjects c la ssified a s low risk tak ers had a sign ifican t advantage over th eir le s s con servative counterparts in term s of th eir observed sco re s on both responding procedures. 4. Conventional responding with a correction for g u essin g and personal probability responding had com parable relia b ilities. 5. The tendency to take a risk on an ob jective exam ination when a co r rection for gu essin g is applied w as not a function of educational lev el. The con clu sion s reached in th is study provide support for Cronbach's (1970) b e lie f that the effects of gu essin g on te st sc o r e s is not serio u s enough to w arrant probability responding.________________________________________________ 66 R ecom m endations 1. It is recom m ended that th is study b e replicated using in telligen ce as an additional independent variable to determ in e whether risk taking is inde pendent of in tellectual ability. 2. It is recom m ended that further r e se a r c h be conducted on the norm alacy of the distribution of r isk taking on obj ective exam inations due to the negatively skewed distributions encountered in this study. B IB L IO G R A P H Y 67 BIBLIOGRAPHY ; C oom bs, C.H. On the u se of objective exam inations. Educational and P sy - ch ological M easurem ent, 1953, 13, 308-310. i _ _ — _ I C oom bs, C.H. , M ilholland, J. E . , and W om er, F. B. The a sse ssm e n t of j partial knowledge. Educational and P sy ch o lo g ica l M easurem ent, j 1956, 16, 13-37. | Cronbaeh, L. J. E sse n tia ls of P sych ological T estin g . New York: Harper and ; Row, 1970. i Cureton, E. The correction for gu essin g. The Journal of Experim ental Education, 1966, 34, 44-47. ■ D avis, F. B. and F ifer, G. The effect on test relia b ility and validity of sco rin g aptitude and achievem ent te sts with w eights for every ch oice. Educational and P sy ch o lo g ica l M easurem ent, 1959, 19, 159-170. : De F in etti, B. Methods for d iscrim in atin g le v e ls of p artial knowledge co n - ! cerning a te st item . The B ritish Journal of M athem atical and S tatis- i tica l P sych ology, 1965, 18, 87-123. I ; Dixon, W. J. (Ed.) B iom ed ical Com puter P rogram s. B erkeley: U niversity | of C alifornia P r e s s , 1970. J D r e s se l, P. L. and Schm id, J. Some m odifications of the m u ltip le-ch o ice ! item . Educational and P sychological M easurem ent, 1953, 13, 5 7 4 - 595. Educational T esting S erv ice. Handbook-Cooperative S ocial Studies T e sts. P rinceton, New J ersey : Educational T esting S erv ice, 1965. j ! Edw ards, A. L. E xperim ental D esign in P sych ological R esea rch . New York: | H olt, R inehart and W inston, 1960. G la ss, G. V. and W iley, D. E. Form ula sco rin g and te s t relia b ility . Journal of Educational M easurem ent, 1964, 1, 43-49. Granich, L. A technique for experim enting on gu essin g in objective te s ts . Journal of Educational P sych ology, 1931, 22, 145-156. 68 69 G reenhouse, S. W. and G e isse r , S. On m ethods in the a n a ly sis of p r o file data. P sv ch o m etrik a , 1959, 2 4 , 95 -1 1 2 . H am m erton, M. The g u essin g co rrectio n in vocabulary te sts. B ritish Journal of Educational P sy ch o lo g y , 1965, 3 4 , 249-251. H ays, W. L. S ta tistic s. N ew York: Holt, R inehart and W inston, 1963. H errell, J. M. P sy ch o m etric p ro p erties of te s ts using su b sea u en t-ch o ice p roced u res. (Unpublished D octoral D isserta tio n , U n iversity o f M ary land, 1967.) Hoyt, C. J. T e st relia b ility estim ated by a n a ly sis of varian ce. P sy ch o m e- trik a, 1941, < 3 , 153-160. International B u sin ess M achines C orporation. S v stem s/3 6 0 S cien tific Sub routine P ackage (360A-CM -03X1 V ersion IH P ro g ra m m er's M anual. New York: International B u sin ess M achines C orporation, 1968. K elley, T. L. The se le c tio n of upper and low er groups for the validation of te s t ite m s. Journal of E ducational P sy ch o lo g y . 1939, 30, 17 -2 4 . Kirk, R. E. E xp erim en tal D esign; P ro ced u res for the B ehavioral S c ie n c e s. B elm ont, C alifornia: B ro o k s-C o le, 1968i K ristof, W. T estin g d ifferen ces betw een relia b ility co efficien ts. The B ritish Journal o f S ta tistica l P sych ology. 1964, 17, P art 2 , 105-111. L ittle, E. G. O v ercorrection for g u essin g in m u ltip le-ch o ice te st scorin g. The Journal of Educational R esea rch , 1962, 55, 2 45-252. Lord, F. M. F orm ula sco rin g and valid ity. Educational and P sych ological M easu rem en t. 1963, 23, 66 3 -6 7 2 . Lord, F. M. and N ovick, M. R. S ta tistica l T h eories of M ental T est S c o r e s. R eading, M ass: A d d ison -W esley, 1968. L yerly, S. B. A note on co rrectio n for chance su c c e ss in objective te s ts . P sv ch o m etrik a . 1951, 16, 2 1 -3 0 . M a ssen g ill, H. E. and Shuford, E. H. The E ffect of "D egree of C onfidence. " Lexington, M ass: Shu ford -M assen gill Corp. , February, 1968. (D istributed by C learin gh ou se, A D 677209.) M assen gill, H. E. and Shuford, E. H. Confidence T estin g at the A cadem ic 70 Instructor C ourse of the A ir U niversity; August and Septem ber. 1968. Lexington, M ass. : Shuford-M assengill Corp., January 1969. (D istributed by C learinghouse, AD685178.) M ichael, J. J. An Experim ental A nalysis of the Relationship Between the Re liab ility of a M ultiple-C hoice Examination and V arious T est-S corin g P roced u res. (Unpublished Doctoral D issertation , U niversity of Southern C alifornia, 1967.) N ie, N. H. , Bent, D. N. , and Hull, C. H. Statistical Package for the Social S cien ces. New York: M cG raw-H ill, 1970. Nunnally, J. C. Psychom etric Theory. New York: M cG raw-H ill, 1967. Q uereshi, M. Y. Mental te st perform ance as a function of payoff conditions, item difficulty, and degree of speeding. Journal of Applied P sychology. 1960, 44, 65-77. Rippey, R. P robab ilistic testing. Journal of Educational M easurem ent. 1968, 5, 211-215. Rippey, R. E ffects of various scoring functions on the reliab ility of valid confidence te sts. Paper presented at the Annual Convention of the National Council on M easurem ent in Education, M inneapolis, M arch 3 -6 , 1970. Rom berg, T. A. and Shepler, J. L. An experim ent involving a probability m easurem ent procedure. Paper presented at the Annual M eeting of the A m erican Educational R esearch A ssociation , Chicago, February 7 -1 0 , 1968. Sherriffs, A. C. and B oom er, D. S. Who is penalized by the penalty for gu essin g. Journal of Educational Psychology, 1954, 45, 81-90. Shuford, E. H. Cybernetics Testing. Bedford. M ass. : D ecision Sciences Laboratory. L. G. Hanscom F ield. March. 1967. (Distributed by C learinghouse, A D 650668.) (a) Shuford, E. H. How to Shorten a T est and Increase its R eliability and V alidity. Lexington, M ass. : Shuford-M assengill C orp ., May, 1967. (Distributed by C learinghouse, AD677207.) (b) Shuford, E. H ., A lbert, A ., and M assengill, H. E. A dm issible probability m easurem ent procedures. Psvchom etrika, 1966, 31, 125-145. 71 Shuford, E. H. and M a ssen g ill, H. E. The Worth of Individualizing Instruction. Lexington, M ass. : S h u ford -M assen gill C o r p ., N ovem ber 1966. (D istributed by C learinghouse, A D 677204.) Shuford, E. H. and M a ssen g ill, H. E. Individual and S ocial J u stice in O bjective T estin g . Lexington, M ass. : S h u ford -M assen gill Corp. , N ovem ber 1967. (D istributed by C learin gh ou se, A D 677208.) Shuford, E. H. and M a ssen g ill, H. E. A irm an Q ualifying E xam ination-66 A dm inistered as a C onfidence T e s t. Lexington, M ass. : Shuford- M a ssen g ill Corp., May 1968. (D istributed by C learin gh ou se, A D 677210.) (a) Shuford, E. H. and M a ssen g ill, H. E. D ecisio n -T h eo r etic P sy c h o m e tr ic s. Lexington, M ass. : S h u ford -M assen gill Corp., N ovem ber 1968. j (D istributed by C learinghouse, A D 682330. ) (b) j i Shuford, E. H. and M a ssen g ill, H. E. C onfidence T estin g at the O fficer Train- j ing School. Lackland A ir F o r c e B ase: Septem ber 1968. Lexington, j M ass. : S h u ford -M assen gill Corp. , January 1969. (D istributed by j C learin gh ou se, AD685179.) (a) Shuford, E.H. and M assen gill, H. E. Item A n alysis B ased on Confidence R e sp o n se s. Lexington, M ass. : S h u ford -M assen gill Corp. , M arch 1969. (D istributed by C learin gh ou se, A D 685182.) Slakter, M. J. R isk taking on o b jective exam ination s. A m erican E ducational R esea rch Journal, 1967, 4 , 3 1 -4 3 . Slakter, M. J. The m easu rem en t and effect of r isk taking on ob jective exam i nations. F in al Report, P ro ject N o. 58428, C ontract No. O E -6 -1 0 - 239. U.S. D epartm ent of H ealth, Education, and W elfare, 1967. (b) S lakter, M. J. G enerality of r isk taking on ob jective exam inations. Educa tional and P sy ch o lo g ica l M easu rem en t, 1969, 29 , 115-128. Slakter, M. J. and K oehler, R. A. A new m e a su re of r isk taking on ob jective exam ination s. C alifornia Journal of E ducational R esea rch , 1968, 19, 132-137. Stanley, J. C. and Wang, M. D ifferen tia l W eighting: A Survey of M ethods and E m p irical Studies. New York: C ollege Entrance Exam ination Board, N ovem ber 1968. Sw ineford, F. The m easu rem en t of a p erson ality tra it. Journal o f Educa 72 tional Psychology, 1938, 2 9 , 295-300. Swineford, F. and M iller, P. M. E ffects of directions regarding gu essin g on item s ta tistic s on a m ultiple choice vocabulary test. Journal of Edu cational P sych ology, 1953, 44, 129-139. Toda, M. M easurem ent of Subjective Probability D istributions. Bedford, M ass. : D ecision Sciences Laboratory, L. G. Hanscom F ield, July 1963. (Distributed by Clearinghouse, AD416405.) Votaw, D. F. The effect of d o-n ot-gu ess directions upon the validity of tru e- fa lse or m u ltip le-ch oice te sts. Journal of Educational P sych ology. 1936, 27, 698-703. W iley, L. and T rim ble, O. C. The ordinary objective te st as a p o ssib le criterio n of certa in personality tra its. School and Society, 1936, 43, 446-448. W illey, C. F. The th ree-d ecisio n m u ltip le-ch oice test: A method of in crea s ing the sen sitivity of the m u ltip le-ch oice item . P sych ological R eports. 1960, 7, 475-477. Winer, B. J. S tatistical P rin cip les in Experim ental D esign. New York: M cG raw -H ill, 1962. Z iller, R. C. A m easure of the gam bling resp o n se-set in objective te sts. P svch om etrik a, 1957, 2 2 , 289-292. A P P E N D I X E S 73 APPENDIX A NONSENSE ITEMS APPENDIX A NONSENSE ITEMS The eighth and twelfth grade nonsense item s designed to provide a m easu re of risk taking on objective exam inations w ere: 1. The monetary policy of a nation is said to be distorged if a. the ratio of gold to paper money is not equal. b. the nation im ports m ore than it exports. c. the nation has foreign debts of m ore than half its g ross national product. d. the nation is unable to m eet its financial obligations in term s of foreign bonds. 2. The Railway Control Act was established by C ongress for which one of the following p urposes? a. to prevent p rice fixing. b. to a ssist in developing a modern rail system . c. to prevent m onopolies among railw ays. d. to encourage a m ore econom ic b ase for rail operations. 3. Under International Law, the concept of Tasarum refers to a. one nation illegally holding the property of another. b. one nation suing another for the return of a political prisoner. c. one nation violating the air space of another. d. one nation establishing m ilitary bases within the borders of another. 4. The Pacific States Treaty Organization (PSTO) established after W. W. II includes which one of the following nations ? a. Chile b. Australia c. Japan d. M alaysia 5. A W rit of Dolci when issu ed by the Supreme Court com pels 75 76 a. a state to forw ard sp e c ific court reco rd s. b. a state to drop a ll ch a rg es again st an individual. c . a sta te to rem ove a law from its books due to the law 's unconstitutionality. d. a state to extrad ite a p riso n er to another state. 6. Two independent nations a r e said to b e e s tr a s s e if a. they have connecting b ord ers. b. they have r e c ip r o c a l trad e agreem en ts. c. they have m utual d efen se agreem en ts. d. they have not d ip lo m a tica lly recogn ized each other. 7. The N arcotic C ontrol A ct of 1968 p la c e s the resp o n sib ility for enforcem ent of v io la tio n s o f fed era l drug law s in the hands of a. the P ure Food and Drug C om m ission . b. the T reasu ry D epartm ent. c. the D epartm ent o f H ealth, Education, and W elfare. d. the D epartm ent of J u stice. 8. The C ooperation A m ong States A ct a s recen tly p a ssed by the C ongress p rovid es fo r the esta b lish m en t of a. co m m ittees to stim u la te in te r sta te trade. b. co m m ittees to study m ean s of equalizing sta te law s. c. fiv e region al co m m ittee s to study com m on eco lo g ica l p rob lem s. d. uniform m a rria g e and d iv o rce law s. 9. The E co lo g ica l C ontrol A gency w as esta b lish ed by the C ongress in 1969 for w hich one of the follow ing p u rp oses ? a. to control and p reven t w ater pollution. b. to con trol and p rev en t further s o il erosion . c . to p ro tect the natural r e so u r c e s. d. to seek out solu tion s for th e prevention of a ir pollution. 10. The P o litic a l P r e ss u r e Group A ct req u ires w hich one of the follow ing: a. reg istra tio n o f lo b b y ists. b. d isc lo su r e of am ount of cam paign m oney spent during an election . c. the public listin g of o ffic er s of p o litica l action groups. _______ d. the fo rfeitu re of ille g a l cam paign contributions._____________ 77 The tenth grade nonsense item s designed to provide a m easu re of risk taking on objective exam inations were: 1. The Egyptian Dynasty under Turkamenon achieved which of the following ? a. unification of the Upper and Lower N ile regions. b. the com pletion of the construction of the Sphinx. c. the fir st recorded u se of the papyrus plant as a m eans of writing. d. the conquest of the F ertile C rescent. 2. Altm a, the Chinese Monk, was honored for a. the invention of gun powder. b. the invention of the water wheel. c. the fir st plotting of the earth's orbital path. d. the fir st printing of the works of Buddha. 3. During the Deformation period, Roland Hentz succeeded in a. uniting the banking houses of Europe. b. establishing the library at the m onestary of St. Cruiox. c. developing the fir st com m ercial publishing house. d. bringing an end to the fighting between the Stewarts and the Tudors of England. 4. The Treaty of Achwatzch established a. the end to the Seven Y ear War. b. the boundaries of the countries of Eastern Europe prior to World War I. c. the su ccesso r to the Spanish Throne after the defeat of the Armada. d. the dominant position of the Byzantine Em pire over the Middle East. 5. The P eace of Hamstead established which one of the follow ing? a. an end to the War of the R oses. b. the reunification of Germany and A ustria. c. an end to the religious q u arrels during the seventeenth century. 78 d. the s u c c e ss o r to the disputed D anish Throne at the end of the nineteenth century. 6. The g rea t s e ig e of St. P etersb u rg took p lace during a period when w hich of the follow ing cou n tries w ere at war ? a. R u ssia and Germany b. A u stria and Hungary c. R u ssia and A ustria d. G erm any, R u ssia, and A ustria 7. During the eighteenth century, the now n on -existen t E astern European country of Slovia w as b est known for a. its reputation as a trading cen ter. b. its g rea t storeh ou se of arts and cra fts. c. its pow erful arm y and navy. d. its lite r a r y accom plishm ents. 8. The Statter Plan, a risin g out of World War I, provided for a. the estab lish m en t of the League of N ations. b. the rendering of econom ic aid by the United States to the w ar dam aged cou n tries of Europe. c. the total and perm anent disarm am ent of Germany. d. a m utual d efen se p act betw een England and the United States. 9. The unification of the w arring trib es of the Crown R egion of A sia took p la ce under a. A ttila the Hun. b. G enghis Khan. c. Ivan the T errib le. d. A lexander the Great. 10. The P a c ific States T reaty Organization established after World War II in clu d es which one of the follow ing nations ? a. C hile b. A u stralia c. Japan d. M alaysia APPENDIX B ANSWER SHEET 79 APPENDIX B ANSWER SHEET NAME_________________________ ______________ GRADE___________ AGE Last F irst Middle 25% 25% 25% 25% 1. a. ( ) ( ) ( ) ( ) 7. a. ( ) ( ) ( ) ( b. ( ) ( ) ( ) ( ) b. ( ) { )( ) ( c. ( ) ( ) ( ) ( ) c. { ) ( ) ( ) ( d. ( ) ( ) ( ) ( ) d. ( ) ( ) ( ) ( 2. a. ( ) ( ) ( ) ( ) 8. a. ( ) ( ) ( ) ( b. ( ) ( ) ( ) ( ) b. ( ) ( ) ( ) ( c. ( ) ( ) ( ) ( ) c. ( ) ( ) ( ) ( d. ( ) ( ) ( ) ( ) d. ( )( )( ) ( 3. a. ( ) ( ) ( ) { ) 9- a. ( ) ( ) ( ) ( b. ( ) ( ) ( ) ( ) b. ( ) ( ) ( ) ( c. ( ) ( ) ( ) ( ) c. ( ) ( ) ( ) ( d. ( ) ( ) < ) ( ) d. ( ) < ) ( ) ( 4. a. ( ) { ) ( ) ( ) 10. a. ( ) ( ) ( ) ( b. ( ) ( ) ( ) ( ) b. ( )( ){ ) ( c. ( ) ( ) ( ) ( ) c. ( ) ( ) ( ) ( d. ( ) ( ) < ) ( ) d. ( ) ( ) ( ) ( 5. a. { ) ( ) ( ) ( ) 11. a. ( ) ( ) ( ) ( b. ( ) ( ) ( ) ( ) b. ( ) ( ) ( ) ( c. { ) ( ) { ) { ) c. ( ) ( ) ( ) ( d. ( ) ( ) ( ) ( ) d. ( ) ( ) ( ) ( 6. a. ( ) ( ) ( ) ( ) 12. a. ( ) ( ) ( ) ( b. ( ) { ) ( ) ( ) b. { )( )() ( c. ( ) ( ) ( ) ( ) c. ( ) ( ) ( ) ( d. ( ) ( ) ( ) ( ) d. ( ) ( ) < ) ( 80 APPENDIX C GENERAL DIRECTIONS 81 APPENDIX C GENERAL DIRECTIONS This is a 40-m inute test. Do not spend too much tim e on any one question. Rem em ber that when this te st is scored, a percentage of the number of wrong answ ers is subtracted from the number of co rrect answ ers as a cor rection for guessing. If you do not know the answ er to a question lea v e it blank and go on to the next one sin ce a gu ess w ill usually result in a low er score. Do not make any m arks in your te st booklet. Mark all answ ers on the separate answ er sheet. Mark only one answer for each question. If you make a m istake or w ish to change an answer, be sure to era se your fir st choice com pletely. Note how the answer to the exam ple below is marked: 1. Chicago is a a. state b. county c. city d. village On your answ er sheet you would mark as below: a b e d ! • < ) ( ) # > ( ) Copyright (cT) 1964, Educational Testing Service. A ll rights reserved. Adapted and reproduced by perm ission. 82 APPENDIX D DIRECTIONS FOR PROBABILITY RESPONDING 83 APPENDIX D DIRECTIONS FOR PROBABILITY RESPONDING T om orrow you w ill be taking a te s t which w ill req u ire you to an sw er in a way w hich w ill be d ifferen t than any you have u sed in the p a st. This sy stem is ca lled p erso n a l probability responding. A ll of you a re fa m ilia r with th e con cep t of probability or chance. W eather fo r e c a sts dealing w ith rain a re u su ally stated in term s of probability of occu rren ce. H ence, when the w eather m an sa y s th ere is a 70% chance of rain today, he sayin g that under id en tical conditions lik e th ese, rain w ill o ccu r 70 days out of 100. If he says that the probability of rain is 10% fo r today, on only 10 days out of 100 w ill rain occu r under th e se sa m e conditions. In answ ering the te s t, you w ill be given the opportunity to state your own p erso n a l p rob ab ilities concerning th e c o r r e c tn e ss o f your a n sw ers in blocks o f 25%. T his p r o c e ss w ill be illu stra ted in the follow in g exam p les: Y our te s t booklet w ill have a q u estion or statem en t follow ed by four ch o ice s, 1. C hicago is a a. state b. county c. city d. v illa g e Your answ er sh e e t w ill look lik e th is, a. ( ) { ) ( ) ( ) b. ( ) ( ) ( ) ( ) c. ( ) ( ) ( ) ( ) d. ( ) ( ) ( ) ( ) E ach m ark that you m ake equals a probability of 25%. For each q u es tion you m u st m ake four m ark s totalin g a probability o f 100%, how ever, you a re fr e e to spread the four m ark s o v er the ch o ices in any m anner you d e sir e . If you a r e 100% certa in that ch o ice c is the c o r r e c t an sw er, you would respond in the follow ing way: 25% 25% 25% 25% 1. a. ( ) ( ) ( ) ( ) b. ( ) ( ) ( ) ( ) 85 o . ( X ) (* ) (X ox d. ( ) ( ) ( ) ( ) If you are alm ost certain that c is the co rrect choice, but som e proba bility rem ains that d could be the correct answ er, you would answ er in the following way : 1. a. n n < 25? n b. ( ) < ) ( ) < ) c. <X> 06 < tI) () d. </> < ) ( ) () If you are com pletely undecided between choices c and d, you would respond in the following w ay: 25% 25% 25% 25% a. ( ) ( ) ( ) ( ) b. ( ) ( ) ( ) ( ) c. w (X) ( ) ( ) d. 1 X J X ( ) ( ) If you are undecided between three ch o ices, one of the choices m ust receive two m arks in order for you to u se all four blocks of probability. 25% 25% 25% 25% a. ( ) ( ) ( ) ( ) b. (X ) ( ) ( ) ( ) c. (X ) (X ) ( ) ( ) d. o o ( ) ( ) ( ) R em em ber, you m ust make a total of four m arks per question thereby accounting for a total probability of 100%. If you a re com pletely unsure of any of the ch oices, you would mark in the following way: a. 8 25% ( ) 25% ( ) 25% ( ) b. ( ) ( ) ( ) c. 0 4 ( ) < ) ( ) d. ( ) ( ) ( ) There is no question of whether to g u ess or not with this type of an- sering procedure. A ll you have to do is place on the answ er sheet your own personal probabilities concerning the co rrectn ess of the ch oices. By following j such a procedure you w ill be able to make the highest p ossib le sco re you are 86 capable of making and to give the teacher the tru est pictu re of your present knowledge. APPENDIX E PROCEDURE FOR COMPARING RELIABILITIES 87 APPENDIX E PROCEDURE FOR COMPARING RELIABILITIES K risto f’s (1964) procedure for com paring the r elia b ilitie s of two te sts g iv en to the sam e sam p le is shown. The two relia b ility co efficien ts to be com pared a r e com puted from the v a ria n ce-co v a ria n ce m atrix com posed of the four half te s ts by r = 2 tt The m atrix Sjk 1 - °1 + 2 a2 - L 2 °1 + Ct + 2V 12 Q 1 a 2 s u 0 S 0 S 2 2 0 S 1 3 0 s 0 0 0 13 33 0 0 0 44 i s determ ined with the aid of equation one through five The determ ination of 2 r = 13 S11 S33 and t = S22 S33 S44 is n ecessa ry in order to employ equation 6 a - S „ (1+r2 ) a ^ + | s u [ S ^ T K r 2 ) - t(l-2 r 2 )]a 11 "11 11 - \ r 2 <s u + t > - 0 The root of th is cubic is found.and the rem aining term s of the m atrix ( a j k ) a re obtained from 5 11 0 S31 0 0 22 0 0 S13 0 ° 33 0 0 0 0 a 44 90 and a 22 a l l ^44 •4 33 using equation 7 taking common logarithm s 0 2 X = 2.3026 (N -l) • [Ig o 22+ lg a 44 + Ig ( c r n cr33 - S13 ) " * S22 - lg S44 ' Ig<Sl l S33 - 4 > 1 2 a X is obtained with one degree of freedom . APPENDIX F NUMBER AND PER CENT OF SUBJECTS COMPLETING VARIOUS PERCENTAGES OF ITEMS ON EACH TEST 91 APPEN D IX F NUMBER AND PER CENT OF SUBJECTS COMPLETING VARIOUS PERCENTAGES OF ITEMS ON EACH TEST Item s A nsw ered Eighth G rade— 70 Item s T otal Conventional Probability 50% (35) 97% (88) 98% (89) 75% (53) 75% (68) 74% (67) 95% (67) 57% (52) 46% (42) 100% (70) 51% (46) 42% (38) 50% (35) Tenth G rade— 70 Item s T otal 88% (80) 99% (90) 75% (53) 84% (76) 96% (88) 95% (67) 56% (60) 88% (80) 100% (70) 47% (43) 88% (80) 50% (33) Tw elfth G rade— 65 Item s Total 98% (89) 98% (89) 75% (49) 92% (84) 77% (70) 95% (62) 60% (55) 45% (41) 100% (65) 52% (47) 37% (34) 92 APPENDIX G NUMBER AND PEE CENT OF PROBABILITY RESPONSES MADE BY EIGHTH GRADE SUBJECTS ACCORDING TO ITEM SCORES 93 APPENDIX G NUMBER AND PER CENT OF PROBABILITY RESPONSES MADE BY EIGHTH GRADE SUBJECTS ACCORDING TO ITEM SCORES Score Total High Low 1. 00 1542 312 455 24% 18% 26% 0. 00 1721 489 70 27% 28% 31% . 94 257 81 70 4% 5% 4% .4 4 110 38 30 2% 2% 2% . 19 152 57 30 2% 3% 2% .7 5 622 184 145 10% 11% 8% .2 5 335 119 56 5% 7% 3% .8 1 153 58 39 2% 3% 2% . 56 276 95 73 4% 5% 4% .3 1 106 40 23 2% 2% 1% . 62 1092 275 291 17% 16% 17% APPENDIX H NUMBER AND PER CENT OF PROBABILITY RESPONSES MADE BY TENTH GRADE SUBJECTS ACCORDING TO ITEM SCORES 95 APPENDIX H NUMBER AND PER CENT OF PROBABILITY RESPONSES MADE BY TENTH GRADE SUBJECTS ACCORDING TO ITEM SCORES Score Total High Low 1.00 1035 244 302 16% 14% 17% 0.00 1191 315 328 19% 18% 19% .94 316 64 72 5% 3% 2% .44 197 51 41 3% 3% 2% .19 256 59 58 4% 3% 3% .75 647 181 179 10% 10% 10% .25 392 127 88 6% 7% 5% .81 227 68 59 4% 4% 3% .56 373 123 84 6% 7% 5% .31 154 46 43 2% 3% 2% .62 1578 471 493 25% 27% 28% APPENDIX I NUMBER AND PER CENT OF PROBABILITY RESPONSES MADE BY TWELFTH GRADE SUBJECTS ACCORDING TO ITEM SCORES APPENDIX I NUMBER AND PER CENT OF PROBABILITY RESPONSES MADE BY TWELFTH GRADE SUBJECTS ACCORDING TO ITEM SCORES Score Total High Low 1.00 1441 329 417 24% 20% 26% 0.00 1670 506 415 28% 31% 26% .9 4 351 77 114 6% 5% 7% .4 4 138 34 43 2% 2% 3% .19 213 50 48 4% 3% 3% .75 546 168 142 9% 10% 9% .25 291 115 55 5% 7% 3% .81 141 41 52 2% 3% 3% .56 229 75 56 4% 5% 3% .31 91 26 22 2% 2% 1% .62 804 204 261 14% 13% 16% ?8
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
An Exploration Of Interpersonal Behavioral Possibilities And Probabilities
PDF
The Relationship Of Multidimensional Scaling Spaces Of Trait Adjectives For Different Reference Persons
PDF
Personality Variables And Intellectual Abilities As Determinants Of Concept Learning
PDF
The relationship between process and presage critera of college teaching effectiveness
PDF
Delay Of Feedback And The Acquisition And Retention Of Verbal Material Inthe Classroom
PDF
An Evaluation Of The Mastery Of Simple English Sentence Structure By Secondary Age Deaf Students
PDF
The Influence Of Communality And N On The Sampling Distributions Of Factor Loadings
PDF
The relationship of scores on a measure of test-wiseness to performance on teacher-made objective achievement examinations and on standardized ability and achievement tests, to grade-point average, an
PDF
The effects of an RNA polymerase in the improvement of verbal learning and memory
PDF
The Systematic Isolation And Validation Of Personality Determiners In The Handwriting Of School Children
PDF
Conflicting Motives In The Prisoner'S Dilemma Game
PDF
A Multidimensional Similarities Analysis Of Twelve Choice Probability Learning With Payoffs
PDF
The Influence Of Other People'S Schooling On An Individual'S Income
PDF
Effects Of A Workshop And Simulation Game On Teacher Ability And Willingness To Individualize Instruction
PDF
The Effects Of Making Social Desirability Judgments On Personality Inventory Scores Of Schizophrenics
PDF
The Effects Of Early Educational Stimulation On Selected Characteristics Of Kindergarten Children
PDF
An exploratory study of the psychological sequelae of surgery and hospitalization in male children
PDF
Relationships between self-concept, specified scholastic variables, and the grade point averages of selected continuation high school students
PDF
The Modification Of Age-Specific Expectations Of Piaget'S Theory Of Development Of Intentionality In Moral Judgments Of Four-Year-Old To Seven-Year-Old Children In Relation To Use Of Puppets In A...
PDF
The Relationship Between Parent-Child Reciprocity And The Child'S Mental Level
Asset Metadata
Creator
Martois, John Sherman (author)
Core Title
An Empirical Analysis Of The Relationship Between Risk Taking And Personal Probability Responding On Multiple-Choice Examinations
Degree
Doctor of Philosophy
Degree Program
Education
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
education, educational psychology,OAI-PMH Harvest
Language
English
Contributor
Digitized by ProQuest
(provenance)
Advisor
Smith, Robert A. (
committee chair
), Cliff, Norman (
committee member
), Fox, Frank H. (
committee member
)
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c18-549398
Unique identifier
UC11362992
Identifier
7203790.pdf (filename),usctheses-c18-549398 (legacy record id)
Legacy Identifier
7203790
Dmrecord
549398
Document Type
Dissertation
Rights
Martois, John Sherman
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA
Tags
education, educational psychology