University of Southern California Dissertations and Theses

Processing the dynamicity of events in language

(USC Thesis Other)

Processing the dynamicity of events in language

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content PROCESSING THE DYNAMICITY OF EVENTS IN LANGUAGE by Sarah Hye-yeon Lee A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (LINGUISTICS) May 2022 Copyright 2022 Sarah Hye-yeon Lee ii Acknowledgements I would first like to express my utmost gratitude to my advisor Elsi Kaiser. I have been extremely lucky to have such a supportive, dedicated, and tireless mentor. Thank you for many, many hours of reading through everything I have written during the last five years. Thank you for having unwavering faith in my ideas and abilities even when I didn’t. Thank you for training and nurturing me to become the researcher I am today. Thank you for everything. I would also like to thank my committee members, Roumyana (Roumi) Pancheva, Deniz Rudin, and Alexis Wellwood. Roumi has been the second biggest source of guidance and support during my graduate career at USC. I thank her for inspiring and pushing me to be a better linguist. Working with her on my screening paper on superlatives has been “the most” delightful experience. I am grateful to Deniz for always being interested in my psycholinguistics and semantics research projects. I could always count on him to have insightful comments that would help better contextualize the contributions of the research. I thank Alexis for providing many incisive comments on both the theory and the data in this dissertation. Her comments have been incredibly beneficial for enriching my thinking. Thanks also to Maria-Luisa Zubizaretta, Stefan Keine, and Rand Wilcox for serving on committees on my screening and qualifying papers, which led up to this dissertation. Their comments and feedback have been invaluable. My linguistics journey has been greatly influenced by my mentors back home at Seoul National University. I thank Eun-jung Yoo, Ki-sun Hong, and Heejeong Ko for showing the beauty of Linguistics to a baby linguist. I especially thank my MA advisor Eun- jung Yoo for her continued support over the years, through my MA and PhD. iii I gratefully acknowledge the assistance of my wonderful RA, Mila Mathias, who helped out with Experiments 3, 4, and 5. Special thanks are also due to Jesse Storbeck for answering all my Webgazer questions. The eye-tracking experiment in Chapter 5 would never have happened without him! I also thank our departmental staff Guillermo Ruiz, not just for helping me with all the paperwork and reimbursements, but for being such a caring and warm presence in the department. I would like to thank my fellow graduate students at USC. Many thanks to: my dear cohort Miran Oh, Daniel Plesniak, Madhumanti Datta (and Yoona Yee and Merouane Benhassine for the first two years) for the friendship and memories. I especially thank Miran Oh for standing by me through everything - professional and personal. Thanks to Silvia Kim, for moving in with me during the most stressful time of writing this dissertation and for feeding me. To Betul Erbasi and Ana Besserman, for being such caring and supportive friends through the ups and downs of grad school. To the USC psycholinguists, Monica Do, Binh Ngo, Ana Besserman, Jesse Storbeck, Jina Song, Silvia Kim, Cindy Chiang, Ian Rigby, and Jun Lyu for hearing about this research so many times and yet always providing insightful suggestions. To my Korean friends Cynthia Yoonjeong Lee, Hayeun Jang, Jina Song, Yoona Yee, Silvia Kim, Miran Oh, and Sung Hah Hwang for sharing with me the laughs, the agonies, and most importantly, the food. I love you all. Thanks also to Tommy Tsz Ming Lee, Sarah Harper, Reed Blaylock, Yijing Lu, Charlie O’Hara, Yifan Yang, Luis Miguel Toquero Perez, Jessie Johnson, Samir Alam, and Yubin Zhang for being great company over the years. Outside of USC, I thank my linguist friends Sanghee Kim and Jinyoung Jo, for listening to my dissertation rants. Much love for anyone that I’m forgetting here. Finally, I would like to thank my family. None of this would have been possible without the unwavering love and support from my parents Sunpyo Lee and Sungeun Kim and iv my brother Andrew Choongwon Lee. This dissertation is dedicated to my late grandfather Dr. Moon Young Lee, who was the first person to inspire me to be an academic and a scholar. He once told me that one’s dissertation work launches a lifelong inquiry – that it should raise questions that she would try to answer for the rest of her academic career. I hope this dissertation achieves that, on some level. I gratefully acknowledge support from the National Science Foundation under the Doctoral Dissertation Research Improvement Grant #BCS-2041261. Parts of the research presented in Chapters 2 and 3 are published in Language, Cognition, and Neuroscience (Lee & Kaiser, 2021). v Table of Contents Acknowledgements ................................................................................................................ ii List of Tables ....................................................................................................................... viii List of Figures ........................................................................................................................ ix Abstract ................................................................................................................................... x Chapter 1: Introduction ......................................................................................................... 1 1.1. Introduction ................................................................................................................... 1 1.2. Change in human cognition .......................................................................................... 1 1.3. Change in verb semantics ............................................................................................. 3 1.4. Going beyond verb semantics ....................................................................................... 4 1.5. Overview of the dissertation ......................................................................................... 6 Chapter 2: Discourse-level information interacts with verb tense to influence object state representations ............................................................................................................... 9 2.1. Introduction ................................................................................................................... 9 2.2. The role of verb tense in object state representations ................................................. 12 2.3. How do object state representations go beyond verb meaning? ................................. 15 2.4. Verbs differ in the information they provide about object state change: result verbs vs. manner verbs ................................................................................................................ 17 2.5. Does discourse-level information interact with verb-level information to guide the representation of object state change? ......................................................................... 20 2.6. Aims of this work ....................................................................................................... 22 2.7. Experiment 1 ............................................................................................................... 23 2.7.1. Methods ............................................................................................................. 25 2.7.1.1. Participants ............................................................................................... 25 2.7.1.2. Design and Materials ................................................................................ 25 2.7.1.3. Procedure .................................................................................................. 28 2.7.2. Predictions ......................................................................................................... 29 2.7.3. Likelihood-of-change Norming Study ............................................................... 31 2.7.4. Data Processing and Analysis ............................................................................ 32 2.7.5. Results ................................................................................................................ 33 2.8. Discussion ................................................................................................................... 35 Chapter 3: Discourse-level information interacts with verb semantics to influence object state representations ....................................................................................................... .... 38 3.1. Introduction ................................................................................................................. 38 3.2. Experiment 2 ............................................................................................................... 40 3.2.1. Methods ............................................................................................................. 40 3.2.1.1. Participants ............................................................................................... 41 3.2.1.2. Design and Materials ................................................................................ 41 3.2.2. Procedure ........................................................................................................... 44 vi 3.2.3. Data Processing and Analysis ............................................................................ 44 3.2.4. Predictions ......................................................................................................... 45 3.2.5. Results ................................................................................................................ 46 3.3. Discussion ................................................................................................................... 49 3.4. General Discussion about Experiments 1 and 2 ......................................................... 50 3.5. A note about lexicalized and inferred results .............................................................. 53 Chapter 4: Grammatical markers of temporal event structure interact with real-world event knowledge during event comprehension .................................................................. 56 4.1. Introduction ................................................................................................................. 56 4.2. Experiment 3: Grammatical aspect ............................................................................. 58 4.2.1. Methods ............................................................................................................. 62 4.2.1.1. Participants ............................................................................................... 62 4.2.1.2. Design and Materials ................................................................................ 63 4.2.1.3. Procedure .................................................................................................. 65 4.2.2. Predictions ......................................................................................................... 65 4.2.3. Data Processing and Analysis ............................................................................ 66 4.2.4. Results ................................................................................................................ 67 4.2.5. Discussion .......................................................................................................... 69 4.3. Experiment 4: Tense ................................................................................................... 71 4.3.1. Methods ............................................................................................................. 73 4.3.1.1. Participants ............................................................................................... 73 4.3.1.2. Design and Materials ................................................................................ 74 4.3.2. Procedure ........................................................................................................... 75 4.3.3. Predictions ......................................................................................................... 75 4.3.4. Data processing and analysis ............................................................................. 76 4.3.5. Results ................................................................................................................ 77 4.3.6. Discussion .......................................................................................................... 79 4.4. General Discussion ..................................................................................................... 80 Chapter 5: Grammatical cues dynamically update object location representations in real-time ................................................................................................................................. 83 5.1.Introduction .................................................................................................................. 83 5.2. Grammatical aspect ..................................................................................................... 84 5.3. Verb semantics ............................................................................................................ 85 5.4. Argument realization patterns ..................................................................................... 87 5.5. Experiment 5 ............................................................................................................... 88 5.5.1. Methods ............................................................................................................. 89 5.5.1.1. Participants ............................................................................................... 89 5.5.1.2. Materials ................................................................................................... 89 5.5.1.2.1. Auditory (sentence) stimuli ........................................................... 89 5.5.1.2.2. Display (Visual scenes) ................................................................. 91 5.5.2. Procedure ........................................................................................................... 94 5.5.3. Predictions ......................................................................................................... 95 5.5.3.1. Predictions about post-sentential interpretations ...................................... 95 5.5.3.2. Predictions about real-time processing ..................................................... 97 5.5.4. Data processing and analysis ............................................................................. 98 5.5.4.1. Click data (Post-sentential interpretations) ............................................... 98 vii 5.5.4.2. Eye gaze data (Real-time processing) ....................................................... 99 5.5.5. Results ................................................................................................................ 99 5.5.5.1. Post-sentential interpretations ................................................................... 99 5.5.5.2. Real-time processing ............................................................................... 102 5.6. Discussion ................................................................................................................. 104 Chapter 6: Conclusions ...................................................................................................... 107 6.1. Overview and summary ................................................................................................. 107 6.2 Theoretical implications ................................................................................................. 109 6.2.1 Language, linguistic theory, and event representations .................................... 109 6.2.2 Going beyond language .................................................................................... 110 6.3 Methodological implications .......................................................................................... 111 6.4 A note about attentional focus ........................................................................................ 112 6.5 Future directions ............................................................................................................. 113 6.6 Final remarks .................................................................................................................. 113 References ............................................................................................................................ 115 Appendix A. Target stimuli for Experiment 1 ..................................................................... 127 Appendix B. Target stimuli for Experiment 2 ..................................................................... 131 Appendix C. Target stimuli for Experiment 3 ..................................................................... 137 Appendix D. Target stimuli for Experiment 4 ..................................................................... 142 Appendix E. Target stimuli for Experiment 5 ..................................................................... 147 Appendix F. Experiment 5: Click data for all conditions (Error bars show +/- 1 SE) ........ 149 Appendix G. Experiment 5: Eye gaze data in imperfective and perfective conditions ....... 150 Appendix H. Experiment 5: Eye gaze data in imperfective + double object, perfective + double object, imperfective + to-, and perfective + to- conditions ...................................... 151 Appendix I. Experiment 5: The time course of GOAL-SOURCE difference scores in imperfective aspect trials and in perfective aspect trials ...................................................... 153 viii List of Tables Table 2.1. Experiment 1: Results of the lmer model ............................................................. 34 Table 2.2. Experiment 1: Planned comparisons, Object-QUD conditions only .................... 35 Table 2.3. Experiment 1: Planned comparisons, Subject-QUD conditions only ................... 35 Table 3.1. Results of lmer models for each word in the target region ................................... 48 Table 4.1 Sample target item for Experiment 3 ..................................................................... 64 Table 4.2 Sample target item for Experiment 4 ..................................................................... 74 Table 5.1. Areas of Interest and analysis regions .................................................................. 93 ix List of Figures Figure 2.1. Mean reaction times by condition in Experiment 1 ............................................ 34 Figure 3.1. Experiment 2: Average reading times by word position ..................................... 46 Figure 4.1 Difference between perfective and imperfective aspect ...................................... 60 Figure 4.2 Average raw RTs to the image by condition (ms) in Experiment 3 ..................... 69 Figure 4.3 Average raw RTs to the image by condition (ms) in Experiment 4 ..................... 78 Figure 5.1 Sample target visual image .................................................................................. 92 Figure 5.2. Areas of interest and regions for analyses ........................................................... 93 Figure 5.3. Proportion of clicks on each area of interest ..................................................... 100 Figure 5.4 Proportion of clicks on the SOURCE and GOAL regions ................................. 101 Figure 5.5. Proportion of clicks on MIDDLE region .......................................................... 102 Figure 5.6. Proportions of GOAL region looks by grammatical aspect (left), Proportions of SOURCE region looks by grammatical aspect (right); Proportions of Center area looks are not plotted; 0 on the x-axis indicates the onset of the verb; Data is collapsed by participant for plotting .................................................................................................................................. 103 x Abstract This dissertation explores the question of how the language comprehension system constructs mental representations of events based on linguistic descriptions of events. When a comprehender encounters linguistic input describing an event, what information guides how a mental representation of that event is constructed? How is information from various linguistic and non-linguistic sources integrated to create an understanding of an event? I specifically focus on the dimension of dynamic changes occurring to objects in an event. I report five experiments that were designed to investigate how comprehenders map linguistic input onto mental representations of events during language processing. I investigate three grammatical and non-grammatical factors: (a) grammatical properties of event descriptions (e.g. verb semantics, grammatical aspect, tense), (b) discourse-level properties, and (c) real-world knowledge about physical events. Chapters 2 and 3 investigate the question of how discourse-level and verb-level cues interact to guide the construction of object state representations. Chapter 2 reports a lexical decision experiment that investigates how verb tense information interacts with discourse- level cues about which event participant is being discussed (Experiment 1). Chapter 3 reports a self-paced reading experiment that investigates how verb type (result verb vs. manner verb) information interacts with discourse-level cues about whether the object’s resultant state is under discussion (Experiment 2). In both experiments, I find an interaction between discourse-level information and verb-level information in guiding object state representations. The findings highlight the need to take into account discourse-level factors in theorizing about the cognitive process of understanding the dynamics of event representation during language comprehension. xi Chapter 4 reports two rebus paradigm experiments that investigate how grammatical cues (Experiment 3: grammatical aspect, Experiment 4: tense) about an event’s temporal structure interact with real-world knowledge about the likelihood of state change (e.g. dropping a wine glass vs. dropping a plastic cup). Results from both experiments suggest that grammatical cues about an event’s temporal structure are rapidly integrated with non- linguistic information about the real-world to influence object state representations. Chapter 5 reports a visual-world eye-tracking study that investigates how different types of grammatical cues (grammatical aspect, verb type, argument realization patterns) influence the real-time comprehension of transfer events (Experiment 5). This experiment looks at the representation of an object’s change-of-location. Results from Experiment 5 show that comprehenders use different grammatical cues to shape their representation of object location, as the sentence unfolds in real-time. Taken together, this dissertation research sheds light on the psycholinguistic processes involved in recruiting information from both grammatical and non-grammatical sources to dynamically update comprehenders’ event representations. First, it highlights the role of grammatical factors in building event representations during sentence processing. The temporal domain of event representations – the temporal contour or the temporal perspective – is a central component of event representations. Second, this work also motivates the need for theories of event representation to incorporate and account for discourse-level information and real-world knowledge about physical events and objects as meaningful components. 1 Chapter 1. Introduction 1.1 Introduction The world is a dynamically changing place, filled with different kinds of events. Events, which are broadly defined in the dictionary as “things that happen” – a girl eating an apple, a man washing dishes, a thief breaking a window, etc. – are a notion that is crucial to informing theories of human perception, action, language, and the human mind. The significance of events has been studied across various disciplines that span across fields including cognitive psychology, linguistics, and philosophy. Much of human communication involves talking about events. During language processing, comprehenders form mental representations of the described events. This dissertation aims to shed light on the question of how the language comprehension system maps linguistic descriptions of events onto mental representations of events. Event representations consist of multiple components, including the participants of the event, the relations between them, and the temporal nature of events, amongst other things. In this dissertation, I study the representation of the internal temporal structure of events by focusing on dynamic events that cause change in the world. In the following sections, I discuss how the notion of change is represented in human cognition and language (Sections 1.2-1.3). Then, I discuss how during language processing, the mental representations of events often go beyond grammatical properties encoded in the linguistic input (Section 1.4). The last section (Section 1.5) provides an overview of the structure of the dissertation. 1.2 Change in human cognition 2 What is central to understanding the dynamicity of events is the notion of change. Events can be defined as ‘a segment of time at a given location that is conceived by an observer to have a beginning and an end’ (Zacks & Tversky, 2001: 3). Often, the state-of- affairs of the world that holds at the beginning of an event and at the end of an event are not entirely the same. For example, an entity involved in an event may undergo a transition from state/location A to state/location B. In an event described by the sentence Kim chopped the onion, the onion transitions from being in an unchopped state to being in a chopped state (change-of-state). In an event described by the sentence Kim moved from New York to Los Angeles, Kim’s location changes from New York to Los Angeles (change-of-location 1 ). Change is a fundamental concept in human cognition. A growing body of work in cognitive psychology suggests that humans are finely attuned to the notion of change. For example, humans use the presence of unexpected changes to identify event boundaries (event segmentation, e.g. Newtson, Engquist, & Bois, 1977; Magliano, Miller, & Zwaan, 2001; Speer, Zacks, & Reynolds, 2004; Zacks, 2004; Hard, Tversky, & Lang, 2006; Zacks, Swallow, Speer, & Reynolds, 2006; Shipley & Mcguire, 2008). In Altmann and Ekves’ (2019) recent theory of event representations (Intersecting Object Histories (IOH) theory), it has been proposed that the trajectory of changes that objects 2 undergo is key to understanding an event’s inner (temporal) structure. In IOH, events are represented as series of intersecting representations of the objects undergoing change across time and space. For example, understanding an event described by the sentence “The chef chopped the onion” requires tracking the changes (in state and location) of the onion, the 1 The conceptual similarities between change-of-state and change-of-location have been discussed by Gruber (1965), Jackendoff (1972, 1978), Gropen et al. (1991), and Pustejovsky (1991), amongst others, and Beavers (2011) proposes a theory to subsume change-of-possession under other types of change. 2 The term ‘object’ does not imply that the entity is an object in the syntactic argument-structure sense. There is an established tradition – which I follow – of using the term ‘objects’ to refer, broadly speaking, to ‘things’ in the real world. 3 chef, and any instruments that (might have) mediated the interaction between the chef and the onion, as the event unfolds. Indeed, the cognitive system can track multiple representations of the same object as it undergoes change-of-location (e.g. Altmann & Kamide 2009) or change- of-state (e.g. Altmann & Kamide, 2007; Hindy et al. 2012; Solomon et al., 2015; Kang et al., 2020) The significance of the notion of change has also been shown by other researchers. For example, humans are sensitive to the nature of the object’s change-of-state, i.e., whether the change has an inherent/specified resultant state (“(un)boundedness”) (e.g. Ji & Papafragou 2020a; 2020b). Change-of-state has also been shown to be cognitively salient both in verbal and non-verbal encoding of simple events (e.g. Sakarias and Flecken 2019). This dissertation builds on these studies to shed light on the real-time language processing mechanisms that influence how comprehenders represent the changes that an object may undergo. This work also aims to broaden our understanding of different kinds of changes by investigating not only change-of-state, but also other types of change such as change-of-location 3 . 1.3 Change in verb semantics The importance of the notion of change has been acknowledged in linguistic theories of event representations as well (e.g. Dowty 1979, Rappaport Hovav & Levin 1998, Vendler 1957, Warglien, Gärdenfors, & Westera 2012). Indeed, the human sensitivity to the presence/absence of change is reflected in the fact that language is equipped with ways to convey the difference between events that necessarily cause change and those that do not. 3 In this chapter and in subsequent chapters, I use the terms “state change” and “location change” interchangeably with “change-of-state” and “change-of-location”. 4 Linguistic theories of verb meaning and events (e.g. Dowty 1979, Rappaport Hovav & Levin 1998) have long recognized the importance of change for event representations. Theories of lexical semantics have identified a dichotomy between verbs that entail change and verbs that do not. The former (e.g. arrive, clean, break, fill, open) include in their semantic representations a well-defined change-of-state entailed by the action. These verbs have been termed result verbs (e.g. Rappaport Hovav & Levin 2010), reflecting the fact that they lexicalize the result of the action: It is obvious that a cleaning-a-shirt event results in the state of the shirt being clean: The result state (clean) of the object is encoded in the meaning of clean. This meaning is derived from the semantic template [x CAUSE [y BECOME clean]] (e.g. Rappaport Hovav & Levin 1998). These result verbs are distinguished from manner verbs (e.g. run, wash, hit, shake), which lexicalize the manner in which an action is carried out, but not the result. For example, wash encodes the particular manner in which the agent comes in physical contact with the object that is acted upon, but does not necessarily entail a result. The washed object may or may not undergo change-of-state. Similarly, run encodes the manner in which the agent moves, but does not entail that the agent ends up at a certain location. The result state of the object, thus, can only be inferred from the context (e.g. Talmy 1991, 2000, Rappaport Hovav & Levin 1998, Wittek 2002, Alexiadou et al. 2017). 1.4 Going beyond verb semantics Lexical semantics theories assume that verb semantics capture the underlying cognitive structure of events (e.g. Rappaport Hovav & Levin, 1998). However, the mental representations that comprehenders build based on sentences often go beyond verb semantics. Inferences drawn from non-grammatical sources can be used to augment the basic ‘skeleton’ 5 of an event representation. For example, successful comprehension of the linguistic input during communication involves understanding the utterances in the context of communicative goals, in addition to understanding the compositional meaning of the individual utterances that are spoken. If Lisa says “Can I wear the blue shirt to my interview? I spilled coffee on it earlier” and Bob says “I washed it”, we can draw the inference that the shirt is now clean – although, grammatically speaking, the meaning of ‘wash’ does not entail that something becomes clean. (Consider: “Bob washed the shirt but the stains did not come out.”) Thus, in addition to the grammatical properties of utterances, discourse-level properties shape language users’ mental representation of the described events. Language comprehension also involves the use of real-world event knowledge (e.g. Generalized Event Knowledge, McRae & Matsuki, 2009): Hearing that Bob dropped a wine glass vs. a plastic mug will elicit the construction of two different kinds of event representations – one with a broken glass and one with an unscathed mug. In this case, verb semantics alone is not sufficient in explaining how we reach different kinds of event representations (one with state change and the other without state change), even with the same verb. We need to account for how comprehenders also integrate knowledge from real- world experience. In other words, in understanding descriptions of events, many things are inferred. Comprehenders often augment their mental representation of the event with what they can infer, going beyond what is explicitly said. For example, even when the verb’s core meaning does not encode a change (e.g. wash), people may still construct an augmented event representation involving state change (e.g. a clean state of a shirt). This raises the question of how different information sources (e.g. grammatical properties of the utterance, discourse- level properties, real-world knowledge about events) interact with each other to guide the 6 inferencing process to update mental representations of events. This is the main question that guides the studies reported in this dissertation. Determining how different kinds of information are integrated in real time is a core issue for all theories of language processing. Early sentence-processing research proposed that the first moments of processing are informationally encapsulated, governed only by syntactic considerations (e.g. the ‘syntax-first’ model, e.g. Frazier 1987, Frazier & Fodor 1978). Since then, a growing body of work has merged arguing that non-syntactic information also guides sentence processing early on (e.g. Trueswell & Tanenhaus 1994, Trueswell, Tanenhaus & Garnsey 1994). Historically, much of this debate has tended to focus on the question of how cues about syntactic structure are integrated with other kinds of information. In this dissertation, I turn our attention to another foundational property of sentences, the semantics of event structure. I look specifically at how different sources of information are used to dynamically update comprehenders’ representations of changing situations. 1.5 Overview of the dissertation The main aim of this dissertation is to investigate how comprehenders map linguistic input onto mental representations of events during language processing. When a comprehender encounters linguistic input describing an event, what information guides how a mental representation of that event is constructed? I report five experiments that shed light on this question. I am especially interested in what influences how comprehenders understand the dynamic changes occurring to objects in an event. I investigate three grammatical and non- grammatical factors: (a) grammatical properties of event descriptions (e.g. verb semantics, 7 grammatical aspect, tense), (b) discourse-level properties, and (c) real-world knowledge about physical events. The interplay between these information types is crucial for understanding how event representations are built during comprehension. Grammatical properties form the basis of compositional meaning, which forms the skeleton of the mental representations of described events, but these may be enriched with inferences that go beyond compositional meaning. Chapters 2 and 3 investigate the question of how discourse-level and verb-level cues interact to guide the construction of object state representations. Chapter 2 reports a lexical decision experiment that investigates how verb tense information interacts with discourse- level cues about which event participant is being discussed (Experiment 1). Chapter 3 reports a self-paced reading experiment that investigates how verb type (result verb vs. manner verb) information interacts with discourse-level cues about whether the object’s resultant state is under discussion (Experiment 2). In both experiments, I find an interaction between discourse-level information and verb-level information in guiding object state representations. The findings highlight the need to take into account discourse-level factors in theorizing about the cognitive process of understanding the dynamics of event representation during language comprehension. Chapter 4 reports two rebus paradigm experiments that investigate how grammatical cues (Experiment 3: grammatical aspect, Experiment 4: tense) about an event’s temporal structure interact with real-world knowledge about the likelihood of state change (e.g. dropping a wine glass vs. dropping a plastic cup). Results from both experiments suggest that grammatical cues about an event’s temporal structure are rapidly integrated with non- linguistic information about the real-world to influence object state representations. Chapter 5 reports a visual-world eye-tracking study that investigates how different types of grammatical cues (grammatical aspect, verb type, argument realization patterns) 8 influence the real-time comprehension of transfer events (Experiment 5). This experiment looks at the representation of an object’s change-of-location. Results from Experiment 5 show that comprehenders use different grammatical cues to shape their representation of object location, as the sentence unfolds in real-time. 9 Chapter 2. Discourse-level information interacts with verb tense to influence object state representations 2.1. Introduction During communication, comprehenders build mental representations of the events described by sentences (e.g., Johnson-Laird, 1983; van Dijk & Kintsch, 1983), using information from the lexical semantics of individual words, grammatical markers such as tense and aspect, prosody, and many other sources. At the same time, comprehenders are also faced with the task of understanding how individual utterances – and the inferences drawn from them – contribute to the conversational goals and the broader discourse. This chapter investigates the construction of event representations during language processing, in particular how information from different levels of linguistic representation, namely (i) grammatically-encoded temporal properties (tense) and (ii) discourse-level conversational context, guides the event representations constructed by comprehenders. Mental representations of events have multiple dimensions, including temporal information, spatial information, and information about the relevant entities, including potential changes they may undergo. We focus on this third component, namely the mental representations of entities that get acted upon during the event (in linguistic terms, entities with the thematic role of ‘theme’ or ‘patient’). The representation of object states is a fundamental component of event representations as it is central to understanding the trajectory of changes that happen in an event (e.g., Altmann & Ekves, 2019; Altmann & Kamide, 2007; Hindy et al., 2012; Kang et al., 2020; Solomon et al., 2015). For example, successfully understanding events described by sentences like The woman broke the window or The man cleaned the shirt involves understanding that the window and the shirt undergo a 10 change-of-state from their initial states (intact window, dirty shirt) to result states (broken window, clean shirt). (The term ‘object state’ does not imply that the entity is an object in the syntactic argument-structure sense. There is an established tradition – which we follow – of using the term ‘objects’ to refer, broadly speaking, to ‘things’ in the real world.) Tracking object states during language processing – i.e., figuring out the changes that an entity may or may not undergo – is not a trivial task, as the compositional meaning of the sentence-level linguistic input is often underspecified. In particular, verbs differ in whether their meaning clearly specifies whether an object undergoes a change-of-state or whether their meaning leaves this underspecified. On the one hand, the meaning of verbs like break and clean specifies that the object undergoes a change-of-state. Following linguistic tradition (e.g., Rappaport Hovav & Levin, 1998), we call these result verbs. On the other hand, verbs like hit and wash describe the manner of the action but are underspecified regarding the result state. These are called manner verbs. Result verbs such as clean are complex events specified for a causing subevent and a result (change-of-state) subevent, as in (1). On the other hand, manner verbs such as wash are simple events as in (2), with no result subevent specified (e.g. Rappaport Hovav & Levin, 1998). (1) clean: [ [x ACT<MANNER> CAUSE [ BECOME [y <CLEAN>] ] ] (2) wash: [x ACT<WASH> y] More concretely, the result subevent encoded in result verbs like to clean specifies that the object ends up in a certain result state (e.g., the shirt ends up clean). In contrast, manner verbs like to wash only describe the manner in which the action is carried out and do not say anything about the result state, e.g., do not indicate whether the shirt ends up being 11 clean or not. For example, note that one can easily say The man washed the shirt but it still has stains on it. In other words, the linguistic meaning of manner verbs like wash is underspecified for whether the thing being acted on undergoes a change-of-state. What does this mean with respect to the mental models that comprehenders construct upon encountering linguistic event descriptions with manner verbs? After encountering a sentence like The man washed the shirt, do comprehenders represent the state of the shirt at the end of this event as dirty or clean? In other words, when the lexical semantics of manner verbs like wash do not include a result subevent, can the comprehenders’ mental event representation include a result subevent or not? In the current chapter, we examine how the information provided by verbs’ lexical semantics (e.g., manner verbs like wash) is augmented by two other kinds of information, namely (i) discourse-level information and (ii) temporal information, to guide comprehenders’ construction of object state representations. Ex.(3) is an example of how discourse-level information can guide object state representations: (3) Lisa: Do you think I can wear my blue shirt to my interview tomorrow? It had a big stain on it but I can’t remember if I did laundry. John: Don’t worry, I washed that shirt. Here, although the verb wash is underspecified about whether the thing being washed becomes clean or not, in this context we can easily infer from John’s utterance that the shirt is now clean. Otherwise, the sentence I washed that shirt would not be contributing to the overarching discourse goal of answering the question of whether the shirt can be worn for an interview. This intuition suggests that, at least on some level, discourse-level information 12 plays a role in guiding the mental representations of object states that comprehenders construct. In this chapter, we report an experiment that tests how (i) temporal information (verb tense) and (ii) discourse-level information (general information about which event participant – subject vs. object – is being talked about) interact to influence object state representations. Specifically, we test whether comprehenders’ representations of object states – i.e., whether an object has undergone a change-of-state (e.g., the shirt changing from dirty to clean) – are influenced by the interplay of discourse-level information and verb-level information, namely tense. Before outlining the research aims and hypotheses in more depth, we review relevant work on tense, verb semantics, and discourse-level representations in the next sections. 2.2 The role of verb tense in object state representations When the state of an object changes in an event, it changes over time. It is at its initial state before the event and ends up at its result state after the event. Therefore, it is natural to consider the temporal ordering of the described event when investigating the factors that can influence comprehenders’ mental representation of object states. One of the linguistic cues that provides information about the temporal ordering is tense. Tense (past, present, future) indicates the temporal ordering of events by temporally situating an event to precede, overlap with, or follow the time at which the sentence is uttered (e.g., Reichenbach, 1947). English marks tense on the verb using grammatical markers, e.g., Mary kicked the ball (past), Mary will kick the ball (future). Prior work in psycholinguistics shows that during language processing, representations of the initial state and the end state of an object can compete with one another (e.g., Hindy et al., 2012) and that the salience of these states in comprehenders’ mental 13 representations can be modulated by information about temporal ordering as indicated by tense (e.g., Altmann & Kamide, 2007; Kang et al., 2020; For related work on grammatical aspect, see e.g., Misersky et al., 2019). Altmann and Kamide (2007) showed that verb tense (past vs. future) rapidly modulates the extent to which participants expect the resulting object state. They conducted a visual-world eye-tracking experiment where participants heard sentences such as ‘The man will drink …’ and ‘The man has drunk …’. Even before hearing the object noun, participants launched more anticipatory looks to the full glass (the initial state) when hearing ‘will drink’, but looked more at the empty glass (the result state) upon hearing ‘has drunk.’ This shows that grammatical cues about the temporal ordering of events (with respect to the utterance time) influence the salience of each object state. Further evidence that verb tense guides object state representations comes from Kang et al. (2020), who showed that the effects of tense interact with general non-linguistic information about events and objects to modulate the availability of the initial state and the end state of the object. Using picture verification, Kang et al. tested how event representations are influenced by tense and by the degree of change described by a sentence (e.g., The woman chose / will choose the ice cream = minimal change to the ice cream, vs The woman dropped / will drop the ice cream = substantial change to the ice cream). They found that only past-tense sentences elicit responses showing sensitivity to degree of change: When the sentences were in past tense, participants were faster to verify the original state of the object (e.g., an upright ice cream) with minimal-change sentences than with substantial- change sentences. No such asymmetry was observed when the sentences were in future tense. These studies suggest that the mental representations of objects (and thus also of events) that comprehenders construct during language comprehension are guided by tense. Specifically, although both states are available in future tense contexts which signal that the event has not yet happened, the initial state can be more available to comprehenders than the 14 result state. Conversely, when past tense signals that the event has already happened, the end state can be more salient to comprehenders. In what follows, we refer to this finding that representations involving a changed object state are more salient with past than with future tense as the ‘past tense advantage’. These prior studies, however, used verbs and/or contexts where the nature of the past- tense sentences made it clear that the object must have undergone a change-of-state (e.g., after you hear ‘The man has drunk the beer,’ you know that the glass is empty). In such contexts, the difference between past and future tense is perfectly correlated with presence and absence of change-of-state (which was exactly what the experimenters intended): Either the object has not undergone a change-of-state (future tense) or it has undergone a change-of- state (past tense). But what about verbs such as wash that are semantically underspecified for whether the object undergoes a change-of-state or not? Even when wash is in past tense, it is not guaranteed that the shirt becomes clean. It is fine to say The man washed the shirt but it still had stains on it – a washing event in the past does not guarantee that the shirt has become clean. But despite this lack of a guarantee, it could still be that comprehenders are more likely to construct an event representation where the object has undergone a change-of-state when the verb is in past tense compared to when it is in future tense. Thus, the question is whether the past tense advantage for the change-of-state representation observed with result verbs like clean also exists with manner verbs like wash whose lexical semantics do not entail a change- of-state. It is not yet known whether and how tense modulates object state representations in events that are inherently underspecified for change-of-state. This is one of the key issues that we investigate in the present chapter. 15 2.3 How do object state representations go beyond verb meaning? The question of how verb tense interacts with the lexical semantics of verbs has implications for how we think about fundamental aspects of how mental representations of events are constructed during language processing. First, let’s note that (i) a linguistic event representation where an object undergoes a change-of-state is arguably semantically more complex than one where the object does not necessarily undergo a change-of-state (e.g., Beavers, 2006; Davis & Koenig, 2000; Dowty, 1991; Goldberg, 1995; Jackendoff, 1996; Rappaport Hovav & Levin, 1998) and that (ii) increased complexity in verb semantics is known to elicit a higher processing load (e.g., Gennari & Poeppel, 2003; McKoon & Love, 2011; McKoon & Macfarland, 2000, 2002; McKoon & Ratcliff, 2003, 2005, 2008). McKoon and Love (2011), in particular, show that processing times for hit verbs and sentences are shorter than those for break verbs and sentences. Given these two considerations, we might expect that comprehenders will opt for the most economical, minimalist approach and only ‘make the effort’ to build event representations involving change-of-state when faced with clear semantic evidence about a change-of-state, such as a verb that entails a change-of-state, like break/clean, but not a verb like hit/wash. Let’s call this the Semantic Entailment Hypothesis: (4) Semantic Entailment Hypothesis: Comprehenders construct representations where the object has undergone a change-of-state only when the verb semantics clearly specifies this (e.g., with result verbs like clean). In all other circumstances (e.g., with manner verbs like wash), comprehenders opt for the simpler representations without a change-of-state, regardless of other sentence-internal or external cues. 16 However, in light of prior work, we should also keep in mind the possibility that verb tense modulates the extent to which comprehenders are likely to consider an event representation involving change-of-state. In particular, it could be that the past tense advantage occurs not only with verbs that guarantee a change-of-state (e.g., clean), but also with verbs that do not guarantee a change-of-state (e.g., wash). Let’s call this the Tense-based Inference Hypothesis: (5) Tense-based Inference Hypothesis: When a sentence is in future tense, comprehenders focus more on the initial (unchanged) object state, and when a sentence is the past tense, comprehenders are relatively more likely to construct a representation where the object has undergone a change-of-state, regardless of the verb’s lexical semantics. Another possibility is that, in addition to verb-level factors like tense, discourse-level information can further modulate the extent to which a comprehender considers an event representation that involves object state change or not (ex.(3)). It could be that the extent to which comprehenders consider representations where the object undergoes a change-of-state depends not only on verb-level factors such as verb type and tense, but also interacts with the discourse context. This would yield an interaction between verb type/tense and discourse context. We call this the discourse-based inference hypothesis: (6) Discourse-based Inference Hypothesis: Discourse-level contextual information that is external to the sentence itself can interact with verb-level information (such as lexical semantics and tense) to further modulate the extent to which 17 comprehenders consider representations where the object has undergone a change- of-state. 2.4 Verbs differ in the information they provide about object state change: result verbs vs. manner verbs To test the three hypotheses outlined above, we first focus on comprehenders’ processing of manner verbs in Experiment 1. In Experiment 1, we keep verb class membership constant, to assess the interplay of discourse-level information and tense information on verbs. We then compare manner and result verbs in Experiment 2 to assess the interplay of discourse-level information and lexical semantics of verbs (Chapter 2). Earlier work on how verb tense influences object representations has tended to focus on scenarios where, due to the verb or real-world knowledge, past tense signals that an object change-of-state is virtually guaranteed – akin to result verbs. However, as we already saw with hit vs. break or wash vs. clean, individual verbs vary considerably in their lexical meaning – their lexical semantics – and not all verbs pattern alike with regard to encoding the object’s change-of-state. However, prior experimental work on object state representations has not systematically manipulated verbs’ lexical semantics. This may seem surprising, given that (i) in other areas of psycholinguistics, verb-related information is regarded as a key aspect of sentence processing (e.g., Garnsey, Pearlmutter, Meyers, & Lotocky, 1997; Trueswell, Tanenhaus, & Kello, 1993), and (ii) there is a long tradition of theoretical linguistics work that recognises the importance of verbs’ lexical semantics on object states in event representations and has identified systematic verb classes (e.g., Dowty, 1979; Fillmore, 1970; Rappaport Hovav & Levin, 1998; Vendler, 1957). 18 Since Fillmore’s (1970) seminal work on the grammatical differences between verbs of hitting and breaking, theoretical work on lexical semantics has identified a dichotomy between (a) verbs that entail change-of-state and (b) verbs that do not. The first class consists of verbs like break, clean, crack, fill, empty, melt, open, and shatter which describe situations with a clear result: the object has to undergo a change-of-state. In other words, the lexical semantic representation of these verbs includes a well-defined change-of-state that is entailed by the action. In this context, the notion of entailment is a logical dependency: if the action occurs, the object changes state. These verbs are called result verbs (e.g., Rappaport Hovav & Levin, 1998, 2010). Result verbs can be distinguished from another class of verbs called manner verbs. These are verbs like hit, wash, kick, pour, shake, shovel, slap, and wipe, which lexicalise the manner in which an action is carried out, but do not entail a result. For example, hit encodes the particular manner in which the agent comes in physical contact with the object that is acted upon, but does not entail that the object becomes broken (unlike the verb break). Thus, whereas use of a result verb reliably indicates to the comprehender that the object has undergone the relevant change-of-state (e.g., Lisa broke the window => window is broken), use of a manner verb fails to do so (e.g., Lisa hit the window => window may or may not break). However, although manner verbs do not linguistically encode a change-of-state, a change-of-state can often be inferred (e.g., Alexiadou, Martin, & Schäfer, 2017; Rappaport Hovav & Levin, 1998; Talmy, 1991, 2000; Wittek, 2002), as we saw in example (3) above. Examples (7a, 8a) also illustrate this: The change-of-state of the object (shirt becoming clean, window breaking) described in the second sentence follows naturally from the first sentence. In what follows, we often refer to this as the result state of the object. (Other researchers have also used the term end state. We consider the two terms to be interchangeable.) However, this 19 change-of-state is not semantically hard-wired into the meaning of the first sentence, as shown by the fact that (7b, 8b) are also natural. In other words, with manner verbs, the inference about a change-of-state occurring is defeasible (e.g., Rappaport Hovav & Levin, 1998). This is not the case with result verbs, where the change-of-state is entailed by the meaning of the verb itself, and thus cannot be denied. This is showed by the oddness (infelicity) of (9a, b). (7) a. Greg washed the shirt. He finally got the stains out! [manner verb] b. Greg washed the shirt but it is still dirty. (8) a. Mary hit the window. It shattered into a thousand pieces. [manner verb] b. Mary hit the window, but it didn’t break. (9) a. # John cleaned the shirt, but it is not clean. [result verb] b. # John shattered the window, but it didn’t break. [result verb] In sum, whereas result verbs semantically entail a change in object state, manner verbs are underspecified in this regard: a change in object state can often be inferred but is not encoded in the semantics of the verb. This brings us back to the fundamental question of how comprehenders build event representations, and specifically, what guides the representation of object states when the verb does not provide deterministic evidence. Do comprehenders pattern in accordance with the Semantic Entailment Hypothesis and only construct mental event representations involving a change-of-state of the object when forced to do so by the verb semantics? Or can other information – either in the discourse or the sentence itself – push comprehenders to construct an event representation where the object undergoes a change-of-state, as predicted by the Tense-based Inference Hypothesis and the Discourse-based Inference Hypothesis respectively? To investigate these 20 issues, we need to move beyond contexts where the verb entails a change-of-state and look at semantically more ambiguous contexts. 2.5 Does discourse-level information interact with verb-level information to guide the representation of object state change? As mentioned above, prior experimental work, e.g., Altmann and Kamide (2007) and Kang et al. (2020), focused on verbs/events that are associated with an obvious result state, akin to result verbs. Altmann and Kamide (2007) used destruction verbs (e.g., drink, eat) where the object disappears at the end of the action. Kang et al. (2020)’s substantial-change event descriptions also necessarily involve a change of the object’s state (e.g., The woman dropped the ice cream.) Thus, their findings provide information about how comprehenders use the result state information inherent in the event together with tense cues, but do not shed light on what happens in situations where the linguistic input is underspecified about whether or not the object undergoes a change-of-state. Their results are compatible with the Semantic Entailment Hypothesis, but do not directly speak to the Tense-based or Discourse-based Inference Hypotheses. This brings us to one of the key questions that we explore in the present work: In situations where the verb does not pre-specify whether the object undergoes a change-of- state, what kinds of information guide the object state representations constructed by comprehenders? The intended meaning of an underspecified event description may be context-sensitive, as we saw in the exchange in (1). In order to investigate how comprehenders understand events based on underspecified event descriptions, we examine the discourse-level factors that affect this process, and how they interact with linguistic information encoded on the verb itself, namely tense marking (past vs. future) and the verb’s 21 lexical semantics. In doing so, we assess the three hypotheses outlined above, namely the Semantic Entailment Hypothesis, the Tense-based Inference Hypothesis, and the Discourse- based Inference Hypothesis. We frame our discussion of discourse factors within the framework of Questions- Under-Discussion (QUD). We use the QUD approach because it allows us to articulate our predictions in a precise way, as it is well-established in theoretical work (e.g., Roberts 1996/2012) – and has proven to be very fruitful for linguistic theorizing (see e.g., Beaver & Clark, 2008; Beaver, & Roberts, 2010; Büring, 2003; Schoubye, 2010; Simons, Tonhauser, Onea, 2016; Umbach, 2005) – and because it is supported by a substantial number of experimental studies (e.g., Clifton & Frazier, 2012, 2018; Cummins & Rohde, 2015; Degen & Goodman, 2014; Delogu, Jachmann, Staudte, Vespignani, & Molinaro, 2020; Grant, Clifton, & Frazier, 2012; Kehler & Rohde, 2017; Tian, Breheny, & Ferguson, 2010; Zondervan, Meroni, & Gualmini, 2008; Zondervan, 2009, 2010). However, it is important to note at the outset that the validity of our claims does not rely on the specific notion of QUDs; a different discourse-based approach could, in principle, also be used. We chose to use the notion of QUDs because they are theoretically and empirically well-understood. The core idea of the QUD approach is that discourse is structured around questions- under-discussion which represent the interlocutors’ joint discourse goals – the aims/goals of the current communicative exchange. QUDs are often implicit, i.e., often they are not explicitly worded as questions, and can be introduced by means of various cues (see e.g., Roberts, 1996/2012, see also Carlson, 1983). Under the QUD approach, a felicitous utterance is one that is relevant to the current QUD(s) and thus the interpretation of a sentence may depend on the QUD(s) that it addresses. For example, in (1), one relevant QUD is ‘Can the shirt be worn for the interview?’ which then leads to the sub-question ‘Is the shirt clean?’ In 22 order for John’s utterance “I washed that shirt” to be relevant to the question of whether the shirt is clean or not, an inference may be drawn: ‘I washed that shirt, so it is now clean.’ A growing body of psycholinguistic evidence suggests that QUDs play a significant role in guiding language processing (e.g., Clifton & Frazier, 2012, 2018; Cummins & Rohde, 2015; Degen & Goodman, 2014; Delogu et al., 2020; Grant, Clifton, & Frazier, 2012; Kehler & Rohde, 2017; Tian, Breheny, & Ferguson, 2010; Zondervan, Meroni, & Gualmini, 2008; Zondervan, 2009, 2010). These findings point to the conclusion that when processing an utterance, comprehenders prefer to interpret it as being relevant to the QUD, i.e., contributing to the current communicative goals of the discourse. This idea is captured in Clifton and Frazier’s (2018) General QUD Processing Principle, which states that comprehenders “preferentially analyze new material such that it comments on the QUD” (p.109). The current work builds on and expands this literature by investigating a novel domain in which QUDs can guide interpretation: the representation of object states during event comprehension. 2.6 Aims of this work Experiment 2 investigates how a key aspect of event representations, namely object state representations, are influenced by information from verb-level grammatical factors (verb tense), and discourse-level information (which we conceptualise in terms of the QUD framework). We test three hypotheses about whether and how grammatical and discourse- level information guide comprehenders’ construction of object states: the Semantic Entailment Hypothesis, the Tense-based Inference Hypothesis and the Discourse-based Inference Hypothesis (as defined in (4)-(6)). We conducted a lexical decision experiment that examine how rapidly comprehenders process linguistic material associated with potential change-of-state inferences, in contexts 23 where the preceding discourse context and verb-level information are manipulated. Experiment 1 (using lexical decision) investigates whether verb tense (past vs. future) and discourse-level information modulate the object state representations that participants construct based on manner verbs, which are verbs that do not semantically specify whether or not the object has undergone a change-of-state. Both lexical decision (and self-paced reading: Experiment 2) methods have been used in various studies to investigate comprehenders’ mental representations in many linguistic domains, including issues related to event representation (e.g., Allbritton, 2004 on predictive inferences; Ferretti et al., 2001 and subsequent work on activation of event-related knowledge; Fincher‐Kiefer, 1993; Magliano et al., 1993; Pickering et al., 2006). We build on prior literature showing that it is easier to process words that relate to expectations comprehenders have already constructed, relative to unexpected words (e.g. expectation- based models of language processing and the notion of surprisal, e.g. Hale, 2001; Levy, 2008). Thus, we predict that, in contexts where comprehenders have already constructed (based on prior context) an event representation where the object has undergone a change-of- state, then words related to the change-of-state will be easier to process – recognised faster – than in contexts where the event representation that the participant has constructed so far does not include a change-of-state for the object. 2.7 Experiment 1 Experiment 1 investigates manner-verb event descriptions which are underspecified for change-of-state, to see how verb tense and discourse-level cues guide how people construct representations of object states. Consider the sentence ‘Arthur poked the balloon.’ 24 Poke is a manner verb and does not specify whether or not the balloon undergoes a change- of-state (such as getting popped). What kind of representation do comprehenders construct upon encountering this kind of sentence? The Semantic Entailment Hypothesis predicts that participants do not construct a representation where the object has undergone a change-of- state. Why bother to construct a more complex event representation that involves a change- of-state, unless the verb demands this? In contrast, the Tense-based Inference Hypothesis and the Discourse-based Inference Hypothesis predict that tense and discourse context can push comprehenders to construct event representations where the object does undergo a change-of- state, even with manner verbs. We used a lexical decision task to test whether (i) verb tense (past vs. future) and (ii) discourse-level cues (subject-oriented vs. object-oriented QUDs) modulate the extent to which participants consider a change-of-state representation for the object. The critical transitive sentences (e.g., Arthur poked / will poke the balloon) were (i) preceded by context clauses that marked either the subject or the object as the focus of the current QUD (Talking about Arthur / Talking about the balloon) and were (ii) followed by a lexical decision task with target words associated with the potential change-of-state (e.g., popped). We assume lexical decision RTs indicate how strongly a comprehender is considering an event representation where the object undergoes a change-of-state, based on prior context (faster RTs indicate more activation of a representation where the object has undergone a change-of- state). 25 2.7.1 Methods 2.7.1.1 Participants Adult native speakers of American English were recruited via Amazon’s Mechanical Turk (MTurk). Participants were compensated for their participation. We only included self- reported US-born native English speakers. Furthermore, we excluded 34 participants 1 who performed poorly on either the lexical decision task (below 75% accuracy, mean accuracy of excluded participants=54.48%, mean accuracy of included participants=96.43%) or on comprehension questions that occurred after each item (below 75% accuracy, mean accuracy of excluded participants=53.77%, mean accuracy of included participants=89.65%). All exclusion criteria reported in this paper were determined before data analysis on the targets was conducted. After excluding these participants, 102 participants were included in the final analysis. All experiments reported in this paper were reviewed and approved by the USC Institutional Review Board. 2.7.1.2 Design and Materials Each item had two main components: A two-line text component (shown on one screen) and a single-word lexical decision component (shown on the next screen). An example is given in (10) and (11). In the text component, we manipulated (i) the QUD- introducing context clause that precedes the critical sentence and (ii) the tense of the critical sentence (2x2 within-subjects design). The study included 24 target items. See Appendix A for the target stimuli. (10) Sample text component: 26 a. Subject-related QUD + Future tense Talking about Arthur: “Arthur will poke the balloon.” b. Subject-related QUD + Past tense Talking about Arthur: “Arthur poked the balloon.” c. Object-related QUD + Future tense Talking about the balloon: “Arthur will poke the balloon.” d. Object-related QUD + Past tense Talking about the balloon: “Arthur poked the balloon.” (11) Sample lexical decision word: popped Text component. The first line of the text component consisted of a context clause that signaled that the QUD (which reflects the immediate topic of discussion) is either about the subject or object of the following sentence (e.g., Talking about {Arthur/the balloon}:). We chose to use the “Talking about X” frame because “about”-phrases provide a means to manipulate attention to either the subject or the object without providing additional (and potentially asymmetric or biasing) information about other aspects of the event or the context. Other studies have used “about”-phrases to effectively manipulate the discourse context: mentioning a referent in an “about”-phrase influences the information structural status of the referent (e.g., Burmester et al., 2014; Burmester et al., 2018; Cowles, 2007; Cowles & Ferreira, 2012). Based on Clifton and Frazier’s (2018) General QUD Processing Principle and attention allocation in discourse (e.g., Cutler & Fodor, 1979; Birch & Rayner, 27 1997; Sturt et al., 2004), we assume that the QUD manipulation modulates whether participants will focus more on the subject (Subject-QUD conditions) vs. the object (Object- QUD conditions) as the answer to the QUD. The second line was the critical sentence. All targets used manner verbs – in past or future tense – that refer to events of contact-by-impact which may or may not cause the object to change its state (e.g., whack, pound, kick, hit, knock, tap, poke), identified based on Levin (1993) as well as established semantic tests. The object nouns had no strong bias about whether the event would or would not lead to a change-of-state of the object. The study had 24 targets, and each target used a different manner verb and a different noun. Lexical decision word (target word). The text component of each trial was followed by a single word on the next screen (e.g., popped). Within an item, the target word was held constant. Participants indicated whether this word was a real word of English (lexical decision). Reaction times were recorded and analysed. On target trials, the lexical decision word (e.g., popped, cracked, squashed) was a past participial form of a result verb associated with a potential change-of-state of the object. These forms are associated with object change-of-state interpretations relative to both the subject and the object of the critical sentence. If a participant reads (10) and constructs an event representation where the object undergoes a change-of-state (e.g., the balloon pops), then ‘popped’ should be easier to process regardless of whether it is construed as (i) an active form related to the subject (Arthur has popped the balloon) or (ii) a passive form related to the object (verbal passive: The balloon was/got popped, adjectival passive: the popped balloon), 2 compared to a situation where the participant constructs an event representation with no object state change. Thus, the lexical decision target words provide a maximally 28 sensitive way of tapping into whether participants’ event representations in the different conditions differ in terms of how strongly an object change-of-state is activated. We did not test target words unrelated to change-of-state; our hypotheses and results are cast in relative terms and focus on comparing the conditions to each other. In other words, our design allows us to test whether the four conditions differ in terms of the extent to which participants consider a representation where the object undergoes a change-of-state. In addition to 24 targets, the study included 36 fillers. Fillers also consisted of a context clause (Talking about …) and a sentence that describes a transitive event in future or past tense. On 26 of the 36 filler trials, the lexical decision word was a nonce word with adjectival morphology (e.g., vulperous, lindful) and on the remaining 10 filler trials, the lexical-decision word was a real word (e.g., edible, magenta, excited) that was not related to the text component associated with it. As this study had a 2x2 design, four lists were created and presented to participants using a standard Latin Square design: each participant was presented with only one condition of each target item and each of the four conditions appeared the same number of times on any given list. Each of the four lists contained the same set of filler items, pseudo-randomly distributed throughout the list. 2.7.1.3 Procedure The experiment was hosted online on PennController IBEX (Zehr & Schwarz, 2018; https://www.pcibex.net/), and participants did it remotely via the internet. Participants completed three practice trials before the start of the main experiment. Each trial began with a presentation of the two-line text component (see (10)). Participants were instructed to read both lines and then press the spacebar to advance. The next screen displayed a fixation cross for 1000ms, which was then replaced by the lexical decision word. Participants indicated 29 whether this target word was a real word of English by pressing F (word) or J (non-word). The target word remained on the screen until a button press was registered. Afterwards, participants answered a yes-no comprehension question about the text component. When participants made errors either on the lexical decision task or on the comprehension questions, they saw an error feedback message. The experiment session lasted approximately 20 minutes. 2.7.2 Predictions If participants only construct event representations where the object has undergone a change-of-state when the verb semantically entails this, as predicted by the Semantic Entailment Hypothesis, we expect no effects of tense or QUD on lexical decision times (given that Experiment 1 tests manner verbs which do not entail a change-of-state). Absence of any tense or QUD effects would be compatible with the claim that regardless of tense or discourse context, participants do not construct a change-of-state representation if the verb semantics do not entail it. (We test result verbs in Experiment 2). In contrast, the Tense-based Inference Hypothesis predicts that tense guides the construction of object state representations, even with manner verbs. According to this hypothesis, with a future tense verb, comprehenders focus more on the initial (unchanged) object state, but with a past tense verb, comprehenders are more likely to construct a representation where the object has undergone a change-of-state, regardless of the verb’s lexical semantics. So, if participants make the tense-based inference that a past action leads to a result state, we expect to see a past tense advantage in the lexical decision response times (faster RTs in past tense conditions than future tense conditions). 30 Furthermore, if effects of verb-level information (in this case tense) on object state representations are modulated by discourse-level information, as predicted by the Discourse- based Inference Hypothesis, we expect an interaction between verb tense and discourse-level (QUD) effects: We predict that tense effects will be stronger when the focus of attention is on the object (Object-QUD conditions), compared to the subject (Subject-QUD conditions). This predicted interaction is rooted in the observation that in transitive sentences of the type we are testing, tense affects the representation of objects more than subjects: It is the objects that undergo the change-of-state. The representation of Arthur is largely the same before and after the poking-the-balloon event, but the state of the balloon is potentially very different after the poking event (past tense conditions). Thus, if discourse-level information interacts with verb tense in guiding the construction of object state representations, we predict an interaction between tense and QUD. If we find effects of verb tense but no effects of the QUD manipulation, this would be compatible with the Tense-based Inference Hypothesis but not the Discourse-based Inference Hypothesis. It would support a view where only core aspects of grammar (e.g., verb tense), but not discourse-level information, shape comprehenders’ construction of object state representations. In addition to an interaction between tense and QUD type, we may also find a main effect of QUD type, which would be orthogonal to the hypotheses we are testing. A main effect of QUD type would be related to the fact that our target (lexical decision) words are past participial forms (e.g., popped), which are ambiguous between (i) active (e.g., John (has) popped …) and (ii) passive forms (e.g., The balloon is/was popped). We chose to use these ambiguous forms because they can be associated with change-of-state event interpretations relative to both the subject and the object of the critical sentence – an active interpretation being subject-oriented and a passive interpretation being object-oriented. Thus, these forms 31 provide a maximally sensitive way of tapping into whether participants’ event representations in the different conditions differ in terms of how strongly a change-of-state representation is activated (tense x QUD interaction). We may also find – for reasons orthogonal to our research aims –that subject-oriented QUDs make it more likely that participants will interpret the target word as an active form, while object-oriented QUDs make it more likely that participants interpret it as a passive form. Given that actives are much more frequent in English than passives (e.g., Hopper & Thompson, 1980; Svartvik, 1966), and that lexical decision RTs are sensitive to word frequency (e.g., Whaley, 1978), this may elicit a main effect of QUD type such that subject QUDs elicit faster lexical decision RTs to popped than object QUDs. However, a main effect of QUD type is not relevant for the hypotheses we are testing in this paper. 2.7.3 Likelihood-of-change Norming Study The targets in Experiment 1 used a range of verbs and nouns in the critical sentences and as the target words in the lexical decision task. To control for differences between items, we ran a norming study to obtain an independent measure of how likely it is that the verb + noun pairs used in target items (e.g., poke + the balloon) make people expect that the noun undergoes the change-of-state described by the target word in the lexical decision task (e.g., popped). We computed the likelihood-of-change measure for each item based on norming data from 35 adult native speakers of American English. (One participant who answered incorrectly on four out of the five attention-check trials was excluded. None of the other participants made more than one error.) Participants saw 24 target items like (12) and rated the likelihood of the object undergoing the change-of-state. The study also included five 32 attention-check trials involving recall of a word from the preceding screen. The mean likelihood-of-change rating for all twenty-four items was 4.78 (sd=1.79). The ratings were z- scored, and we used the z-scored ratings as a fixed effect in the reaction time analyses in order to control for item-level variation (see Results section below). (12) [POKE – the balloon] / popped If you imagine a situation that is related to [POKE – the balloon], how likely are you to imagine that the balloon gets popped? Not likely at all extremely likely 1 2 3 4 5 6 7 2.7.4 Data Processing and Analysis When analysing the reaction times in the lexical decision task, we first removed incorrect lexical decision responses (2.67% of the data) and RTs over 5000ms (2.1% of the data), following Baayen and Milin (2010). Then, RTs more than 2.5 SDs from a participant’s mean RT were excluded. This affected 2.71% of the data. For statistical analyses, we used linear mixed effects models with RT as the dependent variable, and tense (contrast-coded, future tense=0.5, past tense=-0.5), QUD type (contrast-coded, Object QUD=0.5, Subject QUD=-0.5), and the tense x QUD type interaction as fixed effects. Models were estimated using the lme4 package (version 1.1.21) (Bates et al., 2015) and lmertest (version 3.1.1) (Kuznetsova et al., 2017) in the R software environment (R Development Core Team, 2019). We used the same R packages for planned comparisons. Our models also included word frequency (word frequency per million words from the SubtlexUS database, Brysbaert & New, 2009), word length, likelihood-of-result scores (in z-scores) and presentation order as 33 fixed effects. As random effects, we entered intercepts for subjects and items, as well as by- subject and by-item random slopes for the effects of tense type, QUD type, and their interaction (Experiment 1) when justified by model comparison: Random effects started out fully crossed and fully specified with by-subject and by-item effects of tense type, QUD type, and their interaction. They were then reduced (starting with by-item effects) via model comparison, wherein only random effects that contributed significantly to the model (p < 0.05) were included (Baayen et al., 2008). 2.7.5 Results Figure 2.1 shows the average lexical decision RTs by condition, i.e., how quickly participants correctly identified the target word as a real word of English. Two numerical patterns are clearly visible: First, overall RTs in the Subject-QUD conditions are faster than in the Object-QUD conditions. Second, tense has no effect in the Subject-QUD conditions whereas there is a past tense advantage (or future-tense penalty) in the Object-QUD conditions: RTs are slower in the future tense conditions than in the past tense conditions. The main effect of QUD type (RTs in Subject-QUD conditions being faster than in Object-QUD conditions) is not relevant to the claims we are making in this paper. We attribute it to the (intended) active/passive ambiguity of the target word and the fact that active verb forms are much more frequent than passive verb forms (see the Predictions section above). Statistical analyses are reported in Tables 2.1-2.3. As can be seen in Table 2.1, we found a main effect of QUD type, a marginal main effect of tense, but no interaction. However, given that we predicted differential effects of tense in Subject-QUD and Object- QUD conditions, we conducted planned comparisons to test this. Thus, we looked separately 34 at Object-QUD conditions (Table 2.2) and Subject-QUD conditions (Table 2.3), to test our hypothesis that tense effects will be modulated by QUD type (the Discourse-based Inference Hypothesis). As expected based on Figure 2.1, there is a significant effect of tense in the Object-QUD conditions but not in the Subject-QUD conditions: When the QUD inquired about the object, RTs in the past tense condition are faster than in the future tense condition (Table 2.2). In contrast, when the QUD inquired about the subject, this tense effect was not observed (Table 2.3). These results support the Tense-based and the Discourse-based Inference hypotheses. Figure 2.1. Mean reaction times by condition in Experiment 1 (The y-axis shows the raw reaction times to the lexical decision task in milliseconds (ms). Error bars show +/-1 SE) Table 2.1. Experiment 1: Results of the lmer model β SE df t value Pr (>|t|) (Intercept) 1303.2204 95.2597 118.4234 13.681 < 2e-16 *** QUD type 35.5868 11.6873 2156.1279 3.045 0.00236 ** Tense type 22.2045 11.6838 2153.3115 1.900 0.05751 . likelihood-of-result score 12.5101 14.3771 20.0479 0.870 0.39452 35 word length -5.6787 6.1711 19.6034 -0.920 0.36865 word frequency -0.4245 0.1797 19.6837 -2.363 0.02857 * presentation order -6.1973 0.8507 2156.1558 -7.285 4.48e- 13 *** QUD type:Tense type 26.9589 23.3806 2157.4864 1.153 0.24902 Table 2.2. Experiment 1: Planned comparisons, Object-QUD conditions only β SE df t value Pr (>|t|) (Intercept) 1282.6470 111.1086 87.2899 11.544 < 2e-16 *** Tense type 36.5302 16.8896 1017.1898 2.163 0.0308 * likelihood-of-result score 3.2456 21.4307 18.5734 0.151 0.8813 word length 1.0092 9.3177 18.8055 0.108 0.9149 word frequency -0.3547 0.2808 21.1967 -1.263 0.2203 * presentation order -6.8462 1.2285 1017.8299 -5.573 3.21e- 08 *** Table 2.3. Experiment 1: Planned comparisons, Subject-QUD conditions only β SE df t value Pr (>|t|) (Intercept) 1341.6957 96.3847 115.6697 13.920 < 2e-16 *** Tense type 8.2611 15.8885 1018.6536 0.520 0.6032 likelihood-of-result score 23.4310 15.7977 20.4959 1.483 0.1532 word length -14.6623 6.9554 21.4070 -2.108 0.0470 * word frequency -0.5348 0.2025 20.6478 -2.641 0.0154 * presentation order -5.7106 1.1636 1021.6290 -4.908 1.07e- 06 *** 2.8 Discussion In order to gain insights into what kinds of information guide the representation of events, in particular object state changes, Experiment 1 tested the processing of transitive sentences with manner verbs which are underspecified with respect to whether or not the object changes state (e.g., Arthur poked the balloon may or may not result in the balloon popping). This study used a lexical decision task to investigate the effects of (i) discourse- level information (specifically, which event participant the contextual Question Under Discussion (QUD) is related to) and (ii) verb tense (future vs. past) on how quickly comprehenders process words related to the potential change-of-state. 36 We tested three hypotheses about how different kinds of information guide the representation of object states. According to the Semantic Entailment Hypothesis, participants only construct event representations with a change-of-state of the object if the verb semantically entails a change-of-state. Since Experiment 1 tested manner verbs, this hypothesis predicts that no change-of-state representation is constructed in any condition, regardless of tense or QUD. According to the Tense-based Inference Hypothesis, tense guides the construction of object state representations, even with manner verbs. Furthermore, if verb-level effects (in this case verb tense) can interact with discourse-level information, as predicted by the Discourse-based Inference Hypothesis, a stronger past tense advantage is predicted with Object-QUDs than with Subject-QUDs. Our results go against the Semantic Entailment Hypothesis, because we find that tense does have an effect on the lexical decisions times for the target words associated with the potential change-of-state. Indeed, we find a past tense advantage in the Object-QUD conditions but not in the Subject-QUD conditions. We interpret our results as suggesting that, when the QUD drives people to attend to the object (by introducing an expectation that the sentence is interpreted as providing an answer to a question about the object), tense modulates the object state representations and makes a change-of-state inference more likely with past tense verbs than with future tense verbs. This is in line with the predictions of the Tense-based and Discourse-based Inference hypotheses. These findings suggest that the mental representations of events built from linguistic input can go beyond the lexical semantics of verbs. In doing so, other verb-level factors such as tense and discourse-level factors such as the QUD can play a modulating role. Thus, our results highlight the interplay between grammar-level and discourse-level information and support the idea that the mental representations of events are modulated by the QUD (i.e., what is being discussed in the current discourse). This finding is in line with 37 Clifton and Frazier’s (2018) General QUD Processing Principle which states that utterance interpretations are QUD-dependent. A possible concern is whether the past tense advantage that we observed in the Object-QUD conditions could be due to a morphological priming process driven by the presence of the -ed marker in both the critical sentence and in the target word. However, this is not a plausible explanation for our data: It fails to explain the lack of a past tense advantage in the Subject-QUD conditions. If the past tense advantage were only reflective of a morphological priming effect, the same effect should be observed across-the-board, regardless of the QUD manipulation. But this is clearly not the case. In sum, the results of Experiment 1 extend the previous findings on tense effects (e.g., Altmann & Kamide, 2007; Kang et al., 2020) by providing evidence that the past tense advantage also holds for verbs that do not specify a result state and for which the result state can only be inferred. Thus, our results support the Tense-based Inference Hypothesis and go against the Semantic Entailment Hypothesis. Crucially, our results also show that in addition to sentence-level information, discourse-level information interacts with verb tense to contribute to the cognitive process of understanding the dynamics of event representation in language comprehension: We find evidence for both the Discourse-based and the Tense- based Inference hypotheses. 38 Chapter 3. Discourse-level information interacts with verb semantics to influence object state representations 3.1 Introduction In Experiment 2 in Chapter 2, we found that the ease of processing words associated with a potential change-of-state of the object is influenced by discourse-level information in conjunction with verb-level temporal information (past vs. future tense). These results suggest that in discourse contexts with object-oriented QUDs, the inference about the object having changed state – with manner verbs which are underspecified for change-of-state – is more available when the event description is in past tense than in future tense. The finding that both verb-based and discourse-based information play a role provides initial evidence against the Semantic Entailment Hypothesis ((4) in Chapter 2, repeated below as (1)), according to which only the lexical semantics of verbs matters for the purposes of representing the object state. (1) Semantic Entailment Hypothesis: Comprehenders construct representations where the object has undergone a change-of-state only when the verb semantics clearly specifies this (e.g., with result verbs like clean). In all other circumstances (e.g., with manner verbs like wash), comprehenders opt for the simpler representations without a change-of-state, regardless of other sentence-internal or external cues. Experiment 2 uses self-paced reading methodology and has two main aims. First, it tests the semantic entailment hypothesis more directly, by comparing result verbs and manner verbs. Even though the results of Experiment 1 challenge the Semantic Entailment 39 Hypothesis, they do so in a ‘one-sided’ way because only manner verbs were tested. To fully test the hypothesis that comprehenders construct representations involving an object change- of-state only when the verb semantics entails such a change, we need to directly compare verbs that entail a change-of-state (result verbs) to verbs that do not (manner verbs). Prior experimental work on object state representations has not systematically manipulated verbs’ lexical semantics. This may seem surprising, given that (i) in other areas of psycholinguistics, verb-related information is regarded as a key aspect of sentence processing (e.g., Garnsey, Pearlmutter, Meyers, & Lotocky, 1997; Trueswell, Tanenhaus, & Kello, 1993), and (ii) there is a long tradition of theoretical linguistics work that recognises the importance of verbs’ lexical semantics on object states in event representations and has identified systematic verb classes (e.g., Dowty, 1979; Fillmore, 1970; Rappaport Hovav & Levin, 1998; Vendler, 1957). Second, Experiment 2 also takes a closer look at the Discourse-based Inference Hypothesis ((6) in Chapter 2, repeated below as (2)), to see if a finer-grained discourse-level manipulation has effects on object state representations – especially with manner verbs. (2) Discourse-based Inference Hypothesis: Discourse-level contextual information that is external to the sentence itself can interact with verb-level information (such as lexical semantics and tense) to further modulate the extent to which comprehenders consider representations where the object has undergone a change-of-state. Whereas Experiment 1 tested a coarse-grained split of object-related vs. subject- related QUDs, in Experiment 2 we tested two kinds of object-related QUDs: We compared contexts with aboutness QUDs that simply ask about the object (e.g., Trevor asked about the 40 X, similar to Experiment 1) to contexts with change-of-state oriented QUDs that specifically ask what happened to the object (e.g., Trevor asked what happened to the X). A difference between aboutness QUDs and change-of-state oriented QUDs would provide evidence that the construction of event representations is sensitive not only to QUD- based distinctions between event participants (e.g., subject vs. object, Experiment 1) but also to the presence of fine-grained information in QUDs about event structure (Experiment 2). In fact, given that result verbs semantically entail an object state change, we expect to see stronger QUD effects with manner verbs (verb type x QUD type interaction). We are especially interested to see whether change-of-state oriented QUDs can facilitate the processing of change-of-state descriptions after manner verbs so as to render them comparable to the processing of change-of-state descriptions after result verbs. Such a finding would both provide further evidence for the Discourse-based Inference Hypothesis – by showing that effects of verbs’ lexical semantics can be modulated by discourse-level information – and against the Semantic Entailment Hypothesis. Experiment 2 used past tense sentences in all targets; thus, it does not test the Tense- based Inference Hypothesis. Experiment 2 used self-paced reading, instead of lexical decision. This is because self-paced reading allowed us construct contexts that incorporate aboutness QUDs and change-of-state oriented QUDs in a more coherent way. Furthermore, self-paced reading allows us to measure RTs not only on the target word but also on subsequent words. 3.2 Experiment 2 3.2.1 Methods 41 3.2.1.1 Participants Students from the University of Southern California participated in return for course credit. We report data from 40 adult native English speakers with normal or corrected-to- normal vision and no reported reading or learning disabilities. One person was excluded due to dyslexia and one due to low performance on comprehension questions (only 70% correct; mean comprehension question accuracy of other participants=82.05%, SD=0.38). 3.2.1.2 Design and Materials In a 2x2 design, we manipulated (i) QUD type (what happened to X vs. about X) and (ii) verb type (manner verb vs. result verb) to create four conditions. All targets used nonce object nouns (e.g., merick) because we wanted to avoid object properties affecting result state representations (stomp on an egg vs. stomp on a penny, e.g., Hindy et al., 2012; see also Horchak & Garrido, 2020 on the effect of the object being affected by light vs. heavy items). Each target used a different nonce word. An example is in (3). See Appendix B for the full list of target stimuli. (3) Sample target stimuli a. what happened QUD + Manner verb Trevor called and asked Mary what happened to the merick. She replied that she hit it in the morning on Monday. She said that it is damaged and that she feels very sorry about this. b. about QUD + Manner verb Trevor called and asked Mary about the merick. She replied that she hit it in the morning on Monday. She said that it is damaged and that she feels very sorry about this. 42 c. what happened QUD + Result verb Trevor called and asked Mary what happened to the merick. She replied that she broke it in the morning on Monday. She said that it is damaged and that she feels very sorry about this. d. about QUD + Result verb Trevor called and asked Mary about the merick. She replied that she broke it in the morning on Monday. She said that it is damaged and that she feels very sorry about this. All targets were three sentences long, and each sentence was presented on a separate line. All sentences were presented word-by-word. The first sentence mentions two different- gender people using proper names and establishes the QUD. The about QUD is similar in effect to the Object-related QUDs in Experiment 1: it generally asks about the object but does not signal that the speaker is specifically interested in knowing about the result state of the object. In contrast, the what happened QUD indicates that the speaker wants to know about the result state of the object. Based on Clifton and Frazier’s General QUD Processing Principle, which states that comprehenders prefer to interpret utterances so that they relate to the QUD, we assume that when the QUD asks what happened to the object, a change-of-state inference about the object is more available than in the context of aboutness QUDs. In other words, in the what happened QUD + manner verb condition, in order for the second sentence to be interpreted as an answer to the QUD, the change-of-state inference is to be drawn. If the manner verb is interpreted purely as its lexical semantic meaning of providing information about the manner in which the action was carried out, the sentence does not serve to meet the discourse goals. 43 The second sentence provides an answer to the QUD, using a manner verb (e.g., hit) or a result verb (e.g., break). All verbs were in the past tense. We used 33 different result verbs (one was used twice) and 32 different manner verbs (two were used twice), selected based on Levin (1993). The third sentence starts with the structure ‘he/she replied that…’ and uses the target word in predicative position as shown in (3). The target word describes the (changed) result state of the object noun (e.g., damaged). Most of the target words are past participial adjectives (e.g., damaged, shattered, cracked), like Experiment 1, while some are non-past participial adjectives and related expressions (e.g., shiny, rough). All target words describe the changed state of the object. We used 25 different target words (7 were used twice and 1 was used three times.) The target word is followed by a coordinated ‘that’ clause with a pronominal subject with the structure and that he/she feels quite/very/rather…. In the result verb conditions, the changed state is entailed by the verb. In the manner verb conditions, it is not entailed, but can be inferred. Therefore, successful integration of the result state into the existing event representation depends on the generation of the change-of- state inference. The critical region for the RT analysis consists of the target word and the five subsequent words. For completeness, we report RTs for the entire critical sentence, i.e., also for the two words before the adjective (it is), although these are not relevant for our predictions because the critical adjective has not yet been encountered at that point. In addition to 34 targets, the experiment included 4 practice trials and 48 fillers. The study had 32 ‘core’ target items as part of the 2x2 design, as well as two ‘extra’ target items, for a total of 34. As in Experiment 1, four lists were created using the core targets and presented to participants using a standard Latin Square design: each participant was presented with only one condition of each target item and each of the four conditions appeared the same 44 number of times on any given list. Each of the four lists contained the same set of filler items, pseudo-randomly distributed throughout the list. The two extra targets were not integrated fully into the Latin Square because of potential semantic concerns (they differ from the others in using a potentially non-gradable target word: spotless, shiny). However, it turns out that the basic RT patterns with and without these items are the same, so we include them in our analyses. Thus, each participant saw either eight or nine target items per condition. 3.2.2 Procedure Participants were tested individually in the Language Processing Lab on the University of Southern California campus. We used a moving-window self-paced reading paradigm implemented with Linger (D. Rohde, http://tedlab.mit.edu/dr/Linger/) on an HP Spectre X360 laptop computer, running Windows 10. Participants read the sentences one word at a time. With each press of the spacebar, the currently displayed word turned back into dashes and the next word was displayed. Each trial was followed by a yes-no comprehension question which participants answered with the F (yes) or J (no) keys. Incorrect responses triggered an error feedback message. Comprehension questions were related to the content of a balanced range of sentence regions. Each experimental session lasted approximately 20 to 25 minutes. 3.2.3 Data Processing and Analysis Prior to data analysis, RTs faster than 100ms or slower than 2000ms were excluded, which affected 1.77% of the data. We also excluded any RTs more than 2.5 standard 45 deviations from the mean in any given word position. This affected an additional 2.26% of the entire data. We analysed the target word (e.g., damaged) and the five words following it. Statistical analyses were carried out on the raw RT data using linear mixed-effects models. Models were estimated using the lme4 package (version 1.1.21) (Bates et al., 2015) and lmerTest (version 3.1.1) (Kuznetsova et al., 2017) in the R software environment (R Development Core Team, 2019). We used the same R package for planned comparisons. The models included fixed effects of verb type (contrast-coded: manner verb=0.5, result verb=- 0.5), QUD type (contrast-coded: about QUD=0.5, what happened QUD=-0.5), and their interaction. The random effect structures were identified in the same way as in Experiment 1. 3.2.4 Predictions We expect to find a main effect of verb type: The target word – which describes a changed object state – is predicted to be read faster in the result verb conditions (where the verbs semantically entail that the object undergoes a change-of-state) than in the manner verb conditions (where the result state is not encoded in the lexical semantics of the verb itself). Crucially, according to the semantic entailment hypothesis, this verb effect should be unaffected by the discourse-level QUD manipulation (aboutness QUDs vs. change-of-state oriented QUDs). In contrast, the Discourse-based Inference Hypothesis predicts that discourse-level information can interact with verb-level information (here: the lexical-semantic distinction between manner vs. result verbs) to modulate the extent to which comprehenders consider representations where the object has undergone a change-of-state. This leads us to expect that the QUD manipulation can boost the availability of an event representation in which the 46 object changes state, even when the lexical semantics of the verb does not include a notion of change-of-state (manner verbs). More specifically, the prediction is that when the QUD specifically asks about the change-of-state (what happened QUDs), the change-of-state representation will become more available – compared to aboutness QUDs – and this will affect reading times at the target word region. This leads us to expect an interaction between QUD type and verb type: With what happened QUDs, effects of verb type on reading times at the target word are expected to be smaller than with aboutness QUDs, because the inference triggered by what happened QUDs after manner verbs will help comprehenders to process the target word that describes an object change-of-state. In fact, we may find that reading times at the target word in the aboutness QUD conditions are comparable for manner verbs and result verbs, if the effect of QUD on event representations is strong enough to ‘overcome’ effects of verbs’ lexical semantics. This would provide further support for the Discourse-based Inference Hypothesis. 3.2.5 Results Figure 3.1 shows the RTs in the critical region. Visual inspection of the data shows that at and after the target word (e.g., damaged), the RTs were longer in the about QUD + manner verb condition compared to the result verb conditions. On the other hand, RTs in the what happened QUD + manner verb condition do not exhibit such a severe slowdown relative to the result verb conditions. Statistical analyses confirmed these observations: Figure 3.1. Experiment 2: Average reading times by word position (error bars represent +/- 1 SE) 47 Pre-target region. At two words (e.g., it is) before the target word, reading times are faster in the result verb conditions than in the manner verb conditions. This is likely a spillover effect from the preceding sentence (sentence 2), where manner verbs were read more slowly than result verbs (sentence 2 mean RT at the manner verb: 346.63ms vs. result verb: 314.28ms). Target word. At the target word position (e.g., “damaged”), there was again a main effect of verb type, no main effect of QUD type, and, crucially, an interaction of verb type and QUD type. Planned comparisons at the target word position show that in the about QUD conditions (triangles in Figure 3.1), the target word is read slower in the manner verb conditions than in the result verb conditions (effect of verb type: β=49.07, SE=17.31, t=2.83, p=0.008), but in the what happened QUD conditions (circles in Figure 3.1), the target word is read equally fast after both verb types (no effect of verb type: β=1.77, SE=10.88, t=0.16, 48 p=0.87). In other words, in the what happened QUD conditions, the slowdown associated with a preceding manner verb is absent. In other words, in aboutness QUD contexts, when comprehenders are faced with a word that describes a potential changed object state, RTs are slower after manner verbs than after result verbs. But when the QUD specifically highlights the possibility of object state change (what happened QUDs), target words describing potential result states are read equally quickly in both the manner and result verb conditions. This goes against the Semantic Entailment Hypothesis and supports the Discourse-based Inference Hypothesis. (One might wonder why the target word in the about QUD conditions with result verbs seems to be read numerically more slowly than in the what happened QUD conditions with result verbs; crucially, this is not a statistically significant difference (t=-1.22, p > 0.2), nor does it relate to our hypotheses, so we will not discuss it further.) Spillover region. The main effect of verb type is found throughout the spillover region (first, second, third, and fifth spillover word). There is a main effect of QUD type at the second spillover word. There is no interaction between verb type and QUD type in the spillover region (five words after the critical word, see Table 3.1 for details.) Table 3.1 Results of lmer models for each word in the target region word position β SE df t-value Pr(>t) it (Intercept) 298.923 10.759 41.035 27.784 <2e-16 *** verb type 12.675 6.759 1249.027 1.875 0.061 . QUD type 4.857 6.759 1249.128 0.719 0.472 verb type:QUD type -3.361 13.521 1248.689 -0.249 0.804 is (Intercept) 290.327 8 9.5820 42.5583 30.299 < 2e-16 *** verb type 20.2636 5.6627 1247.0603 3.578 0.000359 *** QUD type 0.6366 5.6580 1246.6245 0.113 0.910440 verb type:QUD type -4.9833 11.324 5 1246.6375 -0.440 0.659979 damaged (critical word) (Intercept) 319.152 13.446 47.765 23.736 < 2e-16 *** verb type 25.170 7.795 1242.191 3.229 0.00127 ** QUD type 4.907 7.792 1241.988 0.630 0.52896 verb type:QUD type 46.105 15.586 1242.047 2.958 0.00315 ** 49 and (spillover1) (Intercept) 322.900 12.070 45.042 26.753 < 2e-16 *** verb type 22.724 7.167 1248.340 3.171 0.00156 ** QUD type 5.154 7.157 1247.063 0.720 0.47156 verb type:QUD type 15.122 14.325 1247.517 1.056 0.29135 that (spillover2) (Intercept) 305.423 10.311 43.846 29.622 < 2e-16 *** verb type 21.097 6.162 1246.410 3.424 0.000638 *** QUD type 13.845 6.158 1246.630 2.249 0.024717 * verb type:QUD type 19.976 12.328 1246.886 1.620 0.105385 she (spillover3) (Intercept) 298.334 11.178 43.512 26.689 < 2e-16 *** verb type 19.367 6.531 36.949 2.965 0.00527 ** QUD type -1.071 5.827 1220.703 -0.184 0.85421 verb type:QUD type 4.692 11.661 1217.466 0.402 0.68746 feels (spillover4) (Intercept) 301.003 10.486 46.711 28.704 <2e-16 *** verb type 8.645 5.235 1245.035 1.651 0.0989 . QUD type 2.214 5.229 1244.486 0.423 0.6721 verb type:QUD type 11.814 10.467 1244.615 1.129 0.2592 very (spillover5) (Intercept) 309.158 9.592 46.771 32.230 < 2e-16 *** verb type 13.956 4.868 1250.567 2.867 0.00422 ** QUD type 3.965 4.865 1250.055 0.815 0.41528 verb type:QUD type -3.117 9.736 1250.123 -0.320 0.74891 3.3 Discussion Experiment 2 was designed to investigate how lexical semantics of verbs and discourse-level information from QUDs guide the event representations that comprehenders construct for events where the object may or may not undergo a change-of-state. Experiment 2 goes beyond Experiment 1 in two main ways. First, it tests the Semantic Entailment Hypothesis more directly, by comparing result verbs and manner verbs. Second, it takes a closer look at the Discourse-based Inference Hypothesis by investigating finer grained differences between aboutness QUDs and change-of-state oriented QUDs. According to the Semantic Entailment Hypothesis, verbs’ lexical semantics constrain comprehenders’ construction of event representations: This hypothesis predicts that because manner verbs do not entail a change-of-state, comprehenders do not include a change-of-state notion in their event representations in the manner verb conditions (regardless of QUD type), and because result verbs do entail a change-of-state, comprehenders construct event 50 representations involving change-of-state in the result verb conditions (again, regardless of QUD type). In contrast, the Discourse-based Inference Hypothesis predicts that even with manner verb sentences, comprehenders can be driven by the QUD to consider a change-of-state of the object. An interaction between verb type and QUD type on the RTs at the result state adjective would support the Discourse-based Inference Hypothesis. Indeed, our results support the Discourse-based Inference Hypothesis: In conditions with change-of-state oriented QUDs, participants read the target word equally quickly in the manner verb and the result verb conditions. This suggests that, even if the verb does not semantically entail a change-of-state, the presence of a change-of-state oriented QUD makes participants more likely to construct a representation where the object undergoes a change-of- state. In other words, when the QUD indicates that the inquiry is about the (changed) result state of the object, the event representation can be enriched to include a notion of a changed state, even though this not included in the lexical semantics of manner verbs (e.g., hit). This inferred information can in turn facilitate processing of linguistic material that is dependent on the inferential process, to an extent comparable to how information from verb entailments facilitates processing of linguistic material associated with the entailed meaning. 3.4 General Discussion about Experiments 1 and 2 What kinds of information guide how comprehenders construct mental representation of linguistically-described events? It is unquestionable that the lexical semantics of verbs play a crucial role, as lexical semantics provides the basic skeletons of an event representation, including information about the changes entailed by the action described by 51 the verb. Does verb meaning, then, strictly constrain the event representations available to comprehenders, or can other sources of information also contribute? In this work, we investigated three hypotheses: (i) the Semantic Entailment Hypothesis, which states that comprehenders only consider event representations supported by a verb’s semantic entailments, (ii) the Tense-based Inference Hypothesis and (iii) the Discourse-based Inference Hypothesis, which state that verb-based information and discourse-level information, respectively, can modulate the event representations constructed by comprehenders. As a test case, we examined how comprehenders represent object states when events are described with manner verbs, which do not entail change-of-state of the object, compared to result verbs, which do entail a change-of-state. We report two experiments investigating how comprehenders process linguistic material related to change-of-state when provided with event descriptions with manner and result verbs. Experiment 1 focused specifically on manner verbs, and used a lexical decision task to test the Semantic Entailment Hypothesis, the Tense-based Inference Hypothesis, and the Discourse-based Inference Hypothesis. The results support the Tense-based and the Discourse-based Inference hypotheses over the Semantic Entailment Hypothesis, because we found that tense and discourse context interacted to mediate the availability of the change-of-state inference, even when the event description used manner verbs. In Experiment 2, we directly compared manner verbs and result verbs and manipulated discourse context in more specific ways to further test the Semantic Entailment Hypothesis and the Discourse-based Inference Hypothesis. Results provide evidence for the Discourse-based Inference Hypothesis and against the Semantic Entailment Hypothesis: we found that a change-of-state oriented context can make the change-of-state representation equally available with manner verbs and result verbs. 52 Taken together, results from both experiments suggest that comprehenders use information beyond verb semantics to draw inferences concerning the representation of the object. This provides important insights into the mechanisms underlying online representation of events during language processing. Language is limited to being able to describe only so much of an event, and we can plausibly think that there are numerous details about events that are not captured in linguistic descriptions. Our results suggest that in deciding at what level of detail and granularity the event component should be represented, and in deciding how to represent the unsaid part(s), comprehenders draw from multiple available sources. They are also strategic in choosing how to mentally represent the event being described, especially when the description is underspecified: Our results are compatible with a view where comprehenders prioritise their limited attentional resources toward representing only the event components that are relevant to answering the current communicative goals of the discourse, which we operationalised in terms of the Question- Under-Discussion (QUD) approach (e.g., Roberts, 1996/2012). Under this view, discourse is structured around questions under discussion which represent the interlocutors’ joint discourse goals, i.e., the aims/goals of the current communicative exchange. These studies add support to the growing body of evidence showing that the notion of Questions-under-discussion – or something resembling it – plays an important role in the interpretive processes related to interpreting ambiguous, underspecified, or context-sensitive aspects of language. Our findings are broadly in line with the General QUD Processing Principle (Clifton & Frazier, 2018), which states that utterance interpretation is guided by QUDs. We conclude that event representations, such as how the comprehender keeps track of the dynamic changes that occur during events, are QUD-sensitive as well. Although the methods of the two studies may seem different at first glance – Experiment 1 used a lexical decision task and Experiment 2 used a self-paced reading task – 53 it’s worth noting that these are both reaction-time tasks and in both experiments, the critical word was a past participle or adjective that provided information about the object state (e.g. popped, damaged). Indeed. our results suggest that information about object state representations can be probed both at the offset of the critical sentence (with a 1000ms delay, as in Experiment 1) and in a word embedded in the subsequent sentence (Experiment 2). In light of earlier work in other domains (e.g., Swinney, 1979) on lexical ambiguity, future studies could use cross-modal lexical decision to test different timepoints, in order to gain insights into the time-course of activation of object-state representations. In conclusion, the experiments reported in this paper provide initial evidence that discourse-level information exerts an influence on the mental representation of object states during event comprehension. Prior work on object state representations in language comprehension had largely focused on morphosyntactically-encoded semantic information such as grammatical tense (e.g., Altmann & Kamide, 2007; Kang et al., 2020) and grammatical aspect (e.g., Misersky et al., 2019). Object-specific semantic properties are also known to play a role on the object state representations that comprehenders construct (e.g., Hindy et al., 2021, Horchak & Garrido, 2020), in line with what one might expect based on situation models (e.g., Zwaan & Radvansky, 1998). We believe that our findings open up a fruitful area of research on the role of discourse-level information in event representations and indicates that discourse-level factors need to be incorporated into models of event processing. 3.5 A note about lexicalized and inferred results In this section, I consider how the discussion about object state representations can be recast in terms of what can be called lexicalized and inferred results. As noted earlier, result 54 verbs like clean are complex events specified for a causing subevent and a result (change-of- state) subevent, as in (4) (repeated from Chapter 2). On the other hand, manner verbs such as wash are simple events as in (5), with no result subevent specified (e.g. Rappaport Hovav & Levin, 1998). (4) clean: [ [x ACT<MANNER> CAUSE [ BECOME [y <CLEAN>] ] ] (5) wash: [x ACT<WASH> y] More concretely, the result subevent encoded in result verbs like to clean specifies that the object ends up in a certain result state (e.g., the shirt ends up clean). We will refer to this as LEXICALIZED RESULT. In contrast, manner verbs like to wash only describe the manner in which the action is carried out and do not say anything about the result state, e.g., do not indicate whether the shirt ends up being clean or not. However, as was discussed earlier, comprehenders can infer from sentences containing verbs like wash some kind of result (e.g. the shirt ends up clean.) We will refer to this as INFERRED RESULT. The divide between lexicalized meaning (e.g., LEXICALIZED RESULT) and other sources of meaning (e.g., INFERRED RESULTS) is crucial to the study of lexical semantics. The following quote is from Levin & Rappaport Hovav (2013). (6) “[. . . ] Crucially, a verb’s lexicalized meaning is to be distinguished from additional facets of meaning that can be inferred from a particular use of that verb in context and from the choice of noun phrases serving as arguments of the verb.” (p. 49) 55 A psycholinguistic question that arises is whether this linguistic distinction between lexicalized meaning and “additional facets of meaning” is also psychologically relevant. Are they used in fundamentally different ways during language comprehension? Results from Experiment 2 shed light on how lexicalized results and inferred results are processed during online sentence comprehension. Specifically, inferred results, once activated, can be utilized as effectively as lexicalized results, in line with other recent psycholinguistic findings suggesting that the processing system rapidly integrates multiple constraints that are available (e.g., MacDonald et al. 1994; Tanenhaus & Trueswell 1995; Jurafsky 1996; Altmann 1998; Gibson & Pearlmutter 1998; Levy 2008). Another conclusion that we can draw from results of Experiment 3 is that discourse context (QUD) can be a factor that modulates the availability of inferred meaning. 56 Chapter 4. Grammatical cues about an event’s temporal structure interacts with real-world event knowledge to influence object state representations 4.1 Introduction Successful language comprehension is deeply tied to our real-world knowledge, including our prior knowledge about how events normally take place in the world. For example, we learn throughout our lives that wine glasses often break when dropped but that plastic cups do not, and that eggs typically crack when thrown, but rocks do not. This non- linguistic knowledge plays an important role in guiding language comprehension. Indeed, some accounts of language processing incorporate the mapping between linguistic input and relevant (non-linguistic) event knowledge as a fundamental component of language comprehension (e.g. Altmann & Mirković, 2009; Elman, 2009; Sanford & Garrod, 1998; van Dijk & Kintsch, 1983). In addition to real-world knowledge about the likelihood of event outcomes, the temporal nature of events is also central for our experience and understanding of events: Events always occur at some time for some (if any) duration. One can drop a glass yesterday, today, or tomorrow or swim for a minute, for two hours, or for an entire evening. Understanding how events are temporally structured is a central aspect of event comprehension. Correspondingly, language is equipped with grammatical means for expressing temporal-semantic information about events independently of event knowledge, such as grammatical aspect (e.g. Kevin was dropping the cup vs. Kevin dropped the cup) and tense (e.g. Kevin dropped the cup vs. Kevin will drop the cup). Prior psycholinguistic research has shown that grammatical cues such as grammatical aspect and tense and non-linguistic real-world event knowledge both shape the mental 57 representations of events that we construct based on linguistic input. What is less well- understood, however, is how temporal-semantic grammatical information combines and interacts with non-grammatical event knowledge to guide the construction of event representations (cf. Kang et al., 2020). In the present chapter, we investigate how linguistic cues to an event’s temporal structure (Experiment 3: grammatical aspect, Experiment 4: tense) and non-linguistic real- world knowledge about the likelihood of state change (e.g. that eggs usually crack when thrown, but rocks do not) interact to modulate mental representations of events during incremental sentence processing. Specifically, we focus on actions that may or may not result in state change. For example, whether or not a kicking action changes the object state depends on what the object is. Kicking a snowman will lead to substantial object state change (e.g. the snowman will get damaged), but kicking a chair most likely will not (e.g. the chair will probably not get damaged/broken). This kind of information is not grounded in language, but in our real-world knowledge about events and objects. The aims of this chapter are twofold: First, we investigate how real-world knowledge about the likelihood of state change influences the object state representations that comprehenders construct during real-time sentence comprehension. This question concerns the role of purely non-linguistic knowledge. Does our knowledge about how likely snowmen vs. chairs are to get damaged when kicked how easily comprehenders can integrate visual images depicting different object states into their mental event representations? In order to gain insights into the ease of integration of different object states, we measured response times to intact vs. changed object images in sentences where the target object word (e.g. snowman, chair) was replaced with an image (e.g. Experiment 3: Carlos was kicking/kicked the *snowman*/*chair* …; Experiment 4: Carlos kicked/will kick the *snowman*/*chair* …). 58 Second, we investigate how real-world knowledge and temporal-semantic linguistic cues are combined to contribute to the object state representations constructed during incremental language comprehension. We investigate two different types of temporal- semantic linguistic cues: grammatical aspect (Experiment 3) and tense (Experiment 4). In Experiment 3, we investigate whether there are differential effects of real-world event knowledge in events that are described as ongoing (=imperfective aspect) and completed (=perfective aspect). In Experiment 4, we investigate whether there are differential effects of real-world event knowledge in past tense event descriptions and future tense event descriptions. The English simple past was used in both experiments: as the perfective aspect condition in Experiment 3 and as the past tense condition in Experiment 4. This allowed us to (indirectly) assess the relative strengths of the real-world knowledge effects in different linguistic contexts across the two experiments. The following section reviews theoretical background and prior literature related to the role of grammatical aspect and reports the methods and results of Experiment 3. Section 4.3 reviews prior literature on the role of tense and reports findings from Experiment 4. Section 4.4 concludes. 4.2 Experiment 3: Grammatical aspect Experiment 3 investigates how grammatical aspect (imperfective, perfective) cues interact with real-world event knowledge to influence the processing of visually presented object state information during incremental language comprehension. Grammatical aspect is a morphosyntactic cue that marks ‘different ways of viewing the internal temporal constituency of a situation’ (Comrie, 1976: 3). The present work 59 focuses on the distinction between perfective and imperfective aspect. While perfective aspect views the situation as completed, imperfective aspect views it as ongoing. More specifically, perfective aspect represents the event as a completed whole. The English simple past (e.g. dropped, kicked, nudged) is one way of conveying an event in perfective aspect. The sentence below in (1) is an example. (1) Carlos kicked the chair. In this sentence, the event of kicking the chair is represented as a single whole and as completed (e.g. Comrie, 1976). In contrast, imperfective aspect represents the event as ongoing. The English progressive (see (2)) represents the event in imperfective aspect. The progressive is a subtype of imperfective aspect, and hereafter we use the progressive in discussing imperfective aspect, as it is relevant for the ongoing reading that we are concerned with. Experiment 3 will also use the English progressive to represent imperfective aspect. (2) Carlos was kicking the chair. Here, the imperfective aspect signals that the onset of the event has occurred, but the event has not yet ended. In this example, reference is being made to the internal temporal phases that make up the kicking event (i.e. imperfective aspect imposes an “internal perspective”). Figure 4.1 shows the difference between perfective aspect and imperfective aspect, where the perspective in perfective aspect views the event as a whole, whereas the perspective in imperfective aspect is within the ongoing event (e.g. Comrie, 1976). 60 Figure 4.1 Difference between perfective and imperfective aspect (adapted from Velupillai, 2012) During language processing, comprehenders use aspectual cues to constrain their event representations (e.g. Madden & Zwaan, 2003; Magliano & Schleich, 2010; Morrow, 1985, inter alia). Madden and Zwaan (2003), for example, conducted a series of experiments to probe for effects of perfective and imperfective aspect on comprehenders’ event representations. They showed that when participants read perfective sentences, they preferred pictures showing completed events over pictures showing ongoing events, but that there was no preference upon reading imperfective sentences. Madden and Zwaan concluded that comprehenders use aspectual cues to build their event models. The perfective aspect, in particular, constrains the event representation such that the comprehender’s attentional focus is on the completed situation. Before continuing, a short discussion of the notion of attentional focus is in order. In the subsequent discussion, we use expressions like ‘attentional focus’ and ‘focuses attention on’ to describe a situation where a comprehender’s attention is focused (e.g. by virtue of aspectual cues) on a particular component of an event. We further assume that focusing one’s attention on a certain thing results in more in-depth and detailed processing of that thing. There is experimental evidence that aspectual cues modulate the construction of object state representations during event comprehension, such that perfective aspect focuses the comprehenders’ attention to the resulting state of the object (e.g. Madden-Lombardi, 61 Dominey, & Ventre-Dominey, 2017; Misersky et al., 2021). In Madden-Lombardi et al. (2017), participants read French sentences in perfective or imperfective aspect, word-by- word. The instrument and the object in each sentence were replaced by pictures (e.g. John was using/had used a *corkscrew* to open the *bottle* at the restaurant) that were either congruent or incongruent with the temporal constraints of the verb. For example, in perfective sentences, a closed (no longer in-use) corkscrew and an open wine bottle were aspect-congruent versions of the instrument and the object pictures. The processing of aspect- congruent pictures was faster than that of aspect-incongruent pictures, especially for resulting objects in perfective sentences. The authors take this as evidence that perfective aspect focuses comprehenders’ attention on the resultant state of the object. Further evidence that perfective aspect puts attentional focus on the changed state of the object after the action has been completed come from Misersky et al.’s (2021) ERP studies. Participants read sentences describing a change-of-state event (e.g. to chop an onion), in either perfective aspect (“chopped”) or imperfective aspect (“was chopping”). They then saw a picture that either showed the object that has undergone substantial state change (a chopped onion) or no state change (an onion in its original state) and were asked to decide whether the object in the picture was mentioned in the sentence. Misersky et al. found that state change pictures elicited a higher-amplitude P300 after perfective sentences than after imperfective sentences. This suggests that participants were engaged in more detailed processing of state change pictures after perfective sentences than after imperfective sentences. These results show that perfective aspect, but not imperfective aspect, focuses comprehender’s attention on object state change. In the current study, we used a rebus study paradigm, similar to that of Madden and Therriault (2009) (see also Madden-Lombardi, Dominey, & Ventre-Dominey, 2017). The word ‘rebus’ refers to the fact that the stimuli are a combination of text and images. 62 Participants read sentences in imperfective aspect (e.g. … was dropping …) or perfective aspect (e.g. … dropped …), word by word in a self-paced reading paradigm. The critical object word was replaced by an image of that object in either its intact or changed state. Unlike the post-sentential picture verification task that has been used in much of prior research (e.g. Madden & Zwaan, 2003; Kang et al., 2020; Misersky et al., 2021, inter alia), this method taps into comprehenders’ object state representations during incremental sentence processing, which makes it possible to investigate the cognitive processes that take place as visual stimuli is integrated into comprehenders’ event representations during sentence processing. In order to investigate the role of real-world event knowledge about the likelihood of state change, we used verbs/predicates (e.g. kick, hit, yank on, strike) that do not semantically entail change-of-state of the object (e.g. An object that is kicked may or may not undergo change-of-state.) Information about likelihood of state change is instead grounded in knowledge about real-world events that involve certain actions and objects. Using the same verb/predicate for both types of events (events that are likely to result in state change and events that are unlikely to result in state change) allowed us to assess the effect of incrementally integrating information about real-world object properties. 4.2.1 Methods 4.2.1.1 Participants Adult native speakers of English were recruited on the internet via Prolific. Participants were compensated $4 for their participation. We only included self-reported native English speakers. None of the recruited participants scored below 80% accuracy on the comprehension questions presented after each item (mean accuracy of participants=97.98%). 63 All exclusion criteria reported here were determined before data analysis was conducted. 114 participants were included in the final analysis. All experiments reported in this chapter were reviewed and approved by the USC Institutional Review Board. 4.2.1.2 Design and materials For each of the 24 target items, we created eight different versions. These varied in terms of (a) the grammatical aspect of the sentence, (b) the object type, and (c) the visually- depicted object state. (a) Grammatical aspect: Each sentence was presented in either perfective or imperfective aspect. The English simple past was used to represent perfective aspect (e.g. kicked), and the (past) progressive was used to represent imperfective aspect (e.g. was kicking). (Note that both of these conditions are in past tense. Experiment 4 investigates the role of different tenses.) (b) Object type: The object of the sentence was either likely (e.g. a snowman) or unlikely (e.g. a chair) to undergo state change as a result of the described action (e.g. kicking). The types of events and objects to include in the stimuli were determined based on discussions with a native speaker of English. (c) Object state: The object word(s) (e.g. snowman, chair) was replaced by an image of that object. The image of each object was created in two different versions: a no state change (NSC) version, which shows the intact state of the object, and a state change (SC) version, which shows the object in its changed state. 64 The image stimuli were drawn specifically for this study with detailed instructions, to ensure that objects and object states were clearly recognizable and that for each item, an identical object was depicted in its intact and changed states. That is, the only difference between the NSC and SC versions of each object was their states, not their types. The depicted change(s) were not entailed by verb semantics. See Table 4.1 for a sample target item. (See Appendix C for the full list of target stimuli.) Table 4.1 Sample target item for Experiment 3 Object state pre-image text Object type post-image text Grammatical aspect unlikely to undergo state change likely to undergo state change Imperfective aspect No state change (NSC) Carlos was kicking the on the wooden patio. State change (SC) Carlos was kicking the Perfective aspect No state change (NSC) Carlos kicked the State change (SC) Carlos kicked the In addition to the 24 targets, the study included 32 fillers. Fillers also had one of the words replaced by an image. As this study had a 2x2x2 design, eight lists were created and presented to participants using a standard Latin Square design: each participant was presented with only one condition of each target item and each of the eight conditions appeared the 65 same number of times on any given list. Each of the eight lists contained the same set of filler items, pseudo-randomly distributed throughout the list. 4.2.1.3 Procedure The experiment was hosted online on PennController IBEX (Zehr & Schwarz, 2018; https://www.pcibex.net/). Participants read event descriptions in (a) imperfective aspect (e.g. Kevin was dropping …) or (b) perfective aspect (e.g. Kevin dropped …), word by word (self- paced), with the critical object word(s) replaced with an image of that object that is either (a) likely (e.g. a snowman) or (b) unlikely (e.g. a chair) to undergo the depicted state change as a result of the action. The objects were depicted either in their (a) intact (=NSC) or (b) changed (resultant) (=SC) states. With each keypress, the currently displayed word/image disappeared and the next word/image was displayed. Reaction times to each word/image were measured. Each trial was followed by a comprehension question which participants answered with the F or J keys. Incorrect responses triggered an error feedback message. Comprehension questions were related to the content of a balanced range of sentence regions. 4.2.2 Predictions Prediction 1. Real-world violation penalty If real-world knowledge plays a part in the construction of object state representations during incremental language comprehension, we expect that encountering an object image that goes against real-world knowledge will result in a RT slowdown. We refer to this as the real-world violation penalty. If there is a real-world violation penalty, RTs to unlikely SC images (e.g. damaged chair) will be longer than RTs to likely SC images (e.g. damaged snowman). However, we 66 predict that RTs to both types of NSC images (e.g. intact chair and intact snowman) will not differ. In other words, we predict a main effect of object type in SC image conditions, but not in the NSC image conditions. In other words, we expect an interaction between object type (unlikely to undergo state change vs likely to undergo state change) and object state (intact vs. changed). However, if real-world knowledge has no effect (i.e. there is no real-world violation penalty), any RT differences between the likely and the unlikely SC images should be comparable to any RT differences between the two types of NSC images. (We expect that, due to the images being intrinsically different, we may find independent differences between the two types of NSC images.) Prediction 2. Perfective aspect can amplify the real-world violation penalty Crucially, we may also expect that grammatical aspect information interacts with real- world event knowledge to modulate the object state representations constructed by comprehenders. Specifically, the real-world violation penalty may be bigger in perfective aspect than in imperfective aspect. This prediction is rooted in the idea that perfective aspect focuses attention on state change (e.g. Madden-Lombardi et al., 2017; Misersky et al., 2021), participants would be pushed to more closely evaluate the depicted state change. However, if grammatical aspect plays no role in modulating the strength of the real- world violation penalty, we will see that the RT difference between likely and unlikely SC images in perfective aspect sentences is not any greater than RT difference between likely and unlikely SC images in imperfective aspect sentences. 4.2.3 Data Processing and Analysis 67 When analyzing reaction times to the image, we first removed RTs under 100ms and over 5000ms. No datapoint was more than 3 SDs away from a participant’s mean RT to the object image. For statistical analyses, we used linear mixed effects models with RT as the dependent variable, and grammatical aspect, object type, object state, grammatical aspect x object type interaction, grammatical aspect x object state interaction, and object type x object state interaction as fixed effects. For planned comparisons comparing NSC conditions and SC conditions, grammatical aspect, object type, and their interaction were fixed effects. For planned comparisons comparing perfective aspect conditions and imperfective aspect conditions within SC images, object type was the fixed effect. The criteria for outlier removal were determined prior to data analysis. We used the maximal random effect structure justified by model comparison. Models were estimated using the lme4 package (version 1.1.26) (Bates et al., 2015) and lmertest (version 3.1.3) (Kuznetsova et al., 2017) in the R software environment (R Development Core Team, 2019). We used the same R packages for planned comparisons. Our predictions and analyses crucially depend on comparing RTs to different images, as in Madden & Therriault (2009), Madden-Lombardi et al. (2017), and Kang et al. (2020), where they compared RTs to different images, within the same linguistic context. It is a crucial part of our prediction that RTs to images depicting changed states of different types of objects would be different (=real-world violation penalty). Note, however, that we do not make claims based on absolute differences between RTs to different images. Our central prediction about the differential effects of the real-world violation penalty in imperfective aspect and perfective aspect hinges on an interaction between grammatical aspect object type (i.e. relative differences between the strengths of the RT penalty). 4.2.4 Results 68 Figure 4.2 shows the average RTs at the image position, by condition. Overall, it can be observed that RTs to the SC images are longer than RTs to the NSC images. This observation is confirmed by statistical analysis (β=158.77, SD=14.65, t=10.837, p < 0.0001). This main effect of object state is expected as the SC images are divergent from prototypical depictions of the object, but this is not part of our theoretical predictions. Therefore, we do not discuss it further. To investigate our hypothesis about the real-world violation penalty, we tested whether there was an interaction between object state and object type and conducted separated analyses on NSC conditions and SC conditions. We found an interaction between object state and object type (β=-91.05, SD=29.30, t=-3.108, p=0.00191), driven by differential effects of object type in SC image conditions and in NSC image conditions: Within SC images, we found a main effect of object type (β=-96.329, SD=23.796, t=-4.048, p < 0.0001). RTs to unlikely SC images were longer than RTs to the likely SC images, suggesting a real-world violation penalty. (Within NSC images, there was no main effect of object type (β=-4.231, SD=16.190, t=-0.261, p=0.7939)). In order to test whether perfective aspect amplifies the real-world violation penalty, we looked for an interaction between grammatical aspect and object type within the SC image conditions. We found an interaction between grammatical aspect and object type within SC image conditions (β=99.103, SD=47.596, t=2.082, p=0.0375). This is driven by differential effects of object type in perfective aspect and imperfective aspect conditions. In perfective aspect conditions, there is a main effect of object type: RTs to unlikely SC images are longer than RTs to likely SC images (β=-144.77, SD=36.91, t=-3.922, p < 0.0001), suggesting a real-world violation penalty in perfective aspect. However, in imperfective conditions, there is no such effect (β=-47.53, SD=30.17, t=-1.575, p=0.116). This suggests 69 that the real-world violation penalty holds only in perfective but not in imperfective sentences. Figure 4.2 Average raw RTs to the image by condition (ms) in Experiment 3 (Error bars show +/- 1 SE) 4.2.5 Discussion In order to investigate how real-world event knowledge and grammatical aspect affect comprehenders’ object state representations, Experiment 3 tested the processing of visually- presented object state information during reading of perfective or imperfective sentences. We used a rebus sentence paradigm to investigate effects of (i) object type (likely vs. unlikely to 70 undergo state change), (ii) object state (SC vs. NSC), and (iii) grammatical aspect (perfective vs. imperfective) on the RTs to object images. We first tested how real-world event knowledge affects the processing of object images. Is there a RT penalty associated with real-world knowledge violations? If there is a real-world violation penalty, it will be difficult to integrate an object’s resultant state that violates our knowledge about how the world usually works into the event representation, which will manifest as a RT penalty. If not, real-world properties of objects will not affect the object image RTs. Our results suggest a real-world violation penalty, according to which real-world event knowledge contributes to object state representations. We found that while RTs to either object type in their NSC states did not differ, RTs to unlikely SC images (e.g. a damaged chair) were longer than RTs to likely SC images (e.g. a damaged snowman). This suggests that knowledge about the properties of real-world objects is activated and rapidly integrated into the event representations that comprehenders construct incrementally during real-time processing. The central aim of Experiment 3 is to test whether the strength of the real-world violation penalty is mediated by grammatical aspect cues, which provide information about whether the event is completed or ongoing. Does the real-world violation penalty get amplified in perfective aspect sentences, where we assume that the aspect cue focuses comprehenders’ attention on state change? Our results suggest that the answer is yes. In perfective aspect sentences, there is a penalty associated with integrating an unlikely changed state, whereas in imperfective sentences, there is no significant penalty. Experiment 3 shows that attentional biases introduced by perfective aspect interacts with real-world knowledge information to affect how visually-presented object state representations are integrated into the comprehenders’ mental event representations. These 71 results build upon Misersky et al.’s (2021) findings wherein SC pictures resulted in a higher amplitude P300 after perfective sentences than after imperfective sentences, suggesting that people were engaged in more detailed evaluation of state change after perfective sentences. Results from Experiment 3 show that during detailed evaluation of state change (driven by perfective aspect), processing penalties associated with unlikely state changes can be increased. In Experiment 4, we turn to the question of how tense (past vs future) interacts with real-world knowledge. Unlike grammatical aspect, tense indicates whether the event has already happened or not, but does not necessarily make reference to a particular stage of the event. How do temporal constraints regarding whether the event has already happened or is yet to happen modulate the strength of the real-world violation penalty? 4.3 Experiment 4: Tense Tense is closely related to grammatical aspect but can operate independently of grammatical aspect. Comrie (1985) defines tense as the “grammaticalisation of location in time” and aspect as the “grammaticalisation of expression of internal temporal constituency” (p. 1). Following Comrie (1985), Reichenbach (1947), Klein (1994), and many others, we assume that tense relates utterance time to the time that a sentence is describing. Here, we only discuss simple tense (simple past, simple present, and simple future). The simple past tense indicates that the described event precedes the utterance time, the simple present tense indicates that the described event overlaps with the utterance time, and the simple future tense indicates that the described event follows the utterance time (e.g. Reichenbach, 1947). English marks tense using grammatical markers, e.g. Kevin dropped the glass (past), Kevin will drop the glass (future). 72 The role of tense in sentence comprehension has been studied in psycholinguistic literature. Altmann & Kamide (2007) showed that participants’ looks to the visual scene can be directed differently when the sentences were in past tense or in future tense. They conducted a visual-world eye-tracking experiment where participants heard sentences such as “The man will drink ... ” and “The man has drunk ... ”. Participants looked more to the full glass (the initial state of the object) when hearing “will drink”, but looked more at the empty glass (the resultant state of the object) upon hearing “has drunk.” These results show that tense is a cue that affects object state representations during sentence comprehension. More recently, it has been found that the effect of tense interacts with general non- linguistic information about events and objects to modulate comprehenders’ mental representations of object states. Kang et al. (2020) tested how event representations are influenced by tense and by the degree of change described by a sentence. In one of their studies, participants read past tense sentences that described an action that would leave the object in its original state (e.g. minimal-change sentences: The woman chose the ice cream) or an action that would substantially change the state of the object (e.g. substantial-change sentences: The woman dropped the ice cream). After participants read the sentences, they saw a picture that depicted the object in its original (e.g. an upright ice cream) or changed state (e.g. a crushed ice cream). The task was to decide whether the depicted object was mentioned in the sentence that they just read. In another study, participants read future tense versions of the same sentences (e.g. The woman will choose/drop the ice cream) and performed the same task. Kang et al. found that only past tense sentences elicit responses showing sensitivity to degree of change: When the sentences were in past tense, participants were faster to verify the original state of the object (e.g. an upright ice cream) with minimal- change sentences than with substantial-change sentences. No such asymmetry was observed when the sentences were in future tense. The authors conclude that the interplay between 73 real-world event knowledge and the grammatical tenses of sentences influences the mental representation of object states. In Kang et al.’s (2020) studies, object state change was manipulated by using two different verbs: one indicating no/minimal change (e.g. choose) and the other indicating substantial change to the object (e.g. drop). That is, the degree of object state change was clearly indicated by the semantics of the action itself. However, even the same verb/action can result in different degrees of object state change, depending on other properties of the situation. For example, prior work by Horchak and Garrido (2020) shows that when object state change is inferred (i.e. not driven by the verb’s lexical semantics), other features of the situation (e.g. the weight of an item) are taken into account when representing the target object state. For example, they found that in understanding what happens to the tomato in a sentence like “You drop a bowling ball on a tomato” vs. “You drop a balloon on a tomato”, the weight of the item contributed to the updating of object state information. In Experiment 4, we investigate how tense cues are integrated with real-world knowledge information during the integration of visually-presented object state information into mental event representations. In Experiment 3, we only looked at past tense sentences, but now in Experiment 4, we compare past tense sentences and future tense sentences, which allows us to manipulate where the event is located on the hearer (=participant)’s timeline. Because the past tense sentences in Experiment 4 are string-identical to perfective aspect sentences in Experiment 3, our main focus of investigation in Experiment 4 concerns the future tense sentences. (Past tense sentences act as a baseline condition.) 4.3.1 Methods 4.3.1.1 Participants 74 As in Experiment 3, we recruited adult native speakers of English on Prolific. 115 participants completed the experiment. Two participants were excluded from data analysis for being nonnative speakers of English. One participant was excluded for reporting to have a hearing impairment. One participant was excluded for a technical data error. One participant was excluded from data analysis for scoring below 80% accuracy on the comprehension questions (and for being a nonnative speaker of English.) None of the other recruited participants scored below 80% accuracy on the comprehension questions that occurred after each item (mean accuracy=97.8%). All exclusion criteria reported here were determined before data analysis was conducted. 110 participants were included in the final analysis. 4.3.1.2 Design and materials Experiment 4 was identical to Experiment 3 except that each sentence was now presented in either past (e.g. kicked) or future tense (e.g. will kick). Therefore, the stimuli varied on (a) the tense of the sentence, (b) the object type, and (c) the object state. See Table 4.2 for a sample target item. (See Appendix D for the full list of target stimuli.) Table 4.2 Sample target item for Experiment 4 Object state pre-image text Object type post-image text Tense unlikely to undergo state change likely to undergo state change Past tense No state change (NSC) Carlos kicked the on the wooden patio. State change (SC) Carlos kicked the 75 Future tense No state change (NSC) Carlos will kick the State change (SC) Carlos will kick the 4.3.2 Procedure Experiment 4 was conducted in the same way as Experiment 3, with the exception that the sentences were either in past or future tense. 4.3.3 Predictions 1. Real-world violation penalty The predictions regarding the real-world violation penalty are the same as in Experiment 3. 2. Future tense amplifies the real-world violation penalty In Experiment 3, we found that the real-world violation penalty exists in perfective aspect sentences, which were presented in the English simple past. In Experiment 4, we test past tense (simple past) sentences and future tense sentences. Based on findings from Experiment 3, we predict that a similar pattern of the real-world violation penalty will be found with past sentences in Experiment 4, as they are string-identical conditions. The crucial prediction that we test in Experiment 4 concerns what happens in future tense sentences. Does the real-world violation penalty exist for events described in future tense, which signals that the event has yet to happen in the time of the hearer (=participant)? 76 Furthermore, we also test whether the real-world violation penalty is more severe for state changes that have already happened (past tense) or have yet to happen (future tense). One may expect that it may be more severe in future tense, because it may be easier to integrate an unlikely resultant state when the event is understood to have already happened relative to the time of the hearer (=participant). In more intuitive terms, state changes that have already happened may be seen as more incontestable compared to state change that have yet to be obtained in the time of the hearer. If the real-world violation penalty is stronger in future tense sentences than in past tense sentences, we will see that the RT difference between likely and unlikely SC images in future tense conditions is greater than the RT difference between likely and unlikely SC images in past tense conditions. However, if the strength of the real-world violation penalty is not affected by tense cues, the RT difference between likely and unlikely SC images will be constant across both tense conditions. 4.3.4 Data processing and analysis As in Experiment 3, we first removed RTs under 100ms and over 5000ms. No datapoint was more than 3 SDs away from a participant’s mean RT to the object image. We used linear mixed effects models with RT as the dependent variable, and tense, object type, object state, grammatical aspect x object type interaction, tense x object state interaction, and object type x object state interaction as fixed effects. For planned comparisons comparing NSC conditions and SC conditions, tense, object type, and their interaction were fixed effects. For planned comparisons comparing past tense conditions and future tense conditions within SC images, object type was the fixed effect. The criteria for outlier removal were 77 determined prior to data analysis. We used the maximal random effect structure justified by model comparison. 4.3.5 Results Figure 4.3 shows the average RTs at the image position, by condition. As in Experiment 3, it can be seen that in general, RTs are longer in SC image conditions than in NSC image conditions. This is confirmed by a main effect of object type (β=-99.351, SD=22.593, t=-4.397, p < 0.001). In Experiment 3, the real-world violation penalty was supported by an interaction between object state and object type, driven by differential effects of object type in SC image conditions and in NSC image conditions. In Figure 4.3, we again see a similar pattern. There is an interaction between object type and object state (β=-103.469, SD=45.187, t=-2.290, p=0.0221). This interaction is driven by a stronger effect of object type within SC images than within NSC images. Within SC images, there is a marginal main effect of object type, RTs to unlikely SC images being longer than RTs to likely SC images (β=-155.02, SD=77.12, t=-2.01, p=0.0538). This suggests a RT slowdown associated with processing a state change that violates real-world knowledge. However, within NSC images, there is no effect of object type (β=-46.437, SD=25.771, t=-1.802, p=0.0718). These results again support the real-world violation penalty hypothesis, as in Experiment 3. In order to test whether future tense amplifies the real-world violation penalty, we looked at whether there is an interaction between tense and object type within SC image conditions. The interaction between tense and object type did not reach significance (β=73.62, SD=46.33, t=1.589, p=0.1124). However, because we predicted differential effects of object type in past tense and future tense sentences, we conducted planned comparisons to 78 test this. When the sentence was in future tense, there was a main effect of object type: RTs to unlikely SC images were longer than RTs to likely SC images (β=-152.31, SD=35.63, t=- 4.275, p < 0.0001). However, when the sentence was in past tense, no such effect of object type was found (β=-78.62, SD=75.89, t=-1.036, p=0.311). (Although the effect does not reach significance, there is a numerical trend that is in line with findings from Experiment 3, where there was a significant effect of object type in perfective aspect sentences. In Experiment 4, the effect may not be as strong due to the presence of the other condition (future tense sentences) in the experiment.) These results suggest that the real-world violation penalty is amplified in future tense sentences. Figure 4.3 Average raw RTs to the image by condition (ms) in Experiment 4 (Error bars show +/- 1 SE) 79 4.3.6 Discussion Experiment 4 investigated how real-world event knowledge and tense affect comprehenders’ object state representations during incremental sentence processing. We looked at how fast participants process visually-presented object state information while reading sentences presented in future or past tense. As in Experiment 3, we used a rebus sentence paradigm to investigate effects of (i) object type, (ii) object state, and (iii) tense (future vs. past) on the RTs to object images. We first tested whether findings from Experiment 3 regarding the real-world knowledge violation penalty replicate. Again, we found that while RTs to either object type in their NSC states did not differ, RTs to unlikely SC images were longer than RTs to likely SC images, suggesting that there is a processing penalty associated with integrating an object’s resultant state that is not compatible with what we know about how the world works. The main aim of this experiment was to investigate different to test whether tense mediates the strength of the real-world violation penalty (cf. Experiment 3 only looked at past tense sentences). Is the real-world violation penalty stronger when the described event happened in the past or is yet to happen, relative to the time of the hearer (=participant)? Our results suggest that when the event is described in future tense, there is a stronger penalty to integrating an unlikely changed state representation of an object. There was no such penalty when the event was described in past tense. This could be because for events that have already happened on the timeline, the outcomes have already materialized, whereas there is more room for future outcomes to be contested. Overall, Experiment 4 shows that tense, a linguistic cue, interacts with non-linguistic event knowledge information to influence the integration of object state information. 80 4.4 General Discussion Which factors contribute to how comprehenders’ mental representations of events are shaped during online sentence processing? Presumably, comprehenders consider information from both the sentence itself and from what we know about the world around us. However, the question of how linguistic and non-linguistic information are integrated to guide and constrain object state representations during incremental sentence comprehension is not yet very well understood. In the present study, we examined the role of (a) grammatical markers of an event’s temporal structure and (b) the likelihood of state change informed by real-world event knowledge. The central aims of this study were to investigate (i) how event knowledge contributes to object state representations and (ii) how linguistic cues about an event’s temporal structure combines with event knowledge information to influence people’s object state representations. We report two experiments which investigated the effect of (a) real-world knowledge about the likelihood of state change and (b) temporal-semantic cues in language (Experiment 3: grammatical aspect, Experiment 4: tense). Results from both experiments show that when comprehenders update mental event representations by integrating visually presented object state information, object state information that violates real-world knowledge incurs a processing cost: the real-world violation penalty. Crucially, both experiments show that the real-world violation penalty interacts with temporal-semantic linguistic cues. Specifically, the real-world violation penalty is amplified when comprehenders are pushed to evaluate state change in more detail, either because the event is construed as completed rather than ongoing (grammatical aspect), or because the event (and its outcome) has yet to happen (tense). In sum, results from both experiments suggest that comprehenders integrate information from 81 both non-linguistic event knowledge and linguistic cues about the described event’s temporal structure to build object state representations online. The perfective aspect condition in Experiment 3 and the past tense condition in Experiment 4 are string-identical, due to both conditions being the English simple past (e.g. Carlos kicked …). (The difference between two experiments is the presence of the imperfective aspect condition in Experiment 3 and the future tense condition in Experiment 4.) The repetition of these conditions allows us to (indirectly) compare results across both experiments. The numerical results hint at the real-world violation penalty being the strongest in future tense sentences, while being the weakest in imperfective sentences. However, it is difficult to draw strong conclusions about these comparisons, as we cannot directly compare the two experiments. This is because the experiments were not designed to test between- subjects/between-experiment comparisons and because of the presence of different conditions in the two experiments. A possible concern related to the task is whether participants could have interpreted the SC images in a way that was different from what we had intended. Both experiments were designed to elicit “result state” interpretations of the image stimuli, rather than “initial state” interpretations. In other words, the images are intended to depict the object state after the action – the state after Carlos kicked it, not the state before he kicked it. However, is it possible that participants may have construed the images as depicting the initial states of objects before the action? For example, is it a concern that participants may have understood the "Carlos kicking the [damaged snowman]” image as Carlos kicking an already damaged snowman, as opposed to kicking a snowman and causing it to become damaged? Results from both experiments suggest that this concern is unwarranted. If it were the case that participants had understood the SC images to be initial states of objects, there would be no RT difference between the two different object type conditions within SC images. For 82 example, there should be no a priori reason for someone kicking a damaged chair vs. someone kicking a damaged snowman to differ in real-world likelihood. (In other words, there is no reason to expect a systematic, across-the-board difference in real-world likelihood between the two object type conditions across items. See Appendix C and D for the full list of target items.) However, we observe a main effect of object type within SC images (=real- world violation penalty) in both experiments, suggesting that participants were indeed interpreting the images to be result states of objects (as intended). This study adds to the body of work showing that temporal constraints introduced by grammar constrain and guide event representations. Crucially, we show that these temporal constraints interact with non-linguistic information grounded in real-world knowledge. These findings provide important insights into how linguistic input is mapped onto event representations during online language comprehension. Understanding the dynamics of how the described event unfolds over time – i.e. understanding the change(s) that the objects undergo – is a process that involves rapidly integrating information encoded in grammar and general semantic knowledge about the world. 83 Chapter 5. Grammatical cues dynamically update object location representations in real-time 5.1 Introduction In previous chapters, we investigated how linguistic descriptions of events map onto mental event representations, with a particular focus on the representation of object states. However, objects can undergo changes in physical location as well as in physical states. In linguistics, it has been discussed that there are conceptual similarities between change-of- state and change-of-location (e.g. Gruber, 1965; Gropen et al., 1991; Pustejovsky, 1991). It has also been shown that the cognitive system can not only track multiple representations of the same object as it undergoes change-of-state (e.g. Altmann & Ekves, 2019; Altmann & Kamide, 2007; Hindy et al., 2012; Kang et al., 2020; Solomon et al., 2015) but also as it undergoes change-of-location (e.g. Altmann & Kamide, 2009). In Altmann and Ekves’ (2019) Intersecting Object Histories theory, changes in location are understood as a type of change relevant for the representation of object histories. The current work builds on these insights and aims to investigate the mapping between language and mental representations of object locations. We investigate three different grammatical factors that may influence the representation of object location in transfer-of-possession events: (a) grammatical aspect, (b) verb semantics, and (c) argument realization patterns. How do these grammatical factors influence the mapping from language to object location representations? Another aim of this work is to investigate whether the mental representation of object locations can be dynamically updated during incremental sentence processing. Do comprehenders update their mental representations of object location in real-time as the linguistic input unfolds? 84 To investigate these questions, we conducted a visual-world eye-tracking study using a novel webcam-based eye-tracking paradigm (Webgazer; Papoutsaki et al., 2016), which allowed us to probe how visual attention reflects the real-time mapping of linguistic input onto mental representations of object locations in events where the object undergoes change- of-location from its starting point/origin (Source) to an endpoint (Goal). We focus specifically on transfer-of-possession events, which are transitive events that involve an object being transferred from a Source/agent to a Goal/patient. Using transfer-of-possession events allowed us to investigate effects of verb semantics and argument realization patterns, which we elaborate on in Sections 5.3-5.4. Much of prior literature in the domain of events that involve a Source and a Goal was interested in the linguistic and non-linguistic asymmetry between the Source and the Goal (e.g. Regier, 1996; Ihara & Fujita, 2000; Stefanowitsch & Rohde, 2004; Lakusta & Landau, 2005; Regier & Zheng, 2007; Papafragou, 2010; Lakusta & Landau, 2012; Do, Papafragou, & Trueswell, 2020). In the current work, we shift our attention from the representations of the Source and the Goal to the third element involved in these events: namely, the object undergoing change-of-location. This chapter aims to shed light on how different grammatical properties of the sentence interact in real-time to dynamically update the mental representation of changing situations and to ultimately lead to a final understanding of the event. In the following sections (Sections 5.2-5.4), we provide a brief background on the grammatical factors that are under investigation in this chapter. Section 5.5 reports the methods and results of Experiment 5. Section 5.6 discusses the findings and concludes. 5.2 Grammatical aspect 85 As discussed in Chapter 4, grammatical aspect – in particular the distinction between perfective and imperfective aspect – provides information about whether the described event is represented as completed or ongoing (e.g. Comrie, 1976). When an event is described in perfective aspect, the event is viewed as a completed whole. In contrast, an event described in imperfective aspect is viewed as ongoing and incomplete, with reference being made to the internal temporal phases that make up the event. It has been shown that grammatical aspect can modulate the pronoun interpretation following transfer-of-possession events (e.g. Rohde, Kehler, & Elman, 2006). In sentences like “John was handing/handed a book to Bob. He _____”, the interpretation of the ambiguous pronoun he was sensitive to the grammatical aspect of the sentence: Imperfective sentences resulted in more Source interpretations of the ambiguous pronoun than perfective sentences did. Although Rohde, Kehler, and Elman’s (2006) results suggest that grammatical aspect is a cue that comprehenders consider when interpreting transfer-of-possession event descriptions, their main aim was to understand the representations of the Source and the Goal (as they are relevant for pronoun interpretation), not the representation of the object being transferred. Thus, their results do not directly speak to the research aims of our study, namely: How does grammatical aspect guide the mental representation of object locations? More specifically, does the completed vs. ongoing distinction lead to different representations of the object’s location? We investigate this question in Experiment 5. In addition to grammatical aspect, we consider two additional grammatical factors that may influence the representation of object location in transfer-of-possession events: verb semantics and syntactic argument realization patterns. 5.3 Verb semantics 86 According to Rappaport Hovav and Levin (2008), transfer-of-possession verbs can be classified into different classes, based on whether the verb’s lexical semantics entails (i.e. semantically guarantees) successful transfer or not. 4 With verbs that entail successful transfer (give-type verbs; e.g. give, hand), it is infelicitous to assert that the transfer was unsuccessful, as demonstrated by the infelicity of the continuation in (1). (1) Kim gave her brother the ball, # but he never received it. These verbs differ from verbs that do not guarantee successful transfer (throw-type verbs; e.g. throw, toss), for which it is felicitous to deny that the transfer was unsuccessful, as in (2). (2) Kim threw her brother the ball, but he never received it. Rappaport Hovav and Levin also note that give-type verbs, unlike throw-type verbs, lack a path argument. This claim is supported by the fact that give-type verbs are incompatible with spatial PPs. When give-type verbs take to-phrase arguments, the preposition to can only take animate complements, not inanimate places (e.g., Goldsmith, 1980, Green, 1974). However, to-phrase arguments of throw-type verbs can take both animate and inanimate complements that describe places. This contrast is shown in (3a) and (3b). (3) a. I gave the package to Maria/*London. 4 Rappaport Hovav and Levin (2008) also discuss a third class of verbs: send-type verbs. We do not use this class of verbs in Experiment 5, so we leave it out of the discussion. 87 b. I threw the ball to Maria/the other side of the field. In the current study, we investigate whether and how different verb classes with different entailments (give-type verbs and throw-type verbs) contribute to the mental representations of object locations. More specifically, does the presence vs. absence of the successful transfer entailment lead to different representations of the object’s location? 5.4 Argument realization patterns In addition to differences in the lexical semantics of verbs, we also consider a syntactic dimension of transfer-of-possession sentences, which also has semantic correlates – namely how arguments are ordered. As shown in (4), English ditransitive sentences can be realized in two different ways, as in (4). (4) a. Mary threw the ball to Bill. [to- variant] b. Mary threw Bill the ball. [double object variant] It is widely accepted that these syntactic variants also differ in their semantics – more specifically, that they are associated with different causal structures (e.g. Pinker, 1989, Goldberg, 1992, 1995, Krifka, 1999, 2004, Hale & Keyser, 2002, Harley, 2003, Beck & Johnson, 2004). The to- variant in (4a) is associated with a CAUSED MOTION meaning (Mary causes the ball to go to Bill), which entails change-of-location but not change-of- possession: (4a) could be used in a context where the ball did not reach Bill (e.g. ends up on the ground between Mary and Bill). Here, the ball changed its location but did not change possession from Mary to Bill. The double object variant in (4b), on the other hand, is 88 associated with a CAUSED POSSESSION meaning (Mary causes Bill to have the ball). It has also been claimed that the double object variant is associated with a successful transfer inference, such that in (4b), Bill ends up with the ball, and in (5b) John ends up with knowledge of linguistics (whereas (5a) is neutral about whether the learning was successful, e.g. Green, 1974). (5) a. Mary taught linguistics to John. [to- variant] b. Mary taught John linguistics. [double object variant] In the current study, we investigate whether the different inference patterns associated with each argument realization pattern give rise to different mental representations regarding the location of the object. More specifically, are comprehenders more likely to represent the object as having underwent successful transfer when the sentence is presented in the double object variant than when it is presented in the to- variant? 5.5 Experiment 5 To investigate how grammatical aspect, verb semantics, and argument realization patterns in linguistic input guide the real-time and final representations of object locations, we conducted a visual-world eye-tracking study where participants heard descriptions of transfer-of-possession events and were asked to click on where they think the object is in the scene. The location of the final click allows us to tap into the final representations of object locations, whereas the real-time eye gaze data allows us to tap into the dynamically updating real-time representations of object locations. 89 5.5.1 Methods 5.5.1.1 Participants Participants were recruited on the internet via Prolific and were compensated $5 in return for participating. All participants were native speakers of English and were born in and resided in the United States. 68 participants completed the study. One participant was excluded for reporting that they have a visual impairment. One participant was excluded for reporting a hearing impairment. One participant was excluded for poor accuracy on attention check trials (8.33% accuracy; average accuracy of included participants=99.85%). Five participants were excluded for not following instructions to click on the fixation cross. Four additional participants were excluded due to poor calibration (initial calibration score < 60, mean pre-trial calibration score < 45). All exclusion criteria were determined prior to data analysis. We included data from 56 participants in the data analysis. 5.5.1.2 Materials 5.5.1.2.1 Auditory (sentence) stimuli All target sentences were past tense sentences containing a transfer-of-possession verb and animate Source and Goal individuals that differed in gender. Names of the Source and the Goal characters differed on each trial. The target stimuli varied on three aspects: verb type, grammatical aspect, and argument realization pattern. Verb type was manipulated between-items and grammatical aspect and argument realization patterns were manipulated within-items. (a) Verb type: Twelve of the target items had give-type verbs, and the other twelve target items had throw-type verbs. We only included verbs that sound natural with both 90 argument realization patterns and were natural with “the ball” as the object, without triggering a strong inference about non-canonical types of balls. The give-type verbs that were used were give, hand, and bring. Example (6) shows all conditions of a sample give- type item. The throw-type verbs that were used were throw, kick, and toss. Each verb was used in four different items. Example (7) shows all conditions of a sample throw-type item. (b) Grammatical aspect: Each sentence was presented either in perfective aspect or imperfective aspect. The simple past was used to represent perfective aspect (e.g. gave, threw), and the (past) progressive was used to represent imperfective aspect (e.g. was giving, was throwing), as in Experiment 3. (c) Argument realization pattern: The post-verbal arguments in each sentence (the ball, Goal) were presented in either the double object variant or the to- variant. (6) Give-type example stimuli a. imperfective + to- variant Liam was giving the ball to Paige. b. imperfective + double object variant Liam was giving Paige the ball. c. perfective + to- variant Liam gave the ball to Paige. d. perfective + double object variant Liam gave Paige the ball. (7) Throw-type example stimuli a. imperfective + to- variant Carly was throwing the ball to Oliver. 91 b. imperfective + double object variant Carly was throwing Oliver the ball. c. perfective + to- variant Carly threw the ball to Oliver. d. perfective + double object variant Carly threw Oliver the ball. In addition to 24 target items, the experiment included 34 filler items. Twelve of the filler items also functioned as attention check trials (i.e. had clearly expected click locations). Filler sentences involved a ball or a bird as invisible objects. The invisible “object” was in the sentence subject or object position. All the items in the experiment were in past tense. Sentences were recorded by a female native speaker of American English, using the Praat software (Boersma & Weenink 2021). 5.5.1.2.2 Display (Visual scenes) Visual scenes were created to accompany the auditory stimuli. (See Figure 5.1.) They were hand-drawn on an iPad tablet. They depicted a Source character and a Goal character, illustrated with simple stick figures, whose positions (right/left) on the screen were counterbalanced throughout the experiment. The arrow next to the Source character indicated the direction of the ball’s movement (i.e. signaled who the Source was); this was explained to participants. Crucially, the ball was not visible on the scene. Instead, participants were asked to click on where they think the ball is on the scene. 92 Figure 5.1 Sample target visual image For purposes of data analysis, we specified five different vertical areas of interest, both for measuring the location of the participants’ clicks and for eye gaze data. Each area was identical in width, occupying one fifth of the entire screen’s width: the Source area, the Source-adjacent area, the Center area, the Goal-adjacent area, and the Goal area. These areas are shown in Figure 5.2. Before we proceed, an explanation about the areas of interest and analyses regions are in order. The eye-tracking data was collected by areas of interest (the Source area, the Source-adjacent area, the Center area, the Goal-adjacent area, and the Goal area). For purposes of data analysis, we also define analysis regions: SOURCE region, GOAL region, and MIDDLE region. (Note the use of capital letters for regions.) This is shown in Figure 5.2 and also summarized in Table 5.1. In order to test predictions about whether the object is represented as having reached its end location (e.g. in perfective aspect sentences) or not (e.g. in imperfective aspect sentences), we collapsed the Source and the Source-adjacent areas into the SOURCE region and collapsed the Goal and the Goal-adjacent areas into the GOAL region. This was largely motivated by the assumption that (depending on the verb type and argument realization 93 pattern) a completed event representation may also include the ball as not having successfully reached the Goal, therefore represented as being in the Goal-adjacent area. In order to test predictions about whether the object is represented as being located on the middle-ground/path between the Source and the Goal, we collapsed the Source-adjacent, Center, and Goal-adjacent areas into the MIDDLE region. Figure 5.2. Areas of interest and regions for analyses Table 5.1. Areas of Interest and analysis regions Areas of interest Regions for Source vs. Goal analyses Region for Middle ground analysis Source area SOURCE region 94 Source-adjacent area MIDDLE region Center area Goal-adjacent area GOAL region Goal area 5.5.2 Procedure The experiment was hosted online on PennController IBEX (Zehr & Schwarz, 2018; https://www.pcibex.net/), and participants did it remotely via the internet. Eye gaze data was collected using the Webgazer.js library (Papoutsaki et al., 2016). Webgazer is an open-source webcam-based eye-tracking JavaScript library which runs locally on the participant’s computer and uses the participant’s webcam to compute the gaze position. Only the gaze position was transmitted from the participant’s computer to our web server. Each trial started with fixation cross at the center of the screen. Participants were asked to look at and click on the fixation cross. Participants were given 3.5 seconds to click on the cross before the experiment automatically proceeded to the visual scene. One second (1000ms) after they clicked on the fixation cross, the visual scene appeared. One second (1000ms) after the visual scene appeared, the audio started playing. Participants were asked to imagine that the world is in a freeze-frame during the moment described by the sentence, and to click on where they think the ball is in the scene. The full wordings for these instructions are provided in (8). We did not want participants to think about the location of the ball at the moment the sentence is uttered, as this kind of interpretation may mislead participants to think about the possibility of the presence of other intervening events that may have caused the ball to move after the throwing event. Therefore, we included the phrase “during the moment described by the sentence” to encourage participants to construct event representations relevant to the temporal interval that is discussed by the sentence. 95 (8) You will hear a sentence, for example “The ball is near Mason”. However, the ball is not visible. Now let’s imagine that we freeze the world during the moment described by the sentence. Where do you think the ball is in the scene? Your task is to use your mouse to click where you think the ball is. After the audio finished playing, participants were given five seconds to provide their click response before the experiment automatically proceeded to the next trial. Upon clicking, the trial ended and the next trial started. There was a 250ms pause in between each trial. The experiment began with two practice trials. The experiment lasted around 20 minutes. Participants’ eye movements were recorded during the entire trial (from the onset of the fixation cross until a click was made), along with their final mouse click region and timing. 5.5.3 Predictions In this experiment, we have two different types of measures: participants’ final click locations after the end of the sentence and participants’ real-time eye gaze before the final click. These two types of data can shed light on different aspects of processing. The final click data will inform us of the comprehenders’ post-sentential object location representation. The eye gaze data will inform us of how comprehenders dynamically update the representations of object location in real-time during the unfolding of the sentence. In this section, we discuss predictions regarding (i) the final/post-sentential interpretation, which is informed by the click data and (ii) real-time processing, which is informed by eye gaze data. 5.5.3.1 Predictions about post-sentential interpretations 96 The first goal of this study is to investigate how different grammatical factors contribute to the final interpretations of transfer-of-possession events, in particular with regards the representation of object location. We consider three non-mutually-exclusive hypotheses about each of the grammatical factors. 1. Grammatical Aspect Hypothesis: The Grammatical Aspect Hypothesis states that the mental representations of object locations are guided by grammatical aspect information. This hypothesis predicts that upon hearing imperfective aspect sentences, comprehenders will construe the event as ongoing, leading to an event representation where the ball may have not reached the final goal yet (e.g. The source individual may still have it, or it may be mid-air.) Perfective sentences, however, are predicted to be more likely to lead to a completed event representation where the ball is at its final location (regardless of whether the transfer is successful or not). If the Grammatical Aspect Hypothesis is on the right track, we expect the proportion of SOURCE region clicks to be greater in the imperfective aspect condition than in the perfective aspect condition. Conversely, the proportion of GOAL region clicks should be greater in the perfective aspect condition than in the imperfective aspect condition. 2. Verb Semantics Hypothesis: According to the Verb Semantics Hypothesis, verbs’ entailment patterns constrain the mental representation of events that comprehenders construct, such that give-type verbs, which entail successful transfer, give rise to event representations where successful transfer occurs – i.e. the ball successfully ends up at the Goal. Conversely, throw-type verbs, which do not entail successful transfer, may give rise to event representations in which the ball does not successfully end up at the Goal. 97 The Verb Semantics Hypothesis predicts that there will be more clicks to the MIDDLE region in sentences with throw-type verbs than in sentences with give-type verbs. We interpret clicks on the MIDDLE region as indicating that the participant constructed an event representation where the transfer was not successful (i.e. did not reach the goal.) To test this hypothesis, we focus on the proportion of clicks on the MIDDLE region, instead of the SOURCE or the GOAL regions. This is because we are mainly interested in whether comprehenders construct an event representation in which successful transfer occurs or not. 3. Argument Realization Hypothesis: The Argument Realization Hypothesis states that argument realization patterns mediate the event representations that are constructed. Upon hearing the double object variant, claimed to entail successful transfer, comprehenders are predicted to construct an event representation in which the ball ends up at the goal. This, however, will not be guaranteed with to- variant sentences. This hypothesis predicts that there will be more clicks to the MIDDLE region in to- variant sentences than in double object variant sentences. To test this hypothesis, we again focus on the proportion of clicks on the MIDDLE region, instead of the SOURCE or the GOAL regions because we are mainly interested in whether comprehenders construct an event representation in which successful transfer occurs or not. 5.5.3.2 Predictions about real-time processing The second aim of this experiment is to investigate whether the mental representation of object locations can be dynamically updated during real-time sentence processing. Do comprehenders update mental representations of object location in real-time as the linguistic input unfolds? We assume that eye movements will provide a measure of where participants 98 think the object (the ball) is in the scene. Observing participants’ eye gaze as the sentence unfolds can shed light on their dynamically changing mental representations. The three hypotheses outlined in Section 5.5.3.1 are also relevant here. If mental representations of object locations are dynamically updated in real-time during the unfolding of linguistic input, we expect to see that the patterns outlined for each hypothesis above will be reflected in eye movements while participants are still listening to the sentence. For example, if grammatical aspect cues are considered in real-time to update the mental representation of object location, it can be predicted that the proportion of SOURCE looks will be greater in the imperfective aspect condition than in the perfective aspect condition, during the unfolding of the rest of the sentence. Conversely, the proportion of GOAL region looks should be greater in the perfective aspect condition than in the imperfective aspect condition, again during the unfolding of the rest of the sentence. The same logic goes for other hypotheses. However, it may be difficult to test the exact time course of when each grammatical cue comes into play, due to different sampling rates on each participant’s computer and the uncertainty about time latencies for each participant. 5.5.4 Data processing and analysis 5.5.4.1 Click data (Post-sentential interpretations) In order to investigate the hypotheses regarding the effects of grammatical cues on participants’ final interpretations, as indicated by click locations, we conducted three different statistical analyses on the click data. First, we conducted analyses on the proportions of SOURCE region clicks in different conditions. Second, we conducted analyses on the proportions of GOAL region clicks in different conditions. These two analyses allow us to test the Grammatical Aspect Hypothesis. Lastly, we conducted analyses on the proportions of 99 MIDDLE region clicks in different conditions. This analysis informs us of the Verb Semantics Hypothesis and the Argument Realization Pattern Hypothesis. For statistical analyses, we used Generalized Linear Mixed Effects models (glmer) with grammatical aspect, verb type, argument realization pattern, grammatical aspect x verb type interaction, grammatical aspect x argument realization pattern interaction, and verb type x argument realization pattern interaction as fixed effects. We used the maximal random effect structure justified by model comparison. 5.5.4.2 Eye gaze data (Real-time processing) To investigate whether the mental representation of object location is dynamically updated during the unfolding of the sentence, we looked at the time window from the onset of the verb (e.g. throwing, threw) to the end of the sentence. The analysis time window was offset by 400ms, instead of the usual 200ms, given that it has been reported that there is a systematic delay in Webgazer recordings (e.g. approximately 300ms additional delay in Slim & Hartsuiker, 2021). We conducted three types of analyses. The proportions of looks to the SOURCE region and to the GOAL region were analyzed separately. We also analyzed GOAL-SOURCE difference scores, which were calculated by subtracting the proportion of looks to the SOURCE region from the proportion of looks to the GOAL region. Statistical analyses were conducted using the lme4 package (version 1.1.26) (Bates et al., 2015) and lmertest (version 3.1.3) (Kuznetsova et al., 2017) in the R software environment (R Development Core Team, 2019). 5.5.5 Results 5.5.5.1 Post-sentential interpretations 100 Figure 5.3 shows the proportion of clicks on each of the five areas (Source, Source- adjacent, Center, Goal-adjacent, and goal), by verb type and grammatical aspect. Here, the two argument realization pattern conditions were collapsed, as the two conditions did not show different patterns. See Appendix F for additional plots separating out the two argument realization conditions. Two different patterns can be seen in Figure 5.3. First, there are more GOAL region clicks (and fewer Source clicks) in imperfective conditions (was giving, was throwing) than in perfective conditions (gave, threw). Another pattern that can be inspected is that there are more MIDDLE region clicks in throw-conditions than in give-conditions. We additionally see an overall Source preference, which we attribute to various reasons, including the visual salience of the source character due to the presence of the arrow. Because the overall preference for the Source is not relevant for our hypotheses or the main claims being made in this chapter, we do not discuss it further. Figure 5.3. Proportion of clicks on each area of interest (Error bars show +/- 1 SE) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 was giving gave was throwing threw source sourceADJ center goalADJ goal 101 Figure 5.4 plots the average proportion of looks to the SOURCE region and the GOAL region in each condition, for better visualization of the SOURCE vs. GOAL click patterns. In terms of the proportion of SOURCE region clicks, there is a main effect of grammatical aspect (β=1.35091, SE=0.21338, z=6.331, p < 0.0001): Imperfective sentences elicited more SOURCE region clicks than perfective sentences. There is a main effect of grammatical aspect in the proportion of GOAL region clicks as well (β=-1.66127, SE=0.23944, z=-6.938, p < 0.0001): Perfective sentences elicited more GOAL region clicks than imperfective sentences. These results support the Grammatical Aspect Hypothesis. Figure 5.4 Proportion of clicks on the SOURCE and GOAL regions (Error bars show +/- 1 SE) Figure 5.5 plots the average proportion of looks to the MIDDLE region only. In terms of the proportions of MIDDLE region clicks, there was a main effect of verb type (β=- 1.91883, SE=0.23349, z=-8.218, p < 0.00001), supporting the Verb Semantics Hypothesis. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 was giving gave was throwing threw SOURCE GOAL 102 There was no effect of argument realization patterns (β=0.09667, SE=0.14619, z=0.661, p=0.508), suggesting that the results do not provide support for the Argument Realization Pattern Hypothesis. Figure 5.5. Proportion of clicks on MIDDLE region (Error bars show +/- 1 SE) 5.5.5.2 Real-time processing The analysis of the eye-tracking data focuses on the role of grammatical aspect. Figure 5.6 shows the proportions of looks to the GOAL and the SOURCE regions in imperfective aspect trials (red) and in perfective aspect trials (blue), relative to the onset of the verb (e.g. throwing, threw). 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 was giving gave was throwing threw 103 Figure 5.6. Proportions of GOAL region looks by grammatical aspect (left), Proportions of SOURCE region looks by grammatical aspect (right); Proportions of Center area looks are not plotted; 0 on the x-axis indicates the onset of the verb; Data is collapsed by participant for plotting The plot on the left shows that after the onset of the verb, the proportion of GOAL region looks is greater in perfective aspect trials than in imperfective aspect trials. Conversely, the plot on the right shows that the proportion of SOURCE region looks is greater in imperfective aspect trials than in perfective aspect trials. In other words, there is a weaker Source preference (or a stronger Goal preference) in perfective sentences than in imperfective sentences. To test whether this pattern is statistically significant, we conducted three analyses. We conducted analyses on the proportion of looks to the SOURCE region, the proportion of looks to the GOAL region, and the GOAL-SOURCE difference scores, which were calculated by subtracting the proportion of looks to the SOURCE region from the proportion 104 of looks to the GOAL region. These analyses were conducted on raw data (not averaged by participant or by item) using Generalized Linear Mixed Effects models (glmer). We found a main effect of grammatical aspect in the proportion of SOURCE region looks: The proportions of looks to the SOURCE region were greater in imperfective aspect sentences than in perfective aspect sentences (β=0.036380, SE=0.015872, df=55.438971, t=2.292, p=0.02572). In the proportions of GOAL region looks, there was again a main effect of grammatical aspect: The proportions of looks to the GOAL region were greater in perfective aspect sentences than in imperfective aspect sentences (β=-0.052635, SE=0.003320, df=37274.312172, t=-15.852, p < 0.0001). There was a main effect of grammatical aspect in the GOAL-SOURCE difference scores as well: The GOAL- SOURCE difference scores were greater in perfective aspect sentences than in imperfective aspect sentences (B=-0.090327, SE=0.040924, df=49.179616, t=-2.207, p=0.032). These results suggest that the mental representations of object locations are reflected in the real-time eye movements during the unfolding of the sentence. 5.6 Discussion How does language guide the mental representations of object location during real- time language processing of transfer-of-possession events? We investigated three grammatical factors that can influence the construction of object location representations: grammatical aspect, verb semantics, and argument realization patterns. The study was designed to test how grammatical cues impact real-time eye movements to the visual scene and the final interpretation of the event. The first aim of the study was to assess how different grammatical cues influence the final object location representations that comprehenders post-sententially reach at. Our results 105 support the Grammatical Aspect Hypothesis, according to which the ongoing vs. completed event representations lead to different representations of object locations. The click data suggests that in perfective aspect sentences, comprehenders are more likely than in imperfective aspect sentences to construct an event representation in which the object is located at/near the goal, whereas in imperfective sentences, they are more likely than in perfective sentences to construct an event representation where the object is located at/near the source. The results also support the Verb Semantics Hypothesis, suggesting that verbs’ entailment patterns contribute to the event representations that comprehenders construct. When the sentence describes a throwing event, comprehenders are more likely to construct an event representation in which the object is located in the middle ground than when the sentence describes a giving event. These results suggest that the semantic differences between the two verb classes (the presence vs. absence of the successful transfer entailment, as well as the presence vs. absence of a path argument) lead to different representations of object locations. In sum, comprehenders consider verb semantics and grammatical aspect as cues to guide the final representation of the event. The second aim of the study was to investigate whether the mental representation of object locations can be dynamically updated during incremental sentence processing. Our eye-tracking data indicate that object locations are updated in real-time while the sentence unfolds. Although the effect of verb semantics does not clearly show up in our eye-tracking data, the data suggests that grammatical aspect is indeed a cue that comprehenders consider in order to dynamically update the object location representations during the unfolding of the sentence. That is, the process of language getting mapped onto mental event representations is a dynamic, real-time process. This finding is in line with prior work by Altmann and Kamide (2009), where they showed that in comprehending sentences like The woman will put the glass on the table, the event representations of object locations are dynamically updated. 106 Our study further shows that a temporal-semantic grammatical cue such as grammatical aspect is a relevant cue during this dynamic process. This study uses a novel webcam-based eye-tracking paradigm and shows that it can provide a useful way to collect eye-tracking data. More research is needed to better understand the exact nature of the time latencies and the factors that attribute to them, but this study provides some initial confirmation that the method provides data that can be informative for psycholinguistic research. 107 Chapter 6. Conclusion 6.1 Overview and summary This dissertation set out to explore the question of how the language comprehension system constructs mental representations of events based on linguistic descriptions of events. When a comprehender encounters linguistic input describing an event, what information guides how a mental representation of that event is constructed? I reported a series of experiments that were designed to address this question. I specifically focused on the dimension of dynamic changes occurring to objects in an event. Three grammatical and non- grammatical factors were investigated: (a) grammatical properties of event descriptions (e.g. verb semantics, grammatical aspect, tense) (Chapters 2, 3, 4, 5), (b) discourse-level properties (e.g. Question-under-discussion) (Chapters 2, 3), and (c) real-world knowledge about physical events (Chapter 4). In this way, I investigated information types from multiple levels and domains of representations: verb semantics, which is grammatical information at the lexical level; tense/aspect, which is also grammatical information, but at the level of morphosyntax; discourse context, which is linguistic information, but is not grammatically encoded; and real-world knowledge, which is non-linguistic information. In this endeavor, the dissertation sought out to link insights from lexical semantics, pragmatics, psycholinguistics, and cognitive psychology. Put together, the studies reported in this dissertation show that comprehenders construct event representations through a highly integrated process that rapidly recruits information from multiple sources (grammatical and non-grammatical information sources). Experiments 1 and 2 investigate the question of how discourse-level (QUD) and verb- level cues interact to guide the construction of object state representations. Experiment 1 was 108 designed to investigate how discourse-level cues about which event participant is being discussed interact with tense information to influence the construction of object state representations (Chapter 2). Experiment 2 was designed to investigate how verb type (manner verb vs. result verb) information grounded in the verbal root interacts with discourse-level cues about whether the object’s resultant state is under discussion to influence whether comprehenders infer the object’s change-of-state (Chapter 3). Results from both experiments indicate that discourse-level information and verb-level information interact with one another to guide the object state representations that are constructed. Experiments 3 and 4 investigate how non-linguistic real-world knowledge about the likelihood of state change (e.g. dropping a wine glass vs. dropping a plastic cup) interacts with grammatical cues (grammatical aspect, tense) about an event’s temporal structure to influence the object state representations. Both experiments show that grammatical cues such as grammatical aspect and tense can modulate the effect of real-world knowledge on event representations, suggesting a rapid interplay between grammatical and non-grammatical information during the processing of visually-presented object state information. Finally, Experiment 5 broadens the inquiry beyond the domain of state change by turning to another type of change: namely, object location change. More specifically, it investigates how object location representations are updated in real-time, by specifically focusing on the mapping between linguistic input and object location representations. Results highlight the effects of grammatical cues (e.g. grammatical aspect, verb type) on final and real-time representations of object locations. (Chapter 5) Experiment 5 shows that the cognitive process of updating mental event representations is a dynamic process that occurs during the unfolding of the linguistic input. In sum, this dissertation shows that comprehenders are efficient in constructing the representations of changes that objects undergo (both state change and location change) by 109 rapidly integrating information from multiple sources (grammatical and non-grammatical information sources). 6.2 Theoretical implications In this section, I discuss two major implications of this dissertation research for theories of event representations. 6.2.1 Language, linguistic theory, and event representations The way that events are represented in language has been extensively studied in theoretical linguistics. There also exists a good amount of understanding about how events are perceived and understood by the human cognitive system in general. This dissertation is part of a growing body of work that takes initial steps at bridging the gap between these two bodies of literature. The studies reported in this dissertation draw insights from linguistic theory, in particular theories about verb semantics and tense/aspect. Verb semantics often encodes the temporal contour along which an event develops. Grammatical aspect and tense provide cues as to the temporal perspective from which an event is linguistically described. The outcomes of my experiments highlight the role of these grammatical factors in building event representations during sentence processing. The temporal domain of event representations – the temporal contour or the temporal perspective – is a central component of event representations. This dissertation research thus motivates the need to incorporate insights from linguistic theory in the study of event cognition. Integrating our knowledge about linguistic 110 event representations and about cognitive event representations can help advance us better understand the mechanisms that enable humans to apprehend, think about, and communicate about events. 6.2.2 Going beyond language The first contribution of this dissertation is that it highlights the role that grammatical cues in linguistic input play in guiding the mental representations of events that comprehenders construct. It also highlights, however, that in constructing event representations on the basis of a sentence, comprehenders also use additional information outside of the sentence to construct event representations that go beyond what is encoded in the linguistic representation of the sentence. The sentence comprehension system integrates information from non-grammatical sources, such as discourse-level information (Experiments 1-2) and real-world knowledge (Experiments 3-4) to draw inferences about what happens in the described event. In other words, during the process of building mental representations of events, comprehenders often augment, in systematic and meaningful ways, the event representations provided by the grammatical content of the sentence. This dissertation research also sheds light on the psycholinguistic processes involved in recruiting information from both grammatical and non-grammatical sources to dynamically update comprehenders’ event representations. The results support the view that both grammatical and non-grammatical information can be rapidly integrated at the same time. Historically, much of the debate about how different kinds of information are integrated in real time has tended to focus on the question of how cues about syntactic structure are integrated with other kinds of linguistic and non-linguistic information (e.g. Trueswell & 111 Tanenhaus 1994, Trueswell, Tanenhaus & Garnsey 1994). In this dissertation, we addressed this issue in another foundational domain of language, the semantics of event structure. In sum, this work motivates the need for theories of event representation to incorporate and account for discourse-level information and real-world knowledge about physical events and objects as meaningful components. 6.3 Methodological implications In this section, I briefly discuss the methodological implications of the research reported in this dissertation, as each experiment uses different methodologies. A number of different methods were employed to probe the mental representations of events that comprehenders construct online and offline. Experiment 1 used a lexical decision task after the sentence stimuli and tapped into comprehenders’ object state representations by measuring reaction times to words that describe the object’s changed state. In this study, object state representations were probed by using a post-sentential measure. Experiment 2 used self-paced reading and looked at how fast participants read the part of the sentence that describes the state of the object. This method also taps into comprehenders’ object state representations by measuring reaction times to linguistic stimuli, but in this case during sentence comprehension, not afterwards. In Experiments 3 and 4, I also use visual stimuli to complement our understanding of the mental event representations that comprehenders construct online. By using the rebus sentence paradigm, I look at how people process visual stimuli that depict object state information. The rebus sentence paradigm also allows us to investigate the cognitive processes that take place as visual stimuli is integrated into comprehenders’ event representations during sentence processing, not afterwards. 112 Unlike Experiments 1-4 that tap into the comprehender’s event representation at one point during sentence processing (either at the words describing object state or at the image depicting object state), Experiment 5 uses the visual world eye-tracking paradigm to examine how event representations are updated in real-time during incremental sentence processing. This method helps provide additional information about the time course of the updating process in a continuous way. 6.4 A note about attentional focus At several points in the dissertation, I discuss attentional focus as being relevant to the process of building mental event representations. In Experiment 1, I suggest that discourse context is a factor that can direct attentional focus on different event participants. For instance, a QUD that is about the object can drive people to attend to the object by introducing an expectation that the sentence is interpreted as providing an answer to a question about the object, in which case effects of tense information on object state representations are stronger. In Experiment 3, I discuss grammatical aspect as a factor that can affect attentional focus on object state change. Specifically, perfective aspect can increase attention on state change, in which case effects of real-world event knowledge on object state representations are stronger. Taken together, findings from Experiments 1 and 3 suggest that cues from different sources and levels of information – from discourse context and from grammatical elements – may be involved in the cognitive processes relevant to attentional focus. However, because Experiments 1 and 3 were not designed to directly measure the level of attentional focus on certain components of event structure, it is difficult at this point to discuss the exact nature of 113 the processes involved. Further investigating the role that attentional focus plays in understanding the internal (temporal) structure of events would be an interesting area of research. 6.5 Future directions The studies reported in this dissertation lead to the larger question about the relationship between human language and human cognitive processes more generally. As event representations are a very fundamental component of both human language and other aspects of cognition, I believe that this area of research provides an extremely fruitful testing ground for investigating questions about the relationship between human language and (non- linguistic) cognition. Exploring these questions in the domain of event representations, especially by focusing on the internal structure of events, is a very new and emerging body of literature and there exist numerous avenues for future research. Although this dissertation has focused on the domain of sentence comprehension by adult language users, there are related topics that are starting to emerge as new research areas that merit further work, including cross-linguistic differences in the grammatical encoding of events and their relationship to mental event representations, the cognitive processes involved in mapping mental event representations to language (i.e. language production), and the way in which children learn and understand events in language and the mind. 6.6 Final remarks I would like to leave here some final remarks about the notion of change, which I – building on Altmann and Ekves (2019) – understand in terms of dynamicity. In this 114 dissertation, I made use of the distinction between verbs/events that involve change (state change or location change) and those that do not involve change. It is worth noting that this kind of distinction is linked to other notions that other researchers have employed, such as the distinction between bounded vs. unbounded events (e.g. Ji & Papafragou 2020a, 2020b; Ünal, Ji, & Papafragou 2021), between events vs. processes (e.g. Wellwood, Hespos, & Rips, 2018a, b) between resultative vs. non-resultative events (e.g. Sakarias & Flecken 2019). As I see it, what generally connects these insights is that the distinction between events that have a dynamically defined internal structure and those that do not is central to event understanding, in both language and cognition. Much awaits to be uncovered about the role of the internal structure of events in event cognition. 115 References Alexiadou, A., Martin, F., & Schäfer, F. (2017, June). Optionally causative manner verbs: When implied results get entailed. Paper presented at the Roots V conference, Queen Mary University of London/University College London. Altmann, G. T. M., & Ekves, Z. (2019). Events as intersecting object histories: A new theory of event representation. Psychological Review, 126(6), 817-840. doi:10.1037/rev0000154 Altmann, G. T. M., & Kamide, Y. (2007). The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language, 57(4), 502-518. doi:10.1016/j.jml.2006.12.004 Altmann, G. T. M., & Kamide, Y. (2009). Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation. Cognition, 111(1), 55–71. doi:10.1016/j.cognition.2008.12.005 Altmann, G. T. M., & Mirković, J. (2009). Incrementality and Prediction in Human Sentence Processing. Cognitive Science, 33(4), 583–609. doi:10.1111/j.1551- 6709.2009.01022.x Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01 Baayen, R. H., & Milin, P. (2010). Analyzing reaction times. International Journal of Psychological Research, 3(2), 12-28. doi:10.21500/20112084.807 Beavers, J. T. (2006). Argument/oblique alternations and the structure of lexical meaning (Unpublished doctoral dissertation). Stanford University. 116 Beaver, D., & Clark, B. (2008). Sense and sensitivity: How focus determines meaning. West Sussex, UK: Wiley Blackwell. Beck, S., & Johnson, K. (2004). Double Objects Again. Linguistic Inquiry, 35(1), 97–123. doi:10.1162/002438904322793356 Birch, S., & Rayner, K. (1997). Linguistic focus affects eye movements during reading. Memory & Cognition, 25(5), 653-660. doi:10.3758/BF03211306 Boersma, P., & Weenink, D. (2021). Praat: doing phonetics by computer [Computer program]. Version 6.1.47, retrieved from http://www.praat.org/ Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977- 990. doi:10.3758/BRM.41.4.977 Büring, D. (2003). On D-trees, beans and B-accents. Linguistics and Philosophy, 26, 511- 545. doi:10.1023/A:1025887707652 Chmielewski, M., & Kucker, S. C. (2020). An MTurk crisis? Shifts in data quality and the impact on study results. Social Psychological and Personality Science, 11(4), 464- 473. doi:10.1177/1948550619875149 Clifton Jr, C., & Frazier, L. (2012). Discourse integration guided by the ‘Question under Discussion’. Cognitive Psychology, 65(2), 352-379. doi:10.1016/j.cogpsych.2012.04.001 Clifton Jr, C., & Frazier, L. (2018). Context effects in discourse: The Question under Discussion. Discourse Processes, 55(2), 105-112. doi:10.1080/0163853X.2017.1330029 Comrie, B. (1976). Aspect: An Introduction to the Study of Verbal Aspect and Related Problems. Cambridge University Press. 117 Comrie, B. (1985). Tense. Cambridge University Press. Cummins, C., & Rohde, H. (2015). Evoking context with contrastive stress: Effects on pragmatic enrichment. Frontiers in Psychology, 6, 1-11. doi:10.3389/fpsyg.2015.01779 Cutler, A., & Fodor, J. A. (1979). Semantic focus and sentence comprehension. Cognition, 7, 49-59. doi:10.1016/0010-0277(79)90010-6 Davis, A. R., & Koenig, J. P. (2000). Linking as constraints on word classes in a hierarchical lexicon. Language, 76(1), 56-91. doi:10.1353/lan.2000.0068 Degen, J., & Goodman, N. (2014). Lost your marbles? The puzzle of dependent measures in experimental pragmatics. Proceedings of the Cognitive Science Society, 36, 397-402. Retrieved from https://escholarship.org/uc/cognitivesciencesociety Delogu, F., Jachmann, T., Staudte, M., Vespignani, F., & Molinaro, N. (2020). Discourse expectations are sensitive to the Question Under Discussion: Evidence from ERPs. Discourse Processes, 57(2), 122-140. doi:10.1080/0163853X.2019.1575140 Do, M. L., Papafragou, A., & Trueswell, J. (2020). Cognitive and pragmatic factors in language production: Evidence from source-goal motion events. Cognition, 205, 104447. doi:10.1016/j.cognition.2020.104447 Dowty, D. (1979). Word meaning and Montague grammar. Dordrecht, The Netherlands: Kluwer Academic. Dowty, D. (1991). Thematic proto-roles and argument selection. Language, 67(3), 547-619. doi:10.1353/lan.1991.0021 Elman, J. L. (2009). On the Meaning of Words and Dinosaur Bones: Lexical Knowledge Without a Lexicon. Cognitive Science, 33(4), 547–582. doi:10.1111/j.1551- 6709.2009.01023.x 118 Fillmore. C. J. (1970). The grammar of hitting and breaking. In R. Jacobs & P. Rosenbaum (Eds.), Readings in English transformational grammar (pp. 120-133). Waltham. MA: Ginn. Garnsey, S. M., Pearlmutter, N. J., Myers, E., & Lotocky, M. A. (1997). The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. Journal of Memory and Language, 37(1), 58-93. doi:10.1006/jmla.1997.2512 Gennari, S. & Poeppel, D. (2003). Processing correlates of lexical semantic complexity. Cognition, 89. 27-41. doi:10.1016/S0010-0277(03)00069-6 Goldberg, A. E. (1992). The inherent semantics of argument structure: The case of the English ditransitive construction. Cognitive Linguistics, 3, 37-74. doi:10.1515/cogl.1992.3.1.37 Goldberg, A. E. (1995). Constructions: A construction grammar approach to argument structure. Chicago: University of Chicago Press. Goldsmith, J. (1980). Meaning and mechanism in grammar. In Susumu Kuno (Ed.), Harvard studies in syntax and semantics, 423–449. Cambridge, MA: Harvard University. Grant, M., Clifton, C., & Frazier, L. (2012). The role of non-actuality implicatures in processing elided constituents. Journal of Memory and Language, 66(1), 326-343. doi:10.1016/j.jml.2011.09.003 Green, G. M. (1974). Semantics and syntactic regularity. Bloomington, IN: Indiana University Press. Gropen, J., Pinker, S., Hollander, M., & Goldberg, R. (1991). Affectedness and direct objects: The role of lexical semantics in the acquisition of verb argument structure. Cognition, 41(1), 153–195. doi:10.1016/0010-0277(91)90035-3 Gruber, J. S. (1965). Studies in lexical relation. Doctoral dissertation, Massachusetts Institute of Technology. 119 Hale, K., & Keyser, S. J. (2002). Prolegomenon to a theory of argument structure. Cambridge, MA: MIT Press. Harley, H. (2003). Possession and the double object construction. In P. Pica & J. Rooryck (Eds.), Linguistic Variation Yearbook 2 (pp. 31-70). Amsterdam: John Benjamins. Hindy, N. C., Altmann, G. T. M., Kalenik, E., & Thompson-Schill, S. L. (2012). The effect of object state-changes on event processing: do objects compete with themselves? The Journal of Neuroscience, 32(17), 5795-5803. doi:10.1523/JNEUROSCI.6294- 11.2012 Hopper, P. J., & Thompson, S. A. (1980). Transitivity in grammar and discourse. Language, 56(2), 251-299. doi:10.2307/413757 Ihara, H., & Fujita, I. (2000). A Cognitive Approach to Errors in Case Marking in Japanese Agrammatism: The Priority of the Goal -ni over the Source -kara. Constructions in Cognitive Linguistics, 123. Jackendoff, R. (1996). The proper treatment of measuring out, telicity, and perhaps even quantification in English. Natural Language & Linguistic Theory, 14(2), 305-354. doi:10.1007/BF00133686 Ji, Y., & Papafragou, A. (2020a). Is there an end in sight? Viewers’ sensitivity to abstract event structure. Cognition, 197, 104197. doi:10.1016/j.cognition.2020.104197 Ji, Y., & Papafragou, A. (2020b). Midpoints, endpoints and the cognitive structure of events. Language, Cognition and Neuroscience, 35(10), 1465–1479. doi:10.1080/23273798.2020.1797839 Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press. 120 Kang, X., Eerland, A., Joergensen, G. H., Zwaan, R. A., & Altmann, G. T. M. (2020). The influence of state change on object representations in language comprehension. Memory & Cognition, 390-399. doi:10.3758/s13421-019-00977-7 Kehler, A., & Rohde, H. (2017). Evaluating an expectation-driven question-under-discussion model of discourse interpretation. Discourse Processes, 54(3), 219-238. doi:10.1080/0163853X.2016.1169069 Klein, W. (1994). Time in Language. Psychology Press. Krifka, M. (1999). Manner in dative alternation. In Proceedings of WCCFL 18, 260-271. Somerville, MA: Cascadilla Press. Krifka, M. (2004). Semantic and pragmatic conditions for the dative alternation. Korean Journal of English Language and Linguistics, 4, 1-32. Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software, 82, 1–26. doi:10.18637/jss.v082.i13 Lakusta, L., & Landau, B. (2005). Starting at the end: The importance of goals in spatial language. Cognition, 96(1), 1–33. doi:10.1016/j.cognition.2004.03.009 Lakusta, L., & Landau, B. (2012). Language and Memory for Motion Events: Origins of the Asymmetry Between Source and Goal Paths. Cognitive Science, 36(3), 517–544. doi:10.1111/j.1551-6709.2011.01220.x Lee, S. H., & Kaiser, E. (2021). Does hitting the window break it? Investigating effects of discourse-level and verb-level information in guiding object state representations. Language, Cognition and Neuroscience, 36(8), pp. 921-940. doi:10.1080/23273798.2021.1896013 Levin, B. (1993). English verb classes and alternations: A preliminary investigation. Chicago: University of Chicago Press. 121 Levin, B., & Rappaport Hovav, M. (2013). Lexicalized meaning and manner/result complementarity. In B. Arsenijević, B. Gehrke, & R. Marín (Eds.), Studies in the composition and decomposition of event predicates (pp. 49–70). Springer. Madden, C. J., & Therriault, D. J. (2009). Short article: Verb aspect and perceptual simulations. Quarterly Journal of Experimental Psychology, 62(7), 1294–1303. doi:10.1080/17470210802696088 Madden, C. J., & Zwaan, R. A. (2003). How does verb aspect constrain event representations? Memory & Cognition, 31(5), 663–672. doi:10.3758/BF03196106 Madden-Lombardi, C., Dominey, P. F., & Ventre-Dominey, J. (2017). Grammatical verb aspect and event roles in sentence processing. PLOS ONE, 12(12), e0189919. doi:10.1371/journal.pone.0189919 Magliano, J. P., & Schleich, M. C. (2010). Verb Aspect and Situation Models. Discourse Processes. doi:10.1207/S15326950dp2902_1 McKoon, G., & Love, J. (2011). Verbs in the lexicon: Why is hitting easier than breaking?. Language and cognition, 3(2), 313-330. doi:10.1515/langcog.2011.011 McKoon, G., & Macfarland, T. (2000). Externally and internally caused change of state verbs. Language, 76. 833-858. doi:10.2307/417201 McKoon, G., & Macfarland, T. (2002). Event templates in the lexical representations of verbs. Cognitive Psychology, 45. 1-44. doi: 10.1016/S0010-0285(02)00004-X McKoon, G., & Ratcliff, R. (2003). Meaning through syntax: Language comprehension and the reduced relative clause construction. Psychological Review, 110. 490-525. doi: 10.1037/0033-295X.110.3.490 McKoon, G., & Ratcliff, R. (2005). “Meaning through Syntax” in sentence production and comprehension. Psychological Review, 112. 1032-1039. doi:10.1037/0033- 295X.112.4.1032 122 McKoon, G., & Ratcliff, R. (2008). Meanings, propositions, and verbs. Psychonomic Bulletin and Review, 15. 592-597. doi:10.3758/PBR.15.3.592 Misersky, J., Slivac, K., Hagoort, P., & Flecken, M. (2021). The state of the onion: Grammatical aspect modulates object representation during event comprehension. Cognition, 214, 104744. doi:10.1016/j.cognition.2021.104744 Morrow, D. G. (1985). Prepositions and verb aspect in narrative understanding. Journal of Memory and Language, 24(4), 390–404. doi:10.1016/0749-596X(85)90036-1 Onea, E. (2016). Potential questions at the semantics-pragmatics interface. Leiden: Brill. Papafragou, A. (2010). Source-Goal Asymmetries in Motion Representation: Implications for Language Production and Comprehension. Cognitive Science, 34(6), 1064–1092. doi:10.1111/j.1551-6709.2010.01107.x Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). Webgazer: Scalable webcam eye tracking using user interactions. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 3839–3845. Pinker, S. (1989). Learnability and cognition: The acquisition of argument structure. Cambridge, MA: MIT Press. Pustejovsky, J. (1991). The syntax of event structure. Cognition, 41(1), 47–81. doi:10.1016/0010-0277(91)90032-Y R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https://www.R-project.org/ Rappaport Hovav, M., & Levin, B. (1998). Building verb meanings. In M. Butt & W. Geuder (Eds.), The projection of arguments: Lexical and compositional factors (pp. 97-134). Stanford, CA: CSLI Publications. 123 Rappaport Hovav, M., & Levin, B. (2008). The English dative alternation: The case for verb sensitivity. Journal of Linguistics, 44(1), 129–167. doi:10.1017/S0022226707004975 Rappaport Hovav, M., & Levin, B. (2010). Reflections on manner/result complementarity. In M. Rappaport Hovav, E. Doron, & I. Sichel (Eds.), Syntax, lexical semantics, and event Structure (pp. 21-38). Oxford University Press, Oxford, UK. Regier, T. (1996). The Human Semantic Potential: Spatial Language and Constrained Connectionism. MIT Press. Regier, T., & Zheng, M. (2007). Attention to Endpoints: A Cross-Linguistic Constraint on Spatial Meaning. Cognitive Science, 31(4), 705–719. doi:10.1080/15326900701399954 Reichenbach, H. (1947). Elements of Symbolic Logic. London: Dover Publications. Roberts, C. (1996/2012). Information structure in discourse: Towards an integrated theory of pragmatics. Semantics and Pragmatics, 6, 1-69. doi:10.3765/sp.5.6. (Original work published 1996 in Ohio State University Working Papers in Linguistics, 49) Rohde, H., Kehler, A., Elman, J. L. (2006). Event structure and discourse coherence biases in pronoun interpretation. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 28, No. 28). Sakarias, M., & Flecken, M. (2019). Keeping the Result in Sight and Mind: General Cognitive Principles and Language-Specific Influences in the Perception and Memory of Resultative Events. Cognitive Science, 43(1), e12708. doi:10.1111/cogs.12708 Sanford, A. J., & Garrod, S. C. (1998). The role of scenario mapping in text comprehension. Discourse Processes, 26(2–3), 159–190. doi:10.1080/01638539809545043 Schoubye, A. J. (2010). Descriptions, truth value intuitions, and questions. Linguistics and Philosophy, 32(6), 583-617. doi:10.1007/s10988-010-9069-y 124 Simons, M., Tonhauser, J., Beaver, D., & Roberts, C. (2010). What projects and why. Proceedings of SALT, 20, 309-327. doi:10.3765/salt.v20i0.2584 Slim, M. S., & Hartsuiker, R. (2021). Visual world eyetracking using WebGazer.js. PsyArXiv. doi:10.31234/osf.io/5adgf Solomon, S. H., Hindy, N. C., Altmann, G. T. M., & Thompson-Schill, S. L. (2015). Competition between mutually exclusive object states in event comprehension. Journal of Cognitive Neuroscience, 27(12), 2324-2338. doi:10.1162/jocn_a_00866 Sturt, P., Sanford, A. J., Stewart, A., & Dawydiak, E. (2004). Linguistic focus and good- enough representations: An application of the change-detection paradigm. Psychonomic Bulletin and Review, 11(5), 882-888. doi:10.3758/BF03196716 Svartvik, J. (1966). On voice in the English verb. The Hague: Mouton. Talmy, L. (1991). Path to realization: A typology of event conflation. Proceedings of the Annual Meeting of the Berkeley Linguistics Society, 17. 480-519. doi: 10.3765/bls.v17i0.1620 Talmy, L. (2000). Towards a cognitive semantics II. Cambridge, MA: MIT Press. Tian, Y., Breheny, R., & Ferguson, H. (2010). Why we simulate negated information: A dynamic pragmatic account. Quarterly Journal of Experimental Psychology, 63, 2305-2312. doi: 10.1080/17470218.2010.525712 Trueswell, J. C., & Tanenhaus, M. K. (1994). Toward a lexicalist framework of constraint- based syntactic ambiguity resolution. In Perspectives on sentence processing (pp. 155–179). Lawrence Erlbaum Associates, Inc. Trueswell, J. C., Tanenhaus, M. K., & Garnsey, S. M. (1994). Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution. Journal of Memory and Language, 33(3), 285–318. doi:10.1006/jmla.1994.1014 125 Trueswell, J. C., Tanenhaus, M. K., & Kello, C. (1993). Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(3), 528-553. doi:10.1037/0278-7393.19.3.528 Umbach, C. (2005). Contrast and information structure: A focus-based analysis of but. Linguistics, 43(1), 207-232. doi:10.1515/ling.2005.43.1.207 Ünal, E., Ji, Y., & Papafragou, A. (2021). From Event Representation to Linguistic Meaning. Topics in Cognitive Science, 13(1), 224–242. doi:10.1111/tops.12475 Van Dijk, T.A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. Velupillai, V. (2012). An Introduction to Linguistic Typology. John Benjamins Publishing. Vendler, Z. (1957). Verbs and times. The Philosophical Review, 66(2), 143-160. doi:10.2307/2182371 Wellwood, A., Hespos, S. J., & Rips, L. (2018a). The object : substance :: event : process analogy. In T. Lombrozo, J. Knobe, & S. Nicholas (Eds.), Oxford Studies in Experimental Philosophy, Volume II (pp. 183-212). Oxford University Press. Wellwood, A., Hespos, S. J., & Rips, L. (2018b). How similar are objects and events? Acta Linguistica, 15(2-3), 473-501. doi:10.1556/2062.2018.65.2-3.9 Wittek, A. (2002). Learning the meaning of change-of-state verbs: A case study of German child language. Berlin: Walter de Gruyter. Zehr, J., & Schwarz, F. (2018). PennController for Internet Based Experiments (IBEX). doi:10.17605/OSF.IO/MD832 Zondervan, A. (2009). Experiments on QUD and focus as a contextual constraint on scalar implicature calculation. In U. Sauerland & K. Yatsushiro (Eds.), Semantics and 126 pragmatics: From experiment to theory (pp. 94-112). Basingstoke: Palgrave Macmillan. Zondervan, A. (2010). The role of QUD and focus on the scalar implicature of most. In J. Meibauer & M. Steinbach (Eds.), Experimental semantics/pragmatics (pp. 221-238). Amsterdam: John Benjamins. Zondervan, A., Meroni, L., & Gualmini, A. (2008). Experiments on the role of the Question under Discussion for ambiguity resolution and implicature computation in adults. Proceedings of SALT, 18, 765-777. doi:10.3765/salt.v18i0.2486 127 Appendix A. Target stimuli for Experiment 1 1. Talking about {Trevor/the watermelon}: “Trevor {will whack/whacked} the watermelon.” Test word: cracked 2. Talking about {Susie/the metal}: “Susie {will pound/pounded} the metal.” Test word: bent 3. Talking about {Justin/the gold}: “Justin {will hammer/hammered} the gold.” Test word: deformed 4. Talking about {Brooke/the glass}: “Brooke {will kick/kicked} the glass.” Test word: fragmented 5. Talking about {Alex/the window}: “Alex {will strike/struck} the window.” Test word: shattered 6. Talking about {Tiffany/the boxing bag}: “Tiffany {will batter/battered} the boxing bag.” Test word: torn 128 7. Talking about {Josh/the balloon}: “Josh {will poke/poked} the balloon.” Test word: popped 8. Talking about {Diana/the wall}: “Diana {will punch/punched} the wall.” Test word: dented 9. Talking about {Andy/the tv}: “Andy {will smack/smacked} the tv.” Test word: damaged 10. Talking about {Sally/the sand castle}: “Sally {will tap/tapped} the sand castle.” Test word: crumbled 11. Talking about {Greg/the beanbag}: “Greg {will hit/hit} the beanbag.” Test word: ripped 12. Talking about {Michelle/the can}: “Michelle {will slam/slammed} the can.” Test word: crushed 129 13. Talking about {Arthur/the balloon}: “Arthur {will poke/poked} the balloon.” Test word: popped 14. Talking about {Margaret/the beach ball}: “Margaret {will squeeze/squeezed} the beach ball.” Test word: flattened 15. Talking about {Tim/the dough}: “Tim {will pull/pulled} the dough.” Test word: stretched 16. Talking about {Jenna/the cord} “Jenna {will yank/yanked} the cord.” Test word: broken 17. Talking about {Jim/the pottery}: “Jim {will nudge/nudged} the pottery.” Test word: chipped 18. Talking about {Molly/the boxer}: “Molly {will jab/jabbed} the boxer.” Test word: wounded 19. Talking about {Marilyn/the bottle}: 130 “Marilyn {will bang/banged} the bottle.” Test word: smashed 20. Talking about {Jason/the opponent}: “Jason {will tackle/tackled} the opponent.” Test word: hurt 21. Talking about {Stephen/the vase}: “Stephen {will knock/knocked} the vase.” Test word: fractured 22. Talking about {Rachel/the eggs}: “Rachel {will beat/beat} the eggs.” Test word: foamed 23. Talking about {Brandon/the porcelain figurine}: “Brandon {will push/pushed} the porcelain figurine.” Test word: smashed 24. Talking about {Ashley/the donut}: “Ashley {will prod/prodded} the donut.” Test word: squashed 131 Appendix B. Target stimuli for Experiment 2. (The two extra items that are not part of the core 2x2 design are items 33 and 34, marked with *) 1. Trevor called and asked Mary what happened to the merick/about the merick. She replied that she hit/broke it. She said that it is damaged and that she feels very sorry about this. 2. Susie called and asked John what happened to the jolf/about the jolf. He answered that he whacked/crashed it. He said that it is broken and that he feels quite sad. 3. Justin called and asked Karen what happened to the zat/about the zat. She responded that she pounded/shattered it. She said that it is in pieces and that she feels rather regretful. (“in pieces” was presented together) 4. Brooke called and asked David what happened to the kiff/about the kiff. He replied that he hammered/chipped it. He said that it is damaged and that he feels so glad. 5. Alex called and asked Sarah what happened to the cheel/about the cheel. She answered that she kicked/smashed it. She said that it is shattered and that she feels very ashamed. 6. Tiffany called and asked Mark what happened to the cag/about the cag. 132 He responded that he struck/cracked it. He said that it is fractured and that he feels quite glad. 7. Josh called and asked Lisa what happened to the carn/about the carn. She replied that she battered/crushed it. She said that it is deformed and that she feels rather tired. 8. Diana called and asked Michael what happened to the himp/about the himp. He answered that he thumped/damaged it. He said that it is cracked and that he feels so frustrated. 9. Andy called and asked Maria what happened to the risp/about the risp. She responded that she poked/fractured it. She said that it is cracked and that she feels very apologetic. 10. Sally called and asked Thomas what happened to the sporf/about the sporf. He replied that he slapped/harmed it. He said that it is injured and that he feels quite embarrassed about what he did. 11. Greg called and asked Kelly what happened to the choop/about the choop. She answered that she punched/splintered it. She said that it is broken and that she feels rather upset. 12. Michelle called and asked Jack what happened to the dwag/about the dwag. He responded that he smacked/split it. 133 He said that it is halved and that he feels so guilty. 13. Arthur called and asked Susan what happened to the sheg/about the sheg. She replied that she tapped/snapped it. She said that it is broken and that she feels very shocked. 14. Margaret called and asked Peter what happened to the mert/about the mert. He answered that he pulled/ripped it. He said that it is torn and that he feels quite upset. 15. Tim called and asked Jennifer what happened to the mipper/about the mipper. She responded that she yanked/tore it. She said that it is ripped and that she feels rather indifferent. 16. Jenna called and asked Roger what happened to the ning/about the ning. He replied that he shoved/broke it. He said that it is in pieces and that he feels so sorry. (“in pieces” was presented together) 17. Jim called and asked Jessica what happened to the crail/about the crail. She answered that she jabbed/pierced it. She said that it is punctured and that she feels very responsible. 18. Molly called and asked Patrick what happened to the shemp/about the shemp. He responded that he poked/punctured it. He said that it is pierced and that he feels quite happy. 134 19. Jason called and asked Sandra what happened to the vulper/about the vulper. She replied that she squeezed/burst it. She said that it is popped and that she feels rather content. 20. Marilyn called and asked Ryan what happened to the plorf/about the plorf. He answered that he bit/popped it. He said that it is hurt and that he feels so bad. 21. Stephen called and asked Katie what happened to the jum/about the jum. She responded that she pounded/ground it. She said that it is crushed and that she feels very pleased. 22. Rachel called and asked Bill what happened to the alf/about the alf. He replied that he chewed/ate it. He said that it is all gone and that he feels quite regretful. (“all gone” was presented together) 23. Brandon called and asked Monica what happened to the weam/about the weam. She answered that she slurped/drank it. She said that it is all gone and that she feels rather apologetic. (“all gone” was presented together) 24. Ashley called and asked Tony what happened to the plass/about the plass. He responded that he pressed/squished it. 135 He said that it is deformed and that he feels so bad. 25. Paul called and asked Emily what happened to the blicket/about the blicket. She replied that she scratched/etched it. She said that it is rough and that she feels very dissatisfied with it. 26. Hillary called and asked Brian what happened to the falks/about the falks. He answered that he stirred/creamed them. He said that they are blended and that he feels quite content. 27. Luis called and asked Nicole what happened to the ligots/about the ligots. She responded that she gathered/merged them. She said that they are together and that she feels rather tired from all the work. 28. Natalie called and asked Austin what happened to the harsts/about the harsts. He replied that he glued/fused them. He said that they are connected and that he feels so happy. 29. Shawn called and asked Haley what happened to the selps/about the selps. She answered that she lumped/unified them. She said that they are together and that she feels very proud of herself. 30. Caroline called and asked Dylan what happened to the tems/about the tems. He responded that he shook/combined them. He said that they are blended and that he feels quite exhausted. 136 31. Adam called and asked Emma what happened to the varns/about the varns. She replied that she shuffled/mixed them. She said that they are randomized and that she feels rather overworked. 32. Madison called and asked Robert what happened to the blims/about the blims. He answered that he stirred/blended them. He said that they are mixed and that he feels so pleased. 33*. Daniel called and asked Olivia what happened to the croom/about the croom. She responded that she washed/cleaned it. She said that it is spotless and that she feels very proud. 34*. Hannah called and asked Andrew what happened to the flamp/about the flamp. He replied that he wiped/cleared it. He said that it is shiny and that he feels quite satisfied. 137 Appendix C. Target stimuli for Experiment 3 pre-image text unlikely to undergo state change likely to undergo state change post-image text 1 Sarah {whacked/was whacking} the in the messy bedroom. 2 Andre {punched/was punching} the near the cement driveway. 3 Nicole {punched/was punching} the in the noisy apartment. 4 Carlos {kicked/was kicking} the on the wooden patio. 5 Lindsay {kicked/was kicking} the in the cramped garage. 138 6 Ethan {stomped/was stomping} on the at the peaceful park. 7 Katie {nudged/was nudging} the at the public hospital. 8 Gordon {pushed/was pushing} the at the huge playground. 9 Jasmine {scratched/was scratching} the at the local store. 10 Howard {struck/was striking} the near the front yard. 11 Helen {stomped/was stomping} on the in the modern bedroom. 139 12 Ivan {dropped/was dropping} the at the busy bar. 13 Gina {dropped/was dropping} the at the large supermarket. 14 Jose {bit/was biting} the at the expensive buffet. 15 Ellie {bit/was biting} the at the quiet restaurant. 16 Kevin {threw/was throwing} the in the spacious kitchen. 140 17 Connie {threw/was throwing} the on the bustling street. 18 Louis {bent/was bending} the at the crowded school. 19 Chloe {yanked/was yanking} on the in the tiny hotel. 20 Michael {hammered/was hammering} the in the dirty kitchen. 21 Bridget {stepped/was stepping} on the in the department store. 22 Peter {yanked/was yanking} on the in the crafts store. 141 23 Audrey {hit/was hitting} the at the new studio. 24 Trevor {struck/was striking} the at the lively party. 142 Appendix D. Target stimuli for Experiment 4 pre-image text unlikely to undergo state change likely to undergo state change post-image text 1 Sarah {whacked/will whack} the in the messy bedroom. 2 Andre {punched/will punch} the near the cement driveway. 3 Nicole {punched/will punch} the in the noisy apartment. 4 Carlos {kicked/will kick} the on the wooden patio. 5 Lindsay {kicked/will kick} the in the cramped garage. 143 6 Ethan {stomped/will stomp} on the at the peaceful park. 7 Katie {nudged/will nudge} the at the public hospital. 8 Gordon {pushed/will push} the at the huge playground. 9 Jasmine {scratched/will scratch} the at the local store. 10 Howard {struck/will strike} the near the front yard. 144 11 Helen {stomped/will stomp} on the in the modern bedroom. 12 Ivan {dropped/will drop} the at the busy bar. 13 Gina {dropped/will drop} the at the large supermarket. 14 Jose {bit/will bite} the at the expensive buffet. 15 Ellie {bit/will bite} the at the quiet restaurant. 16 Kevin {threw/will throw} the in the spacious kitchen. 145 17 Connie {threw/will throw} the on the bustling street. 18 Louis {bent/will bend} the at the crowded school. 19 Chloe {yanked/will yank} on the in the tiny hotel. 20 Michael {hammered/will hammer} the in the dirty kitchen. 21 Bridget {stepped/will step} on the in the department store. 146 22 Peter {yanked/will yank} on the in the crafts store. 23 Audrey {hit/will hit} the at the new studio. 24 Trevor {struck/will strike} the at the lively party. 147 Appendix E. Target stimuli for Experiment 5 Imperfective + Double object Imperfective + to - Perfective + Double object Perfective + to - 1 Lauren was handing Patrick the ball. Lauren was handing the ball to Patrick. Lauren handed Patrick the ball. Lauren handed the ball to Patrick. 2 Aaron was handing Sarah the ball. Aaron was handing the ball to Sarah. Aaron handed Sarah the ball. Aaron handed the ball to Sarah. 3 Helen was handing Travis the ball. Helen was handing the ball to Travis. Helen handed Travis the ball. Helen handed the ball to Travis. 4 Daniel was handing Samantha the ball. Daniel was handing the ball to Samantha. Daniel handed Samantha the ball. Daniel handed the ball to Samantha. 5 Hannah was giving Jack the ball. Hannah was giving the ball to Jack. Hannah gave Jack the ball. Hannah gave the ball to Jack. 6 Tyler was giving Claire the ball. Tyler was giving the ball to Claire. Tyler gave Claire the ball. Tyler gave the ball to Claire. 7 Charlotte was giving David the ball. Charlotte was giving the ball to David. Charlotte gave David the ball. Charlotte gave the ball to David. 8 Marco was giving Christina the ball. Marco was giving the ball to Christina. Marco gave Christina the ball. Marco gave the ball to Christina. 9 Vivian was bringing Ross the ball. Vivian was bringing the ball to Ross. Vivian brought Ross the ball. Vivian brought the ball to Ross. 10 Ethan was bringing Gina the ball. Ethan was bringing the ball to Gina. Ethan brought Gina the ball. Ethan brought the ball to Gina. 11 Rebecca was bringing Howard the ball. Rebecca was bringing the ball to Howard. Rebecca brought Howard the ball. Rebecca brought the ball to Howard. 12 Luke was bringing Sophia the ball. Luke was bringing the ball to Sophia. Luke brought Sophia the ball. Luke brought the ball to Sophia. 13 Megan was throwing Derek the ball. Megan was throwing the ball to Derek. Megan threw Derek the ball. Megan threw the ball to Derek. 14 Charles was throwing Monica the ball. Charles was throwing the ball to Monica. Charles threw Monica the ball. Charles threw the ball to Monica. 15 Ashley was throwing Drew the ball. Ashley was throwing the ball to Drew. Ashley threw Drew the ball. Ashley threw the ball to Drew. 16 Isaac was throwing Valerie the ball. Isaac was throwing the ball to Valerie. Isaac threw Valerie the ball. Isaac threw the ball to Valerie. 17 Beth was kicking Scott the ball. Beth was kicking the ball to Scott. Beth kicked Scott the ball. Beth kicked the ball to Scott. 18 Nick was kicking Sally the ball. Nick was kicking the ball to Sally. Nick kicked Sally the ball. Nick kicked the ball to Sally. 19 Paula was kicking Matthew the ball. Paula was kicking the ball to Matthew. Paula kicked Matthew the ball. Paula kicked the ball to Matthew. 20 William was kicking Michelle the ball. William was kicking the ball to Michelle. William kicked Michelle the ball. William kicked the ball to Michelle. 21 Sabrina was tossing Noah the ball. Sabrina was tossing the ball to Noah. Sabrina tossed Noah the ball. Sabrina tossed the ball to Noah. 22 Christian was tossing Emily the Christian was tossing the ball to Emily. Christian tossed Emily the ball. Christian tossed the ball to Emily. 148 ball. 23 Natalie was tossing Adam the ball. Natalie was tossing the ball to Adam. Natalie tossed Adam the ball. Natalie tossed the ball to Adam. 24 Kevin was tossing Nicole the ball. Kevin was tossing the ball to Nicole. Kevin tossed Nicole the ball. Kevin tossed the ball to Nicole. 149 Appendix F. Experiment 5: Click data for all conditions (Error bars show +/- 1 SE) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 was giving GOAL the ball gave GOAL the ball was giving the ball to GOAL gave the ball to GOAL source sourceADJ center goalADJ goal 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 was throwing GOAL the ball threw GOAL the ball was throwing the ball to GOAL threw the ball to GOAL source sourceADJ center goalADJ goal 150 Appendix G. Experiment 5: Eye gaze data in imperfective and perfective conditions 151 Appendix H. Experiment 5: Eye gaze data in imperfective + double object, perfective + double object, imperfective + to-, and perfective + to- conditions 152 153 Appendix I. Experiment 5: The time course of GOAL-SOURCE difference scores in imperfective aspect trials and in perfective aspect trials

Abstract (if available)

Abstract This dissertation explores the question of how the language comprehension system constructs mental representations of events based on linguistic descriptions of events. When a comprehender encounters linguistic input describing an event, what information guides how a mental representation of that event is constructed? How is information from various linguistic and non-linguistic sources integrated to create an understanding of an event? I specifically focus on the dimension of dynamic changes occurring to objects in an event. ❧ I report five experiments that were designed to investigate how comprehenders map linguistic input onto mental representations of events during language processing. I investigate three grammatical and non-grammatical factors: (a) grammatical properties of event descriptions (e.g. verb semantics, grammatical aspect, tense), (b) discourse-level properties, and (c) real-world knowledge about physical events. ❧ Chapters 2 and 3 investigate the question of how discourse-level and verb-level cues interact to guide the construction of object state representations. Chapter 2 reports a lexical decision experiment that investigates how verb tense information interacts with discourse- level cues about which event participant is being discussed (Experiment 1). Chapter 3 reports a self-paced reading experiment that investigates how verb type (result verb vs. manner verb) information interacts with discourse-level cues about whether the object’s resultant state is under discussion (Experiment 2). In both experiments, I find an interaction between discourse-level information and verb-level information in guiding object state representations. The findings highlight the need to take into account discourse-level factors in theorizing about the cognitive process of understanding the dynamics of event representation during language comprehension. ❧ Chapter 4 reports two rebus paradigm experiments that investigate how grammatical cues (Experiment 3: grammatical aspect, Experiment 4: tense) about an event’s temporal structure interact with real-world knowledge about the likelihood of state change (e.g. dropping a wine glass vs. dropping a plastic cup). Results from both experiments suggest that grammatical cues about an event’s temporal structure are rapidly integrated with non- linguistic information about the real-world to influence object state representations. ❧ Chapter 5 reports a visual-world eye-tracking study that investigates how different types of grammatical cues (grammatical aspect, verb type, argument realization patterns) influence the real-time comprehension of transfer events (Experiment 5). This experiment looks at the representation of an object’s change-of-location. Results from Experiment 5 show that comprehenders use different grammatical cues to shape their representation of object location, as the sentence unfolds in real-time. ❧ Taken together, this dissertation research sheds light on the psycholinguistic processes involved in recruiting information from both grammatical and non-grammatical sources to dynamically update comprehenders’ event representations. First, it highlights the role of grammatical factors in building event representations during sentence processing. The temporal domain of event representations – the temporal contour or the temporal perspective – is a central component of event representations. Second, this work also motivates the need for theories of event representation to incorporate and account for discourse-level information and real-world knowledge about physical events and objects as meaningful components.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Exploring the effects of Korean subject marking and action verbs’ repetition frequency: how they influence the discourse and the memory representations of entities and events

PDF

Discourse-level processing of nominal possessive constructions

PDF

When things are left unsaid: existential and anaphoric implicit objects in discourse

PDF

Dynamics of multiple pronoun resolution

PDF

The morphosyntax of states: deriving aspect and event roles from argument structure

PDF

Discourse level processing and pronoun interpretation

PDF

Syntactic and non-syntactic factors in reflexive pronoun resolution in Mandarin Chinese

PDF

The grammar of individuation, number and measurement

PDF

What 'you' and 'I' can say about reference resolution and non-structural constraints

PDF

Considering the effects of disfluent speech on children’s sentence processing capabilities and language development

PDF

Schema architecture for language-vision interactions: a computational cognitive neuroscience model of language use

PDF

Narrowing the focus: experimental studies on exhaustivity and contrast

PDF

Comparative iIlusions at the syntax-semantics interface

PDF

Interaction between prosody and information structure: experimental evidence from Hindi and Bangla

PDF

Syntax-prosody interactions in the clausal domain: head movement and coalescence

PDF

Interpretation of pronominal forms across languages

PDF

Silence in answers: a study of ellipsis in Hindi

PDF

Vietnamese pronouns in discourse

PDF

The prosodic substrate of consonant and tone dynamics

PDF

Linking eyes to mouth: a schema-based computational model for describing visual scenes

Asset Metadata

Creator Lee, Sarah Hye-yeon (author)

Core Title Processing the dynamicity of events in language

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Linguistics

Degree Conferral Date 2022-05

Publication Date 02/01/2022

Defense Date 12/06/2021

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag change-of-state,event cognition,event structure,language processing,OAI-PMH Harvest,psycholinguistics,sentence comprehension

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Kaiser, Elsi (committee chair), Pancheva, Roumyana (committee member), Rudin, Deniz (committee member), Wellwood, Alexis (committee member)

Creator Email hyeyeon.sarah.lee@gmail.com,sarahhl@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC110582766

Unique identifier UC110582766

Legacy Identifier etd-LeeSarahHy-10368

Document Type Dissertation

Format application/pdf (imt)

Rights Lee, Sarah Hye-yeon

Type texts

Source 20220201-usctheses-batch-910 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu