University of Southern California Dissertations and Theses

Temporal dynamics of attention: attention gating in rapid serial visual presentation

(USC Thesis Other)

Temporal dynamics of attention: attention gating in rapid serial visual presentation

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

Temporal Dynamics of Attention:
Attention Gating in Rapid Serial Visual Presentation

by

Yukai Zhao

__________________________________________________________________

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(NEUROSCIENCE)

May 2016

Copyright 2016 Yukai Zhao

II
Acknowledgements
My gratitude to my advisor, Zhong-Lin Lu, is beyond words. Zhong-Lin has patiently guided me
through the Ph.D. training, while giving me great autonomy to be myself and to learn from my
own mistakes. His enthusiasm towards science and his support for me has always inspired me to
strive for better. I owe as much thanks to the other main collaborator, Barbara Dosher, for the
development of this dissertation. I would like to thank members in my qualification and/or
dissertation committee: Irving Biederman, Norberto Grzywacz, Judith Hirsch, Laurent Itti and
Bosco Tjan. I especially thank Bosco Tjan for his intellectual support, for his generosity and for
sharing his lab space. Special thanks to Michael Dawson and Jason Zevin for sharing their office
space, without which my final stage of dissertation writing would not be as enjoyable. Thanks to
former and current LOBEs lab members: Jongsoo Baek, Carlos Cabrera, Fang Hou, Chang-Bing
Huang, Jiajuan Liu and Miao Wei. Thanks to all my friends at USC, who have made my
experience more memorable in the past few years. Last but not least, thank my family for their
unconditional love.

III

Table of Contents
Acknowledgements II

Table of Contents III

Abstract VI

Overview IX

Chapter One: Introduction 1

1. Temporal dynamics of attention: Empirical findings 1
1.1. Simultaneous presentation 1
1.1.1. Cue-target onset asynchrony (CTOA) 1
1.1.1.1. Spatial attention 2
1.1.1.2. Feature-based attention 2
1.1.2. Signal response speed-accuracy trade-off (SAT) 3
1.1.3. Visual search 4
1.2. Sequential presentation 6
1.2.1. Minimum sequence: two items 6
1.2.1.1. Attentional dwell time (AD) 6
1.2.1.2. Temporal discrimination 7
1.2.2. Multiple items 8
1.2.2.1. One at a time presented in different spatial locations 8
1.2.2.2. Rapid serial visual presentation (RSVP) at one spatial location 9
1.2.2.3. Multiple RSVP streams in separated spatial locations 11
1.2.2.4. Measuring reaction time of covert visual attention shift: Attention
reaction time (ART) paradigm 12

2. Temporal dynamics of attention: Attention gating model 15
2.1 Attention gating in visual short-term memory(VSTM) 15
2.1.1. Attention gating model 16
2.1.2. Applications of AGM 16

3. Un-answered questions of RSVP studies 18

4. New approach: the external noise plus attention reaction time paradigm and
attention gating perceptual template model (agPTM) 19
4.1. External noise method 19
4.1.1. The equivalent input noise method 20
4.2. Observer models 22
4.2.1. Attentional effects on perception: the Perceptual Template Model (PTM) 22

5. Significance 24

IV

Chapter Two: Measuring the Temporal Dynamics of Attention in the External Noise Plus
Attention Reaction Time Paradigm 26

1. Method 27
1.1. Apparatus 27
1.2. Stimuli 27
1.2.1. Signal (letter) frame 27
1.2.1.1. Training, Experiment 1 and Experiment 2B 27
1.2.1.2. Experiment 2A 28
1.2.2. External noise frame 28
1.2.2.1. Training, Experiment 1 and Experiment 2B 28
1.2.2.2. Experiment 2A 28
1.2.3. Cue 28
1.3. Procedure 28
1.4. Tasks 29
1.5. Feedback 29
1.6. Design 30
1.7. Statistical analysis 31

2. Results 32
2.1. Experiment 1 32
2.1.1. Item scores, P
i
(r) 32
2.1.2. Composite item scores, P
i
33
2.1.3. Order scores, P
iBj
34
2.1.4. Individual subjects 35
a. YZ 35
b. KS 36
c. BF 36
2.2. Experiment 2 37
2.2.1 Individual data in Experiment 2A 37
a. YZ 37
b. KS 40
c. BF 42
2.2.2. Individual data in Experiment 2B 44
a. YZ 44
b. KS 47

Chapter Three: Attention Gating Perceptual Template Model (agPTM) 49

1. Attention gating perceptual template model (agPTM) 50
1.1. Step-by-Step description 50
1.1.1. Perceptual templates 50
1.1.2. Nonlinear transducer function 51
1.1.3. Pre-contrast-gain internal noise 51
1.1.4. Divisive contrast-gain control 51

V

1.1.5. Attention gate function 52
1.1.6. Decision 52
1.1.6.1 Discriminability 53
1.1.6.2 Memory strength 53
1.2. Qualitative illustrations of the function of each parameter 54
1.3. Qualitatively different predictions by the two strength models for Experiment 2: signal-to-
noise ratio vs. total energy (AGM) 56

2. Model evaluation 57
2.1. Fitting procedure 57
2.2. Fitting results 58
2.2.1. Joint fit of Experiment 1 and Experiment 2B 58
2.2.2. Experiment 1 59
2.2.3. Experiment 2A 59

3. Discussion 60
3.1. Alternative models 61
3.1.1. Alternative model 1 (AM 1) 61
3.1.1.1. Joint fit of Experiment 1 and Experiment 2B 61
3.1.1.2. Experiment 1 61
3.1.1.3. Experiment 2A 62
3.1.2. Alternative model 2 (AM 2) : PTM+AGM 63

Chapter Four: Conclusions and Discussions 64

1. Conclusions 64

2. Discussions 65

References 69
Appendix 1: Tables 78
Appendix 2: Figures 83

VI
Abstract
Temporal dynamics of attention have been a central topic in visual attention. The temporal
dynamics are inferred behaviorally from changes in performance, such as response accuracy
and/or reaction time, while systematically varying certain temporal aspect(s) of the experiments,
such as cue-target onset asynchrony (CTOA). Using rapid serial visual presentation (RSVP),
important temporal properties of attention have been found and theoretical interpretations and
models about these empirical results have been developed. Stimuli with high signal-to-noise
ratios are usually presented in noiseless displays in RSVP studies. However, attention has been
found to affect perception in stimulus conditions other than high signal-to-noise ratios. Theories
and models have also been developed to characterize and distinguish the underlying attentional
mechanisms. Therefore, it is essential to measure and quantify attentional effects on perception
by systematically manipulating stimulus signal-to-noise ratios at different levels of external
noise. Furthermore, models need to be developed to fully characterize and distinguish
mechanisms underlying the temporal dynamics of attention in RSVP.
A new paradigm, the external noise plus attention reaction time paradigm, is developed by
combining the attention reaction paradigm (ART, Reeves, 1977; Sperling and Reeves, 1980)
with the external noise method (Lu and Dosher, 1998). Using this paradigm, the time course of
attention cueing is systematically measured and quantified under a wide range of stimulus
conditions, including low signal contrast, high external noise, and un-equal item signal-to-noise
ratios. The task is to remember four letters immediately following a visual cue embedded in an
RSVP of letters at the fovea and to report them in order in the end of a trial. The temporal
dynamics of attention are quantified by report probabilities and report orders as a function of the
temporal positions of stimuli in the presentation sequence.
VII
In experiment 1, signal contrasts spanning a full range of performance levels were tested in both
zero and high external noise. Performance was better at higher signal contrasts at a given
external noise level, as shown by higher report probabilities and more consistent report orders of
the letters presented within the critical set (temporal positions close to the visual cue included in
the data analysis). Higher contrast was required in high external noise to achieve similar
performance level as in zero noise. In experiment 2A and 2B, signal-to-noise ratios of the input
stimuli were made unequal by adding external noise (phase-scrambled noise in experiment 2A
and 33% Gaussian noise in experiment 2B) to only one randomly picked letter out of the four
letters immediately following the cue onset. Compared to the report probability presented at the
same temporal position in the control condition without any added noise, the report probability
of the noise-perturbed letter decreased to chance level. Furthermore, the report probabilities of
letters presented in close temporal proximity to the noise-perturbed letter changed systematically.
In order to account for these new results, a new model, the attention gating perceptual template
model (agPTM; Zhao et al., 2014), was developed based on the perceptual template model
(PTM; Lu and Dosher, 1998) and the attention gating model (AGM; Reeves and Sperling, 1986).
In the agPTM, both signal and noise of input stimuli go through a contrast-gain control process
(PTM; Lu and Dosher, 1998; Dao et al., 2006) and then are gated by attention (AGM; Reeves
and Sperling, 1986). The attention gated outputs determine the discriminability of the input
stimuli and the report order. The report order is determined by an order strength, which is
hypothesized to be proportional to either the signal-to-noise ratio or the total energy of the gated
output.
The data in experiment 1 and 2B were first fit jointly with a single set of parameters for each
subject. The strength model of signal-to-noise ratio well accounted for all the data. The strength
VIII
model of total energy failed to predict the systematic changes in report probabilities due to the
added external noise in experiment 2B. The data of experiment 1 and experiment 2A were also
fit separately with one set of parameters in each experiment for each subject. The data in
experiment 1 were well accounted for by agPTM with a single set of parameters for each subject.
Given equal item signal-to-noise ratios in Experiment 1, predictions by the two strength models
are qualitatively and quantitatively similar. In experiment 2A and 2B, on the other hand, using
un-equal item signal-to-noise ratios, evidence was found for the strength model of signal-to-
noise ratio and against the model of total energy.
In conclusion, the temporal dynamics of attention cueing in RSVP at fovea were systematically
measured and quantified in a wide range of stimulus conditions by the external noise plus
attention reaction time paradigm. The agPTM was developed to unveil the underlying attentional
mechanisms. The attention gated contrast gain control process well accounted for the perceptual
process in all the experiments. The report order was found to be determined by the signal-to-
noise ratio, not the total energy, of the attention gated output by using un-equal input signal-to-
noise ratios in experiment 2A and 2B.

IX
Overview
Attention refers to the collective processes in which limited resources of brain are being
allocated to certain aspect of the overwhelming information available at any given time. In
vision, every waking moment with eyes open, light brings more information to the retina than the
brain can fully process simultaneously. Without overt eye movements, covert visual attention
(referred as attention for simplicity in this dissertation unless otherwise specified) selects task
relevant information for further analysis and filters out or attenuates other information not
relevant to the current task. Depending on the tasks, attention can be directed to different aspects
of visual scene, such as spatial locations (spatial attention), features (feature-based attention),
and objects (object-based attention). Effects of attentional selection manifest behaviorally as
differences in accuracy and/or reaction time in different attention states (review: eg. Carrasco,
2011).
One important aspect of attention is the diversity of the temporal dynamics of attention observed
in different paradigms and/or tasks in which qualitatively distinct and/or quantitatively different
patterns have been observed. These paradigms can be divided into two main categories:
simultaneous and sequential presentations of visual stimuli. When all stimuli are presented
simultaneously, the temporal dynamics of attention can be characterized and quantified by some
measurement of the behavioral performance (eg. accuracy) at different cue-target onset
asynchronies (CTOAs) or by speed-accuracy trade-off analysis (SAT analysis). CTOA
manipulations are used to quantify the time course of attention cueing, while SAT analysis are
usually applied to investigate the attentional effects on the speed of information accrual.
Temporal dynamics of attention can be further explored by presenting stimuli one after another
in one or more spatial locations to characterize how attention evolves over time and interacts
X
with dynamic inputs. In particular, the rapid serial visual presentation (RSVP) is a good way to
quantify the temporal dynamics of attention because (1) visual persistence is controlled by
presenting one stimulus after another at the same spatial location and (2) both accuracy and
latency are measured simultaneously, and therefore data are collected more efficiently than
CTOA manipulations and SAT analysis. By presenting multiple RSVP streams at different
spatial positions, the attention reaction time paradigm (ART: Reeves, 1977; Sperling and Reeves,
1980) was used to study the time course of attention cueing and shifts (Reeves and Sperling,
1986; Sperling and Weichselgartner, 1995) and was further employed to separate the attentional
effects on performance from the effects of the iconic memory decay in partial reports (Shih and
Sperling, 2002). All these results were well accounted for in a general framework based on the
attention gating model (Reeves and Sperling ,1986).
Since only high signal-to-noise-ratio visual stimuli in clear displays were used in these studies, it
leaves open the questions whether the temporal dynamics of attention are different and how
attention affects perception in other stimulus conditions with lower signal-to-noise ratios and/or
with added external noise in RSVP. Furthermore, models based on AGM have only been applied
to RSVP experiments with high item signal-to-noise ratios. AGM and models derived from
AGM do not provide possible attentional mechanisms on perception in RSVP when stimulus
contrast is lowered to threshold and/or external noise is added. However, different attentional
effects have been observed in different stimulus conditions with or without external noise.
Theoretical interpretations and mathematical models have been developed to distinguish the
underlying mechanisms of attention in paradigms in which all potential targets are presented
simultaneously (see Introduction). Therefore, it is essential to measure performance by
systematically manipulating signal-to-noise ratios and to develop models to fully characterize
XI
and identify the mechanisms underlying the temporal dynamics of attention in different stimulus
conditions in RSVP.
This dissertation measures the temporal dynamics of attention cueing in RSVP in a wide range of
stimulus condition with a novel paradigm, the external noise plus attention reaction time
paradigm, by combining the external noise method with the ART. A new model, the agPTM, is
developed to characterize and distinguish the underlying mechanisms of attention at the observer
level. Chapter 1 reviews the previous literature of the empirical results and the theoretical
interpretations of the temporal dynamics of attention. The unsolved questions in the literature of
temporal dynamics of attention in RSVP are briefly discussed. Goals and strategies of the
dissertation are presented. Chapter 2 introduces the external noise plus attention reaction time
paradigm and presents the new results of two experiments regarding the temporal dynamics of
attention cueing in RSVP at fovea. Chapter 3 presents the new model, the agPTM and the model
fitting results to the data of the two experiments in Chapter 2. Two alternative models are
considered. Chapter 4 summarizes the new empirical results and theoretical interpretations about
the temporal dynamics of attention cueing in RSVP. Limitations of the dissertation and the
potential applications of the new paradigm and the new model to other studies of temporal
dynamics of attention are also discussed.

1

Chapter One: Introduction
1. Temporal dynamics of attention: Empirical findings
1.1 Simultaneous presentation
When all stimuli are presented simultaneously, the temporal dynamics of attention can be
characterized and quantified in several ways. First, the time intervals between the attention cue
and the stimulus presentation (Cue-target onset asynchrony, CTOA) can be varied and
performance can be quantified at different CTOAs. Effects of varying CTOAs on the temporal
dynamics of spatial and feature-based attention are reviewed. Second, the temporal duration
between the stimulus presentation and the response can be manipulated and the accuracy of the
response can be quantified in relation to reaction time (speed-accuracy trade-off analysis, SAT).
Applications of SAT in studies of attention effects in visual search are reviewed.
1.1.1. Cue-target onset asynchrony (CTOA)
The attention cue often precedes the stimulus display and all stimuli are usually presented
simultaneously after the cue onset (but see iconic memory studies, eg. Sperling 1960). The cue-
target onset asynchrony (CTOA), the time interval between the onset of the cue and the onset of
the stimulus frame is manipulated. As CTOA increases, performance usually improves (higher
accuracy and/or shorter reaction time) until reaches asymptotic level (but see inhibition of return,
Posner et al., 1985). Temporal dynamics of attention are inferred from the changes in
performance (accuracy and/or reaction time) measured at different CTOAs. Better performance
at longer CTOA supports the notion that deployment of attention to the cued aspect of visual
inputs is not instantaneous.

2
1.1.1.1. Spatial attention
The temporal dynamics of spatial attention have been studied extensively at different CTOAs.
Endogenous voluntary attention directed by central symbolic cue reaches peak effectiveness
around CTOA of 300 ms, while exogenous involuntary attention elicited by peripheral non-
informative cue reaches asymptotic levels at about CTOA of 150 ms (Müller & Rabbitt, 1989;
Nakayama and Makeben ,1989). However, some studies (Eriksen and Collins, 1969; Lu et al.,
2009; Warner et al., 1990) found that central and peripheral cues improved performance with
similar time course. As Lu et al. (2009) pointed out that many factors might affect the time
course of spatial attention, such as stimulus durations, practice levels, etc. Nevertheless, both
endogenous and exogenous attention gradually improves behavioral performance as CTOA
increases.
1.1.1.2. Feature-based attention
Fewer studies have been done regarding the temporal dynamics of feature-based attention.
Performance improves with longer CTOA before reaching asymptotic levels (Adams and
Chambers, 2012; Liu et al., 2007). Interestingly, different feature cues evoked different temporal
dynamics of attention relative to spatial cues given the exact same stimuli. While color cues
showed greater attentional advantages over spatial cues (Adams and Chambers, 2012), motion
direction cues acted slower than the spatial attention did but improved performance to similar
extent at longer CTOAs (Liu et al., 2007). In both studies, however, spatial factors might also
contribute to the different time courses. In a detection task (O as the target) using iso-eccentric
(5.1° radius) Landolt stimuli, Adams and Chambers (2012) found that color cues were more
effective at increasing perceptual sensitivity than spatial cues at all CTOAs tested (100 ~
1000ms). However, when target stimuli were presented within 2° visual angle of spatial
3
attention spotlight, effects of color cues and attention cues were the same at all CTOAs. The
authors argued that the inferior performance with spatial cue was due to the inability of spatial
attention to maintain a “sufficiently broad focus (>2°)”. In a task to detect a possible speed
increment of coherently moving dots, Liu et al. (2007) found central spatial cueing (to attend to
dots on either the left or the right to the fixation) reached asymptotic effect on perceptual
sensitivity at CTOA of 300ms while direction cueing (to attend to dots moving either left or right
on both side of the fixation) reached the same asymptotic level at a longer CTOA of 500ms.
Since the only difference in the two cueing condition was the instruction, the authors argued
feature-based attention was slower than spatial attention. However, the slower time course could
also be partly due to the requirement to attend two different locations simultaneously in direction
cueing rather than the only one attended location in spatial cueing condition. Even though spatial
rather than feature factors might lead to the different time courses between feature-based and
spatial attention, magnitude of feature-based attention effects increase with CTOAs as spatial
attention.
1.1.2. Signal response speed-accuracy trade-off (SAT)
With only accuracy or reaction time as dependent variable, speed-accuracy trade-off might occur
and confound the interpretations of the measured temporal dynamics of attention. First, faster
reaction time could result from response bias rather than attentional effects on perception and
sensitivity. Secondly, lower accuracy could result from faster reaction time in which a decision is
made before enough information is accumulated. The speed of information accrual can be
reliably inferred from the speed-accuracy trade-off analysis of both accuracy and reaction time.
Speed-accuracy trade-off is one useful paradigm in quantifying dynamics of perceptual, memory
and other cognitive processes (see review: Wickelgren,1977; Dosher,1979). Among them,
4
signal response speed-accuracy tradeoff paradigm (referred to as SAT in this dissertation) has
been extensively used in attention research. In a typical SAT study of attention, the shortest
CTOA required to achieve the largest attentional effect is usually used for a given type of
attention cue and a response cue, and often an audio tone as the response cue presented at
different moments after the visual stimuli instructs subject to respond as soon as. Response time
is defined as the time interval between the onset of the stimuli and the onset of the response cue.
For each cueing condition, accuracy increases with response time and then reaches asymptote.
The rate at which performance (accuracy) reaches asymptote quantifies how fast information is
accumulated and processed. Effects of attention are reflected in differences in rates and/or
asymptotic accuracy.
1.1.3. Visual search
Visual search is another widely used paradigm in attention. There are two types of search,
feature search and conjunction search. In feature search a unique value in one feature dimension
defines the target (eg. a green square among red squares and red circles) and in conjunction
search unique values in two or more feature dimensions are required to jointly define the target
(eg. a green square among green circles and red squares). Earlier free-viewing studies (eg.
Treisman and Gelade, 1980) with unlimited presentation time found that reaction time was short
and did not change with set size in feature search but that reaction time was longer and increased
with set size in conjunction search. These reaction time data suggested that the easy feature
“pop-out” search evokes parallel processing and is pre-attentive while conjunction search is
serial and requires attention. However, the intrinsically serial nature of eye movements in these
free-viewing search paradigms might confound the interpretations. Furthermore, the pre-attentive
notion of feature search has been recently challenged (see review: Nakayama and Martini, 2011).
5
Time-limited presentations are another major display configuration used in visual search. The
duration of the stimulus presentation is usually brief enough to prevent any saccades. Temporal
dynamics of attention in visual search have been studied using both CTOA manipulations and
SAT analysis. Attention cueing improves performance in visual search and performance
becomes better as CTOA increases before reaching asymptotic levels (eg. see 1.1.1.1. spatial
attention, Nakayama and Makeben ,1989; 1.1.1.2. feature-based attention, Adams and
Chambers, 2012). Attention also increases processing rate in visual search. Using SAT analysis,
Carrasco and her colleagues found that exogenous covert attention accelerated the information
accrual in both feature and conjunction search (Carrasco and McElree, 2001) and that attention
increased the information processing speed uniformly across different eccentricities at both
parafoveal (4°) and peripheral (9°) (Carrasco et al., 2006), therefore did not compensate or
reverse the faster rate of information processing at peripheral (Carrasco et al., 2003). At iso-
eccentric locations of the visual field, however, exogenous covert attention sped up information
processing and eliminated the difference in processing speed across all iso-eccentric locations
(Carrasco et al., 2004). In another study (Giordano et al., 2009), effects of endogenous but not
exogenous attention on discriminability and rate of information accrual increased with cue
validity. Although recent empirical findings and modeling studies using SAT analysis suggest
that visual search within one eye fixation are parallel with unlimited capacity (McElree and
Carrasco, 1999; Dosher et al., 2004; Dosher et al., 2010), more research need to be done to
characterize and quantify what information attention selects and/or excludes during the parallel
processing of covert visual search tasks.

6
1.2. Sequential presentation
Temporal dynamics of attention can be further explored by presenting stimuli one after another
in one or more spatial locations in order to measure how attention selects temporally dynamic
inputs. Many different paradigms of sequential presentation have been developed, including two
items presented at one location (temporal discrimination) or two randomly selected locations
(attentional dwell time; prior entry; temporal discrimination), multiple items presented one at a
time in locations randomly selected without repetition (visual search), single rapid serial visual
presentation (RSVP) in the same location ( attention blink, prior entry) and multiple
synchronized RSVP streams (spatial attention shifts, attention selection in partial report and
whole report).
1.2.1. Minimum sequence: two items
1.2.1.1. Attentional dwell time (AD)
Duncan et al. (1994) proposed an attentional dwell time (AD) paradigm to address the question
whether attention acts serially or in parallel in visual search. Instead of presenting all stimuli
simultaneously, only two items with post-masks were presented one after another separated by a
time interval randomly selected from a set of pre-determined SOAs and at two unpredictable
spatial locations. Processing the first item interfered with processing the second one. In the first
experiment, when both items had to be identified, probability of correct identification of the
second item decreased with increasing SOA up to about 200ms and then increased and recovered
to similar performance level at longer SOA (1000ms) relative to when identification of only one
item was required. In the second experiment, a detection task was employed to adapt the
paradigm more closely to visual search. The target was a letter L and non-target distractors were
Ls rotated either clockwise or counter-clockwise. The target letter L occurred first or second with
7
equal probabilities. Similar AD time was found when the first item was a rotated L (non-target).
The authors concluded that attention could not be switched between items rapidly as proposed in
serial models of visual search (eg. Treisman & Gelade, 1980). However, post-masks are used in
AD but not in visual search. Delaying or eliminating the mask of the first item shortened the
attentional dwell time (Moore et al., 1996). Interruption masks (masks presented 90ms after the
stimuli offset) but not integration masks (masks presented simultaneously and at the same
location of the stimuli) induced the attentional dwell time (Brehaut et al., 1999). Habituation to a
fixed mask after practice attenuated the AD effects and eye movements also confounded the
attentional dwell time (Petersen and Kyllingsbæk, 2013).
1.2.1.2. Temporal discrimination
In temporal discrimination tasks, two stimuli are presented one after the other either in the same
spatial location (temporal gap detection: Chica and Christie, 2009; Yeshurun and Levy, 2003) or
two different locations (temporal order judgment: Hein et al., 2006; Stelmach and
Herdman,1991). The best-known attention effect on temporal order perception is the prior entry
described by Titchener (1908) as ‘‘the object of attention comes to consciousness more quickly
than the objects which we are not attending to”. Stelmach and Herdman (1991) provided one of
the first direct evidence for the attentional effects on temporal order perception. In a temporal
order judgment task, the perceived temporal order of the two dots (one to the left and the other to
the right of the fixation) was biased towards the dot on the attended side. Performance in
different attention states have also been compared when two stimuli were equally attended but
with different amount of attention across different cueing conditions. In a series of temporal
order judgment tasks, Hein et al. (2006) further showed that both exogenous attention elicited by
informative peripheral cue and endogenous attention elicited by non-informative fovea cue led to
8
lower accuracy and faster reaction time while endogenous attention elicited by informative fovea
cue led to higher accuracy and faster reaction time in valid cue condition than in invalid
condition. Prior entry has been observed within a single sensory modality such as vision, as well
as across two different modalities, but this is beyond the scope of this dissertation (see a recent
review: Spence and Parise, 2010)
Interestingly, attention has been shown to impair temporal discrimination when two stimuli were
presented in the same spatial position one after the other. In a temporal gap detection task
(Yeshurun and Levy, 2003), temporal discriminability was higher in neutral cue condition than
in 100% valid cue condition which suggests that transient exogenous attention degrades temporal
resolution. In the study of Chica and Christie (2009), combining SAT analysis with a similar
paradigm as Yeshurun and Levy (2003), exogenous attention improved temporal discrimination
in valid than in invalid cue condition and performance was better in neutral condition than
performance in both valid and invalid conditions, consistent with Yeshurun and Levy (2003).
1.2.2. Multiple items
1.2.2.1. One at a time presented in different spatial locations
Saarinen and Julesz (1991) measured the speed of attention shifts in space by presenting one to
four randomly selected (without replacement) numerals (30ms each) with post-stimulus masks
(30ms each) one at a time in randomly selected locations without replacement (12 possible
locations in an imaginary circle of 1.5°). The post-stimulus mask of an item was presented
simultaneously at the onset of the following item. The task was to identify all letters in order.
Identification accuracy decreased with presentation rate (shorter SOAs) and number of numerals
presented. Performance was above chance level even with presentation rate of 33ms per numeral.
Accuracy of temporal order was worse than identification. Hung et al. (1995) used the same
9
paradigm with an additional simultaneous condition in which all four numbers were presented at
once. Performance in simultaneous presentation was better than in sequential conditions in all
number length (1~4) and presentation rates tested. Given the better performance in simultaneous
condition, the author concluded that the factor(s) limiting performance (bottleneck) must occur
before short-term memory but the exact cause of the difference was not addressed in their study.
Although the paradigms of these two studies (Saarinen and Julesz, 1991; Hung et al., 1995)
resemble those used in AD experiments regarding the sequential presentation at randomly
selected spatial locations, one critical difference is that SOAs between the two stimuli in AD
were randomly selected trial by trial while the presentation rates of the former two studies did
not change within a block. Therefore, the temporal uncertainty of the onset of the second item
could contribute to the impaired performance in AD. On the other hand, keeping the presentation
rate constant within a block of trials eliminated the temporal uncertainty and thus the AD deficit.
These differences in paradigms might have evoked mechanisms with different temporal
dynamics.
1.2.2.2. Rapid serial visual presentation (RSVP) at one spatial location
Presenting visual stimuli one after another at the same spatial location (RSVP) limits the visual
persistence of each item on the retina to the physical duration of the screen presentation and can
characterize and quantify the temporal properties of the information processing in the visual
system more efficiently than SAT analysis. Presentation rate of an RSVP is determined by the
SOA of two sequentially presented stimuli and is often expressed as number of items presented
per second. Rates of RVSP streams are selected high enough to fully engage attention, but not
too fast (about 20/s) to blur perception of each individual image due to limited temporal
resolution of the visual system (eg. Reeves and Sperling, 1986). Early studies have demonstrated
10
that both simple stimuli such as letters (Haber and Nathanson, 1969; Kolers and Katzman, 1966;
Travers, 1973) and digits (Norman, 1966) and complex images such as pictures of natural
outdoor scene, animals, people, etc. (Potter and Levy, 1969) could be processed quickly in RSVP
streams. Performance deteriorated with increasing presentation rate. The probability of recalling
a letter did not vary with different ratios between stimulus-on time and stimulus-off time (blank
interval) given a fixed presentation rate (Haber and Nathanson, 1969). Also the probability of
recognizing a picture was proportional to the presentation rate of that particular picture and
independent of the presentation rates of other pictures in the same RSVP stream (Potter and
Levy, 1969). These two studies indicate that the critical determinant of the performance in RSVP
is the SOA between the onset of one stimuli and the onset of the one immediately following it
and that each stimulus is being processed for the duration of the SOA, not only the physical
presentation of the stimulus but the blank interval before the onset of the next item as well.
Using a single RSVP stream at one spatial position, some interesting phenomenon regarding the
temporal dynamics of attention have been discovered, characterized, quantified and modeled.
Attentional blink (AB) has been extensively studied in which detection and identification of the
first target interferes with the detection and/or identification of the second target, if the second
target is presented within 500ms after the first target and at least one distractor is presented
between the two targets (Broadbent and Broadbent, 1987; Raymond et al., 1992; a recent review,
Dux and Marois, 2009). AB like deficits were also observed when a target digit and the three
digits immediately following the target in a RSVP stream were to be reported in each trial
(Weichselgartner and Sperling, 1987). The digits at the temporal positions around 200ms after
the target were least likely to be reported compared to the target digits and the digits presented at
500ms or later after the target onset. AB was not observed when subject was instructed to ignore
11
the first target and report only the second target (Raymond et al., 1992) or when the whole
sequence instead of a subset of items were to be reported (Nieuwenstein and Potter, 2006). Given
that the physical presentation of RSVP were the exactly same and that only instructions varied,
the occurrence of AB was attributed to the task demand for attentional selection of the first target
which was eliminated in the single task to report only the second target or to report the whole
sequence. Neither was AB observed when T2 was presented immediately after T1 without any
distractors in between the two targets, a phenomenon called lag-1 sparing (Potter et al., 1998).
Furthermore, the perceived order of the two targets are often reversed in that T2 is reported as
presented before T1. This order reversal was studied in a series of experiments (Hilkenmeier et
al., 2012a; Hilkenmeier et al., 2012b; Olivers et al., 2010) in which the temporal and feature
properties of a temporal cue were systematically manipulated to direct attention toward either T1
or T2. Directing attention to T2 increased the order reversals and directing attention to T1
decreased the reversals. Therefore, the authors concluded that attention affected the temporal
perception of the two targets via prior entry and that prior entry also existed when stimuli were
presented in the same spatial location.
1.2.2.3. Multiple RSVP streams in separate spatial locations
By using multiple streams of RSVP at different spatial positions, Sperling and his colleagues
measured the time course of attentional shift (Attention reaction time paradigm, Sperling and
Reeves, 1980; Reeves and Sperling ,1986), provided evidence for the quantal movement of
attention across space (Sperling and Weichselgartner, 1995) and quantified the interaction
between attention and iconic memory in partial report (Shih and Sperling, 2002).

12
1.2.2.4. Measuring reaction time of covert visual attention shift: Attention reaction time (ART)
paradigm
Attention reaction time (ART) paradigm (Reeves, 1977; Sperling and Reeves, 1980) was
developed to measure the covert shift of visual attention from one spatial location to another.
Subjects had to maintain fixation at a fixation dot throughout a trial. Two rapid serial visual
presentation (RVSP) streams were presented simultaneously, one to the left and the other to the
right of the fixation dot. The left stream consisted of only letters and the right one consisted of
only numbers. The letter stream was presented at a fixed rate of 4.6 letters per second while the
numeral stream was presented at any one of the rates of 4.6, 6.9, 9.1, or 13.4 numbers per
second. The task was to detect a target in the letter stream and then to report the earliest seen
number from the numeral stream after the detection without moving the eyes. The motor
reaction time (MRT) was also measured by the release of a key as soon as the target was
detected. The target was any one of the letters U, C or an outline square and occurred in a
randomly selected temporal position in the letter stream. The critical set was displayed
immediately after each trial as feedback. A critical set was defined as the seven numerals
consecutively presented at or right after the target. The critical set, together with the two
numerals right before and one right after, comprised 10 different numbers presented at randomly
determined order in each trial. Attention reaction time (ART) was defined as the temporal
interval between the target and the reported numeral. Similar to MRT, ART varied from trial to
trial and the distributions of ART were obtained from many trials in each combination of
presentation rate and target.
Results: Subjects reported that them had to pay full attention to the letter stream to detect the
target and then had to shift attention to the other stream. Subjects usually reported a numeral
13
occurred after the presentation of the target, except that the simultaneous number or even the
number presented right before the target was occasionally reported when the target was the
outline square at the slowest numeral rate of 4.6/s. No random guessing occurred (Sperling and
Melchner, 1976). Average ART was longest with the target letter C and shortest with the outline
square. In addition, average ART decreased as the rate of the to-be-reported numeral stream
increased. However, there were no significant interactions of the main effects of the target
difficulty and the presentation rate. Therefore, the relative speeds of processing different targets
would be the same at all the numeral rates tested. Another important result was the loss of order
information as the presentation rate increased. At high rates, the subjects could not tell the
relative temporal position of the reported numeral within the critical set as “early, middle, or
late”. At lower rates, however, order information was preserved. Similar patterns of disrupted
order perception were observed in another 4-item-recall experiment in which the three numbers
immediately after the earliest-seen number were also reported.
Item scores and order scores: Because all numerals were reported about with the same
frequency (except number “1”) and because no number tended to be reported earlier in the
response than the other numbers, the data were collapsed over the numeral identity and only
stimulus positions within the critical set were included in the analysis. The position
simultaneous with the target (in the letter stream) was defined A position 0. Positive integers
represented the positions after position 0 and negative integers before position 0.
Data were tabulated as item scores and order scores. Item scores Pi(r) referred to the possibility
of the stimulus (number) presented in the position i in the numeral (right, to-be-reported) RSVP
stream being reported in the response position r ( r = 1, 2, 3, or 4 for the first, second, third and
fourth reported number, respectively). For example, P3(2)= 0.2 means that the 3rd number
14
presented after the target (stimulus position 3) is reported as response 2 (the second numeral
reported in the four-number response) in 20% of trials. Composite item scores Pi referred to the
proportion of the trials in which number from stimulus position i was reported in any of the four
response positions. Therefore,
Pi = Pi(1) + Pi(2) + Pi(3) + Pi(4);
Order scores PiBj represented the summed probability that (1) number from stimulus position i
was reported before number from position j in the 4-item response and (2) number from position
i was reported but not number from position j. PiBj + PjBi = 1 and PiBi = 0.5.
The x axis is the stimulus position of the critical set and the y axis is the probability (item scores
or order scores; eg. Figure 2.1). In the item score plot, Item scores of the same response position
r are connected with a line. The envelope line represents the composite item scores Pi.
In the order score plot, each curve represents the probability (y axis) of a given stimulus position
i being reported earlier than position j (x axis).
Data patterns: The number presented about 400ms after the target was most likely to be reported
and the report probabilities of other positions decreased as the absolute temporal distance
increased relative to this “center position”. The order score curves were U-shaped, parallel to
each other, and folded around the center position. The center position was most likely to be
reported first as the corresponding curve was above all the other lines. The further a position was
from the center, either presented before or after, the less likely the numeral from that position
was to be reported earlier and the lower the corresponding curve was in the order scores plot.

15
2. Temporal dynamics of attention: Attention gating model
As the temporal dynamics of attention have shown qualitatively distinct and/or quantitatively
different patterns in different paradigms empirically, different aspects of the temporal properties
of attention have been theorized and modeled in simple detection task (Smith and Ratcliff, 2009)
as well as other more complex phenomenon such as attentional blink (see review: Oliver and
Meeter, 2008) and prior entry (Hilkenmeier et al., 2012b). In these models, the initial facilitation
effect of attention is modeled as a gating process. The concept of an attention gate was first
proposed by Sperling and his colleagues (Sperling, 1960; Reeves and Sperling, 1986) and was
further developed to account for behavioral results in all major attention paradigms (Reeves and
Sperling, 1986; Sperling and Weichselgartner, 1995; Shih and Sperling, 2002).
2.1. Attention gating in visual short-term memory (VSTM)
Reeves and Sperling (1986) further analyzed the same data of 4-item-recall experiment in the
ART paradigm (Reeves, 1977). The data measured the temporal dynamics of attention at the to-
be-reported RSVP stream and quantified the perceived orders in VSTM. They developed the
Attention Gating Model (AGM) to capture the temporal dynamics of attention. Because the
number of stimulus items presented in the to-be-reported RSVP stream is far more than the
number of items could be kept in the VSTM, attention, acting like a gate, allows the flow of
information to capacity limited memory only during a short interval. In this model, perceived
orders of rapidly presented items in VSTM were primarily determined by the amount of attention
allocated to each item during input. MRTs were used only to exclude trials with too fast
(expectation) or too slow (misses) reaction times.

16
2.1.1. Attention gating model
Attention gating model (AGM) further elaborated the mental snapshot model (Sperling and
Reeves, 1980; Figure 1.1.1) to quantify the temporal dynamics of attention at a single location
and its effects on the contents in VSTM. AGM proposes that attention gates the information from
the sensory organs (eyes) into the capacity limited VSTM. Upon the detection of target (in the
letter stream), an attention gate gradually opens at the location of the to-be-reported numeral
stream and then closes shortly after to prevent overloading the capacity-limited VSTM. The
amount of attention each stimulus receives depends on the width of the gate and the duration of
the stimulus presentation when the gate is open. No central forgetting is assumed. An order
precedence of each stimulus (numeral in this experiment) is defined as the product of the
stimulus strength and the amount of attention allocated by the gate at the time of the input and
stimuli are reported in ascending order of their precedence. The precedence is also perturbed by
memory noise so that the responses vary trial by trial. The attention gate is defined as a second-
order gamma function and the area under the function is the amount of attention allocated during
that time period (Figure 1.1.2). Since each input stimulus has the same high contrast, the
precedence equals to the corresponding area under the attention gate function. The memory noise
is randomly sampled from a normal distribution with 0 mean and unit variance. The model well
accounts for hundreds of data points of each subject (average r
2
~=0.85) with only five to eight
parameters. The attention gate function was found to be independent of the presentation rates of
the to-be-reported RSVP stream within subject (Reeves and Sperling ,1986), but was affected by
the complexity of the to-be-reported stimuli (Reeves, 1986)
2.1.2. Applications of AGM
AGM (Reeves and Sperling, 1986) quantified the temporal dynamics of attention at one spatial
17
location and stated that noisy memory strength, whose expected values are proportional to the
amount of attention available during the time of input, led to the systematic distortion in report
order of multiple targets in an RSVP stream (Norman, 1967). Sperling and Weichselgartner
(1995) conducted two experiments using ART paradigm and showed that the time it took for
attention to shift from one spatial location to another was independent of the distance traveled
and the distractors/obstacles along the path of the covert attention shift. The episodic quantal
theory was developed to account for this quantal movement of spatial covert attention. In the
theory, the quantal movement was a result of the temporal transition from one episode of
attention to another. Each attention episode described a state of attention with a static distribution
across space. In the first episode, attention was directed at the one RVSP stream for the target
detection. In the second episode, attention was allocated to the other to-be-reported RSVP
stream. A temporal transition function described the transition from one episode to the next. The
two attention episodes together with the temporal transition function described the following
process of attention shift. As soon as the target was detected, attention resource was quickly
withdrawn from the target and an attention gate gradually opened at the other location where the
to-be-reported RSVP stream was presented. Then the attention gate closed, which was described
as the third attention episode. The theory was further developed by Shih and Sperling (2002) to
quantify both the dynamics of attention and the iconic memory decay in partial report using a
novel RSVP choice attention-gating experiment. With some elaborations, the model can account
for all major spatial attention paradigms: spatially cued Go/No-Go reaction time (Shulman et al.,
1979), spatially cued choice reaction time (Tsal, 1983) and spatially cued discrimination
accuracy (Shaw and Shaw 1977; Lyon, 1987, 1990; Cheal and Lyon 1991, 1994).

18
3. Un-answered questions of RSVP studies
The RSVP experiments reviewed above, were only run and tested with supra-threshold stimuli
without any external noise. Attention has been shown to recruit different mechanisms in different
stimulus conditions and at different external levels which can be distinguished when
performance are measured in a wide range of stimulus contrasts at different levels of external
noise (see review: Lu and Dosher, 2008). Therefore, our understanding of how the temporal
dynamics of attention affects behavior in RSVP tasks is limited without measuring performance
in other stimulus conditions. With supra-threshold stimuli, identification reaches asymptotic
performance where attention is less likely to show any effects on perception. Therefore,
measuring behavioral performance in a wide range of stimulus conditions in RSVP is an
important first step to unveil possible attention effects on perception due to its temporal
dynamics. Moreover, AGM and its related models have only been applied to data obtained in
noiseless displays with high signal-to-noise-ratio stimuli and have not provided any theoretical
interpretations of potential attentional effects on perception in other stimulus conditions. In these
models, amount of attention allocated to an item at the input determines its memory strength,
which in turn decide the report order, while identification of a reported item is assumed to be
perfect. Without specifying the perceptual process and possible attentional effects on perception,
these models cannot account for any results when identification performance is impaired due to
lowered signal-to-noise ratio by decreasing signal strength and/or adding external noise.

19
4. New approach: the external noise plus attention reaction time paradigm and attention
gating perceptual template model (agPTM)
A new paradigm, the external noise plus attention reaction time paradigm developed by
combining the attention reaction paradigm (ART, Reeves, 1977; Sperling and Reeves, 1980)
with the external noise method (Lu and Dosher, 1998), is used to systematically measure and
quantify the time course of attention cueing in a wide range of stimulus conditions, including low
signal contrast, high external noise, (Experiment 1) and un-equal item signal-to-noise ratios
(Experiment 2A and 2B). The new paradigm is introduced and the new data are presented in
Chapter 2.
To account for the new data, the agPTM is developed by combining the perceptual template
model (PTM; Lu and Dosher, 1998) with the AGM (Reeves and Sperling ,1986). In the agPTM,
both signal and noise of input stimuli go through a contrast-gain control process (PTM; Lu and
Dosher, 1998; Dao et al., 2006) and then are gated by attention (AGM; Reeves and Sperling,
1986). The gated outputs determine the discriminability of the input stimuli. The report order is
determined by memory strength, which is hypothesized to be proportional to either the signal-to-
noise ratio or total energy of the gated output. The new agPTM is developed and evaluated and
alternative models are considered in Chapter 3.
The external noise method and the PTM model are introduced in this section.
4.1. External noise method
Human responses vary from trial to trial even given the exact same input stimulus. This might be
due to the different sources of fluctuations (noise) along the way of information processing in
both peripheral and central nervous system. Psychophysical models construct specific internal
processes of encoding, representation and interpretation (decision making) to predict behavior
20
outcome in certain tasks. Signal detection theory (SDT) has been used extensively to
quantitatively model the human behavior at the observer/ system level in different tasks, such as
Yes-no and two-alternative forced-choice (2AFC) task (Green and Swets, 1966; Macmillan and
Creelman, 1991).
Although SDT as a theoretical frame can be applied to any underlying response distributions,
Gaussian internal response distributions have be shown to account for performance in a wide
range of tasks (Wilkens, 2002). A Gaussian distribution is determined by its mean and its
variance. However, traditional applications of SDT rarely refer these internal distributions,
especially the variances, to external stimuli. In recent years, more research has been done to
quantify the relationship between the internal response distributions and the characteristics of
external stimuli. One effective approach is to add external noise to the signal stimuli to
externalize the internal responses. Many paradigms employ this approach, such as the critical
band masking, the classification image method, the double-pass consistency test, and the
equivalent input noise method (see a review in Lu and Dosher, 2008). The former two have been
mainly used to characterize the detailed features of the perceptual templates, while the latter two
have been mostly used to study the intrinsic noise of the observer. Intrinsic noise can be either
independent of external stimuli (eg. additive), or determined by the external input (eg.
multiplicative). The double-pass consistency test measures and refers the total intrinsic noise
relative to the external noise. The equivalent input noise method can further distinguish and
quantify different types of internal noise. This dissertation only used the equivalent input noise
method.
4.1.1. The equivalent input noise method
The equivalent input noise method was first developed by engineers (eg. North,1942) to measure
21
the response of electronic amplifiers and to estimate the intrinsic noise. Mixtures of signal and
external noise of various amplitudes, generated and known by the engineers, are used as inputs to
the amplifier. Outputs of the amplifiers are then analyzed to obtain the signal-to-noise ratios in
each signal and external noise conditions. The signal-to-noise ratio is defined as the average
output amplitude divided by its standard deviation. The known external noise and the unknown
intrinsic noise are the two sources of variability in the output. When the intrinsic noise is much
larger than the external noise, the output variability and therefore the output signal-to-noise ratio
for a given input signal amplitude is mainly determined by intrinsic noise. The output signal-to-
noise ratios for a given signal condition remains relative constant within this range of external
noise levels. When the external noise is much stronger than the intrinsic noise, the external
noise mostly determines the output variability and therefore the output signal-to-noise ratio. In
order to keep the output signal-to-noise ratio constant, an increasing amount of signal is required
as the external noise increases. The transition point is where the intrinsic and external noise is of
similar magnitude and contribute equally to the output signal-to-noise ratio.
Psychologist (Barlow, 1956; Burgess et al., 1981; Legge et al., 1987; Nagaraja, 1964; Pelli,1981)
adopted/adapted the equivalent input noise method to measure the internal noise of perceptual
system. The perceptual system of the observer is assumed to function as a noisy amplifier. The
output signal-to-noise ratios can be inferred from the measured behavior performance (eg.
accuracy). In simple tasks such as yes-no and forced choice, the signal-to-noise ratios can be
calculated directly from the behavioral measurement based on SDT. A constant-signal-to-noise
ratio contour of an observer in a given task qualitatively is called threshold-versus-external-
noise-contrast (TvC) function. A TvC function describes the required signal (eg. contrast)
threshold to maintain a certain performance level across different external noise levels. A single
22
TvC function resembles the contour of an electronic amplifier. The threshold remains relative
constant in low external noise conditions (a straight line parallel to the ordinate) and increases
with the external noise when the external noise is much greater than the intrinsic noise. Internal
noise and external noise are equally damaging to the performance near the transition point.
Triple TvC functions, three single TvC function measured at three different performance levels,
are usually obtained to fully characterize the relationship between signal thresholds and signal-
to-noise ratios.
4.2. Observer models
Many observer models have been developed to interpret the empirical results and model the
internal process and response distributions of the perceptual system. Lu and Dosher (2008)
reviewed five prominent noisy observer models in vision and concluded that the perceptual
template model gave the best account for the combined data of triple TvC functions and double-
pass agreement tests. All the other four models gave inferior fits to the results of at least one of
the two experiments.
4.2.1. Attentional effects on perception: the Perceptual Template Model (PTM)
The perceptual template model (PTM: Lu and Dosher, 1998; Figure 1.2) was proposed to
explicitly model nonlinear psychometric functions (Pelli, 1985) and the Weber law behavior of
the perceptual system (Burgess and Colborne, 1988).
There are five main components in the PTM: (1) Two perceptual templates: one in the signal
pathway with certain selectivity for task relevant properties; the other in the contrast-gain control
pathway, probably broadly tuned. The perceptual templates can be either very simple such as a
spatial frequency filter or far more complex depending on tasks. (2) Two nonlinear transducers
correspond to the two perceptual templates. (3) An independent multiplicative internal noise is
23
determined by the total amount of contrast energy of the input stimulus (signal and external
noise) (4) An independent additive internal noise is independent of signal contrasts, external
noise and other internal noise. It remains constant in a given attentional state for individual
subject. (5) Decision process depends on specific tasks.
According to PTM, attention can influence perception in the following ways:
1. Stimulus enhancement turns up the gain of templates to both signal and external noise and is
mathematically equivalent to the reduction of additive internal noise. It improves performance at
low external noise levels when the constant additive internal noise dominates, and neither
improves nor damages performance in high external noise conditions
2. External noise exclusion reduces the effective external noise probably by narrowing the tuning
function of the perceptual templates. Opposite to stimulus enhancement, it manifests the
influence under high external noise.
3. Multiplicative noise reduction lowers the gain of the independent multiplicative internal noise.
Since the multiplicative noise is proportional to the sum of the energy in signal and external
noise, it affects performance at all levels of external noise, with slightly larger effects in high
external noise.
With central pre-cues, external noise exclusion (Lu and Dosher, 1998; Lu and Dosher, 2000;
Dosher and Lu, 2000a, 2000b) was found to be the main effect with stimulus enhancement as
secondary mechanism in certain displays for some subjects (Dosher and Lu, 2000a; but see Ling
and Carrasco 2006). Using peripheral pre-cues, stimulus enhancement and external noise
exclusion were identified as the attentional mechanisms (Lu and Dosher, 1998; Lu and Dosher,
2000). It has also been shown that attention is distributed relatively equally within the related
space (Dosher et al., 2004) and time (Lu et al., 2004). So far, few experiments have identified
24
reduction in multiplicative noise or changes in non-linear transducer.

5. Significance
This dissertation addresses two questions. The first question is how attention selects information
over time? Does attention amplify signal, exclude external noise, or affect other process in
perception? In noiseless display with above-thresholds contrasts, the stimulus condition usually
used in the RSVP studies reviewed above, the perfect performance in identification does not tell
anything about the possible attention effects on perception when the signal-to-noise ratios are
lowered to thresholds or when external noise is added. Therefore, the external noise plus
attention reaction time paradigm is used to systematically measure and quantify the time course
of attention cueing in a wide range of stimulus conditions, including low signal contrast, high
external noise (Experiment 1), and un-equal item signal-to-noise ratios (Experiment 2A and 2B).
The agPTM is developed to quantify the rich data obtained in the two experiments and to unveil
the mechanism of the attention gating.
The second question is what determines the report orders of multiple targets presented in rapid
succession at one spatial location. Does it depend only on the amount of attention each stimulus
receives as proposed in AGM? Or do stimulus properties, such as total stimulus energy or signal-
to-noise ratio, also play some role in order reports? In the agPTM framework, the memory
strength determines the order report. With un-equal input signal-to-noise rations, qualitatively
distinct predictions are generated by two hypothesis of the memory strength, one proportional to
signal-to-noise ratio and the other proportional to total energy. With external noise added to only
one stimulus position in each trial, Experiment 2A and 2B are designed to test these two
competing hypotheses.
25
Addressing the first question will extend our understanding of how attention selects and gates
information in a wide range of stimulus conditions in RSVP and elucidate the underlying
mechanism of the temporal dynamics of attention. Answering the second question will help
characterize the influence of stimulus properties and attention on order perception. It could also
identify the potential strategy used in order report when multiple items presented in rapid
succession at one spatial location are reported.
The external noise plus attention reaction time paradigm and the agPTM developed in this
dissertation provide a general framework to characterize and quantify the temporal dynamics of
attention and can be readily applied to further study the dynamics of attention cueing and shifts
in other RSVP paradigms reviewed in Chapter 1.

26

Chapter Two: Measuring the Temporal Dynamics of Attention in the External Noise Plus
Attention Reaction Time Paradigm
The attention reaction time paradigm (ART) is a useful and efficient paradigm to quantify the
temporal dynamics of attention cueing and switching in a variety of attention tasks (citations).
Since only high item signal-to-noise ratios have been tested in ART, it is unknown how attention
affects performance in other stimulus conditions in RSVP. Given different attentional
mechanisms found in different stimulus conditions with and without external noise (citations), it
is important to measure performance in a wide range of stimulus conditions to fully quantify
attentional effects on perception and order decision and to characterize and distinguish different
attentional mechanisms. A new paradigm, the external noise plus attention reaction time
paradigm developed by combining the attention reaction paradigm (ART, Reeves, 1977;
Sperling and Reeves, 1980) with the external noise method (Lu and Dosher, 1998), is used to
systematically measure and quantify the time course of attention cueing under a wide range of
stimulus conditions, including low signal contrast, high external noise, and un-equal item signal-
to-noise ratios. The task is to report four letters in order immediately following a visual cue
embedded in an RSVP of letters at the fovea. The temporal dynamics of attention are quantified
by report probabilities and report orders as a function of temporal stimulus presentation position.
In experiment 1, signal contrasts spanning a full range of performance levels were tested in both
zero and high external noise. Performance was better at higher signal contrasts at a given
external noise level, as shown by higher report probabilities and more consistent report orders of
the letters presented within the critical set. Higher contrast was required in high external noise to
achieve similar performance level as in zero noise. In experiment 2, item signal-to-noise ratios
were made uneven by adding external noise to only one randomly picked letter out of the four
27
letters immediately following the cue. Compared to the report probability presented at the same
temporal position in the control condition without any added noise, the report probability of the
noise-perturbed letter decreased to chance level. Furthermore, the report probabilities of letters
presented in close temporal proximity to the noise-perturbed position increased and became more
likely to be reported before the noise-perturbed letter.
1. Method
1.1. Apparatus
All programs were run in Matlab 7.0.1 with Psychtoolbox 3.0.8 on a Dell desktop with Windows
XP Professional 2002. Stimuli were viewed binocularly and were displayed on a 19-Inch Hewlett
Packard (D8911) with a refresh rate of 100Hz and a resolution of 1024 × 768. A special circuit
was used to generate 14-bit grayscale images (VGA Video Switcher) (Li et. al, 2003). Responses
were collected by a keyboard and a Response Time Box (RTBox) (Li et al., 2010). Subjects were
stabilized by a chin rest 85 cm away from the monitor. Experiments were performed in a dark
room with the monitor as the only light source.
1.2. Stimuli
1.2.1. Signal (letter) frame: 19 capital letters in the English alphabet (excluding B, I, O, Q, S, V
and Z) were presented at a constant rate of 10 letters per second in each trial (SOA= 100ms).
Each signal frame only consisted one letter, which covered 64 X 64 pixels in the center of the
display, extending 1.5º X 1.5º visual angle in fovea at a viewing distance of 85cm.
1.2.1.1. Training, Experiment 1 and Experiment 2B: Each signal frame was presented for 20
ms (Figure 1.4.a) .
28
1.2.1.2. Experiment 2A: Each signal frame was presented for 60ms, except that the noise
perturbed frame (selected randomly from stimulus position 0, 1, 2 and 3 in each trial) was
presented for 20ms (Figure 1.4.b).
1.2.2. External noise frame: noise covered the same 64 X 64 pixels area in the center of the
display as the signal letters. Each noise frame was presented for 20ms. Two noise frames were
temporally integrated with a signal frame in the order of noise, signal and noise.
1.2.2.1. Training, Experiment 1 and Experiment 2B: each noise element consisted of 2 × 2
pixels and its contrast was randomly sampled from a Gaussian distribution with zero mean and
standard deviation of 33% (Figure 1.4.a; Figure 1.5, left).
1.2.2.2. Experiment 2A: each noise image was generated by combining randomly scrambled
phase spectrum and intact power spectrum of the discrete Fourier transformation of the
corresponding letter at the noise-perturbed position. (Figure 1.4.b; Figure 1.5, right).
1.2.3. Cue: consists of four 30-degree arches in the 68-pixel circle centered at the fixation
(Figure 1.6 ) .
1.3. Procedure
At the beginning of each trial, a black fixation was presented in the center of the screen. Subject
initiated the trial by pressing the space bar and the black fixation turned white for 0.5s. An
RSVP stream was presented at the center after a blank interval (uniform background luminance )
for a random duration between 0.3~0.5 s with a uniform distribution. The RSVP stream
consisted of 19 uppercase letters (B I O Q S V Z excluded) randomly sampled without
replacement (random permutation). The cued letter was selected randomly between the 6th-13th
letters, by an exponential distribution:
29



  
 
12
6
13
3
0585 . 0 1
12 ~ 6 )
3
2
(
3
1
j
j
i
i
p p
i p

Responses were collected in the end of the 19-letter presentation and feedbacks were given
immediately after each trial.
1.4. Tasks
Subjects were instructed to press a button (any of the four buttons of the RTBox) as soon as the
cue was presented and remembered the identity and the presentation order of the cued letter and
the three letters immediately after. If the reaction time, defined as the time between the onset of
the cue and the button press, was shorter than 0.15s or longer than 0.4s, the trial was aborted
immediately without collecting the 4-letter response and re-run at a random time later in the
same session. In the end of the RSVP stream, subjects reported the four letters in the perceived
order on a keyboard. The typed letters were presented on the screen and could be edited before
confirmation by pressing the “ENTER” key on the keyboard
1.5. Feedback
A payoff matrix [0,10, 7, 5, 1, 1, 1] was used to encourage subjects to report letters as close as to
the cued letter but not earlier. Points were only given to response matching the letters presented
at the stimulus position from -1 to +5, regardless of the reported order. After confirmation of the
4-letter response upon pressing the “ENTER” key , 7-letter critical set (stimulus position -1 ~ 5),
4-letter response just entered, the reaction time and the points were shown on the same screen.

30
1.6. Design
Subject:
Three subjects, including the author, were recruited and informed consent were obtained. YZ and
KS participated in all the experiments. BF participated in Experiment 1 and Experiment 2A.
Training:
Staircase procedures was used to train subjects and estimate contrast thresholds for each
individual subjects. Two staircases, 2-down-1-up and 4-down-1-up were run with zero and high
noise in a mixed design. Each subject was trained until the performance reaches asymptotic
levels. Five contrast levels with equal distance in log space at each external noise level were then
calculated from the results of the staircases procedures. These estimated contrast plus the 100%
contrast were used as the signal contrasts in experiment 1 (Table 2.1).
Experiment 1:
The method of constant stimuli was used. Six signal contrasts (Table 2.1) were tested in both
zero and high noise conditions, resulting in twelve stimulus combinations of signal contrast and
noise. Each subject ran 2400 trials, divided into 8 blocks of 300 trials each. Therefore, 200 trials
were run in each combination of signal contrast and external noise level. Trials of different
stimulus combinations were mixed randomly.
Experiment 2A:
Contrast was set at 0.5 for all letters. In a noise-perturbed trial, two phase-scrambled noise
frames were temporally integrated with one letter frame randomly selected from stimulus
position 0, 1, 2 or 3. A control condition without any noise was also included. Each subject ran
four blocks of 250 trials. Within each block, 50 trials of noise-perturbed position 0, 1, 2, and 3
and control were run in random order.
31
Experiment 2B:
Contrast was set at 0.4 for all letters. In a noise-perturbed trial, two 33% Gaussian noise frames
were temporally integrated with one letter frame randomly selected from position 0, 1, 2 or 3. A
control condition without any noise was also included. Each subject ran four blocks of 250 trials.
Within each block, 50 trials of noise-perturbed position 0, 1, 2, and 3 and control were run in
random order.
1.7 Statistical analysis
For each subjects, item scores and order scores (Chapter 1: 1.2.2.4 item scores and order
scores) were calculated based on the collapsed data in each stimulus condition (one
combination of signal contrast and external noise level) in terms of the stimulus positions, not
the actual letter identities. 95% confidence intervals (95% C.I.s) of item scores and order scores
were generated by the following bootstrap method in each stimulus condition in each
experiment for each subject.
a. 200 four-letter responses were randomly sampled with replacement from the 200 four-
letter responses collected in the experiments;
b. Item scores and order scores were calculated from the re-sampled 200 four-letter
responses
c. Procedure a and b were repeated 10,000 times
d. Means and 95% confidence intervals of item scores and order scores were calculated
from the 10,000 sets of samples.

32
2. Results
Qualitatively similar patterns of items scores and order scores are observed in all three subjects.
Quantitative conclusions are based on comparisons of the 95% C.I.s.
2.1. Experiment 1
Performance became better as letter contrast increased in both no noise and high external noise
conditions. Higher letter contrasts were required to achieve same performance in high external
noise than in zero external noise. . Overall patterns of the data are summarized in section
2.1.1~2.1.3. Data for each subject are analyzed individually in section 2.1.4 .
2.1.1. Item scores, Pi(r)
Item score, Pi(r), is the probability of letters presented at stimulus position i being reported in
response r (-3 < -i < 5, -3 < j < 5, i and j are integers), which equals to the number of trials in
which letters presented at stimulus position i are reported in the response r divided by the total
number of trials. Given i, r, and external noise level, Pi(r) increased with increasing letter
contrasts. Given i and r, higher contrasts were required to achieve similar Pi(r)s in high external
noise than in zero noise given. The item scores of a given response r are connected with a line,
which shows the report probability distribution of response r across all stimulus positions. The
peak of an item score curve in a stimulus condition for a subject is defined as:
Pj(r) = max (Pi(r)) and the C.I.s of Pi(r)s of no more than two other stimulus positions overlap
with the C.I. of Pj(r).
The curves (top rows in Figure 2.1, Figure 2.2, and Figure 2.3 for subject YZ, KS, and BF,
respectively) showed distinct and separate peaks, more evident at high signal contrasts.
The peak values of the item score curves decreased with increasing response position r (r = 1,
2, 3, or 4). Peaks did not exist in some stimulus conditions and /or at some response positions.
33
The earlier (after the cue) the stimulus position was, the more likely it was to be reported in
earlier responses, which was shown as earlier peaks in curves of earlier responses. In other
words, the temporal distance between the peak and the cue onset (position 0) increased with
response position r in a given stimulus condition for a given subject. The exact locations of
peaks varied between subjects and within a subject across different stimulus conditions for a
given response r. The overlapping C.I.s between the peak positions and the positions temporally
approximate to the peaks reflect the limits of the statistical power in the experiments. Letters
presented before the cue onset were rarely reported above chance (1/19, 1/18, 1/17, and 1/16 for
response 1, 2, 3, and 4, respectively) in any of the four response positions, except that letters at
the position -1, the temporal position immediately preceding the cue, were reported above
chance in some response positions in a subset of stimulus conditions for some subjects.
2.1.2. Composite item scores, Pi
Composite item score, Pi , is the probability of letters presented at stimulus position i being
reported in any of the four response positions. Therefore, the composite item score of a stimulus
position equals to the sum of the item scores of the stimulus position in all four response
positions: Pi = Pi(1) + Pi(2) + Pi(3) + Pi(4). Pi s are shown as the envelope curves in the item
scores plots (top rows in Figure 2.1, Figure 2.2, and Figure 2.3 for subject YZ, KS, and BF,
respectively). Given i , Pi s increased with increasing letter contrasts with or without external
noise and higher letter contrasts were required to achieve similar Pi in high external noise than
in zero external noise. The peak of a composite item score curve in a stimulus condition for a
subject is defined as:
Pj = max (Pi) and the C.I.s of Pi of no more than two other stimulus positions overlap with the
C.I. of Pj.
34
The distributions of Pi s showed similar peak positions across all stimulus conditions. Pi peaked
at either position 0, the cued position, or position 1, the temporal position immediately after the
cue. The exact peak positions of Pi varied between subjects and within a subject in different
stimulus conditions. The overlapping C.I.s between the peak positions and the positions
temporally approximate to the peaks reflect the limits of the statistical power in the
experiments. Letters presented at stimulus positions temporally further away (either before i < -
1 or after i > 4 ) from the cue were rarely reported above chance (4/19~ 0.21). For a given
stimulus condition (particular high signal contrast), Pi rose quickly above chance (at position -2
or -1) to peak performance (at position 0 or 1) and then fall gradually back to chance at later
stimulus positions.
2.1.3. Order scores, PiBj
Order score, PiBj, is the probability of letters presented at stimulus position i being reported
‘before’ letters presented at the stimulus position j, which refers to either of the following two
events (1) letters at the stimulus position i and j are both reported and the letter at the position i
is reported before the letter at the position ; (2) only letter at position i is reported. PiBj equals to
the number of trials in which i is reported ‘before’ j divided by the number of trials in which at
least letters at one of the two positions are reported in any of the four responses. The order
scores of a given stimulus position i are connected with a line. The U-shaped curves lied one on
top of another and most of them did not cross each other, which defined an order of precedence.
The higher up the curve relative to the other curves, the larger the precedence of the
corresponding position was and the more likely stimulus presented at that position was to be
reported first. The position most likely to be reported first defines the folding point where all
the curves folded around (the bottom of the U). PiBj s deviated more from the chance level of
35
0.5 as letter contrast increased. Moreover, the order of precedence was not the same as the order
of presentation and some PiBj curves crossed each other. The result summaries focus on the
order precedence of positions whose item scores were above chance levels.
2.1.4. Individual subjects
Item scores, order scores, and bootstrap 95% confidence intervals in experiment 1 are plotted in
Figure 2.1, Figure 2.2 and Figure 2.3, for YZ, KS and BF, respectively. The peaks of the item
scores and the composite item scores are summarized in Table 2.1.
a. YZ (Figure 2.1):
The composite item scores Pi s peaked at position 1 across all stimulus conditions except at
position 2 at 42.6% contrast and high external noise, indicating that letters presented
immediately after, not simultaneous with, the visual cue were most often reported. Furthermore,
letters at position 1 were most likely to be reported first as shown by the peak of the item scores
of the first response at position 1 (max (Pi(1)) = P1(1)). Response 2 most often came from
stimulus position 1 and 2, and response 3 from position 2 and 3. The item scores curve of
response 4 (Pi(4)) showed peaks only at position 3 in 100% contrast in zero noise, and 74.1% in
high noise. Similar data patterns were observed in the order scores. Position 1 had the highest
precedence as reflected by the top-most position above all the other order score curves across
all stimulus conditions. Precedence of the other five stimulus positions with above-chance
composite item scores showed a general trend of precedence in descending order as position 0,
2, 3, 4, and -1 with some variations in certain stimulus conditions. Crossings of order score
curves were observed between P0Bj and P2Bj in the three highest letter contrasts in both external
noise conditions.

36
b. KS (Figure 2.2):
The peaks of composite item scores Pi at position 0 in all but the two lowest signal contrasts in
both external noise conditions indicate that subject KS was able to report letters presented
simultaneously with the visual cue. Pi (1) reached maximum at position 0 (P0(1)) in all but
8.2% in zero noise stimulus condition at position 1. Response 2 most often came from stimulus
position 1 and 2. Pi(3), the item scores of the third response, showed peaks only at position 2 in
the three highest signal contrasts (25.2%, 36.7% and 100%) in zero noise condition. Pi (4), the
item scores of the fourth/last response, did not show peaks at all. Based on the order scores,
position 0 had the highest precedence in the four highest letter contrasts in both external noise
conditions. In the two lowest contrasts at both external noise levels, position 1 had the highest
precedence but the order score C.I.s of position 1 overlapped with the C.I.s of position 0.
Precedence of the five stimulus positions with above-chance composite item scores showed a
general trend of precedence in descending order as the position 0, 1, 2, -1, 3, with some
variations in certain stimulus conditions.
c. BF (Figure 2.3):
The composite item scores Pi peaked at either position 0 or 1 in all but the lowest contrasts in
both external noise conditions. Pi (1) reached maximum at either position 0 or position 1 in all
but the lowest contrasts in both external noise conditions. Response 2 most often came from
stimulus position 1, and response 3 from position 2. The item scores curve of response 4 (Pi(4))
showed peaks only at position 3 in 100% contrast and zero noise, and at position 2 in 74.1%
contrast and high noise. Similar data patterns were observed in the order scores. Position 0 and
position 1 had the two highest precedence. Precedence of the other three stimulus positions with
above-chance composite item scores, position -1, 2 and 3, varied from one stimulus condition to
37
another. The remaining four positions with below-chance composite item scores had the lowest
precedence in all stimulus conditions.
2.2. Experiment 2
Experiment 2A and 2B showed similar results. Letters presented at positions adjacent to the
noise-perturbed positions were reported earlier and more often compared to the control
condition, which resulted in the following changes in item scores and order scores. The
composite item scores and item scores at noise-perturbed positions decreased significantly and
the composite items scores of the positions close to the noise-perturbed positions increased.
Item scores (top panels in Figure 2.4, Figure 2.5, and Figure 2.6, for YZ, KS and BF,
respectively) showed systematic shifts of the peaks of one or more responses (Table 2.3) and
increases at positions close to the noise perturbed positions. Precedence of the noise-perturbed
positions decreased while those of the nearby positions increased, which resulted in an inverted
U-shaped folding around the noise-perturbed position in the order score curves.
2.2.1 Individual data in Experiment 2A
a. YZ (Figure 2.4, left)
In control, the order precedence of stimulus positions were 1, 2, 0, 3, 4, -1, and 5 in descending
order. The 95% confidence intervals of the order scores of position -1 and 5 were not
statistically significantly different. The order score curve P0Bj, crossed the other two curves, P2Bj
and P3Bj but the C.I.s at the crossed stimulus positions were not statistically significantly
different. The composite item scores peaked at position 1 and the item scores of response 1, 2,
3, and 4 peaked at position 1, 2, 3, and 3, respectively.
When noise was added to position 0, the order precedence of stimulus positions became 1, 2, 3,
4, -1, 5, and 0. The 95% confidence intervals of the order scores of position 5 and 0 were not
38
statistically significantly different. The order score curve P-1Bj, crossed the other two curves,
P3Bj and P4Bj Changes in item scores further showed changes in report order as well as in report
probability. All the item scores and the composite item score of position 0 were at chance
levels. For response 1, P1(1)(0.780, C.I.=(0.720,0.835); control, 0.535, C.I.=(0.465,0.605))
increased significantly. Significant increase was also observed in the composite item score P2
(0.985, C.I.= (0.965,1.000); control, 0.925, C.I.=(0.885,0.960))
When noise was added to position 1, the order precedence of stimulus positions became 0, 2, 3,
4, 1, 5, and -1. The 95% confidence intervals of the order scores of position 1, 5, and -1 were
not statistically significantly different. The order score curves P0Bj and P2Bj crossed each other.
Changes in item scores further showed changes in report order as well as in report probability.
All the item scores and the composite item score of position 1 were at chance levels. For
response 1, the peak of Pi(1) shifted from position 1 to position 0. P0(1) (0.640, C.I.=
(0.575,0.705); control, 0.285, C.I.= (0.225,0.350)) and P2(1) (0.255, C.I.= (0.195,0.315);
control, 0.115, C.I.= (0.070,0.160)) increased significantly. For response 2, P3(2) (0.255, C.I.=
(0.200,0.315); control, 0.110, C.I.= (0.070,0.155)) increased significantly. For response 3, P4(3)
(0.220, C.I.= (0.165,0.280); control, 0.095, C.I.= (0.055,0.135)) increased significantly. For
response 4, P0(4) (0.140, C.I.= (0.095,0.190); control, 0.050, C.I.= (0.020,0.080)) decreased
significantly. Significant increases were also observed in the composite item scores P0
(0.810,C.I.= (0.755,0.860); control, 0.565, C.I.= (0.495,0.635) ), P3 (0.880,C.I.= (0.830,0.925);
control, 0.715, C.I.= (0.650,0.780)), and P4 (0.520,C.I.= (0.450,0.590); control, 0.260, C.I.=
(0.200,0.320)).
When noise was added to position 2, the order precedence of stimulus positions became 1, 3, 0,
4, 5, 2, and -1. The 95% confidence intervals of the order scores of position 5, 2, and -1 were
39
not statistically significantly different. Changes in item scores also showed these changes in
report order as well as in report probability. All the item scores and the composite item score of
position 2 were at chance levels. For response 1, P1(1) (0.705, C.I.= (0.640, 0.770); control,
0.535, C.I.=(0.465, 0.605)) increased significantly. For response 2, position 1 and 3 were the
most often reported in the second response instead of position 2 in control. P0(2) (0.210, C.I.=
(0.155, 0.265); control, 0.085, C.I.= (0.050, 0.125)) and P3(2) (0.330, C.I.= (0.265,0.395);
control, 0.110, C.I.=( 0.070, 0.155)) increased significantly. For response 3, P4(3) (0.270, C.I.=
(0.210,0.335); control, 0.095 , C.I.=(0.055, 0.135)) increased significantly. For response 4,
P4(4) (0.265, C.I.= (0.205,0.325); control, 0.145 , C.I.=(0.095, 0.195)) and P5(4) (0.115, C.I.=
(0.075,0.160); control, 0.040, C.I.=(0.015, 0.070)) increased significantly. Significant increases
were also observed in the composite item scores P3 (0.900,C.I.= (0.855,0.940); control, 0.715,
C.I.= (0.620,0.750)), P4 (0.685,C.I.= (0.620,0.750); control, 0.260, C.I.= (0.200,0.320)), and P5
( 0.190,C.I.= (0.140,0.245); control, 0.070, C.I.= (0.035,0.105) ),
When noise was added to position 3, the order precedence of stimulus positions became 1, 2, 0,
4, 5, -1 and 3. The 95% confidence intervals of the order scores of position 5, -1, and 3 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 3 were at chance levels. For response 3, position 2 and 4 became the most often
reported in the third response instead of position 3 in control. P0(3) (0.270,C.I.= (0.210,0.335);
control, 0.095, C.I.= (0.055,0.135)) and P4(3) (0.270,C.I.= (0.210,0.335); control, 0.095, C.I.=
(0.055,0.135)) increased significantly. Significant increases were also observed in the
composite item scores P2 (0.985,C.I.= (0.965, 1.000); control, 0.925 (C.I.=(0.885, 0.960)), P4
40
(0.540,C.I.= (0.470,0.610); control, 0.260, C.I.= (0.200,0.320)), and P5 ( 0.185,C.I.=
(0.130,0.240); control, 0.070, C.I.= (0.035,0.105) ).
b. KS (Figure 2.5, left)
In control, the order precedence of stimulus positions were 0, 1, 2, -1, and 3 in descending
order. The 95% confidence intervals of the order scores of position -1 and 3 were not
statistically significantly different. The composite item scores peaked at position 0 and the item
scores of response 1, 2, and 3 peaked at position 0, 1, and 2, respectively.
When noise was added to position 0, the order precedence of stimulus positions became 1, 2, -1,
3, and 0. The 95% confidence intervals of the order scores of position -1, 3, and 0 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 1 were at chance levels. For response 1, the peak of Pi(1) shifted from position 0 in
control to position 1. P-1(1) (0.190, C.I.= (0.135,0.245); control, 0.065, C.I.= (0.035,0.100)) and
P1(1) (0.640, C.I.= (0.575,0.705); control, 0.185, C.I.= (0.135,0.240)) increased significantly.
For response 2, the peak of Pi(2) shifted from position 1 in control to 2. P1(2) (0.235, C.I.=
(0.175,0.295); control, 0.425, C.I.= (0.335,0.490)) decreased significantly and P2(2) (0.445,
C.I.= (0.375,0.515); control, 0.235, C.I.= (0.175,0.295)) increased significantly. For response 3,
the peak of Pi(3) shifted from position 2 in control to 3. P2(3) (0.115, C.I.= (0.075,0.160);
control, 0.225, C.I.= (0.170,0.285)) decreased significantly. Significant increase was also
observed in the composite item score P1 (0.950, C.I.= (0.920,0.980); control, 0.815, C.I.=
(0.760,0.870)).
When noise was added to position 1, the order precedence of stimulus positions became 0, 2, 3,
-1, 1, and 4. The 95% confidence intervals of the order scores of position -1, 1, and 4 were not
41
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 1 were at chance levels. For response 1, P0(1) (0.825, C.I.=(0.770,0.875); control,
0.695, C.I = (0.630,0.755)) increased significantly. For response 2, the peak of Pi(2) shifted
from position 1 in control to position 2. P2(2) (0.495, C.I.= (0.425,0.565); control, 0.235, C.I.=
(0.175,0.295)) and P3(2) (0.215, C.I.= (0.160,0.270); control, 0.055, C.I.= (0.025,0.090))
increased significantly. For response 3, the peak of Pi(3) shifted from position 2 in control to
position 3. P3(3) (0.265, C.I.=(0.205,0.325); control, 0.115, C.I = (0.075,0.160)) increased
significantly. Significant increases were also observed in the composite item scores P2 (
0.790,C.I.= (0.735,0.845); control, 0.545, C.I.= (0.475,0.615)) and P3 ( 0.585,C.I.=
(0.515,0.650); control, 0.305, C.I.= (0.245,0.370)).
When noise was added to position 2, the order precedence of stimulus positions became 0, 1, 3,
-1, and 4. The 95% confidence intervals of the order scores of position 3, -1, and 4 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 2 were at chance levels. For response 1, P0(1) (0.555, C.I.= (0.485, 0.625); control,
0.695, C.I.= (0.630,0.755)) decreased significantly and P1(1) (0.305, C.I.= (0.240, 0.370);
control, 0.185, C.I.= (0.135,0.240)) increased significantly. For response 2, P3(2) (0.200, C.I.=
(0.145, 0.255); control, 0.055, C.I.= (0.025,0.090)) increased significantly. Significant increases
were also observed in the composite item scores P3 ( 0.450,C.I.= (0.385,0.520); control, 0.305,
C.I.= (0.245,0.370)) and P4 ( 0.290,C.I.= (0.230, 0.355); control, 0.160, C.I.= (0.110,0.215)).
When noise was added to position 3, the order precedence of stimulus positions became 0, 1, 2,
-1 and 4. Changes in item scores further showed changes in report probability. All the item
42
scores and the composite item score of position 3 were at chance levels. No other significant
changes in item scores were observed.
C. BF (Figure 2.6)
In control, the order precedence of stimulus positions were 1, 0, 2, 3, -1 and 4 in descending
order. The 95% confidence intervals of the order scores of position 1 and 0 were not
statistically significantly different, nor were those of position 3, -1, and 4. The composite item
scores peaked at position 1 and the item scores of response 1, 2, and 3 peaked at position 1, 1,
and 2, respectively.
When noise was added to position 0, the order precedence of stimulus positions became 1, 2, 3,
-1, 0, and 4. The 95% confidence intervals of the order scores of position -1, 0, and 4 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 0 were at chance levels. For response 1, P1(1) (0.840, C.I.= (0.785,0.890); control,
0.485, C.I.= (0.415,0.555)) increased significantly. For response 2, the peak of Pi(2) shifted
from position 1 in control to position 2. P1(2) (0.115, C.I.= (0.075,0.160); control, 0.345, C.I.=
(0.280,0.410)) decreased significantly and P2(2) (0.645, C.I.= (0.575,0.710); control, 0.245,
C.I.= (0.185,0.305)) increased significantly. For response 3, the peak of Pi(3) shifted from
position 2 in control to position 3. P2(3) (0.150, C.I.= (0.105,0.200); control, 0.375, C.I.=
(0.310,0.440)) decreased significantly and P3(3) (0.380, C.I.= (0.310,0.445); control, 0.145,
C.I.= (0.100,0.195)) increased significantly. Significant increases were also observed in the
composite item score P2 (0.885, C.I.= (0.840,0.930); control, 0.750, C.I.= (0.690,0.810)) and P3
(0.615,C.I.= (0.545,0.680); control, 0.350, C.I.= (0.285,0.420)).
43
When noise was added to position 1, the order precedence of stimulus positions became 0, 2, 3,
1, -1, and 4. The 95% confidence intervals of the order scores of position 1, -1 and 4 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. For response 1, the peak of Pi(1) shifted from position 1,
in control to position 0. P1(1) (0.015, C.I.= (0.000, 0.035); control, 0.485, C.I.= (0.415,0.555))
decreased significantly. P0(1) (0.760, C.I.= (0.700,0.820); control, 0.430 (C.I.= (0.360,0.500))
and P2(1) (0.015, C.I.= (0.000, 0.035); control, 0.485, C.I.= (0.415,0.555)) increased
significantly. For response 2, the peak of Pi(2) shifted from position 1 in control to position 2.
P0(2) ( 0.055, C.I.= (0.025,0.090); control, 0.265, C.I.= (0.205,0.330)) and P1(2) (0.125, C.I.=
(0.080,0.175); control, 0.345, C.I.= (0.280,0.410)) decreased significantly. P2(2) (0.405, C.I.=
(0.335,0.475); control, 0.245, (C.I.= (0.180,0.305)) and P3(2) (0.195, C.I.= (0.140,0.250);
control, 0.030, (C.I.= (0.010,0.055)) increased significantly. For response 3, the peak of Pi(3)
shifted from position 2 in control to position 3. P0(3) (0.035, C.I.= (0.010,0.060); control,
0.125, C.I.= (0.080,0.175)) and P2(3) (0.210, C.I.= (0.155,0.270); control, 0.375, C.I.=
(0.310,0.440)) decreased significantly. The composite item score P3 (0.580,C.I.= (0.510,0.650);
control, 0.350, C.I.= (0.285,0.420)) increased significantly and the composite item score P1
(0.315, C.I.= (0.250,0.380); control, 0.965, C.I.= (0.935,0.985)) decreased significantly.
When noise was added to position 2, the order precedence of stimulus positions became 1, 0, 3,
4, and -1. The 95% confidence intervals of the order scores of position 1 and 0 were not
statistically significantly different, nor were those of position 4 and -1. Changes in item scores
further showed changes in report order as well as in report probability. All the item scores and
the composite item score of position 2 were at chance levels. For response 3, the peak of Pi(3)
shifted from position 2 in control to position 3. P3(3) (0.275, C.I.= (0.215,0.335); control,
44
0.145, C.I.= (0.100,0.195)) increased significantly. Significant increases were also observed in
the composite item score P3 (0.550,C.I.= (0.480,0.620); control, 0.350, C.I.= (0.285,0.420)) and
P4 (0.330,C.I.= (0.265,0.395); control, 0.165, C.I.= (0.115,0.220)).
When noise was added to position 3, the order precedence of stimulus positions became 1, 0, 2,
4, -1 and 5. The 95% confidence intervals of the order scores of position 1 and 0 were not
statistically significantly different, nor were those of position 4, -1, and 5. Changes in item
scores further showed changes in report order as well as in report probability. All the item
scores and the composite item score of position 3 were at chance levels. Significant increases
were observed in the composite item score P4 (0.345,C.I.= (0.280,0.410); control, 0.165, C.I.=
(0.115,0.220)) and P5 (0.195,C.I.= (0.140,0.250); control, 0.090, C.I.= (0.005,0.130)).
2.2.2. Individual data in Experiment 2B
a. YZ (Figure 2.4, right)
In control, the order precedence of stimulus positions were 1, 2, 0, 3 and 4 in descending order.
The order score curve of position 0, P0Bj, crossed the curve of position 2, P2Bj. The composite
item scores peaked at position 1 and the item scores of response 1, 2, 3, and 4 peaked at
position 1, 2, 3, and 3, respectively.
When noise was added to position 0, the order precedence of stimulus positions became 1, 2, 3,
4, 5, and -1. The 95% confidence intervals of the order scores of position 5 and -1 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 0 were at chance levels. For response 1, P1(1) (0.785, C.I.=(0.725,0.840); control,
0.475, C.I.=(0.405,0.545)) increased significantly. For response 2, P1(2) (0.105, C.I.=
(0.065,0.150); control, 0.320, C.I.= (0.255,0.385)) decreased significantly and P2(2) (0.615,
45
C.I.=(0.545,0.685); control, 0.380, C.I.=(0.310,0.450)) increased significantly. For response 3,
P2(3) (0.140, C.I.= (0.095,0.190); control, 0.265, C.I.= (0.205,0.330)) decreased significantly.
Significant increases were also observed in the composite item score P5 (0.165, C.I.=
(0.115,0.220) ;control, 0.065, C.I.= (0.035,0.100))
When noise was added to position 1, the order precedence of stimulus positions became 0, 2, 3,
4, 5, and -1. The 95% confidence intervals of the order scores of position 5 and -1 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 1 were at chance levels. For response 1, the peak of Pi(1) shifted from position 1 to
position 0. P0(1) (0.820, C.I.= (0.765,0.870); control, 0.385, C.I.= (0.315,0.455)) increased
significantly. For response 2, P2(2) (0.580, C.I.= (0.510,0.650); control, 0.380, C.I.=
(0.310,0.450)) and P3(2) (0.195, C.I.= (0.140,0.250); control, 0.125, C.I.= (0.080,0.170))
increased significantly. For response 4, the peak of Pi(4) shifted from position 3 to position 4.
P2(4) (0.035, C.I.= (0.010,0.060); control, 0.135, C.I.= (0.090,0.185)) decreased significantly
and P4(4) (0.275, C.I.= (0.215,0.340); 0.100, C.I.= (0.060,0.145)) increased significantly.
Significant increases were also observed in the composite item scores P0 (0.880,C.I.=
(0.835,0.920); control, 0.605, C.I.= (0.535,0.675) ), P3 (0.880,C.I.= (0.835,0.925); control,
0.695, C.I.= (0.630,0.760)), and P4 (0.505,C.I.= (0.435,0.570); control, 0.245, C.I.=
(0.185,0.305)).
When noise was added to position 2, the order precedence of stimulus positions became 1, 0, 3,
4, 5 and -1. The 95% confidence intervals of the order scores of position 5 and -1 were not
statistically significantly different. Changes in item scores also showed these changes in report
order as well as in report probability. All the item scores and the composite item score of
46
position 2 were at chance levels. For response 1, P1(1) (0.660, C.I.=(0.595, 0.725); control,
0.475, C.I.=(0.405, 0.545)) increased significantly. For response 2, position 0, 1, and 3 were the
most often reported in the second response instead of position 2 in control. P0(2) (0.240,
C.I.=(0.180, 0.300); control, 0.095, C.I.=(0.055, 0.135)), P3(2) (0.260, C.I.=(0.200, 0.325);
control, 0.125, C.I.=(0.080, 0.170)), P4(2) (0.130 C.I.=(0.085, 0.185); control, 0.025,
C.I.=(0.005, 0.050)) increased significantly. For response 3, P4(3) (0.270, C.I.=(0.210, 0.335);
control, 0.120, C.I.=(0.075, 0.165)) increased significantly. For response 4, P4(4) (0.235,
C.I.=(0.180, 0.295); control, 0.100, C.I.=(0.060, 0.145)) increased significantly. Significant
increases were also observed in the composite item scores P4 (0.635,C.I.= (0.570,0.700);
control, 0.245, C.I.= (0.185,0.305)), and P5 (0.225, C.I.= (0.170,0.285) ; control, 0.065, C.I.=
(0.035,0.100)).
When noise was added to position 3, the order precedence of stimulus positions became 1, 2, 0,
4, and 5. The order score curve P2Bj, crossed two other curves P0Bj and P1Bj. Changes in item
scores further showed changes in report order as well as in report probability. All the item
scores and the composite item score of position 3 were at chance levels. For response 2, P2(2)
(0.580, C.I.=(0.510, 0.650); control, 0.380, C.I.=(0.310, 0.450)) increased significantly. For
response 3, position 2 and 4 became the most often reported in the third response instead of
position 3 in control. P4(3) (0.230, C.I.=(0.175, 0.290); control, 0.120, C.I.=(0.075, 0.165)) and
P5(3) (0.100, C.I.=(0.060, 0.145); control, 0.015, C.I.=(0.000, 0.035)) increased significantly.
Significant increases were also observed in the composite item scores P2 (0.985,C.I.= (0.965,
1.000); control, 0.875 (C.I.=(0.830, 0.920)), P4 (0.450,C.I.= (0.380,0.520); control, 0.245, C.I.=
(0.185,0.305)), and P5 (0.215, C.I.= (0.160,0.275) ;control, 0.065, C.I.= (0.035,0.100)).

47
b. KS (Figure 2.5, right)
In control, the order precedence of stimulus positions were 0, 1, 2, and 3 in descending order.
The composite item scores peaked at position 0 and the item scores of response 1, 2, and 3
peaked at position 0, 1, and 2, respectively.
When noise was added to position 0, the order precedence of stimulus positions became 1, 2, 3
and 4. Changes in item scores further showed changes in report order as well as in report
probability. All the item scores and the composite item score of position 0 were at chance
levels. For response 1, the peak of Pi(1) shifted from position 0 in control to position 1. P1(1)
(0.730, C.I.=(0.665, 0.790); control, 0.235, C.I.=(0.180, 0.295)) and P2(1) (0.195, C.I.=(0.140,
0.250); control, 0.025, C.I.=(0.005, 0.050)) increased significantly. For response 2, the peak of
Pi(2) shifted from position 1 in control to position 2. P1(2) (0.120, C.I.=(0.075, 0.165); control,
0.425, C.I.=(0.355, 0.495)) decreased significantly and P2(2) (0.575, C.I.=(0.505, 0.645);
control, 0.210, C.I.=(0.155, 0.265)) increased significantly. For response 3, the peak of Pi(3)
shifted from position 2 in control to position 3. P2(3) (0.090, C.I.=(0.050, 0.130); control,
0.215, C.I.=(0.160, 0.270)) decreased significantly and P3(3) (0.245, C.I.=(0.185, 0.305);
control, 0.125, C.I.=(0.080, 0.170)) increased significantly. Significant increases were also
observed in the composite item score P1 (0.915, C.I.= (0.875,0.955); control, 0.800, C.I.=
(0.745,0.855 )), P2 (0.880, C.I.= (0.835,0.920); control, 0.540, C.I.= (0.470,0.610)), and P3 (
0.430, C.I.= (0.360,0.500); control, 0.280, C.I.= (0.220,0.345)).
When noise was added to position 1, the order precedence of stimulus positions became 0, 3, 2,
4, and 5. The order scores of the remaining positions were not significantly different. Changes
in item scores further showed changes in report order as well as in report probability. All the
item scores and the composite item score of position 1 were at chance levels. For response 1,
48
P0(1) (0.955, C.I.=(0.925, 0.980); control, 0.690, C.I.=(0.625, 0.750)) increased significantly.
For response 2, the peak of Pi(2) shifted from position 1 in control to position 3. P3(2) (0.585,
C.I.=(0.515, 0.650); control, 0.085, C.I.=(0.050, 0.125)) increased significantly. Significant
increases were also observed in the composite item scores P0 ( 0.980,C.I.= (0.960,0.995);
control, 0.865, C.I.= (0.815,0.910)), P3 ( 0.830, C.I.= (0.775,0.880); control, 0.280, C.I.=
(0.220,0.345)), P4 ( 0.385,C.I.= (0.32,0.450); control, 0.140, C.I.= (0.095,0.190)), and P5(
0.055,C.I.= (0.025,0.090); control, 0.150, C.I.= (0.100,0.200)),
When noise was added to position 2, the order precedence of stimulus positions became 0, 1, 3,
4, and -1. The 95% confidence intervals of the order scores of position 3, 4, and -1 were not
statistically significantly different. Changes in item scores further showed changes in report
probability. All the item scores and the composite item score of position 2 were at chance
levels. No significant increases in item scores were observed. Significant increase was observed
in the composite item score P4 (0.350,C.I.= (0.285,0.420); control, 0.140, C.I.= (0.095,0.190)).
When noise was added to position 3, the order precedence of stimulus positions became 0, 1, 2,
and -1. The 95% confidence intervals of the order scores of position 1 and 2 were not
statistically significantly different. Changes in item scores further showed changes in report
order as well as in report probability. All the item scores and the composite item score of
position 3 were at chance levels. For response 2, P2(2) (0.445, C.I.=(0.375, 0.515); control,
0.210, C.I.=(0.155, 0.265)) increased significantly.

49

Chapter Three: Attention Gating Perceptual Template Model (agPTM)
The framework of AGM well accounts for the temporal dynamics of attention cueing and shifts
within and between RSVP streams in high item signal-to-noise ratios in all major attention
paradigms (reviewed in Chapter 2, section 2.1.2). However, it does not specify what goes
through the attention gate and potential attentional effects on perception. Moreover, it does not
address the possible different contributions of input signal and noise to order report when
contrast is lowered and/or external noise is added. Therefore, the AGM cannot account for the
results of the two experiments in Chapter 2.
The agPMT is developed to quantitatively fit to the data collected in the two experiments in
Chapter 2 and characterize the underlying mechanism of the temporal dynamics of attention. In
the agPTM, input stimuli (both signal and external noise) goes through a contrast-gain control
process, and the outputs of the initial perceptual process are gated by attention. The time to
initiate the opening of the attention gate is set as a variable to reflect the trail-to-trial
fluctuations of the attention reaction time. Furthermore, with un-equal item signal-to-noise
ratios (Experiment 2), the order strength model of signal-to-noise ratio, but not total energy
generates predictions qualitatively and quantitatively consistent with the data, which indicates
that items with higher signal-to-noise ratios (discriminability) are more likely to be reported
before items with lower signal-to-noise ratios.
Two more alternative models are considered in the discussion section in the end of this chapter.
One model (Alternative model 1, AM 1) assumes a constant attention gate opening time and the
other model (Alternative model 2, AM 2) assumes no effects of attention on perceptual process
(discriminability). The worse fit of AM 1 to experiment 1 suggests that a constant attention gate
opening time is not sufficient and that attention reaction time varies from trial to trial. AM 2
50
generates qualitatively meaningful predictions of Experiment 1 only with the strength model of
total energy, which cannot reconcile with the data in Experiment 2. Therefore, the attention
gated contrast-gain control, with variable attention gate opening time and the order strength of
signal-to-noise ratio, gives the best account among all the alternative models presented here.
This suggests that attention gates information for further analysis and signal-to-noise ratio
determines the report order when attention is required to process multiple items in rapid
succession in one spatial location (eg. at fovea in this dissertation).
1. Attention gating perceptual template model (agPTM)
The attention gating perceptual template model (agPTM; Zhao et al., 2014), was developed
based on the attention gating model (AGM; Reeves and Sperling, 1986) and the perceptual
template model (PTM; Lu and Dosher, 1998). In the agPTM (Figure 1.3), input signal and noise
goes through a contrast-gain control process (PTM; Lu and Dosher, 1998; Dao et al., 2006) and
then the output is gated by attention (AGM; Reeves and Sperling, 1986). The gated output
determines both the discriminability of the input and the report order. The discriminability is
proportional to the signal-to-noise ratio of the gated output. The report order is determined by
strength, a function of the gated output. Two independent perceptual internal noises are added,
one before and the other after the contrast-gain control process, to account for the inefficiency in
perception. Another independent decision noise added to the strength is also required to account
for the trial-by-trial variances of report order. All model predictions are obtained by Monte Carlo
simulations.
1.1. Step-by-Step description
1.1.1. Perceptual templates. In the signal pass, the gain of the perceptual template for a signal
valued stimulus is equal to β. For a signal stimulus of contrast c, the perceptual template output
51
has an amplitude S: S= βc. In the gain-control path, the template gain to the signal stimulus is
equal to β2. In this dissertation, β = β2. In addition to the signal are two types of external noise:
(1) Gaussian noise (Experiment 1 and Experiment 2B) with equal energy across all spatial
frequencies (2) phase-scrambled noise (Experiment 2A) with the same Fourier power spectrum
of the corresponding template. Because its total gain is equal to 1.0, the perceptual template
output for the Gaussian noise has a standard deviation of σext (equal to the standard deviation Next
of the Gaussian noise) and the output for the phase-scrambled noise is equal to c
2
, the total
energy in the signal.
1.1.2. Nonlinear transducer function. Output = sign (input) | Input|
ϒ
, raises the outputs from
the perceptual templates to the power of ϒ.
1.1.3. Pre-contrast-gain internal noise is an internal noise source after the nonlinear transducer.
It is modeled as a Gaussian random variable with mean 0 and standard deviation N1. Till this
point, the total noise in the system is the sum of the external noise and the additive internal noise

2
1
2
N N
ext


Equation 1.

and the net energy E in the contrast-gain control pathway is
2
1
2 2
) ( N N c E
ext cgc
  
 

Equation 2.

1.1.4. Divisive contrast-gain control process modifies the signal gain:
b E c
cgc
 / ) (



Equation 3.

and the noise energy :
) /( ) (
2
1
2
b E N N
cgc ext
 


Equation 4.

Dao et. al (2006) included the non-zero “b” in the contrast-gain-control to keep the denominator
non-zero when both signal contrast and external noise are close to zero. However the non-zero
52
pre-contrast-gain internal noise N1 serves the same function, therefore b is a redundant parameter
and is set to zero in agPTM.
1.1.5. Attention gate function. The outputs of the contrast-gain control process is gated by
attention in which both normalized signal and noise is multiplied by the cumulative attention
resources Ai , the area under the attention gate function within the time interval.
The attention gate function is a gamma probability density function of time t (cue onset defined
as t = 0):





 
   


 


T t t
T t t e T t t
t a
T t t
0
0
1
0
0
) (
) (
1
) (
0



  

Equation 5.

 is the scaling factor,

is the time constant and

is the number of exponential stage(s).
0
t
- T
is the opening time of the gate and T is a constant. The probability distribution of
0
t
is defined as
another gamma probability density function (Shih and Sperling, 2002):






 




0 0
0 ) (
) (
1
) (
0
0
1
0
0
0
0
0
0
0 0
t
t e T t
t p
t t
t
t
t t
 

 
Equation 6.

0
t

is the time constant,
0
t

is the number of exponential stage(s) and T is a constant.
The attention resource allocated to position i equals to
dt t a A
i
i
t
t
i



1
) (

Equation 7.

1.1.6. Decision. The outputs from the attention gate go through a two-stage decision
process in which and (1) report accuracy are determined by the signal-to-noise ratios (d’ s) and
(2) the four stimuli with the largest strength are reported in descending order.
53
1.1.6.1. Discriminability
2 2
1
2 2 2
2
2
1
2
2
2
2
1
2 2
'
/ ] ) [( ) (
) (
) /( ) (
/ ) (
i ext ext
cgc ext i
cgc i
i
A N N c N N N
c
N b E N N A
b E c A
d
   

  


  






Equation 8.

N2 is the post-contrast-gain internal noise. Since the scaling factor  in the attention gate function
also scales N2 as shown in the equation 8, N2 is modeled as a Gaussian random variable with
mean 0 and standard deviation of 1. Therefore,
2 2
1
2 2 2
1
2
'
/ ] ) [( ) (
) (
i ext ext
i
A N N c N N
c
d
   

  




Equation 9.

The four items are identified in descending order of the order strength. Since the four-letter
response is a sample without replacement from 19 letters, the probabilities of correct
identification are calculated from N-alternative forced-identification task in SDT with N=19, 18,
17 and 16 for response 1, 2, 3 and 4, respectively.
1.1.6.2. Memory strength: The memory strength is corrupted by a memory noise which leads to
trial-to-trial variability of the order report. The strength Vi can take one of the two formulations.
Strength model of signal-to-noise ratio:
'
' d
i i
memory
d V   

Equation 10.

Strength model of total energy (AGM):
E
i
E
cgc
ext i
i
memory memory
A
b E
N N c A
V  

 
  

 

) ( ) (
2
1
2 2

Equation 11.

Therefore, the strength model of total energy is mathematically equivalent to the strength model
of the original AGM, in which report orders are determined by the amount of attention allocated
54
to the items during input.

The memory noise of the two models are normal random variables with zero means and the
standard deviations of
E d
memory memory
and  
'
for strength model of signal-to-noise ratio and of
total energy, respectively.
1.2. Qualitative illustrations of the function of each parameter
Parameters used in the simulations are listed in Table 3.1.1. The gamma functions of the
attention gate and the attention gate opening time distributions are listed in Table 3.1.2 and Table
3.1.3, respectively. All the changes in item scores and/or order scores are relative to the
simulations in Figure 3.1.1. The memory noise,
memory
 , only affects report order without
changes in discriminability. However due to the variable gate opening time, increase in
memory

by a factor of 5 result in only small decrease in report order consistency (shallower folding of
order score curves), mostly evident at the lowest contrast in zero and high noise condition
(Figure 3.1.2). Changes in all the other parameters discussed here lead to changes in
discriminability due to changes either in perceptual process or in temporal distribution of
attention (attention gate itself or attention gate opening time distribution). Item scores increase
with discriminability due to higher report accuracy and less guessing. Report orders become
more consistent as shown by stronger folding of order score curves. Increases in the perceptual
template gain, β , result in increases in discriminability in all but saturated signal contrasts
regardless of external noise levels (Figure 3.1.3). Increases in the nonlinear transducer function,
ϒ , result in decreases in zero noise and increases in high noise in discriminability in all but
saturated signal contrasts (Figure 3.1.4). Increases in the pre-contrast-gain internal noise,
1
N ,
result in decreases in discriminability in all but saturated signal contrasts only in zero noise
55
condition when the external noise is much smaller than
1
N (Figure 3.1.5). Increases in  or 
result in wider attention gate without change in the total area under the gate (Figure 3.1.12: line
M1, control; increased  , line M3; increased  , line M4). Multiplying  by a factor of X
results in multiplication of each value of the attention gate function by 1/ X (Figure 3.1.12, line
M2). Therefore, increases in any of the three parameters (  ,  , or  ) of the attention gate
function result in decreases in the area under the attention gate for a given time period around the
peak and increases near the tails of the gate. Decreases in the amount of attention allocated (the
area under the gate) lead to decreases in discriminability, thus decreases in item scores at
stimulus positions around the peak of the attention gate across all stimulus condition (  : Figure
3.1.6;  : Figure 3.1.7;  : Figure 3.1.8). The exact extent of the changes depend on the
underlying changes in Ai s, which are the amounts of attention defined as the areas under the
attention gate function during the time of the inputs. Increases in
0
t

or
0
t

result in wider
density function of attention gate opening time (Figure 3.1.13: line M1, control; increased
0
t

,
line M2; increased
0
t

, line M3) and decrease in attention allocated to the stimulus positions
near the peak of the attention gate and increase at tail positions, which leads to decrease in item
scores and report order consistency (
0
t

: Figure 3.1.7;
0
t

: Figure 3.1.10). When the product of
0
t

and
0
t

is kept constant, increase in
0
t

results in sharper attention gate opening time (Figure
3.1.13, line M4). When
0
t

>>
0
t

, t0 can be treated as a constant and the model reduces to a
model with a constant attention gate opening time.

56
1.3. Qualitatively different predictions by the two strength models for Experiment 2: signal-to-
noise ratio vs. total energy (AGM)
The strength model of signal-to-noise ratio predicts systematic changes of report orders and
report probabilities. Adding noise to a stimulus lowers the signal-to-noise ratio and thus the
strength, so the noise-perturbed stimulus is less likely to be reported and other stimuli are
reported before the noise-perturbed stimulus instead. This is shown as systematic changes in item
scores of the four responses and order scores, especially for position 0 and 1. Left column in
Figure 3.1.14 shows sample simulations of the strength model of signal-to-noise ratio. In control
condition, position 0 and position 1 are most likely to be reported as the first and the second
response, respectively. The peaks of the item scores of the first and the second response is:
max(Pi(1)) = P0(1)≈0.53 ; max(Pi(2)) = P1(2) ≈ 0.31. The precedence of positions with above-
chance composite item scores as shown in order scores is position 0, 1, 2, 3, -1. When noise is
added to position 0, the peaks shift to position 1 and position 2, respectively: max(Pi(1)) =
P1(1)≈0.54 ; max(Pi(2)) = P2(2) ≈ 0.25. The precedence with above-chance composite item
scores is position 1, 2, 3, -1, and 4. When noise is added to position 1, the peak of the item scores
of the first response increases from around 0.53 to about 0.70 and the peak of the item scores of
the second response shifts from position 1 to position 2. The precedence with above-chance
composite item scores is position 0, 2, 3, -1, and 4. A secondary inverted U-shaped folding also
occurs at the noise-perturbed position.
In the strength model of total energy, adding external noise does not change the total energy thus
the memory strength. Because of decreased discriminability by added external noise, guessing
occur when the noise-perturbed stimulus is being reported. Guessing results in the decreased
items scores only at the noise-perturbed position while the items scores of all the other positions
57
remain relatively same as those in the control condition (right column in Figure 3.1.14). For
example, when noise is added to position 0, all item scores of position 0 (P0, P0(1), P0(2), P0(3),
and P0(4)) fall below chance while item scores of all the other positions remain the same. The
changes in precedence resemble the changes in the strength model of signal-to-noise ratio.
The empirical results of Experiment 2A and 2B (Figure 2.4, Figure 2.5, and Figure 2.6) are
qualitatively consistent with the predictions by the strength model of signal-to-noise ratio, not
total energy. In this section, both strength models were fit to the data. The strength model of
signal-to-noise ratio accounted well for all the experiments. The strength model of total energy
showed significant difference between its predictions and the data in Experiment 2A and
Experiment 2B for all subjects.

2. Model evaluation:
Two strength models (signal-to-noise ratio vs. total energy/AGM) of the agPTM are evaluated
by fitting them to the data in experiment 1 and experiment 2. In experiment 1, the two models
fit equally well to the data. In experiment 2, item score predictions by the strength model of
total energy/AGM were significantly different from the shifted peaks of the observed item
scores in all three subjects (see Chapter 3, section 1.3.). The strength model of the signal-to-
noise ratios accounted well for both experiments. These results suggest that attention gated
contrast-gain control with internal noise at each stage and memory strength determined by
signal-to-noise ratio (not total energy) well account for all the experiments.
2.1. Fitting procedure
The weighted least squares (squared the difference between observed and predicted item scores
and order scores (10000 trials of simulations per set of item and order scores divided by their
58
respective confidence intervals obtained in Chapter 2, section 1.7) were used to find the best
fitting parameters of models. Grid search was used to explore parameter space and to find initial
values for optimization by simulated annealing (Kirkpatrick et al., 1983). The models were fit
jointly to experiment 1 and experiment 2B with a single set of parameters for each of the two
subjects, YZ and KS. The models were also fit separately to experiment 1 and experiment 2A
with one set of parameters in each experiment for each of the three subjects (YZ, KS, and BF).
Because of the different temporal configurations of stimuli used in experiment 1 and
experiment 2A, these two experiments cannot be fit jointly with a single set of parameters.
2.2. Fitting results
2.2.1. Joint fit of Experiment 1 and Experiment 2B:
The best fitting parameters are listed in Table 3.2.1 and Table 3.2.2. For strength model of
signal-to-noise ratio, r
2
is 0.9572 and 0.9262 for YZ and KS, respectively. For strength model
of energy, r
2
is 0.9035 and 0.8868 for YZ and KS, respectively. In terms of the 95% C.I.s of the
data, the strength model of signal-to-noise ratio well accounts for data in both experiments
(Figure 3.2.1, YZ; Figure 3.2.3., KS), while the strength model of total energy, with more
predicted item scores and order scores outside the C.I.s of the data, showed worse fits in both
experiments (Figure 3.2.2, YZ; Figure 3.2.4, KS). Moreover, predictions by the strength model
of signal-to-noise ratio (bottom row: Figure 3.2.1, YZ; Figure 3.2.3, KS), but not total energy
(bottom row: Figure 3.2.2, YZ; Figure 3.2.4, KS) show qualitatively consistent patterns with the
data in Experiment 2B (see Chapter 2, section 2.2.; Chapter 3, section 1.3.). The significant
difference between the predictions by the strength model of total energy and the data in
Experiment 2B (bottom row: Figure 3.2.2, YZ; Figure 3.2.4, KS) indicates that the item signal-
to-noise ratio, not the total energy, is the determinant in order report. The joint fit results
59
indicate that the attention gated contrast gain control is sufficient to account for perceptual
process in both experiments and the signal-to-noise ratio but not the total energy determines the
report order.
2.2.2. Experiment 1:
The best fitting parameters are listed in Table 3.2.3 and Table 3.2.4. For all three subjects, both
strength model of energy and of signal-to-noise ratio show similar fitting results in terms of r
2
.
For strength model of signal-to-noise ratio, r
2
is 0.9692, 0.9497 and 0.9281 for YZ, KS and BF,
respectively. For strength model of energy, r
2
is 0.9689, 0.9424 and 0.9302 for YZ, KS and BF,
respectively. In terms of the 95% C.I. of the data, predictions by both models well accounted
for the data for all three subjects (signal-to-noise ratio: Figure 3.2.5, YZ; Figure 3.2.7, KS;
Figure 3.2.9, BF; total energy: Figure 3.2.6, YZ; Figure 3.2.8, KS; Figure 3.2.10, BF).
Therefore, when stimulus input signal-to-noise ratios are equal, both strength models well
account for all the data with a single set of parameters for each subject. This suggests that the
attention gated contrast gain control is sufficient to account for all the data in Experiment 1 but
the two strength models could not be distinguished with equal stimulus input signal-to-noise
ratios.
2.2.3. Experiment 2A:
For all three subjects, strength model of signal-to-noise ratio (Table 3.2.3) generated better
fitting results than strength model of energy (Table 3.2.4) in terms of the r
2
. For the strength
model of energy, r
2
is 0.9619, 0.9496 and 0.9563 for YZ, KS and BF, respectively. For the
strength model of signal-to-noise ratio, r
2
is 0.9531, 0.9205 and 0.9385 for YZ, KS and BF,
respectively. More importantly, predictions by the strength model of signal-to-noise ratio (left
column: Figure 3.2.11, YZ; Figure 3.2.12, KS; Figure 3.2.13, BF), but not total energy (right
60
column: Figure 3.2.11, YZ; Figure 3.2.12, KS; Figure 3.2.13, BF) show qualitatively consistent
patterns with the data of all the three subjects (see Chapter 2: 2.2.; Chapter 3, 2.3.). The
significant difference between the predictions by the strength of total energy (bottom row:
Figure 3.2.11, YZ; Figure 3.2.12, KS; Figure 3.2.13, BF) and the data in all three subjects
indicates that the item signal-to-noise ratio, not the total energy, is the determinant in order
report.

3. Discussion
Two alternative models are presented here. The first alternative model assumes a constant
attention gate opening time. This model showed inferior fit to experiment 1, especially for
subject YZ, who exhibited order score curves crossings, evidence against a constant gate
opening time. The second alternative model assumes the amount of attention allocated to an
item during its input does not affect its discriminability. In this model, only the strength model
of the AGM could generate predictions of Experiment 1. But the strength model of AGM could
not account for the observed shifted peaks of item scores in experiment 2. Therefore, the
agPTM with variable attention gate opening time and the memory strengths proportional to
item signal-to-noise ratios is the only model that well accounts for all the experiments in this
dissertation. Attention gates the output of the contrast-gain control process and the report order
is determined by the memory strength proportional to the signal-to-noise ratio.

61
3.1. Alternative models
3.1.1. Alternative model 1 (AM 1): assumes attention gate opening time is constant
3.1.1.1. Joint fit of Experiment 1 and Experiment 2B:
The best fitting parameters are listed in Table 3.3.1 and Table 3.3.2. For the strength model of
signal-to-noise ratio, r
2
is 0.9286 and 0.9212 for YZ and KS, respectively. For the strength
model of energy, r
2
is 0.9038 and 0.88623 for YZ and KS, respectively. Compared to the
agPTM (strength model of signal-to-noise ratio: Figure 3.2.1, YZ; Figure 3.2.3, KS; strength
model of total energy: Figure 3.2.2, YZ; Figure 3.2.4, KS), the assumption of a constant
attention gate opening time in AM 1 led to inferior fit and more predicted item scores and order
scores outside the 95% C.I.s. of the data for both strength models (strength model of signal-to-
noise ratio: Figure 3.3.1, YZ; Figure 3.3.3, KS; strength model of total energy: Figure 3.3.2,
YZ; Figure 3.3.4, KS). For YZ, peaks of AM1 predicted item scores in the strength model of
total energy overlapped (Figure 3.3.2), which were clearly separated in the data in Experiment 1
(Figure 2.1; Table 2.2). The AM1 also failed to generate order score crossings observed in YZ
(Figure 2.1; Chapter 2, section 2.1.4, a.YZ). Even given the inferior, the strength model of
signal-to-noise ratio generated better fitting results than the strength model of total energy in
terms of the C.I.s. The strength model of total energy showed qualitatively inconsistent patterns
and quantitatively significant different item scores.
3.1.1.2. Experiment 1:
The best fitting parameters are listed in Table 3.3.3 and Table 3.3.4. For the strength model of
signal-to-noise ratio, r
2
is 0.9362, 0.9537 and 0.9218 for YZ, KS and BF, respectively. For the
strength model of energy, r
2
is 0.9390, 0.9434 and 0.9243 for YZ, KS and BF, respectively. In
terms of the 95% C.I. of the data, predictions by both models show significant deviations from
62
the data for all three subjects (strength model of signal-to-noise ratio: Figure 3.3.5, YZ; Figure
3.3.7, KS; Figure 3.3.9, BF; strength model of total energy: Figure 3.3.6, YZ; Figure 3.3.8, KS;
Figure 3.3.10, BF). For YZ, peaks of item scores overlapped (Figure 3.3.5 and Figure 3.3.6),
which were clearly separated in the data (Figure 2.1; Table 2.2). Furthermore, the agPTM with
a constant attention gate opening time could not generate crossings of order score curves
observed in YZ (Figure 2.1; Chapter 2, section 2.1.4, a.YZ).
3.1.1.3. Experiment 2A:
For all three subjects, strength model of signal-to-noise ratio (Table 3.3.5) generated better
fitting results than strength model of energy in terms of the r
2
(Table 3.3.6). For the strength
model of signal-to-noise ratio, r
2
is 0.9388, 0.9491 and 0.9548 for YZ, KS and BF, respectively.
For the strength model of energy, r
2
is 0.9218, 0.9428 and 0.9402 for YZ, KS and BF,
respectively. Furthermore, predictions by the strength model of signal-to-noise ratio (left
column, top row: Figure 3.3.11, YZ; Figure 3.3.12, KS; Figure 3.3.13, BF), but not total energy
(left column, bottom row: Figure 3.3.11, YZ; Figure 3.3.12, KS; Figure 3.3.13, BF) show
qualitatively consistent patterns with the data of all the three subjects (see Chapter 2: 2.2.;
Chapter 3, 2.3.). The significant difference between the predictions by the strength of total
energy and the data in all three subjects (left column, bottom row: Figure 3.3.11, YZ; Figure
3.3.12, KS; Figure 3.3.13, BF) further support that the item signal-to-noise ratio, not the total
energy, is the determinant in order report although the assumption of constant attention gate
opening time is not consistent with the data.

63
3.1.2. Alternative model 2 (AM2) : PTM+AGM assumes attention does not affect
discriminability, d’

] ) [(
) (
2 2 2
2
2
1
2
'
  



ext ext
i
N c N N N
c
d
  

Equation 12

In a given signal contrast and external noise level, d’ is constant and does not change with the
temporal distribution of attention. In the strength model of signal-to-noise ratio, strength of
different stimulus positions are equal in a given stimulus condition (in a single trial), and report
orders of items presented at different temporal positions would be completely random due to
the memory noise. Item scores and order scores are all at chance levels (Figure 3.4.1). In the
strength model of AGM, report order is solely determined by the amount of attention allocated
during input and the report accuracy is determined by discriminability (d’), which result in
qualitatively consistent patterns with the data in experiment 1 (Figure 3.4.2).
Predictions by this alternative model are qualitatively consistent with data in experiment 1 only
when the strength is proportional to the area under the attention gating function (strength model
in the original AGM) (Figure 3.4.2). However, the predictions by any model with the strength
model of AGM are inconsistent with data in experiment 2 (see Chapter 3, 2.2), which requires
additional assumption about the report order of the noise-perturbed position in experiment 2.

64

Chapter Four: Conclusions and Discussions
1. Conclusions
When attention is maintained at fovea, a temporal cue can quickly initiate attention selection as
shown in the distributions of item score P
i
(1) of the first response (in the four-letter response
task) which peaked within 100ms after the cue onset (blue curves in top rows panels in Figure
2.1, Figure 3.1 and Figure 4.1 ) in all stimulus conditions tested. The asymmetric distributions
of P
i
, the summed report probabilities of stimulus positions, indicates that the disengagement
of attention from the fovea to prevent overloading the capacity limited short-term memory is
slower compared to the initiation of attention. The faster rising and slower decay of attention at
fovea is consistent with previous studies using single RSVP stream at fovea (Hilkenmeier et al.
2012b; Weichselgartner and Sperling, 1987). The results of the two experiments in this
dissertation suggests that this temporal dynamics of attention persist in a wide range of stimulus
conditions.
The newly developed agPTM quantifies the temporal dynamics of attention at fovea and
characterizes the underlying attentional mechanism. In the agPTM, both signal and noise of
input stimuli go through a contrast-gain control process (PTM; Lu and Dosher, 1998) and then
are gated by attention (AGM; Reeves and Sperling, 1986). The gated outputs determine the
discriminability of the input stimuli and the report order. Independent internal noise was added
both before and after the contrast-gain control process. The agPTM suggests that when multiple
items presented at the same place in rapid succession have to be reported, attention gates the
input for further analysis. The agPTM well accounted for performance in a wide range of
stimulus conditions in experiment 1, 2A and 2B for all subjects. By adding external noise to
only one stimulus position in otherwise noiseless display, two strength models of agPTM
65
predicts qualitatively and quantitatively different changes in item scores. The results in
experiment 2A and 2B were qualitatively consistent with the predictions by the strength model
of signal-to-noise ratio but not of total energy. The strength model of signal-to-noise ratio
generated better results in fitting experiment 1 and 2B jointly as well as fitting experiment 2A.

2. Discussions
The attention gated contrast gain control is the main mechanism of the temporally distributed
attention in one spatial location initiated by a cue presented at the same position in this
dissertation. In the previous studies of spatially distributed attention, central pre-cues mainly
elicited the external noise exclusion and peripheral cues induced a combination of the stimulus
enhancement and the external noise exclusion (Lu and Dosher, 1998; Lu and Dosher, 2000;
Dosher and Lu, 2000a; Dosher and Lu, 2000b). It remains unknown whether the agPTM can
account for the temporal dynamics of attention at different CTOAs initiated by a cue to re-
distribute attention across space during the simultaneous presentation of all stimuli. If agPTM
also captures the temporal dynamics of attention in these paradigms using simultaneous
presentation, it would indicate that attention acts as a gate at any single spatial location by an
attention gated contrast gain control process, while the spatial re-distribution of attention
mainly enhances signal with peripheral pre-cue and excludes external noise with central pre-
cue.
In this dissertation, the specific presentation rate (10 items per second) of RSVP stream was
selected to fully engage attention at fovea (Reeves and Sperling, 1986). Although a single
attention gate function was sufficient to account for performance at various presentation rates
with inputs of high signal-to-noise ratios (Reeves and Sperling, 1986), further experiments are
66
needed to test whether a single gate function or multiple gates are required in the agPTM to
account for the temporal properties of attention using a combination of a wide range of stimulus
conditions and presentation rates. The stimuli to be reported might also affect the temporal
dynamics of attention since attention gates with different temporal distribution were elicited by
numerals and shapes as to-be-reported items (Reeves, 1986).
Guessing in the current study could result from two processes. One process is true guessing
about low signal-to-noise ratio (discriminability) stimuli and the other is confusion between two
similar stimuli (eg. E and F). Because the stimulus positions of any two given stimuli are
selected randomly and independently in each trial, reporting one stimulus instead of another due
to confusion cannot be distinguished from true guessing when stimulus positions, not stimulus
identities, are the independent variable. Therefore, discriminability might be under-estimated.
The likelihood of a set of item scores and order scores given a set of parameters has not been
calculated due to complexity brought by the inter-dependence between these scores. Therefore,
the maximum likelihood method has not been applied to optimize the best-fitting models.
Another way to find the best-fitting parameters is to fit models directly to the frequencies of the
4-letter responses. However, due to the small number of responses collected and the large
number of possible combinations of the 4-letter responses, it requires a lot of computation time
even if the probability of a single 4-letter response is to be estimated given a set of parameters
by Monte Carlo simulations. Analytical solutions might be a more efficient way to calculate the
probabilities but have not been realized due to the same reason, the large number of possible
combinations of 4-letter responses and the guessing. Therefore, it remains unknown whether
this method could better constrain the models.
67
Although the strength model of signal-to-noise ratio generated qualitatively consistent patterns
and quantitatively better fit in Experiment 2, the observed report orders might not be the same
as the perceived order. The pay-off matrixes only rewarded responses matching the presented
letters so subject was more likely to report letters with higher discriminability even though the
reported letters might be perceived as being presented later than the noise-perturbed letter.
Therefore, further experiments are needed to distinguish between the report order and the
perceived order. Given the model mimicry shown between the two strength models in
Experiment 1 (r
2
s in Table 1.1 and Table 1.2), however, the agPTM can still be applied to
characterize and quantify the mechanisms of the temporal dynamics of attention with equal
item signal-to-noise ratios without distinguishing between report order and perceived order.
The results in this dissertation support the notion of prior entry (Titchener, 1908) to the extent
that the amount of attention allocated to an item is a key factor in determining its report order.
Given the same input signal-to-noise ratios, the more attention an item received, the earlier it
tends to be reported than the other items receiving less attention. When the input signal-to-noise
ratios are different, the evidence for the strength model of signal-to-noise ratio indicates that the
item with higher input signal-to-noise ratio is more likely to be reported first than the item with
lower input signal-to-noise ratio given the same amount of attention. The agPTM can also
quantitatively predict report orders when input signal-to-noise ratio of and the amount of
attention allocated to each item are both different as in experiment 2. However, it remains
unknown whether the report orders are the same as the perceived orders as discussed in the
previous paragraph.
Given the similarity between the paradigms of Experiment 1 and of Weichselgartner and
Sperling (1987), the current results seems to contradict the AB-like deficits observed in the
68
previous study. The disparity could be due to the differences in task instructions (Olivers,
2007). In Experiment 1, subject was only asked to report the first four letter after the detection
of the cue while subject in the previous study (Weichselgartner and Sperling, 1987) was also
required to separate items attached to the cue (first glimpse) from the other items presented
latter (second glimpse). This additional requirement might have induced an AB like deficit.
The simpler task in this dissertation was able to elicit a single transient attentional episode
(gate) without evoking additional attentional operation observed in AB paradigm.
The external noise plus attention reaction time paradigm and the agPTM provide a general
framework to characterize and quantify the temporal dynamics of attention and the underlying
mechanism. Although only a single RSVP stream at fovea was used to quantify and
characterize attention switching at fovea in this dissertation, the newly developed paradigm and
the agPTM can be applied to study attention switching at different eccentricities by presenting
RSVP streams at different eccentricities and the time course of attention shift by presenting
multiple RSVPs at different spatial locations simultaneously (eg. Reeves and Sperling, 1986).

69

References
Adams, R. C., & Chambers, C. D. (2012). Mapping the timecourse of goal-directed attention to
location and colour in human vision. Acta Psychologica, 139(3), 515–523.
Barlow, H. B. (1956). Retinal noise and absolute threshold. Journal of the Optical Society of
America, 46, 634–639.
Brehaut, J. C., Enns, J. T., & Lollo, V. D. (1999). Visual masking plays two roles in the
attentional blink. Perception & Psychophysics, 61(7), 1436–1448.
Broadbent, D. E., & Broadbent, M. H. P. (1987). From detection to identification: Response to
multiple targets in rapid serial visual presentation. Perception & Psychophysics, 42(2), 105–113.
Burgess, A. E., & Colborne, B. (1988). Visual signal detection: IV. Observer inconsistency.
Journal of the Optical Society of America, 2(A), 617–627.
Burgess, A. E., Wagner, R. F., Jennings, R. J., & Barlow, H. B. (1981). Efficiency of human
visual signal discrimination. Science, 214(4516), 93–94.
Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research, 51(13), 1484–1525.
Carrasco, M., Giordano, A.M., & McElree, B. (2004). Temporal performance fields: visual and
attentional factors. Vision Research, 44(12), 1351–1365.
Carrasco, M., Giordano, A. M., & McElree, B. (2006). Attention speeds processing across
eccentricity: Feature and conjunction searches. Vision Research, 46(13), 2028–2040.
Carrasco, M., & McElree, B. (2001). Covert attention accelerates the rate of visual information
processing. Proceedings of the National Academy of Sciences, 98(9), 5363–5367.
Carrasco, M., McElree, B., Denisova, K., & Giordano, A. M. (2003). Speed of visual processing
increases with eccentricity. Nature Neuroscience, 6(7), 699–700.
Cheal, M., & Lyon, D. R. (1991). Central and peripheral precuing of forced-choice
70
discrimination. Quarterly Journal of Experimental Psychology, 43A, 859–880.
Cheal, M., & Lyon, D. R. (1994). Allocation of attention in texture segregation, visual search,
and location-precuing paradigms. Quarterly Journal of Experimental Psychology: Human
Experimental Psychology, 47A, 49–70.
Chica, A. B., & Christie, J. (2009). Spatial attention does improve temporal discrimination.
Attention, Perception, & Psychophysics, 71(2), 273–280.
Dao, D. Y., Lu, Z.-L., & Dosher, B. A. (2006). Adaptation to sine-wave gratings selectively
reduces the contrast gain of the adapted stimuli. Journal of Vision, 6(7), 739–759.
Dosher, B. A. (1979). Empirical approaches to information processing: Speed-accuracy tradeoff
functions or reaction time — a reply. Acta Psychologica, 43(5), 347–359.
Dosher, B. A., Liu, S.-H., Blair, N., & Lu, Z.-L. (2004). The spatial window of the perceptual
template and endogenous attention. Vision Research, 44(12), 1257–1271.
Dosher, B. A., Han, S. M., & Lu, Z.-L. (2004). Parallel processing in visual search asymmetry.
Journal of Experimental Psychology-Human Perception and Performance, 30(1), 3–27.
Dosher, B. A., Han, S., & Lu, Z.-L. (2010). Information-Limited Parallel Processing in Difficult
Heterogeneous Covert Visual Search. Journal of Experimental Psychology-Human Perception
and Performance, 36(5), 1128–1144.
Dosher, B. A., & Lu, Z.-L. (2000a). Mechanisms of perceptual attention in precuing of location.
Vision Research, 40(10-12), 1269–1292.
Dosher, B. A., & Lu, Z.-L. (2000b). Noise exclusion in spatial attention. Psychological Science,
11(2), 139–146.

71
Duncan, J., Ward, R., & Shapiro, K. (1994). Direct measurement of attentional dwell time in
human vision. Nature, 369(6478), 313–315.
Dux, P. E., & Marois, R. (2009). The attentional blink: A review of data and theory. Attention
Perception & Psychophysics, 71(8), 1683–1700.
Eriksen, C., & Collins, J. (1969). Temporal Course of Selective Attention. Journal of
Experimental Psychology, 80(2P1), 254–261.
Giordano, A. M., McElree, B., & Carrasco, M. (2009). On the automaticity and flexibility of
covert attention: A speed-accuracy trade-off analysis. Journal of Vision, 9(3), 30.
Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York:
Wiley.
Haber, R. N., & Nathanson, L. S. (1969). Processing of sequentially presented letters. Perception
& Psychophysics, 5(6), 359–361.
Hein, E., Rolke, B., & Ulrich, R. (2006). Visual attention and temporal discrimination:
Differential effects of automatic and voluntary cueing. Visual Cognition, 13(1), 29–50.
Hilkenmeier, F., Olivers, C. N. L., & Scharlau, I. (2012a). Prior Entry and Temporal Attention:
Cueing Affects Order Errors in RSVP. Journal of Experimental Psychology-Human Perception
and Performance, 38(1), 180–190.
Hilkenmeier, F., Scharlau, I., Weiss, K., & Olivers, C. N. L. (2012b). The dynamics of prior
entry in serial visual processing. Visual Cognition, 20(1), 48–76.
Hung, G. K., Wilder, J., Curry, R., & Julesz, B. (1995). Simultaneous better than sequential for
brief presentations. Journal of the Optical Society of America A, 12(3), 441–449.
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by Simulated Annealing.
Science, 220(4598), 671–680.
72
Kolers, P. A., & Katzman, M. T. (1966). Naming Sequentially Presented Letters and Words.
Language & Speech, 9(2), 84–95.
Legge, G. E., Kersten, D., & Burgess, A. E. (1987). Contrast discrimination in noise. Journal of
the Optical Society of America, 4(A), 391–404.
Li, X., Lu, Z.-L., Xu, P., Jin, J., & Zhou, Y. (2003). Generating high gray-level resolution
monochrome displays with conventional computer graphics cards and color monitors. Journal of
Neuroscience Methods, 130(1), 9–18.
Li, X., Liang, Z., Kleiner, M., & Lu, Z.-L. (2010). RTbox: a device for highly accurate response
time measurements. Behavior Research Methods, 42(1), 212–225.
Ling, S., & Carrasco, M. (2006). Sustained and transient covert attention enhance the signal via
different contrast response functions. Vision Research, 46(8-9), 1210–1220.
Liu, T., Stevens, S. T., & Carrasco, M. (2007). Comparing the time course and efficacy of spatial
and feature-based attention. Vision Research, 47(1), 108–113.
Lu, Z.-L., & Dosher, B. A. (1998). External noise distinguishes attention mechanisms. Vision
Research, 38(9), 1183–1198.
Lu, Z.-L., & Dosher, B. A. (2008). Characterizing observers using external noise and observer
models: Assessing internal representations with external noise. Psychological Review, 115(1),
44–82.
Lu, Z.-L., & Dosher, B. A. (2000). Spatial attention: Different mechanisms for central and
peripheral temporal precues? Journal of Experimental Psychology-Human Perception and
Performance, 26(5), 1534–1548.
Lu, Z.-L., Jeon, S.-T., & Dosher, B. A. (2004). Temporal tuning characteristics of the perceptual
template and endogenous cuing of spatial attention. Vision Research, 44(12), 1333–1350.
73
Lu, Z.-L., Tse, H. C.-H., Dosher, B. A., Lesmes, L. A., Posner, C., & Chu, W. (2009). Intra- and
cross-modal cuing of spatial attention: Time courses and mechanisms. Vision Research, 49(10),
1081–1096
Lyon, D. L. (1987). How quickly can attention affect form perception? (Tech. Rep. No. AFHRL-
TR-87-28). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.
Lyon, D. R. (1990). Large and rapid improvement in form discrimination accuracy following a
location precue. Acta Psychologica, 73, 69–82.
Macmillan, N. A., & Creelman, C. D. (1991). Detection theory: A user’s guide. New York:
Cambridge University Press.
McElree, B., & Carrasco, M. (1999). The temporal dynamics of visual search: Evidence for
parallel processing in feature and conjunction searches. Journal of Experimental Psychology:
Human Perception and Performance, 25(6), 1517–1539.
Moore, C. M., Egeth, H., Berglan, L. R., & Luck, S. J. (1996). Are attentional dwell times
inconsistent with serial visual search? Psychonomic Bulletin & Review, 3(3), 360–365.
Müller, H. J., & Rabbitt, P. M. (1989). Reflexive and voluntary orienting of visual attention:
Time course of activation and resistance to interruption. Journal of Experimental Psychology:
Human Perception and Performance, 15(2), 315–330.
Nagaraja, N. (1964). Effect of Luminance Noise on Contrast Thresholds. Journal of the Optical
Society of America, 54(7), 950–&.
Nakayama, K., & Makeben, M. (1989). Sustained and Transient Components of Focal Visual-
Attention. Vision Research, 29(11), 1631–1647.
Nakayama, K., & Martini, P. (2011). Situating visual search. Vision Research, 51(13), 1526–
1537.
74
Nieuwenstein, M. R., & Potter, M. C. (2006). Temporal limits of selection and memory encoding
- A comparison of whole versus partial report in rapid serial visual presentation. Psychological
Science, 17(6), 471–475.
Norman, D. A. (1966). Acquisition and retention in short-term memory. Journal of Experimental
Psychology, 72(3), 369–381.
Norman, D. A. (1967). Temporal confusions and limited capacity processors. Acta Psychologica,
27, 293–297.
North, D. O. (1942). The absolute sensitivity of radio receivers. RCA Review, 6, 332–344.
Olivers, C. N. L. (2007). The time course of attention - It is better than we thought. Current
Directions in Psychological Science, 16(1), 11–15.
Olivers, C. N. L., & Meeter, M. (2008). A Boost and Bounce Theory of Temporal Attention.
Psychological Review, 115(4), 836–863.
Olivers, C. N. L., Hilkenmeier, F., & Scharlau, I. (2011). Prior entry explains order reversals in
the attentional blink. Attention, Perception & Psychophysics, 73(1), 53–67.
Pelli, D. G. (1981). Effects of visual noise. Doctoral dissertation, Cambridge University,
Cambridge, England.
Pelli, D. G. (1985). Uncertainty explains many aspects of visual contrast detection and
discrimination. Journal of the Optical Society of America A, 2(9), 1508–1531.
Petersen, A., & Kyllingsbæk, S. (2013). Eye movements and practice effects in the attentional
dwell time paradigm. Experimental Psychology, 60(1), 22–33.
Posner, M., Rafal, R., Choate, L., & Vaughan, J. (1985). Inhibition of Return - Neural Basis and
Function. Cognitive Neuropsychology, 2(3), 211–228.
75
Potter, M., & Levy, E. (1969). Recognition Memory for a Rapid Sequence of Pictures. Journal of
Experimental Psychology, 81(1), 10–15.
Potter, M. C., Chun, M. M., Banks, B. S., & Muckenhoupt, M. (1998). Two attentional deficits
in serial target search: the visual attentional blink and an amodal task-switch deficit. Journal of
Experimental Psychology. Learning, Memory, and Cognition, 24(4), 979–992.
Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression of visual
processing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human
Perception and Performance, 18(3), 849–860.
Reeves, A. (1977). The identification and recall of rapidly displayed letters and digits.
Unpublished doctoral dissertation, City University of New York.
Reeves, A. (1986). Attention and the order of items in short-term visual memory. Psychological
Research, 48(4), 239–250.
Reeves, A., & Sperling, G. (1986). Attention Gating in Short-Term Visual Memory.
Psychological Review, 93(2), 180–206.
Saarinen, J., & Julesz, B. (1991). The speed of attentional shifts in the visual field. Proceedings
of the National Academy of Sciences of the United States of America, 88(5), 1812.
Shaw, M. L., & Shaw, P. (1977). Optimal allocation of cognitive resources to spatial locations.
Journal of Experimental Psychology: Human Perception and Performance, 3, 201–211.
Shih, S. I., & Sperling, G. (2002). Measuring and modeling the trajectory of visual spatial
attention. Psychological Review, 109(2), 260–305.
Shulman, G. L., Remington, R. W., & McLean, J. P. (1979). Moving attention through visual
space. Journal of Experimental Psychology: Human Perception and Performance, 2, 14–22.
76
Smith, P. L., & Ratcliff, R. (2009). An Integrated Theory of Attention and Decision Making in
Visual Signal Detection. Psychological Review, 116(2), 283–317.
Spence, C., & Parise, C. (2010). Prior-entry: A review. Consciousness and Cognition, 19(1),
364–379.
Sperling, G. (1960). The information available in brief visual presentations. Psychological
Monographs: General and Applied, 74(11), 1.
Sperling, G., & Melchner, M. J. (1976). Estimating item and order information. Journal of
Mathematical Psychology, 13(2), 192–213.
Sperling, G., & Reeves, A. (1980). Measuring the reaction time of an unobservable response: A
shift of visual attention. In R. S. Nickerson (Ed.), Attention and performance VIII (pp. 347-360).
New York: Academic Press
Sperling, G., & Weichselgartner, E. (1995). Episodic Theory of the Dynamics of Spatial
Attention. Psychological Review, 102(3), 503–532.
Stelmach, L., & Herdman, C. (1991). Directed Attention and Perception of Temporal-Order.
Journal of Experimental Psychology-Human Perception and Performance, 17(2), 539–550.
Titchener, E. M. (1908). Lectures on the elementary psychology of feeling and attention. New
York, NY: MacMillan.
Travers, J.R. (1973). The Effects of Forced Serial Processing on Identification
of Words and Random Letter Strings. Cognitive Psychology, 5, 109-137
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive
Psychology, 12, 97–136.
Tsal, Y. (1983). Movement of attention across the visual field. Journal of Experimental
Psychology: Human Perception and Performance, 8, 194–214.
77
Warner, C. B., Juola, J. F., & Koshino, H. (1990). Voluntary allocation versus automatic capture
of visual attention. Perception & Psychophysics, 48(3), 243–251.
Weichselgartner, E., & Sperling, G. (1987). Dynamics of automatic and controlled visual
attention. Science (New York, N.Y.), 238(4828), 778–780.
Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta
Psychologica, 41(1), 67–85.
Wilkens, T. D. (2002). Elementary signal detection theory. New York: Oxford University Press.
Yeshurun, Y., & Levy, L. (2003). Transient spatial attention degrades temporal resolution.
Psychological Science, 14(3), 225–231.
Zhao, Y., Lu, Z.-L., & Dosher, B.A. (2014). External noise distinguishes mechanisms underlying
attention gating in visual short-term memory. Journal of Vision, 14(10), 1121–1121.

78

Appendix 1: Tables
Table 2.1. Letter contrasts in Experiment 1.

Noise Level
Zero noise High noise
Signal (letter) contrast level Signal (letter) contrast level
Subject 1 2 3 4 5 6 1 2 3 4 5 6
YZ
0.080 0.098 0.119 0.146 0.178 1.000 0.426 0.489 0.562 0.645 0.741 1.000
KS
0.082 0.119 0.173 0.252 0.367 1.000 0.401 0.482 0.578 0.694 0.833 1.000
BF
0.059 0.076 0.098 0.126 0.162 1.000 0.290 0.345 0.409 0.486 0.578 1.000

Table 2.2. Peaks of item scores and composite item scores in Experiment 1.
Note:
The peak position is the first digit. The following digits, if any, represent stimulus
positions in descending orders of item scores whose 95% C.I.s overlap with the C.I. of the peak
position. “=” represents the equal item scores of the two stimulus positions.

Subject
Zero noise High noise
Contrast
Level

Pi(1) Pi(2) Pi(3) Pi(4) Pi Pi(1) Pi(2) Pi(3) Pi(4) Pi
YZ
1, 0 - - - 1, 2 1, 0, 2 - - - 2, 1, 0
1
1 - 3, 4, 2 - 1 1, 0 1, 2, 3 - - 1, 2, 0
2
1, 0 2, 1 - - 1 1 2, 1, 3 - - 1, 2
3
1, 0 1, 2 2, 3 - 1 1, 0 2 2, 3, 4 - 1, 2
4
1, 0 2, 1 2, 3 - 1, 2 1 2, 1, 3 3, 2 3, 4 1, 2, 3
5
1, 0 1 = 2 2, 3 3, 2, 4 1 1, 0 2 3, 2 - 1, 2
6
KS
1, 0 - - - 1, 2, 3 0,-1, 1 2, 1, 3 - - -
1
0,-1, 1 1 = 2 - - 1, 2, 0 0,-1, 1 2, 3 - - -
2
0 1, 2 - - 0, 1 0 2, 1 - - 0, 1, 2
3
0 1, 2 2, 1, 3 - 0, 1 0 1, 2 - - 0, 1
4
0 1 2, 1 - 0, 1 0 2, 1 - - 0, 1
5
0 1 2, 1 - 0, 1 0 1, 2 - - 0
6
BF
-1,0, 1 - - - - - - - - -
1
0 1 2, 1 - 0 = 1 0, 1,-1 1 - - 1, 0
2
0,-1 1, 0 - - 1, 0,-1 0, 1 - - - 1, 0
3
0 1 2, 1 - 0, 1 0, 1 1, 0 - 2, 4,-2 0, 1
4
0 1, 0 2, 1 - 0 = 1 0, 1 1, 0, 2 2, 3 - 1, 0
5
1 1, 2, 0 2 3, 2 1 1, 0 1, 2, 0 2 - 1
6
79
Table 2.3. Peaks of item scores and composite item scores in Experiment 2.
subject
Experiment 2A Experiment 2B
position
Pi(1) Pi(2) Pi(3) Pi(4) Pi Pi(1) Pi(2) Pi(3) Pi(4) Pi
YZ
1 2 3 3, 4, 0 1, 2 1 2 3 3 1, 2
control
1 2 3 4, 3 1 = 2 1 2 3 3, 4, 5 2, 1
0
0 2 3 4, 3 2 0 2 3 4, 3 2, 3, 0
1
1 3, 1 3, 4 4, 3 1 1 1, 0, 3 3, 4 4,3 1
2
1 2 2 = 4 4, 0 2, 1 1 2 2, 4 4 2
3
KS
0 1 2 - 0, 1 0 1 2, 3, 1 - 0, 1
control
1 2 3, 2,-1 - 1 1 2 3 - 1, 2
0
0 2 3 - 0 0 3 4, 3, 2 - 0
1
0 1 - - 1, 0 0 1 3, 4 - 1, 0
2
0 1 - - 0, 1 0 2, 1 1, 2 - 0, 1, 2
3
BF
1, 0 1, 0, 2 2 - 1
control
1 2 3 3, 4, 5 1
0
0 2 3, 2 - 0, 2
1
0, 1 1, 0 3 3, 4 1
2
0, 1 1 2 4, 5, 2 1
3
Note: The peak position is the first digit. The following digits, if any, represent stimulus
positions in descending orders of item scores whose 95% C.I.s overlap with the C.I. of the peak
position. “=” represents the equal item scores of the two stimulus positions.

Table 3.1.1. Parameters of simulation plots in Figure 3.1.1~3.1.11.
Figure 



T







1
N

memory

0
t

0
t

3.1.1 1.2 1.6 100 50 0.05 1.6 0.014 0.2 15 20
3.1.2 1.2 1.6 100 50 0.05 1.6 0.014 1 15 20
3.1.3 2 1.6 100 50 0.05 1.6 0.014 0.2 15 20
3.1.4 1.2 3 100 50 0.05 1.6 0.014 0.2 15 20
3.1.5 1.2 1.6 100 50 0.05 1.6 0.05 0.2 15 20
3.1.6 1.2 1.6 100 50 0.2 1.6 0.014 0.2 15 20
3.1.7 1.2 1.6 100 100 0.05 1.6 0.014 0.2 15 20
3.1.8 1.2 1.6 100 50 0.05 3 0.014 0.2 15 20
3.1.9 1.2 1.6 550 50 0.05 1.6 0.014 0.2 15 50
3.1.10 1.2 1.6 700 50 0.05 1.6 0.014 0.2 50 20
3.1.11 1.2 1.6 100 50 0.05 1.6 0.014 0.2 60 5

80
Table 3.1.2 Parameters of the gamma functions of the attention gate in Figure 3.1.12.
M 





1 1.6 50 0.05
2 1.6 50 0.20
3 1.6 100 0.05
4 3.0 50 0.05

Table 3.1.3. Parameters of the gamma functions of the attention gate opening time distribution
in Figure 3.1.13.
M
0
t

0
t

1 15 20
2 15 50
3 50 20
4 60 5

Table 3.2.1. Best fitting parameters of the strength model of signal-to-noise ratios in Experiment
1 and Experiment 2B




T







1
N

memory

0
t

0
t
 r
2

YZ 0.82 2.89 216.0 334.1 0.0556 0.82 0.0004 0.0108 19.9 13.8 0.9572
KS 1.12 1.58 357.5 683.4 0.0389 1.02 0.0189 0.2747 435.1 0.7 0.9262

Table 3.2.2. Best fitting parameters of the strength model of total energy in Experiment 1 and
Experiment 2B




T







1
N

memory

0
t

0
t
 r
2

YZ 0.60 3.18 104.3 136.9 0.0228 0.60 0.0001 0.0087 8.7 23.9 0.9035
KS 0.60 2.69 289.8 337.7 0.0510 1.16 0.0005 0.5950 132.0 1.9 0.8868

Table 3.2.3. Best fitting parameters of the strength model of signal-to-noise ratios in Experiment
1.




T







1
N

memory


0
t

0
t
 r
2

YZ 1.36 1.40 284.5 220.8 0.0621 1.12 0.0243 0.4630 26.8 12.0 0.9692
KS 1.36 1.54 265.6 371.2 0.0608 1.06 0.0205 0.5412 375.3 0.59 0.9497
BF 1.20 1.61 203.3 52.0 0.0467 1.60 0.0143 0.1147 13.4 17.4 0.9281
81
Table 3.2.4. Best fitting parameters of the strength model of total energy in Experiment 1




T







1
N

memory

0
t

0
t
 r
2

YZ 0.89 2.77 200.1 128.6 0.0718 1.39 0.0006 0.0066 10.4 23.9 0.9689
KS 0.87 2.02 303.6 874.5 0.0359 1.01 0.0049 0.2479 964.5 0.2 0.9424
BF 1.30 1.93 200.5 65.2 0.0752 1.64 0.0071 0.0068 9.1 24.6 0.9302

Table 3.2.5. Best fitting parameters of the strength model of signal-to-noise ratios in Experiment
2A.




T







1
N

memory

0
t

0
t
 X r
2

YZ 0.84 1.48 235.7 312.1 0.0436 1.00 0.0144 0.6449 32.2 8.7 0.0000 0.9619
KS 0.43 1.19 319.6 155.7 0.0511 0.58 0.0040 0.1737 44.4 8.2 0.0006 0.9496
BF 0.96 1.92 223.2 91.6 0.0440 1.33 0.0686 0.0491 35.0 7.3 0.0246 0.9563

Table 3.2.6. Best fitting parameters of the strength model of total energy in Experiment 2A.




T







1
N

memory

0
t

0
t
 X r
2

YZ 0.78 1.21 123.3 289.5 0.0314 1.19 0.0145 0.8743 27.6 9.4 0.0010 0.9531
KS 0.85 1.30 102.4 75.6 0.0190 0.46 0.0045 0.0026 15.8 16.2 0.0002 0.9205
BF 0.13 1.32 101.7 117.0 0.0605 1.05 0.0002 0.0216 16.7 15.3 0.0091 0.9385
*Note: X is the equivalent signal contrast of the noise-perturbed letter, set as a free parameter

Table 3.3.1. AM 1 best fitting parameters of strength model of signal-to-noise ratio in
Experiment 1 and Experiment 2B.



0
t







1
N

memory


r
2

YZ 1.18 1.49 237.8 361.4 0.0392 1.33 0.0149 0.4401 0.9286
KS 1.26 1.01 231.7 292.2 0.0561 1.36 0.0639 0.3489 0.9212

Table 3.3.2. AM 1 best fitting parameters of the strength model of total energy in Experiment 1
and Experiment 2B.



0
t







1
N

memory


r
2

YZ 0.66 3.31 159.7 1652.4 0.0104 1.17 0.0001 0.1076 0.9038
KS 0.65 3.43 256.8 309.7 0.0496 1.17 0.0001 0.7517 0.8823
82
Table 3.3.3. AM 1 best fitting parameters of strength model of signal-to-noise ratio in
Experiment 1.



0
t







1
N

memory


r
2

YZ 1.30 1.41 318.8 226.6 0.0671 1.26 0.0212 0.7642 0.9362
KS 1.35 1.60 237.3 447.4 0.0551 1.11 0.0169 0.3798 0.9537
BF 2.26 1.08 202.2 145.1 0.0725 2.08 0.0658 0.7643 0.9218

Table 3.3.4. AM 1 best fitting parameters of the strength model of total energy in Experiment 1.



0
t







1
N

memory


r
2

YZ 1.15 1.42 304.0 236.0 0.0588 1.43 0.0280 0.7152 0.9390
KS 1.03 1.60 236.4 228.4 0.0602 1.38 0.0140 0.9633 0.9434
BF 1.76 1.18 200.6 176.3 0.0647 1.88 0.0574 0.7721 0.9243

Table 3.3.5. AM 1 best fitting parameters of strength model of signal-to-noise ratio in
Experiment 2A.



0
t







1
N

memory


x r
2

YZ 0.7385 1.1383 307.6 286.6 0.0548 1.3353 0.0208 0.5231 0.0159 0.9388
KS 0.8443 1.1687 252.7 209.3 0.0678 1.3643 0.0698 0.4997 0.0084 0.9491
BF 2.6321 1.1745 232.7 179.5 0.0630 1.7939 0.0381 0.6143 0.0105 0.9548

Table 3.3.6. AM 1 best fitting parameters of the strength model of total energy in Experiment
2A.



0
t







1
N

memory


x r
2

YZ 1.0232 1.9553 278.0 257.1 0.0421 1.6522 0.0199 0.6962 0.0115 0.9218
KS 0.8467 1.0746 240.0 278.2 0.0521 1.3193 0.0460 0.7526 0.0094 0.9428
BF 2.1272 1.3609 207.1 192.6 0.0483 1.9673 0.0516 0.7591 0.0136 0.9402
*Note: x is the equivalent signal contrast of the noise-perturbed position, set as a free
parameter

83

Appendix 2: Figures

Figure 1.1.1. Attention Gating Model (AGM: Reeves and Sperling, 1986).

Figure 1.1.2. The gamma function of an attention gate.

Note: the shaded area under the gate is the amount of attention allocated to input.
memory noise
attention gate
84
Figure 1.2. Perceptual template model (PTM: Lu and Dosher, 1998).

Figure 1.3. Attention gating perceptual template model (agPTM).

Input Perceptual template
Nonlinearity Decision
Multiplicative noise Additive noise
85
Figure 1.4.a. Temporal integration of signal and noise frame in Experiment 1 and Experiment
2B.

Zero noise

High noise
two frames of 33% Gaussian noise were added to each stimulus position

Figure 1.4.b. Temporal integration of signal and noise frame in Experiment 2A. In this example,
noise is added to position 1.

10 letters / second
. . . . . .
20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms
10 letters / second
. . . . . .
20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms
10 letters / second
. . . . . .
20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms 20 ms
86
Figure 1.5. Sample 33% Gaussian noise in Experiment 1 and Experiment 2B (left); phase-
scrambled noise in Experiment 2A (right).

Figure 1.6. Cue in a single trial.

87
Figure 2.1. Items scores, order scores, and bootstrap 95% confidence intervals in Experiment 1
for YZ (left: zero noise; right: high noise).

Figure 2.2. Items scores, order scores and bootstrap 95% confidence intervals in Experiment 1
for KS (left: zero noise; right: high noise).

Figure 2.3. Items scores, order scores and bootstrap 95% confidence intervals in Experiment 1
for BF (left: zero noise; right: high noise).

88
Figure 2.4. Items scores, order scores and bootstrap 95% confidence intervals in Experiment 2A
(left) and Experiment 2B (right) for YZ.

Figure 2.5. Items scores, order scores and bootstrap 95% confidence intervals in Experiment 2A
(left) and Experiment 2B (right) for KS.

Figure 2.6. Items scores, order scores and bootstrap 95% confidence intervals in Experiment 2A
(left) for BF.

89
Figure 3.1.1. Simulation of agPTM with parameters in the first row in Table 3.1.1.

Figure 3.1.2. Simulation of agPTM when
memory

increases from 0.2 to 1.

Figure 3.1.3. Simulation of agPTM when β increases from 1.2 to 2.

Figure 3.1.4. Simulation of agPTM when

increases from 1.6 to 3.

90
Figure 3.1.5. Simulation of agPTM when
1
N increases from 0.014 to 0.05.
Figure 3.1.6. Simulation of agPTM when

increases from 0.05 to 0.2.

Figure 3.1.7. Simulation of agPTM when  increases from 50 to 100.

91
Figure 3.1.8. Simulation of agPTM when  increases from 1.6 to 3.

Figure 3.1.9. Simulation of agPTM when
0
t

increases from 20 to 50.

Figure 3.1.10. Simulation of agPTM when
0
t

increases from 15 to 50.

92
Figure 3.1.11. Simulation of agPTM when
0
t

=60 and
0
t

=5.

Figure 3.1.12. The gamma functions of the attention gate.

Figure 3.1.13. The gamma functions of the attention gate opening time distribution.

93
Figure 3.1.14. Predictions by the strength model of signal-to-noise ratio (left) and total energy
(right).

Figure 3.2.1. Predictions by the strength model of signal-to-noise ratio (left column) and data
(right column) in Experiment 1 and Experiment 2B for YZ (r
2
=0.9572).

94
Figure 3.2.2. Predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 and Experiment 2B for YZ (r
2
=0.9035).

Figure 3.2.3. Predictions by the strength model of signal-to-noise ratio (left column) and data
(right column) in Experiment 1 and Experiment 2B for KS (r
2
=0.9262).

95

Figure 3.2.4. Predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 and Experiment 2B for KS (r
2
=0.8868).

96

Figure 3.2.5. Predictions by the strength model of signal-to-noise ratio (left column) and data
(right column) in Experiment 1 for YZ (r
2
=0.9692).

97
Figure 3.2.6. Predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 for YZ (r
2
=0.9689).

Figure 3.2.7. Predictions by the strength model of signal-to-noise ratio (left column) and data
(right column) in Experiment 1 for KS (r
2
=0.9497)

98
Figure 3.2.8. Predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 for KS (r
2
=0.9424).

Figure 3.2.9. Predictions by the strength model of signal-to-noise ratio (left column) and data
(right column) in Experiment 1 for BF (r
2
=0.9281).

99
Figure 3.2.10. Predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 for BF (r
2
=0.9302).

Figure 3.2.11. Predictions by the strength model of signal-to-noise ratio (r
2
=0.9619) and total
energy (r
2
=0.9531) (left column) and data (right column) in Experiment 2A for YZ.

100
Figure 3.2.12. Predictions by the strength model of signal-to-noise ratio (r
2
=0.9496) and total
energy (r
2
=0.9205) (left column) and data (right column) Experiment 2A for KS.

Figure 3.2.13. Predictions by the strength model of signal-to-noise ratio (r
2
=0.9563) and total
energy (r
2
=0.9385) (left column) and data (right column) in Experiment 2A for BF.

101
Figure 3.3.1. AM 1 predictions by the strength model of signal-to-noise ratio (left column) and
data (right column) in Experiment 1 and Experiment 2B for YZ (r
2
= 0.9286).

Figure 3.3.2. AM 1 predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 and Experiment 2B for YZ (r
2
=0.9038).

102

Figure 3.3.3. AM 1 predictions by the strength model of signal-to-noise ratio (left column) and
data (right column) in Experiment 1 and Experiment 2B for KS (r
2
= 0.9212).

103

Figure 3.3.4. AM 1 predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 and Experiment 2B for KS (r
2
=0.8823).

104
Figure 3.3.5. AM 1 predictions by the strength model of signal-to-noise ratio (left column) and
data (right column) in Experiment 1 for YZ (r
2
=0.9362).

Figure 3.3.6. AM 1 predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 for YZ (r
2
=0.9390).

105

Figure 3.3.7. AM 1 predictions by the strength model of signal-to-noise ratio (left column) and
data (right column) in Experiment 1 for KS (r
2
=0.9537).

Figure 3.3.8. AM 1 predictions by the strength model of total energy (left column) and data (right
column) in Experiment 1 for KS (r
2
=0.9434).

106

Figure 3.3.9. AM 1 predictions by the strength model of signal-to-noise ratio (left column) and
data (right column) in Experiment 1 for BF (r
2
=0.9218).

Figure 3.3.10. AM 1 predictions by the strength model of total energy (left column) and data
(right column) in Experiment 1 for BF (r
2
=0.9243).

107

Figure 3.3.11. AM 1 predictions by the strength model of signal-to-noise ratio (r
2
=0.9388) and
total energy (r
2
=0.9218) (left column) and data (right column) in Experiment 2A for YZ.

Figure 3.3.12. AM 1 predictions by the strength model of signal-to-noise ratio (r
2
=0.9491) and
total energy (r
2
=0.9428) (left column) and data (right column) in Experiment 2A for KS.

108

Figure 3.3.13. AM 1 predictions by the strength model of signal-to-noise ratio (r
2
=0.9548) and
total energy (r
2
=0.9402) (left column) and data (right column) in Experiment 2A for BF.

Figure 3.4.1. AM 2 predictions by the strength model of signal-to-noise ratio in Experiment 1.

109
Figure 3.4.2. AM 2 predictions by the strength model of total energy in Experiment 1.

Abstract (if available)

Abstract Attention is essential in selecting task-relevant sensory information for further processing. It has been shown that attentional selection affects the discriminability of sensory inputs in terms of report accuracy. Furthermore, the temporal dynamics of attention could interact with short-term memory and decision. Specifically, the amount of the attention allocated to the sensory inputs has been related to the systematic distortion of perceived order. This dissertation is among the first to characterize, quantify, and model the effects of attentional selection on perception, and the interactions of the temporal dynamics of attention with short-term memory and decision in a single study. A new behavioral paradigm, the external noise plus attention reaction time paradigm, was developed in which four letters immediately following a cue in a RSVP stream of letters at fovea were reported in each trial. Data were collected at a wide range of stimulus conditions, including low signal contrast, high external noise and un-equal item signal-to-noise ratios. Subjects’ performance were characterized as item scores and order scores. Item scores were defined as the probability of reporting an item in each of the target positions and reflected the discriminability of each target item. Order scores were defined as the probability that an item in one target position is reported before an item in another target position and reflected the relative strengths of items in short-term memory. We found that 35% of the order scores were reversed, that is, the probability that an item presented later was reported before an item presented earlier was 0.35. A new model, the attention gating perceptual template model (agPTM), was developed to characterize and quantify the underlying cognitive processes. In the agPTM, the sensory input goes through a contrast gain control process and then is gated by attention into short-term memory, with added internal noise at each stage. The signal-to-noise ratios determine both the item discriminability and memory strength. Items are reported in descending orders of their memory strengths. The agPTM well accounted for all the data of item scores and order scores with a single set of parameters for each subject, suggesting invariance of the temporal dynamics of attention switching across all the stimulus conditions tested. In conclusion, the temporal dynamics of attention switching and its interactions with perception, short-term memory and decision have been systematically measured and quantified by the external noise plus attention reaction time paradigm and accounted for by the agPTM. Our results indicate that attention gates information from perception (contrast gain control) into short-term memory. The attention gating determines the item signal-to-noise ratios in memory, thus both the discriminability and memory strength which are quantified as item scores and order scores, respectively. The timecourse of attention switching was found to be invariant to stimulus inputs. The novel paradigm and the new model provide a framework to further study the temporal properties of attention and its interactions with perception as well as other cognitive processes.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Spatiotemporal processing of saliency signals in the primate: a behavioral and neurophysiological investigation

PDF

Temporal dynamics of perceptual decision

PDF

Computational modeling and utilization of attention, surprise and attention gating

PDF

Computational modeling and utilization of attention, surprise and attention gating [slides]

PDF

Emotion, attention and cognitive aging: the effects of emotional arousal on subsequent visual processing

PDF

Selectivity for visual speech in posterior temporal cortex

PDF

Explicit encoding of spatial relations in the human visual system: evidence from functional neuroimaging

PDF

The role of similarity in restoring missing notes in music

PDF

Quantification and modeling of sensorimotor dynamics in active whisking

PDF

Aging, Azheimer’s disease and the visual sensory memory system

PDF

Functional models of fMRI BOLD signal in the visual cortex

PDF

The symbolic working memory system

PDF

Learning lists and gestural signs: dyadic brain models of non-human primates

PDF

A million-plus neuron model of the hippocampal dentate gyrus: role of topography, inhibitory interneurons, and excitatory associational circuitry in determining spatio-temporal dynamics of granul...

PDF

Synaptic integration in dendrites: theories and applications

PDF

Exploring sensory responses in the different subdivisions of the visual thalamus

PDF

Trunk control during dynamic balance: effects of cognitive dual-task interference and a history of recurrent low back pain

Asset Metadata

Creator Zhao, Yukai (author)

Core Title Temporal dynamics of attention: attention gating in rapid serial visual presentation

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Neuroscience

Defense Date 11/05/2015

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag attention,attention cueing,memory strength,OAI-PMH Harvest,short-term memory

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Tjan, Bosco (committee chair), Biederman, Irving (committee member), Itti, Laurent (committee member), Lu, Zhong-Lin (committee member)

Creator Email yukaizha@usc.edu,zhaoyukai1986@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-219709

Unique identifier UC11277535

Identifier etd-ZhaoYukai-4184.pdf (filename),usctheses-c40-219709 (legacy record id)

Legacy Identifier etd-ZhaoYukai-4184.pdf

Dmrecord 219709

Document Type Dissertation

Format application/pdf (imt)

Rights Zhao, Yukai

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA