Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Moving object detection on a runway prior to landing using an onboard infrared camera
(USC Thesis Other)
Moving object detection on a runway prior to landing using an onboard infrared camera
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING AN ONBOARD
INFRARED CAMERA
by
Cheng-Hua Jeff Pai
A Thesis Presented to the
FACULTY OF THE VITERBI SCHOOL OF ENGINEERING
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF SCIENCE
(COMPUTER SCIENCE)
May 2007
Copyright 2007 Cheng-Hua Jeff Pai
ii
Table of Contents
List of Tables iv
List of Figures vi
Abstract viii
1. Introduction 1
2. System Overview 3
3. Stabilization Module 5
3.1. Runway Identification 5
3.2. Image Stabilization 7
3.3. Reference Frame Update 8
4. Motion Detection Module 11
4.1. Runway Filter 14
4.2. Gain Compensation 14
4.3. Background Modeling 18
4.4. Identify Constant Pixels 19
4.5. Evaluating Warp Quality 20
4.6. Noise Reduction 20
5. Experimental Result 27
5.1. Synthetic Data 27
5.2. Real World Data 28
6. Discussion 31
7. Conclusion 33
Bibliography 39
iii
Appendix A: Synthetic object moving in diagonal direction 40
Appendix B: Synthetic object moving in horizontal direction 45
Appendix C: Synthetic object moving in vertical direction 50
iv
List of Tables
Table A-1: Speed vs. detection of 2-by-2 object moving in diagonal direction 40
Table A-2: Speed vs. noise of 2-by-2 object moving in diagonal direction 40
Table A-3: Speed vs. detection of 3-by-3 object moving in diagonal direction 41
Table A-4: Speed vs. noise of 3-by-3 object moving in diagonal direction 41
Table A-5: Speed vs. detection of 4-by-4 object moving in diagonal direction 42
Table A-6: Speed vs. noise of 4-by-4 object moving in diagonal direction 42
Table A-7: Speed vs. detection of 5-by-5 object moving in diagonal direction 43
Table A-8: Speed vs. noise of 5-by-5 object moving in diagonal direction 43
Table A-9: Speed vs. detection of 6-by-6 object moving in diagonal direction 44
Table A-10: Speed vs. noise of 6-by-6 object moving in diagonal direction 44
Table B-1: Speed vs. detection of 2-by-2 object moving in horizontal direction 45
Table B-2: Speed vs. noise of 2-by-2 object moving in horizontal direction 45
Table B-3: Speed vs. detection of 3-by-3 object moving in horizontal direction 46
Table B-4: Speed vs. noise of 3-by-3 object moving in horizontal direction 46
Table B-5: Speed vs. detection of 4-by-4 object moving in horizontal direction 47
Table B-6: Speed vs. noise of 4-by-4 object moving in horizontal direction 47
Table B-7: Speed vs. detection of 5-by-5 object moving in horizontal direction 48
Table B-8: Speed vs. noise of 5-by-5 object moving in horizontal direction 48
v
Table B-9: Speed vs. detection of 6-by-6 object moving in horizontal direction 49
Table B-10: Speed vs. noise of 6-by-6 object moving in horizontal direction 49
Table C-1: Speed vs. detection of 2-by-2 object moving in vertical direction 50
Table C-2: Speed vs. noise of 2-by-2 object moving in vertical direction 50
Table C-3: Speed vs. detection of 3-by-3 object moving in vertical direction 51
Table C-4: Speed vs. noise of 3-by-3 object moving in vertical direction 51
Table C-5: Speed vs. detection of 4-by-4 object moving in vertical direction 52
Table C-6: Speed vs. noise of 4-by-4 object moving in vertical direction 52
Table C-7: Speed vs. detection of 5-by-5 object moving in vertical direction 53
Table C-8: Speed vs. noise of 5-by-5 object moving in vertical direction 53
Table C-9: Speed vs. detection of 6-by-6 object moving in vertical direction 54
Table C-10: Speed vs. noise of 6-by-6 object moving in vertical direction 54
vi
List of Figures
Figure 1: Flow chart of the system 4
Figure 2: Detailed flow chart of the Stabilization module 6
Figure 3: Extracted SIFT features in two consecutive images 8
Figure 4: Ratio of the lower edge length. Reference frame (a), and current 11
frame after stabilization (b)
Figure 5: Flow chart of Motion Detection module 13
Figure 6: Runway image before runway filtering (a), and after runway 14
filtering (b)
Figure 7: Gain compensation result. (a) is frame 1 before gain 15
compensation while (b) is frame 1 after gain compensation
Figure 8: Difference between frame 105 and 106 before gain 16
compensation (a) and after gain compensation (b)
(The intensity is scaled up by 5 to enhance readability)
Figure 9: Reference image intensity vs. runway image intensity before 16
intensity normalization (a) and after intensity normalization (b)
Figure 10: (a) is the foreground mask and (b) is the constant foreground 20
pixels of the same frame
Figure 11: Foreground mask of a poorly stabilized image 21
Figure 12: Flow chart of the transformation refinement process 23
Figure 13: A runway image before stabilization (a), after stabilization (b) 24
and extracted moving objects (c)
vii
Figure 14: Foreground intensity of a noisy image vs. Foreground intensity 25
of background model
Figure 15: Frame number vs. number of foreground pixels before applying 25
normalization process on the foreground pixels (a) and after
applying normalization process on the foreground pixels (b)
Figure 16: Foreground mask before noise reduction (a) and after noise 26
reduction (b)
Figure 17: Speed vs. Detection rate 30
Figure 18: Speed vs. Detection rate 35
Figure 19: Speed vs. full detection 36
Figure 20: Speed vs. partial detection 37
Figure 21: Speed vs. > 1 / 2 partial detection 38
viii
Abstract
DETECTING MOVING OBJECTS IN RUNWAY SEQUENCE CAPTURED BY AN
ONBOARD INFRARED CAMERA
Determining the status of a runway prior to landing is essential for any aircraft,
whether manned or unmanned. In this thesis, we present a method that can detect
moving objects on the runway from an onboard infrared camera prior to landing.
Since the runway is a planar surface, we first locally stabilize the sequence to
automatically selected reference frames using feature points in the neighborhood
of the runway. We normalize the stabilized sequence to compensate for the global
intensity variation caused by the gain control of the infrared camera. Then we
create an appearance model to learn the background. And finally we identify
moving objects by comparing the image sequence with the background model.
We have tested our system with both synthetic and real world data and provide
a quantitative analysis of the performance with respect to variations in size,
direction and speed of the target.
1
1. Introduction
A system for detecting moving objects on the runway prior to landing using an
onboard camera is helpful not only for unmanned air vehicles (UAVs), but also for
pilots to determine whether it is safe to land.
Such system should have the following properties:
1. Ability to detect distant moving objects, in order to give enough response time
for UAVs and pilots.
2. Robustness to plane motion.
3. Robustness to illuminations, as to provide around-the-clock functionality.
In this thesis, we present a method designed to detect moving objects from an
infrared runway sequence taken by an airplane prior to landing.
Previous methods for detecting motion on a moving camera include optical flow
based approach [3, 5, 7] and background subtraction based approach [9]. Our
system belongs to the latter group.
Optical flow approaches require the availability of camera motion parameters
(position and velocity) to estimate object range. [3, 5, 7] In [5], the optical flow is
first calculated for extracted features. A Kalman filter then uses the optical flow to
calculate the range of those features. Finally, the range map is used to detect
obstacles. In [7], the model flow field and residual flow field are first initialized with
the camera motion parameters. Obstacles are then detected by comparing the
2
expected residual flow with the observed residual flow field. Instead of calculating
optical flow for the whole image as in [7], [3] only calculates optical flow for
extracted features since full optical flow is unnecessary and unreliable.
In contrast to the optical flow approaches, the background subtraction approach
[9] does not need camera motion parameters. Camera motion is compensated by
estimating the transformation between two images from feature points. Moving
objects are detected by finding the frame differences between the
motion-compensated pairs.
Comparing to previous methods, the scale of our moving objects is
considerably smaller. Therefore, the optical flow method may not be able to detect
moving objects in the scale of our interest.
Our work is different from [9] in that, instead of stabilizing consecutive frames
and comparing them to find the changing parts in the sequence, we stabilize
multiple frames to a local reference frame and use a background model to detect
changes caused by moving objects.
We first give an overview of the system in section 2, followed by detailed
descriptions in section 3 and 4 of this thesis. Experimental results and discussions
are given in section 5 and 6, and finally a conclusion in section 7.
3
2. System Overview
Background modeling is a method for detecting moving objects in sequences
captured by static cameras [6]. However, this method is not appropriate if the
camera moves or the background intensity changes. Therefore, we divide the
problem into two steps: stabilization and motion detection. The first step
compensates for the camera movement by stabilizing the runway. Once the
stabilized runway is obtained, we use a background model to segment moving
blobs on the runway.
Our approach is summarized in Figure 1. First, a Mark Runway step identifies a
4-sided polygon which contains the runway in the first image.
Once this 4-sided polygon region is selected, a Stabilize process estimates the
homographies between the selected regions in each pair of consecutive images.
Images are then warped to automatically selected reference frames to form a
locally stabilized image sequence. The reference frame is updated when necessary
and the homgraphy is reset to the identity matrix. The homographies along with
the locally stabilized image sequence are then given to the Motion Detection
process where global intensity variation is compensated and moving objects on the
image are identified.
4
Video in
Runway Identification
Image Stabilization
Motion Detection
Blobs in
Motion
Locally
stabilized
image
sequence
Homographies
H
i,ref
Update
Reference
Frame?
Reference
Frame
Yes
Update Reference
Frame
Figure 1: Flow chart of the system.
5
3. Stabilization Module
Since it is very hard to directly detect moving objects when the camera also
moves, our first goal is to stabilize the image sequence. We assume that the ground
is a planar surface, which is a reasonable assumption in the neighborhood of a
runway. With this assumption, the change of viewpoint between two adjacent
frames can be represented by a homography. The problem now is to estimate the
parameters of the homography. Figure 2 shows the detailed algorithm of the
module.
3.1. Runway Identification
Because the stabilization process requires a planar surface, and non-planar areas
in the image sequence do not fit the transformation and may invalidate it, we need
to restrict the region of interest. We first tried a vanishing line method to stabilize
the image below the vanishing line. However, since the bottom of the image is
closer to the airplane, the height of the buildings and trees make the process
unstable. Knowing that, we select the planar region around the runway, and the
stabilization process is applied only to this region. We hand-picked the polygons for
our test sequences; however, with the help of onboard devices such as GPS, the
vertex location can be calculated automatically in an operational scenario.
6
input
Locally stabilized image sequence and
ref i
H
,
for all i s
3. Match features to previous frame
to establish correspondence
1. Extract SIFT features
Landing UAV
image sequence
Manually labeled
planar region
Stabilization:
For each image in the sequence
2. Region of Interest
4. Use RANSAC to remove outliers
and estimate homography
6. Warp to the reference frame
Figure 2: Detailed flow chart of the Stabilization module.
output
5. Update reference frame if
necessary
7
3.2. Image Stabilization
Let
i
I be the
th
i frame in the video sequence starting at 0,
i
R be the region
of interest in
i
I , and
1 , - i i
H be the homography between
i
R and
1 - i
R . We
have
i i i i
R H R
1 , 1 - - = and
1 , 1 - - =
i i i i
R H R . A reference frame can be any frame in the
image sequence. We use ref to represent the index of the reference frame which
the current image registers to. It is initialized to 0, and is automatically updated.
Hence we have
ref m if
ref n if
R H
R
R
m ref m
n
ref
>
=
=
,
,
,
The homography of the current region
m
R with respect to its (local) reference
frame
ref
R is derived as
∏
+ =
- =
m
ref i
i i ref m
H H
1
1 , ,
In this step, we decided to use feature points to find a robust estimation of the
perspective transformation. We first used Harris corner features [2] for this task,
but these corner features are not stable enough to approximate the correct
transformation. We then used SIFT (Scale Invariant Feature Transform) [4] features,
and obtained a much better result.
SIFT points within the runway in each pair of consecutive images are extracted
and matched (Figure 3). The matching process checks the 128 feature descriptors
8
for each point on adjacent images to find the best match. Then a RANSAC [1]
process is applied to estimate the best perspective transformation for the pair of
images from the matched feature points. The RANSAC process chooses 8 random
correspondences from the matched feature pairs (2 pairs from each quadrant), and
calculates a perspective transformation from it. The RANSAC process is applied
2000 times and the transformation is applied to all the matched pair to check for
correctness. The transformation that is correct for the most matched pairs is chosen
as the best approximation.
Figure 3: Extracted SIFT features in two consecutive images.
3.3. Reference Frame Update
In our early implementation, a single reference frame was chosen for the whole
sequence. However, it is not a good idea for a long sequence since small errors are
inevitable when doing registration and these errors may accumulate to affect the
results of later frames. To resolve this problem, we allow updating of reference
frame in the stabilization module.
Correspondence
9
One option is to swap reference frame on a fixed interval. The problem with this
approach is that the interval is too short when the airplane is very far from the
runway, since the scale of runway does not change much during the interval, but
the interval is too long when the airplane is close to the runway as the aspect
changes quickly.
We need to define a measure that gives a longer interval when the runway is far
and a shorter interval when the runway is near.
Since the transformation is close to affine, we first check the scale terms of the
transformation matrix to decide whether to swap the reference frame. In the
warping process, the location of each pixel is mapped to its corresponding position
in the reference frame by the equation
=
⋅
1 1
'
'
y
x
i h g
f e d
c b a
y
x
k , where
i h g
f e d
c b a
is the transformation from the current frame to the reference frame
and k is a scaling factor. We observed that the scale terms in the transformation
matrix, namely term a and e , increase continuously throughout the sequence.
Therefore, we set a threshold on the two terms and swap the reference frame
when either term is more than 25% different from that of the reference frame.
10
While this approach works, it is not stable as the runway in two consecutive
reference frames does not appear to have a 25% scale difference.
We then examine the tilting angle of the warped image to determine when to
update the reference frame. It turns out that we could not find this angle because
the depth information is missing in this transformation matrix. Therefore, we have
to base our decision on other measures.
We noticed that after warping, the image corners changes its angle. We made
the decision by thresholding on those angles, but we found that this method is not
stable, either. As the plane turns by rotates towards one direction, the angles of the
warped image corners also changes, which may affect the decision.
Finally, we base our decision on the ratio of the lower edge length before and
after the warping as in figure 4 to decide whether to swap a reference frame. We
chose the threshold to be 8 . 0 ) /( ) ( = before length after length . Since this ratio is
related to the tilting angle, thresholding on this measure has the same effect as
thresholding on the tilting angle. This measure gives the most stable result of the
four measures we've tried.
11
(a) (b)
Figure 4: Ratio of the lower edge length. Reference frame (a), and current frame after
stabilization (b).
4. Motion Detection Module
Now that we have an online stabilization process, the next problem is to find the
moving objects in the sequance. The method we choose for this task is background
modeling. The approach for this module can be summarized by the flow chart in
Figure 4.
Let
i
f be the runway filter with respect to the reference image
i
I ,
i
C and
i
fg be the constant foreground pixels and foreground pixels of the image
i
I , and
μ ,
2
σ be the mean and standard deviation of the background model with respect
to the reference image
i
I .
The module first determines if there is a reference frame change, and updates
the background model accordingly. Then the locally stabilized runway sequence
with respect to reference frame
i
I is passed through a runway mask
i
f to filter
out non-runway areas. Then the image intensity values are compensated to account
12
for global intensity changes by comparing it to the intensities in the reference frame.
After the intensity normalization, the image is compared with the background
model. Pixels incongruent to the background are marked as foreground. We then
examine the quality of the foreground mask by counting the number of foreground
pixels. If the number is low, we output the foreground mask and update the
background model. Otherwise, the image is refined with a noise reduction process.
13
Locally
Stablized
Runway
Sequence
Reference
Frame update?
Filter Runway
Intensity
Normalization
Intensity
compensated
runway image
Image Subtraction
Foreground
mask &
Quality
Indicator
QI. Score
Noise reduction
Update Runway
Filter
Runway
Filter
Update Reference
Frame
Reference
Frame
Update Background
Model
Background
model
Foreground
mask
Yes
No
Good
Bad
Motion Detection Module
Homographies
H i,ref
Figure 5: Flow chart of Motion Detection module.
14
4.1. Runway Filter
Since the area of interest is strictly the runway, other areas can be filtered out.
The runway filter
i
f is a binary image in the shape of the runway. The process
simply applies an “and” operation on the image and the binary mask to single out
the area of interest.
(a) (b)
Figure 6: Runway image before runway filtering (a), and after runway filtering (b).
When the reference frame changes from
i
I to
j
I , we apply the following
equation to the runway filter:
j i j i
f H f
,
=
4.2. Gain Compensation
According to [8], the intensity between any two images with different gains can
be modeled by an affine transformation.
15
j i j i i j i j
b y x I m y x I y x
, , ,
) , ( ) , ( ) , ( ε + + = ∀
By ignoring the saturated pixels, the transformation can be estimated by LMSE
(Least Mean Square Estimation) and the gain can be compensated. (Figure 6, 7, and
8)
(a) (b)
Figure 7: Gain compensation result. (a) is frame 1 before gain compensation while (b) is frame 1
after gain compensation.
Because LMSE is used, small errors can be introduced in the compensation. If the
whole sequence is compensated recursively, the errors accumulate to affect the
global intensity of later frames. To increase the accuracy of the gain compensation,
we correct the global intensity with respect to the reference frame of the current
frame, rather than between two adjacent frames.
i ref i ref ref i ref i
b I m I
, , ,
ε + + =
As a result, errors no longer accumulate and the intensity of the image is stable.
16
(a) (b)
Figure 8: Difference between frame 105 and 106 before gain compensation (a) and after gain
compensation (b). (The intensity is scaled up by 5 to enhance readability.)
(a) (b)
Figure 9: Reference image intensity vs. runway image intensity before intensity normalization (a)
and after intensity normalization (b).
Since we have multiple reference frames, we have to adjust the intensity of all
reference frames to a standard background intensity before compensating other
frames against them. We first tried to normalize the reference frames with the
mean of the background model, but found this method not stable.
i ref i ref ref i ref i
b m I
, , ,
ε μ + + =
The runway in the initial frame may be small and have narrow intensity range,
and after each update of the reference frame, as the runway gets closer, the
17
intensity range of the runway becomes wider and the image is more detailed. Our
background model, on the other hand, remains narrow in intensity range. Even if
the learning rate is increased, we still get very dark or very bright images at the end
of some sequences. The reason is that, by normalizing the reference frame against
the background model and the rest of the frames against the reference frame, we
force every frame to behave like the initial background. As a result, the intensity
range of the initial background greatly affects the compensation quality. We also
tried to linearly scale the first reference frame to increase its intensity range using
the following equation, but this produced similar results.
T I I
I I
S
I + - - = )] min( [
) min( ) max(
0 0
0 0
'
0
In which S is the range of
'
0
I and T is the minimum value of
'
0
I .
Knowing that normalizing the reference frame against our background model
won’t work, we tried to adapt our background model to new reference frames.
Instead of normalizing a new reference frame against the background model, we
normalize our background model against the new reference frame.
ref ref ref ref ref
b I m ε μ + + =
By doing so, we not only reduce the effect of the first reference frame, but also
increase the accuracy of our background model.
18
4.3. Background Modeling
A Gaussian distribution is used to model the intensity of each pixel [6]. The mean
of the Gaussian model is initialized to the value of the base image.
0 0
I = μ
The variance is initialized to a constant value (currently 5). Both mean and
variance are updated for each new frame according to the following formula where
is the learning rate (currently 0.02).
) ( ) )( 1 (
1 i i i
I ρ μ ρ μ + - =
-
) ( ) )( 1 (
2
1
2
i i i i
I μ ρ σ ρ σ - + - =
-
Pixels having intensity difference greater than
2
4
i
σ from μ are marked as
foreground pixels in the foreground mask.
When the reference frame updates, we need to adjust the orientation of our
background model. Since the background model models the previous reference
frame, by applying the perspective transformation between the reference frames to
the background model, we obtain a background model for the new reference
frame.
j i j i
H μ μ
,
=
j
i j
i H
2
,
2
σ σ =
19
However, each reference frame update also brings more details. These details
may be classified as foreground because they were not present in our background
model. To reduce the effect of details, we increase the learning rate of our
background model from 0.005 to 0.02 so it can quickly learn them. We also update
the background model with the new reference frame right after the transformation.
4.4. Identify Constant Pixels
Since the scale of our moving objects is small, we cannot apply morphological
operations to each foreground mask to reduce noise, as it may remove our targets,
too. However, random noise can be suppressed by comparing two consecutive
foreground masks.
After the binary foreground mask is produced, it is compared with the previous
mask for constant foreground pixels. Constant pixels include pixels that are marked
as foreground in both the previous and the current mask, and pixels that have
moved one pixel in any direction. To find the constant pixels, the previous
foreground mask is first dilated, and then a binary “and” operation is applied to the
dilated mask and the current mask. (Figure 9(b))
) , ( ) , ( ) , ( ) , (
1 , 1
1 , 1
1
y x fg y x fg y x C y x
i
y b x a
y b x a
i i
∩
= ∀
+ = + =
- = - =
- U
20
In which
i
C and
i
fg denotes constant pixels and foreground in frame i
respectively.
(a) (b)
Figure 10: (a) is the foreground mask and (b) is the constant foreground pixels of the same frame.
4.5. Evaluating Warp Quality
To evaluate the quality of a resulting foreground mask, we need a quality
indicator. One such indicator is the total number of foreground pixels in the
foreground mask, as illustrated in Figure 10. If the number is greater than a
threshold (currently 250), the result is considered poor.
4.6. Noise Reduction
If we know that a foreground mask is poor, we can go back and try to refine the
stabilized image in order to improve the result. Since we used feature points during
the stabilization step, we need to use a different source of information for the
refinement. We considered using linear features, such as edges, which can be
detected in the image, to improve the stabilization. By aligning edges of a poorly
21
stabilized image with those of the base image, we hope to find a better
transformation. The results, however, were inconclusive.
Figure 11: Foreground mask of a poorly stabilized image.
Instead, we found that we could apply the same stabilization technique on the
gradient of the two poorly stabilized images to refine the transformation (Figure 11).
Since the SIFT features [4] are extracted from gradient maps, they contain edge
information. By using these features to estimate transformation, we take edge
information into account.
One modification that dramatically improves the stabilization quality can be
achieved by making sure that correspondences from each of the 4 quadrants are
selected to build the homography model in each iteration of RANSAC. The
previously mentioned restabilization process is no longer needed because after the
modification, it can't improve stabilization anymore.
Even though random noises are removed when we check for constant moving
pixels, there are still some noises caused by local intensity variation. For example,
sunlight penetrating through clouds creates a bright area in our image.
22
After studying the foreground pixels in noisy images, we found that there exist a
linear relationship between the intensity of foreground pixels in the runway image
and the intensity of the same pixels in the mean of the background model, as
shown on Figure 13.
fg
i
fg
i
fg
i
fg
i
fg
i
b m I ε μ + + =
Where
fg
i
I denotes the intensity on the position of foreground pixels at index
i .
By applying a normalization process on the foreground pixels, we are able to
reduce the noise by 75% in some cases, as illustrated in Figure 14 and 15.
23
Figure 12: Flow chart of the transformation refinement process.
24
(a)
(b)
(c)
Figure 13: A runway image before stabilization (a), after stabilization (b) and extracted moving
objects (c).
25
Figure 14: Foreground intensity of a noisy image vs. Foreground
intensity of background model.
(a) (b)
Figure 15: Frame number vs. number of foreground pixels before applying normalization process
on the foreground pixels (a) and after applying normalization process on the foreground pixels (b).
26
(a)
(b)
Figure 16: Foreground mask before noise reduction (a) and after noise reduction (b).
27
5. Experimental Results
We both synthesized and real world runway sequences to test our program.
For the synthetic experiment, we produced 150 runway sequences from a test
sequence. The synthetic sequences have 351 frames and for each sequence, a
different simulated object is added. The added objects are of different sizes, moving
in different directions at different speeds. For the real world test, we ran the
program on 18 real world runway sequences with moving objects.
5.1. Synthetic Data
The three variables for the simulated objects are size, direction and speed. For
the size variable, we vary from 2-by-2 to 6-by-6 (pixel). For the direction variable,
diagonal, vertical, and horizontal directions are considered. For the speed variable,
the range is from 0.1 pixels per frame to 2.8 pixels per frame with 0.3 pixels per
frame increment. By mixing and matching the three variables, 150 test sequences
were generated.
Since we know the object positions in the simulated sequences, we were able to
collect statistical data for analysis. (Summarized in figure 17, 18 and 19) We count
the number of frames in a sequence that missed the object completely, detected
the object fully, detected the object partially, and detected partially but more than
28
half of the object. We have also collected for each sequence a histogram of number
of foreground pixels.
As for the detection rate, it is greater than 73% in diagonal direction. In horizontal
and vertical directions, it is greater than 77% except objects of size 2-by-2 moving at
a speed higher than 1.9 pixels per frame.
For the detection rate of detecting more than half of the object, it is greater than
69% in diagonal direction except objects of size 2-by-2 and 3-by-3 moving faster
than 1.6 pixels per frame. In horizontal direction, the detection rate is greater than
73% except object of size 2-by-2 and 3-by-3 moving at more than 1.9 pixels per
frame. In vertical direction it is greater than 60%, except for object of size 6-by-6
and object of all sizes with speed greater than 1.9 pixels per frame.
5.2. Real Data
We applied our method to 18 sequences of the same runway taken in 4 different
days to test our program, and it succeeded in detecting most cars on the road just
before the runway. We found that when the contrast in a sequence is low, for
example, the sequences captured in 20050127, fewer features can be extracted for
the stabilization. As a result, the stabilized runway slowly slides. Fortunately, the
slide is slow and our background model can adapt to the changes due to increase of
the learning rate. We also noticed that in the beginning of the second runway
29
sequence produced in 20050127, a slowly moving vehicle was not detected fully.
After studying the background model, we found that the contrast of the sequence is
low and the learning rate is too fast for the slow moving truck. After lowering the
learning rate, the truck shows up as foreground completely (figure 16).
30
(a)
(b)
(c)
Figure 17: A frame in 20050127 video containing the truck (a), foreground with a lower
learning rate (b), and with a higher learning rate (c).
31
6. Discussion
By study both the synthetic and the real data collected, we have observed the
following properties for the system.
First, for the synthetic data, speed and direction of the moving object does not
affect the number of noise pixels. Noise pixels are quite stable across different
speed and direction; however, we noticed a small increase of noise pixels as the size
of the objects increase. One reason for this behavior is that since we use a dilation
with a 3-by-3 kernel and an “and” operation on adjacent frames to find the
constant foreground pixels, larger objects will allow more noise around the object
to pass through this filter.
Second, the detection rate is affected by speed as well as size of the object. As
the speed increases, a sharp decrease of full detection at greater than 1 pixel per
frame in horizontal and vertical directions and around 1.6 pixels per frame in
diagonal direction is observed. Moreover, when we compare the detection of more
than half of the object, when the size is 2-by-2 or 3-by-3, we observed a drop of
detection rate at greater than 1.6 pixels per frame in diagonal direction and greater
than 1.9 pixels per frame in horizontal and vertical directions. These behaviors are
expected as we only allow constant foreground pixels to move one pixel distance in
adjacent frames. As a result, moving objects with speed greater than 1 pixel per
frame in horizontal and vertical directions and 1.4142 pixels per frame in diagonal
32
direction will not be fully detected. Also, a moving object with speed greater than
half of its dilated size will lose more than half of its size in the foreground mask.
Another observation is that comparing to smaller objects, larger objects generally
have fewer full detections and more partial detections. This behavior is caused by
the adaptive property of the background model. From the data collected, it is
discovered that a 4-by-4 object moving at 0.1 pixels per frame in vertical or
horizontal direction or a 3-by-3 object moving at 0.1 pixels per frame in diagonal
direction will have more partial detections than smaller objects. In other words, the
background will start to adapt the foreground object after about 40 frames.
Another reason for larger objects having more partial detection is the way constant
foreground pixels are determined. Since the constant foreground pixels are defined
as pixels that move one pixel distance in any direction in adjacent frames, as the
speed increases, the module may start to lose track of small objects while detecting
large objects partially.
We also found that vertical direction has lower detection rate. For the direction
variable, while diagonal and horizontal directions behave similarly, vertical direction
has noticeably less detection rate for greater than half of the object. One
explanation is that in a runway sequence, many background objects are aligned
vertically. Therefore, if an object moving vertically was between two vertical
33
background objects, those two objects will affect it continuously throughout the
sequence.
Even though the chosen threshold parameters work for most of our sequences,
they may need fine-tuning for some special cases such as slow moving vehicles and
low contrast sequences.
Sometimes, in the real-world sequences, moving objects fade in and out, and
because there is no tracking capability in our program, it can only detect the objects
when they appear.
Finally, the noise reduction method we use, which applies intensity
normalization on the foreground pixels, seems to work very well when the
stabilization quality is good. However, near the end of the sequence, where there
are not enough features to stabilize the runway, and the stabilization quality is bad,
the runway will slide, and some background edges may be classified as foreground
pixels. In this case, our program applies the noise reduction method, and remove
those edges, which is an unwanted behavior.
7. Conclusion
We have presented a two step method to detect distant moving objects on the
runway with an onboard infrared camera. The system is able to detect distant
moving objects thus give enough response time for UAVs or pilots, is indifferent to
34
plane motion and since it utilizes infrared, it is also unaffected by illumination. We
have performed extensive validation on both synthetic and real image sequences.
Our next steps are further validation on a larger data set, and speed improvement
so that the system can run in real time.
35
Figure 18: Speed vs. Detection rate.
36
Figure 19: Speed vs. full detection.
37
Figure 20: Speed vs. partial detection.
38
Figure 21: Speed vs. > 1 / 2 partial detection.
39
Bibliography
[1] M. Fischler and R. Bolles. Random sample consensus: A paradigm for model
fitting with applications to image analysis and automated cartography.
Communications of the ACM, 24(6):381-395, 1981.
[2] C. Harris and M. Stephens. A combined corner and edge detector. Proceedings
of The Fourth Alvey Vision Conference, (1):147-152, 1988.
[3] T. G. R. Kasturi, O. Camps and S. Devadiga. Detection of obstacles on runway
using ego-motion compensation and tracking of signicant features. Proceedings
3rd IEEE Workshop on Applications of Computer Vision, 1996 (WACV'96), pages
168-173, 1996.
[4] D. G. Lowe. Distinctive image features from scale invariant keypoints.
International Journal of Computer Vision, 60(2):91-110, 2004.
[5] R. S. B. Sridhar and B. Hussien. Passive range estimation for rotor-craft
low-altitude flight. Machine Vision and Applications, 6(1):10-24, 1993.
[6] W. G. Chris Stauffer. Adaptive background mixture models for real-time tracking.
1999 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR'99), 2:2246, 1999.
[7] S. Sull and B. Sridhar. Runway obstacle detection by controlled spatiotemporal
image Low disparity. IEEE Transactions on Robotics and Automation,
15(3):537-547, 1999.
[8] R. C. H. Yalcin and M. Hebert. Background estimation under rapid gain change in
thermal imagery. Second IEEE Workshop on Object Tracking and Classification in
and Beyond the Visible Spectrum (OTCBVS'05), 2005.
[9] Q. Zheng and R. Chellappa. Motion detection in image sequences acquired from
a moving platform. Proc. Int. Conf. Acoustics, Speech, and Signal Processing,
Minneapolis, 5:201-204, 1993.
40
Appendix A: Synthetic object moving in diagonal direction
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 60 56 67 36 70 57 63 82 66 91
Full 258 240 218 277 239 223 154 100 54 9
Partial 31 53 64 36 40 69 132 167 229 249
> 1/2 31 43 51 32 34 34 36 28 29 21
Detection
Rate
0.82808 0.83954 0.80802 0.89685 0.79943 0.83668 0.81948 0.76504 0.81089 0.73926
Detecting >
1/2
0.82808 0.81089 0.77077 0.88539 0.78223 0.73639 0.54441 0.36676 0.23782 0.08596
Table A-1: Speed vs. detection of 2-by-2 object moving in diagonal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 104 104 103 106 102 106 105 103 104 103
1~5 71 69 69 68 70 67 69 71 69 68
6~10 48 49 52 49 51 48 47 47 48 50
11~20 46 46 46 45 45 47 46 48 46 47
21~30 22 22 21 23 23 23 23 22 24 23
31~50 12 12 12 11 12 13 12 13 12 12
51~100 13 14 13 14 13 12 14 12 13 13
101~200 18 18 18 18 18 19 18 19 18 18
200+ 15 15 15 15 15 14 15 14 15 15
> 20 rate 0.22923 0.23209 0.22636 0.23209 0.23209 0.23209 0.23496 0.22923 0.23496 0.23209
> 30 rate 0.16619 0.16905 0.16619 0.16619 0.16619 0.16619 0.16905 0.16619 0.16619 0.16619
> 10 rate 0.36103 0.3639 0.35817 0.36103 0.36103 0.36676 0.36676 0.36676 0.36676 0.36676
Table A-2: Speed vs. noise of 2-by-2 object moving in diagonal direction.
41
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 44 23 41 18 43 29 41 49 40 55
Full 174 200 171 231 182 186 128 88 48 6
Partial 131 126 137 100 124 134 180 212 261 288
> 1/2 115 93 107 87 94 79 68 45 32 25
Detection
Rate
0.87393 0.9341 0.88252 0.94842 0.87679 0.91691 0.88252 0.8596 0.88539 0.84241
Detecting >
1/2
0.82808 0.83954 0.79656 0.91117 0.79083 0.75931 0.5616 0.38109 0.22923 0.08883
Table A-3: Speed vs. detection of 3-by-3 object moving in diagonal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 88 82 86 88 83 88 85 83 84 84
1~5 82 89 85 83 86 82 87 87 86 84
6~10 51 52 49 49 51 54 48 50 51 54
11~20 46 45 46 47 47 44 48 49 47 46
21~30 24 23 25 24 23 23 22 22 23 23
31~50 11 12 11 13 12 11 12 13 11 12
51~100 14 13 14 12 14 13 14 12 14 12
101~200 18 19 19 19 18 21 18 19 18 19
200+ 15 14 14 14 15 13 15 14 15 15
> 20 rate 0.23496 0.23209 0.23782 0.23496 0.23496 0.23209 0.23209 0.22923 0.23209 0.23209
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16905 0.16619 0.16905 0.16619 0.16619 0.16619
> 10 rate 0.36676 0.36103 0.36963 0.36963 0.36963 0.35817 0.36963 0.36963 0.36676 0.3639
Table A-4: Speed vs. noise of 3-by-3 object moving in diagonal direction.
42
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 20 19 26 13 36 18 19 28 15 33
Full 23 185 146 223 170 168 117 82 43 3
Partial 306 145 177 113 143 163 213 239 291 313
> 1/2 261 126 138 105 114 138 169 179 231 240
Detection
Rate
0.94269 0.94556 0.9255 0.96275 0.89685 0.94842 0.94556 0.91977 0.95702 0.90544
Detecting >
1/2
0.81375 0.89112 0.81375 0.93983 0.81375 0.87679 0.81948 0.74785 0.7851 0.69628
Table A-5: Speed vs. detection of 4-by-4 object moving in diagonal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 85 80 86 82 80 82 82 81 86 84
1~5 77 82 79 73 84 78 79 82 72 78
6~10 53 61 56 62 55 63 58 58 63 61
11~20 55 43 47 50 50 47 47 47 46 43
21~30 21 25 24 24 22 21 25 24 25 25
31~50 12 12 12 10 13 12 11 11 11 12
51~100 14 14 14 15 14 13 14 13 14 12
101~200 17 17 16 18 16 18 19 18 17 19
200+ 15 15 15 15 15 15 14 15 15 15
> 20 rate 0.22636 0.23782 0.23209 0.23496 0.22923 0.22636 0.23782 0.23209 0.23496 0.23782
> 30 rate 0.16619 0.16619 0.16332 0.16619 0.16619 0.16619 0.16619 0.16332 0.16332 0.16619
> 10 rate 0.38395 0.36103 0.36676 0.37822 0.37249 0.36103 0.37249 0.36676 0.36676 0.36103
Table A-6: Speed vs. noise of 4-by-4 object moving in diagonal direction.
43
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 21 10 18 4 19 7 9 17 6 23
Full 0 160 122 199 147 154 101 67 36 3
Partial 328 179 209 146 183 188 239 265 307 323
> 1/2 276 147 162 124 143 143 186 196 229 236
Detection
Rate
0.93983 0.97135 0.94842 0.98854 0.94556 0.97994 0.97421 0.95129 0.98281 0.9341
Detecting >
1/2
0.79083 0.87966 0.81375 0.9255 0.83095 0.851 0.82235 0.75358 0.75931 0.68481
Table A-7: Speed vs. detection of 5-by-5 object moving in diagonal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 75 84 84 83 78 86 82 83 87 86
1~5 69 54 55 47 60 51 50 54 48 50
6~10 67 75 74 77 73 74 78 74 77 75
11~20 57 56 57 55 57 58 55 57 57 57
21~30 23 23 22 24 24 22 25 24 20 23
31~50 12 12 12 10 11 12 11 13 9 13
51~100 14 12 13 16 15 13 16 11 18 12
101~200 17 17 17 18 16 18 16 18 17 17
200+ 15 16 15 19 15 15 16 15 16 16
> 20 rate 0.23209 0.22923 0.22636 0.24928 0.23209 0.22923 0.24069 0.23209 0.22923 0.23209
> 30 rate 0.16619 0.16332 0.16332 0.18052 0.16332 0.16619 0.16905 0.16332 0.17192 0.16619
> 10 rate 0.39542 0.38968 0.38968 0.40688 0.39542 0.39542 0.39828 0.39542 0.39255 0.39542
Table A-8: Speed vs. noise of 5-by-5 object moving in diagonal direction.
44
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 21 5 10 3 13 2 6 14 2 16
Full 0 148 113 175 141 137 93 60 37 3
Partial 328 196 226 171 195 210 250 275 310 330
> 1/2 276 152 178 149 125 167 203 208 243 252
Detection
Rate
0.93983 0.98567 0.97135 0.9914 0.96275 0.99427 0.98281 0.95989 0.99427 0.95415
Detecting >
1/2
0.79083 0.8596 0.83381 0.92837 0.76218 0.87106 0.84814 0.76791 0.80229 0.73066
Table A-9: Speed vs. detection of 6-by-6 object moving in diagonal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 60 78 82 83 85 85 81 85 82 80
1~5 67 50 51 45 52 51 49 51 48 57
6~10 73 57 57 52 49 49 51 50 53 52
11~20 64 82 77 79 78 83 81 82 77 80
21~30 28 26 26 25 28 24 27 25 25 25
31~50 12 11 11 8 11 9 11 14 8 10
51~100 13 13 13 16 15 15 14 9 15 13
101~200 17 16 17 18 16 14 19 18 23 17
200+ 15 16 15 23 15 19 16 15 18 15
> 20 rate 0.24355 0.23496 0.23496 0.25788 0.24355 0.23209 0.24928 0.23209 0.25501 0.22923
> 30 rate 0.16332 0.16046 0.16046 0.18625 0.16332 0.16332 0.17192 0.16046 0.18338 0.15759
> 10 rate 0.42693 0.46991 0.45559 0.48424 0.46705 0.46991 0.48138 0.46705 0.47564 0.45845
Table A-10: Speed vs. noise of 6-by-6 object moving in diagonal direction.
45
Appendix B: Synthetic object moving in horizontal direction
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 15 29 41 41 72 77 66 123 210 386
Full 325 307 297 291 190 111 31 1 1 1
Partial 9 13 11 17 87 161 252 225 138 62
> 1/2 9 9 10 15 83 160 249 224 137 55
Detection
Rate
0.95702 0.91691 0.88252 0.88252 0.7937 0.77937 0.81089 0.64756 0.39828 0.14031
Detecting >
1/2
0.95702 0.90544 0.87966 0.87679 0.78223 0.7765 0.80229 0.6447 0.39542 0.12472
Table B-1: Speed vs. detection of 2-by-2 object moving in horizontal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 103 98 98 102 98 95 101 102 105 101
1~5 71 74 73 71 76 76 73 71 68 73
6~10 49 50 51 49 48 52 49 50 49 47
11~20 48 45 47 47 45 46 45 47 48 50
21~30 20 24 22 22 24 22 23 21 21 20
31~50 12 12 11 11 12 11 12 12 12 11
51~100 12 13 14 14 13 13 12 13 13 14
101~200 19 18 18 19 18 20 19 19 19 19
200+ 15 15 15 14 15 14 15 14 14 14
> 20 rate 0.2235 0.23496 0.22923 0.22923 0.23496 0.22923 0.23209 0.22636 0.22636 0.2235
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619
> 10 rate 0.36103 0.3639 0.3639 0.3639 0.3639 0.36103 0.36103 0.36103 0.3639 0.36676
Table B-2: Speed vs. noise of 2-by-2 object moving in horizontal direction.
46
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 15 26 35 35 60 70 61 54 71 48
Full 315 298 283 276 180 112 30 6 4 2
Partial 19 25 31 38 109 167 258 289 274 299
> 1/2 10 10 20 28 94 160 252 229 135 74
Detection
Rate
0.95702 0.9255 0.89971 0.89971 0.82808 0.79943 0.82521 0.84527 0.79656 0.86246
Detecting >
1/2
0.93123 0.88252 0.86819 0.87106 0.7851 0.77937 0.80802 0.67335 0.39828 0.21777
Table B-3: Speed vs. detection of 3-by-3 object moving in horizontal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 80 64 63 69 68 66 76 77 80 76
1~5 90 105 104 99 102 102 93 91 88 93
6~10 49 49 53 50 51 54 53 52 50 53
11~20 48 48 47 51 46 46 45 46 49 47
21~30 24 25 24 22 24 23 24 25 24 22
31~50 10 11 10 11 10 12 11 10 10 12
51~100 14 15 15 14 15 12 13 15 15 13
101~200 20 17 19 19 19 20 20 19 19 19
200+ 14 15 14 14 14 14 14 14 14 14
> 20 rate 0.23496 0.23782 0.23496 0.22923 0.23496 0.23209 0.23496 0.23782 0.23496 0.22923
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619
> 10 rate 0.37249 0.37536 0.36963 0.37536 0.36676 0.3639 0.3639 0.36963 0.37536 0.3639
Table B-4: Speed vs. noise of 3-by-3 object moving in horizontal direction.
47
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 15 22 31 27 43 57 53 46 59 42
Full 213 287 272 257 172 104 31 1 0 1
Partial 121 40 46 65 134 188 265 302 290 306
> 1/2 112 22 36 48 109 169 253 284 260 276
Detection
Rate
0.95702 0.93696 0.91117 0.92264 0.87679 0.83668 0.84814 0.86819 0.83095 0.87966
Detecting >
1/2
0.93123 0.88539 0.88252 0.87393 0.80516 0.78223 0.81375 0.81662 0.74499 0.7937
Table B-5: Speed vs. detection of 4-by-4 object moving in horizontal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 86 72 78 76 82 77 75 84 85 83
1~5 71 91 84 84 80 79 80 74 76 79
6~10 51 58 59 59 59 57 58 60 58 55
11~20 52 49 47 51 47 47 53 49 49 52
21~30 22 22 23 21 23 23 25 23 23 21
31~50 17 11 13 12 12 10 11 12 12 11
51~100 16 14 13 14 14 17 14 14 14 16
101~200 18 16 19 18 17 23 16 17 18 17
200+ 16 16 13 14 15 16 17 16 14 15
> 20 rate 0.25501 0.22636 0.23209 0.22636 0.23209 0.25501 0.23782 0.23496 0.23209 0.22923
> 30 rate 0.19198 0.16332 0.16619 0.16619 0.16619 0.18911 0.16619 0.16905 0.16619 0.16905
> 10 rate 0.40401 0.36676 0.36676 0.37249 0.36676 0.38968 0.38968 0.37536 0.37249 0.37822
Table B-6: Speed vs. noise of 4-by-4 object moving in horizontal direction.
48
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 17 21 24 20 37 52 42 38 51 25
Full 53 276 262 244 166 100 29 0 0 0
Partial 279 52 63 85 146 197 278 311 298 324
> 1/2 258 27 43 62 111 176 251 282 258 276
Detection
Rate
0.95129 0.93983 0.93123 0.94269 0.89398 0.851 0.87966 0.89112 0.85387 0.92837
Detecting >
1/2
0.89112 0.86819 0.87393 0.87679 0.7937 0.79083 0.80229 0.80802 0.73926 0.79083
Table B-7: Speed vs. detection of 5-by-5 object moving in horizontal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 61 54 49 72 78 78 77 82 86 78
1~5 78 85 89 65 59 52 53 45 50 59
6~10 61 69 74 73 74 69 72 78 69 73
11~20 57 59 56 58 55 55 55 51 64 56
21~30 20 24 24 23 25 22 20 23 22 23
31~50 8 12 12 12 12 8 8 15 12 10
51~100 13 14 12 14 14 19 16 23 14 14
101~200 26 17 17 18 18 18 24 15 18 21
200+ 25 15 16 14 14 28 24 17 14 15
> 20 rate 0.26361 0.23496 0.23209 0.23209 0.23782 0.27221 0.26361 0.26648 0.22923 0.23782
> 30 rate 0.2063 0.16619 0.16332 0.16619 0.16619 0.20917 0.2063 0.20057 0.16619 0.17192
> 10 rate 0.42693 0.40401 0.39255 0.39828 0.39542 0.4298 0.4212 0.41261 0.41261 0.39828
Table B-8: Speed vs. noise of 5-by-5 object moving in horizontal direction.
49
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 18 19 21 19 33 44 35 28 45 24
Full 0 256 240 212 144 88 26 0 0 0
Partial 331 65 88 118 172 217 288 321 304 325
> 1/2 309 38 62 88 133 183 257 281 263 281
Detection
Rate
0.94842 0.94412 0.93983 0.94556 0.90544 0.87393 0.89971 0.91977 0.87106 0.93123
Detecting >
1/2
0.88539 0.86471 0.86533 0.8596 0.7937 0.7765 0.81089 0.80516 0.75358 0.80516
Table B-9: Speed vs. detection of 6-by-6 object moving in horizontal direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 49 47 53 72 71 77 81 77 78 80
1~5 69 81 84 66 64 53 48 51 59 57
6~10 66 51 48 49 48 48 45 44 45 45
11~20 65 81 81 80 80 76 82 80 85 81
21~30 28 32 25 25 28 23 19 25 23 22
31~50 7 12 13 12 12 11 12 8 13 8
51~100 13 13 12 14 14 16 15 17 13 14
101~200 24 16 17 16 18 19 21 24 18 24
200+ 28 16 16 15 14 26 26 23 15 18
> 20 rate 0.28653 0.25501 0.23782 0.23496 0.24642 0.27221 0.26648 0.27794 0.23496 0.24642
> 30 rate 0.2063 0.16332 0.16619 0.16332 0.16619 0.2063 0.21203 0.2063 0.16905 0.18338
> 10 rate 0.47278 0.48711 0.46991 0.46418 0.47564 0.48997 0.50143 0.50716 0.47851 0.47851
Table B-10: Speed vs. noise of 6-by-6 object moving in horizontal direction.
50
Appendix C: Synthetic object moving in vertical direction
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 33 38 22 45 52 56 42 104 193 293
Full 284 275 310 272 187 108 32 4 3 1
Partial 32 36 17 32 110 185 275 241 153 55
> 1/2 19 30 15 25 106 178 256 230 141 51
Detection
Rate
0.90544 0.89112 0.93696 0.87106 0.851 0.83954 0.87966 0.70201 0.44699 0.16046
Detecting >
1/2
0.86819 0.87393 0.93123 0.851 0.83954 0.81948 0.82521 0.67049 0.41261 0.149
Table C-1: Speed vs. detection of 2-by-2 object moving in vertical direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 104 105 106 106 106 106 106 105 105 105
1~5 68 68 68 67 68 68 68 69 69 68
6~10 49 47 49 49 49 48 47 49 49 50
11~20 44 48 45 46 46 45 48 44 45 46
21~30 26 23 23 23 22 24 22 24 23 22
31~50 12 12 11 12 12 12 12 12 10 13
51~100 13 13 14 13 13 13 13 14 15 13
101~200 19 19 19 19 19 19 18 18 19 17
200+ 14 14 14 14 14 14 15 14 14 15
> 20 rate 0.24069 0.23209 0.23209 0.23209 0.22923 0.23496 0.22923 0.23496 0.23209 0.22923
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619
> 10 rate 0.36676 0.36963 0.36103 0.3639 0.36103 0.3639 0.36676 0.36103 0.36103 0.36103
Table C-2: Speed vs. noise of 2-by-2 object moving in vertical direction.
51
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 34 31 15 34 45 52 39 41 49 69
Full 214 196 242 190 145 68 24 1 0 0
Partial 101 122 92 125 159 229 286 307 300 280
> 1/2 73 89 75 97 129 190 244 208 134 46
Detection
Rate
0.90258 0.91117 0.95702 0.90258 0.87106 0.851 0.88825 0.88252 0.8596 0.80229
Detecting >
1/2
0.82235 0.81662 0.90831 0.82235 0.7851 0.73926 0.76791 0.59885 0.38395 0.13181
Table C-3: Speed vs. detection of 3-by-3 object moving in vertical direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 97 98 97 98 101 101 99 101 100 92
1~5 75 76 76 73 69 70 72 70 73 80
6~10 46 47 50 51 52 51 49 52 50 51
11~20 49 47 46 44 47 46 48 46 45 44
21~30 24 23 22 25 21 24 22 22 22 24
31~50 12 12 12 10 14 11 13 13 13 13
51~100 14 13 13 15 13 13 15 13 13 14
101~200 19 19 19 20 19 19 17 17 19 17
200+ 13 14 14 13 13 14 14 15 14 14
> 20 rate 0.23496 0.23209 0.22923 0.23782 0.22923 0.23209 0.23209 0.22923 0.23209 0.23496
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16905 0.16332 0.16905 0.16619 0.16905 0.16619
> 10 rate 0.37536 0.36676 0.36103 0.3639 0.3639 0.3639 0.36963 0.36103 0.36103 0.36103
Table C-4: Speed vs. noise of 3-by-3 object moving in vertical direction.
52
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 35 33 11 35 48 51 36 38 43 62
Full 64 130 147 117 91 46 14 0 0 0
Partial 250 186 191 197 210 252 299 311 306 287
> 1/2 223 139 164 147 173 207 243 236 200 159
Detection
Rate
0.89971 0.90544 0.96848 0.89971 0.86246 0.85387 0.89685 0.89112 0.87679 0.82235
Detecting >
1/2
0.82235 0.77077 0.89112 0.75645 0.75645 0.72493 0.73639 0.67622 0.57307 0.45559
Table C-5: Speed vs. detection of 4-by-4 object moving in vertical direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 93 92 93 95 93 91 91 91 94 91
1~5 75 80 78 75 77 78 78 79 78 79
6~10 52 48 49 50 52 53 52 51 49 52
11~20 46 47 49 48 46 44 48 47 48 46
21~30 24 25 22 23 24 26 22 23 22 23
31~50 13 12 12 12 11 11 12 13 11 13
51~100 15 13 13 15 15 13 14 14 15 13
101~200 18 16 18 17 16 19 17 16 19 18
200+ 13 16 15 14 15 14 15 15 13 14
> 20 rate 0.23782 0.23496 0.22923 0.23209 0.23209 0.23782 0.22923 0.23209 0.22923 0.23209
> 30 rate 0.16905 0.16332 0.16619 0.16619 0.16332 0.16332 0.16619 0.16619 0.16619 0.16619
> 10 rate 0.36963 0.36963 0.36963 0.36963 0.3639 0.3639 0.36676 0.36676 0.36676 0.3639
Table C-6: Speed vs. noise of 4-by-4 object moving in vertical direction.
53
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 35 32 11 35 44 55 33 35 44 61
Full 2 80 35 57 39 22 7 0 0 0
Partial 312 237 303 257 266 272 309 314 305 288
> 1/2 232 132 239 159 194 190 204 176 156 116
Detection
Rate
0.89971 0.90831 0.96848 0.89971 0.87393 0.84241 0.90544 0.89971 0.87393 0.82521
Detecting >
1/2
0.67049 0.60745 0.7851 0.61891 0.66762 0.60745 0.60458 0.5043 0.44699 0.33238
Table C-7: Speed vs. detection of 5-by-5 object moving in vertical direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 91 89 89 84 89 91 88 87 88 86
1~5 71 67 67 72 70 63 65 65 66 67
6~10 53 63 65 63 60 67 64 69 65 65
11~20 55 51 49 49 50 49 50 49 50 50
21~30 21 21 21 23 22 21 25 20 21 24
31~50 12 12 11 11 12 13 11 13 11 11
51~100 15 14 14 16 15 12 14 14 15 14
101~200 17 17 18 15 16 19 15 16 18 18
200+ 14 15 15 16 15 14 17 16 15 14
> 20 rate 0.22636 0.22636 0.22636 0.23209 0.22923 0.22636 0.23496 0.22636 0.22923 0.23209
> 30 rate 0.16619 0.16619 0.16619 0.16619 0.16619 0.16619 0.16332 0.16905 0.16905 0.16332
> 10 rate 0.38395 0.37249 0.36676 0.37249 0.37249 0.36676 0.37822 0.36676 0.37249 0.37536
Table C-8: Speed vs. noise of 5-by-5 object moving in vertical direction.
54
Speed vs
detection
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
Miss 35 46 12 37 46 55 40 37 50 53
Full 0 27 3 26 11 5 3 0 0 0
Partial 314 276 334 286 292 289 306 312 299 296
> 1/2 102 176 246 167 199 191 182 157 138 107
Detection
Rate
0.89971 0.86819 0.96562 0.89398 0.86819 0.84241 0.88539 0.89398 0.85673 0.84814
Detecting >
1/2
0.29226 0.58166 0.71347 0.55301 0.60172 0.5616 0.53009 0.44986 0.39542 0.30659
Table C-9: Speed vs. detection of 6-by-6 object moving in vertical direction.
Speed vs
Noise
0.1 0.4 0.7 1 1.3 1.6 1.9 2.2 2.5 2.8
0 81 87 86 84 85 87 84 86 87 85
1~5 73 67 64 66 64 64 68 64 66 65
6~10 58 55 53 52 53 50 47 54 49 52
11~20 59 63 65 63 67 69 69 63 66 63
21~30 22 20 24 26 21 22 24 24 22 27
31~50 10 14 10 11 13 11 12 13 12 11
51~100 14 11 14 16 15 13 13 13 15 14
101~200 16 16 17 15 15 18 15 16 18 17
200+ 16 16 16 16 16 15 17 16 14 15
> 20 rate 0.2235 0.22063 0.23209 0.24069 0.22923 0.22636 0.23209 0.23496 0.23209 0.24069
> 30 rate 0.16046 0.16332 0.16332 0.16619 0.16905 0.16332 0.16332 0.16619 0.16905 0.16332
> 10 rate 0.39255 0.40115 0.41834 0.4212 0.4212 0.42407 0.4298 0.41547 0.4212 0.4212
Table C-10: Speed vs. noise of 6-by-6 object moving in vertical direction.
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Motion segmentation and dense reconstruction of scenes containing moving objects observed by a moving camera
PDF
Effective incremental learning and detector adaptation methods for video object detection
PDF
Spatio-temporal probabilistic inference for persistent object detection and tracking
PDF
Part based object detection, segmentation, and tracking by boosting simple shape feature based weak classifiers
PDF
Incorporating aggregate feature statistics in structured dynamical models for human activity recognition
PDF
Exploitation of wide area motion imagery
Asset Metadata
Creator
Pai, Cheng-Hua Jeff
(author)
Core Title
Moving object detection on a runway prior to landing using an onboard infrared camera
School
Viterbi School of Engineering
Degree
Master of Science
Degree Program
Computer Science (Multimedia
Publication Date
04/18/2007
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
OAI-PMH Harvest,object detection onboard infrared camera
Language
English
Advisor
Medioni, Gerard (
committee chair
), Francois, Alexandre R.J. (
committee member
), Nevatia, Ramakant (
committee member
)
Creator Email
chenghup@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m408
Unique identifier
UC166832
Identifier
etd-Pai-20070418 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-478587 (legacy record id),usctheses-m408 (legacy record id)
Legacy Identifier
etd-Pai-20070418.pdf
Dmrecord
478587
Document Type
Thesis
Rights
Pai, Cheng-Hua Jeff
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
object detection onboard infrared camera