Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 914 (2010)

(USC DC Other)

USC Computer Science Technical Reports, no. 914 (2010)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Online Anomaly Detection for Sensor Systems: a Simple and E±cient
Approach
Yuan Yao
a
, Abhishek Sharma
b
, Leana Golubchik
a,b
, Ramesh Govindan
b
a
Department of Electrical Engineering-Systems, USC, Los Angeles, CA 90089
b
Department of Computer Science, USC, Los Angeles, CA 90089
Abstract
Wireless sensor systems aid scienti¯c studies by instrumenting the real world and collecting measurements.
Given the large volumes of measurement collected by sensor systems, one problem arises, namely an au-
tomated approach to identifying the \interesting" parts of these data sets, or anomaly detection. A good
anomaly detection methodology should be able to accurately identify many types of anomalies, be robust,
require relatively little resources, and perform detection in (near) real-time. Thus, in this paper we focus on
an approach to online anomaly detection in measurements collected by sensor systems (that exhibits these
characteristics) and on its evaluation.
1. Introduction
1000 2000 3000 4000 5000 6000
Sample Number
6400 6600 6800 7000 7200 7400 7600
−2
0
2
4
6
8
10
12
Sample Number
(a)Long duration Anomaly (b)Short duration Anomaly
Figure 1: Examples of Anomalies in Sensor Readings
Wireless sensor systems have signi¯cant potential for aiding scienti¯c studies by instrumenting the real
world and collecting measurements, with the aim of observing, detecting, and tracking scienti¯c phenomena
that were previous only partially observable or understood. However, one obstacle to achieving the full
potential of such systems, is the ability to process, in a timely and meaningful manner, the huge amounts
of measurements they collect. Given such large volumes of collected measurements, one natural question
might be: Can we devise an e±cient automated approach to identifying the \interesting" parts of these
data sets?. For instance, consider a marine biology application collecting ¯ne-grained measurements in near
real-time (e.g., temperature, light, micro-organisms concentrations) { one might want to rapidly identify
\abnormal" measurements that might lead to algal blooms which can have devasting consequences. We
Email addresses: yuanyao@usc.edu (Yuan Yao), absharma@usc.edu (Abhishek Sharma), leana@usc.edu (Leana
Golubchik), ramesh@usc.edu (Ramesh Govindan)
Preprint submitted to Performance Review March 22, 2010
can view identi¯cation of such \interesting" or \abnormal" measurements (or events) in collected data as
anomaly detection. In the remainder of the paper, we use the generic term \anomaly" for all interesting
(typically, other-than-normal) events occurring either on the measured phenomena or the measuring equip-
ment. Automated online (or near real-time) anomaly detection in measurements collected by sensor systems
is the focus of this paper.
Anomalies can have a variety of lengths, magnitudes, and patterns. For instance, Figure 1(a) depicts
a long duration, relatively gradual change in sensor reading, whereas 1(b) depicts a short duration, quite
abrupt change in sensor readings. Both scenarios correspond to anomalous events and should be accurately
detectedbyananomalydetectionmethodology. Speci¯cally,agoodanomalydetectionmethodologyshould
have the following properties. First, it should be able to accurately identify all types of anomalies as well as
normalbehavior(i.e.,itshouldhavelowfalsenegativeandfalsepositiverates,asde¯nedlaterinthepaper).
Second, it should be robust, i.e., the methodology should be relatively insensitive to parameter settings as
well as pattern changes in the data sets. Third, it should require relatively small amounts of resources, as
these are typically limited in sensor systems. That is, to run on sensor systems, it should ideally have low
computational complexity, occupy little memory space, and require little transmission power. Last, it is
also desirable for a detection algorithm to be able to detect anomalies in real-time or near real-time. This
is particularly important for sensor systems corresponding to temporary deployments (as it might not be as
usefulto detect anomalies oncethe deployment isover)and thosemonitoring hazardousnatural phenomena
(e.g.,spreadofcontaminantsinaquaticecosystems),wherepromptdetection(andreaction)canbeessential
to reducing loss of life and money.
Anomaly detection, in general, has been studied in a number of systems contexts, most notably in
networking. Several techniques have been proposed in the literature for detecting network tra±c anomalies
such as DDoS attacks, port scans, °ash crowds, and network outages { these include approaches based on
waveletanalysis[1,2],combiningofKalman¯lteringandstatisticalmethodssuchaslikelihoodratiotests[3],
and Principal Component Analysis (PCA) [4]. While one might take the approach of adapting one (or
more) of these techniques to sensor systems, we believe that they do not satisfy all the desirable properties
described above. For instance, the PCA based approaches are typically computationally intensive and
di±cult to perform in an online fashion. A major di±culty in using ¯ltering and wavelets based techniques
is determining their parameter values that would lead to accurate detection. Additionally, our experience
with applying these techniques to sensor data anomaly detection (as detailed in Section 5) indicates that
their performance is sensitive to the choice of parameters, i.e., we did not ¯nd these methods to be robust
when used on data from real sensor system deployments.
In summary, the properties required of an e®ective anomaly detection method for sensor data, as well as
ourexperiencewithapplyingnetworktra±canomalydetectiontechniquestosensorsystemesmeasurements,
motivated us to explore methods di®erent from prior work in network anomaly detection. That said, in
Section 5), we provide (a) quantitative results from applying network anomaly detection techniques to data
collected by real sensor systems deployments, and (b) intuition for why these techniques do not yield good
results.
We also note that little exists in the literature on the topic of anomaly detection in sensor systems
data. Most e®orts are focused on detection of faulty sensor readings, such as those depicted in Figures 1(b)
and 6(b) { these are typically short duration events, with values signi¯cantly deviating from the \normal"
sensor readings [5]. Often, such sensor data faults are modeled as outliers and can be detected using simple
Rule-based approaches or by using statistical models to capture the pattern of normal sensor readings and
°agging any signi¯cantly di®erent samples as faulty [6]. In this work, we view faulty sensor readings as a
special case of anomalies. As illustrated in Section 4, our approach is able to capture such faulty readings,
as well as other long duration, \gradual" anomalies such as the one depicted in Figure 1(a). To the best of
our knowledge, the only e®ort focused on anomaly detection in sensor systems data is described in [7, 8].
Brie°y, this work views measurements collected by a sensor system as coming from the same (unknown)
distribution and \pre-de¯nes" anomalies as outliers. The main focus of that e®ort, which is an o®-line
approach, is on minimizing communication overhead (in transmitting data needed for anomaly detection)
andcorrespondingenergyconsumption. Incontrast,wefocusonan online approach,without\pre-de¯ning"
what is an anomaly. For instance, the approach in [7, 8] might only °ag the most extreme measurement
2
in Figure 1(a) as an anomaly, whereas our approach would °ag the entire event (outlined by the dashed
rectangle) as an anomaly. We give a more detailed description of [7, 8] and a quantitative comparison in
Section 5.
In this work, we formulate the problem of anomaly detection in sensor systems as an instance of iden-
tifying unusual patterns in time series data problem. Of course, one possible direction would then be to
construct a timeseries-based approach, e.g., based on [6]. However, we also did not ¯nd this direction to
be e®ective since such techniques are not well-suited for detecting long duration anomalies. So, we do not
pursue this direction further here, but in Section 5, we do illustrate quantitative results corresponding to
applying a representative timeseries-based approach to data collected by real sensor systems deployments
and provide intuition for why such a technique did not yield good results.
In contrast, the basic idea behind our approach is to compare the collected measurements against a
reference time series. But, to do this e±ciently and robustly, the following challenging problems need to
be solved: (1) How do de¯ne a reference time series?; (2) How to compare two time series e±ciently?; (3)
What metric to use in deciding whether two sensor data time series are similar or di®erent?; and (4) How
to update the reference time series, to adapt to (normal) changes in sensor data patterns?
We address these challenges by proposing and evaluating an anomaly detection algorithm, termed SSA
(Segmented Sequence Analysis), that exhibits the desirable characteristics stated above. Brie°y, SSA lever-
ages temporal and spatial correlations in sensor systems measurements and constructs a piecewise linear
model of sensor data time series. This is motivated by [9] which focused on searching for known patterns in
time series ( details in Section) 5. To detect anomalies, we compare the piecewise linear models of sensor
data(collectedduringatimeinterval)andareferencemodel,wheresigni¯cantdi®erences(asdeterminedby
aproposedsimilaritymetric)are°aggedasanomalies. Weusedatafromreal-worlddeploymentstoevaluate
our approach and demonstrate its accuracy, robustness, and e±ciency. Thus, the main contributions of this
paper are as follows:
² We propose an approach to anomaly detection in sensor systems that is able to detect anomalies
accurately and in an online manner (Section 2).
² We perform an extensive study using data sets from real deployments, which illustrates that our
approach is accurate, robust, as well as e±cient (Sections 3 and 4).
² We show that our (online) SSA-based based approach is more accurate than potential other (o²ine)
techniques
1
, which are more computationally intensive (Section 5).
2. Methodology
Inthissection,we¯rstdescribeatieredsensorsystemarchitecturethatisrepresentativeofdatacollection
deployments. We then formulate the problem of anomaly detection in sensor readings as an instance of the
problem of identifying unusual patterns in time series data. Lastly, we describe our method for detecting
anomalous sensor readings.
2.1. Sensor systems for data collection
We consider a typical tiered sensor system [10, 11] consisting of two tiers: a lower-tier of resource-
constrained battery-operated wireless motes with one or more attached sensors (e.g., temperature, humid-
ity, acceleration), and an upper tier of more capable master nodes each of which has signi¯cantly higher
computation, storage, and communication capabilities than the motes. Figure 2 depicts such a tiered sensor
system. Here, we are interested in the class of data collection sensor systems, where each mote (usually)
collects periodic sensor data, possibly performs some local processing on the data, and then transfers the
resulting data over multiple hops. We model the measurements collected by a sensor m as a time series
D
m
[t];t = 1;2;:::. For example, suppose the system in Figure 2 had 20 motes, each collecting data from 3
1
Most of these were designed in other contexts, but constitute possible directions that could have been taken for sensor
systems anomaly detection.
3
Figure 2: Tiered sensor system
sensors. Then, wewouldhaveatotalof60timeseries(3fromeachofthe20motes), andwewouldrepresent
these as a setfD
m
[t];m=1;2;:::;60;t=1;2;:::g.
In many data collection applications, these time series exhibit a high degree of temporal and spatial
correlations due to the nature of the physical phenomenon being monitored (e.g., temperature or light
conditions). We leverage such correlations to detect anomalies (interesting events) in the sensor data time
series. AsnotedinSection1,anomalieshavevariouslengths,magnitudes,andpatterns,andagoodanomaly
detection methodology should be robust to such variations.
We¯rstdescribethebuildingblocksofourapproach,wherethebasisinvolvesbuilding(andcontinuously
updating) a model of the \normal" and then determining how similar new sensor measurements are to the
\normal". We then describe our approach to anomaly detection.
2.2. Building blocks
At a high level, our approach answers the following question: How similar is a time series of sensor
measurements to a given \reference" time series?. Suppose we are given two time series, D
new
[t] and
D
ref
[t], where D
new
[t] is the time series of new sensor data, and D
ref
[t] is the reference time series
2
. Then,
ananomalydetectionmethodcan: (1)Constructmodelscorrespondingto D
new
[t]andD
ref
[t]; (2)Compare
thesetwomodelsusingasimilaritymeasure; and(3)Ifthemodelfor D
new
[t]isnotsu±cientlysimilartothe
model for D
ref
[t], conclude that there are anomalies in the time series D
new
[t]. Thus, our method involves
solving three main problems: (1) how to construct the models for D
new
[t] and D
ref
[t], (2) which similarity
measure to use for comparing these models, and (3)howto decide whether the models for twodi®erenttime
series data are su±ciently similar, given our similarity measure.
Piecewise linear model. We use a piecewise linear model to represent D
new
[t] and D
ref
[t]. Figure 3
depicts an example piecewise linear representation of sensor measurements collected by the SensorScope
deployment [12]. Each line segment represents a small subset of sensor readings, determined using linear
least-squares regression. The advantages of a piecewise linear representation of time series data are: (a) It
is succinct, since only a few line segments are needed to represent a large amount of time series data; (b) It
is representative as essential information (e.g., signi¯cant patterns) in the data is captured; (c) It is robust
to changes in model parameters as well as to faults and noise in sensor measurements (as demonstrated in
Section 4).
A succinct, representative, and robust piecewise linear model of sensor data time series is desirable for
online anomaly detection. First, we can compute such a model in near real-time (Section 2.3). Second, it
enables us to create a data driven reference model that is easy to update { hence, we do not need prior
knowledge about the types of anomalies that sensor data might contain. Third, because it is succinct, it
enables us to compare two di®erent time series e±ciently and transmit models with low overhead. Finally,
because it is representative of the sensor data patterns, it enables accurate detection of anomalous patterns.
2
We discuss how to obtain D
ref
[t] in Section 2.4.
4
275 290 302
4.9
5
5.1
5.2
Sample Number

X(j)
Y(j)
Y(j+1)
X(j+1)
Figure 3: Piecewise linear model for time series data
Due to their usefulness in modeling time series data, linearization based approaches have also been used
in other contexts. For example, [9] developed an e±cient technique to search for occurrences of a known
pattern within a time series. However, the problem of searching for a known pattern in time series data is
di®erent from anomaly detection because often we do not have any prior information about the patterns
exhibited by anomalous sensor readings
3
.
Linearization Error. In order to compute a piecewise linear model, we need to de¯ne the linearization
error between a sensor data point j and the line segment l covering it. We de¯ne this error as the perpen-
dicular distance between the point j and the line l. Accordingly, we de¯ne the linearization error ² for a
piecewise linear model representing a time series fD[t];t=1;2;3:::;ng, as the maximum linearization error
across all the data points in D[t].
How many line segments to use? We also need to determine the number of line segments, k, to use.
Intuitively, using a large number of line segments will result in a small linearization error { as explained
below, this leads to lower computation cost but larger communication cost. (This tradeo® is explored in
detail in Section 4.2.)
We automatically determine the number of line segments in our piecewise linear model based on the
maximum allowed linearization error ², which is a (free) parameter in our approach. For a ¯xed choice of
maximum linearization error ², we use a greedy approach to determine the number of line segments needed
to represent a time series. We start with the ¯rst two data points of the time series and ¯t a line segment,
(say)l
1
,tothem. Thenweconsiderthedatapointsoneatatimeandrecompute l
1
usinglinearleast-squares
regressiontocoveranewdatapoint. Wecomputethedistanceofthenewdatapointfromtheline l
1
. Ifthis
distance is greater than ², then we start a new line segment, l
2
such that the distance between the new data
point and l
2
is at most ². We keep repeating this process until we exhaust all data points. Note that our
approach is suited for both o²ine and online processing. In an online setting, whenever the sensor collects
a new reading, we can either recompute the current line segment to cover it or start a new line segment
(depending on the linearization error).
We represent the k line segments that constitute a piecewise linear model of a time series using their end
points f(X[i];Y[i]);i = 1;2;:::;kg, where X[i] denotes a sample number (or the time at which a sample
was collected). The corresponding Y[i] is one of the end points of a line segment and represents an estimate
of the actual sensor reading collected at time X[i]. For example, in Figure 3, the line segments approximate
actual sensor readings (shown using dots) { here we indicate two measurement collection times, X[j] and
X[j+1] that correspond to two end points, Y[j] and Y[j+1], that are part of a piecewise linear model.
Similarity measure. Let f(
^
X[i];
^
Y[i]);i = 1;2;:::;
^
kg and f(
~
X[i];
~
Y[i]);i = 1;2;:::;
~
kg denote the
piecewise linear representation of two time series
^
D[t] and
~
D[t], respectively. In order to de¯ne a similarity
measurebetweenanytwopiecewiselinearrepresentations,weneedto¯rstalignthemsothattheirX[i]values
3
More details about [9] are given in Section 5.
5
(end points on the x-axis) line up. For example, consider two representations f(
^
X[i];
^
Y[i]);i = 1;2g and
f(
~
X[i];
~
Y[i]);i = 1;2;3g such that
~
X[1] =
^
X[1] and
~
X[3] =
^
X[2], and hence,
~
X[2] <
^
X[2]. In order to align
the two representations, we choose the X values asfX[1]=
^
X[1]=
~
X[1];X[2]=
~
X[2];X[3]=
^
X[2]=
~
X[3]g.
Hence, after alignment, the new representations are f(X[i];
~
Y[i]);i = 1;2;3g, and f(X[i];Y[i]);i = 1;2;3g,
where Y[1]=
^
Y[1], Y[3]=
^
Y[2] and the Y[2] value (corresponding to the sample at time X[2]) is computed
using the equation of the line segment joining Y[1] and Y[3].
We de¯ne the di®erence between the (aligned) piecewise linear representations of two time series
^
D[t]
and
~
D[t] as:
S(
^
D;
~
D)=
1
k
k
X
i=1
jY[i]¡
~
Y[i]j (1)
Here, S(
^
D;
~
D) represents the average di®erence between the Y values of the piecewise linear representations
of
^
D[t] and
~
D[t] over the k line segments. The
1
k
normalization term is needed to have S(
^
D;
~
D) be of a
similar \scale" as the sensor measurements (since it is being compared to a threshold, described below). We
chose this metric because: (1) it is e±cient to compute, and (2) it indirectly captures the di®erence between
the two time series
^
D[t] and
~
D[t] since Y[i] and
~
Y[i] depend on the corresponding time series data values at
sample time X[i].
Thresholdcomputation. Wesetthethreshold° (fordecidingwhetherS(
^
D;
~
D)issu±cientlylarge)to
the standard deviation of the initial D
ref
[t]. We remove any CONSTANT anomalies (described in Section
3.2), before computing the standard deviation - intuitively such measurements are not a good indication of
variability in sensor data as they typically correspond to faulty data, e.g., due to low voltage supply to the
sensor [5]. Intuitively, the standard deviation is a reasonable indication of the variability in the \normal"
data.
A multiple of standard deviation could also be used, but our more conservative approach already results
(Section 3) in a reasonably low false positive rate; more sophisticated (than threshold-based) approaches
are part of future e®orts.
Putting it all together. Given a time series of new sensor data, D
new
[t], and a reference time series,
D
ref
[t], ourSegmentedSequenceAnalysis(SSA)basedapproachtoanomalydetectionutilizesthefollowing
steps (all detailed above):
1. Linearization: We apply our linearization technique to obtain the two piecewise linear models
f(X
new
[i];Y
new
[i])g andf(X
ref
[i];Y
ref
[i])g.
2. Alignment: We align the two linear representations so that they have the same X values.
3. Similarity computation: We compute the similarity,
S(D
new
;D
ref
), between the reference model and the model for new sensor data using Equation (1).
4. Anomaly detection: We detect an anomaly using a simple threshold-based approach. Speci¯cally,
if
S(D
new
;D
ref
)isgreaterthanathreshold°,thenweconcludethatthesensorreadingsD
new
[t]contain
an anomaly.
We now describe in detail our SSA-based anomaly detection framework.
2.3. Using SSA on a tiered sensor network
We perform anomaly detection in a tiered sensor network in two stages { (1) a local step, executed at
each mote, followed by (2) an aggregation step, executed at the master nodes. In the local step we exploit
temporal correlations (in sensor readings), and in the aggregation step we exploit spatial correlations, as
described next.
Local step. During the local phase (executed at individual motes), each mote m performs the following
tasks: (1) construct or update a reference time series, D
ref
m
[t], for its sensor readings, (2) collect new sensor
readings fD
new
m
[t];t = 1;2;:::;Tg over a period T, (3) construct or update linear models for D
new
m
[t] and
D
ref
m
[t], and (4) perform anomaly detection using the SSA-based method (refer to Section 2.2).
6
Reference time series. To construct a reference time series at mote m, D
ref
m
[t], we use the following
approach. For physical phenomena such as ambient temperature or humidity variations that exhibit a
diurnal pattern, we initially start with a time series D[t] consisting of measurements collected over a period
of 24 hours, (say) on day 1 of the deployment. Let D
new
[t] be the new sensor readings collected by mote m
over time period T corresponding to (say) 9-9:30 a.m. on day 2 of the deployment. For these new readings,
wede¯nethedatapointsin D[t]thatwerecollectedbetween9-9:30a.m.(onday1)as D
ref
[t]. We¯rstlook
for anomalies in the new sensor readings D
new
[t], and then use the data points in D
new
[t] to update D
ref
[t]
using weighted averaging. For example, we can use exponential weighted moving averaging to (pointwise)
update D
ref
[t], i.e.,
~
D
ref
[t]=(1¡®)£D
ref
[t]+®£D
new
[t], where
~
D
ref
[t] denotes the updated reference
time series.
2500 3000 3500 4000 4500 5000 5500
0
20
40
60
80
100
Sample Number

Sensor readings
Reference (including anomalous readings)
Reference (excluding anomalous readings)
Anomalies
Figure 4: Reference time series
Figure 4 depicts the time series of humidity readings collected by a sensor from the Jug Bay deploy-
ment[13]alongwithtworeferencetimeseriesforitconstructedusing T =12hours(36datapointswithone
data point collected every 20 minutes). The reference time series labeled \Reference time series (including
anomalous readings)" is computed using both non-anomalous as well as anomalous readings in D
new
[t] to
update the reference time series, while the \Reference time series (excluding anomalous readings)" excludes
the anomalous readings in D
new
[t]. The humidity measurements contain two anomalies { sharp changes
in the sensor reading (marked by bounding rectangles in Figure 4) which cause the humidity readings to
increase sharply and then decay over time. It is important to detect these sharp changes in sensor readings.
However, not using anomalous readings in D
new
[t] for updates results in a reference time series that
diverges from the sensor data time series resulting in too many samples being °agged as anomalies and
failure to \zoom in" on the samples where the sharp changes in sensor readings occur. If we include the
anomalous readings in D
new
[t] for updating of the reference time series, then the reference time series
exhibits the same patterns as D
new
[t] but with a time lag. Our evaluation results in Section 4 show that
this lag is long enough for SSA to identify the anomalous readings. There is a potential downside in using
anomalousreadingsinupdating D
ref
[t]. Ifananomalya®ectsalargenumberofsamples, thenSSA willfail
to detect many of them. We discuss this in detail in Section 4 and show that for long duration anomalies,
SSA can identify anomalous samples that correspond to the start and end of these anomalies, which is also
quite useful.
For scenarios where the \normal" pattern of sensor readings might not be known or might not exhibit
any periodicity { e.g., sensors deployed for monitoring of birds' nests [10], in the absence of any domain
expertise,weassumethatthesensorreadingscollectedoveralargeduration(a24hourperiodinmostcases)
capturethenormalpatternsinthesensordatatimeseries,andstartwithsuchatimeseriesasourreference.
Clearly, the performance of our local anomaly detection step depends on the quality of the reference data.
A reference data that does not capture the normal sensor readings or is corrupted by anomalies can lead
to false positives and/or false negatives. In Section 4, using real-world sensor readings for periodic (e.g.,
ambient temperature) as well as aperiodic (e.g., soil moisture variations) phenomena, we show that our
approach for selecting and updating D
ref
m
[t] is robust and works well in practice.
7
Aggregationstep. Afterperformingitslocalstep,eachmotemsendsitslinearmodel,f(X
new
m
[i];Y
new
m
[i]);i=
1;:;kg, for the new sensor readings, D
new
m
[t], and the results of its local anomaly detection step to its mas-
ter node. For each mote m, the master node performs another round of anomaly detection by comparing
its linear model against the models from other motes (treating them as reference). Hence, a master node
managing n slave motes performs O(n
2
) model comparisons. The design of our aggregation step is based
on the observation from several real-world deployments that often the readings from sensors deployed at
di®erent locations are correlated [12, 14]. The aggregation step exploits these spatial correlations to detect
additional anomalies (if any) that might not have been detected during the local step.
The ¯nal set of anomalies is the union of the anomalies detected during the local and the aggregation
steps. In our current framework, the master node does not provide any feedback to its slave motes. Hence,
the anomalous readings from mote m detected only by the aggregation step are not currently leveraged to
improve the accuracy of the local anomaly detection step. Incorporating a feedback mechanism between the
aggregation and local steps is part of future e®orts.
Online fault detection. To achieve online detection, we run the local and aggregation anomaly detec-
tion steps periodically, every T minutes. For example, if T = 30 min, we ¯rst collect new sensor readings
for half an hour and then perform anomaly detection using the framework described above. The anomaly
detection interval, T, controls the trade-o® between real-time anomaly detection and resource consumption,
as discussed in detail in Section 4.2.
2.4. Hybrid approach
AsnotedinSection2.2, ourpiecewiselinearrepresentationisverysuccinct{inpractice, asmallnumber
of line segments is su±cient to capture the essential information (diurnal patterns, trends lasting for a long
duration, etc.) in a time series. However, because it is designed to capture signi¯cant trends, a piecewise
linear representation will mask faults or anomalies that a®ect a very small number of sensor samples. The
top plot in Figure 5 shows a temperature reading time series from the SensorScope datasets [12], and the
bottom plot shows whether each sensor reading was identi¯ed as \normal" or \anomalous" by SSA. While
SSA is able to detect instances of long duration anomalies (marked by circles) it fails to detect the three
very short duration anomalies (marked by rectangles in the top plot).
To improve the accuracy of SSA on short duration anomalies, next we propose a hybrid approach.
Combining SSA with Rule-based methods. We can view data faults in sensor readings as short
durationanomalies(refertoSection5). Thus,itisreasonabletoadapttechniquesdesignedforfaultdetection
for detection of short duration anomalies. Speci¯cally, [15, 16] are representative of such techniques and
they consider: SHORT anomalies (a sharp change in the measured sensor readings between two successive
samples), NOISE anomalies (increase in the variance of sensor readings) and CONSTANT or \Stuck-at"
anomalies (the sensor reports a constant value). And, we can use the Rule-based methods [16] (originally
designed for fault detection), for detection of short range anomalies in our hybrid approach by adding the
following rules.
2000 3000 4000 5000 6000 7000 8000 9000
−5
0
5
10
Temperature
2000 3000 4000 5000 6000 7000 8000 9000
Normal
Anomalous
Sample Number
Short Anomaly
Figure 5: SHORT anomalies (marked by rectangles)
8
SHORT Rule: To detect SHORT anomalies in the time series fD[t];t = 1;2;3:::g, we keep track of the
change in sensor readings between two successive samples, jD[t]¡D[t¡1]j. If this value is larger than a
threshold ¾
s
, then we °ag D[t] as anomalous.
CONSTANT Rule: To detect CONSTANT anomalies we calculate moving variance statistics of time
series fD[t];t = 1;2;3:::g. Let V[t] = variance(fD[j]g
j=t
j=t¡c+1
) be the variance of c consecutive data
readings prior to time t. If V[t] is less than a threshold ¾
c
, then we °ag the set of samplesfD[j]g
j=t
j=t¡c+1
as
anomalous.
A rule-based method also exists for detecting NOISE data faults. But, as shown in Section 4, SSA is
accurate at detecting NOISE faults anomalies; thus, we do not include the NOISE rule as part of our hybrid
method.
Toautomaticallydeterminethedetectionthresholds,¾
s
and¾
c
,weusethehistogrambasedapproach[16].
We plot the histogram of the change in sensor readings between two successive samples (for SHORT rule)
or the variance of c samples (for CONSTANT rule) and select one of the modes of the histogram as the
threshold.
Thus, inscenariosinwhichweexpectbothshortandlongdurationanomaliesinsensordata, wepropose
a hybrid approach for anomaly detection. Speci¯cally, every T minutes, we use the Rule-based methods
to detect and mark short duration anomalies, and then use SSA to detect the remaining anomalies. It
is possible to combine other detection methods with SSA to design variants of our hybrid approach, e.g.,
[16] proposes other techniques, such as HMM-based methods, for detecting sensor data faults. However,
other methods in [16] are much more computationally intensive and require a much longer training phase
(than our reference model). We evaluate our hybrid approach using real-world datasets in Section 4 and
show that it is e®ective at detecting both short and long duration anomalies. Our evaluation also shows
that Rule-based methods boost the e±cacy of SSA only in situations where we are interested in detecting
short duration anomalies along with interesting long duration events or anomalies (e.g., changes in sensor
readings patterns). Hence, in situations where we are not interested in detecting short duration anomalies,
we do not need the additional complexity of using Rule-based methods, since SSA su±ces. Note that we
do not need to remove short duration anomalies (or data faults) from the time series { e.g., by replacing
the sensor readings D[j] corrupted by a SHORT anomaly with the average of its adjacent samples D[j¡1]
and D[j +1] { in order for SSA to be able to detect long duration anomalies. Our evaluation results in
Section 4 show that the presence of short duration anomalies does not impact the accuracy of SSA when it
comes to detecting long duration anomalies.
Complexity and Overhead. Of all the steps in SSA, linearization requires the most computation,
with the worst case complexity being O(n
2
), where n is the number of measurements accumulated in a time
slot of length T. Since we use linear least-squares regression to determine the best-¯t line segment, the
cost of approximating d (one dimensional) data points with a line segment is O(d). However, our greedy
approach performs a least-squares regression ¯t every time a new sensor sample is recorded. In the worst
case, we may need to perform least-squares regression n times (once for each data point) resulting in O(n
2
)
computational complexity for the linearization step, and hence, for SSA. In practice, SSA is quite e±cient,
as shown in Section 4 (as n is typically not very large). We note that the Rule-based methods used in our
hybrid approach are simple and have O(n) computational complexity; thus, they do not increase the overall
complexity of the hybrid approach.
SSA incurs a communication overhead every time a mote conveys its linear model to its master node.
Note that a mote needs to convey 4 data points per line segment { two X[i] values (sample times) and the
corresponding two Y[i] values. Since a mote's linear model consists of k line segments, the communication
overhead is O(k). Note that this overhead is incurred every T minutes since a mote recomputes its linear
model once every T minutes.
3. Experimental setup
Inthissectionwedescribereal-worlddatasetsusedforevaluatingSSAandcorrespondinganomalytypes.
9
3.1. Sensor datasets
Thesensordatatimeseriesusedinourevaluationscomefromtwosources: theSensorScopedatasets[12],
and the data sets from the Life Under Your Feet project [13]. We chose these two datasets because both
projects represent the state-of-the-art in sensor systems, and as described next, they collect measurements
in very di®erent environments. Hence, the two datasets allow us to evaluate SSA on representative and
diverse sensor system data.
SensorScope datasets. In the SensorScope project, large networks of sensors are deployed to collect
environmental data such as temperature, humidity, solar radiation, etc. In this paper, we use temperature
readings collected from 23 sensors deployed in the Grand St. Bernard pass between Switzerland and Italy
in 2007. Each sensor collected samples every 2 minutes for 43 days. Since the temperature measurements
exhibit a diurnal pattern, the sensor data time series are periodic with the period being 720 data points
(collected every 24 hours). In what follows, we show results for all 23 sensor data time series. We refer to
these time series as SensorScope 1 through SensorScope 23.
Jug Bay datasets. Our second sensor data source is from the Life Under Your Feet project [13], which
studies soil ecology in a number of locations. We use data sets collected at the Jug Bay Wetland Sanctuary
in Anne Arundel County, Maryland between June, 2007 and April, 2008. In this deployment, sensors were
placed in the nests of Box Turtles to study the e®ect of soil temperature and soil moisture on the incubation
of turtle eggs.
Measurements of soil temperature, soil moisture, box temperature
4
, and box humidity are collected
every 20 minutes for more than 5 months. These measurements exhibit very diverse patterns. For example,
as depicted in Figure 6(a), the soil moisture data are non-periodic { here the soil moisture readings are close
to 8% when it is not raining, but they exhibit a sharp jump followed by a gradual decay when it rains.
Hence, for the soil moisture time series, instances of rainfall are the anomalies (or events) of interest that
we try to detect using SSA. In contrast, the box humidity data sets are periodic with a period of 72 data
points (or 24 hours). The Jug Bay dataset consists of readings from 3 di®erent sensors. In what follows, we
show results for soil moisture readings collected (we refer to them as Soil Moisture 1, Soil Moisture 2, and
Soil Moisture 3), as well as the box humidity data time series (we refer to them as Box Humidity 1, Box
Humidity 2, and Box Humidity 3).
3.2. Anomalies in the data sets
Tothebestofourknowledge,therearenopubliclyavailabledatasetswithanomaliesalreadyannotated.
Thus, to obtain the ground truth, we visually inspected the sensor data time series from the SensorScope
and the Jug Bay deployments, to identify both long and short duration anomalies. This is consistent with
current practice for other data sets (e.g., Internet traces) that lack ground truth. To identify long duration
anomalies, we used the (subjective) criterion of \what kind of patterns would a human ¯nd interesting?".
The short duration anomalies that we identi¯ed resemble sensor data faults types (SHORT, NOISE, and
CONSTANTfaults)describedin[5,16]. Wecategorizetheanomaliesidenti¯edthroughourvisualinspection
into ¯ve groups as follows. We note that this categorization is done for ease of results presentation (Section
4) only and is no way used in our anomaly detection approach.
Change in mean. The anomalous sensor readings di®er signi¯cantly from the average value of the
normal sensor readings. An instance of this anomaly is shown in Figure 1(a). It is a long duration anomaly.
Change in variance. The anomalous sensor readings exhibit less variability than the normal sensor
readings. An instance this anomaly is shown in Figure 6(c). These anomalies can last for long as well as
short durations. The NOISE data fault type described in [5, 16] is equivalent to short duration Change in
variance anomaly.
Short spike. An example of this type of anomaly is shown in Figure 1(b). This type of anomaly is
equivalent to the SHORT sensor data fault type de¯ned in [5]. This is a short duration anomaly a®ecting a
single sensor reading.
4
The motes in the Jug Bay deployment were ¯tted inside a box before being buried in the ground to protect the sensors.
10
0 1000 2000 3000 4000 5000
5
10
15
20
25
Sample Number
7950 8000 8050 8100 8150 8200
−2
0
2
4
6
8
10
12
Sample Number
(a) Soil Moisture Readings (b) Anomaly: Constant
3800 4000 4200 4400 4600 4800
10
20
30
40
50
60
70
Sample Number
5600 5700 5800 5900 6000 6100 6200 6300 6400
10
20
30
40
50
60
Sample Number
(c) Anomaly: Change in Variance (d) Anomaly: Change in Shape
Figure 6: Soil Moisture Readings, Constant, Change in Variance and Change in Shape anomalies
Constant reading. This anomaly results in the sensor reporting a constant value over a period of
time. An instance of this anomaly is shown in Figure 6(b). This anomaly can be both long as well as short
duration.
Change in shape. Two instances of this anomaly are shown in Figure 6(d). This type of anomaly
usually results in a change in mean and/or variance of the sensor readings, but its duration is shorter as
compared to the Change in mean and the Change in variance anomalies.
4. Experimental Results
We now evaluate our SSA-based approach and illustrate its goodness using the following criteria (a
comparison with related literature is presented in Section 5).
² Accuracy: SSA alone detects most long duration anomalies (plus a signi¯cant fraction of the short
duration ones), and that our hybrid approach detects both, long and short duration anomalies accu-
rately.
² Sensitivity: OurresultsarenotsensitivetoSSA'sparameter,namelytothesettingsofthelinearization
period T, and the maximum linearization error ².
² Complexity and Overhead: SSA has low computation complexity and overhead, so that it can be
e®ectively implemented in sensing systems.
² Robustness: SSA is robust to the presence of sensor data faults in the reference time series (i.e., there
is no need to \clean" the data before running SSA).
11
4.1. Accuracy evaluation
We¯rstdescribeourmethod'saccuracy,usingthedatasetsandthegroundtruthidenti¯cationdescribed
in Section 3. We use (1) number of false positives (detecting non-exist anomalies), and (2) number of false
negatives (not being able to detect an anomaly) as our metrics. Speci¯cally, the results in the tables below
are presented as follows - the x=y number indicates that x out of y anomalies were detected correctly
(corresponding to y¡x false negatives) plus we also indicate the number of corresponding false positives
(FP). Note that a long duration anomaly may consist of many consecutive data points. In this paper, we
focus on detecting these events rather than on identifying each and every anomalous data point within an
event. Thus, when 50% or more data points of a long duration anomaly are identi¯ed by SSA as an
anomaly, we consider it as being successfully detected.
The accuracy results of our hybrid approach on all data sets are given in Tables 1 and 2. Our hybrid
method is able to detect both long duration and short duration anomalies, with a small number of false
positives, and often without any false negatives. Most of the false positives are due to the Rule-based part
of the hybrid approach rather than to the SSA part (as explained below).
Tables 1 and 2 also show that long duration anomalies { particularly the Change in Mean and Change
in Shape anomalies { occur quite often in the SensorScope and the Jug Bay datasets (refer to the last row
of both tables). For example, over the course of 43 days, a total of 84 instances of Change in Mean and
139 instances Change in Shape anomalies occurred in the SensorScope datasets; on average, 2 instances of
Change in Mean and 3 instances of Change in Shape anomalies per day. Previously, others have shown that
short duration anomalies or data faults (Short spikes, Constant readings, Noise faults) are quite prevalent
in real-world datasets [5, 16]; this work is the ¯rst to demonstrate that long duration anomalies occur quite
often as well.
Under our hybrid approach, anomalies can be detected at three di®erent stages { the Rule-based meth-
ods, the local step in SSA, and the aggregator step in SSA. For both the SensorScope and the Jug Bay
datasets, we found that the aggregator step in SSA did not add signi¯cantly to the accuracy of SSA. This
is because the combination of the Rule-based methods and the local step in SSA was able to detect most of
the anomalies. We now focus on understanding the contribution to our hybrid approache's accuracy of SSA
vs. the Rule-based methods. The ¯rst two rows of Table 3 show the results of applying SSA alone (without
the use of Rule-based methods) on the Soil Moisture 1 and the SensorScope 1 time series. Based on these
results, we make the following observations: (1) SSA is accurate at detecting long duration anomalies such
as Change in Average, Change in Variance, and Change in Shape, and (2) SSA can fail to detect short
duration anomalies such as Short spikes. For example, while it is able to detect more than 70% of the Short
spikes in Soil Moisture 1, it detects only about 50% of the Short spikes in SensorScope 1. This makes sense,
as SSA is intended more for longer duration anomalies.
The utility of the hybrid approach can be seen, e.g., by comparing the Short results for Soil Moisture 1
and SensorScope 1 in Tables 1 and 2 with those in Table 3. The hybrid approach outperforms SSA on short
duration anomalies because it uses Rule-based methods, designed speci¯cally for short duration anomalies
like Short spikes and Constant readings. However, our hybrid approach incurred a higher false positive rate
thanSSA,andadetailedinspectionofthesamplesfalselyidenti¯edasanomalousrevealedthatthisincrease
was due to the Rule-based methods. [16] showed that Rule-based methods can incur a high false positive
ratemainlybecausethehistogrammethodfordeterminingtheirfaultdetectionthreshold(Section2.4)does
notalwaysidentifyagoodthreshold. Wealsoveri¯edthisbycomputingthehistogramusingtheentiredata
set, which signi¯cantly reduced the false positive rate. However, such an approach would not be online and
hence not used here.
Wealsoveri¯edthattheRule-basedmethodsalonedonotprovideanybene¯tsindetectinglongduration
anomalies. For instance, this can be seen by comparing the results for Soil Moisture 1 and the SensorScope
1 data in Tables 1 and 2 with those in Table 3, where our hybrid method performs the same as SSA w.r.t.
to detecting long duration anomalies like Change in Average and Change in Shape. In fact, the Rule-based
methodscanperformquitepoorlywhenusedforidentifyinglongdurationanomalies. ThelastrowofTable3
shows the results of applying the Rule-based methods alone on the Box Humidity 2 data - compare that
to the Box Humidity 2 results in Table 2. As expected, the Rule-based methods detect the short duration
anomalies (Short spikes and Constant readings), but fail to detect most of the long duration anomalies.
12
Data Set Change in Mean Change in Var Change in Shape Short Constant False Positives
SensorScope 1 3/4 0/0 6/7 90/90 2/2 7
SensorScope 2 8/8 0/0 6/7 86/86 6/6 6
SensorScope 3 7/7 2/2 9/10 64/64 5/5 12
SensorScope 4 5/5 2/2 12/13 220/222 13/13 27
SensorScope 5 6/6 4/4 9/9 726/819 34/34 0
SensorScope 6 7/7 0/0 8/10 206/206 1/1 7
SensorScope 7 8/8 4/4 12/12 555/567 54/54 0
SensorScope 8 6/6 0/0 9/10 243/243 2/2 4
SensorScope 9 6/6 2/2 11/12 65/65 23/23 6
SensorScope 10 5/5 0/0 10/12 46/46 2/2 3
SensorScope 11 7/7 0/0 8/10 122/122 1/1 6
SensorScope 12 7/7 0/0 11/13 84/84 13/13 7
SensorScope 13 8/8 2/2 13/14 250/250 15/15 5
Sensosrscope 14 5/5 4/4 5/7 595/633 26/26 19
Sensosrscope 15 6/6 2/2 8/9 464/475 24/24 12
Sensosrscope 16 4/4 2/2 7/7 120/120 12/12 25
Sensosrscope 17 5/5 4/4 6/6 166/166 17/17 13
Sensosrscope 18 6/7 2/2 16/18 56/56 9/9 12
Sensosrscope 19 3/3 0/0 4/6 98/98 1/1 15
Sensosrscope 20 3/3 0/0 3/4 77/78 1/1 9
Sensosrscope 21 2/2 1/1 3/4 332/337 26/26 5
Sensosrscope 22 3/3 0/0 3/5 88/88 1/1 11
Sensosrscope 23 3/3 0/0 4/4 84/84 2/2 17
Total 121/123 31/31 183/209 4837/4999 290/290 238
Table 1: Hybrid method: SensorScope
Data Set Change in Mean Change in Var Change in Shape Short Constant False Positives
Soil Moisture 1 7/8 1/1 8/10 53/55 1/1 0
Soil Moisture 2 8/9 1/1 9/11 74/74 1/1 2
Soil Moisture 3 5/5 0/0 6/7 42/42 0/0 4
Box Humidity 1 2/2 4/4 18/19 15/15 0/0 2
Box Humidity 2 5/5 7/7 27/28 16/16 2/2 2
Box Humidity 3 3/3 2/2 1/1 17/17 2/2 3
Total 30/32 15/15 69/76 217/219 6/6 13
Table 2: Hybrid method: Jug Bay
4.2. Sensitivity evaluation
The linearization period T and the maximum linearization error ² are the two main parameters in our
SSA-based approach. Next, we present an analysis of the sensitivity of our results to these parameters'
settings.
Impact of T. SSA computes the similarity measure S(
^
D;
~
D) every T time units. The smaller the value
of T, the more real-time is SSA's anomaly detection. However, if T is too small, there may be too few data
points forSSA to accurately capture the pattern of a long duration anomaly. Thus, T controls the trade-o®
between (near) real-time anomaly detection and the accuracy of detecting long duration anomalies.
To characterize the sensitivity of SSA's accuracy to T, we ran SSA using di®erent values of T. For
SensorScope datasets, we varied the value of T from 30 minutes (the time to collect 15 data points) to 8
hours (the time to collect 240 data points). For Jug Bay datasets, we used T values ranging from 2 hours
(the time to collect 6 data points) to 24 hours (the time to collect 72 data points).
We found that changing T's value did not a®ect SSA's accuracy w.r.t. Change in Average and Change
in Variance anomalies, but it did a®ect the accuracy w.r.t. Change in Shape anomalies. We show examples
of SSA's performance in detecting instances of the Change in Shape anomaly in SensorScope 2 and Box
Humidity 1 time series (for di®erent T values) in Tables 4 and 5, respectively. Very small T values (corre-
Method Data Set Change in Mean Change in Var Change in Shape Short Constant FP
SSA Soil Moisture 1 7/8 1/1 8/10 40/55 1/1 0
SSA SensorScope 1 3/4 0/0 6/7 46/90 1/2 1
Rule based Box Humidity 2 2/5 0/7 5/28 16/16 2/2 2
Table 3: SSA vs. Rule-based methods
13
sponding to few data points in a linearization period) result in a signi¯cant number of false positives. As T
grows, the number of false positives improves and becomes reasonably insensitive to T. The false negative
rate is quite insensitive to the value of T, with a small increase for very large values of T. Intuitively, this
can be explained as follows. For a small value of T, SSA considers only a few data points at a time and
even small di®erences in these data points (e.g., due to measurement noise) can cause SSA to misclassify
these points as an instance of Change in Shape anomaly resulting in an increase in the false positive rate.
The Change in Average and the Change in Variance anomalies occur over longer periods of time; thus, to
cause a false positive corresponding to these anomalies, (random) noise would have to a®ect a much greater
number of samples, which is unlikely to happen. The small increase in false negatives for large values of T
is due to very short duration Change in Shape anomalies being \averaged out" (with a large T).
T Value 0.5 1 2 4 8
Detected 7/7 7/7 6/7 6/7 6/7
FP 8 4 0 0 0
Table 4: Change in shape anomaly; SensorScope 2
T Value 2 4 6 8 12 24
Detected 19/19 19/19 19/19 19/19 18/19 18/19
FP 17 10 6 4 2 2
Table 5: Change in shape anomaly; Box Humidity 1
Data sets SensorScope 1 SensorScope 2
jS1[t]¡S2[t]j (1:35;1:40) (1:28;2:05)
jS
2
[t]¡S
3
[t]j (0:22;0:08) (0:25;0:12)
jS3[t]¡S4[t]j (0:25;0:11) (0:29;0:17)
jS
4
[t]¡S
6
[t]j (0:29;0:14) (0:30;0:20)
Table 6: Similarity metric as a function of T
Insummary,ourevaluationshowsthatourmethodisnotsensitivetothelinearizationperiod T,provided
it is long enough to collect a reasonable number of data points. The main reason for this is that beyond a
certain value of T, our similarity metric
S(D
new
;D
ref
) does not change signi¯cantly with T, as illustrated next.
For a ¯xed T value, we ran SSA on SensorScope 1 and SensorScope 2 separately, and recorded the
similarity values (w.r.t. to the reference time series) computed every T time units. For example, for T = 1
hour, SSA computes a similarity value for new data points collected every hour using Equation (1). We
then computed the mean and the variance of the di®erences in the similarity values for di®erent values of
T for SensorScope 1 (and separately for SensorScope 2). For example, consider a set of SensorScope 1 data
points collected over 2 hours. Let ®
2
be the similarity value for these data points when T =2 hours, and for
T =1 hour, let ®
11
and ®
12
be the similarity values for data points collected during the ¯rst and the second
hour, respectively. The mean and variance of the di®erences in the similarity values for T = 1 hour and
T = 2 hours are computed using the values j®
2
¡®
11
j and j®
2
¡®
12
j. These values capture the di®erence
in the similarity values associated with a set of data points for di®erent T values.
Table 6 shows the results, where jS
q
[t]¡S
r
[t]j is the di®erence in similarity values corresponding to
T = q hrs and T = r hrs and the (x;y) entry is the corresponding mean and variance of that di®erence.
The similarity values for T ¸ 2 are close, but the similarity values for T = 1 hr are di®erent from those
for T = 2 hrs. Recall that SSA compares similarity values against a threshold to detect anomalies. Hence
(for SensorScope 1 and SensorScope 2), SSA's performance with T =1 hr di®ers from its performance with
T =2hrs;butforT ¸2,SSA'sperformanceisinsensitivetothechoiceofT. Weobservedasimilarbehavior
fortheotherSensorScopetimeseries. FortheJugBaydataset, weobservedsimilarbehaviorfor T ¸12hrs.
The range of T values over which SSA's performance is insensitive is di®erent for the SensorScope and Jug
Bay datasets primarily because of the di®erences in sampling intervals (2 minutes for SensorScope and 20
14
minutes for Jug Bay). So, it makes sense that it takes much longer to collect a su±cient number of samples
in the Jug Bay data sets and hence requires a larger T to achieve robust SSA performance.
Impact of ². As discussed in Section 2.4, for n data points collected during an interval of length T,
the worst case running time of our linearization algorithm is O(n
2
). Such worst case scenarios arise when a
small number of line segments are su±cient to model all n data points. That is, in the extreme case where
a single line segment is su±cient to cover all the n data points, our greedy approach will be forced to solve
larger and larger instances of the least-square regression problem in each iteration { the ¯rst iteration will
have 2 samples, the second 3 samples, and the (n-1)st will have n samples, resulting in O(n
2
) complexity.
At the other extreme, is the case where each pair of consecutive samples de¯nes a new line segment, leading
to O(n) line segments and O(n) computation. The number of line segments, k, used to model a sensor data
time series depends on the desired linearization error ². Intuitively, a small value of ² will force us to use
more line segments (larger values of k), which would lead to a lower computational cost (as shown below).
However, the communication overhead of our approach is O(k). Thus, ² controls the trade-o® between
computational cost and communication overhead (or size of the model).
We found that in practice (e.g., in SensorScope and Jug Bay datasets) a small value of ² results in each
linesegmentcoveringasmallnumberofpoints. Intuitively, thishappensastypicallysensorreadingsexhibit
non-linear patterns (e.g., diurnal or sharp increases in valuewhen an event occurs), and approximating non-
linear patterns using line segments results in only a few points being covered by a single line. Table 7 shows
theaveragenumberoflinesegmentsusedtomodel120datapointscollectedwhenT=4hrs, forSensorScope
1 and SensorScope 2, for di®erent ² values. As the ² value is reduced from 1 to 0:01, the average number of
line segments increases from 1:85 (1:98) to 92:55 (86:51) for SensorScope 1 (SensorScope 2). (Note that we
can use at most 119 line segments to model 120 data points). Table 7 shows our linearization approache's
running time (on a PC with a 2.8 GHz processor with 2GB of RAM) on the entire time series; as expected,
it is smaller for smaller ² values.
Table 7 results support our intuition that choosing a small value for ² results in faster execution time.
However, the overhead of the communication between a mote and its master is O(k) (Section 2.4) - a small
value of ² reduces the computation time at the expense of a larger communication overhead. In scenarios
where the aggregator step does not boost the accuracy of SSA (as is the case with SensorScope and the
Jug Bay datasets; Section 4.1), we can either do away with the aggregator step or invoke it less frequently
than after every T minutes. This can help us reduce the computational cost of the local step (by selecting
a small ²) while not incurring a signi¯cant communication overhead.
The choice of ² can also determine how well a sensor time series is approximated by our piecewise linear
model. Intuitively, weshouldchoosean ²valuethatisverysmallcomparedtothethreshold ° againstwhich
the similarity measure S(D
new
;D
ref
) is compared to detect anomalies (see Section 2). With ² << °, it is
unlikely that linearization errors will signi¯cantly impact the similarity measure, and hence, not impact the
accuracy of SSA. As discussed in Section 2, we set ° to be equal to the standard deviation of the initial
reference time series; ° for the SensorScope and Jug Bay Box Humidity datasets was within the interval
[4;7] and [6;9], respectively. In this paper, we conservatively set ²=0:1. We also investigated how ² a®ects
SSA's accuracy by trying di®erent values between 0:1 and 1 for it and found that the accuracy of SSA was
the same for the di®erent values of ². (Due to space limitations, we do not show detailed results.) As shown
in Table 7 ² = 0:1 achieves a good computational cost vs. communication overhead trade-o® { choosing a
smaller value did not reduce the running time signi¯cantly but led to a large increase in the number of line
segments k.
² value
SensorScope1 SensorScope2
# Lines Running Time # Lines Running Time
0.01 92.55 0.08 86.51 0.10
0.05 47.49 0.08 50.50 0.11
0.10 25.45 0.10 29.39 0.12
0.50 3.38 0.34 3.80 0.29
1.00 1.85 0.46 1.98 0.40
Table 7: Impact of ²
15
4.3. CPU and Memory Usage
In Section 2.4, we discussed the computation complexity and overhead of SSA. We also measured the
running time and memory usage of SSA on a low-end netbook with Atom 1.6 GHz processor and 1.5 GB of
RAM. The processing power and available memory of this netbook are comparable to that of the emerging
master class devices used in today's tiered sensor network deployments [10, 11]. We ¯rst run a monitoring
program in the background, and then run SSA over all 23 time series in the SensorScope data sets. We
record the running time for each times series is recorded. The monitoring program also records the memory
usage by SSA every second.
We perform two sets of experiments with di®erent linearization period T. In Table 8, we show the
maximum running time and memory usage of SSA over all the 23 time series. For both T = 60 and 120,
SSA takes less than 5 seconds to process approximately 30;000 samples with a small memory footprint.
These results show that the computation and memory requirements of SSA are small and well within the
resources available on today's master class devices.
T (sample number) Max Running Time (Sec) Max Memory Usage (KB)
60 4.876 2048
120 4.796 2052
Table 8: Running time and Memeory Usage of SSA
4.4. Robustness evaluation
Data faults are quite prevalent in real-world sensor system deployments [5, 16] and can be caused by
bugs in hardware or software, improper sensor calibration, or due to motes running out of battery power[5].
Hence, in a real-world scenario, it is quite likely that the reference time series D
ref
[t] used by SSA may
itself contain anomalous readings. Note that, as described in Section 2.3, when updating D
ref
[t] using new
sensor data, we do not discard the anomalous readings. with an anomaly-free reference time series initially,
as a result of updating it with these anomalous data points, the reference time series may eventually exhibit
the same pattern as the anomalous readings. For example, we observed an instance of this problem in a
time series from the SensorScope datasets, as described next.
Figure 7 (top plot) shows a SensorScope time series with a long duration Constant reading anomaly that
lasted for 6 days. The bottom plot in Figure 7 shows the readings identi¯ed as anomalous by SSA alone
and the Rule-based methods. We can see that SSA is able to correctly identify the samples corresponding
to the start and the ¯nish of the Constant reading anomaly but misses most of the \in-between" anomalous
samples. ThisisduetoourdesignchoicetoupdatethereferencetimeseriesD
ref
[t]usinganomaloussamples
as well. We can see in Figure 7 (top plot) that after (approximately) 400 successive samples corrupted by
the Constant reading anomaly, the reference time series values are quite close to the anomalous readings,
and as a result, SSA does not stops °agging the subsequent readings as anomalous. In Section 2, we
justi¯ed our use of anomalous readings to update D
ref
[t] by demonstrating that it helps us \zoom in" on
samples where the sharp changes in sensor readings happen (see Figure 4). However, as this example shows,
updating D
ref
[t] using anomalous readings can cause SSA to miss a large number of samples a®ected by
a long duration anomalies, and only identify the beginning and end of a long duration anomalous event.
This again motivates the use of our hybrid approach - i.e., for the time series in Figure 7 (bottom plot), we
identify the samples missed by SSA using the CONSTANT rule.
TheSensorScopetimeseriesinFigure7(topplot)alsocontainstwoinstancesof Short spikes (thatoccur
before the long duration Constant anomaly). Even though SSA alone fails to detect them, their presence
does not impairSSA's ability to detect the long duration anomaly that occurs later. Hence, we do not need
to \clean" the time series data before running SSA. We can see in Figure 7 (top plot) that the SHORT
faults do not a®ect the reference time series signi¯cantly, i.e., SSA is robust to faults.
16
0.6 0.8 1 1.2 1.4
x 10
4
0
5
10
15

6000 8000 10000 12000 14000 16000
Normal
Rule
SSA
Sample Number
Sensor reading
Reference
Figure 7: Data set with faulty readings
5. Related Work and Comparison
Fault detection. Short duration anomalies can be viewed as instances of data faults, errors in measure-
ments, or outliers (see [5]. for a taxonomy of sensor data faults types). Sharma et al. focus on SHORT
spikes, NOISE faults, and CONSTANT readings data faults and show that these are quite prevalent in real-
world sensor datasets [16]. They also evaluate the performance of several methods { Rule-based methods,
a least-squares estimation based method, Hidden Markov model (HMM) based method, and a time series
analysis (ARIMA model) based method { that are targeted towards detecting transient sensor data faults.
However, thesemethods performpoorlyat detectinglongduration anomalies, e.g., in Section4.1 weshowed
that Rule-based methods are not su±cient for detecting long duration anomalies. Out of these methods, [6]
shows that the ARIMA based method works best in case of data faults a®ecting more than a few samples;
hence, weexpectittohavesomesuccessatdetectinganomaliesandcompareitagainstourhybridapproach
below. Other approaches to sensor data faults detection include [17, 18, 19, 20]. However, these are not
suited for detecting (unknown) long duration anomalies due to one or more of the following assumptions
they make: (1) the anomalous data pattern is known a priori [17, 18, 19], (2) the distribution of normal
sensor readings is known [19], and (3) focusing on short duration trends is enough to capture anomalies [20].
Comparison with ARIMA based method. ARIMA (Autoregressive Integrated Moving Average) models are a
standard tool for modeling and forecasting time series data with periodicity [21], and [6] leverages temporal
correlations in sensor measurements to construct an ARIMA model of sensor data. This model is used to
predict the value of future readings, with new sensor readings compared against their predicted value - if
the di®erence between these values is above a threshold (the 95% con¯dence interval for the predictions)
then the new data is marked as anomalous. We compare our hybrid approach against the ARIMA L-step
method from [6], where the ARIMA model is used to predict the next L sensor readings.
We ¯rst trained the ARIMA model to estimate its parameters using sensor readings (from SensorScope
1 and SensorScope 2) collected over 3 days as training data (a separate training phase was required for
SensorScope1andSensorScope2,[6]alsousestrainingdatafrom3days;thesearemorefavorableconditions
for ARIMA asSSA uses a shorter period for its reference model). The ARIMA L-step method with L=24
hrs °agged 12,135 (of the 30,356 data points in SensorScope 1) as anomalies. Our inspection revealed a
total of 107 anomalies that a®ect more than 7,500 data points. While the ARIMA L-step method identi¯ed
most anomalous samples, it also falsely identi¯ed a large number of samples as anomalous. The extremely
high number of false positives resulting from the ARIMA L-step method reduces its utility.
We failed to train ARIMA on the SensorScope 2 due to the fact that the training data (¯rst 3 days of
the deployment) contains a Constant readings anomaly that a®ects almost two-thirds of the samples. This
failure highlights the fact that ARIMA based methods are not robust to long duration anomalies in the
training data. SSA's counterpart to the ARIMA training phase is the selection of an initial reference time
series, and SSA tolerates anomalies in its initial reference time series (see Section 4.4).
Anomalies in network tra±c. Detecting anomalies such as DDoS attacks, °ash crowds, worm propagat-
17
ing, bulk data transfer in enterprise or ISP networks is an area of active research [1, 4, 3, 2]. The techniques
such as Principal Component Analysis (PCA) [4], and wavelet analysis [1, 3, 2] used for detecting net-
work tra±c anomalies are not well suited for online detection in sensor systems primarily because they are
computationally intensive and di±cult to perform in an online manner. However, one can still ask: How
accurately would those technique perform on sensor systems data?,i.e.,inano²inemanner. Asareasonably
representative technique, we use the PCA based approach in [4], which collects periodic measurements of
the tra±c on all network links and splits it into \normal" and \residual" tra±c using PCA. The residual
tra±c is then mined for anomalies using the Q-statistic test in which L2-norm jjyjj
2
of the residual tra±c
vector y for a link is compared against a threshold. If it is greater than the threshold, then the tra±c vector
y is declared anomalous [4].
We ran the PCA based anomaly detection method on the data from SensorScope time series 8, 10, 11,
12, 13, and 14. We chose these as their start and end times are the same. The input to PCA was a data
matrix with 6 columns (one for each sensor) and 29,518 rows (the total number of samples collected). Note
that the PCA based method is applied in an o²ine fashion using the entire time series data from 6 di®erent
sensors whereas in our SSA-based hybrid approach, at best, the aggregator step would get access to linear
models from 6 sensors during the past T =4 hours only.
7000 8000 9000 10000 11000
−5
0
5
10
7000 8000 9000 10000 11000
Normal
Anomalous
Sample Number
Change of Mean
Figure 8: SensorScope 14: PCA result
The results for the PCA based method are summarized in Table 9. It fails to detect most long duration
anomalies (5 out of 38 Change in Mean anomalies and 4 out of 67 Change in Shape anomalies). It does
better at detecting Short spikes but is still not as accurate as our hybrid approach. Thus, even under a
best case scenario (o²ine with access to the entire time series), the PCA based method does not perform
as well as our hybrid approach. Recall that it identi¯es anomalies by looking at the L2-norm of the data
vector in the residual space. As pointed out in [4], the PCA based method works best when the intensity
of an anomaly is large compared to the variability of the normal data. This is not the case with most of
the long duration anomalies present in the sensor data analyzed in Table 9. For instance, ¯gure 8 shows a
Change in Mean anomaly in SensorScope 14 time series that the PCA based method fails. It also shows a
Short anomaly (spike) that the PCA based method is able to detect.
Data Set Change in Mean Change in Var Change in Shape Short Constant
SensorScope 8 1/7 0/0 2/10 45/206 0/1
SensorScope 10 1/6 0/0 1/10 59/243 0/2
SensorScope 11 1/6 1/2 0/12 47/65 4/23
SensorScope 12 0/5 0/0 0/12 31/46 0/2
SensorScope 13 1/7 0/0 0/10 33/122 0/1
SensorScope 14 1/7 0/0 1/13 27/84 6/13
Total 5/38 1/2 4/67 242/766 22/42
Table 9: PCA based method: SensorScope
To further analyze the impact of the intensity of anomalies on the accuracy of the PCA based method,
we injected anomalies (Short, Noise, and Constant) into the time series SensorScope 9 using the approach
18
in [16]. We injected the Short anomalies by replacing the temperature value reported in a sample by a large
valuerelativetothenormalsensorreadings,andhenceasshowninFigure9(a)(topplot),samplescorrupted
by the Short anomalies look like spikes. To inject NOISE anomalies, we took a window of 1000 samples and
added zero mean Gaussian Noise to them in such a way that it increased the standard deviation of these
samples by a factor of three (Figure 9(b) (top plot)). We injected the Constant anomalies by replacing the
values of 1000 samples by a ¯xed constant (Figure 9(c)).
0.7 0.8 0.9 1 1.1 1.2 1.3 1.4
x 10
4
0
10
20
30
40
50
60
0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5
x 10
4
Normal
Anomaly
Sample Number
0.6 0.8 1 1.2 1.4 1.6
x 10
4
−10
−5
0
5
10
15
0.6 0.8 1 1.2 1.4 1.6
x 10
4
Normal
Anomaly
Sample Number
Anomaly Injected
(a) Injected Short anomalies (b) Injected Noise anomaly
0.6 0.8 1 1.2 1.4 1.6 1.8
x 10
4
0
20
40
60
80
100
Sample Number
0.5 1 1.5
x 10
4
−20
0
20
40
60
80
Sample Number
Normal Space
(c) Injected Constant anomaly (d) Constant anomaly: PCA Normal subspace
Figure 9: PCA with injected anomalies
We tried to detect these injected anomalies using PCA. The data samples corrupted by Short and Noise
anoamlies had a higher variance compared to the normal data whereas, by de¯nition, the variance of the
samples corrupted by the Constant anomaly was lower than the normal data. As shown in Figure 9(a) and
(b) (bottom plots), the PCA based method was able to detect most of the samples corrupted by SHORT
andNOISEanomalies. However, thePCAbasedmethodfailedtodetecttheConstantanomaly. Thereason
whyPCAwasnotabletodetecttheinjectedConstantanomalywasbecausethisanomalycontaminatedthe
normal subspace. As shown in Figure 9(d), with even just one principal component in normal subspace, the
Constant anomaly shows up in the normal subspace, and hence we cannot detect it by looking at the data
in the residual subspace. The problem of anomalies contaminating the normal subspace of PCA has been
observed in the context of network tra±c anomaly detection as well [22, 23], and the only way to handle
such situations is to ¯lter out such anomalies before applying PCA.
Apart from the intensity of the anomaly, our PCA results might also be impacted by several other
factors such as sensitivity of PCA to its parameters, and lack of preprocessing of sensor data. Ringberg et
al. show that the performance of PCA is sensitive to the number of principal components include in the
normal subspace, and the threshold used for anomaly detection [22]. We did vary the number of principal
components in the normal subspace and in Table 9 show the best results obtained with having only 1
principal component in the normal space. To our knowledge the Q-statistic based technique that we use in
thispaperistheonlyknownmethodforautomaticallydecidingthedetectionthreshold. Itisalsowell-known
19
that anomalies can contaminate the normal subspace and hence, avoid detection using PCA [23, 22]. One
way to ameliorate this situation can be to preprocess the data to identify and remove large anomalies before
applying PCA. However, in our context, de¯ning a large anomaly would itself have required us to introduce
another heuristic (with its own shortcomings), and we decided against it. The main goal of our PCA based
evaluation in this paper was to illustrate that, as in case of network anomaly detection, it is not straight
forward to apply PCA to detect anomalies in sensor data (hence our choice of a di®erent approach in this
paper). We do not claim that the PCA based method cannot be made more e®ective at detecting sensor
data anomalies.
We also explore wavelet based methods for detecting sensor data anomalies. We picked the method
presented in [1] as a representative technique. This method ¯rst separates a time series of network data
(for e.g., aggregate byte counts on a link) into low, medium and high frequency components using wavelet
decomposition. For detecting anomalies, ¯rst a (time varying) deviation score is computed by combining
the local variance of the medium and high frequency components. Local variance is de¯ned as the variance
of the data falling within a moving time window. The deviation score is then compared against a threshold
to detect anomalies.
Data Set Change in Mean Change in Var Change in Shape Short Constant False Positive
SensorScope 8 1/7 0/0 2/10 205/206 1/1 26
SensorScope 10 1/6 0/0 2/10 243/243 1/2 24
SensorScope 11 1/6 2/2 3/12 65/65 13/23 42
SensorScope 12 1/5 0/0 3/12 46/46 0/2 47
SensorScope 13 1/7 0/0 2/10 122/122 1/1 29
SensorScope 14 1/7 0/0 3/13 80/84 9/13 22
Total 6/38 2/2 15/67 761/766 25/42 190
Table 10: Wavelet based method: SensorScope
We ran the wavelet based anomaly detection method described above on the same set of data that we
used for evaluating the PCA based method. The results are summarized in Table 10. While the wavelets
basedmethoddetectsmoreanomaliescomparedtoPCA,itdoesnotperformaswellasourhybridapproach
at detecting long duration anomalies. In particular, it fails to detect most of the Change in Mean and
Change in Shape anomalies. In our evaluation, this method also incurred a large number of false positives.
Onepossiblereasonforthewaveletsbasedmethodnotbeingverye®ectiveontheSensorScopedatasetcould
be that it looks for anomalies in the medium and high frequency components of a time series, whereas the
long duration anomalies fall into the low frequency component.
LikeincaseofPCA,themaingoalofourwaveletsbasedanalysisofsensordatawastodemonstratethata
straight forward application of a technique designed for network anomaly detection is not very e®ective. We
do not claim that wavelets based techniques will not work on sensor data. However, this requires addressing
several challenges related selecting a wavelet system and its associated parameters that will work best on a
given data set. As shown by the results presented in Section 4, our SSA based hybrid approach provides an
e±cient and robust alternative to the techniques developed for network anomaly detection.
Anomalies in sensor systems. The only e®ort we are aware of that focuses on anomaly detection
in sensor systems data is [7, 8]. Brie°y, this work views measurements collected by a sensor system as
coming from the same (unknown) distribution and \pre-de¯nes" anomalies as outliers, with main focus of
these (o²ine) techniques being on minimizing the communication overhead (in transmitting data needed
for anomaly detection) and the corresponding energy consumption. In contrast, we focus on an online
approach, without \pre-de¯ning" what is an anomaly. However, we do compare our hybrid approach with
theclusteringbasedmethodin[7], which¯rstgroupsthevectorofreadingsfromseveralsensorsintoclusters
of¯xed-width. Afterthis, theaverageinter-clusterdistancebetweenaclusterandits K nearestneighboring
clusters is used to determine the abnormal clusters. An SVM-based method [8] comparison is omitted as
it assumes that few measurements are anomalous, thus unlikely to detect long duration anomalies that can
corrupt most of the samples (e.g., the Constant anomaly in Figure 7).
WeranthismethodonSensorScopetimeseries6,8,9,10,11and12. Theinputstotheclusteringmethod
are vectors of readings with the same time stamp from the 6 time series. We found that the performance
20
of this method depends strongly on the cluster width w, as also noted in [7]. We tried a large number of w
values (from 0.03 to 8) and used the best results found to compare with our method.
This method also can only identify which data vectors contain anomalies, but cannot identify anomalous
readings within a data vector. We determine the number of detections and false negatives as in the PCA-
based method; so, it is not possible to report false positives for individual time series. This method did
incorrectly °ag 18 data vectors as anomalous; again, the actual number of false positives is between 18 and
108.
WegivedetailedresultsinTable11andnotethattheclusterbasedmethodperformspoorlyindetecting
long term anomalies such as Change in mean, Change in shape, and Constant. Intuitively, this is because
this method is designed to detect outliers and does not exploit temporal correlations within the time series.
We can also see that the method has a lot of false negatives in detecting Short spikes. This makes sense as a
spike is determined by the corresponding data point's relative position to the previous and next point, but
in the cluster based method all data points are considered as a whole and this temporal relationship is lost.
Data Set Change in Mean Change in Var Change in Shape Short Constant
SensorScope 6 2/7 0/0 4/10 56/206 1/1
SensorScope 8 3/6 0/0 5/10 54/243 1/2
SensorScope 9 3/6 1/2 5/12 20/65 5/23
SensorScope 10 2/5 0/0 5/12 16/46 1/2
SensorScope 11 3/7 0/0 6/10 49/122 1/1
SensorScope 12 3/7 0/0 5/13 36/84 3/13
Total 16/38 1/2 30/67 243/766 12/42
Table 11: Cluster based method: SensorScope
Piecewise linear models. Piecewise linear models have been used to model time series data in the other
contexts(i.e., not for anomaly detection). Forexample, Keoghdeveloped an e±cienttechniqueto searchfor
occurrences of a known pattern within a time series using a piecewise linear representation [9]. Speci¯cally,
foratimeserieswithnpoints, Keogh'salgorithmstartswith k =b
n
3
clinesegmentsandde¯nesa\goodness
of ¯t" metric for k line segments as B
k
=std(e
1
;:::;e
k
), where e
i
is the average linearization error for line
segment i. It then iteratively merges two consecutive line segments until B
k
cannot be reduced any further;
this process is continued until a single line approximates the entire time series. This process ends with a
family of piecewise linear models for a single time series { one for each value of k, 1 · k · b
n
3
c. Each
linear model has a B
k
value associated with it, and the one with the smallest B
k
is selected as the ¯nal
representation.
In contrast, our greedy linearization algorithm di®ers as follows: (1) we start with a single line segment
andcontinueaddingmoresegments,(2)weuseadi®erent\goodnessof¯t"criterion(maximumlinearization
error;seeSection2)and(3)wecomputeasinglerepresentationinsteadofafamilyofpiecewiselinearmodels.
The main reason behind these choices (rather than, e.g., using Keogh's algorithm) is computational cost
as our goal is an e±cient online approach. Brie°y, computing a family of models to ¯nd the \best" one
is wasteful for our purposes, if it takes signi¯cantly greater computation. Hence, we opt for our greedy
approach.
6. Conclusions
We proposed an online anomaly detection approach for sensor systems measurements. Our approach
utilized piecewise linear models of time series, which are succinct, representative, and robust, and as result
enabled us to (a) compute such models in near real-time, (b) create models without prior knowledge about
anomalytypesthatsensordatamightcontain,(c)compareandcommunicatedi®erenttimeseriese±ciently.
Our extensive evaluation study, using real sensor systems deployments data, illustrated that our approach
is accurate, robust, and e±cient.
Our future work includes (1) investigation of dynamic setting of parameters (such as the linearization
periodandthesizeoftheinitialreferencemodel),(2)feedbackmechanismsbetweenthelocalandaggregation
steps of our approach, and (3) investigation of other techniques that are useful in our hybrid approach.
21
References
[1] P. Barford, J. Kline, D. Plonka, A. Ron, A signal analysis of network tra±c anomalies, in: Proceedings of the Workshop
on Internet measurment (IMW), 2002.
[2] C.-T. Huang, S. Thareja, Y.-J. Shin, Wavelet-based Real Time Detection of Network Tra±c Anomalies, International
Journal of Network Security 6 (2008) 309{320.
[3] A. Soule, K. Salamatian, N. Taft, Combining ¯ltering and statistical methods for anomaly detection, in: Proceedings of
the ACM conference on Internet Measurement (IMC), 2005.
[4] A. Lakhina, M. Crovella, C. Diot, Diagnosing network-wide tra±c anomalies, in: ACM SIGCOMM, 2004.
[5] K.Ni,N.Ramanathan,M.Chehade,L.Balzano,S.Nair,S.Zahedi,G.Pottie,M.Hansen,M.Srivastava.,SensorNetwork
Data Fault Types, Transactions on Sensor Networks.
[6] A. B. Sharma, L. Golubchik, R. Govindan, Sensor Faults: Detection Methods and Prevalence in Real-World Datasets,
Tech. Rep. 09-906, Computer Science, USC (2009).
[7] S. Rajasegarar, C. Leckie, M. Palaniswami, J. Bezdek, Distributed anomaly detection in wireless sensor networks, in:
ICCS, 2006.
[8] S. Rajasegarar, C. Leckie, M. Palaniswami, J. Bezdek, Quarter sphere based distributed anomaly detection in wireless
sensor networks, in: IEEE ICC, 2007.
[9] E. Keogh, Fast similarity search in the presence of longitudinal scaling in time series databases, in: ICTAI, 1997.
[10] J. Hicks, J. Paek, S. Coe, R. Govindan, D. Estrin, An Easily Deployable Wireless Imaging System, in: Proceedings of
ImageSense Workshop, 2008.
[11] J. Paek, O. Gnawali, K.-Y. Jang, D. Nishimura, R. Govindan, J. Ca®rey, M. Wahbeh, S. Masri, A Programmable
Wireless Sensing System for Structural Monitoring, in: Proceedings of the 4th World Conference on Structural Control
and Monitoring(4WCSCM), 2006.
[12] http://sensorscope.ep°.ch/.
[13] http://www.lifeunderyourfeet.org/.
[14] NAMOS: Networked Aquatic Microbial Observing System.
[15] N. Ramanathan, L. Balzano, M. Burt, D. Estrin, E. Kohler, T. Harmon, C. Harvey, J. Jay, S. Rothenberg, M. Srivastava,
Rapid Deployment with Con¯dence: Calibration and Fault Detection in Environmental Sensor Networks, Tech. Rep. 62,
CENS (April 2006).
[16] A.B.Sharma,L.Golubchik,R.Govindan,OnthePrevalenceofSensorFaultsinReal-WorldDeployments,in: Proceedings
of the IEEE Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON), 2007.
[17] N. Khoussainova, M. Balazinska, D. Suciu, Towards Correcting Input Data Errors Probabilistically Using Integrity Con-
straints, in: Proceedings of the ACM Workshop on Data Engineering and Mobile Access (MobiDE), 2006.
[18] S. R. Je®ery, G. Alonso, M. J. Franklin, W. Hong, J. Widom, Declarative Support for Sensor Data Cleaning, in: Proceed-
ings of the International Conference on Pervasive Computing, 2006.
[19] E. Elnahrawy, B. Nath, Cleaning and Querying Noisy Sensors, in: Proceedings of the ACM International Workshop on
Wireless Sensor Networks and Applications (WSNA), 2003.
[20] D. Tulone, S. Madden, PAQ: Time series forecasting for approximate query answering in sensor networks, in: Proceedings
of the European Conference on Wireless Sensor Networks (EWSN), 2006.
[21] G. E. P. Box, G. M. Jenkins, G. C. Reinsen, Time Series Analysis: Forecasting and Control, 3rd Edition, Prentice Hall,
1994.
[22] H.Ringberg, A.Soule, J.Rexford, C.Diot, Sensitivityofpcafortra±canomalydetection, in: ACMSIGMETRICS,2007.
[23] A. Lakhina, M. Crovella, C. Diot, Characterization of network-wide anomalies in tra±c °ows, Tech. Rep. 2004-20, Com-
puter Science, Boston University (May 2004).
22

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 906 (2009)

PDF

USC Computer Science Technical Reports, no. 888 (2007)

PDF

USC Computer Science Technical Reports, no. 923 (2012)

PDF

USC Computer Science Technical Reports, no. 905 (2009)

PDF

USC Computer Science Technical Reports, no. 920 (2011)

PDF

USC Computer Science Technical Reports, no. 930 (2012)

PDF

USC Computer Science Technical Reports, no. 904 (2009)

PDF

USC Computer Science Technical Reports, no. 915 (2010)

PDF

USC Computer Science Technical Reports, no. 939 (2013)

PDF

USC Computer Science Technical Reports, no. 773 (2002)

PDF

USC Computer Science Technical Reports, no. 872 (2005)

PDF

USC Computer Science Technical Reports, no. 839 (2004)

PDF

USC Computer Science Technical Reports, no. 692 (1999)

PDF

USC Computer Science Technical Reports, no. 852 (2005)

PDF

USC Computer Science Technical Reports, no. 771 (2002)

PDF

USC Computer Science Technical Reports, no. 938 (2013)

PDF

USC Computer Science Technical Reports, no. 745 (2001)

PDF

USC Computer Science Technical Reports, no. 913 (2009)

PDF

USC Computer Science Technical Reports, no. 873 (2005)

PDF

USC Computer Science Technical Reports, no. 774 (2002)

Description

Yuan Yao, Abhishek B. Sharma, Leana Golubchik, and Ramesh Govindan. "Online anomaly detection for sensor systems: A simple and efficient approach." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 914 (2010).

Asset Metadata

Creator Golubchik, Leana (author), Govindan, Ramesh (author), Sharma, Abhishek (author), Yao, Yuan (author)

Core Title USC Computer Science Technical Reports, no. 914 (2010)

Alternative Title Online anomaly detection for sensor systems: A simple and efficient approach (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 22 pages (extent), technical reports (aat)

Language English

Unique identifier UC16269707

Identifier 10-914 Online Anomaly Detection for Sensor Systems a Simple and Efficient Approach (filename)

Legacy Identifier usc-cstr-10-914

Format 22 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)