Chakravarthy / Jiang Stream Data Processing: A Quality of Service Perspective

Modeling, Scheduling, Load Shedding, and Complex Event Processing
1. Auflage 2009
ISBN: 978-0-387-71003-7
Verlag: Springer-Verlag
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 324 Seiten

In recent years, a new class of applications has come to the forefront { p- marily due to the advancement in our ability to collect data from multitudes of devices, and process them e ciently. These include homeland security - plications, sensor/pervasive computing applications, various kinds of mo- toring applications, and even traditional applications belonging to nancial, computer network management, and telecommunication domains. These - plications need to process data continuously (and as long as data is available) from one or more sources. The sequence of data items continuously gen- ated by sources is termed a data stream. Because of the possible never-ending nature of a data stream, the amount of data to be processed is likely to be unbounded. In addition, timely detection of interesting changes or patterns or aggregations over incoming data is critical for many of these applications. Furthermore, the data arrival rates may uctuate over a period of time and may be bursty at times. For most of these applications, Quality of Service (or QoS) requirements, such as response time, memory usage, and throughput are extremely important. These application requirements make it infeasible to simply load the incoming data streams into a persistent store and process them e ectively using currently available database management techniques.

Autoren/Hrsg.

Chakravarthy, Sharma

Jiang, Qingchun

Inhaltsverzeichnis

1;Preface;7
1.1;Objectives;8
1.2;Intended Audience;9
1.3;Acknowledgements;9
2;How to Use the Book;11
3;Contents;13
4;List of Figures;19
5;List of Tables;22
5.1;List of Algorithms;23
6;1 INTRODUCTION;24
6.1;1.1 Paradigm Shift;26
6.2;1.2 Data Stream Applications;28
6.3;1.3 Book Organization;29
7;2 OVERVIEW OF DATA STREAM PROCESSING;31
7.1;2.1 Data Stream Characteristics;31
7.2;2.2 Data Stream Application Characteristics;32
7.3;2.3 Continuous Queries;34
7.4;2.4 Data Stream Management System Architecture;41
8;3 DSMS CHALLENGES;44
8.1;3.1 QoS-Related Challenges;44
8.2;3.2 Concise Overview of Book Chapters;48
9;4 LITERATURE REVIEW;53
9.1;4.1 Data Stream Management Systems;53
9.2;4.2 QoS-Related Issues;58
9.3;4.3 Complex Event Processing;61
9.4;4.4 Commercial and Open Source Stream and CEP Systems;67
10;5 MODELING CONTINUOUS QUERIES OVER DATA STREAMS;69
10.1;5.1 Continuous Query Processing;70
10.2;5.2 Problem Denition;74
10.3;5.3 Modeling Relational Operators;77
10.4;5.4 Modeling Continuous Queries;89
10.5;5.5 Intuitive Observations;102
10.6;5.6 Experimental Validation;105
10.7;5.7 Summary of Chapter 5;113
11;6 SCHEDULING STRATEGIES FOR CQs;114
11.1;6.1 Scheduling Model and Terminology;115
11.2;6.2 Impact of Scheduling Strategies on QoS;122
11.3;6.3 Novel Scheduling Strategies for CQs;124
11.4;6.4 Experimental Validation;145
11.5;6.5 Summary of Chapter 6;155
12;7 LOAD SHEDDING IN DATA STREAM MANAGEMENT SYSTEMS;156
12.1;7.1 The Load Shedding Problem;157
12.2;7.2 Integrating Load Shedders;159
12.3;7.3 Load Shedding Framework;162
12.4;7.4 Experimental Validation;177
12.5;7.5 Summary of Chapter 7;184
13;8 N F M: AN INTER-DOMAIN NETWORK FAULT MANAGEMENT SYSTEM;186
13.1;8.1 Network Fault Management Problem;187
13.2;8.2 Data Processing Challenges for Fault Management;189
13.3;8.3 Stream- and Event-Based N F M Architecture;192
13.4;8.4 Three-Phase Processing Model for N F M;197
13.5;8.5 Transactional Needs of Network Management Applications;203
13.6;8.6 Summary of Chapter 8;205
14;9 INTEGRATING STREAM AND COMPLEX EVENT PROCESSING;206
14.1;9.1 Motivation;207
14.2;9.2 Event Processing Model;210
14.3;9.3 Complex Event Vs. Stream Processing;214
14.4;9.4 MavEStream: An Integrated Architecture;219
14.5;9.5 Stream-Side Extensions;222
14.6;9.6 Event-Side Extensions;226
14.7;9.7 Summary of Chapter 9;232
15;10 MavStream: DEVELOPMENT OF A DSMS PROTOTYPE;234
15.1;10.1 MavStream Architecture;235
15.2;10.2 Windows Types;239
15.3;10.3 Stream Operators and CQs;241
15.4;10.4 Buffers and Archiving;248
15.5;10.5 Run-time Optimizer;250
15.6;10.6 QoS-Delivery Mechanisms;262
15.7;10.7 System Evaluation;267
16;11 INTEGRATING CEP WITH A DSMS;280
16.1;11.1 MavStream: Integrated Issues;281
16.2;11.2 Design of the Integrated System;285
16.3;11.3 Implementation Details of Integration;294
16.4;11.4 Stream Modiers;300
16.5;11.5 Additional Benets of CEP Integration;303
16.6;11.6 Summary of Chapter 11;304
17;12 CONCLUSIONS AND FUTURE DIRECTIONS;305
17.1;12.1 Looking Ahead;305
17.2;12.2 Stream Processing;306
17.3;12.3 Integration of Stream and Event Processing;309
17.4;12.4 Epilogue;311
18;References;312
19;Index;332