Khan / Zomaya Handbook on Data Centers

1. Auflage 2015
ISBN: 978-1-4939-2092-1
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 1309 Seiten

This handbook offers a comprehensive review of the state-of-the-art research achievements in the field of data centers. Contributions from international, leading researchers and scholars offer topics in cloud computing, virtualization in data centers, energy efficient data centers, and next generation data center architecture.  It also comprises current research trends in emerging areas, such as data security, data protection management, and network resource management in data centers. Specific attention is devoted to industry needs associated with the challenges faced by data centers, such as various power, cooling, floor space, and associated environmental health and safety issues, while still working to support growth without disrupting quality of service. The contributions cut across various IT data technology domains as a single source to discuss the interdependencies that need to be supported to enable a virtualized, next-generation, energy efficient, economical, and environmentally friendly data center. This book appeals to a broad spectrum of readers, including server, storage, networking, database, and applications analysts, administrators, and architects. It is intended for those seeking to gain a stronger grasp on data center networks: the fundamental protocol used by the applications and the network, the typical network technologies, and their design aspects. The Handbook of Data Centers is a leading reference on design and implementation for planning, implementing, and operating data center networks.

Autoren/Hrsg.

Khan, Samee U.

Zomaya, Albert Y.

Inhaltsverzeichnis

1;Preface;5
2;Contents;8
3;Part I Energy Efficiency;13
3.1;Energy-Efficient and High-Performance Processing of Large-Scale Parallel Applications in Data Centers;14
3.1.1;1 Introduction;14
3.1.1.1;1.1 Motivation;14
3.1.1.2;1.2 Our Contributions;16
3.1.2;2 Related Work;17
3.1.3;3 Preliminaries;18
3.1.3.1;3.1 Power and Task Models;19
3.1.3.2;3.2 Problems;21
3.1.3.3;3.3 Lower Bounds;21
3.1.4;4 Heuristic Algorithms;22
3.1.4.1;4.1 Precedence Constraining;22
3.1.4.2;4.2 System Partitioning;23
3.1.4.3;4.3 Task Scheduling;25
3.1.5;5 Optimal Energy/Time/Power Allocation;26
3.1.5.1;5.1 Minimizing Schedule Length;26
3.1.5.1.1;5.1.1 Level 1;26
3.1.5.1.2;5.1.2 Level 2;27
3.1.5.1.3;5.1.3 Level 3;27
3.1.5.1.4;5.1.4 Level 4;28
3.1.5.2;5.2 Minimizing Energy Consumption;32
3.1.5.2.1;5.2.1 Level 1;32
3.1.5.2.2;5.2.2 Level 2;32
3.1.5.2.3;5.2.3 Level 3;33
3.1.5.2.4;5.2.4 Level 4;33
3.1.6;6 Simulation Data;36
3.1.7;7 Summary and Future Research;43
3.1.8;References;44
3.2;Energy-Aware Algorithms for Task Graph Scheduling, Replica Placement and Checkpoint Strategies;47
3.2.1;1 Introduction;47
3.2.2;2 Energy Models;49
3.2.2.1;2.1 Literature Survey;50
3.2.2.1.1;2.1.1 DVFS and Optimization Problems;51
3.2.2.1.2;2.1.2 Energy Models;52
3.2.2.2;2.2 Example;52
3.2.3;3 Minimizing the Energy of a Schedule;54
3.2.3.1;3.1 Optimization Problem;54
3.2.3.2;3.2 The CONTINUOUS Model;55
3.2.3.2.1;3.2.1 Special Execution Graphs;56
3.2.3.2.2;3.2.2 General DAGs;57
3.2.3.3;3.3 Discrete Models;57
3.2.3.3.1;3.3.1 The VDD-HOPPING Model;58
3.2.3.3.2;3.3.2 NP-Completeness and Approximation Results;58
3.2.3.4;3.4 Final Remarks;59
3.2.4;4 Replica Placement;59
3.2.4.1;4.1 Framework;60
3.2.4.1.1;4.1.1 Replica Servers;61
3.2.4.1.2;4.1.2 With Power Consumption;62
3.2.4.1.3;4.1.3 Objective Functions;63
3.2.4.1.4;4.1.4 Summary of Results;63
3.2.4.2;4.2 Complexity Results: Update Strategies;64
3.2.4.2.1;4.2.1 Running Example;64
3.2.4.2.2;4.2.2 Dynamic Programming Algorithm;65
3.2.4.3;4.3 Complexity Results with Power;67
3.2.4.3.1;4.3.1 Running Example;67
3.2.4.3.2;4.3.2 NP-Completeness of MINPOWER;68
3.2.4.3.3;4.3.3 A Pseudo-polynomial Algorithm for MINPOWER-BOUNDEDCOST;70
3.2.4.4;4.4 Simulations;71
3.2.4.4.1;4.4.1 Impact of Pre-existing Servers;71
3.2.4.4.2;4.4.2 With Power Consumption;73
3.2.4.4.3;4.4.3 Running Time of the Algorithms;74
3.2.4.5;4.5 Concluding Remarks;74
3.2.5;5 Checkpointing Strategies;75
3.2.5.1;5.1 Framework;76
3.2.5.1.1;5.1.1 Model;76
3.2.5.1.2;5.1.2 Optimization Problems;77
3.2.5.2;5.2 With a Single Chunk;78
3.2.5.2.1;5.2.1 SINGLESPEED Model;78
3.2.5.2.2;5.2.2 MULTIPLESPEEDS Model;79
3.2.5.3;5.3 Several Chunks;80
3.2.5.3.1;5.3.1 Single Speed Model;81
3.2.5.3.2;5.3.2 Multiple Speeds Model;82
3.2.5.4;5.4 Simulations;83
3.2.5.4.1;5.4.1 Simulation Settings;83
3.2.5.4.2;5.4.2 Comparison with Single Speed;85
3.2.5.4.3;5.4.3 Comparison Between EXPECTED-DEADLINE and Hard-Deadline;86
3.2.5.5;5.5 Concluding Remarks;86
3.2.6;6 Conclusion;87
3.2.7;References;88
3.3;Energy Efficiency in HPC Data Centers: Latest Advances to Build the Path to Exascale;91
3.3.1;1 Introduction;91
3.3.2;2 Computing Systems Architectures;92
3.3.2.1;2.1 Architecture of the Current HPC Facilities;92
3.3.2.2;2.2 Overview of the Main HPC Components;95
3.3.2.3;2.3 HPC Performance and Energy Efficiency Evaluation;99
3.3.3;3 Energy-Efficiency in HPC Data-Center: Overview & Challenges;102
3.3.3.1;3.1 The Exascale Challenge;102
3.3.3.2;3.2 Hardware Approaches Using Low-Power processors;103
3.3.3.3;3.3 Energy Efficiency of Virtualization Frameworks over HPC Workloads;105
3.3.3.4;3.4 Energy Efficiency in Resource and Job Management Systems (RJMSs);110
3.3.4;4 Conclusion: Open Challenges;114
3.3.5;References;115
3.4;Techniques to Achieve Energy Proportionality in Data Centers: A Survey;118
3.4.1;1 Introduction;118
3.4.2;2 Energy Proportionality;120
3.4.2.1;2.1 Energy Proportionality at the Server Level;121
3.4.2.2;2.2 Energy Proportionality at Data Center Level;123
3.4.2.3;2.3 Overview on Power Proportionality Techniques at Different Data Center Levels;124
3.4.3;3 Energy Proportionality at Component Level;127
3.4.3.1;3.1 Energy Proportionality at the CPU;127
3.4.3.2;3.2 Energy Proportionality at the Memory;129
3.4.3.3;3.3 Energy Proportionality at the Disk;131
3.4.3.4;3.4 Energy Proportionality at the Networking Interface;132
3.4.4;4 Power Management Techniques at Server Level;133
3.4.5;5 Data Center/Cluster Level Power Management;135
3.4.5.1;5.1 Server Provisioning in Internet Data Centers (IDCs);136
3.4.5.2;5.2 Virtual Machine Management;144
3.4.5.3;5.3 Other Data Center Level Power Management Techniques;148
3.4.6;6 Energy Cost Minimization Through Workload Distribution Across Data Centers;152
3.4.7;7 Data Center Simulation Tools;157
3.4.8;8 Performance of Server and Data Center Level Power Management Techniques;159
3.4.9;9 Conclusions;161
3.4.10;References;162
3.5;A Power-Aware Autonomic Approach for Performance Management of Scientific Applications in a Data Center Environment;172
3.5.1;1 Introduction;172
3.5.2;2 Background;175
3.5.3;3 An Online Look-Ahead Control-based Management Approach;182
3.5.4;4 Case Study: Performance Management of a Parallel Loop Execution Environment;187
3.5.5;5 Benefits of the Proposed Approach;193
3.5.6;6 Combining DLS Techniques with the Proposed Approach;194
3.5.7;7 Conclusion;195
3.5.8;References;196
3.6;CoolEmAll: Models and Tools for Planning and Operating Energy Efficient Data Centres;199
3.6.1;1 Introduction;199
3.6.1.1;1.1 The CoolEmAll Project;202
3.6.1.2;1.2 RelatedWork;204
3.6.2;2 Simulation, Visualisation and Decision Support Toolkit;205
3.6.2.1;2.1 Architecture;206
3.6.2.2;2.2 Application Profiler;208
3.6.2.3;2.3 Data Center Workload and Resource Management Simulator;209
3.6.2.3.1;2.3.1 Architecture;209
3.6.2.3.2;2.3.2 Workload Modelling;210
3.6.2.3.3;2.3.3 Resource Description;211
3.6.2.3.4;2.3.4 Simulation of Energy Efficiency;212
3.6.2.3.5;2.3.5 Application Performance Modelling;213
3.6.2.4;2.4 Interactive Computational Fluid Dynamics Simulation;214
3.6.2.5;2.5 Visualization;216
3.6.3;3 Data centre Efficiency Building Blocks;217
3.6.3.1;3.1 DEBB Concept and Structure;217
3.6.3.2;3.2 Hardware Models for Workload Simulation;220
3.6.3.2.1;3.2.1 Hardware Modelling in DCworms Workload Simulator;220
3.6.3.2.2;3.2.2 Hardware Power Profiles;222
3.6.3.2.3;3.2.3 Electrical Model of the Power Supply Unit 2.0;222
3.6.3.3;3.3 Hardware Models for Thermodynamic Profiles and Cooling Equipment;223
3.6.3.4;3.4 Hardware Models for CFD Simulation;225
3.6.3.5;3.5 Assessment of DEBBs;227
3.6.4;4 Energy Efficiency Metrics;227
3.6.4.1;4.1 State of the Art;228
3.6.4.2;4.2 Selected Metrics for CoolEmAll;229
3.6.4.2.1;4.2.1 Resource Usage Metrics;230
3.6.4.2.2;4.2.2 Energy Based Metrics;231
3.6.4.2.3;4.2.3 Heat-Aware Metrics;232
3.6.4.3;4.3 Application Power Model;233
3.6.5;5 Validation of the CoolEmAll Approach;234
3.6.5.1;5.1 Validation Approach;234
3.6.5.1.1;5.1.1 Capacity Management;236
3.6.5.1.2;5.1.2 Optimisation of Rack Arrangement in a Compute Room Using Open Data Centre Building Blocks;236
3.6.5.1.3;5.1.3 Analysis of Free Cooling Efficiency for Various Inlet Temperatures;237
3.6.5.2;5.2 Testbed;237
3.6.5.3;5.3 Analysis and Optimization of Data Centre Efficiency;239
3.6.5.3.1;5.3.1 Capacity Management;239
3.6.5.3.2;5.3.2 Analysing Cooling Efficiency in Compute-room;246
3.6.6;6 Business Impact;248
3.6.7;7 Summary;250
3.6.8;References;251
3.7;Smart Data Center;254
3.7.1;1 Introduction;254
3.7.2;2 System Model;255
3.7.2.1;2.1 Long Term Power Purchase;256
3.7.2.2;2.2 Real Time Power Purchase;257
3.7.3;3 Constraints;257
3.7.3.1;3.1 Purchasing Accuracy and Cost;257
3.7.3.2;3.2 Data Center Availability;258
3.7.3.3;3.3 UPS Lifetime;258
3.7.4;4 Cost Minimization;259
3.7.5;5 Algorithm Design;259
3.7.5.1;5.1 Drift Plus Penalty Upper Bound;260
3.7.5.2;5.2 Relaxed Optimization;262
3.7.5.3;5.3 Two Timescale Smart Data Center Algorithm;263
3.7.6;6 Performance Analysis;264
3.7.7;7 Related Work;267
3.7.8;8 Conclusions;267
3.7.9;References;268
3.8;Power and Thermal Efficient Numerical Processing;270
3.8.1;1 Introduction;270
3.8.2;2 Floating-Point Representation;271
3.8.2.1;2.1 Formats;272
3.8.2.2;2.2 Rounding Modes;272
3.8.2.3;2.3 Operations;273
3.8.2.4;2.4 Exceptions;273
3.8.3;3 Floating-Point Addition;273
3.8.4;4 Floating-Point Multiplication;275
3.8.5;5 Floating-Point Fused Multiply-Add;277
3.8.6;6 Floating-Point Division;279
3.8.6.1;6.1 Division by Digit Recurrence;279
3.8.6.1.1;6.1.1 Radix-4 Division Algorithm;280
3.8.6.1.2;6.1.2 Intel Penryn Division Unit;281
3.8.6.1.3;6.1.3 Radix-16 by Overlapping Two Radix-4 Stages;281
3.8.6.2;6.2 Division by Multiplication;283
3.8.7;7 Energy dissipation in FP-units;286
3.8.7.1;7.1 Energy Metrics;286
3.8.7.2;7.2 Implementation of the FP-Units;287
3.8.7.3;7.3 Energy Consumption in Floating-Point Workloads;288
3.8.7.4;7.4 Thermal Analysis;290
3.8.8;8 Conclusions and Outlook on FP-Units;292
3.8.9;References;292
3.9;Providing Green Services in HPC Data Centers: A Methodology Based on Energy Estimation;294
3.9.1;1 Introduction;294
3.9.2;2 Identifying Operations in a Service;297
3.9.2.1;2.1 Fault Tolerance Case;297
3.9.2.2;2.2 Data Broadcasting Case;298
3.9.2.3;2.3 Associated Parameters;299
3.9.3;3 Energy Calibration Methodology;300
3.9.3.1;3.1 Calibration of the Power Consumption op;301
3.9.3.2;3.2 Calibration of the Execution Time top;302
3.9.3.2.1;3.2.1 Fault Tolerance Case;303
3.9.3.2.2;3.2.2 Data Broadcasting Case;304
3.9.4;4 Energy Estimation Methodology;305
3.9.4.1;4.1 Fault Tolerance Case;306
3.9.4.1.1;4.1.1 Checkpointing;307
3.9.4.1.2;4.1.2 Message Logging;307
3.9.4.1.3;4.1.3 Coordination;308
3.9.4.2;4.2 Data Broadcasting Case;309
3.9.4.2.1;4.2.1 MPI/SAG and Hybrid/SAG;309
3.9.4.2.2;4.2.2 MPI/Pipeline and Hybrid/Pipeline;310
3.9.5;5 Validation of the Estimations;311
3.9.5.1;5.1 Calibration Results of the Platform;311
3.9.5.1.1;5.1.1 Calibrating the Power Consumption;311
3.9.5.1.2;5.1.2 Calibration of the Execution Time;314
3.9.5.2;5.2 Accuracy of the Estimations;320
3.9.5.2.1;5.2.1 Fault Tolerance Case;321
3.9.5.2.2;5.2.2 Data Broadcasting Case;323
3.9.6;6 Energy-Aware Choice of Services for HPC applications;325
3.9.6.1;6.1 Fault Tolerance Protocols;325
3.9.6.2;6.2 Data Broadcasting Algorithms;326
3.9.7;7 Conclusion;327
3.9.8;References;329
4;Part II Networking;331
4.1;Network Virtualization in Data Centers: A Data Plane Perspective;332
4.1.1;1 Introduction;332
4.1.1.1;1.1 Network Link Virtualization;333
4.1.1.2;1.2 Network Node Virtualization;333
4.1.1.3;1.3 Organization;334
4.1.2;2 Flexible Flow Matching for Network Link Virtualization;334
4.1.2.1;2.1 Background;334
4.1.2.2;2.2 Existing Solutions;336
4.1.2.3;2.3 Algorithmic Solution for Efficient Flexible Flow Matching;337
4.1.2.3.1;2.3.1 Motivations;337
4.1.2.3.2;2.3.2 Algorithms;339
4.1.2.3.3;2.3.3 Architecture;340
4.1.2.4;2.4 Performance Evaluation;342
4.1.2.4.1;2.4.1 Experimental Setup;342
4.1.2.4.2;2.4.2 Algorithm Evaluation;342
4.1.2.4.3;2.4.3 Hardware Implementation;344
4.1.3;3 Resource Consolidation in Network Node Virtualization;344
4.1.3.1;3.1 Background;345
4.1.3.2;3.2 Existing Solutions;346
4.1.3.3;3.3 Efficient Algorithm for Resource Consolidation;346
4.1.3.3.1;3.3.1 Motivations;346
4.1.3.3.2;3.3.2 Trie Merging;348
4.1.3.3.3;3.3.3 Lookup Process;349
4.1.3.3.4;3.3.4 Traffic Isolation;349
4.1.3.4;3.4 Analysis and Evaluation;350
4.1.3.4.1;3.4.1 Theoretical Comparison;350
4.1.3.4.2;3.4.2 Experimental Setup;350
4.1.3.4.3;3.4.3 Scalability;351
4.1.3.4.4;3.4.4 Execution Time;352
4.1.4;4 Summary and Discussion;352
4.1.5;References;353
4.2;Optical Data Center Networks: Architecture, Performance, and Energy Efficiency;355
4.2.1;1 Introduction;355
4.2.2;2 Optical Switches Used in Optical Data Center Networks;357
4.2.2.1;2.1 Optical Packet Switches;357
4.2.2.2;2.2 Optical Circuit Switches;358
4.2.3;3 Approach 1: Optical Data Center Networks to Provide Large Bandwidth for All-to-All Communication;360
4.2.3.1;3.1 Optical Packet Switches with Large Bandwidth;361
4.2.3.2;3.2 Data Center Network Structure Using Optical Packet Switches;362
4.2.3.2.1;3.2.1 Connection Within Group;364
4.2.3.2.2;3.2.2 Connection Between Groups;364
4.2.3.2.3;3.2.3 Routing in Topology;364
4.2.3.3;3.3 Parameter Settings;366
4.2.3.3.1;3.3.1 Parameters for Connection Between Groups;367
4.2.3.3.2;3.3.2 Parameters for Connection within Group;367
4.2.3.4;3.4 Evaluation;368
4.2.3.4.1;3.4.1 Topologies;368
4.2.3.4.2;3.4.2 Properties of Topologies;370
4.2.3.4.3;3.4.3 Maximum Link Load;373
4.2.4;4 Approach 2: Networks to Achieve Low Energy Consumption;374
4.2.4.1;4.1 Overview;376
4.2.4.2;4.2 Virtual Network Topologies Suitable for Optical Data Center Networks;377
4.2.4.2.1;4.2.1 Requirements;377
4.2.4.2.2;4.2.2 Existing Network Structures for Data Centers;378
4.2.4.2.3;4.2.3 Generalized Flattened Butterfly;380
4.2.4.3;4.3 Control of Virtual Network Topology to Achieve Low Energy Consumption;388
4.2.4.3.1;4.3.1 Outline;388
4.2.4.3.2;4.3.2 Control of Topology to Satisfy Requirements;389
4.2.4.4;4.4 Evaluation;391
4.2.5;5 Conclusion;393
4.2.6;References;394
4.3;Scalable Network Communication Using Unreliable RDMA;396
4.3.1;1 Introduction;396
4.3.1.1;1.1 The Significance of Data Communication;397
4.3.1.2;1.2 Datacenter Computing and RDMA;399
4.3.1.3;1.3 High-Performance Computing and RDMA;399
4.3.1.4;1.4 RDMA and the Current Unreliable Datagram Network Transports;400
4.3.2;2 Overview of RDMA Technology;401
4.3.2.1;2.1 Overview of the iWARP Standard;402
4.3.2.2;2.2 Overview of the InfiniBand Standard;404
4.3.3;3 The Case for RDMA over Unreliable Transports;405
4.3.3.1;3.1 Importance of Unreliable Connectionless RDMA;405
4.3.3.2;3.2 Benefits of RDMA over Unreliable Datagrams for iWARP;406
4.3.4;4 RDMA over Unreliable Datagrams;408
4.3.4.1;4.1 Related Work and Development History;409
4.3.4.2;4.2 iWARP Extension Methodology;410
4.3.4.3;4.3 iWARP Design Changes;410
4.3.4.4;4.4 RDMA Write-Record;413
4.3.4.5;4.5 Packet Loss Design Considerations;416
4.3.5;5 Datagram-iWARP Software Implementation;416
4.3.5.1;5.1 iWARP Socket Interface;418
4.3.6;6 Experimental Results and Analysis;418
4.3.6.1;6.1 Verbs-Layer Microbenchmarks;419
4.3.6.2;6.2 Send/Recv Broadcast;419
4.3.6.3;6.3 Packet Loss and Performance;420
4.3.6.4;6.4 Datacenter Application Results;422
4.3.7;7 Summary;425
4.3.8;References;426
4.4;Packet Classification on Multi-core Platforms;428
4.4.1;1 Introduction;428
4.4.2;2 Background;429
4.4.2.1;2.1 Multi-field Packet Classification;429
4.4.2.2;2.2 Related Work;430
4.4.2.3;2.3 Multi-core Processor;431
4.4.3;3 Decision-Tree Based Approaches;432
4.4.3.1;3.1 Algorithms;432
4.4.3.2;3.2 Challenges and Prior Work;434
4.4.4;4 Decomposition-Based Approaches;435
4.4.4.1;4.1 Overview;435
4.4.4.2;4.2 Challenges and Prior Work;436
4.4.4.3;4.3 Preprocessing;437
4.4.4.4;4.4 Searching;440
4.4.4.5;4.5 Merging;441
4.4.5;5 Performance Evaluation and Summary of Results;441
4.4.5.1;5.1 Experimental Setup;441
4.4.5.2;5.2 Latency;443
4.4.5.3;5.3 Throughput;444
4.4.5.4;5.4 Cache Performance;445
4.4.5.5;5.5 Impact of the Number of Threads;447
4.4.5.6;5.6 Comparison with Existing Approaches;447
4.4.6;6 Conclusion;449
4.4.7;References;449
4.5;Optical Interconnects for Data Center Networks;451
4.5.1;1 Introduction;451
4.5.2;2 Need for Optical Interconnects in Data Center Networks;452
4.5.3;3 Optical Components in Data Centers;455
4.5.3.1;3.1 Semiconductor Optical Amplifier (SOA);456
4.5.3.2;3.2 Silicon Micro Ring Resonator;456
4.5.3.3;3.3 ArrayedWaveguide Grating;456
4.5.3.4;3.4 Wavelength Selective Switch;458
4.5.3.5;3.5 MEMS Switch(Optical Switching Matrix, Optical Crossbar);459
4.5.3.6;3.6 Circulators;461
4.5.3.7;3.7 Optical Multiplexer and De-multiplexer;461
4.5.4;4 Optical Interconnects in Data Center Networks and their Performance;461
4.5.4.1;4.1 Reconfigurable Architectures;461
4.5.4.1.1;4.1.1 An Enhanced Optically Connected Network Architecture;462
4.5.4.1.2;4.1.2 OSA, a Novel Optical Switching Architecture for DCNs;462
4.5.4.1.3;4.1.3 Wavelength-reconfigurable optical packet and circuit switched platform for DCNs;463
4.5.4.1.4;4.1.4 Next-Generation Optically-Interconnected High-Performance Data Centers;464
4.5.4.1.5;4.1.5 The Data Vortex Optical Packet Switched Interconnection Network;465
4.5.4.1.6;4.1.6 Proteus: A Topology Malleable Data Center Network;465
4.5.4.1.7;4.1.7 A Hybrid Optical Packet and Wavelength Selective Switch for High-Performance DCNs;466
4.5.4.2;4.2 Power Saving Architectures;467
4.5.4.2.1;4.2.1 VCSEL Based Energy Efficient and Bandwidth Reconfigurable Architecture;467
4.5.4.2.2;4.2.2 A Wavelength Striped, Packet Switched, Optical Interconnection Network;468
4.5.4.2.3;4.2.3 SPRINT: Scalable Photonic Switching Fabric for HIGH PERFORMANCE COMPUTING;468
4.5.4.3;4.3 Low Latency Architectures;470
4.5.4.3.1;4.3.1 DOS: A Scalable Optical Switch for Data Centers;470
4.5.4.3.2;4.3.2 Scalable Optical Packet Switch Architecture for Low Latency and High Load;471
4.5.4.3.3;4.3.3 AWGR Based Data Center Switches Using RSOA-based Optical Mutual Exclusion;472
4.5.4.3.4;4.3.4 A Petabit Photonic Packet Switch (P3S);472
4.5.4.3.5;4.3.5 Optical Interconnection Networks: The OSMOSIS Project;473
4.5.4.3.6;4.3.6 A Scalable Optical Multi-Plane Interconnection Architecture;474
4.5.4.3.7;4.3.7 Low Latency and Large Port Count OPS for Data Center Network Interconnects;474
4.5.4.4;4.4 Link Bandwidth Scaling Architectures;476
4.5.4.4.1;4.4.1 Data Center Network Based on Flexible Bandwidth MIMO OFDM Optical Interconnects;476
4.5.4.4.2;4.4.2 Photonic Terabit Routers Employing WDM;477
4.5.4.5;4.5 High Radix Switch Design;478
4.5.5;5 Data center traffic characteristics;478
4.5.6;6 Energy Requirements for Data Center Networks;480
4.5.7;7 Routing in Data Centers;482
4.5.8;References;483
4.6;TCP Congestion Control in Data Center Networks;486
4.6.1;1 Introduction;486
4.6.2;2 TCP Impairments in Data Center Networks;487
4.6.2.1;2.1 TCP Incast;488
4.6.2.2;2.2 TCP Outcast;489
4.6.2.3;2.3 Queue Buildup;490
4.6.2.4;2.4 Buffer Pressure;491
4.6.2.5;2.5 Pseudo-Congestion Effect;491
4.6.2.6;2.6 Summary: TCP Impairments and Causes;492
4.6.3;3 TCP Variants for Data Center Networks;493
4.6.3.1;TCP with FG-RTO + Delayed ACKs Disabled [3];493
4.6.3.1.1;3.3.1 Explicit Congestion Notification (ECN);494
4.6.4;4 Summary: TCP Variants for DCNs;503
4.6.5;5 Open Issues;505
4.6.6;6 Concluding Remarks;505
4.6.7;References;505
4.7;Routing Techniques in Data Center Networks;507
4.7.1;1 Introduction;507
4.7.2;2 Classification of Routing Schemes in Data Centers;510
4.7.2.1;2.1 Topology-Aware Routing;511
4.7.2.1.1;2.1.1 Server-Centric Approach;511
4.7.2.1.2;2.1.2 Switch-centric Approach;512
4.7.2.2;2.2 Energy-Aware Routing;516
4.7.2.2.1;2.2.1 Green Routing;516
4.7.2.2.2;2.2.2 Power-Aware Routing;518
4.7.2.3;2.3 Traffic-sensitive Routing;519
4.7.2.3.1;2.3.1 DARD;520
4.7.2.3.2;2.3.2 Hedera;522
4.7.2.3.3;2.3.3 ESM: Multicast Routing for Data Centers;523
4.7.2.3.4;2.3.4 GARDEN;524
4.7.2.4;2.4 Routing for Content Distribution Networks (CDN);525
4.7.2.4.1;2.4.1 Request-Routing in CDNs;526
4.7.2.4.2;2.4.2 Symbiotic Routing;527
4.7.2.4.3;2.4.3 fs-PGBR: A Scalable and Delay Sensitive Cloud Routing Protocol;528
4.7.2.5;2.5 Summary of All Routing and Forwarding Techniques;528
4.7.3;3 Open Issues and Challenges;529
4.7.4;4 Conclusions;530
4.7.5;References;531
5;Part III Cloud Computing;533
5.1;Auditing for Data Integrity and Reliability in Cloud Storage;534
5.1.1;1 Introduction;534
5.1.2;2 Information Auditing: Objective and Approaches;536
5.1.2.1;2.1 Definition of Information Auditing;536
5.1.2.2;2.2 Three Approaches of Information Auditing;537
5.1.3;3 Auditing for Data Integrity in Distributed Systems;538
5.1.3.1;3.1 Strategies of Auditing Data Integrity;538
5.1.3.2;3.2 Proof of Retrievability;539
5.1.3.3;3.3 Provable Data Possession;542
5.1.3.3.1;3.3.1 Preliminaries;543
5.1.3.3.2;3.3.2 Defining the PDP Protocol;544
5.1.3.3.3;3.3.3 The Secure PDP Scheme (S-PDP);545
5.1.3.3.4;3.3.4 The Efficient PDP Scheme (E-PDP);547
5.1.3.4;3.4 Compact Proof of Retrievability;547
5.1.3.4.1;3.4.1 System Model;547
5.1.3.4.2;3.4.2 Private Verification Construction;548
5.1.3.4.3;3.4.3 Public Verification Construction;549
5.1.4;4 Auditing in Cloud Storage Platform;550
5.1.4.1;4.1 Challenges;551
5.1.4.2;4.2 Public Verifiability;552
5.1.4.3;4.3 Dynamic Data Operations Support;552
5.1.4.4;4.4 Privacy Preserving;554
5.1.4.5;4.5 Multiple Verifications;555
5.1.5;5 Open Questions;556
5.1.6;6 Conclusions;557
5.1.7;References;557
5.2;I/O and File Systems for Data-Intensive Applications;559
5.2.1;1 Parallel File Systems vs. Data-Intensive File Systems: A Comparison;559
5.2.2;2 Chunk-Aware I/O: Enabling HPC on Data-Intensive File Systems;562
5.2.2.1;2.1 Motivation;562
5.2.2.2;2.2 Chunk-Aware I/O Design;564
5.2.2.3;2.3 Chunk-Aware I/O Implementation;569
5.2.2.4;2.4 Chunk-Aware I/O Analysis;569
5.2.2.5;2.5 CHAIO Performance;570
5.2.2.5.1;2.5.1 Experiment Setup;570
5.2.2.5.2;2.5.2 Performance with Different Request Sizes;570
5.2.2.5.3;2.5.3 Performance with Two Replicas;571
5.2.2.5.4;2.5.4 Performance with Different Number of Nodes;572
5.2.2.5.5;2.5.5 Overhead Analysis in Large-Scale Computing Environments;573
5.2.2.5.6;2.5.6 Load Balance;575
5.2.3;3 Related Works;575
5.2.3.1;3.1 HPC on Data-Intensive File Systems;576
5.2.3.2;3.2 N-1 Data Access and its Handling;577
5.2.4;4 Summary;578
5.2.5;References;579
5.3;Cloud Resource Pricing Under Tenant Rationality;581
5.3.1;1 Introduction;581
5.3.2;2 The Game Model;582
5.3.2.1;2.1 User Model and Virtual Instances Pricing;582
5.3.2.2;2.2 Modeling Cloud Revenue and Tenant Surplus;583
5.3.2.2.1;2.2.1 Stage I: Cloud Revenue Maximization;583
5.3.2.2.2;2.2.2 Stage II: Tenant Surplus Maximization;584
5.3.2.3;2.3 Stackelberg Equilibrium;584
5.3.3;3 Usage-Based Cloud Resource Pricing;585
5.3.3.1;3.1 Non-Uniform Pricing;585
5.3.3.1.1;3.1.1 Stage II: Tenant Surplus Maximization;585
5.3.3.1.2;3.1.2 Stage I: Cloud Pricing Choices;586
5.3.3.2;3.2 Uniform Pricing;590
5.3.3.2.1;3.2.1 Stage II: Tenant Surplus Maximization;590
5.3.3.2.2;3.2.2 Stage I: Cloud Pricing Choices;591
5.3.4;4 The Effectiveness of Stackelberg Strategies;592
5.3.4.1;4.1 Centralized Aggregate Network Utility Maximization;592
5.3.4.2;4.2 Total Network Utility Under Selfish Interactions;595
5.3.4.3;4.3 Asymptotic Analysis of Price of Anarchy;597
5.3.5;5 Broker Resource Pricing;598
5.3.6;6 Performance Evaluation;600
5.3.6.1;6.1 Setup;600
5.3.6.2;6.2 Economic Implications of Cloud Resource Pricing;600
5.3.6.3;6.3 Social Welfare Tradeoffs, and Hidden Effects;601
5.3.7;7 Related Work;602
5.3.8;8 Concluding Remarks;603
5.3.9;References;603
5.4;Online Resource Management for Carbon-Neutral Cloud Computing;604
5.4.1;1 Introduction;604
5.4.1.1;1.1 Background;605
5.4.1.2;1.2 Carbon Neutrality: Benefits and Challenges;606
5.4.1.3;1.3 Current Research and Limitations;606
5.4.1.4;1.4 Contributions;607
5.4.2;2 Model;608
5.4.2.1;2.1 Some Assumptions;609
5.4.2.2;2.2 Energy Sources;609
5.4.2.3;2.3 Data Center;610
5.4.2.4;2.4 Workload;611
5.4.3;3 Problem Formulation;612
5.4.3.1;3.1 Objective and Constraints;612
5.4.3.2;3.2 Offline Problem Formulation;614
5.4.4;4 Algorithm for Cost Optimization and Carbon Neutrality;614
5.4.4.1;4.1 Carbon Deficit Queue;614
5.4.4.2;4.2 Optimizing for Cost Minimization and Carbon Neutrality;615
5.4.4.2.1;4.2.1 Working Principle of COCA;615
5.4.4.2.2;4.2.2 Distributed Implementation;616
5.4.4.3;4.3 Performance Analysis;617
5.4.5;5 Simulation;619
5.4.5.1;5.1 Data Sets;619
5.4.5.2;5.2 Results;621
5.4.5.2.1;5.2.1 Efficiency of COCA;621
5.4.5.2.2;5.2.2 Comparison with Prediction-Based Method;623
5.4.6;6 Extension to Geographic Load Balancing;624
5.4.7;7 Conclusions;625
5.4.8;References;625
5.5;A Big Picture of Integrity Verification of Big Data in Cloud Computing;628
5.5.1;1 Introduction;628
5.5.2;2 Motivating Examples;630
5.5.3;3 Problem Analysis---Framework and Lifecycle;631
5.5.4;4 Representative Approaches and Analysis;633
5.5.4.1;4.1 Preliminaries;633
5.5.4.1.1;4.1.1 RSA Signature;633
5.5.4.1.2;4.1.2 Bilinear Pairing and BLS Signature;634
5.5.4.1.3;4.1.3 Merkle Hash Tree;634
5.5.4.2;4.2 Representative Schemes;635
5.5.4.2.1;4.2.1 PDP;635
5.5.4.2.2;4.2.2 Compact POR;636
5.5.4.2.3;4.2.3 DPDP;637
5.5.4.2.4;4.2.4 Public Auditing of Dynamic Data;637
5.5.4.2.5;4.2.5 Authorized Auditing with Fine-Grained Data Updates;638
5.5.5;5 Other Related Work;638
5.5.6;6 Conclusions and Future Work;639
5.5.7;References;640
5.6;An Out-of-Core Task-based Middleware for Data-Intensive Scientific Computing;643
5.6.1;1 Introduction;643
5.6.2;2 Related Work;646
5.6.3;3 An Out-of-Core Task-based Middleware;647
5.6.3.1;3.1 Global and Local Schedulers;649
5.6.3.2;3.2 Storage Service;650
5.6.4;4 Linear Algebra Frontend (LAF);651
5.6.5;5 A Case Study: Block Iterative Eigensolver Using DOoC+LAF;652
5.6.5.1;5.1 Eigenvalue Problem in the Configuration Interaction Approach;652
5.6.5.2;5.2 Implementation Using 1D partitioning;654
5.6.5.3;5.3 Implementation Using a 2D Partitioning;656
5.6.6;6 Experiments;656
5.6.6.1;6.1 Practical Considerations;657
5.6.6.2;6.2 Performance Results for Nmax=8;658
5.6.7;7 Conclusions;660
5.6.8;References;661
5.7;Building Scalable Software for Data Centers: An Approach to Distributed Computing at Enterprise Level;664
5.7.1;1 Introduction to Big Data Problems;664
5.7.2;2 Known Solutions at Design Phase: Overview of Design Patterns for Parallel & Distributed Computing;666
5.7.3;3 Introduction to MapReduce Programming Model;669
5.7.4;4 Overview of Apache Hadoop: A Framework for Distributed Computing;672
5.7.4.1;4.1 Distributed File System: HDFS;672
5.7.4.2;4.2 MapReduce Framework & API;674
5.7.4.3;4.3 Database Support: HBase;678
5.7.4.4;4.4 High Level Programming Language: Pig;679
5.7.4.5;4.5 Hive: Another Database Support & High Level Programming Language;680
5.7.5;5 Conclusions;682
5.7.6;References;682
5.8;Cloud Storage over Multiple Data Centers;685
5.8.1;1 Introduction;685
5.8.2;2 Cloud Storage in a Nutshell;687
5.8.2.1;2.1 Architecture;687
5.8.2.2;2.2 Metadata Service;689
5.8.2.2.1;2.2.1 Layout Manager;689
5.8.2.2.2;2.2.2 Meta-Server;689
5.8.2.2.3;2.2.3 Lock Service;690
5.8.2.3;2.3 Storage Service;690
5.8.2.3.1;2.3.1 Namenode;690
5.8.2.3.2;2.3.2 Chunk Servers;691
5.8.3;3 Replication Strategies;691
5.8.3.1;3.1 Introduction;691
5.8.3.2;3.2 Asynchronous Replication;692
5.8.3.3;3.3 Synchronous Replication;694
5.8.3.4;3.4 Placement of Replicas;695
5.8.4;4 Data Striping Methods;696
5.8.4.1;4.1 Introduction;696
5.8.4.2;4.2 Erasure Code Types;697
5.8.4.3;4.3 Erasure Codes in Data Centers;698
5.8.5;5 Consistency Models;699
5.8.5.1;5.1 Introduction;699
5.8.5.2;5.2 Strong Consistency;700
5.8.5.3;5.3 Weak Consistency;701
5.8.6;6 Cloud of Multiple Clouds;703
5.8.6.1;6.1 Introduction;703
5.8.6.2;6.2 Architecture;704
5.8.6.3;6.3 Data Striping;705
5.8.6.4;6.4 Retrieving Strategy;707
5.8.6.5;6.5 Mutual Exclusion;707
5.8.7;7 Privacy and Security of Storage System;709
5.8.7.1;7.1 Introduction;709
5.8.7.2;7.2 Fine-Grained Data Access Control;710
5.8.7.3;7.3 Security on Storage Server;712
5.8.8;8 Conclusion and Future Directions;714
5.8.9;References;715
6;Part IV Hardware;720
6.1;Realizing Accelerated Cost-Effective Distributed RAID;721
6.1.1;1 Introduction;721
6.1.2;2 Background;723
6.1.2.1;2.1 Rationale;723
6.1.2.1.1;2.1.1 Backend vs. Client-driven Parity Generation;723
6.1.2.1.2;2.1.2 Block-Based vs. Per-File RAID;724
6.1.2.1.3;2.1.3 Hardware vs. Accelerated Software RAID;724
6.1.2.1.4;2.1.4 Discussion;725
6.1.2.2;2.2 Enabling Technologies;725
6.1.2.2.1;2.2.1 Erasure Codes;725
6.1.2.2.2;2.2.2 The Lustre Parallel File System;727
6.1.2.2.3;2.2.3 KGPU;727
6.1.3;3 Design;728
6.1.3.1;3.1 System Overview;728
6.1.3.2;3.2 RAID-enabled PFS Design;729
6.1.3.3;3.3 Control Flow;730
6.1.3.4;3.4 Degraded Array Reconstruction;732
6.1.4;4 Implementation;732
6.1.4.1;4.1 Basic GPU Implementation;733
6.1.4.2;4.2 Optimizations;733
6.1.5;5 Evaluation;734
6.1.5.1;5.1 Experimental Setup;734
6.1.5.2;5.2 I/O Throughput Measurement;735
6.1.5.2.1;5.2.1 Raw Throughput;735
6.1.5.2.2;5.2.2 Encoding Throughput;736
6.1.5.2.3;5.2.3 Impact of Number of Disks on Throughput;737
6.1.5.2.4;5.2.4 End-to-End Data Integrity;739
6.1.5.3;5.3 RAID Reconstruction Cost;739
6.1.5.4;5.4 Impact on Applications;740
6.1.6;6 Related Work;740
6.1.7;7 Conclusion;742
6.1.8;References;742
6.2;Efficient Hardware-Supported Synchronization Mechanisms for Manycores;745
6.2.1;1 Introduction;745
6.2.2;2 The G-Lines Technology;746
6.2.3;3 Hardware Barrier Synchronization;747
6.2.4;4 The GBarrier Synchronization Mechanism;748
6.2.4.1;4.1 Dedicated On-Chip Network Architecture;749
6.2.4.2;4.2 Synchronization Protocol;750
6.2.4.3;4.3 Programmability Issues;753
6.2.5;5 Performance Implications;754
6.2.5.1;5.1 Implementation Technologies;754
6.2.5.1.1;5.1.1 G-Lines Technology;754
6.2.5.1.2;5.1.2 Standard Technology;754
6.2.5.2;5.2 Raw Performance Statistics;755
6.2.6;6 Evaluation;757
6.2.6.1;6.1 Experimental Setup;757
6.2.6.2;6.2 Barrier Implementations;758
6.2.6.3;6.3 Performance Results;759
6.2.6.3.1;6.3.1 Execution Time;759
6.2.6.3.2;6.3.2 Network Traffic;763
6.2.6.3.3;6.3.3 Energy Efficiency;765
6.2.7;7 Related Work;766
6.2.8;8 Hardware Lock Synchronization;768
6.2.9;9 The GLock Synchronization Mechanism;770
6.2.9.1;9.1 Dedicated On-Chip Network Architecture;770
6.2.9.2;9.2 Synchronization Protocol;771
6.2.9.3;9.3 Programmability Issues;774
6.2.10;10 Performance Implications;776
6.2.10.1;10.1 Implementation Technologies;776
6.2.10.1.1;10.1.1 G-Lines Technology;776
6.2.10.1.2;10.1.2 Standard Technology;777
6.2.10.2;10.2 Raw Performance Statistics;778
6.2.11;11 Evaluation;779
6.2.11.1;11.1 Experimental Setup;779
6.2.11.2;11.2 Post-mortem Analysis of Benchmarks;781
6.2.11.3;11.3 Lock Implementations;782
6.2.11.4;11.4 Performance Results;783
6.2.11.4.1;11.4.1 Execution Time;783
6.2.11.4.2;11.4.2 Network Traffic;786
6.2.11.4.3;11.4.3 Energy Efficiency;788
6.2.12;12 Related Work;789
6.2.13;13 Conclusions;791
6.2.14;References;793
6.3;Hardware Approaches to Transactional Memory in Chip Multiprocessors;796
6.3.1;1 Introduction;796
6.3.2;2 Why Transactional Memory Is Going Mainstream;798
6.3.2.1;2.1 The Drawbacks of Lock-Based Synchronization;799
6.3.2.2;2.2 The Transactional Abstraction;799
6.3.2.3;2.3 High-Performance Transactional Memory;800
6.3.2.4;2.4 Industrial Adoption of Hardware Transactional Memory;801
6.3.3;3 Fundamentals of Transactional Memory;802
6.3.4;4 Hardware Mechanisms for Transactional Memory;803
6.3.4.1;4.1 ISA Extensions;803
6.3.4.2;4.2 Transactional Book-Keeping;804
6.3.4.3;4.3 Data Versioning;805
6.3.4.4;4.4 Conflict Detection and Resolution;805
6.3.4.5;4.5 Transaction Commit;807
6.3.4.6;4.6 Transaction Abort;807
6.3.5;5 Intel TSX: TM Support in Mainstream Processors;808
6.3.5.1;5.1 Hardware Lock Elision;809
6.3.5.2;5.2 Restricted Transactional Memory;810
6.3.6;6 Analysing Intel TSX Performance on Haswell;810
6.3.7;7 An Overview of Hardware TM Research;815
6.3.8;8 Conclusions;821
6.3.9;References;821
7;Part V Modeling and Simulation;827
7.1;Data Center Modeling and Simulation Using OMNeT++;828
7.1.1;1 Introduction to Modeling and Simulation (M&S) Methodology;829
7.1.1.1;1.1 Parallel Discrete Event Simulation---PDES;830
7.1.2;2 Data Center Architectures;831
7.1.3;3 Data Center Modeling Using OMNeT++;833
7.1.3.1;3.1 Simple Two Node Simulation;833
7.1.3.2;3.2 Advance Level Simulation;836
7.1.3.3;3.3 Data Center Simulation Model;839
7.1.4;4 Wrap Up;843
7.1.5;References;843
7.2;Power-Thermal Modeling and Control of Energy-Efficient Servers and Datacenters;845
7.2.1;1 Introduction;845
7.2.1.1;1.1 Overall Datacenter Architecture;847
7.2.1.2;1.2 Datacenter Workload Characteristics;848
7.2.1.3;1.3 Energy Efficiency of Datacenters;850
7.2.1.4;1.4 Chapter Organization;851
7.2.2;2 State-of-the-Art in Datacenter Design;852
7.2.2.1;2.1 Computing Servers;852
7.2.2.2;2.2 Cooling Infrastructure;854
7.2.3;3 Power and Temperature Modeling and Monitoring;857
7.2.3.1;3.1 Server Modeling;858
7.2.3.2;3.2 Datacenter Modeling;861
7.2.3.3;3.3 Monitoring System for Datacenters;863
7.2.4;4 Power and Thermal Managements of Servers;864
7.2.4.1;4.1 Overview of CPU Power and Thermal Management Techniques;865
7.2.4.2;4.2 Run-Time Hierarchical Power and Thermal Management for Server Architectures;867
7.2.4.3;4.3 Design-Time Power and Thermal Optimizations;871
7.2.5;5 Power and Thermal Managements for Server Clusters;876
7.2.5.1;5.1 Conventional Solution to Minimize Power Consumption for Server Clusters;876
7.2.5.2;5.2 Correlation-Aware Power and Temperature Management;877
7.2.6;6 Power Minimization of Datacenters with Hybrid Cooling Architectures;886
7.2.6.1;6.1 Formal Problem Definition;888
7.2.6.2;6.2 Multi-objective Trade-offs Exploration Between Cooling Mode and Utilization Threshold;889
7.2.6.3;6.3 Simulation Results;893
7.2.7;7 Conclusions;895
7.2.8;References;896
7.3;Thermal Modeling and Management of Storage Systems in Data Centers;902
7.3.1;1 Introduction;902
7.3.2;2 Related Work;904
7.3.2.1;2.1 Efficient Data Centers;904
7.3.2.2;2.2 Thermal Modeling;905
7.3.2.3;2.3 Thermal Management;905
7.3.3;3 Thermal Modeling;906
7.3.3.1;3.1 CPU Thermal Model;907
7.3.3.2;3.2 Disk Thermal Model;909
7.3.3.3;3.3 Thermal Model of Data Nodes;911
7.3.3.4;3.4 Evaluation of Temperature Models;912
7.3.4;4 Thermal Management Strategies;913
7.3.4.1;4.1 Task Scheduling;914
7.3.4.2;4.2 Predictive Thermal-Aware Data Transmission;917
7.3.5;5 Results;919
7.3.5.1;5.1 Task Scheduling;919
7.3.5.1.1;5.1.1 CPU-Intensive Workload;920
7.3.5.1.2;5.1.2 I/O-Intensive Workloads;922
7.3.5.2;5.2 Predictive Thermal-Aware Management System;922
7.3.6;6 Conclusion;926
7.3.7;References;927
7.4;Modeling and Simulation of Data Center Networks;931
7.4.1;1 Data Centers and Cloud Computing;931
7.4.2;2 DCN Architectures;933
7.4.3;3 DCN Graph Modeling;935
7.4.3.1;3.1 ThreeTier DCN Model;936
7.4.3.2;3.2 FatTree DCN Model;937
7.4.3.3;3.3 DCell DCN Model;938
7.4.4;4 DCNs Implementation in ns-3;939
7.4.4.1;4.1 ThreeTier DCN Implementation Details;939
7.4.4.2;4.2 FatTree DCN Implementation Details;940
7.4.4.3;4.3 DCell DCN Implementation Details;942
7.4.5;References;944
8;Part VI Security;945
8.1;C2Hunter: Detection and Mitigation of Covert Channels in Data Centers;946
8.1.1;1 Introduction;946
8.1.2;2 Background;949
8.1.3;3 Threat Model, Scenarios and Assumptions;950
8.1.3.1;3.1 Threat of Data Center;950
8.1.3.2;3.2 Threat Categories of Covert Channels;951
8.1.3.3;3.3 Threat Scenarios of Covert Channels;952
8.1.3.4;3.4 Assumptions;953
8.1.4;4 Overview of C2Hunter;953
8.1.4.1;4.1 Challenges;953
8.1.4.2;4.2 Formal Requirements;954
8.1.4.3;4.3 C2Hunter Framework Summary;954
8.1.4.4;4.4 Covert Channel Modeling;956
8.1.5;5 Two-Phase Synthesis Detection Algorithm;958
8.1.5.1;5.1 Markov Detection Algorithm;959
8.1.5.2;5.2 Bayesian Detection Algorithm;962
8.1.6;6 Mitigation Algorithm;963
8.1.7;7 Implementation and Evaluation;964
8.1.7.1;7.1 Covert Channels Scenarios;965
8.1.7.2;7.2 Captor and Detector;966
8.1.7.3;7.3 Interrupter in Hypervisor;967
8.1.7.4;7.4 Experimental Settings;967
8.1.7.5;7.5 Detection Analysis;969
8.1.7.6;7.6 Mitigation Analysis;972
8.1.8;8 Discussion;974
8.1.9;9 Related Work;976
8.1.10;10 Conclusion;977
8.1.11;References;978
8.2;Selective and Private Access to Outsourced Data Centers;982
8.2.1;1 Introduction;982
8.2.2;2 Access Control Enforcement;984
8.2.2.1;2.1 Selective Encryption;984
8.2.2.2;2.2 Updates to the Access Control Policy;988
8.2.2.3;2.3 Write Privileges;992
8.2.2.4;2.4 Attribute-Based Encryption;994
8.2.3;3 Efficient Access to Encrypted Data;995
8.2.4;4 Protecting Access Privacy;998
8.2.4.1;4.1 Oblivious RAM;999
8.2.4.2;4.2 Dynamically Allocated Data Structures;1000
8.2.4.3;4.3 Shuffle Index;1002
8.2.5;5 Combining Access Control and Indexing Techniques;1007
8.2.6;6 Conclusions;1010
8.2.7;References;1010
8.3;Privacy in Data Centers: A Survey of Attacks and Countermeasures;1013
8.3.1;1 Introduction;1013
8.3.2;2 Privacy;1015
8.3.3;3 Privacy Enhancing Technologies;1016
8.3.4;4 Anonymous Communications;1017
8.3.5;5 Mix Networks;1019
8.3.6;6 Traffic Analysis;1019
8.3.7;7 Mix Systems Attacks;1020
8.3.8;8 The Disclosure Attack;1020
8.3.9;9 The Statistical Disclosure Attack (SDA);1021
8.3.10;10 Extending and Resisting Statistical Disclosure;1022
8.3.11;11 Two Sided Statistical Disclosure Attack (TS-SDA);1022
8.3.12;12 Perfect Matching Disclosure Attack (PMDA);1023
8.3.13;13 Vida: How to Use Bayesian Inference to De-anonymize Persistent Communications;1024
8.3.14;14 SDA with Two Heads (SDA-2H);1024
8.3.15;15 Conclusions;1025
8.3.16;References;1025
9;Part VII Data Services;1028
9.1;Quality-of-Service in Data Center Stream Processing for Smart City Applications;1029
9.1.1;1 Introduction;1029
9.1.2;2 Distributed Stream Processing Systems;1030
9.1.2.1;2.1 Abstract Model;1031
9.1.2.2;2.2 Development Model;1033
9.1.2.3;2.3 Execution Model;1034
9.1.3;3 Platforms for Distributed Stream Processing;1036
9.1.3.1;3.1 IBM InfoSphere Streams;1036
9.1.3.2;3.2 Apache S4;1037
9.1.3.3;3.3 Storm;1038
9.1.4;4 QoS-Aware Stream Processing;1039
9.1.5;5 Quasit;1041
9.1.5.1;5.1 Quasit Abstract Model;1042
9.1.5.2;5.2 Quasit Development Model;1043
9.1.5.3;5.3 Quasit Execution Model;1048
9.1.6;6 Load-Adaptive Active Replication (LAAR);1049
9.1.7;7 Conclusions;1054
9.1.8;References;1055
9.2;Opportunistic Databank: A context Aware on-the-fly Data Center for Mobile Networks;1059
9.2.1;1 Introduction;1059
9.2.2;2 Data Replication in Manets---A Brief Overview;1062
9.2.3;3 Data Replication in DTNs;1064
9.2.3.1;3.1 System Model;1065
9.2.3.2;3.2 Hybrid Scheme for Message Replication (HSM) for DTNs;1067
9.2.3.3;3.3 Empirical Setups and Results;1069
9.2.3.3.1;3.3.1 Performance Metrics;1070
9.2.3.3.2;3.3.2 Related DTN Replication Schemes;1071
9.2.3.3.3;3.3.3 Simulation Results;1072
9.2.4;4 Conclusions;1074
9.2.5;References;1074
9.3;Data Management: State-of-the-Practice at Open-Science Data Centers;1077
9.3.1;1 Introduction;1077
9.3.2;2 Data Storage Infrastructure;1079
9.3.2.1;2.1 Data Storage Media;1079
9.3.2.2;2.2 General Architecture of a Data Storage System;1080
9.3.2.3;2.3 Supporting Databases for Structured and Semi-Structured Datasets;1080
9.3.2.4;2.4 Examples of Notable Storage Systems at Open-Science Data Centers;1081
9.3.3;3 Data Movement;1082
9.3.3.1;3.1 Parallel File-System Associated with Computational Resources---Secondary Storage;1082
9.3.3.2;3.2 Optimizing Data Movement in Context of Secondary Storage System;1085
9.3.3.3;3.3 Optimizing Data Movement in Context of Tertiary Storage System;1086
9.3.4;4 Data Archiving;1087
9.3.5;5 Data Preservation;1088
9.3.6;6 Conclusion;1089
9.3.7;References;1089
9.4;Data Summarization Techniques for Big Data---A Survey;1091
9.4.1;1 Introduction;1091
9.4.2;2 Applications of Data Summarization;1093
9.4.3;3 Clustering Algorithms;1095
9.4.3.1;3.1 Background;1095
9.4.3.2;3.2 Hierarchical Clustering;1097
9.4.3.3;3.3 Partitioning Clustering;1101
9.4.3.4;3.4 Density-Based Clustering Algorithms;1103
9.4.3.5;3.5 Grid-Based Clustering Algorithms;1105
9.4.4;4 Sampling;1107
9.4.4.1;4.1 Probability Sampling;1108
9.4.4.2;4.2 Non-Probabilistic Sampling;1114
9.4.5;5 Compression;1115
9.4.6;6 Wavelets;1120
9.4.7;7 Histograms;1123
9.4.8;8 Micro-Clustering;1125
9.4.9;9 Conclusion;1126
9.4.10;References;1126
10;Part VIII Monitoring;1135
10.1;Central Management of Datacenters;1136
10.1.1;1 Introduction;1136
10.1.2;2 Organization of the Chapter;1137
10.1.2.1;2.1 Management Layer Network;1137
10.1.2.2;2.2 Provisioning of Servers;1139
10.1.2.2.1;2.2.1 Reason to Use Provisioning Servers;1139
10.1.2.3;2.3 Platform Configuration Management System;1140
10.1.2.4;2.4 Resource Utilization Monitoring;1140
10.1.2.5;2.5 Alerting and Alarming System;1142
10.1.2.6;2.6 Central Logging System;1142
10.1.2.6.1;2.6.1 Security Information Event Management;1144
10.1.2.7;2.7 Intrusion Detection and Prevention System;1144
10.1.2.7.1;2.7.1 Types of Intrusion Detection System (IDS);1145
10.1.2.7.2;Network-Based Intrusion Detection System (NIDS);1146
10.1.2.7.3;Host-Based Intrusion Detection System (HIDS);1146
10.1.2.7.4;2.7.2 How Intrusion Detection System Works?;1146
10.1.2.7.5;Anomaly-Based Intrusion Detection System;1146
10.1.2.7.6;Signature-Based Intrusion Detection System;1146
10.1.2.8;2.8 Datacenter Backup and Restore;1147
10.1.2.8.1;2.8.1 The Components of Data Backup and Recovery;1148
10.1.2.8.2;Cold and Hot Backup;1148
10.1.2.8.3;Enterprise Backup and Restore Software;1148
10.1.2.8.4;Online and Offline Storage;1148
10.1.2.9;2.9 Security Management Systems;1149
10.1.3;3 Conclusion;1149
10.1.4;References;1151
10.2;Monitoring of Data Centers using Wireless Sensor Networks;1152
10.2.1;1 Introduction;1152
10.2.2;2 Survey Study;1155
10.2.3;3 Conclusion;1163
10.2.4;References;1163
10.3;Network Intrusion Detection Systems in Data Centers;1165
10.3.1;1 Introduction;1165
10.3.2;2 Origin and Standardization;1170
10.3.3;3 Architecture;1172
10.3.4;4 Subjects of Study;1175
10.3.5;5 Detection Strategies;1177
10.3.6;6 Alert Correlation;1181
10.3.7;7 Summary;1183
10.3.8;References;1184
10.4;Software Monitoring in Data Centers;1188
10.4.1;1 Introduction;1188
10.4.1.1;1.1 Performance Degradation;1189
10.4.1.2;1.2 Function Failure;1190
10.4.1.3;1.3 Energy Conservation;1191
10.4.2;2 Monitoring Content;1192
10.4.2.1;2.1 Basic Software;1193
10.4.2.2;2.2 Middleware;1193
10.4.2.3;2.3 Database;1194
10.4.2.4;2.4 Application Software;1194
10.4.2.5;2.5 PM (Physical Machine) and VM (Virtual Machine);1196
10.4.2.6;2.6 User Behavior Analysis;1198
10.4.2.7;2.7 Hot-Spot Evaluation;1198
10.4.2.8;2.8 Performance Prediction and Advanced Warning;1200
10.4.2.9;2.9 The Performance Bottlenecks Analysis;1201
10.4.3;3 Monitoring Timing;1202
10.4.3.1;3.1 Resource-Oriented Monitoring;1202
10.4.3.2;3.2 Business-Oriented Monitoring;1205
10.4.4;4 Participators;1207
10.4.4.1;4.1 Resource Managers;1207
10.4.4.2;4.2 Service Operators;1208
10.4.4.3;4.3 Data Owner;1209
10.4.4.4;4.4 Software Developers;1209
10.4.5;5 Monitoring Site;1210
10.4.5.1;5.1 On-Site Monitor;1211
10.4.5.2;5.2 Off-Site Monitor;1211
10.4.6;6 Monitoring Methods;1212
10.4.6.1;6.1 Visualization Monitoring;1212
10.4.6.2;6.2 Hot-Spot Evaluation;1214
10.4.6.3;6.3 Performance Prediction;1220
10.4.6.4;6.4 Analyzing User's Habits;1226
10.4.6.5;6.5 Tools;1227
10.4.7;References;1229
11;Part IX Resource Management;1233
11.1;Usage Patterns in Multi-tenant Data Centers: a Large-Case Field Study;1234
11.1.1;1 Introduction;1234
11.1.2;2 Multi-tenant Datacenters;1236
11.1.2.1;2.1 Evolution of Resource Demands;1236
11.1.2.2;2.2 CPU Load Balancing;1237
11.1.2.3;2.3 The Impact of Time Scales;1240
11.1.3;3 Summary;1242
11.1.4;References;1242
11.2;On Scheduling in Distributed Transactional Memory: Techniques and Tradeoffs;1244
11.2.1;1 Introduction;1244
11.2.2;2 Preliminaries and System Model;1246
11.2.2.1;2.1 Distributed Transactions;1246
11.2.2.2;2.2 Definitions;1247
11.2.2.3;2.3 Transactional Scheduler;1247
11.2.3;3 Bi-interval;1248
11.2.3.1;3.1 Motivation;1248
11.2.3.2;3.2 Scheduler Design;1249
11.2.3.3;3.3 Analysis;1250
11.2.3.4;3.4 Evaluation;1252
11.2.4;4 Cluster-Based Transactional Scheduler;1253
11.2.4.1;4.1 Motivation;1253
11.2.4.2;4.2 Scheduler Design;1254
11.2.4.3;4.3 Analysis;1256
11.2.4.4;4.4 Evaluation;1257
11.2.5;5 Summary and Conclusion;1258
11.2.6;References;1259
11.3;Dependability-Oriented Resource Management Schemes for Cloud Computing Data Centers;1261
11.3.1;1 Introduction;1261
11.3.2;2 System Model and Failure Behavior of Data Center Components;1262
11.3.2.1;2.1 Overview of the Data Center Architecture;1262
11.3.2.2;2.2 Failure Behavior of Servers;1263
11.3.2.3;2.3 Failure Behavior of Network Components;1264
11.3.2.4;2.4 Analysis of the Impact of Failures on Applications;1265
11.3.3;3 Resource Management in Data Center Environments;1266
11.3.3.1;3.1 Global Constraints;1268
11.3.3.2;3.2 Infrastructure-Oriented Constraints;1269
11.3.3.3;3.3 Application-Oriented Constraints;1270
11.3.4;4 Initial Allocation of Virtual Machines in Data Center Environments;1271
11.3.4.1;4.1 A Comprehensive Scheme for Virtual Machines Allocation;1271
11.3.4.2;4.2 Other Schemes for Virtual Machines Allocation;1273
11.3.5;5 Runtime Adaption of Virtual Machine Allocation in Data Center Environments;1275
11.3.5.1;5.1 Runtime Adaption to Balance Availability and Performance;1276
11.3.5.2;5.2 Other Schemes for Runtime Virtual Machines Allocation Adaption;1277
11.3.6;6 Conclusions;1279
11.3.7;References;1279
11.4;Resource Scheduling in Data-Centric Systems;1282
11.4.1;1 Introduction;1282
11.4.2;2 Terminology;1284
11.4.3;3 Classification and State-of-the-Art;1285
11.4.3.1;3.1 Hierarchy of Resource Scheduling in DCS;1285
11.4.3.2;3.2 Resource Provision;1287
11.4.3.2.1;3.2.1 Economic-Based Resource Provision;1287
11.4.3.2.2;3.2.2 SLA-Oriented Resource Provision;1288
11.4.3.2.3;3.2.3 Utility-Oriented Resource Provision;1288
11.4.3.3;3.3 Job Scheduling;1289
11.4.3.3.1;3.3.1 Static Job Scheduling;1290
11.4.3.3.2;3.3.2 Dynamic Job Scheduling;1290
11.4.3.4;3.4 Data Scheduling;1292
11.4.3.4.1;3.4.1 Online Data Scheduling;1293
11.4.3.4.2;3.4.2 Offline Data Scheduling;1293
11.4.4;4 Case Studies;1294
11.4.4.1;4.1 Amazon EC2;1294
11.4.4.2;4.2 Dawning Nebulae;1295
11.4.4.3;4.3 Taobao Yunti;1296
11.4.4.4;4.4 Microsoft SCOPE;1297
11.4.5;5 Future Trends and Challenges;1298
11.4.6;6 Conclusions;1299
11.4.7;References;1300
12;Index;1306