Dünnweber / Gorlatch Higher-Order Components for Grid Programming

Making Grids More Usable
1. Auflage 2009
ISBN: 978-3-642-00841-2
Verlag: Springer
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 186 Seiten

A major challenge in grid computing remains the application software development for this new kind of infrastructure. Grid application programmers have to take into account several complicated aspects: distribution of data and computations, parallel computations on different sites and processors, heterogeneity of the involved computers, load balancing, etc. Grid programmers thus demand novel programming methodologies that abstract over such technical details while preserving the beneficial features of modern grid middleware. For this purpose, the authors introduce Higher-Order Components (HOCs). HOCs implement generic parallel/distributed processing patterns, together with the required middleware support, and they are offered to users via a high-level service interface. Users only have to provide the application-specific pieces of their programs as parameters, while low-level implementation details, such as the transfer of data across the grid, are handled by the HOCs. HOCs were developed within the CoreGRID European Network of Excellence and have become an optional extension of the popular Globus middleware. The book provides the reader with hands-on experience, describing a broad collection of example applications from various fields of science and engineering, including biology, physics, etc. The Java code for these examples is provided online, complementing the book. The expected application performance is studied and reported for extensive performance experiments on different testbeds, including grids with worldwide distribution. The book is targeted at graduate students, advanced professionals, and researchers in both academia and industry. Readers can raise their level of knowledge about methodologies for programming contemporary parallel and distributed systems, and, furthermore, they can gain practical experience in using distributed software. Practical examples show how the complementary online material can easily be adopted in various new projects.


Jan Dünnweber has been working as a reasearcher in grid computing at the University of Münster since 2003. His work focuses on programming methodologies for grid platforms. Sergei Gorlatch is a professor in computer science at the University of Münster, Germany. His areas of expertise include high-performance computing, program calculi, as well as performance modeling, prediction and MPI programming.

Autoren/Hrsg.

Dünnweber, Jan

Gorlatch, Sergei

Inhaltsverzeichnis

1;Preface;5
2;Contents;11
3;Chapter 1 Introduction;14
3.1;Challenges of Grid Programming;15
3.2;The Role of Middleware for the Grid;18
3.3;Communication Technologies for Distributed Computing;20
3.3.1;Java Remote Method Invocation (RMI);20
3.3.2;Common Object Request Broker Architecture (CORBA);22
3.3.3;Containers for Components & Services;23
3.4;Shortcomings in State-Of-The-Art Grid Middleware;29
3.4.1;Responsibilities of the Middleware User;29
3.4.2;Requirements for the Software Components;30
4;Chapter 2 HOCs: Software Components for Grid Programming;31
4.1;Higher-Order Components (HOCs);32
4.1.1;Motivation for HOCs;33
4.1.2;Grid Programming Using HOCs;34
4.1.3;Introducing Code Mobility to the Middleware;36
4.1.4;Polymorphism and Type Checking for Code Parameters;38
4.1.5;First Application Case Study: Julia Sets;42
4.2;HOCs and Grid Middleware;43
4.2.1;An Analysis of the Requirements of the Grid Platform without Components;43
4.2.2;Bridging Middleware and Application with HOCs;48
4.2.3;Case Study Revisited: Using the Farm-HOC;50
4.2.4;Performance Experiments on a Wide-Area Testbed;52
4.2.5;HOCs and Hand-Written Code: A Performance Comparison;53
4.3;APIs for Grid Application Programming with HOCs;54
4.4;Adaptability of HOCs;63
4.4.1;Code Parameters for Adaptation;64
4.4.2;Case Study: From Farm to Wavefront;65
4.5;Discussion: Adaptation vs. AOP;72
5;Chapter 3 Higher-Order Component Service Architecture (HOC-SA);74
5.1;Service-Oriented Grid Programming Using the HOC-SA;75
5.1.1;How Code Mobility Works: HOC-SA Code Service & Remote Code Loader;77
5.1.2;Parameter Databases in the HOC-SA;80
5.2;HOCs and Web Services;82
5.2.1;Web Services;82
5.2.2;Components and Resources;84
5.2.3;The HOC-SA Component Repository;85
5.2.4;The HOC-SA Portal;86
5.3;A Comparison of the HOC-SA and Globus WS-GRAM;89
5.3.1;Grid Programming with WS-GRAM and the HOC-SA;90
5.3.2;Application Types for HOC-SA and WS-GRAM;91
5.3.3;Response Times: HOC-SA vs. WS-GRAM ;95
5.4;MPI, Skeletons and Web Services: Integrating Grid Technologies;98
5.4.1;A Gateway for Bridging between Web Services and MPI;99
5.4.2;Example: Discrete Wavelet Transform (DWT);101
5.4.3;Wavelet Transform in General;101
5.4.4;DWT for Image Processing;103
5.4.5;DWT on the Grid Using the Lifting-HOC;104
5.4.6;Portable Parameters for the Lifting-HOC;107
5.4.7;An Adaptation of the Lifting-HOC;108
5.4.8;Experimental Performance Evaluation;110
5.4.9;Discussion: Interoperability and Portable Code;111
5.5;A HOC-SA Based Map/Reduce Implementation;114
5.5.1;Cloud Computing Technologies for the HOC-SA;114
5.5.2;MapReduce and Hadoop;116
5.5.3;HOC-SA Features for Map/Reduce on the Grid;116
5.6;Summary of HOC-SA Features;118
6;Chapter 4 Applications of Higher-Order Components;120
6.1;Clayworks: A Collaborative Simulation Environment;121
6.1.1;The 3-tier Architecture of Clayworks;123
6.1.2;The Deformation-HOC for Parallel Simulations;126
6.2;Protein Sequence Analysis with HOCs;132
6.2.1;The Alignment Problem in Bioinformatics;132
6.2.2;Circular Permutations of DNA;133
6.2.3;The Alignment-HOC and its Code Parameters;134
6.2.4;Using an Alternative Traceback;137
6.2.5;Optimizations of the Alignment-HOC;137
6.2.6;Experiments with the Alignment-HOC;139
6.3;Conclusions from Using HOCs in Large-Scale applications;140
7;Chapter 5 HOCs with Embedded Scheduling and Loop Parallelization;142
7.1;User-Transparent Grid Scheduling;143
7.1.1;The KOALA Grid Scheduling Infrastructure;144
7.1.2;Extensions of KOALA for User-Transparent Scheduling;146
7.1.3;Integrating KOALA & HOC-SA;147
7.1.4;A HOC-Aware Scheduling Algorithm;149
7.1.5;HOC Scheduling Cost-Functions;149
7.1.6;Scheduling Large-Scale Applications;151
7.1.7;Experiments with HOCs and KOALA;152
7.1.8;Conclusions from the Scheduling Experiments;155
7.2;Parallelization of Code Parameters in HOCs;155
7.2.1;The Internal Compute Farm of the LooPo-HOC;156
7.2.2;Transforming Loop Nests into Task Graphs;157
7.2.3;Integrating Loop Parallelization with the Grid;159
7.2.4;Case Study: The SOR Equation System Solver;161
7.2.5;Experiments;163
7.3;Combining HOCs with Related technologies: ProActive, SOFA and the GCM;164
7.3.1;Combining HOCs with ProActive, the GCM and SOFA;166
7.3.2;Creation of Web Services Using ProActive;169
7.4;Discussion: HOCs and Different Tools for Distributed Computing;170
8;Chapter 6 Conclusions and Related Work;171
8.1;New Contributions;171
8.2;Related Work;172
8.2.1;The Real-Time Framework (RTF);173
8.2.2;A Survey of Related Component Models;174
8.2.3;The Skeleton Model;174
8.2.4;CCA: The Common Component Architecture;175
8.2.5;CCM: The CORBA Component Model;176
8.2.6;Java Servlets and JSPs;176
8.2.7;Enterprise Java Beans and .NET Components;176
8.2.8;The Web 2.0;178
8.2.9;The Semantic Web;178
8.3;Future Work;179
9;Bibliography;181
9.1;References;181
9.2;(a) Textbooks;181
9.3;(b) Research Papers;182
9.4;(c) Online References;187
10;Index;191