Prabhat / Koziol High Performance Parallel I/O

1. Auflage 2014
ISBN: 978-1-4665-8235-4
Verlag: Taylor & Francis
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 434 Seiten

Reihe: Chapman & Hall/CRC Computational Science

Gain Critical Insight into the Parallel I/O Ecosystem

Parallel I/O is an integral component of modern high performance computing (HPC), especially in storing and processing very large datasets to facilitate scientific discovery. Revealing the state of the art in this field, High Performance Parallel I/O draws on insights from leading practitioners, researchers, software architects, developers, and scientists who shed light on the parallel I/O ecosystem.

The first part of the book explains how large-scale HPC facilities scope, configure, and operate systems, with an emphasis on choices of I/O hardware, middleware, and applications. The book then traverses up the I/O software stack. The second part covers the file system layer and the third part discusses middleware (such as MPIIO and PLFS) and user-facing libraries (such as Parallel-NetCDF, HDF5, ADIOS, and GLEAN).

Delving into real-world scientific applications that use the parallel I/O infrastructure, the fourth part presents case studies from particle-in-cell, stochastic, finite volume, and direct numerical simulations. The fifth part gives an overview of various profiling and benchmarking tools used by practitioners. The final part of the book addresses the implications of current trends in HPC on parallel I/O in the exascale world.

Zielgruppe

High performance computing researchers, computational scientists, and graduate students.

Autoren/Hrsg.

Prabhat,

Koziol, Quincey

Fachgebiete

• Mathematik | Informatik EDV | Informatik EDV & Informatik Allgemein

Inhaltsverzeichnis

Parallel I/O in Practice
Parallel I/O at HPC Facilities Galen Shipman
National Energy Research Scientific Computing Center Jason Hick
National Center for Supercomputing Applications William Kramer, Michelle Butler, Gregory Bauer, Kalyana Chadalavada, and Celso Mendes
Argonne Leadership Computing Facility William E. Allcock and Kevin Harms
Livermore Computing Center Richard Hedges and Blaise Barney
Los Alamos National Laboratory Gary Grider
Texas Advanced Computing Center Karl W. Schulz

File Systems
Lustre Eric Barton and Andreas Dilger
GPFS Dean Hildebrand and Frank Schmuck
OrangeFS Walt Ligon and Boyd Wilson
OneFS Nick Kirsch

I/O Libraries
I/O Libraries: Past, Present and Future Mike Folk
MPI-IO Wei-keng Liao and Rajeev Thakur
PLFS: Software-Defined Storage for HPC John Bent
Parallel-NetCDF Rob Latham
HDF5 Quincey Koziol, Russ Rew, Mark Howison, Prabhat, and Marc Poinot
ADIOS Norbert Podhorszki, Scott Klasky, Qing Liu, Yuan Tian, Manish Parashar, Karsten Schwan, Matthew Wolf, and Sriram Lakshminarasimhan
GLEAN Venkatram Vishwanath, Huy Bui, Mark Hereld, and Michael E. Papka

I/O Case Studies
Parallel I/O for a Trillion-Particle Plasma Physics Simulation Surendra Byna, Prabhat, Homa Karimabadi, and William Daughton
Stochastic Simulation Data Management Dimitris Servis
Silo: A General-Purpose API and Scientific Database Mark Miller
Scaling Up Parallel I/O in S3D to 100-K Cores with ADIOS Scott Klasky, Gary Liu, Hasan Abbasi, Norbert Podhorszki, Jackie Chen, and Hemanth Kolla
In-Transit Processing: Data Analysis Using Burst Buffers Christopher Mitchell, David Bonnie, and Jonathan Woodring

I/O Profiling Tools
Overview of I/O Benchmarking Katie Antypas and Yushu Yao
TAU Sameer Shende and Allen D. Malony
Integrated Performance Monitoring David Skinner
Darshan Philip Carns
Iota Mark Howison, Prabhat, and Surendra Byna

Future Trends
Parallel Computing Trends for the Coming Decade John Shalf
Storage Models: Past, Present, and Future Dries Kimpe and Robert Ross
Resilience Gary Grider and Nathan DeBardeleben
Multi/Many Core Ramon Nou, Toni Cortes, Stelios Mavridis, Yannis Sfakianakis, and Angelos Bilas
Storage Networks and Interconnects Parks Fields and Benjamin McClelland
Power Consumption Matthew L. Curry, H. Lee Ward, and David Martinez, Jill Gemmill, Jay Harris, Gary Grider, and Anna Maria Bailey

Index

Über Autor(innen)

Prabhat leads the NERSC Data and Analytics Services Group at Lawrence Berkeley National Laboratory. His main research interests include Big Data analytics, scientific data management, parallel I/O, HPC, and scientific visualization. He is also interested in atmospheric science and climate change.

Quincey Koziol is the director of core software development and HPC at The HDF Group, where he leads the HDF5 software project. His research interests include HPC, scientific data storage, and software engineering and management.