1st Edition

Attaining High Performance Communications A Vertical Approach

Edited By Ada Gavrilovska Copyright 2010
    443 Pages 114 B/W Illustrations
    by Chapman & Hall

    Technological Advances and Problems of High Performance Communications
    An ecosystem of solutions along a stack of technology layers

    Cohesively collecting state-of-the-art contributions from leading researchers in industry, national laboratories, and academia, Attaining High Performance Communications: A Vertical Approach discusses various issues pertaining to high performance communications in a particular layer of a vertical stack. It explores efficient interconnection hardware, the architectural aspects of network adapters and their integration with processor cores, the design of scalable and robust high performance end-to-end communications services and protocols, and system services and tools for new multi-core environments.

    No single solution applied at one particular layer can help applications solve all performance-related issues with communication services. Instead, this book shows that a coordinated effort is needed among the layers. It covers many different types of technologies and layers across the stack, from the architectural features of the hardware, through the protocols and their implementation in operating system kernels, to the manner in which application services and middleware are using underlying platforms. The book also describes key developments in high-end platforms, high performance interconnection fabrics and communication libraries, and multi- and many-core systems.

    This volume addresses the challenges involved in emerging types of communications applications, platforms, and services. Examining each layer in the vertical stack, it illustrates how to eliminate bottlenecks and provide optimization opportunities.

    High Performance Interconnects for Massively Parallel Systems

    Introduction

    Performance

    Network Topology

    Network Features

    Future Directions

    Summary

    Commodity High Performance Interconnects

    Introduction

    Overview of Past Commodity Interconnects, Features, and Trends

    InfiniBand Architecture

    Existing InfiniBand Adapters and Switches

    Existing InfiniBand Software Stacks

    Designing High-End Systems with InfiniBand: Case Studies

    Current and Future Trends of InfiniBand

    Ethernet vs. EtherNOT

    Overview

    Introduction

    Background

    Ethernet vs. EtherNOT?

    Commercial Perspective

    Concluding Remarks

    System Impact of Integrated Interconnects

    Introduction

    Technology Trends

    Integrated Interconnects

    Case Study: Implementation of Global Address Spaces

    Future Trends and Expectations

    Network Interfaces for High Performance Computing

    Introduction

    Network Interface Design Issues

    Current Approaches to Network Interface Design Issues

    Research Directions

    Summary

    Network Programming Interfaces for High Performance Computing

    Introduction

    The Evolution of High Performance Computing (HPC) Network Programming Interfaces

    Low-Level Network Programming Interfaces

    Distinguishing Characteristics

    Supporting Message Passing Interface (MPI)

    Supporting Shared Memory (SHMEM) and Partitioned Global Address Space (PGAS)

    Portals 4.0

    High Performance IP-Based Transports

    Introduction

    Transmission Control Protocol (TCP)

    TCP Performance Tuning

    User Datagram Protocol (UDP)-Based Transport Protocols

    Stream Control Transmission Protocol (SCTP)

    Summary

    Remote Direct Memory Access and iWARP

    Introduction

    Remote Direct Memory Access (RDMA)

    iWARP

    Summary

    Accelerating Communication Services on Multi-Core Platforms

    Introduction

    The "Simple" Onload Approach

    Partitioned Communication Stacks

    Specialized Network Multi-Cores

    Toward Heterogeneous Multi-Cores

    Summary

    Virtualized I/O

    Introduction

    Split Device Driver Model

    Direct Device Access Model

    Opportunities and Tradeoffs

    Summary

    The Message Passing Interface

    Introduction

    MPI’s Layer in the Network Stack

    Threading and MPI

    Point-to-Point Communications

    Collective Operations

    Implementation Strategies

    Summary

    High Performance Event Communication

    Introduction

    Design Points

    The EVPath Architecture

    Performance Microbenchmarks

    Usage Scenarios

    Summary

    The Case of the Fast Financial Feed

    Introduction

    Market Data Processing Systems

    Performance Requirements

    The OPRA Case Study

    Summary

    Data-Movement Approaches for High Performance Computing Storage Systems

    Introduction

    Lustre

    Panasas

    Parallel Virtual File System 2 (PVFS2)

    Lightweight File Systems

    Other Microsoft Project Plan (MPP) File Systems

    Summary

    Network Simulation

    Introduction

    Discrete Event Simulation

    Maintaining the Event List

    Modeling Routers, Links, and End-Systems

    Modeling Network Packets

    Modeling the Network Applications

    Visualizing the Simulation

    Distributed Simulation

    Summary

    References

    Biography

    Ada Gavrilovska is a Research Scientist in the College of Computing and the Center for Experimental Research in Computer Systems (CERCS) at Georgia Tech.