This textbook covers the new development in processor architecture and parallel hardware. It provides detailed descriptions of parallel programming techniques that are necessary for developing efficient programs for multicore processors as well as for parallel cluster systems and supercomputers.

The book is structured in three main parts, covering all areas of parallel computing: the architecture of parallel systems, parallel programming models and environments, and the implementation of efficient application algorithms. The emphasis lies on parallel programming techniques needed for different architectures. In particular, this third edition includes an extended update of the chapter on computer architecture and performance analysis taking new developments such as the aspect of energy consumption into consideration. The description of OpenMP has been extended and now also captures the task concept of OpenMP. The chapter on message-passing programming has been extended and updated to include new features of MPI such as extended reduction operations and non-blocking collective communication operations. The chapter on GPU programming also has been updated. All other chapters also have been revised carefully.

The main goal of this book is to present parallel programming techniques that can be used in many situations for many application areas and to enable the reader to develop correct and efficient parallel programs. Many example programs and exercises are provided to support this goal and to show how the techniques can be applied to further applications. The book can be used as a textbook for students as well as a reference book for professionals. The material of the book has been used for courses in parallel programming at different universities for many years.

Table of Contents

Chapter 1 Introduction

Chapter 2 Parallel Computer Architecture

Chapter 3 Parallel Programming Models

Chapter 4 Performance Analysis of Parallel Programs

Chapter 5 Message-Passing Programming

Chapter 6 Thread Programming

Chapter 7 General Purpose GPU Programming

Chapter 8 Algorithms for Systems of Linear Equations