All Parallel Collective Operations and Communication Avoidance

Algorithms and operations that organize communication among the processors of a parallel computation. It covers the collective patterns by which processors broadcast, gather and combine data, and algorithms restructured to move less data between processors or between levels of memory. An entry belongs here when its main concern is the cost of communication in parallel programs. It excludes protocols for ordering events and sharing resources among independent processes, which are covered by the wider Parallel and Distributed Algorithm subcategory and by the techniques placed under Concurrency and Distributed Systems Techniques.