Collective operation

A collective operation is a concept in parallel computing in which data is simultaneously sent to or received from many nodes.

Common examples of collective operations are gather (in which data is collected from all nodes), scatter (in which a set of data is broken up into pieces, and a different piece is sent to all nodes), and broadcast (in which the same data is sent to all nodes).

Parallel computing frameworks with collective operations

Collective operations are present in the following parallel programming frameworks:

Google's MapReduce framework
MPI^[1]

References

↑ Intercommunicator Collective Operations. The Message Passing Interface (MPI) standard, chapter 7.3.1. Mathematics and Computer Science Division, Argonne National Laboratory.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Intercommunicator Collective Operations. The Message Passing Interface (MPI) standard, chapter 7.3.1. Mathematics and Computer Science Division, Argonne National Laboratory.