Editing Single program, multiple data (section)

{{Short description|Computing technique used to achieve parallelism}}
{{redirect|SPMD|the membrane technology|Semipermeable membrane device}}
{{missing information|GPUs|date=November 2019}}
<!-- SPMD has some similarities with SIMD, but it is actually a form of MIMD do not redirect it to SIMD -->
In [[computing]], '''single program, multiple data''' ('''SPMD''')  is a term that has been used to refer to computational models for exploiting [[Parallel computing|parallelism]] whereby multiple processors cooperate in the execution of a program in order to obtain results faster.

The term SPMD was introduced in 1983 and was used to denote two different computational models:

# by Michel Auguin (University of Nice Sophia-Antipolis) and François Larbey (Thomson/Sintra),<ref name=":0">{{Cite journal |last=M. Auguin, F. Larbey |date=1983 |title=OPSILA: an advanced SIMD for numerical analysis and signal processing |journal=Microcomputers: Developments in Industry, Business, and Education / Ninth EUROMICRO Symposium on Microprocessing and Microprogramming, Pp 311-318 Madrid, September 13–16, 1983}}</ref><ref name=":1">{{Cite journal |last=M. Auguin, F. Labrey |date=1985 |title=A Multi-processor SIMD Machine: OPSILA |journal=K. Waldschmidt and B. Myhrhaug Eds, @EUROMICRO, 1985, Elsevier Science Publishers B. V. – North Holland}}</ref><ref name=":2">{{Cite journal |date=1987 |title=Experience Using a SIMD/SPMD Multiprocessor Architecture |journal=Multiprocessing and Microprogramming |volume=21 |issue=1–5 |pages=171–178|doi=10.1016/0165-6074(87)90034-2 |last1=Auguin |first1=M. |last2=Boeri |first2=F. |last3=Dalban |first3=J.P |last4=Vincent-Carrefour |first4=A. }}</ref> as a "''fork-and-join''" and data-parallel approach where the ''parallel tasks ("single program")'' are split-up and run simultaneously ''in lockstep'' on multiple ''SIMD processors'' with different inputs, and
# by Frederica Darema (IBM),<ref name=":3">{{Cite journal |last=Darema |first=Frederica |date=2001 |title=SPMD model: past, present and future, Recent Advances in Parallel Virtual Machine and Message Passing Interface |journal=8th European PVM/MPI Users' Group Meeting, Santorini/Thera, Greece, September 23–26, 2001. Lecture Notes in Computer Science 2131}}</ref><ref name=":4">{{Cite journal |last=F. Darema-Rogers, D. A. George, V. A. Norton, and G. F. Pfister |date=1985 |title=A VM Parallel Environment |journal=IBM/RC11225 (1/23/85) and IBM/RC11381(9/19/85)}}</ref><ref name=":5">{{Cite journal |title=A single-program-multiple-data computational model for EPEX/FORTRAN |journal=Journal of Parallel Computing |volume=7 |year=1988 |pages=11–24|doi=10.1016/0167-8191(88)90094-4 |last1=Darema |first1=F. |last2=George |first2=D.A. |last3=Norton |first3=V.A. |last4=Pfister |first4=G.F. }}</ref> where "''all (processors) [[Process (computing)|processes]]  begin executing the same program... but through synchronization directives ... self-schedule themselves  to execute different instructions and act on different data''"  and enabling MIMD parallelization of a given program, and is a more general approach than [[Data parallelism|data-parallel]] and more efficient than the fork-and-join for parallel execution on general purpose multiprocessors.

The (IBM) SPMD is the most common style of parallel programming and can be considered a subcategory of MIMD in that it refers to MIMD execution of a given ("single") program.<ref name=":6">{{Cite journal |last=Flynn |first=Michael |title=Some Computer Organizations and Their Effectiveness |url=https://www.cs.utah.edu/~hari/teaching/paralg/Flynn72.pdf |journal=IEEE Transactions on Computers |date=1972 |volume=C-21 |issue=9 |pages=948–960|doi=10.1109/TC.1972.5009071 |s2cid=18573685 }}</ref> It is also a prerequisite for research concepts such as [[active message]]s and [[distributed shared memory]].{{Flynn's Taxonomy}}