Editing Single program, multiple data (section)

==History==
The acronym SPMD for "Single-Program Multiple-Data" has been used to describe two different computational models for exploiting parallel computing, and this is due to both terms being natural extensions of Flynn's taxonomy.<ref name=":6" /> The two respective groups of researchers were unaware of each other's use of the term SPMD to independently describe different models of parallel programming.

The term SPMD was proposed first in 1983 by Michel Auguin (University of Nice Sophia-Antipolis) and François Larbey (Thomson/Sintra) in the context of the OPSILA parallel computer and in the context of a fork-and-join and data parallel computational model approach.<ref name=":0" /> This computer consisted of a master (controller processor) and SIMD processors (or vector processor mode as proposed by Flynn). In Auguin's SPMD model, the same (parallel) task ("''same program''") is executed on different (SIMD) processors ("''operating in lock-step mode''"<ref name=":0" />  acting on a part ("slice") of the data-vector. Specifically, their 1985 paper<ref name=":1" /> and others<ref name=":2" /><ref name=":0" /> stated:<blockquote>We consider the SPMD (Single Program, Multiple Data) operating mode. This mode allows simultaneous execution of the same task (one per processor) but prevents data exchange between processors. Data exchanges are only performed under SIMD mode by means of vector assignments. We assume synchronizations are summed-up to switchings (sic) between SIMD and SPMD operatings [sic] modes using global fork-join primitives.</blockquote>Starting around the same timeframe (in late 1983 – early 1984), the SPMD term was proposed by Frederica Darema (at IBM at that time, and part of the RP3 group) to define a different SPMD computational model that she proposed,<ref name=":5" /><ref name=":4" /><ref name=":3" /> as a programming model which in the intervening years has been applied to a wide range of general-purpose high-performance computers (including RP3 - the 512-processor IBM Research Parallel Processor Prototype) and has led to the current parallel computing standards. The (IBM) SPMD programming model assumes a multiplicity of processors which operate cooperatively, all executing the same program but can take different paths through the program based on parallelization directives embedded in the program:<ref name=":5" /><ref name=":4" /><ref name=":3" /><ref name=":7">{{Cite journal |last=Darema |first=Frederica |date=1987 |title=Applications Environment for the IBM Research Parallel Processor Prototype (RP3) |journal=IBMRC12627 (3/27/87) and in Proceedings of the 1st International Conference on Supercomputing (ICS'87), by Springer-Verlag (1987)}}</ref><ref name=":8">{{Cite journal |last=Darema |first=Frederica |date=1988 |title=Parallel Applications Development for Shared Memory Systems |journal=IBM/RC12229(1986) and in Parallel Systems and Computation, G. Paul and G. S. Almasi Eds, Elsevier Science Publishers B. V. (North Holland), 1988}}</ref><blockquote>All processes participating in the parallel computation are created at the beginning of the execution and remain in existence until the end ... [the processors/processes] execute different instructions and act on different data ... the job [(work)] to be done by each process is allocated dynamically ... [i.e. the processes] self-schedule themselves to execute different instructions and act on different data [thus self-assign themselves to cooperate in execution of serial and parallel tasks (as well as replicate tasks) in the program.]</blockquote>The notion ''[[Process (computing)|process]]'' generalized the term ''processor'' in the sense that multiple processes can execute on a processor (to for example exploit larger degrees of parallelism for more efficiency and load-balancing). The (IBM) SPMD model was proposed by Darema as an approach different and more efficient than the fork-and-join that was pursued by all others in the community at that time; it is also more general than just "data-parallel" computational model and can encompass fork-and-join (as a subcategory implementation). The original context of the (IBM) SPMD was the RP3 computer (the 512-prosessor IBM Research Parallel Processor Prototype), which supported general purpose computing, with both distributed and (logically) shared memory.<ref name=":7" /> The (IBM) SPMD model was implemented by Darema and IBM colleagues into the EPEX (Environment for Parallel Execution), one of the first prototype programming environments.<ref name=":5" /><ref name=":4" /><ref name=":3" /><ref name=":7" /><ref name=":8" /><ref>{{Cite journal |last=J. M. Stone, F. Darema-Rogers, V. A. Norton, G. F. Pfister |date=1985 |title=Introduction to the VM/EPEX Preprocessor and Reference |journal=IBM/RC11407(9/30/85) and IBM/RC11408 (9/30/85)}}</ref> The effectiveness of the (IBM) SPMD was demonstrated for a wide class of applications,<ref name=":7" /><ref name=":3" /> and was implemented in the IBM FORTRAN in 1988,<ref>{{Cite journal |date=1988 |title=IBM Parallel FORTRAN |journal=IBM Systems Journal |volume=27 |issue=4 |pages=416–435|doi=10.1147/sj.274.0416 |last1=Toomey |first1=L. J. |last2=Plachy |first2=E. C. |last3=Scarborough |first3=R. G. |last4=Sahulka |first4=R. J. |last5=Shaw |first5=J. F. |last6=Shannon |first6=A. W. }}</ref> the first vendor-product in parallel programming; and in [[Message Passing Interface|MPI]] (1991 and on), [[OpenMP]] (1997 and on), and other environments which have adopted and cite the (IBM) SPMD Computational Model.

By the late 1980s, there were many distributed computers with proprietary message passing libraries. The first SPMD standard was [[Parallel Virtual Machine|PVM]]. The current de facto standard is [[Message Passing Interface|MPI]].

The [[Cray]] parallel directives were a direct predecessor of [[OpenMP]].