Editing Single program, multiple data (section)

==SPMD vs SIMD==
[[File:SPMD Model.png|thumb|An example of "Single program, multiple data".]]
In SPMD parallel execution, multiple autonomous processors simultaneously execute the same program at independent points, rather than in the [[lockstep (computing)|lockstep]] that [[Single instruction, multiple data|SIMD]] or [[Single instruction, multiple threads|SIMT]] imposes on different data. With SPMD, tasks can be executed on general purpose [[CPU]]s. In SIMD the same operation (instruction) is applied on multiple data to manipulate data streams (a version of SIMD is [[vector processing]] where the data are organized as vectors). Another class of processors, [[Graphics processing unit|GPUs]] encompass multiple SIMD streams processing. SPMD and SIMD are not mutually exclusive; SPMD parallel execution can include SIMD, or vector, or GPU sub-processing. SPMD has been used for parallel programming of both message passing and shared-memory machine architectures.