Editing Parallel computing (section)

==History==
{{broader|History of computing}}

[[File:ILLIAC 4 parallel computer.jpg|right|thumbnail|[[ILLIAC IV]], "the most infamous of supercomputers"<ref name="infamous"/>]]

The origins of true (MIMD) parallelism go back to [[Luigi Federico Menabrea]] and his ''Sketch of the [[Analytic Engine]] Invented by [[Charles Babbage]]''.<ref>[[Luigi Federico Menabrea|Menabrea, L. F.]] (1842). [http://www.fourmilab.ch/babbage/sketch.html ''Sketch of the Analytic Engine Invented by Charles Babbage'']. Bibliothèque Universelle de Genève. Retrieved on November 7, 2007.
quote: "when a long series of identical computations is to be performed, such as those required for the formation of numerical tables, the machine can be brought into play so as to give several results at the same time, which will greatly abridge the whole amount of the processes."
</ref><ref name=PH753>Patterson and Hennessy, p.&nbsp;753.</ref><ref>
R.W. Hockney, C.R. Jesshope. [https://books.google.com/books?id=6HcBQ67-Fb4C ''Parallel Computers 2: Architecture, Programming and Algorithms, Volume 2'']. 1988. p. 8 quote: "The earliest reference to parallelism in computer design is thought to be in General L. F. Menabrea's publication in… 1842, entitled ''Sketch of the Analytical Engine Invented by Charles Babbage''".
</ref>

In 1957, [[Compagnie des Machines Bull]] announced the first computer architecture specifically designed for parallelism, the [[Bull Gamma 60|Gamma 60]].<ref>{{Cite journal |last=Bataille |first=M. |date=1972-04-01 |title=Something old: the Gamma 60 the computer that was ahead of its time |url=https://dl.acm.org/doi/10.1145/641276.641278 |journal=ACM SIGARCH Computer Architecture News |volume=1 |issue=2 |pages=10–15 |doi=10.1145/641276.641278 |s2cid=34642285 |issn=0163-5964|url-access=subscription }}</ref> It utilized a [[Fork–join model|fork-join model]] and a "Program Distributor" to dispatch and collect data to and from independent processing units connected to a central memory.<ref>{{Cite web |title=Architecture Sketch of Bull Gamma 60 -- Mark Smotherman |url=http://www.feb-patrimoine.com/projet/gamma60/architecture_sketch_of_bull_gamma_60_jbourb_--_mark_smotherman.htm |access-date=2023-08-14 |website=www.feb-patrimoine.com}}</ref><ref>{{Cite web |last=Tumlin, Smotherman |date=2023-08-14 |title=An Evaluation of the Design of the Gamma 60 |url=https://db.aconit.org/dbaconit/medias.view.php?media=../dbmedia_0/pdf_10/10174.pdf&cotemedia=An%20Evaluation%20of%20the%20Design%20of%20the%20Gamma%2060.pdf&format=pdf |access-date=2023-08-14 |website=ACONIT Computer History Museum |publisher=Department of Computer Science, Clemson University}}</ref>

In April 1958, Stanley Gill (Ferranti) discussed parallel programming and the need for branching and waiting.<ref>"Parallel Programming", S. Gill, ''The Computer Journal'' Vol. 1 #1, pp2-10, British Computer Society, April 1958.</ref> Also in 1958, IBM researchers [[John Cocke (computer scientist)|John Cocke]] and [[Daniel Slotnick]] discussed the use of parallelism in numerical calculations for the first time.<ref name=G_Wilson>{{cite web | url = http://ei.cs.vt.edu/~history/Parallel.html | title = The History of the Development of Parallel Computing | access-date = 2008-01-08 | first = Gregory V. | last = Wilson | year = 1994|publisher=Virginia Tech/Norfolk State University, Interactive Learning with a Digital Library in Computer Science}}</ref> [[Burroughs Corporation]] introduced the D825 in 1962, a four-processor computer that accessed up to 16 memory modules through a [[crossbar switch]].<ref>{{cite web | url = http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=65878 | title = The Power of Parallelism | author = Anthes, Gry | access-date = 2008-01-08 | date = November 19, 2001 | work = [[Computerworld]] | url-status = dead | archive-url = https://web.archive.org/web/20080131205427/http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=65878 | archive-date = January 31, 2008 }}</ref> In 1967, Amdahl and Slotnick published a debate about the feasibility of parallel processing at American Federation of Information Processing Societies Conference.<ref name=G_Wilson/> It was during this debate that [[Amdahl's law]] was coined to define the limit of speed-up due to parallelism.

In 1969, [[Honeywell]] introduced its first [[Multics]] system, a symmetric multiprocessor system capable of running up to eight processors in parallel.<ref name=G_Wilson/> [[C.mmp]], a multi-processor project at [[Carnegie Mellon University]] in the 1970s, was among the first multiprocessors with more than a few processors. The first bus-connected multiprocessor with snooping caches was the Synapse N+1 in 1984.<ref name=PH753/>

SIMD parallel computers can be traced back to the 1970s. The motivation behind early SIMD computers was to amortize the [[Propagation delay|gate delay]] of the processor's [[control unit]] over multiple instructions.<ref>Patterson and Hennessy, p.&nbsp;749.</ref> In 1964, Slotnick had proposed building a massively parallel computer for the [[Lawrence Livermore National Laboratory]].<ref name=G_Wilson/> His design was funded by the [[US Air Force]], which was the earliest SIMD parallel-computing effort, [[ILLIAC IV]].<ref name=G_Wilson/> The key to its design was a fairly high parallelism, with up to 256&nbsp;processors, which allowed the machine to work on large datasets in what would later be known as [[vector processor|vector processing]]. However, ILLIAC IV was called "the most infamous of supercomputers", because the project was only one-fourth completed, but took 11&nbsp;years and cost almost four times the original estimate.<ref name="infamous">Patterson and Hennessy, pp.&nbsp;749–50: "Although successful in pushing several technologies useful in later projects, the ILLIAC IV failed as a computer. Costs escalated from the $8&nbsp;million estimated in 1966 to $31&nbsp;million by 1972, despite the construction of only a quarter of the planned machine&nbsp;. It was perhaps the most infamous of supercomputers. The project started in 1965 and ran its first real application in 1976."</ref> When it was finally ready to run its first real application in 1976, it was outperformed by existing commercial supercomputers such as the [[Cray-1]].