Editing Central processing unit (section)

===Parallelism===
{{Main|Parallel computing}}
[[File:Nopipeline.png|thumb|upright=2|Model of a subscalar CPU, in which it takes fifteen clock cycles to complete three instructions]]

The description of the basic operation of a CPU offered in the previous section describes the simplest form that a CPU can take. This type of CPU, usually referred to as ''subscalar'', operates on and executes one instruction on one or two pieces of data at a time, that is less than one [[Instructions per cycle|instruction per clock cycle]] ({{nowrap|IPC < 1}}).

This process gives rise to an inherent inefficiency in subscalar CPUs. Since only one instruction is executed at a time, the entire CPU must wait for that instruction to complete before proceeding to the next instruction. As a result, the subscalar CPU gets "hung up" on instructions which take more than one clock cycle to complete execution. Even adding a second [[execution unit]] (see below) does not improve performance much; rather than one pathway being hung up, now two pathways are hung up and the number of unused transistors is increased. This design, wherein the CPU's execution resources can operate on only one instruction at a time, can only possibly reach ''scalar'' performance (one instruction per clock cycle, {{nowrap|1=IPC = 1}}). However, the performance is nearly always subscalar (less than one instruction per clock cycle, {{nowrap|IPC < 1}}).

Attempts to achieve scalar and better performance have resulted in a variety of design methodologies that cause the CPU to behave less linearly and more in parallel. When referring to parallelism in CPUs, two terms are generally used to classify these design techniques:
* ''[[instruction-level parallelism]]'' (ILP), which seeks to increase the rate at which instructions are executed within a CPU (that is, to increase the use of on-die execution resources);
* ''[[task-level parallelism]]'' (TLP), which purposes to increase the number of [[Thread (computing)|threads]] or [[Process (computing)|processes]] that a CPU can execute simultaneously.

Each methodology differs both in the ways in which they are implemented, as well as the relative effectiveness they afford in increasing the CPU's performance for an application.{{Efn|Neither [[Instruction-level parallelism|ILP]] nor [[Task-level parallelism|TLP]] is inherently superior over the other; they are simply different means by which to increase CPU parallelism. As such, they both have advantages and disadvantages, which are often determined by the type of software that the processor is intended to run. High-TLP CPUs are often used in applications that lend themselves well to being split up into numerous smaller applications, so-called "[[embarrassingly parallel]] problems". Frequently, a computational problem that can be solved quickly with high TLP design strategies like [[symmetric multiprocessing]] takes significantly more time on high ILP devices like superscalar CPUs, and vice versa.}}

====Instruction-level parallelism====
{{Main|Instruction-level parallelism}}
[[File:Fivestagespipeline.png|thumb|left|upright=1.5|Basic five-stage pipeline. In the best case scenario, this pipeline can sustain a completion rate of one instruction per clock cycle.]]

One of the simplest methods for increased parallelism is to begin the first steps of instruction fetching and decoding before the prior instruction finishes executing. This is a technique known as [[instruction pipelining]], and is used in almost all modern general-purpose CPUs. Pipelining allows multiple instructions to be executed at a time by breaking the execution pathway into discrete stages. This separation can be compared to an assembly line, in which an instruction is made more complete at each stage until it exits the execution pipeline and is retired.

Pipelining does, however, introduce the possibility for a situation where the result of the previous operation is needed to complete the next operation; a condition often termed data dependency conflict. Therefore, pipelined processors must check for these sorts of conditions and delay a portion of the pipeline if necessary. A pipelined processor can become very nearly scalar, inhibited only by pipeline stalls (an instruction spending more than one clock cycle in a stage).

[[File:Superscalarpipeline.svg|thumb|upright=1.5|A simple superscalar pipeline. By fetching and dispatching two instructions at a time, a maximum of two instructions per clock cycle can be completed.]]

Improvements in instruction pipelining led to further decreases in the idle time of CPU components. Designs that are said to be superscalar include a long instruction pipeline and multiple identical [[execution unit]]s, such as [[load–store unit]]s, [[arithmetic–logic unit]]s, [[floating-point unit]]s and [[address generation unit]]s.<ref>{{cite web |last=Huynh |first=Jack |year=2003 |title=The AMD Athlon XP Processor with 512KB L2 Cache |url=http://courses.ece.uiuc.edu/ece512/Papers/Athlon.pdf |url-status=dead |archive-url=https://web.archive.org/web/20071128061217/http://courses.ece.uiuc.edu/ece512/Papers/Athlon.pdf |archive-date=2007-11-28 |access-date=2007-10-06 |publisher=University of Illinois |pages=6–11 |publication-place=Urbana–Champaign, Illinois}}</ref> In a superscalar pipeline, instructions are read and passed to a dispatcher, which decides whether or not the instructions can be executed in parallel (simultaneously). If so, they are dispatched to execution units, resulting in their simultaneous execution. In general, the number of instructions that a superscalar CPU will complete in a cycle is dependent on the number of instructions it is able to dispatch simultaneously to execution units.

Most of the difficulty in the design of a superscalar CPU architecture lies in creating an effective dispatcher. The dispatcher needs to be able to quickly determine whether instructions can be executed in parallel, as well as dispatch them in such a way as to keep as many execution units busy as possible. This requires that the instruction pipeline is filled as often as possible and requires significant amounts of [[CPU cache]]. It also makes [[Hazard (computer architecture)|hazard]]-avoiding techniques like [[branch prediction]], [[speculative execution]], [[register renaming]], [[out-of-order execution]] and [[transactional memory]] crucial to maintaining high levels of performance. By attempting to predict which branch (or path) a conditional instruction will take, the CPU can minimize the number of times that the entire pipeline must wait until a conditional instruction is completed. Speculative execution often provides modest performance increases by executing portions of code that may not be needed after a conditional operation completes. Out-of-order execution somewhat rearranges the order in which instructions are executed to reduce delays due to data dependencies. Also in case of [[Single instruction, multiple data|single instruction stream, multiple data stream]], a case when a lot of data from the same type has to be processed, modern processors can disable parts of the pipeline so that when a single instruction is executed many times, the CPU skips the fetch and decode phases and thus greatly increases performance on certain occasions, especially in highly monotonous program engines such as video creation software and photo processing.

When a fraction of the CPU is superscalar, the part that is not suffers a performance penalty due to scheduling stalls. The Intel [[P5 (microarchitecture)|P5]] [[Pentium]] had two superscalar ALUs which could accept one instruction per clock cycle each, but its FPU could not. Thus the P5 was integer superscalar but not floating point superscalar. Intel's successor to the P5 architecture, [[P6 (microarchitecture)|P6]], added superscalar abilities to its floating-point features.

Simple pipelining and superscalar design increase a CPU's ILP by allowing it to execute instructions at rates surpassing one instruction per clock cycle. Most modern CPU designs are at least somewhat superscalar, and nearly all general purpose CPUs designed in the last decade are superscalar. In later years some of the emphasis in designing high-ILP computers has been moved out of the CPU's hardware and into its software interface, or [[instruction set architecture]] (ISA). The strategy of the [[very long instruction word]] (VLIW) causes some ILP to become implied directly by the software, reducing the CPU's work in boosting ILP and thereby reducing design complexity.

====Task-level parallelism====
{{Main|Multithreading (computer architecture)|l1=Multithreading|Multi-core processor}}
Another strategy of achieving performance is to execute multiple [[Thread (computing)|threads]] or [[Process (computing)|processes]] in parallel. This area of research is known as [[parallel computing]].<ref>{{cite book |last1=Gottlieb |first1=Allan |url=http://dl.acm.org/citation.cfm?id=160438 |title=Highly parallel computing |last2=Almasi |first2=George S. |publisher=Benjamin/Cummings |year=1989 |isbn=978-0-8053-0177-9 |location=Redwood City, California |language=en-us |access-date=2016-04-25 |archive-url=https://web.archive.org/web/20181107043726/https://dl.acm.org/citation.cfm?id=160438 |archive-date=2018-11-07 |url-status=live}}</ref> In [[Flynn's taxonomy]], this strategy is known as [[Multiple instruction, multiple data|multiple instruction stream, multiple data stream]] (MIMD).<ref>{{Cite journal|last1=Flynn|first1=M. J. |s2cid=18573685 |author-link1=Michael J. Flynn|doi=10.1109/TC.1972.5009071|title=Some Computer Organizations and Their Effectiveness|journal=[[IEEE Transactions on Computers]]|volume=C-21|issue=9| pages=948–960| date=September 1972}}</ref>

One technology used for this purpose is [[multiprocessing]] (MP).<ref>{{cite journal |last1=Lu |first1=N.-P. |last2=Chung |first2=C.-P. |year=1998 |title=Parallelism exploitation in superscalar multiprocessing  |journal=IEE Proceedings - Computers and Digital Techniques |volume=145 |issue=4 |pages=255 |doi=10.1049/ip-cdt:19981955|doi-broken-date=7 December 2024 }}</ref> The initial type of this technology is known as [[symmetric multiprocessing]] (SMP), where a small number of CPUs share a coherent view of their memory system. In this scheme, each CPU has additional hardware to maintain a constantly up-to-date view of memory. By avoiding stale views of memory, the CPUs can cooperate on the same program and programs can migrate from one CPU to another. To increase the number of cooperating CPUs beyond a handful, schemes such as [[non-uniform memory access]] (NUMA) and [[directory-based coherence protocols]] were introduced in the 1990s. SMP systems are limited to a small number of CPUs while NUMA systems have been built with thousands of processors. Initially, multiprocessing was built using multiple discrete CPUs and boards to implement the interconnect between the processors. When the processors and their interconnect are all implemented on a single chip, the technology is known as chip-level multiprocessing (CMP) and the single chip as a [[multi-core processor]].

It was later recognized that finer-grain parallelism existed with a single program. A single program might have several threads (or functions) that could be executed separately or in parallel. Some of the earliest examples of this technology implemented [[input/output]] processing such as [[direct memory access]] as a separate thread from the computation thread. A more general approach to this technology was introduced in the 1970s when systems were designed to run multiple computation threads in parallel. This technology is known as [[Multithreading (computer architecture)|multi-threading]] (MT). The approach is considered more cost-effective than multiprocessing, as only a small number of components within a CPU are replicated to support MT as opposed to the entire CPU in the case of MP. In MT, the execution units and the memory system including the caches are shared among multiple threads. The downside of MT is that the hardware support for multithreading is more visible to software than that of MP and thus supervisor software like operating systems have to undergo larger changes to support MT. One type of MT that was implemented is known as [[temporal multithreading]], where one thread is executed until it is stalled waiting for data to return from external memory. In this scheme, the CPU would then quickly context switch to another thread which is ready to run, the switch often done in one CPU clock cycle, such as the [[UltraSPARC T1]]. Another type of MT is [[simultaneous multithreading]], where instructions from multiple threads are executed in parallel within one CPU clock cycle.

For several decades from the 1970s to early 2000s, the focus in designing high performance general purpose CPUs was largely on achieving high ILP through technologies such as pipelining, caches, superscalar execution, out-of-order execution, etc. This trend culminated in large, power-hungry CPUs such as the Intel [[Pentium 4]]. By the early 2000s, CPU designers were thwarted from achieving higher performance from ILP techniques due to the growing disparity between CPU operating frequencies and main memory operating frequencies as well as escalating CPU power dissipation owing to more esoteric ILP techniques.

CPU designers then borrowed ideas from commercial computing markets such as [[transaction processing]], where the aggregate performance of multiple programs, also known as [[throughput]] computing, was more important than the performance of a single thread or process.

This reversal of emphasis is evidenced by the proliferation of dual and more core processor designs and notably, Intel's newer designs resembling its less superscalar [[P6 (microarchitecture)|P6]] architecture. Late designs in several processor families feature chip-level multiprocessing, including the [[x86-64]] [[Opteron]] and [[Athlon 64 X2]], the [[SPARC]] [[UltraSPARC T1]], IBM [[POWER4]] and [[POWER5]], as well as several [[video game console]] CPUs like the [[Xbox 360]]'s triple-core PowerPC design, and the [[PlayStation 3]]'s 7-core [[Cell (microprocessor)|Cell microprocessor]].

====Data parallelism====
{{Main|Vector processor|SIMD}}
A less common but increasingly important paradigm of processors (and indeed, computing in general) deals with data parallelism. The processors discussed earlier are all referred to as some type of scalar device.{{Efn|Earlier the term ''scalar'' was used to compare the IPC count afforded by various ILP methods. Here the term is used in the strictly mathematical sense to contrast with vectors. See [[scalar (mathematics)]] and [[vector (geometric)]].}} As the name implies, vector processors deal with multiple pieces of data in the context of one instruction. This contrasts with scalar processors, which deal with one piece of data for every instruction. Using [[Flynn's taxonomy]], these two schemes of dealing with data are generally referred to as ''single instruction'' stream, ''multiple data'' stream ([[Single instruction, multiple data|SIMD]]) and ''single instruction'' stream, ''single data'' stream ([[Single instruction, single data|SISD]]), respectively. The great utility in creating processors that deal with vectors of data lies in optimizing tasks that tend to require the same operation (for example, a sum or a [[dot product]]) to be performed on a large set of data. Some classic examples of these types of tasks include [[multimedia]] applications (images, video and sound), as well as many types of [[Scientific computing|scientific]] and engineering tasks. Whereas a scalar processor must complete the entire process of fetching, decoding and executing each instruction and value in a set of data, a vector processor can perform a single operation on a comparatively large set of data with one instruction. This is only possible when the application tends to require many steps which apply one operation to a large set of data.

Most early vector processors, such as the [[Cray-1]], were associated almost exclusively with scientific research and [[cryptography]] applications. However, as multimedia has largely shifted to digital media, the need for some form of SIMD in general-purpose processors has become significant. Shortly after inclusion of [[floating-point unit]]s started to become commonplace in general-purpose processors, specifications for and implementations of SIMD execution units also began to appear for general-purpose processors in the mid-1990s. Some of these early SIMD specifications – like HP's [[Multimedia Acceleration eXtensions]] (MAX) and Intel's [[MMX (instruction set)|MMX]] – were integer-only. This proved to be a significant impediment for some software developers, since many of the applications that benefit from SIMD primarily deal with [[floating-point arithmetic|floating-point]] numbers. Progressively, developers refined and remade these early designs into some of the common modern SIMD specifications, which are usually associated with one [[instruction set architecture]] (ISA). Some notable modern examples include Intel's [[Streaming SIMD Extensions]] (SSE) and the PowerPC-related [[AltiVec]] (also known as VMX).{{Efn|Although SSE/SSE2/SSE3 have superseded MMX in Intel's general-purpose processors, later [[IA-32]] designs still support MMX. This is usually done by providing most of the MMX functionality with the same hardware that supports the much more expansive SSE instruction sets.}}