Editing Central processing unit (section)

====Instruction-level parallelism====
{{Main|Instruction-level parallelism}}
[[File:Fivestagespipeline.png|thumb|left|upright=1.5|Basic five-stage pipeline. In the best case scenario, this pipeline can sustain a completion rate of one instruction per clock cycle.]]

One of the simplest methods for increased parallelism is to begin the first steps of instruction fetching and decoding before the prior instruction finishes executing. This is a technique known as [[instruction pipelining]], and is used in almost all modern general-purpose CPUs. Pipelining allows multiple instructions to be executed at a time by breaking the execution pathway into discrete stages. This separation can be compared to an assembly line, in which an instruction is made more complete at each stage until it exits the execution pipeline and is retired.

Pipelining does, however, introduce the possibility for a situation where the result of the previous operation is needed to complete the next operation; a condition often termed data dependency conflict. Therefore, pipelined processors must check for these sorts of conditions and delay a portion of the pipeline if necessary. A pipelined processor can become very nearly scalar, inhibited only by pipeline stalls (an instruction spending more than one clock cycle in a stage).

[[File:Superscalarpipeline.svg|thumb|upright=1.5|A simple superscalar pipeline. By fetching and dispatching two instructions at a time, a maximum of two instructions per clock cycle can be completed.]]

Improvements in instruction pipelining led to further decreases in the idle time of CPU components. Designs that are said to be superscalar include a long instruction pipeline and multiple identical [[execution unit]]s, such as [[load–store unit]]s, [[arithmetic–logic unit]]s, [[floating-point unit]]s and [[address generation unit]]s.<ref>{{cite web |last=Huynh |first=Jack |year=2003 |title=The AMD Athlon XP Processor with 512KB L2 Cache |url=http://courses.ece.uiuc.edu/ece512/Papers/Athlon.pdf |url-status=dead |archive-url=https://web.archive.org/web/20071128061217/http://courses.ece.uiuc.edu/ece512/Papers/Athlon.pdf |archive-date=2007-11-28 |access-date=2007-10-06 |publisher=University of Illinois |pages=6–11 |publication-place=Urbana–Champaign, Illinois}}</ref> In a superscalar pipeline, instructions are read and passed to a dispatcher, which decides whether or not the instructions can be executed in parallel (simultaneously). If so, they are dispatched to execution units, resulting in their simultaneous execution. In general, the number of instructions that a superscalar CPU will complete in a cycle is dependent on the number of instructions it is able to dispatch simultaneously to execution units.

Most of the difficulty in the design of a superscalar CPU architecture lies in creating an effective dispatcher. The dispatcher needs to be able to quickly determine whether instructions can be executed in parallel, as well as dispatch them in such a way as to keep as many execution units busy as possible. This requires that the instruction pipeline is filled as often as possible and requires significant amounts of [[CPU cache]]. It also makes [[Hazard (computer architecture)|hazard]]-avoiding techniques like [[branch prediction]], [[speculative execution]], [[register renaming]], [[out-of-order execution]] and [[transactional memory]] crucial to maintaining high levels of performance. By attempting to predict which branch (or path) a conditional instruction will take, the CPU can minimize the number of times that the entire pipeline must wait until a conditional instruction is completed. Speculative execution often provides modest performance increases by executing portions of code that may not be needed after a conditional operation completes. Out-of-order execution somewhat rearranges the order in which instructions are executed to reduce delays due to data dependencies. Also in case of [[Single instruction, multiple data|single instruction stream, multiple data stream]], a case when a lot of data from the same type has to be processed, modern processors can disable parts of the pipeline so that when a single instruction is executed many times, the CPU skips the fetch and decode phases and thus greatly increases performance on certain occasions, especially in highly monotonous program engines such as video creation software and photo processing.

When a fraction of the CPU is superscalar, the part that is not suffers a performance penalty due to scheduling stalls. The Intel [[P5 (microarchitecture)|P5]] [[Pentium]] had two superscalar ALUs which could accept one instruction per clock cycle each, but its FPU could not. Thus the P5 was integer superscalar but not floating point superscalar. Intel's successor to the P5 architecture, [[P6 (microarchitecture)|P6]], added superscalar abilities to its floating-point features.

Simple pipelining and superscalar design increase a CPU's ILP by allowing it to execute instructions at rates surpassing one instruction per clock cycle. Most modern CPU designs are at least somewhat superscalar, and nearly all general purpose CPUs designed in the last decade are superscalar. In later years some of the emphasis in designing high-ILP computers has been moved out of the CPU's hardware and into its software interface, or [[instruction set architecture]] (ISA). The strategy of the [[very long instruction word]] (VLIW) causes some ILP to become implied directly by the software, reducing the CPU's work in boosting ILP and thereby reducing design complexity.