Editing Microarchitecture (section)

=== Multiprocessing and multithreading ===
{{Main|Multiprocessing|Multithreading (computer architecture)}}

Computer architects have become stymied by the growing mismatch in CPU operating frequencies and [[Dynamic random-access memory|DRAM]] access times. None of the techniques that exploited instruction-level parallelism (ILP) within one program could make up for the long stalls that occurred when data had to be fetched from main memory. Additionally, the large transistor counts and high operating frequencies needed for the more advanced ILP techniques required power dissipation levels that could no longer be cheaply cooled. For these reasons, newer generations of computers have started to exploit higher levels of parallelism that exist outside of a single program or [[thread (computing)|program thread]].

This trend is sometimes known as ''throughput computing''. This idea originated in the mainframe market where [[OLTP|online transaction processing]] emphasized not just the execution speed of one transaction, but the capacity to deal with massive numbers of transactions. With transaction-based applications such as network routing and web-site serving greatly increasing in the last decade, the computer industry has re-emphasized capacity and throughput issues.

One technique of how this parallelism is achieved is through [[multiprocessing]] systems, computer systems with multiple CPUs. Once reserved for high-end [[Mainframe computer|mainframes]] and [[supercomputer]]s, small-scale (2–8) multiprocessors servers have become commonplace for the small business market. For large corporations, large scale (16–256) multiprocessors are common. Even [[personal computer]]s with multiple CPUs have appeared since the 1990s.

With further transistor size reductions made available with semiconductor technology advances, [[Multi-core processor|multi-core CPUs]] have appeared where multiple CPUs are implemented on the same silicon chip. Initially used in chips targeting embedded markets, where simpler and smaller CPUs would allow multiple instantiations to fit on one piece of silicon. By 2005, semiconductor technology allowed dual high-end desktop CPUs ''CMP'' chips to be manufactured in volume. Some designs, such as [[Sun Microsystems]]' [[UltraSPARC T1]] have reverted to simpler (scalar, in-order) designs in order to fit more processors on one piece of silicon.

Another technique that has become more popular recently is [[Multithreading (computer architecture)|multithreading]]. In multithreading, when the processor has to fetch data from slow system memory, instead of stalling for the data to arrive, the processor switches to another program or program thread which is ready to execute. Though this does not speed up a particular program/thread, it increases the overall system throughput by reducing the time the CPU is idle.

Conceptually, multithreading is equivalent to a [[context switch]] at the operating system level. The difference is that a multithreaded CPU can do a thread switch in one CPU cycle instead of the hundreds or thousands of CPU cycles a context switch normally requires. This is achieved by replicating the state hardware (such as the [[register file]] and [[program counter]]) for each active thread.

A further enhancement is [[simultaneous multithreading]]. This technique allows superscalar CPUs to execute instructions from different programs/threads simultaneously in the same cycle.