Editing Non-uniform memory access

{{Short description|Computer memory design used in multiprocessing}}

[[File:HP Z820 motherboard.jpg|thumb|The motherboard of an [[HP Z|HP Z820]] workstation with two CPU sockets, each with their own set of eight [[DIMM]] slots surrounding the socket.]]
'''Non-uniform memory access''' ('''NUMA''') is a [[computer storage|computer memory]] design used in [[multiprocessing]], where the memory access time depends on the memory location relative to the processor. Under NUMA, a processor can access its own [[local memory]] faster than non-local memory (memory local to another processor or memory shared between processors).<ref>{{FOLDOC|Non-uniform+memory+access}}</ref> NUMA is beneficial for workloads with high memory [[locality of reference]] and low [[lock contention]], because a processor may operate on a subset of memory mostly or entirely within its own cache node, reducing traffic on the memory bus.<ref name="nyu-numa">{{Cite web
 | url = http://cs.nyu.edu/~lerner/spring10/projects/NUMA.pdf
 | title = Non-Uniform Memory Access (NUMA)
 | date = 2010-05-04
 | access-date = 2014-01-27
 | author1 = Nakul Manchanda
 | author2 = Karan Anand
 | publisher = New York University
 | archive-url = https://web.archive.org/web/20131228092942/http://www.cs.nyu.edu/~lerner/spring10/projects/NUMA.pdf
 | archive-date = 2013-12-28
 | url-status = dead
 }}</ref>

NUMA architectures logically follow in scaling from [[symmetric multiprocessing]] (SMP) architectures. They were developed commercially during the 1990s by [[Unisys]], [[Convex Computer]] (later [[Hewlett-Packard]]), [[Honeywell]] Information Systems Italy (HISI) (later [[Groupe Bull]]), [[Silicon Graphics]] (later [[Silicon Graphics International]]), [[Sequent Computer Systems]] (later [[IBM]]), [[Data General]] (later [[EMC Corporation|EMC]], now [[Dell Technologies]]), [[Digital Equipment Corporation|Digital]] (later [[Compaq]], then [[Hewlett-Packard|HP]], now [[Hewlett Packard Enterprise|HPE]]) and [[International Computers Limited|ICL]]. Techniques developed by these companies later featured in a variety of [[Unix-like]] [[operating system]]s, and to an extent in [[Windows NT]].

The first commercial implementation of a NUMA-based Unix system was{{where|date=January 2022}}{{when|date=March 2023}} the Symmetrical Multi Processing XPS-100 family of servers, designed by Dan Gielan of VAST Corporation for [[Honeywell Information Systems]] Italy.

== {{Anchor|Basic concept}}Overview ==
[[Image:NUMA.svg|right|300px|thumb|One possible architecture of a NUMA system. The processors connect to the bus or crossbar by connections of varying thickness/number. This shows that different CPUs have different access priorities to memory based on their relative location.]]

Modern CPUs operate considerably faster than the main memory they use. In the early days of computing and data processing, the CPU generally ran slower than its own memory. The performance lines of processors and memory crossed in the 1960s with the advent of the first [[supercomputer]]s. Since then, CPUs increasingly have found themselves "starved for data" and having to stall while waiting for data to arrive from memory (e.g. for Von-Neumann architecture-based computers, see [[Von Neumann architecture#Von Neumann bottleneck|Von Neumann bottleneck]]). Many supercomputer designs of the 1980s and 1990s focused on providing high-speed memory access as opposed to faster processors, allowing the computers to work on large data sets at speeds other systems could not approach.

Limiting the number of memory accesses provided the key to extracting high performance from a modern computer. For commodity processors, this meant installing an ever-increasing amount of high-speed [[cache memory]] and using increasingly sophisticated algorithms to avoid [[cache miss]]es. But the dramatic increase in size of the operating systems and of the applications run on them has generally overwhelmed these cache-processing improvements. Multi-processor systems without NUMA make the problem considerably worse. Now a system can starve several processors at the same time, notably because only one processor can access the computer's memory at a time.<ref>{{cite web
 | url = https://www.usenix.org/legacy/event/atc11/tech/final_files/Blagodurov.pdf
 | title = A Case for NUMA-aware Contention Management on Multicore Systems
 | date = 2011-05-02 | access-date = 2014-01-27
 | author1 = Sergey Blagodurov | author2 = Sergey Zhuravlev | author3 = Mohammad Dashti | author4 = Alexandra Fedorov
 | publisher = Simon Fraser University }}</ref>

NUMA attempts to address this problem by providing separate memory for each processor, avoiding the performance hit when several processors attempt to address the same memory. For problems involving spread data (common for [[Server (computing)|server]]s and similar applications), NUMA can improve the performance over a single shared memory by a factor of roughly the number of processors (or separate memory banks).<ref name="acm-zmajo">{{cite web
 | url = http://people.inf.ethz.ch/zmajo/publications/11-systor.pdf
 | title = Memory System Performance in a NUMA Multicore Multiprocessor
 | year = 2011
 | access-date = 2014-01-27
 | author1 = Zoltan Majo
 | author2 = Thomas R. Gross
 | publisher = ACM
 | archive-url = https://web.archive.org/web/20130612210800/http://people.inf.ethz.ch/zmajo/publications/11-systor.pdf
 | archive-date = 2013-06-12
 | url-status = dead
 }}</ref> Another approach to addressing this problem is the [[multi-channel memory architecture]], in which a linear increase in the number of memory channels increases the memory access concurrency linearly.<ref>{{cite web
 | publisher = Infineon Technologies North America and Kingston Technology
 | date = September 2003
 | url = http://www.kingston.com/newtech/MKF_520DDRwhitepaper.pdf
 | archive-url = https://web.archive.org/web/20110929024052/http://www.kingston.com/newtech/MKF_520DDRwhitepaper.pdf
 | title = Intel Dual-Channel DDR Memory Architecture White Paper
 | edition = Rev. 1.0 | format = PDF, 1021&nbsp;[[kilobyte|KB]]
 | access-date = 2007-09-06 | archive-date = 2011-09-29}}
</ref>

Of course, not all data ends up confined to a single task, which means that more than one processor may require the same data. To handle these cases, NUMA systems include additional hardware or software to move data between memory banks. This operation slows the processors attached to those banks, so the overall speed increase due to NUMA heavily depends on the nature of the running tasks.<ref name="acm-zmajo" />

== Implementations ==
[[Advanced Micro Devices|AMD]] implemented NUMA with its [[Opteron]] processor (2003), using [[HyperTransport]]. [[Intel]] announced NUMA compatibility for its x86 and [[Itanium]] servers in late 2007 with its [[Nehalem (microarchitecture)|Nehalem]] and [[Tukwila (processor)|Tukwila]] CPUs.<ref>Intel Corp. (2008). Intel QuickPath Architecture [White paper]. Retrieved from http://www.intel.com/pressroom/archive/reference/whitepaper_QuickPath.pdf</ref> Both Intel CPU families share a common [[chipset]]; the interconnection is called Intel [[Intel QuickPath Interconnect|QuickPath Interconnect]] (QPI), which provides extremely high bandwidth to enable high on-board scalability and was replaced by a new version called Intel [[Intel UltraPath Interconnect|UltraPath Interconnect]] with the release of [[Skylake microarchitecture|Skylake]] (2017).<ref>{{cite press release | url = https://www.intel.com/pressroom/archive/releases/2007/20070918corp_b.htm | title = Gelsinger Speaks To Intel And High-Tech Industry's Rapid Technology Cadence | date = September 18, 2007 | publisher = Intel Corporation | accessdate = March 29, 2025}}</ref>

=={{Anchor|CCNUMA}}Cache coherent NUMA (ccNUMA)==
[[Image:Hwloc.png|right|300px|thumb|Topology of a ccNUMA [[Bulldozer (microarchitecture)|Bulldozer]] server extracted using hwloc's lstopo tool.]]

{{details|Directory-based cache coherence}}

Nearly all CPU architectures use a small amount of very fast non-shared memory known as [[CPU cache|cache]] to exploit [[locality of reference]] in memory accesses. With NUMA, maintaining [[cache coherence]] across shared memory has a significant overhead. Although simpler to design and build, non-cache-coherent NUMA systems become prohibitively complex to program in the standard [[von Neumann architecture]] programming model.<ref>{{Cite web
 | url = http://www.slideshare.net/networksguy/ccnuma-cache-coherent-nonuniform-memory-access
 | title = ccNUMA: Cache Coherent Non-Uniform Memory Access
 | year = 2014 | access-date = 2014-01-27
 | publisher = slideshare.net
}}</ref>

Typically, ccNUMA uses inter-processor communication between cache controllers to keep a consistent memory image when more than one cache stores the same memory location. For this reason, ccNUMA may perform poorly when multiple processors attempt to access the same memory area in rapid succession.  Support for NUMA in [[operating system]]s attempts to reduce the frequency of this kind of access by allocating processors and memory in NUMA-friendly ways and by avoiding scheduling and locking algorithms that make NUMA-unfriendly accesses necessary.<ref>{{Cite web
 | url = http://www.cs.berkeley.edu/~kubitron/cs258/handouts/papers/p80-stenstrom.pdf
 | title = Comparative Performance Evaluation of Cache-Coherent NUMA and COMA Architectures
 | year = 2002 | access-date = 2014-01-27
 | author1 = Per Stenstromt | author2 = Truman Joe | author3 = Anoop Gupta
 | publisher = ACM }}</ref>

Alternatively, cache coherency protocols such as the [[MESIF protocol]] attempt to reduce the communication required to maintain cache coherency. [[Scalable Coherent Interface]] (SCI) is an [[IEEE]] standard defining a directory-based cache coherency protocol to avoid scalability limitations found in earlier multiprocessor systems. For example, SCI is used as the basis for the NumaConnect technology.<ref>{{Cite web |title= The Scalable Coherent Interface and Related Standards Projects |author= David B. Gustavson |publisher= [[Stanford Linear Accelerator Center]] |date= September 1991 |work= SLAC Publication 5656 |url= http://www.slac.stanford.edu/cgi-wrap/getdoc/slac-pub-5656.pdf |archive-url=https://ghostarchive.org/archive/20221009/http://www.slac.stanford.edu/cgi-wrap/getdoc/slac-pub-5656.pdf |archive-date=2022-10-09 |url-status=live |access-date= January 27, 2014 }}</ref><ref>{{cite web |url=http://www.numascale.com/numa_technology.html |title=The NumaChip enables cache coherent low cost shared memory |publisher=Numascale.com |access-date=2014-01-27 |archive-url=https://web.archive.org/web/20140122115025/http://www.numascale.com/numa_technology.html |archive-date=2014-01-22 |url-status=dead }}</ref>

==NUMA vs. cluster computing==
One can view NUMA as a tightly coupled form of [[cluster computing]].  The addition of [[virtual memory]] paging to a cluster architecture can allow the implementation of NUMA entirely in software. However, the inter-node latency of software-based NUMA remains several orders of magnitude greater (slower) than that of hardware-based NUMA.<ref name="nyu-numa" />

== Software support ==
Since NUMA largely influences memory access performance, certain software optimizations are needed to allow scheduling threads and processes close to their in-memory data.

* [[Microsoft]] [[Windows 7]] and [[Windows Server 2008 R2]] added support for NUMA architecture over 64 logical cores.<ref>[http://msdn.microsoft.com/en-us/library/windows/desktop/aa363804%28v=vs.85%29.aspx NUMA Support (MSDN)]</ref>
* [[Java 7]] added support for NUMA-aware memory allocator and [[Garbage collection (computer science)|garbage collector]].<ref>[http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#numa Java HotSpot Virtual Machine Performance Enhancements]</ref>
* [[Linux kernel]]:
**Version 2.5 provided a basic NUMA support,<ref>{{Cite web
 | url = http://lse.sourceforge.net/numa/
 | title = Linux Scalability Effort: NUMA Group Homepage
 | date = 2002-11-20 | access-date = 2014-02-06
 | website = SourceForge.net
}}</ref> which was further improved in subsequent kernel releases.
**Version 3.8 of the Linux kernel brought a new NUMA foundation that allowed development of more efficient NUMA policies in later kernel releases.<ref>{{Cite web
 | url = http://kernelnewbies.org/Linux_3.8#head-c16d4288b51f0b50fbf615657e81b0db643fa7a0
 | title = Linux kernel 3.8, Section 1.8. Automatic NUMA balancing
 | date = 2013-02-08 | access-date = 2014-02-06
 | website = kernelnewbies.org
}}</ref><ref>{{Cite web
 | url = https://lwn.net/Articles/524977/
 | title = NUMA in a hurry
 | date = 2012-11-14 | access-date = 2014-02-06
 | author = Jonathan Corbet | publisher = [[LWN.net]]
}}</ref> 
**Version 3.13 of the Linux kernel brought numerous policies that aim at putting a process near its memory, together with the handling of cases such as having [[memory page]]s shared between processes, or the use of transparent [[huge page]]s; new [[sysctl]] settings allow NUMA balancing to be enabled or disabled, as well as the configuration of various NUMA memory balancing parameters.<ref>{{Cite web
 | url = http://kernelnewbies.org/Linux_3.13#head-d29c7db2e73bc464eb67ed8de953d0bfc9841636
 | title = Linux kernel 3.13, Section 1.6. Improved performance in NUMA systems
 | date = 2014-01-19 | access-date = 2014-02-06
 | website = kernelnewbies.org
}}</ref><ref>{{Cite web
 | url = https://www.kernel.org/doc/Documentation/sysctl/kernel.txt
 | title = Linux kernel documentation: Documentation/sysctl/kernel.txt
 | access-date = 2014-02-06
 | publisher = [[kernel.org]]
}}</ref><ref>{{Cite web
 | url = https://lwn.net/Articles/568870/
 | title = NUMA scheduling progress
 | date = 2013-10-01 | access-date = 2014-02-06
 | author = Jonathan Corbet | publisher = [[LWN.net]]
}}</ref>
* [[OpenSolaris]] models NUMA architecture with lgroups.
* [[FreeBSD]] added support for NUMA architecture in version 9.0.<ref>{{Cite web|title=numa(4)|url=https://www.freebsd.org/cgi/man.cgi?numa(4)|access-date=2020-12-03|website=www.freebsd.org}}</ref>
*[[Silicon Graphics]] [[IRIX]] (discontinued as of 2021) support for ccNUMA architecture over 1240 CPU with Origin server series.

== Hardware support ==
As of 2011, ccNUMA systems are multiprocessor systems based on the [[AMD Opteron]] processor, which can be implemented without external logic, and the Intel [[Itanium Processor Family|Itanium processor]], which requires the chipset to support NUMA. Examples of ccNUMA-enabled chipsets are the SGI Shub (Super hub), the Intel E8870, the [[Hewlett-Packard|HP]] sx2000 (used in the Integrity and Superdome servers), and those found in NEC Itanium-based systems. Earlier ccNUMA systems such as those from [[Silicon Graphics]] were based on [[MIPS architecture|MIPS]] processors and the [[Digital Equipment Corporation|DEC]] [[Alpha 21364]] (EV7) processor.

==See also==
{{Div col|colwidth=25em}}
* [[Uniform memory access]] (UMA)
* [[Cache-only memory architecture]] (COMA)
* [[HiperDispatch]]
* [[Partitioned global address space]]
* [[Nodal architecture]]
* [[Scratchpad memory]] (SPM)
{{div col end}}

==References==
{{Reflist}}

== External links ==
{{Div col|colwidth=25em}}
* [http://lse.sourceforge.net/numa/faq/ NUMA FAQ]
* [https://web.archive.org/web/20060505025517/http://cs.gmu.edu/cne/modules/dsm/yellow/page_dsm.html Page-based distributed shared memory]
* [https://web.archive.org/web/20060924140915/http://opensolaris.org/os/community/performance/numa/ OpenSolaris NUMA Project]
* [https://web.archive.org/web/20040606042837/http://h18002.www1.hp.com/alphaserver/nextgen/overview.wmv Introduction video for the Alpha EV7 system architecture]
* [http://www.alphaprocessors.com/ More videos related to EV7 systems: CPU, IO, etc]
* [https://web.archive.org/web/20091108151203/http://developer.amd.com/pages/1162007106.aspx NUMA optimization in Windows Applications]
* [https://web.archive.org/web/20110128015336/http://oss.sgi.com/projects/numa/ NUMA Support in Linux at SGI]
* [http://www.realworldtech.com/page.cfm?NewsID=361&amp;date=05-05-2006#361/ Intel Tukwila]
* [http://www.realworldtech.com/page.cfm?ArticleID=RWT082807020032 Intel QPI (CSI) explained]
* [https://web.archive.org/web/20071103124627/http://www.sql-server-performance.com/articles/per/high_call_volume_NUMA_p1.aspx current Itanium NUMA systems]
{{div col end}}

{{Parallel Computing}}

[[Category:Parallel computing]]
[[Category:Computer memory]]