Editing X86 virtualization

{{Short description|Hardware-assisted virtualization on x86/x86-64 CPUs}}
{{Lowercase title}}
'''x86 virtualization''' is the use of [[Hardware virtualization|hardware-assisted virtualization]] capabilities on an [[x86]]/[[x86-64]] [[Central processing unit|CPU]].

In the late 1990s x86 virtualization was achieved by complex [[software]] techniques, necessary to compensate for the processor's lack of hardware-assisted virtualization capabilities while attaining reasonable [[Computer performance|performance]]. In 2005 and 2006, both [[Intel]] ([[#Intel virtualization (VT-x)|VT-x]]) and [[Advanced Micro Devices|AMD]] ([[#AMD virtualization (AMD-V)|AMD-V]]) introduced limited hardware virtualization support that allowed simpler virtualization software but offered very few speed benefits.<ref>[https://web.stanford.edu/class/cs240/readings/hwsw.pdf A Comparison of Software and Hardware Techniques for x86 Virtualization, Keith Adams and Ole Agesen, VMware, ASPLOS’06 October 21–25, 2006, San Jose, California, USA] {{webarchive |url= https://web.archive.org/web/20221024045801/https://web.stanford.edu/class/cs240/readings/hwsw.pdf |date=2022-10-24}} "Surprisingly, we find that the first-generation hardware support rarely offers performance advantages over existing software techniques. We ascribe this situation to high VMM/guest transition costs and a rigid programming model that leaves little room for software flexibility in managing either the frequency or cost of these transitions."</ref> Greater hardware support, which allowed substantial speed improvements, came with later processor models.

== {{Anchor|SWBASED}}Software-based virtualization ==
The following discussion focuses only on virtualization of the x86 architecture [[protected mode]].

In protected mode the operating system kernel runs at a higher privilege such as [[Protection ring|ring]] 0, and applications at a lower privilege such as ring 3.{{Citation needed|date=January 2019}} In software-based virtualization, a host OS has direct access to hardware while the guest OSs have limited access to hardware, just like any other application of the host OS. One approach used in x86 software-based virtualization to overcome this limitation is called ''ring deprivileging'', which involves running the guest OS at a ring higher (lesser privileged) than 0.<ref>{{cite web |url=http://software.intel.com/sites/default/files/m/0/2/1/b/b/1024-Virtualization.pdf |title=Intel Virtualization Technology Processor Virtualization Extensions and Intel Trusted execution Technology |publisher=Intel.com |date=2007 |access-date=2016-12-12 |url-status=live |archive-url=https://web.archive.org/web/20150521014611/https://software.intel.com/sites/default/files/m/0/2/1/b/b/1024-Virtualization.pdf |archive-date=2015-05-21}}</ref>

Three techniques made virtualization of protected mode possible:
* [[Binary translation]] is used to rewrite certain ring 0 instructions in terms of ring 3 instructions, such as [[Interrupt flag#Setting and clearing|POPF]], that would otherwise fail silently or behave differently when executed above ring 0,<ref>{{cite web |url=http://www.usenix.org/events/sec2000/robin.html |title=USENIX Technical Program - Abstract - Security Symposium - 2000 |publisher=Usenix.org |date=2002-01-29 |access-date=2010-05-02 |url-status=live |archive-url=https://web.archive.org/web/20100610064709/http://www.usenix.org/events/sec2000/robin.html |archive-date=2010-06-10}}</ref><ref name=asplos/>{{rp|3}} making the classic [[Hardware-assisted virtualization|trap-and-emulate virtualization]] impossible.<ref name="asplos">{{cite web |title=A Comparison of Software and Hardware Techniques for x86 Virtualization |url=http://www.vmware.com/pdf/asplos235_adams.pdf |publisher=VMware |access-date=8 September 2010 |url-status=live |archive-url=https://web.archive.org/web/20100820201944/http://www.vmware.com/pdf/asplos235_adams.pdf |archive-date=20 August 2010}}</ref>{{rp|1}}<ref name=pat242>{{US patent|6397242}}</ref> To improve performance, the translated [[basic block]]s need to be cached in a coherent way that detects [[code patching]] (used in [[VxD]]s for instance), the reuse of pages by the guest OS, or even [[self-modifying code]].<ref>{{US patent|6704925}}</ref>
* A number of key data structures used by a processor need to be [[Shadow memory|shadowed]]. Because most operating systems use [[paged virtual memory]], and granting the guest OS direct access to the [[memory management unit|MMU]] would mean loss of control by the [[hypervisor|virtualization manager]], some of the work of the x86 MMU needs to be duplicated in software for the guest OS using a technique known as ''shadow page tables''.<ref>{{cite web |url=http://www.vmware.com/pdf/virtualization_considerations.pdf |title=Virtualization: architectural considerations and other evaluation criteria |publisher=VMware |access-date=8 September 2010 |url-status=live |archive-url=https://web.archive.org/web/20110206114316/http://www.vmware.com/pdf/virtualization_considerations.pdf |archive-date=6 February 2011}}</ref>{{rp|5}}<ref name=asplos/>{{rp|2}} This involves denying the guest OS any access to the actual page table entries by trapping access attempts and emulating them instead in software. The x86 architecture uses hidden state to store [[segment descriptor]]s in the processor, so once the segment descriptors have been loaded into the processor, the memory from which they have been loaded may be overwritten and there is no way to get the descriptors back from the processor. ''Shadow descriptor tables'' must therefore be used to track changes made to the descriptor tables by the guest OS.<ref name=pat242/>
* I/O device emulation: Unsupported devices on the guest OS must be emulated by a [[Emulator|device emulator]] that runs in the host OS.<ref name=pat847/>

These techniques incur some performance overhead due to lack of MMU virtualization support, as compared to a VM running on a natively virtualizable architecture such as the [[IBM System/370]].<ref name=asplos/>{{rp|10}}<ref name=tac9>{{cite web |url=http://download3.vmware.com/vmworld/2006/tac9463.pdf |title=VMware and Hardware Assist Technology |access-date=2010-09-08 |url-status=live |archive-url=https://web.archive.org/web/20110717231322/http://download3.vmware.com/vmworld/2006/tac9463.pdf |archive-date=2011-07-17}}</ref>{{rp|17 and 21}}

On traditional mainframes, the classic [[Hypervisor#Classification|type 1]] hypervisor was self-standing and did not depend on any operating system or run any user applications itself. In contrast, the first x86 virtualization products were aimed at workstation computers, and ran a guest OS inside a host OS by embedding the hypervisor in a kernel module that ran under the host OS (type 2 hypervisor).<ref name=pat847>{{US patent|6496847}}</ref>

There has been some controversy whether the x86 architecture with no hardware assistance is virtualizable as described by [[Popek and Goldberg virtualization requirements|Popek and Goldberg]]. [[VMware]] researchers pointed out in a 2006 [[International Conference on Architectural Support for Programming Languages and Operating Systems|ASPLOS]] paper that the above techniques made the x86 platform virtualizable in the sense of meeting the three criteria of Popek and Goldberg, albeit not by the classic trap-and-emulate technique.<ref name=asplos/>{{rp|2–3}}

A different route was taken by other systems like [[Denali (operating system)|Denali]], [[L4 microkernel family|L4]], and [[Xen]], known as [[paravirtualization]], which involves [[porting]] operating systems to run on the resulting virtual machine, which does not implement the parts of the actual x86 instruction set that are hard to virtualize. The paravirtualized I/O has significant performance benefits as demonstrated in the original [[Symposium on Operating Systems Principles|SOSP]]'03 Xen paper.<ref>{{cite web |url=http://www.cl.cam.ac.uk/research/srg/netos/papers/2003-xensosp.pdf |title=Xen and the Art of Virtualization |url-status=live |archive-url=https://web.archive.org/web/20140929081638/http://www.cl.cam.ac.uk/research/srg/netos/papers/2003-xensosp.pdf |archive-date=2014-09-29}}</ref>

The initial version of [[x86-64]] ([[X86-64#AMD64|AMD64]]) did not allow for a software-only full virtualization due to the lack of segmentation support in [[long mode]], which made the protection of the hypervisor's memory impossible, in particular, the protection of the trap handler that runs in the guest kernel address space.<ref>{{cite web |url=http://www.pagetable.com/?p=25 |title=How retiring segmentation in AMD64 long mode broke VMware |publisher=Pagetable.com |date=2006-11-09 |access-date=2010-05-02 |url-status=live |archive-url=https://web.archive.org/web/20110718104331/http://www.pagetable.com/?p=25 |archive-date=2011-07-18}}</ref><ref>{{cite web |url=http://download3.vmware.com/vmworld/2005/pac346.pdf |title=VMware and CPU Virtualization Technology |publisher=VMware |access-date=2010-09-08 |url-status=live |archive-url=https://web.archive.org/web/20110717231306/http://download3.vmware.com/vmworld/2005/pac346.pdf |archive-date=2011-07-17}}</ref>{{rp|11 and 20}} Revision D and later 64-bit AMD processors (as a rule of thumb, those manufactured in 90&nbsp;nm or less) added basic support for segmentation in long mode, making it possible to run 64-bit guests in 64-bit hosts via binary translation. Intel did not add segmentation support to its x86-64 implementation ([[Intel 64]]), making 64-bit software-only virtualization impossible on Intel CPUs, but Intel VT-x support makes 64-bit hardware assisted virtualization possible on the Intel platform.<ref>{{cite web |url=http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003945 |title=VMware KB: Hardware and firmware requirements for 64bit guest operating systems |publisher=Kb.vmware.com |access-date=2010-05-02 |url-status=live |archive-url=https://web.archive.org/web/20100419032716/http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003945 |archive-date=2010-04-19}}</ref><ref>{{cite web |url=http://www.vmware.com/files/pdf/software_hardware_tech_x86_virt.pdf |title=Software and Hardware Techniques for x86 Virtualization |access-date=2010-05-02 |url-status=dead |archive-url=https://web.archive.org/web/20100105124056/http://www.vmware.com/files/pdf/software_hardware_tech_x86_virt.pdf |archive-date=2010-01-05}}</ref>{{rp|4}}

On some platforms, it is possible to run a 64-bit guest on a 32-bit host OS if the underlying processor is 64-bit and supports the necessary virtualization extensions.

== {{Anchor|Hardware support}}Hardware-assisted virtualization ==
{{Main|Hardware-assisted virtualization}}

In 2005 and 2006, [[Intel]] and [[Advanced Micro Devices|AMD]] (working independently) created new [[x86#Extensions|processor extensions]] to the x86 architecture. The first generation of x86 hardware virtualization addressed the issue of privileged instructions. The issue of low performance of virtualized system memory was addressed with [[Memory management unit|MMU]] virtualization that was added to the chipset later.

=== Central processing unit ===

==== Virtual 8086 mode ====
{{Main|Virtual 8086 mode}}

Because the [[Intel 80286]] could not run concurrent DOS applications well by itself in protected mode, Intel introduced the [[virtual 8086 mode]] in their [[Intel 80386|80386]] chip, which offered virtualized 8086 processors on the 386 and later chips. Hardware support for virtualizing the protected mode itself, however, became available 20 years later.<ref>{{cite web |last=Yager |first=Tom |url=http://www.infoworld.com/article/2664741/computer-hardware/sending-software-to-do-hardware-s-job.html |title=Sending software to do hardware's job &#124; Hardware - InfoWorld |publisher=Images.infoworld.com |date=2004-11-05 |access-date=2014-01-08 |url-status=live |archive-url=https://web.archive.org/web/20141018133427/http://www.infoworld.com/article/2664741/computer-hardware/sending-software-to-do-hardware-s-job.html |archive-date=2014-10-18}}</ref>

==== {{Anchor|AMD-V}}AMD virtualization (AMD-V) ====
[[File:AMD Phenom die equalized.png|thumb|[[AMD Phenom]] die]]

AMD developed its first generation virtualization extensions under the code name "Pacifica", and initially published them as AMD Secure Virtual Machine (SVM),<ref>{{cite web |url=http://www.mimuw.edu.pl/~vincent/lecture6/sources/amd-pacifica-specification.pdf |title=33047_SecureVirtualMachineManual_3-0.book |access-date=2010-05-02 |url-status=live |archive-url=https://web.archive.org/web/20120305061511/http://www.mimuw.edu.pl/~vincent/lecture6/sources/amd-pacifica-specification.pdf |archive-date=2012-03-05}}</ref> but later marketed them under the trademark ''AMD Virtualization'', abbreviated ''AMD-V''.

On May 23, 2006, AMD released the Athlon 64 ([[List of AMD Athlon 64 microprocessors#"Orleans" (F2 & F3, 90 nm)|"Orleans"]]), the Athlon 64 X2 ([[List of AMD Athlon 64 microprocessors#"Windsor" (F2 & F3, 90 nm)|"Windsor"]]) and the Athlon 64 FX ([[List of AMD Athlon 64 microprocessors#"Windsor" (F2, 90 nm)|"Windsor"]]) as the first AMD processors to support this technology.

AMD-V capability also features on the [[Athlon 64]] and [[Athlon 64 X2]] family of processors with revisions "F" or "G" on [[socket AM2]], [[AMD Turion#Turion 64 X2|Turion 64 X2]], and [[Opteron]] 2nd generation<ref>{{cite web |url=http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8806~111165,00.html#111166 |title=What are the main differences between Second-Generation AMD Opteron processors and first-generation AMD Opteron processors? |website=amd.com |access-date=2012-02-04 |url-status=dead |archive-url=https://web.archive.org/web/20090415210555/http://www.amd.com/us-en/Processors/ProductInformation/0%2C%2C30_118_8796_8806~111165%2C00.html#111166 |archive-date=April 15, 2009}}</ref> and third-generation,<ref>{{cite web |url=http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8806~119722,00.html#119726 |title=What virtualization enhancements do Quad-Core AMD Opteron processors feature? |website=amd.com |access-date=2012-02-04 |url-status=dead |archive-url=https://web.archive.org/web/20090416073603/http://www.amd.com/us-en/Processors/ProductInformation/0%2C%2C30_118_8796_8806~119722%2C00.html#119726 |archive-date=April 16, 2009}}</ref> [[AMD Phenom|Phenom]] and [[Phenom II]] processors. The [[AMD Accelerated Processing Unit|APU Fusion]] processors support AMD-V. AMD-V is not supported by any Socket 939 processors. The only [[Sempron]] processors which support it are APUs and [[List of AMD Sempron microprocessors#"Huron" (65&nbsp;nm, Low power)|Huron]], [[List of AMD Sempron microprocessors#"Regor" (Socket AM3, Dual-core, C3, 45 nm)|Regor]], [[List of AMD Sempron microprocessors#"Sargas" (Socket AM3, Single-core, C2 & C3, 45&nbsp;nm)|Sargas]] desktop CPUs.

AMD Opteron CPUs beginning with the Family 0x10 Barcelona line, and Phenom II CPUs, support a second generation hardware virtualization technology called [[Rapid Virtualization Indexing]] (formerly known as Nested Page Tables during its development), later adopted by Intel as [[Extended Page Table]]s (EPT).

As of 2019, all [[Zen (microarchitecture)|Zen]]-based AMD processors support AMD-V.

The [[CPU flag (x86)|CPU flag]] for AMD-V is "svm". This may be checked in [[Comparison of BSD operating systems|BSD derivatives]] via [[dmesg]] or [[sysctl]] and in [[Linux]] via <code>/proc/[[cpuinfo]]</code>.<ref name=cpuflag/> Instructions in AMD-V include VMRUN, VMLOAD, VMSAVE, CLGI, VMMCALL, INVLPGA, SKINIT, and STGI.

With some [[motherboard]]s, users must enable AMD SVM feature in the [[BIOS]] setup before applications can make use of it.<ref>{{Cite web |title=How to enable Intel VTx and AMD SVM? |website=Support |publisher=QNAP Systems, Inc. |url=https://www.qnap.com/en/how-to/faq/article/how-to-enable-intel-vtx-and-amd-svm|access-date=2020-12-23|archive-url=https://web.archive.org/web/20180307204640/https://www.qnap.com/en/how-to/faq/article/how-to-enable-intel-vtx-and-amd-svm |archive-date=2018-03-07 |url-status=dead}}</ref>

==== {{Anchor|Intel-VT-x|VMCS-SHADOWING}}Intel virtualization (VT-x) ====
{{Redirect|Intel VT-x|the Itanium virtualization extensions|Intel VT-i}}

<!--
  IOMMU stuff ("VT-d") belongs in [[IOMMU]] page or section
-->
[[File:Intel Core i7-940 bottom.jpg|thumb|right|[[List of Intel Core i7 microprocessors#"Bloomfield" (45 nm)|Intel Core i7]] (Bloomfield) CPU]]

Previously codenamed "Vanderpool", VT-x represents Intel's technology for virtualization on the x86 platform. On November 14, 2005, Intel released two models of [[Pentium 4]] (Model 662 and 672) as the first Intel processors to support VT-x. The CPU flag for VT-x capability is "vmx"; in Linux, this can be checked via <code>/proc/cpuinfo</code>, or in [[macOS]] via <code>sysctl machdep.cpu.features</code>.<ref name=cpuflag>[http://software.intel.com/en-us/blogs/2012/03/12/how-to-start-intel-hardware-assisted-virtualization-hypervisor-on-linux-to-speed-up-intel-android-x86-gingerbread-emulator To see if your processor supports hardware virtualization] {{webarchive |url=https://web.archive.org/web/20121125081532/http://software.intel.com/en-us/blogs/2012/03/12/how-to-start-intel-hardware-assisted-virtualization-hypervisor-on-linux-to-speed-up-intel-android-x86-gingerbread-emulator/ |date=2012-11-25}} Intel 2012.</ref><ref>https://www.cpu-world.com/CPUs/Pentium_4/Intel-Pentium%204%20662%203.6%20GHz%20-%20HH80547PG1042MH.html</ref><ref>https://www.cpu-world.com/CPUs/Pentium_4/Intel-Pentium%204%20672%203.8%20GHz%20-%20HH80547PG1122MH.html</ref>

"VMX" stands for Virtual Machine Extensions, which adds 13 new instructions: VMPTRLD, VMPTRST, VMCLEAR, VMREAD, VMWRITE, VMCALL, VMLAUNCH, VMRESUME, VMXOFF, VMXON, INVEPT, INVVPID, and VMFUNC.<ref>
{{cite web |url=http://software.intel.com/en-us/articles/intel-sdm |title=Intel® 64 and IA-32 Architectures Software Developer's Manual |last1=INTEL |date=October 2019 |website=intel.com |publisher=Intel Corporation |access-date=2020-01-04 }}</ref> These instructions permit entering and exiting a virtual execution mode where the guest OS perceives itself as running with full privilege (ring 0), but the host OS remains protected.

{{As of|2015}}, almost all newer server, desktop and mobile Intel processors support VT-x, with some of the [[Intel Atom]] processors as the primary exception.<ref>{{cite web |url=http://ark.intel.com/VTList.aspx |title=Intel Virtualization Technology List |publisher=Ark.intel.com |access-date=2010-05-02 |url-status=live |archive-url=http://archive.wikiwix.com/cache/20101027065321/http://ark.intel.com/VTList.aspx |archive-date=2010-10-27}}</ref> With some [[motherboard]]s, users must enable Intel's VT-x feature in the [[BIOS]] setup before applications can make use of it.<ref>{{cite web |url=http://www.microsoft.com/windows/virtual-pc/support/configure-bios.aspx |title=Windows Virtual PC: Configure BIOS |publisher=Microsoft |access-date=2010-09-08 |url-status=dead |archive-url=https://web.archive.org/web/20100906162731/http://www.microsoft.com/windows/virtual-pc/support/configure-bios.aspx |archive-date=2010-09-06}}</ref>

Intel started to include [[Extended Page Table]]s (EPT),<ref>{{cite journal |last=Neiger |first=Gil |author2=A. Santoni |author3=F. Leung |author4=D. Rodgers |author5=R. Uhlig |title=Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization |journal=Intel Technology Journal |year=2006 |volume=10 |issue=3 |pages=167–178 |publisher=Intel |url=http://download.intel.com/technology/itj/2006/v10i3/v10-i3-art01.pdf |doi=10.1535/itj.1003.01 |access-date=2008-07-06 |url-status=dead |archive-url=https://web.archive.org/web/20120925205120/http://download.intel.com/technology/itj/2006/v10i3/v10-i3-art01.pdf |archive-date=2012-09-25}}</ref> a technology for page-table virtualization,<ref>{{cite web |last=Gillespie |first=Matt |title=Best Practices for Paravirtualization Enhancements from Intel Virtualization Technology: EPT and VT-d |work=Intel Software Network |publisher=Intel |date=2007-11-12 |url=http://software.intel.com/en-us/articles/best-practices-for-paravirtualization-enhancements-from-intel-virtualization-technology-ept-and-vt-d |access-date=2008-07-06 |url-status=live |archive-url=https://web.archive.org/web/20081226043414/http://software.intel.com/en-us/articles/best-practices-for-paravirtualization-enhancements-from-intel-virtualization-technology-ept-and-vt-d |archive-date=2008-12-26}}</ref> since the [[Nehalem (microarchitecture)|Nehalem]] architecture,<ref>{{cite press release |title=First the Tick, Now the Tock: Next Generation Intel Microarchitecture (Nehalem) |publisher=Intel |url=http://www.intel.com/pressroom/archive/reference/whitepaper_Nehalem.pdf |access-date=2008-07-06 |url-status=live |archive-url=https://web.archive.org/web/20090126145628/http://www.intel.com/pressroom/archive/reference/whitepaper_Nehalem.pdf |archive-date=2009-01-26}}</ref><ref>{{cite web |title=Technology Brief: Intel Microarchitecture Nehalem Virtualization Technology |publisher=Intel |date=2009-03-25 |url=http://download.intel.com/business/resources/briefs/xeon5500/xeon_5500_virtualization.pdf |access-date=2009-11-03 |url-status=live |archive-url=https://web.archive.org/web/20110607125400/http://download.intel.com/business/resources/briefs/xeon5500/xeon_5500_virtualization.pdf |archive-date=2011-06-07}}</ref> released in 2008. In 2010, [[Westmere (microarchitecture)|Westmere]] added support for launching the logical processor directly in [[real mode]]{{snd}} a feature called "unrestricted guest", which requires EPT to work.<ref>[http://2013.asiabsdcon.org/papers/abc2013-P5A-paper.pdf] "Intel added unrestricted guest mode on Westmere micro-architecture and later Intel CPUs, it uses EPT to translate guest physical address access to host physical address. With this mode, VMEnter without enable paging is allowed."</ref><ref>[https://web.archive.org/web/20130418014840/http://download.intel.com/products/processor/manual/326019.pdf] "If the “unrestricted guest” VM-execution control is 1, the “enable EPT” VM-execution control must also be 1"</ref>

Since the [[Haswell (microarchitecture)|Haswell]] microarchitecture (announced in 2013), Intel started to include ''VMCS shadowing'' as a technology that accelerates [[Virtualization#Nested virtualization|nested virtualization]] of VMMs.<ref>{{cite web | url = http://www-ssl.intel.com/content/dam/www/public/us/en/documents/white-papers/intel-vmcs-shadowing-paper.pdf | title = 4th-Gen Intel Core vPro Processors with Intel VMCS Shadowing | year = 2013 | access-date = 2014-12-16 | publisher = [[Intel]] }}</ref>
The ''virtual machine control structure'' (VMCS) is a [[data structure]] in memory that exists exactly once per VM, while it is managed by the VMM. With every change of the execution context between different VMs, the VMCS is restored for the current VM, defining the state of the VM's virtual processor.<ref>[http://download.microsoft.com/download/9/8/f/98f3fe47-dfc3-4e74-92a3-088782200fe7/TWAR05015_WinHEC05.ppt Understanding Intel Virtualization Technology (VT).] {{webarchive |url=https://web.archive.org/web/20140908110038/http://download.microsoft.com/download/9/8/f/98f3fe47-dfc3-4e74-92a3-088782200fe7/TWAR05015_WinHEC05.ppt |date=September 8, 2014}} Retrieved 2014-09-01</ref> As soon as more than one VMM or nested VMMs are used, a problem appears in a way similar to what required shadow page table management to be invented, as described [[#SWBASED|above]]. In such cases, VMCS needs to be shadowed multiple times (in case of nesting) and partially implemented in software in case there is no hardware support by the processor. To make shadow VMCS handling more efficient, Intel implemented hardware support for VMCS shadowing.<ref>[http://searchservervirtualization.techtarget.com/feature/The-what-where-and-why-of-VMCS-shadowing The 'what, where and why' of VMCS shadowing.] {{webarchive |url=https://web.archive.org/web/20140903165257/http://searchservervirtualization.techtarget.com/feature/The-what-where-and-why-of-VMCS-shadowing |date=2014-09-03}} Retrieved 2014-09-01</ref>

==== VIA virtualization (VIA VT) ====
[[VIA Nano]] 3000 Series Processors and higher support VIA VT virtualization technology compatible with Intel VT-x.<ref>[http://www.via.com.tw/en/resources/pressroom/pressrelease.jsp?press_release_no=4247 VIA Introduces New VIA Nano 3000 Series Processors] {{webarchive |url=https://web.archive.org/web/20130122011049/http://www.via.com.tw/en/resources/pressroom/pressrelease.jsp?press_release_no=4247 |date=January 22, 2013}}</ref> EPT is present in [[Zhaoxin]] ZX-C, a descendant of [[List of VIA Nano microprocessors#QuadCore-E|VIA QuadCore-E]] & [[List of VIA Eden microprocessors#Eden C|Eden X4]] similar to Nano [[List of VIA Nano microprocessors#Nano C|C4350AL]].<ref>{{cite web |url=http://en.zhaoxin.com/Upload/201707061728050030.pdf |title=Notebook Solution: Kaixian ZX-C Processor + VX11PH Chipset}}</ref>

==== {{Anchor|INTERRUPT}}Interrupt virtualization (AMD AVIC and Intel APICv) ====
In 2012, AMD announced their ''Advanced Virtual Interrupt Controller'' (''AVIC'') targeting interrupt overhead reduction in virtualization environments.<ref>Wei Huang, [http://www.slideshare.net/xen_com_mgr/introduction-of-amd-virtual-interrupt-controller Introduction of AMD Advanced Virtual Interrupt Controller] {{webarchive |url=https://web.archive.org/web/20140714160016/http://www.slideshare.net/xen_com_mgr/introduction-of-amd-virtual-interrupt-controller |date=2014-07-14}}, XenSummit 2012</ref> This technology, as announced, does not support [[Advanced Programmable Interrupt Controller|x2APIC]].<ref>{{cite web
 |url         = http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-interrupt-virt-kvm-roedel.pdf
 |title       = Next-generation Interrupt Virtualization for KVM
 |date        = August 2012
 |access-date  = 2014-07-12
 |author      = Jörg Rödel
 |publisher   = AMD
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20160304203744/http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-interrupt-virt-kvm-roedel.pdf
 |archive-date = 2016-03-04
}}</ref>
In 2016, AVIC is available on the AMD family 15h models 6Xh
(Carrizo) processors and newer.<ref>{{cite web |url=http://www.mail-archive.com/xen-devel@lists.xen.org/msg81719.html |title=[Xen-devel] [RFC PATCH 0/9] Introduce AMD SVM AVIC |website=www.mail-archive.com |access-date=4 May 2018 |url-status=live |archive-url=https://web.archive.org/web/20170202040420/https://www.mail-archive.com/xen-devel@lists.xen.org/msg81719.html |archive-date=2 February 2017}}</ref>

Also in 2012, Intel announced a similar technology for interrupt and [[Advanced Programmable Interrupt Controller|APIC]] virtualization, which did not have a brand name at its announcement time.<ref>{{cite web
 |url         = http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-intel-vt-feat-nakajima.pdf
 |title       = Reviewing Unused and New Features for Interrupt/APIC Virtualization
 |date        = 2012-12-13
 |access-date  = 2014-07-12
 |author      = Jun Nakajimaa
 |publisher   = Intel
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20150421073147/http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/09/2012-lpc-virt-intel-vt-feat-nakajima.pdf
 |archive-date = 2015-04-21
}}</ref>
Later, it was branded as ''APIC virtualization'' (''APICv'')<ref>{{cite web
 |url         = http://software.intel.com/en-us/blogs/2013/12/17/apic-virtualization-performance-testing-and-iozone
 |title       = APIC Virtualization Performance Testing and Iozone
 |date        = 2013-12-17
 |access-date  = 2014-07-12
 |author      = Khang Nguyen
 |website     = software.intel.com
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20140714142551/https://software.intel.com/en-us/blogs/2013/12/17/apic-virtualization-performance-testing-and-iozone
 |archive-date = 2014-07-14
}}</ref>
and it became commercially available in the [[Ivy Bridge (microarchitecture)|Ivy Bridge EP]] series of Intel CPUs, which is sold as Xeon E5-26xx v2 (launched in late 2013) and as Xeon E5-46xx v2 (launched in early 2014).<ref>{{cite web
 |url         = http://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/xeon-e5-4600-v2-brief.pdf
 |title       = Product Brief Intel Xeon Processor E5-4600 v2 Product Family
 |date        = 2014-03-14
 |access-date  = 2014-07-12
 |publisher   = Intel
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20140714145052/http://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/xeon-e5-4600-v2-brief.pdf
 |archive-date = 2014-07-14
}}</ref>

=== {{Anchor|GPU}}Graphics processing unit ===
Graphics virtualization is not part of the x86 architecture. Intel [[Graphics Virtualization Technology]] (GVT) provides graphics virtualization as part of more recent Gen graphics architectures. Although [[AMD Accelerated Processing Unit|AMD APUs]] implement the [[x86-64]] instruction set, they implement AMD's own graphics architectures ([[TeraScale (microarchitecture)|TeraScale]], [[Graphics Core Next|GCN]] and [[RDNA (microarchitecture)|RDNA]]) which do not support graphics virtualization.{{cn|date=October 2022}} [[Larrabee (microarchitecture)|Larrabee]] was the only graphics [[microarchitecture]] based on x86, but it likely did not include support for graphics virtualization.

=== Chipset ===
{{Main|I/O virtualization}}

Memory and I/O virtualization is performed by the [[chipset]].<ref>{{cite web |url=http://www.intel.com/technology/itj/2006/v10i3/2-io/5-platform-hardware-support.htm |title=Intel platform hardware support for I/O virtualization |publisher=Intel.com |date=2006-08-10 |access-date=2012-02-04 |url-status=live |archive-url=https://web.archive.org/web/20070120024219/http://www.intel.com/technology/itj/2006/v10i3/2-io/5-platform-hardware-support.htm |archive-date=2007-01-20}}</ref> Typically these features must be enabled by the BIOS, which must be able to support them and also be set to use them.

==== {{Anchor|Intel-VT-d|FLR}}I/O MMU virtualization (AMD-Vi and Intel VT-d) ====
{{See also|Input–output memory management unit#Virtualization}}
[[File:AMD-Vi boot log screenshot.png|upright=1.5|thumb|A [[Linux kernel]] log showing AMD-Vi information]]
An input/output memory management unit (IOMMU) allows guest [[virtual machine]]s to directly use [[peripheral]] devices, such as Ethernet, accelerated graphics cards, and hard-drive controllers, through [[direct memory access|DMA]] and [[interrupt]] remapping. This is sometimes called ''PCI passthrough''.<ref>{{cite web |title=Linux virtualization and PCI passthrough |url=http://www.ibm.com/developerworks/linux/library/l-pci-passthrough/ |publisher=IBM |access-date=10 November 2010 |url-status=dead |archive-url=https://web.archive.org/web/20091101161431/http://www.ibm.com/developerworks/linux/library/l-pci-passthrough/ |archive-date=1 November 2009}}</ref>

An IOMMU also allows operating systems to eliminate bounce buffers needed to allow themselves to communicate with peripheral devices whose memory address spaces are smaller than the operating system's memory address space, by using memory address translation. At the same time, an IOMMU also allows operating systems and hypervisors to prevent buggy or malicious hardware from [[DMA attack|compromising memory security]].
 
Both AMD and Intel have released their IOMMU specifications:
* AMD's I/O Virtualization Technology, "AMD-Vi", originally called "IOMMU"<ref>{{cite web |title=AMD I/O Virtualization Technology (IOMMU) Specification Revision 1.26 |url=http://support.amd.com/us/Processor_TechDocs/34434-IOMMU-Rev_1.26_2-11-09.pdf |access-date=2011-05-24 |url-status=live |archive-url=https://web.archive.org/web/20110124134140/http://support.amd.com/us/Processor_TechDocs/34434-IOMMU-Rev_1.26_2-11-09.pdf |archive-date=2011-01-24}}</ref>
* Intel's "Virtualization Technology for Directed I/O" (VT-d),<ref>{{cite web |url=http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/vt-directed-io-spec.html |title=Intel Virtualization Technology for Directed I/O (VT-d) Architecture Specification |access-date=2012-02-04 |url-status=dead |archive-url=https://web.archive.org/web/20130403045524/http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/vt-directed-io-spec.html |archive-date=2013-04-03}}</ref> included in most high-end (but not all) newer Intel processors since the Core 2 architecture.<ref>{{cite web |url=http://ark.intel.com/search/advanced?VTD=true |title=Intel Virtualization Technology for Directed I/O (VT-d) Supported CPU List |publisher=Ark.intel.com |access-date=2012-02-04 |url-status=dead |archive-url=http://archive.wikiwix.com/cache/20101027065321/http://ark.intel.com/search/advanced?VTD=true |archive-date=2010-10-27}}</ref>

In addition to the CPU support, both [[motherboard]] [[chipset]] and system firmware ([[BIOS]] or [[Unified Extensible Firmware Interface|UEFI]]) need to fully support the IOMMU I/O virtualization functionality for it to be usable. Only the [[Conventional PCI|PCI]] or [[PCI Express]] devices supporting ''function level reset'' (FLR) can be virtualized this way, as it is required for reassigning various [[PCI device function|device functions]] between virtual machines.<ref>{{Cite web
 |url         = http://www.pcisig.com/specifications/pciexpress/specifications/ECN_Function_Level_Reset_27jun2006.pdf
 |title       = PCI-SIG Engineering Change Notice: Function Level Reset (FLR)
 |date        = 2006-06-27
 |access-date  = 2014-01-10
 |website     = pcisig.com
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20160304001637/http://pcisig.com/specifications/pciexpress/specifications/ECN_Function_Level_Reset_27jun2006.pdf
 |archive-date = 2016-03-04
}}</ref><ref>{{Cite web
 |url         = http://wiki.xen.org/wiki/VTd_HowTo
 |title       = Xen VT-d
 |date        = 2013-06-06
 |access-date  = 2014-01-10
 |website     = xen.org
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20140209124212/http://wiki.xen.org/wiki/VTd_HowTo
 |archive-date = 2014-02-09
}}</ref>  If a device to be assigned does not support [[Message Signaled Interrupts]] (MSI), it must not share [[interrupt]] lines with other devices for the assignment to be possible.<ref>{{cite web
 |url         = http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
 |title       = How to assign devices with VT-d in KVM
 |date        = 2014-04-23
 |access-date  = 2015-03-05
 |website     = linux-kvm.org
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20150310220832/http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
 |archive-date = 2015-03-10
}}</ref>
All [[conventional PCI]] devices routed behind a PCI/[[PCI-X]]-to-PCI Express bridge can be assigned to a guest virtual machine only all at once; PCI Express devices have no such restriction.

==== Network virtualization (VT-c) ====
* Intel's "Virtualization Technology for Connectivity" (VT-c).<ref>{{cite web |url=http://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/virtualization-technology-connectivity-brief.pdf |title=Intel Virtualization Technology for Connectivity (VT-c) |publisher=Intel.com |access-date=2018-02-14 |url-status=live |archive-url=https://web.archive.org/web/20160222022822/http://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/virtualization-technology-connectivity-brief.pdf |archive-date=2016-02-22}}</ref>

===== {{Anchor|PCI-SIG|SR-IOV}}PCI-SIG Single Root I/O Virtualization (SR-IOV) =====
{{Main|Single-root input/output virtualization}}
''PCI-SIG Single Root I/O Virtualization (SR-IOV)'' provides a set of general (non-x86 specific) I/O virtualization methods based on [[PCI Express]] (PCIe) native hardware, as standardized by PCI-SIG:<ref>{{cite web |url=http://www.pcisig.com/specifications/iov/ats |title=PCI-SIG I/O Virtualization (IOV) Specifications |publisher=Pcisig.com |date=2011-03-31 |access-date=2012-02-04 |url-status=dead |archive-url=https://web.archive.org/web/20120115211058/http://www.pcisig.com/specifications/iov/ats/ |archive-date=2012-01-15}}</ref>

* ''Address translation services (ATS)'' supports native IOV across PCI Express via address translation. It requires support for new transactions to configure such translations.
* ''[[Single-root input/output virtualization|Single-root IOV]] (SR-IOV or SRIOV)'' supports native IOV in existing single-root complex PCI Express topologies. It requires support for new device capabilities to configure multiple virtualized configuration spaces.<ref name="intel-grantley">{{cite web
 |url         = http://www.intel.com/content/dam/technology-provider/secure/us/en/documents/product-marketing-information/tst-grantley-launch-presentation-2014.pdf#page=104
 |title       = Intel Look Inside: Intel Ethernet
 |date        = November 27, 2014
 |access-date  = March 26, 2015
 |publisher   = [[Intel]]
 |format      = PDF
 |page        = 104
 |url-status     = dead
 |archive-url  = https://web.archive.org/web/20160304003829/http://www.intel.com/content/dam/technology-provider/secure/us/en/documents/product-marketing-information/tst-grantley-launch-presentation-2014.pdf#page=104
 |archive-date = March 4, 2016
}}</ref>
* ''Multi-root IOV (MR-IOV)'' supports native IOV in new topologies (for example, blade servers) by building on SR-IOV to provide multiple root complexes which share a common PCI Express hierarchy.

In SR-IOV, the most common of these, a host VMM configures supported devices to create and allocate virtual "shadows" of their configuration spaces so that virtual machine guests can directly configure and access such "shadow" device resources.<ref>{{cite web
 |url         = http://www.usenix.org/conference/wiov-08/sr-iov-networking-xen-architecture-design-and-implementation
 |title       = SR-IOV Networking in Xen: Architecture, Design and Implementation
 |date        = 2008
 |access-date  = 2014-01-10
 |author1     = Yaozu Dong |author2=Zhao Yu |author3=Greg Rose
 |publisher   = [[USENIX]]
 |work        = usenix.org
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20140109052753/https://www.usenix.org/conference/wiov-08/sr-iov-networking-xen-architecture-design-and-implementation
 |archive-date = 2014-01-09
}}</ref> With SR-IOV enabled, virtualized network interfaces are directly accessible to the guests,<ref>{{cite web
 |url         = http://www.intel.com/content/dam/www/public/us/en/documents/solution-briefs/10-gbe-ethernet-flexible-port-partitioning-brief.pdf
 |title       = An Introduction to Intel Flexible Port Partitioning Using SR-IOV Technology
 |date        = September 2011
 |access-date  = September 24, 2015
 |author1     = Patrick Kutch
 |author2     = Brian Johnson
 |author3     = Greg Rose
 |publisher   = [[Intel]]
 |url-status     = dead
 |archive-url  = https://web.archive.org/web/20150807015402/http://www.intel.com/content/dam/www/public/us/en/documents/solution-briefs/10-gbe-ethernet-flexible-port-partitioning-brief.pdf
 |archive-date = August 7, 2015
}}</ref>
avoiding involvement of the VMM and resulting in high overall performance;<ref name="intel-grantley" /> for example, SR-IOV achieves over 95% of the [[Bare machine|bare metal]] network bandwidth in [[NASA]]'s virtualized datacenter<ref>{{cite web | url=http://www.intel.com/content/dam/www/public/us/en/documents/case-studies/10-gigabit-ethernet-nasa-case-study.pdf | title=NASA's Flexible Cloud Fabric: Moving Cluster Applications to the Cloud | publisher=[[Intel]] | access-date=2014-01-08 | url-status=dead | archive-url=https://web.archive.org/web/20121222083815/http://www.intel.com/content/dam/www/public/us/en/documents/case-studies/10-gigabit-ethernet-nasa-case-study.pdf | archive-date=2012-12-22}}</ref> and in the [[Amazon Web Services|Amazon Public Cloud]].<ref>{{cite web | url=http://blogs.scalablelogic.com/2013/12/enhanced-networking-in-aws-cloud.html | title=Enhanced Networking in the AWS Cloud | publisher=Scalable Logic | date=2013-12-31 | access-date=2014-01-08 | url-status=live | archive-url=http://archive.wikiwix.com/cache/20140109035046/http://blogs.scalablelogic.com/2013/12/enhanced-networking-in-aws-cloud.html | archive-date=2014-01-09}}</ref><ref>{{cite web |url=http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html |title=Enhanced Networking in the AWS Cloud - Part 2 |publisher=Scalable Logic |date=2013-12-31 |access-date=2014-01-08 |url-status=live |archive-url=http://archive.wikiwix.com/cache/20140110122946/http://blogs.scalablelogic.com/2014/01/enhanced-networking-in-aws-cloud-part-2.html |archive-date=2014-01-10}}</ref>

== See also ==
{{Div col|colwidth=25em}}
* [[Comparison of application virtualization software]]
* [[Comparison of platform virtualization software]]
* [[Hardware-assisted virtualization]]
* [[Hypervisor]]
* [[I/O virtualization]]
* [[Network virtualization]]
* [[OS-level virtualization]]
* [[Timeline of virtualization development]]
* [[Virtual machine]]
* [[List of IOMMU-supporting hardware]]
* [[Second Level Address Translation]] (SLAT)
* [[Message Signaled Interrupts]] (MSI)
{{div col end}}

== References ==
{{Reflist|30em}}

== External links ==
* [http://www.hardwaresecrets.com/everything-you-need-to-know-about-the-intel-virtualization-technology/ Everything You Need to Know About the Intel Virtualization Technology] Archived at [https://ghostarchive.org/archive/7xRW3?kreymer=true ghostarchive.org] at 10 May 2022
* [http://www.cs.usfca.edu/~cruse/cs686s07/ A special course at the University of San Francisco on Intel EM64T and VT Extensions] (2007)
* [http://opensecuritytraining.info/AdvancedX86-VTX.html 2 day open source & open access class on writing a VT-x VMM]

{{DEFAULTSORT:X86 Virtualization}}
[[Category:X86 architecture]]
[[Category:Hardware virtualization]]