I'm trying to find if the latest Apple Silicon chips support fully nested virtualization. Namely the ability for it to run a VM inside another VM.
Its really hard to find definitive answers. Most discussions about this are on fan forums that say that the Apple chips support the ARM instruction set. Which does have a fully fleshed out procedure for infinitely nested virtualization.
The short answer is yes. The technical details are below:
It depends a bit on what you mean by "support" exactly. To better understand the nuances in that word, first a quick and rough overview of how ARM CPUs (such as the Apple Silicon M-series) runs software at different privilege levels:
ARM CPUs divide the world into different exceptions levels - namely EL0, EL1, EL2 and EL3. Here EL0 is the least privileged level and EL3 is the most privileged level.
Software running at a specific exception level is restricted in various ways. When that software tries to perform some operation that has been restricted, the CPU will generally pause that software, and instead start running the software that executes at a higher exception level and let that software decide whether or not, and how, the operation should be allowed to continue.
As an example a standard application such as "Calendar.app" would be running at EL0. If it tries to do something privileged, such as access to disk storage, the CPU would effectively pause the program and let operating system code running at EL1 decide if and how that operation should be allowed to continue.
When you add hardware assisted virtualisation into the mix, you would often find the software performing the virtualisation management (known as the hypervisor) running at EL2.
Apple does not currently make use of EL3.
This way of organising privilege levels is also found on Intel CPUs. Here you might have heard of "rings", which is basically the same concept. They do the numbering in the opposite way though, so that ring 0 is the most privileged.
Now with the theoretical overview out of the way, we can start answering the question at hand:
It is of course entirely possible to use nested virtualisation with the Apple Silicon series like with almost any other CPU. You can just run everything at EL0 without requiring any special operating system or hardware features. It can be done with emulation and will usually be quite slow.
Another possibility is to use ordinary hardware assisted virtualisation. The first layer would be hardware virtualised like usual. Then you would use paravirtualisation software to run a modified operating system inside the first level guest at EL0, in order to have second level guests. It would be complicated, limited in performance and require customised guest software. It is usally not what you want.
I assume you're talking about hardware assisted nested virtualisation instead:
In that case you have a host hypervisor that provides the first level of virtualisation, and a guest hypervisor that provides the second level (or more) level of virtualisation. As mentioned before, the host hypervisor runs at EL2. This means that the guest hypervisor runs at EL1.
The hardware features for virtualisation are controlled via registers (and other means). On the Apple M1 CPU the registers at EL2 cannot be changed by EL1 software at all. This mans that a guest hypervisor running at EL1 inside a virtual machine cannot use hardware virtualisation features to provide nested virtualisation.
ARM introduced support for nested virtualisation with their v8.3-A specification. This means that a guest hypervisor running at EL1 modifying the EL2 registers for hardware assisted nested virtualisation (HCR_EL2.NV, HCR_EL2.NV1 and HCR_EL2.AT) will cause the CPU to pause the guest hypervisor, and let the host hypervisor running at EL2 decide whether and how to proceed. That is a big part in enabling nested virtualisation. There are other details, for example related to memory and interrupt management.
One thing to note is that this process of pausing the software and letting more privileged software run takes some time. This is what is known as a context switch - which you would also known from just standard macOS where a context switch happens every time you transition from running application code to running operating system code. So you want to preferably have as few context switches as possible to get optimal performance.
As nested virtualisation means changing multiple registers quite often you will get a performance hit here. With ARM's v8.4-A specification they introduced a performance optimisation, which can be though of as letting the guest hypervisor in EL1 modify a number of registers and then letting the host hypervisor in EL2 process them in a batch. That improves performance for nested virtualisation. This feature is enabled throught the HCR_EL2.NV2 register.
The ARM specification also includes a way for the CPU to inform the software of which features it support. Specifically for nested virtualisation the CPU can signal either that it doesn't support nested virtualisation at all, that is supports the v8.3-A level (HCR_EL2.NV, HCR_EL2.NV1 and HHCR_EL2.AT) or that it supports the v8.4-A level (addition of HCR_EL2.NV2).
The Apple M1 CPU does not support hardware assisted nested virtualisation at all.
The Apple M2 and M3 CPUs support the v8.4-A level of hardware assisted nested virtualisation.
However, Apple has not supplied user level software for that feature. I.e. the Virtualisation and Hypervisor frameworks that comes with macOS does not (yet) support hardware assisted nested virtualization.
This means that for example when running Windows 11 inside VMware Fusion or Parallels Desktop you won't have access to the WSL2 feature (Windows Subsystem for Linux 2).
I know progress has been made on Asahi Linux in supporting this, so at some point in the future, the existing M2 and M3 hardware will most probably get support.
According to Apple Documentation:
Nested virtualization is available for Mac with the M3 chip, and later.
Finally, macOS 15 introduces support for Nested Virtualization! THANK YOU, Apple!!!
Now, for developing WSL2 support and building cloud sandboxes with virtualization within virtual machines, a separate computer will no longer be needed!
I’ve been waiting for this since the introduction of Apple Silicon processors, checking for this feature every release, and now the API is finally here!
I’m looking forward to the Parallels Desktop update more than ever! https://developer.apple.com/documentation/virtualization/vzgenericplatformconfiguration/4360552-isnestedvirtualizationenabled?changes=latest_minor
