In our last post on Large Pages, Zero Pages, TPS and Boot-Storm in vSphere 4.x, we made the following conclusions regarding TPS:
- Transparent Page Sharing runs on all vSphere 4.x systems with processors that have no hardware-assisted MMU virtualization support.
- On systems with processors that support hardware-assisted MMU virtualization (e.g., Intel EPT), ESX will use large pages and TPS will only kick-in when ESX is under memory pressure.
After further investigation, we realized that the first statement above is not necessarily correct.
We later found out that the Large Pages support was actually available since ESX version 3.5 as documented in the VMware whitepaper – Large Page Performance. This whitepaper states that if the VM can use Large Pages, then ESX 3.5 will support the feature. Furthermore, Windows® Server 2008/2008 R2 as well as Windows Vista® and Windows 7 have Large Memory Pagesenabled by default. Now the question is: Would ESX server without the EPT support still utilize TPS to share memory among Windows VMs that have Large Memory Pages enabled by default? We will re-run the tests now with a Windows 2008 R2 VM to find out.
Lab Environment Configurations
Hardware:
- HP ProLiant DL380 G5 with dual Intel Xeon E5335 processors and 32GB memory
- HP ProLiant DL360 G6 with dual Intel Xeon E5540 processors and 32GB memory
Both HP servers are running VMware ESX 4.1 update 1. The Windows Guest VM for this test has Windows 2008 R2 SP1 installed and with 4GB of configured memory.
Test on Server without the EPT support
We first powered on the Windows 2008 R2 VM on the HP DL380 G5 server and the memory usage statistics are shown as follows:
Figure 1. Memory Usage on ESX server without EPT
Test on Server with Intel EPT Support
We powered up the Windows 2008 R2 VM on the HP DL360 G6 server and the memory usage statistics during the power up are shown as follows:
Figure 2. Memory Usage on ESX server with EPT support
Comparing the memory statistics from Fig. 1 and Fig. 2, we can see that:
- Consumed memory stabilizes quickly upon powering up the VMs in both servers. This strongly suggests that there is no TPS activity on either ESX servers when Windows 2008 R2 is powered on, regardless of the hardware-assisted MMU virtualization support.
- The amount of Consumed Memory of the Windows 2008 R2 VM when running on the ESX server with the EPT support is almost four times that consumed by the ESX server without the EPT.
But our further examination using the Windows Resource Monitor indicates that there is 3559MB of available memory and only 528MB of memory in use as shown below:
Figure 3. Memory Usage seen by the Windows O/S
The above results are more in line with the memory statistics in Figure one when running on ESX server without the EPT support.
We are really puzzled by the big discrepancy in terms of the Consumed Memory when the Windows 2008 R2 VM is running on the server with EPT support. This scenario may also have bearing on the consolidation ratios due to the extra memory allocated by the ESX server. Currently, we have no explanation and would like to reach out to the VMware community to see if someone could provide some insight to explain these findings.
Author: YP Chien, Ph.D. - Kingston Technology