Why Do We Have So Much Ballooned Memory? Part II

Published 5/26/2011

Part II of III - Analysis of the memory reclamation using a little bit of math

Before we present our findings, we needed to confirm one precondition: the hypervisor during memory pressure needs to reclaim "physical" memory regardless of the techniques used (e.g., Balloon, Memory Compression, and/or Swap) from the VMs. To help us gain insight into the actual memory activities, we will first perform a memory size analysis using esxtop memory counters shown in the following table. The descriptions of these esxtop memory counters are obtained from p. 97 of the vSphere Resource Management Guide:

We also use information from the VMware community document, Interpreting esxtop 4.1 Statistics, to supply the following two memory size equations:

1. "MEMSZ" = "GRANT" + "MEMCTL" + "SWCUR" + "never touched"
2. "Consumed" = "GRANT" - "SHRDSVD", or "GRANT" = "Consumed" + "SHRDSVD"
   
   Note that the "Consumed" memory usage is defined as the amount of host memory ( in other words, host physical memory) that is allocated to the VM, as described in p. 2 of Understanding Memory Resource Management in VMware ESX 4.1.
   
   Since the "never touched" is a negligible amount, we could safely modify the first equation and include the "ZIP saved" (when Memory Compression is enabled) as follows:
3. "MEMSZ" = "GRANT" + "MEMCTL" + "SWCUR" + "ZIP Saved"
   
   We now substitute "GRANT" using equation 2 above and have the following:
4. "MEMSZ" = ("Consumed" + "SHRDSVD") + ( "MEMCTL" + "SWCUR" + "ZIP Saved")
   
   Since the "MEMSZ" is actually the configured memory size of the VM, it is 4096MB in our test and a constant value. So if we were to examine the memory size variations caused by Memory Balloon, Memory Compression, and/or Swapping we simply take the differentiation operation to the above equation and now we have:
   
   Δ("Consumed" + "SHRDSVD") + Δ( "MEMCTL" + "SWCUR" + "ZIP Saved") = 0
   or, Δ( "MEMCTL" + "SWCUR" + "ZIP Saved") = -Δ("Consumed" + "SHRDSVD")

The above equation says that changes in memory consumed are inversely proportional to changes in memory reclaimed. In other words, the amount of memory reclaimed using Balloon, Swap and/or Memory Compression is inversely correlated to the amount of "Consumed" plus "SHRDSVD".

The important point to notice here is that only the "Consumed" memory is backed by physical memory. We further noticed from our other tests that the memory reclaimed from memory compression (ZIP Saved) and swapping (SWCUR) comes solely from the VM's consumed memory. Hence, the amount of Ballooned Memory reported should include both the "Consumed" and the "SHRDSVD".

The amount of "SHRDSVD" memory is normally nonzero for Windows VM which is either freshly booted or has deflated ballooned memory recently. Since we reboot all our VMs after each memory performance test, we had excessive ballooned memory reported!

We will prove our points using actual test results in the final blog post.

Author: Yungteh Chien