!heap -stat -h doesn't show allocations - heap-memory

Question

I'm trying to determine why my applications consumes 4GB of Private Bytes. So I took a full memory dump, loaded it in windbg. But analyzing using !heap -stat -h produces weird results which don't add up:
0:000> !heap -s
(...)
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-------------------------------------------------------------------------------------
000002d0a0000000 00000002 2800804 2780508 2800700 2984 1980 177 0 6 LFH
000002d09fe10000 00008000 64 4 64 2 1 1 0 0
000002d09ff70000 00001002 1342924 1334876 1342820 13042 3342 87 0 0 LFH
Ok, I got a 2.8GB heap and a 1.34GB heap. Let's look at the allocations of the first one:
0:000> !heap -stat -h 000002d0a0000000
heap # 000002d0a0000000
group-by: TOTSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
651 291 - 1035e1 (16.00)
79c 1df - e3ce4 (14.06)
28 156d - 35908 (3.31)
(...)
IIUC, the first line means block size 0x651(=1617 bytes), number blocks 0x291(=657), for total bytes of 0x103531(=1062369 bytes =~1MB), and that's 16% of total busy bytes. But looking at the summary, there should be ~2.8GB of busy bytes!
Another disparity:
0:000> !heap -stat -h 000002d0a0000000 -grp A
heap # 000002d0a0000000
group-by: ALLOCATIONSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
a160 1 - a160 (0.62)
7e50 2 - fca0 (0.97)
0:000> !heap -h 000002d0a0000000
(...)
(509 lines that note allocations with size 7e50, like this one:)
000002d0a3f48000: 11560 . 07e60 [101] - busy (7e50)
Edit: Many lines also say Internal at the end, which appears to mean HEAP_ENTRY_VIRTUAL_ALLOC - but the 509 lines with (7e50) don't.
My question: How can I get !heap -stat -h to show all the allocations, so they add up to the output of !heap -s?

Answer 1

At the moment I can only explain the busy percentage, but that may already be helpful. Its value is a bit misleading.
Virtual memory is memory taken from VirtualAlloc(). The C++ heap manager uses that basic mechanism to get memory from the operating system. That virtual memory can be committed (ready to use) or reserved (can be committed later). The output of !heap -s tells you the status of the heaps with respect to that virtual memory.
So we agree that any memory the C++ heap manager can use is committed memory. This coarse granular virtual memory is split into finer blocks by the C++ heap manager. The heap manager may allocate such smaller blocks and free them, depending on the need of malloc()/free() or new/delete operations.
When blocks become free, they are no longer busy. At the same time, the C++ heap manager may decide to not give the free block back to the OS, because
it can't, since other parts of the 64k virtual memory are still in use
or it doesn't want to (internal reasons we can't exactly know, e.g. performance reasons)
Since the free parts do not count as busy, the busy percentage seems to be too high when compared to the virtual memory.
Mapped to your case, this means:
you have 2.8 GB of virtual memory
in heap 000002d0a0000000, you have ~1 MB / 16% = 6.25 MB of memory in use, the rest could be in free heap blocks (it possibly isn't)
The following example is based on this C++ code:
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <string>
#include <iomanip>
int main()
{
HANDLE hHeap = HeapCreate(0, 0x1000000, 0x10000000); // no options, initial 16M, max 256M
HeapAlloc(hHeap, HEAP_GENERATE_EXCEPTIONS, 511000); // max. allocation size for non-growing heap
std::cout << "Debug now, handle is 0x" << std::hex << std::setfill('0') << std::setw(sizeof(HANDLE)) << hHeap << std::endl;
std::string dummy;
std::getline(std::cin, dummy);
return 0;
}
The only 511kB block will be reported as 100%, although it is only ~1/32 of the 16 MB:
0:001> !heap -stat -h 009c0000
heap # 009c0000
group-by: TOTSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
7cc18 1 - 7cc18 (100.00)
To see the free parts as well, use !heap -h <heap> -f:
0:001> !heap -h 0x01430000 -f
Index Address Name Debugging options enabled
3: 01430000
Segment at 01430000 to 11430000 (01000000 bytes committed)
Flags: 00001000
ForceFlags: 00000000
Granularity: 8 bytes
Segment Reserve: 00100000
Segment Commit: 00002000
DeCommit Block Thres: 00000200
DeCommit Total Thres: 00002000
Total Free Size: 001f05c7
Max. Allocation Size: 7ffdefff
Lock Variable at: 01430138
Next TagIndex: 0000
Maximum TagIndex: 0000
Tag Entries: 00000000
PsuedoTag Entries: 00000000
Virtual Alloc List: 014300a0
Uncommitted ranges: 01430090
FreeList[ 00 ] at 014300c4: 01430590 . 0240e1b0
0240e1a8: 7cc20 . 21e38 [100] - free <-- no. 1
02312588: 7f000 . 7f000 [100] - free <-- no. 2
[...]
01430588: 00588 . 7f000 [100] - free <-- no. 32
Heap entries for Segment00 in Heap 01430000
address: psize . size flags state (requested size)
01430000: 00000 . 00588 [101] - busy (587)
01430588: 00588 . 7f000 [100]
[...]
02312588: 7f000 . 7f000 [100]
02391588: 7f000 . 7cc20 [101] - busy (7cc18)
0240e1a8: 7cc20 . 21e38 [100]
0242ffe0: 21e38 . 00020 [111] - busy (1d)
02430000: 0f000000 - uncommitted bytes.
0:001> ? 7cc18
Evaluate expression: 511000 = 0007cc18
Here we see that I have a heap of 256 MB (240 MB uncommitted, 0x0f000000 + 16 MB committed, 0x01000000). Summing up the items in the FreeList, I get
0:001> ? 0n31 * 7f000 + 21e38
Evaluate expression: 16264760 = 00f82e38
So almost everything (~16 MB) is considered as free and not busy by the C++ heap manager. Memory like that 16 MB is reported by !heap -s in this way in WinDbg 6.2.9200:
0:001> !heap -s
LFH Key : 0x23e41d0e
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
004d0000 00000002 1024 212 1024 6 5 1 0 0 LFH
00750000 00001002 64 20 64 9 2 1 0 0
01430000 00001000 262144 16384 262144 15883 32 1 0 0
External fragmentation 96 % (32 free blocks)
-----------------------------------------------------------------------------
IMHO there's a bug regarding reserved and committed memory: it should be 262144k virtual - 16384 committed = 245760k reserved.
Note how the list length matches the number of free blocks reported before.
Above explains the busy percentage only. The remaining question is: the free memory reported in your case doesn't match this scenario.
Usually I'd say the remaining memory is in virtual blocks, i.e. memory blocks that are larger than 512 kB (32 bit) or 1 MB (64 bit) as mentioned on MSDN for growable heaps. But that's not the case here.
There is no output about virtual blocks and the number of virtual blocks is reported as 0.
A program that generates a virtual block would be
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <string>
#include <iomanip>
int main()
{
HANDLE hHeap = HeapCreate(0, 0x1000000, 0); // no options, initial 16M, growable
HeapAlloc(hHeap, HEAP_GENERATE_EXCEPTIONS, 20*1024*1024); // 20 MB, force growing
std::cout << "Debug now, handle is 0x" << std::hex << std::setfill('0') << std::setw(sizeof(HANDLE)) << hHeap << std::endl;
std::string dummy;
std::getline(std::cin, dummy);
return 0;
}
and the !heap command would mention the virtual block:
0:001> !heap -s
LFH Key : 0x7140028b
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
006d0000 00000002 1024 212 1024 6 5 1 0 0 LFH
001d0000 00001002 64 20 64 9 2 1 0 0
Virtual block: 01810000 - 01810000 (size 00000000)
00810000 00001002 16384 16384 16384 16382 33 1 1 0
External fragmentation 99 % (33 free blocks)
-----------------------------------------------------------------------------
In your case however, the value virtual blocks is 0. Perhaps this what is reported as "Internal" in your version of WinDbg. If you have not upgraded yet, try version 6.2.9200 to get the same output as I do.