CaptureStackBackTrace inconsistencies using FramesToSkip - c

Question

On windows you can capturing the stack trace using CaptureStackBackTrace as
void* frames[USHRT_MAX];
USHORT framesCount = CaptureStackBackTrace(0, USHRT_MAX, frames, NULL);
However, capturing it by smaller chunks in a loop to avoid allocating a USHRT_MAX buffer doesn't provide the same result.
This code
#include <Windows.h>
#include <assert.h>
#include <stdio.h>
__declspec(noinline) void CheckStack(void)
{
printf("Checking stack...\n");
void* entireStack[USHRT_MAX];
USHORT frameCount = CaptureStackBackTrace(0, USHRT_MAX, entireStack, NULL);
printf("Stack size is: %u\n", frameCount);
ULONG frameOffset = 1;
for (;;)
{
void* chunk[64];
USHORT framesFound = CaptureStackBackTrace(frameOffset, 64, chunk, NULL);
if (framesFound)
{
if (memcmp(entireStack + frameOffset, chunk, sizeof(chunk)) != 0)
{
printf("Incorrect content\n");
}
frameOffset += (ULONG)framesFound;
}
else
{
break;
}
}
if (frameCount != frameOffset)
{
printf("Incorrect count (%u != %u)\n", frameCount, frameOffset);
}
printf("Done\n");
}
__declspec(noinline) void Test(int i)
{
if (i != 500)
Test(++i);
else
CheckStack();
}
int main()
{
Test(0);
}
produces the following output
Checking stack...
Stack size is: 507
Incorrect count (507 != 257)
Done
when building as cl /Od main.c /link /OUT:main.exe.
Am I using the FramesToSkip parameter incorrectly or why are the counts not equal?

Answer 1

If you are using Windows Server 2003 and Windows XP,
The sum of the FramesToSkip and FramesToCapture parameters must be
less than 63.
That's in document.
Else, as #RbMm says, In the API source code, there is the following logic:
if(FramesToSkip>0xfe)
{
return 0; //There are too many stack structures skipped, returning directly to 0.
}
However, this is not metioned on msdn both in the CaptureStackBackTrace and RtlCaptureStackBackTrace.
I am not going to post the source code here, but prove it in debugging:
1.Create a sample:
#include <Windows.h>
#include <assert.h>
#include <stdio.h>
__declspec(noinline) void CheckStack(void)
{
void* entireStack[USHRT_MAX];
USHORT frameCount = CaptureStackBackTrace(255, USHRT_MAX, entireStack, NULL);
}
__declspec(noinline) void Test(int i)
{
if (i != 500)
Test(++i);
else
CheckStack();
}
int main()
{
Test(0);
}
2. Step into CaptureStackBackTrace in Disassembly:
You can see that dword ptr[ebp+8](the first parameter of CaptureStackBackTrace pushed in stack) will be compared with 0feh(254). If true, return 0.