What is the stack limit when MATLAB calls function in DLL - c

Question

I am trying to figure out, what is the stack size limitation, when MATLAB calls function in DLL.
Is there a way to configure the limit?
I am using loadlibrary, and calllib functions to call function implemented in C (in Dynamic-link library).
I created a test to figure out the stack limit.
I am using MATLAB 2016a (64 bits), and Visual Studio 2010 for building the DLL.
Here is my MATLAB source code:
loadlibrary('MyDll','MyDll.h')
size_in_bytes = 1000000;
res = calllib('MyDll', 'Test', size_in_bytes);
if (res == -1)
disp(['Stack Overflow... (size = ', num2str(size_in_bytes), ')']);
else
disp(['Successful stack allocation... (size = ', num2str(size_in_bytes), ')']);
end
unloadlibrary MyDll
Here is my C source code:
MyDll.h
// MyDll.h : DLL interface.
#ifndef MY_DLL_H
#define MY_DLL_H
#ifdef MY_DLL_EXPORTS
#define MY_DLL_API __declspec(dllexport)
#else
#define MY_DLL_API __declspec(dllimport)
#endif
extern MY_DLL_API int Test(int size);
#endif
MyDll.c
// MyDll.c
#include "MyDll.h"
#include <windows.h>
#include <stdio.h>
#include <string.h>
#include <malloc.h>
//Allocate <size> bytes in stack using _alloca(size).
//Return 0 if OK.
//Return (-1) in case of stack overflow.
int Test(int size)
{
//Not allocated on the stack...
static wchar_t errorMsg[100];
static wchar_t okMsg[100];
int errcode = 0;
void *pData = NULL;
//Prepare messages from advance.
swprintf_s(errorMsg, 100, L"Stack Overflow (size = %d)", size);
swprintf_s(okMsg, 100, L"Successful stack allocation (size = %d)", size);
__try
{
pData = _alloca(size);
}
// If an exception occurred with the _alloca function
__except (GetExceptionCode() == STATUS_STACK_OVERFLOW)
{
MessageBox(NULL, errorMsg, TEXT("Error"), MB_OK | MB_ICONERROR);
// If the stack overflows, use this function to restore.
errcode = _resetstkoflw();
if (errcode)
{
MessageBox(NULL, TEXT("Could not reset the stack!"), TEXT("Error"), MB_OK | MB_ICONERROR);
_exit(1);
}
pData = NULL;
};
if (pData != NULL)
{
//Fill allocated buffer with zeros
memset(pData, 0, size);
MessageBox(NULL, okMsg, TEXT("OK"), MB_OK);
return 0;
}
return -1;
}
The __try and __except block is taken from Microsoft example:
https://msdn.microsoft.com/en-us/library/wb1s57t5.aspx
DLL Compiler flags:
/Zi /nologo /W4 /WX- /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_USRDLL" /D "MY_DLL_EXPORTS" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /Gm /EHsc /RTC1 /MTd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Fp"x64\Debug\MyDll.pch" /Fa"x64\Debug\" /Fo"x64\Debug\" /Fd"x64\Debug\vc100.pdb" /Gd /errorReport:queue
DLL Linker flags:
/OUT:"x64\Debug\MyDll.dll" /INCREMENTAL:NO /NOLOGO /DLL "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /MANIFEST /ManifestFile:"x64\Debug\MyDll.dll.intermediate.manifest" /ALLOWISOLATION /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"c:\Tmp\MyDll\x64\Debug\MyDll.pdb" /SUBSYSTEM:CONSOLE /PGD:"c:\Tmp\MyDll\x64\Debug\MyDll.pgd" /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X64 /ERRORREPORT:QUEUE
I executed the MATLAB code using different values of size_in_bytes:
size_in_bytes = 1000000: Pass!
size_in_bytes = 10000000: Pass!
size_in_bytes = 50000000: Pass!
size_in_bytes = 60000000: Pass!
size_in_bytes = 70000000: Stack Overflow!
Looks like the limit in my system is about 64MByte (but I don't know if this number is true for all systems).
I tried to modify stack size of Matlab.exe using editbin tool.
I tried the following command (for example):
editbin /STACK:250000000 "c:\Program Files\MATLAB\R2016a\bin\matlab.exe".
This option sets the size of the stack in bytes and takes arguments in decimal or C-language notation. The /STACK option applies only to an executable file.
It seems to have no affect...

Answer 1

Seems that on windows the size of the stack is set at compile time. So you can use option /F or the binary EDITBIN.
For example, you could to edit the following file:
EDITBIN /STACK:134217728 "C:\Program Files\MATLAB\R2016a\bin\win64\MATLAB.exe"
This would set the stack size to 128 MB (128 x 1024 x 1024 Bytes = 134217728 Bytes).
Note: be aware that editing the C:\Program Files\MATLAB\R2016a\bin\matlab.exe will have no effect.