I would like to declare and initialize a large 3D array on stack. The c function declares the large 3D array as:
#define NMATS 36
#define ROWS 10000
#define COLS 9
void myfunc(void)
{
double mat[NMATS][ROWS][COLS];
// Initialize later ...
}
In VS Code, the command cl.exe /Zi /EHsc /Fe: C:\Users\usr\project\c\build\main.exe c:\Users\usr\project\c\src\main.c successfully builds the code. However, during runtime I get the error:
Unable to open 'chkstk.asm': File not found.
This indicates that my Stack Reserve Size is too small. However, I am relatively new to VS Code and would like to know how to increase the stack reserve size and specify the option for cl.exe.
As mentioned in the comments, do a dynamic allocation so you don't have to rely on compiler switches and architecture limitations:
#define NMATS 36
#define ROWS 10000
#define COLS 9
typedef double MAT[NMATS][ROWS][COLS];
void myfunc(void)
{
MAT* mat = (MAT*)malloc(sizeof(MAT));
// initialize the matrix by referencing it as `*mat`
(*mat)[0][0][0] = 0;
free(mat);
}
But if you insist on needing a stack allocation:
Assuming sizeof(double)==8, 36*10000*9*sizeof(double) == 25,920,000. So you'd need at least that many bytes in stack space plus a little bit more for additional function call and local variable overhead. So let's add on another million bytes and round up to 27 million bytes.
If you really insist on having a stack allocation of 27 MB available. And I'm assuming you are using the Microsoft compiler:
Set the linker option: /STACK:27000000
The above works for all Visual Studio projects.
If your custom build step is just a single run of cl.exe (and not a separate linker step), you can have the compiler forward the command line option via the /F command parameter:
cl.exe /F27000000 /Zi /EHsc /Fe: C:\Users\usr\project\c\build\main.exe c:\Users\usr\project\c\src\main.c
https://learn.microsoft.com/en-us/cpp/build/reference/f-set-stack-size?view=msvc-170
https://learn.microsoft.com/en-us/cpp/build/reference/stack-stack-allocations?view=msvc-170
Related
This question already has answers here:
Find size of a function in C
(9 answers)
Closed 4 years ago.
I am writing code for an embedded system where it is more efficient to copy code from ROM to the SOC's internal memory then execute it. Is there any way to programmatically get the size in bytes of the function so I can use a function like memcpy to copy the function's instructions to internal memory.
Is there a better way to do this?
Is there a better way to do this?
There likely is, and I truly hope someone else responds with an answer that provides a simpler approach, but for now I'll shed some details on your suggested method.
If the program is compiled in ELF format (I don't know for the other formats), all your functions will be included in the .text section of the ELF file. You can use the symbol table to find this function in the text section. To get the size of this function, you might be able to use the st_size member of the Elf64_Sym or Elf32_Sym struct, but I'm not entirely certain that will give the correct size. What you could do (a little hacky, admittedly) is iterate through the other symbols, and find the one immediately after it, and subtract to get the size. Of course you'd have to keep in mind alignment rules, but that's not too much of an issue- if you copy extra bytes, they won't be executed anyways.
Also keep in mind that some code get compiled with certain assumptions about its offset in memory. You'll might need to manually patch the GOT and/or PLT if you copy the function directly into memory. Know you should probably compile the function you want to include with -PIC and -fPIC for position independent code, at least in GCC.
If you need more details on how to access the symbol table, or the text section of your ELF, I could add more details.
With some compilers you can get a function size by computing the difference between the function's address and the address of another function that immediatly follows the first one.
But it really depends of the compiler. With Visual C++ for example, both functions has to be static functions. With GCC, it does not work anymore if optimization O2 or better is activated.
And even if you manage to copy your function elsewhere in memory, you may not be able to use it, especially if it refers other functions, or if it refers global/static variables, or if the code is not position independant, etc.
So this is a simple solution, it may work in your case, but it can't be considered as a general solution.
Below there's an example that works with gcc and visual C++, tested on windows 10 and WSL (do not activate optimizations with gcc).
#include <stdio.h>
#include <string.h>
#ifdef _WIN32
#include <windows.h>
#endif
#ifdef __linux
#include <sys/mman.h>
#endif
// The function to copy
static int fib(int m)
{
int v1 = 0, v2 = 1, n;
if (m == 0) return 0;
for (n = 1; n < m; n++)
{
int v = v1 + v2;
v1 = v2;
v2 = v;
}
return v2;
}
static void endFib(void)
{
// This function follow immediatly the fib function
// and it exists only to get its address and compute the size of fib function
}
int main(int argc, char *argv)
{
long sizeFib;
int (*copyFib)(int);
printf("&fib=%p\n", (char *)fib);
sizeFib = (char *)endFib - (char *)fib;
printf("size of fib : %ld\n", sizeFib);
printf("fib(8) : %d\n", fib(8));
// For the example the allocated copy must be in an executable part of the memory.
#ifdef _WIN32
copyFib = VirtualAlloc(NULL, sizeFib, MEM_COMMIT, PAGE_EXECUTE_READWRITE);
#endif
#ifdef __linux
copyFib = mmap(NULL, sizeFib, PROT_EXEC | PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
#endif
memcpy(copyFib, fib, sizeFib);
printf("©Fib=%p\n", copyFib);
printf("copyFib(8) : %d\n", copyFib(8));
return 0;
}
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have not touched C for long really long time. My first language was C. But then we have been taught C++, Java and C# (in college days). Now mostly my work involve Java and groovy. And suddenly I have to do C again. I am not familiar with how industry uses C as I never was in project doing stuff in C.
I started in eclipse CDT, with MinGW on Windows 10. My new program grew big quickly. And my lack of experience was unrevealed to me. When I run, my program used to crash showing Windows dialog saying "MyProgram.exe has stopped working". Now I had no clue whats going wrong. Compilation was clean with no error, but only one warning. Now since from Java world, I was like warnings are not that fatal. So I simply just ignored it (I thats my lack of C experience). And went on debugging found the cause. Realised that the warning was indeed about the cause. Suffered mental frustation of wasting hours in debugging.
So here is code replicating issue in my original code:
1 #include "stdio.h"
2 #include "limits.h"
3
4 typedef struct TempStruct TempStruct;
5
6 struct TempStruct
7 {
8 int a;
9 TempStruct *next;
10 };
11
12 int function1(TempStruct *param)
13 {
14 return param == NULL;
15 }
16
17 int function2(TempStruct **param)
18 {
19 if(function1(param))
20 {
21 return INT_MIN;
22 }
23 *param = (*param)->next;
24 return 0;
25 }
26
27 int main()
28 {
29 TempStruct *tempStructObj = NULL;
30 function2(&tempStructObj);
31 printf("Does not reach here!!!");
32 return 0;
33 }
My C-noob eyes did not see anything wrong in it. Good that I knew how to do debugging. I got following in debugging:
In main() this is done: *tempStructObj = NULL. So, I was expecting function1() to return 1, making function2() returning from line 21.
The issue was that function1() takes TempStruct*. But on line 19, I passed it **param. So inside function1(), param wasnt NULL. So it returned false (0 I guess). So function2() did not returned from line 21. It executed (*param)->next and hence the program crashed.
My questions:
Q1. Shouldn't be such issue be reported as Error's instead of Warnings? Is there any setting which can report such potentially fatal warnings to error?
Q2. Does eclipse logs the reasons of such sudden app crash somewhere? So instead of debugging by stepping through each line, I can simply refer to the report which can possibly specify the line number which caused the crash
Q3. What is industry-standard approach to deal with such mistakes? Of course one will say dont commit the mistake. But I am asking about precautions that are taken to avoid such mistakes or auto detect them. Like above I asked about settings to make eclipse report the issue as fatal one or making crashes to generate report so that stuff can be fixed quickly instead of hours long debuggins. Do you use any better (and possibly involving smaller learning curve) alternative to (eclipse CDT + MinGW + Windows) that will provide more powerful debugging so that I can avoid such errors.
Q4. In point 1 of above diagram, what is that Error: Multiple errors reported.... This stuff occurred occasionally, but not always, say once in 5 debugging sessions. What can be the reason behind such ad hoc behavior?
Q5. In point 3 of above diagram, what is that (0x62ff2c) value of param inside function1()? If I keep signature of function1() correctly as int function1(TempStruct **param) and change inside reference correctly to *param, *param is correctly 0x0 (i.e. NULL):
Edit
This on Ideone works (with C) & prints "Does not reach here!!!". So dont know how it handled (*param)->next.
This on ideone (with C99 Strict) does gives error (not warning).
Q1. No, they are warnings as they are legit C code. It could be possible that you want such code. You can use -Werror on gcc to make warnings to errors. Also add some other flags for turning on more warnings like -Wall -Wpedantic -Wextra -Wshadow -Wconversion -Wno-sign-compare etc. This'd be a bit closer to what you're probably used to when using Java ;-)
Q2. Atleast on Linux you have coredumps, iirc Windows was minidumps. These can be loaded together with the corresponding executable into a debugger. Then you can access backtraces, values etc.
Q3.
Like above I asked about settings to make eclipse report the issue as fatal one or making crashes to generate report so that stuff can be fixed quickly instead of hours long debuggins.
Log yourself. Also there can be macros for easing this.
Do you use any better (and possibly involving smaller learning curve) alternative to (eclipse CDT + MinGW + Windows) that will provide more powerful debugging so that I can avoid such errors.
IDE is pretty irrelevant for C imho. Use Linux with native GCC instead, MinGW is nice but it can be daunting (my experience).
Ofcourse MS VSC++ can also compile C but its just for C++ compatible thus not really specific to one standard.
Q4. Well, multiple errors occured which are listed. If it's difficult to reproduce it might be a problem in your setup, this is exactly the experience I had with MinGW on Windows.
Q5. It's the address -- you have a pointer to a pointer, so the first ("outer") pointer is that address, pointing to another pointer which is NULL.
Or more verbosely:
tempStructObj is a pointer to NULL (ie. an int_ptr which holds the value 0x0.
To function2 you pass another int_ptr which holds the semi-random value/address of the automatic variable int_ptr tempStructObj is stored
Ie. you have such:
Address &tempStructObj: tempStructObj
in the RAM.
When you then call function1, you pass the value of this (not-NULL) pointer. Of course the comparison is thus always false.
You'd need to compare
*param with NULL.
Even more:
If you compile with GCC (on Linux) and use really verbose flags you get:
gcc -std=c99 -Wall -Wpedantic -Wextra -Wshadow -Wconversion -Wno-sign-compare -o main main.c
main.c: In function ‘function2’:
main.c:19:18: warning: passing argument 1 of ‘function1’ from incompatible pointer type [-Wincompatible-pointer-types]
if(function1(param))
^
main.c:12:5: note: expected ‘TempStruct * {aka struct TempStruct *}’ but argument is of type ‘TempStruct ** {aka struct TempStruct **}’
int function1(TempStruct *param)
^
So exactly the problem you had ^^
Also I'd remove the function1 altogether, it's completely unnecessary and just obfuscates the code. Also I'd use a different name for the struct and the typedef, appending the latter with a _t. Also I'd move it into one shorter piece of code.
On a side note: add a \n in the printf()-call.
Edited code:
#include <stdio.h>
#include <limits.h>
typedef struct TempStruct_s {
int a;
struct TempStruct_s *next;
} TempStruct_t;
int function(TempStruct_t **param)
{
if(!*param) {
return INT_MIN;
}
*param = (*param)->next;
return 0;
}
int main()
{
TempStruct_t *tempStructObj = NULL;
function(&tempStructObj);
printf("Does not reach here!!!\n");
return 0;
}
When you
TempStruct *tempStructObj = NULL;
function2(&tempStructObj);
you are sending into function2 the address of your variable tempStructObj (0x62ff2c). When doing
if(function1(param)){
return INT_MIN;
}
you're sending the same address (0x62ff2c). Therefore, param == NULL is false.
Q5: if you use the proper signature, as you suggest, then you check the value that tempStructObj is pointing at, which is what you want, and everything works.
The error you get about not being able to access memory 0x4 is due to your structure and wrong null checking. (*param)->next is expected work on a memory zone with an int and another pointer. However, *param is pointing at address 0x0, so the int is at address 0x0 and the next pointer is at address 0x4, hence the error.
I have a program problem for which I would like to declare a 256x256 array in C. Unfortunately, I each time I try to even declare an array of that size (integers) and I run my program, it terminates unexpectedly. Any suggestions? I haven't tried memory allocation since I cannot seem to understand how it works with multi-dimensional arrays (feel free to guide me through it though I am new to C). Another interesting thing to note is that I can declare a 248x248 array in C without any problems, but no larger.
dims = 256;
int majormatrix[dims][dims];
Compiled with:
gcc -msse2 -O3 -march=pentium4 -malign-double -funroll-loops -pipe -fomit-frame-pointer -W -Wall -o "SkyFall.exe" "SkyFall.c"
I am using SciTE 323 (not sure how to check GCC version).
There are three places where you can allocate an array in C:
In the automatic memory (commonly referred to as "on the stack")
In the dynamic memory (malloc/free), or
In the static memory (static keyword / global space).
Only the automatic memory has somewhat severe constraints on the amount of allocation (that is, in addition to the limits set by the operating system); dynamic and static allocations could potentially grab nearly as much space as is made available to your process by the operating system.
The simplest way to see if this is the case is to move the declaration outside your function. This would move your array to static memory. If crashes continue, they have nothing to do with the size of your array.
Unless you're running a very old machine/compiler, there's no reason that should be too large. It seems to me the problem is elsewhere. Try the following code and tell me if it works:
#include <stdio.h>
int main()
{
int ints[256][256], i, j;
i = j = 0;
while (i<256) {
while (j<256) {
ints[i][j] = i*j;
j++;
}
i++;
j = 0;
}
printf("Made it :) \n");
return 0;
}
You can't necessarily assume that "terminates unexpectedly" is necessarily directly because of "declaring a 256x256 array".
SUGGESTION:
1) Boil your code down to a simple, standalone example
2) Run it in the debugger
3) When it "terminates unexpectedly", use the debugger to get a "stack traceback" - you must identify the specific line that's failing
4) You should also look for a specific error message (if possible)
5) Post your code, the error message and your traceback
6) Be sure to tell us what platform (e.g. Centos Linux 5.5) and compiler (e.g. gcc 4.2.1) you're using, too.
I have this code:
#include <stdio.h>
#include <math.h>
#define gridSize 400
void main() {
float grid[gridSize][gridSize];
short height[gridSize][gridSize];
short power[gridSize][gridSize];
}
I'm using visual studio 2010, the program seems to crash instantly when I run it. However this code:
#include <stdio.h>
#include <math.h>
#define gridSize 400
void main() {
float grid[gridSize][gridSize];
short height[gridSize][gridSize];
//short power[gridSize][gridSize];
}
Seems to work fine, and the program doesn't crash. What could be the problem?
Here grid height and power are auto variable and going to store in stack.
In any Os each process has some fixed default size stack.
Here you are allocating too much data on stack so process has no other memory left on stack for other operation. so it crash
you have two option
1> Increase stack size for this process
On Linux with gcc you can increase it by
–stack 16777216
adding this in gcc command
2> you can store this data on heap section by using malloc.
You're allocating too much stack. Move one or more into heap instead.
Just read the name of this website, stack overflow.
You can:
1, move those three arrays out of main function(maybe you will get a large .exe after compilation if you initialize those arrays).
or
2, use malloc().
A program has three sections: text, data and stack. The function body lives in the text section. Can we let a function body live on heap? Because we can manipulate memory on heap more freely, we may gain more freedom to manipulate functions.
In the following C code, I copy the text of hello function onto heap and then point a function pointer to it. The program compiles fine by gcc but gives "Segmentation fault" when running.
Could you tell me why?
If my program can not be repaired, could you provide a way to let a function live on heap?
Thanks!
Turing.robot
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
void
hello()
{
printf( "Hello World!\n");
}
int main(void)
{
void (*fp)();
int size = 10000; // large enough to contain hello()
char* buffer;
buffer = (char*) malloc ( size );
memcpy( buffer,(char*)hello,size );
fp = buffer;
fp();
free (buffer);
return 0;
}
My examples below are for Linux x86_64 with gcc, but similar considerations should apply on other systems.
Can we let a function body live on heap?
Yes, absolutely we can. But usually that is called JIT (Just-in-time) compilation. See this for basic idea.
Because we can manipulate memory on heap more freely, we may gain more freedom to manipulate functions.
Exactly, that's why higher level languages like JavaScript have JIT compilers.
In the following C code, I copy the text of hello function onto heap and then point a function pointer to it. The program compiles fine by gcc but gives "Segmentation fault" when running.
Actually you have multiple "Segmentation fault"s in that code.
The first one comes from this line:
int size = 10000; // large enough to contain hello()
If you see x86_64 machine code generated by gcc of your
hello function, it compiles down to mere 17 bytes:
0000000000400626 <hello>:
400626: 55 push %rbp
400627: 48 89 e5 mov %rsp,%rbp
40062a: bf 98 07 40 00 mov $0x400798,%edi
40062f: e8 9c fe ff ff call 4004d0 <puts#plt>
400634: 90 nop
400635: 5d pop %rbp
400636: c3 retq
So, when you are trying to copy 10,000 bytes, you run into a memory
that does not exist and get "Segmentation fault".
Secondly, you allocate memory with malloc, which gives you a slice of
memory that is protected by CPU against execution on Linux x86_64, so
this would give you another "Segmentation fault".
Under the hood malloc uses system calls like brk, sbrk, and mmap to allocate memory. What you need to do is allocate executable memory using mmap system call with PROT_EXEC protection.
Thirdly, when gcc compiles your hello function, you don't really know what optimisations it will use and what the resulting machine code looks like.
For example, if you see line 4 of the compiled hello function
40062f: e8 9c fe ff ff call 4004d0 <puts#plt>
gcc optimised it to use puts function instead of printf, but that is
not even the main problem.
On x86 architectures you normally call functions using call assembly
mnemonic, however, it is not a single instruction, there are actually many different machine instructions that call can compile to, see Intel manual page Vol. 2A 3-123, for reference.
In you case the compiler has chosen to use relative addressing for the call assembly instruction.
You can see that, because your call instruction has e8 opcode:
E8 - Call near, relative, displacement relative to next instruction. 32-bit displacement sign extended to 64-bits in 64-bit mode.
Which basically means that instruction pointer will jump the relative amount of bytes from the current instruction pointer.
Now, when you relocate your code with memcpy to the heap, you simply copy that relative call which will now jump the instruction pointer relative from where you copied your code to into the heap, and that memory will most likely not exist and you will get another "Segmentation fault".
If my program can not be repaired, could you provide a way to let a function live on heap? Thanks!
Below is a working code, here is what I do:
Execute, printf once to make sure gcc includes it in our binary.
Copy the correct size of bytes to heap, in order to not access memory that does not exist.
Allocate executable memory with mmap and PROT_EXEC option.
Pass printf function as argument to our heap_function to make sure
that gcc uses absolute jumps for call instruction.
Here is a working code:
#include "stdio.h"
#include "string.h"
#include <stdint.h>
#include <sys/mman.h>
typedef int (*printf_t)(char* format, char* string);
typedef int (*heap_function_t)(printf_t myprintf, char* str, int a, int b);
int heap_function(printf_t myprintf, char* str, int a, int b) {
myprintf("%s", str);
return a + b;
}
int heap_function_end() {
return 0;
}
int main(void) {
// By printing something here, `gcc` will include `printf`
// function at some address (`0x4004d0` in my case) in our binary,
// with `printf_t` two argument signature.
printf("%s", "Just including printf in binary\n");
// Allocate the correct size of
// executable `PROT_EXEC` memory.
size_t size = (size_t) ((intptr_t) heap_function_end - (intptr_t) heap_function);
char* buffer = (char*) mmap(0, (size_t) size,
PROT_EXEC | PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
memcpy(buffer, (char*)heap_function, size);
// Call our function
heap_function_t fp = (heap_function_t) buffer;
int res = fp((void*) printf, "Hello world, from heap!\n", 1, 2);
printf("a + b = %i\n", res);
}
Save in main.c and run with:
gcc -o main main.c && ./main
In principle in concept it is doable. However... You are copying from "hello" which basically contains assembly instructions that possibly call or reference or jump to other addresses. Some of these addresses get fixed up when the application loads. Just copying that and calling into it would then crash. Also some systems like windows have data execution protection that would prevent code in data form being executed, as a security measure. Also, how large is "hello"? Trying to copy past the end of it would likely also crash. And you are also dependent on how the compiler implements "hallo". Needless to say, this would be very compiler and platform dependent, if it worked.
I can imagine that this might work on a very simple architecture or with a compiler designed to make it easy.
A few of the many requirements for this work:
All memory references would need to be absolute ... no pc-relative addresses, except . . .
Certain control transfers would need to be pc-relative (so your copied function's local branches work) but it would be nice if other ones would just happen to be absolute, so your module's external control transfers, like printf(), would work.
There are more requirements. Add to this the wierdness of doing this in what is likely to already be a highly complex dynamically linked environment (did you static link it?) and you simply are not ever going to get this to work.
And as Adam points out, there are security mechanisms in place, at least for the stack, to prevent dynamically constructed code from executing at all. You may need to figure out how to turn these off.
You might also be getting clobbered with the memcpy().
You might learn something by tracing this through step-by-step and watching it shoot itself in the head. If the memcpy hack is the problem, perhaps try something like:
f() {
...
}
g() {
...
}
memcpy(dst, f, (intptr_t)g - (intptr_t)f)
You program is segfaulting because you're memcpy'ing more than just "hello"; that function is not 10000 bytes long, so as soon as you get past hello itself, you segfault because you're accessing memory that doesn't belong to you.
You probably also need to use mmap() at some point to make sure the memory location you're trying to call is actually executable.
There are many systems that do what you seem to want (e.g., Java's JIT compiler creates native code in the heap and executes it), but your example will be way more complicated than that because there's no easy way to know the size of your function at runtime (and it's even harder at compile time, when the compiler hasn't yet decide what optimizations to apply). You can probably do what objdump does and read the executable at runtime to find the right "size", but I don't think that's what you're actually trying to achieve here.
After malloc you should check that the pointer is not null buffer = (char*) malloc ( size );
memcpy( buffer,(char*)hello,size ); and it might be your problem since you try to allocate a big area in memory. can you check that?
memcpy( buffer,(char*)hello,size );
hello is not a source get copied to buffer. You are cheating the compiler and it is taking it's revenge at run-time. By typecasting hello to char*, the program is making the compiler to believe it so, which is not the case actually. Never out-smart the compiler.