I can find plenty of examples of developers complaining that a big array initialized on the stack create a stack overflow error
int main(int argc, const char * argv[])
{
int v[100000000];
memset(&v, 0, sizeof(v));
}
When compiling on Apple LLVM 7.0, this does not cause a stack overflow, this puzzles me as the array has a size of ~400Mb, significantly more than what is usually the size of the stack.
Why does the above code not cause stack overflow?
Since you are not using v then probably the compiler is not allocating it, try something like
int v[100000000];
for (int i = 0 ; i < sizeof(v) / sizeof(*v) ; ++i)
v[i] = 0;
Your array is more than 100 Mb (*) but assuming it is 100 Mb, that means that either your stack size is larger than 100 Mb, or either your compiler ignored it because you do not use it. That's a compiler optimization.
(*) Indeed 1M = 1024 * 1024, not 1000 * 1000. And one int is more than 1 bit, more than 1 Byte too. And finally, Mb means Megabit and MB means Megabyte.
Related
This question already has answers here:
Getting a stack overflow exception when declaring a large array
(8 answers)
Closed 10 months ago.
the following code caused segmentation fault (core dumped) with 1000000000 times loop.
but by reducing the looping time to 100000, it goes ok.
so is it causing any thing wrong in cpu, hardware, or anywhere? is it caused by watchdog timer?
can anybody help to explain it for this? what happened when cpu goes to huge loops(finite loops with huge number repeating)? how does cpu tell the computing is infinite? many thanks.
#include <stdio.h>
int main () {
int a[1000000000];
int i = 0;
for (i = 0;i < 1000000000; i++){
if(i % 4 == 0){
a[i] = i;
}else {
a[i] = 321;
}
}
printf("run over");
return 0;
}
The overflowing of the stack is observed here. 1000000000 * sizeof(int) memory is supposed to be there for storing this array. In short, the problem is coming from the size of the array not the number iterations.
You can either make the array static or dynamically allocate the memory.
Are you perhaps running out of memory ? Your 1 billion int array weighs 30 Gb if using 32bit ints.
This happened because stack has a memory limit. Your array will occupy a total of 1000000000 * sizeof(int) bytes, which will be equal to 3.725 Gigabytes on 64 bit machine.
You need to dynamically store the array on the heap memory like this:
int *array = malloc(1000000000 * sizeof(int));
Or, better break your array into several parts and process them and after processing store those results on a hard disk.
Also, you can see the maximum stack size on linux by using ulimit:
ulimit -s # stack size
ulimit -a # full details
I am trying to get code that was working on Linux to also work on my Windows 7.
When I retried the same code, it crashed with stack overflow. I then removed everything I could to find out the line which is causing it to crash, and it left me with this:
#include <stdio.h>
#include <stdlib.h>
#include <cuda_runtime.h>
/* 256k == 2^18 */
#define ARRAY_SIZE 262144
#define ARRAY_SIZE_IN_BYTES (sizeof(float) * (ARRAY_SIZE))
int main(void)
{
float a[ARRAY_SIZE] = { };
float result = 0;
printf("sum was: %f (should be around 1 300 000 with even random distribution)\n", result);
return 0;
}
If I change ARRAY_SIZE to 256, the code runs fine. However with the current value, the float a[ARRAY_SIZE] line crashes runtime with stack overflow. It doesn't matter if I use float a[ARRAY_SIZE]; or float a[ARRAY_SIZE] = { };, they both crash the same way.
Any ideas what could be wrong?
Using Visual Studio 2010 for compiling.
Ok, the stack sizes seem to be explained here, saying 1M is the default on Windows.
Apparently it can be increased in VS 2010 by going Properties -> Linker -> System -> Stack Reserve Size and giving it some more. I tested and the code works by pumping up the stack to 8M.
In the long run I should probably go the malloc way.
Your array is too large to fit into the stack, try using the heap:
float *a;
a = malloc(sizeof(float) * ARRAY_SIZE);
Segmentation fault when allocating large arrays on the stack
Well, let me guess. I've heard default stack size on Windows is 1 MB. Your ARRAY_SIZE_IN_BYTES is exactly 1 MB btw (assuming float is 4 bytes). So probably that's the reason
See this link: C/C++ maximum stack size of program
I have a toy cipher program which is encountering a bus error when given a very long key (I'm using 961168601842738797 to reproduce it), which perplexes me. When I commented out sections to isolate the error, I found it was being caused by this innocent-looking for loop in my Sieve of Eratosthenes.
unsigned long i;
int candidatePrimes[CANDIDATE_PRIMES];
// CANDIDATE_PRIMES is a macro which sets the length of the array to
// two less than the upper bound of the sieve. (2 being the first prime
// and the lower bound.)
for (i=0;i<CANDIDATE_PRIMES;i++)
{
printf("i: %d\n", i); // does not print; bus error occurs first
//candidatePrimes[i] = PRIME;
}
At times this has been a segmentation fault rather than a bus error.
Can anyone help me to understand what is happening and how I can fix it/avoid it in the future?
Thanks in advance!
PS
The full code is available here:
http://pastebin.com/GNEsg8eb
I would say your VLA is too large for your stack, leading to undefined behaviour.
Better to allocate the array dynamically:
int *candidatePrimes = malloc(CANDIDATE_PRIMES * sizeof(int));
And don't forget to free before returning.
If this is Eratosthenes Sieve, then the array is really just flags. It's wasteful to use int if it's just going to hold 0 or 1. At least use char (for speed), or condense to a bit array (for minimal storage).
The problem is that you're blowing the stack away.
unsigned long i;
int candidatePrimes[CANDIDATE_PRIMES];
If CANDIDATE_PRIMES is large, this alters the stack pointer by a massive amount. But it doesn't touch the memory, it just adjusts the stack pointer by a very large amount.
for (i=0;i<CANDIDATE_PRIMES;i++)
{
This adjusts "i" which is way back in the good area of the stack, and sets it to zero. Checks that it's < CANDIDATE_PRIMES, which it is, and so performs the first iteration.
printf("i: %d\n", i); // does not print; bus error occurs first
This attempts to put the parameters for "printf" onto the bottom of the stack. BOOM. Invalid memory location.
What value does CANDIDATE_PRIMES have?
And, do you actually want to store all the primes you're testing or only those that pass? What is the purpose of storing the values 0 thru CANDIDATE_PRIMES sequentially in an array???
If what you just wanted to store the primes, you should use a dynamic allocation and grow it as needed.
size_t g_numSlots = 0;
size_t g_numPrimes = 0;
unsigned long* g_primes = NULL;
void addPrime(unsigned long prime) {
unsigned long* newPrimes;
if (g_numPrimes >= g_numSlots) {
g_numSlots += 256;
newPrimes = realloc(g_primes, g_numSlots * sizeof(unsigned long));
if (newPrimes == NULL) {
die(gracefully);
}
g_primes = newPrimes;
}
g_primes[g_numPrimes++] = prime;
}
I have written the code for the sieve but the program runs for only array size less than or equal to 1000000. For the rest of the cases which are larger, a simple SIGSEGV occurs. Can this be made to run cases > 1000000. Or where am I wrong?
#include <stdio.h>
int main()
{
unsigned long long int arr[10000001] = {[0 ... 10000000] = 0};
unsigned long long int c=0,i,j,a,b;
scanf("%llu%llu",&a,&b);
for(i=2;i<=b;i++)
if(arr[i] == 0)
for(j=2*i;j<=b;j+=i)
arr[j] = 1;
for(i=(a>2)?a:2;i<=b;i++)
if(arr[i] == 0)``
c++;
printf("%llu",c);
return 0;
}
This line allocates memory on the stack (which is a limited resource)
unsigned long long int arr[10000001] = {[0 ... 10000000] = 0};
If you are allocating 10,000,000 entries at 4 bytes each, that is 40 million bytes, which will be more than your stack can handle.
(or, on your platform, there is a good chance that a long-long-int is 8 or more bytes, indicating 80 million bytes in use!)
Instead, allocate the memory from the heap, which is much more plentiful:
int* arr = malloc(10,000,000 * sizeof(int)); // commas for clarity only. Remove in real code!
Or, if you want the memory initialized to zero, use calloc.
Then at the end of your program be sure you also free it:
free(arr);
PS The syntax {[0 ... 10000000] = 0}; is needlessly verbose.
To initialize an array to zero, simply:
int arr[100] = {0}; // Thats all!
You declared an array that can hold 10000001 items; if you want to handle larger numbers, you need a bigger array. I'm mildly surprised that it works for 1000000 already - that's a lot of stack space to be using.
Edit: sorry - didn't notice you had a different number of zeroes there. Don't use the stack to allocate your array and you should be fine. Just add static to the array declaration and you'll probably be okay.
I've noticed that a few of my classmates have actually tried asking questions about this same assignment on StackOverflow over the past few days so I'm going to shamelessly copy paste (only) the context of one question that was deleted (but still cached on Google with no answers) to save time. I apologize in advance for that.
Context
I am trying to write a C program that measures the data throughput (MBytes/sec) of the L2 cache of my system. To perform the measurement I have to write a program that copies an array A to an array B, repeated multiple times, and measure the throughput.
Consider at least two scenarios:
Both fields fit in the L2 cache
The array size is significantly larger than the L2 cache size.
Using memcpy() from string.h to copy the arrays, initialize both arrays with some values (e.g. random numbers using rand()), and repeat at least 100 times, otherwise you do not see a difference.
The array size and number of repeats should be input parameters. One of the array sizes should be half of my L2 cache size.
Question
So based on that context of the assignment I have a good idea of what I need to do because it pretty much tells me straight out. The problem is that we were given some template code to work with and I'm having trouble deciphering parts of it. I would really appreciate it if someone would help me to just figure out what is going on.
The code is:
/* do not add other includes */
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
#include <time.h>
#include <string.h>
double getTime(){
struct timeval t;
double sec, msec;
while (gettimeofday(&t, NULL) != 0);
sec = t.tv_sec;
msec = t.tv_usec;
sec = sec + msec/1000000.0;
return sec;
}
/* for task 1 only */
void usage(void)
{
fprintf(stderr, "bandwith [--no_iterations iterations] [--array_size size]\n");
exit(1);
}
int main (int argc, char *argv[])
{
double t1, t2;
/* variables for task 1 */
unsigned int size = 1024;
unsigned int N = 100;
unsigned int i;
/* declare variables; examples, adjust for task */
int *A;
int *B;
/* parameter parsing task 1 */
for(i=1; i<(unsigned)argc; i++) {
if (strcmp(argv[i], "--no_iterations") == 0) {
i++;
if (i < argc)
sscanf(argv[i], "%u", &N);
else
usage();
} else if (strcmp(argv[i], "--array_size") == 0) {
i++;
if (i < argc)
sscanf(argv[i], "%u", &size);
else
usage();
} else usage();
}
/* allocate memory for arrays; examples, adjust for task */
A = malloc (size*size * sizeof (int));
B = malloc (size*size * sizeof (int));
/* initialise arrray elements */
t1 = getTime();
/* code to be measured goes here */
t2 = getTime();
/* output; examples, adjust for task */
printf("time: %6.2f secs\n",t2 - t1);
/* free memory; examples, adjust for task */
free(B);
free(A);
return 0;
}
My questions are:
What could the purpose of the usage method be?
What is the parameter passing part supposed to be doing because as far as I can tell it will just always lead to usage() and won't take any parameters with the sscanf lines?
In this assignment we're meant to record array sizes in KB or MB, and I know that malloc allocates size in bytes and with a size variable value of 1024 would result in 1MB * sizeof(int) (I think at least). In this case would the array size I should record be 1MB or 1MB * sizeof(int)?
If parameter passing worked properly and we passed parameters to change the size variable value would the array size always be the size variable squared? Or would the array size be considered to be just the size variable? It seems very unintuitive to malloc size*size instead of just size unless there's something I'm missing about all this.
My understanding of measuring the throughput is that I should just multiply the array size by the number of iterations and then divide by the time taken. Can I get any confirmation that this is right?
These are the only hurdles in my understanding of this assignment. Any help would be much appreciated.
What could the purpose of the usage method be?
The usage function tells you what arguments are supposed to be passed to the program on the command-line.
What is the parameter passing part supposed to be doing because as far as I can tell it will just always lead to usage() and won't take any parameters with the sscanf lines?
It leads the calling the usage() function when an invalid argument is passed to the program.
Otherwise, it sets the number of iterations to the variable N to the value of the argument no_iterations (default value of 100), and it sets the size of the array to the variable size to the value of the argument array_size (default value of 1024).
In this assignment we're meant to record array sizes in KB or MB, and I know that malloc allocates size in bytes and with a size variable value of 1024 would result in 1MB * sizeof(int) (I think at least). In this case would the array size I should record be 1MB or 1MB * sizeof(int)?
If your size is supposed to be 1 MB, then that is probably what the size should be.
If you want to make it sure the size is a factor of the size of the data type, then you can do:
if (size % sizeof(int) != 0)
{
size = ((int)(size / sizeof(int))) * sizeof(int);
}
If parameter passing worked properly and we passed parameters to change the size variable value would the array size always be the size variable squared? Or would the array size be considered to be just the size variable? It seems very unintuitive to malloc size*size instead of just size unless there's something I'm missing about all this.
You probably just want to allocate size bytes. Unless you are supposed to be working with matrices, rather than just arrays. In that case, it would be size * size bytes.
My understanding of measuring the throughput is that I should just multiply the array size by the number of iterations and then divide by the time taken. Can I get any confirmation that this is right?
I guess so.