When I run the following code in C language, my compiler shows the error "xxx has stopped working ".
However, when I take array sizes as 1000 instead of 100000 it runs fine. What is the problem and how can I fix it? If there is some memory problem then how can I take input of 100000 numbers in these arrays without exceeding it?
Code I tried :
int main()
{
int a[100000],l[100000],r[100000],ans[100000],x[100000],y[100000];
/*
some code
*/
return 0;
}
Declare a, l, r, ans, x and y as global variables so that they will be allocated in the heap instead of the stack.
int a[100000], l[100000], r[100000], ans[100000], x[100000], y[100000];
int main()
{
The stack is typically a limited resource. Use dynamic allocation (such as malloc) instead.
Most systems limits the stack to something between one and four megabytes. Since your arrays are well over 2MB you are most likely going over the stack limit of your system.
In C there are a couple of ways to solve that problem:
Make the arrays global
Make the arrays static
Dynamically allocate the memory for them of the heap (e.g. malloc and friends)
Simply make the arrays smaller
Welcome in stackoverflow ;)
use dynamic allocation (malloc/free) in order to use all your ram capacities.
Most systems have a limited stack size and since your arrays are local(automatic) variables they will be allocated on the stack, so you are very likely overflowing the stack. If you need to allocated large arrays malloc is going to be the better choice.
Related
I have been writing a program where I have a 2d array that changes size if the user wants, as follows:
#include <stdlib.h>
#include <stdio.h>
int max_length = 1024;
int memory_length = 16;
int block_length = 64;
void process_input(int memory[memory_length][block_length], char* user_input) {
...
}
int main(void) {
printf("Not sure what to do? Enter 'help'\n");
while (0 == 0) {
int memory[memory_length][block_length];
char user_input[max_length];
printf(">> ");
fgets(user_input, max_length, stdin);
printf("\n");
process_input(memory, user_input);
if (user_input[0] == 'e' && user_input[1] == 'n' && user_input[2] == 'd') {
break;
}
printf("\n");
}
return 0;
}
NOTE: The process_input() function that I made allows the user to play around with the values inside the array 'memory' as well as change the value of memory_length or block_length, hence then changing the length of the array. After the user is done the cycle repeats with a fresh array.
I can use the 2d array perfectly fine, parsing it to any function. However one day I discover that there are functions such as malloc() that allow you to dynamically allocate memory through a pointer. This made me then question:
Should I re-write my whole very complicated program to use malloc and other 'memory functions', or is it okay to keep it this way?
Also as a side question that might be answered by answering the main question:
Every time I declare the 2d array, does the previous contents of the array get free, or do I keep filling up my memory like an amateur?
Finally if there is anything else that you may notice in the code or in my writing please let me know.
Thanks.
Should I re-write my whole very complicated program to use malloc and other 'memory functions', or is it okay to keep it this way?
Probably rewrite it indeed. int memory[memory_length][block_length]; in main() is a variable-length array (VLA). It is allocated with automatic storage and gets the size of those size variables at the point where its declaration is encountered, then it can't be resized from there.
For some reason a lot of beginners seem to think you can resize the VLA by changing the variables originally used to determine it's size, but no such magic relation between the VLA and those variables exists. How to declare variable-length arrays correctly?
The only kind of array in C that allows run-time resizing is one which was allocated dynamically. The only alternative to that is to allocate an array "large enough" and then keep track of how much of the array you actively are using - but it will sit there in memory (and that is likely no big deal).
However, it is not recommended to allocate huge arrays with automatic storage, since those usually end up on the stack and can cause stack overflows. Use either static storage duration or allocated storage (with malloc etc).
Every time I declare the 2d array, does the previous contents of the array get free, or do I keep filling up my memory like an amateur?
You can only declare it once. In case you do so inside a local scope, with automatic storage duration, it does indeed get cleared up every time you leave the scope which it was declared. But that also means that it can't be used outside that scope.
Finally if there is anything else that you may notice in the code or in my writing please let me know.
Yes, get rid of the global variables. There is no reason to use them in this example, avoid them like the plague. For example a function using an already allocated array might pass the sizes along, like in this example:
void process_input (size_t memory_length,
size_t block_length,
int memory[memory_length][block_length],
char* user_input)
In C, local variables, i.e. variables declared within a function, are allocated on the stack. They are only allocated once when the function is first called. The fact that you can declare variables within a while loop can lead to some confusion. The loop does not somehow allocate the memory again and again.
The memory allocated for all local variables is released when the function return.
The main reason that you might want declare a variable inside a loop (besides convenience) is to limit the scope of the variable. In your code above, you cannot access the "memory" variable outside of the while loop. You can easily check this for yourself. Your compiler should raise an error.
Whether the stack or the heap contains more memory depends on your computer architecture. In an embedded system you can often specify whether to allocate more or less memory to the heap or the stack. On a computer with virtual memory, such as a PC, the size of the heap and the stack are only limited by the size of your hard drive and the address space.
Allocating arrays on the heap is not as simple as it might seem. Single dimensional arrays work just as you might imagine, but things get more complicated with multidimensional arrays, so it is probably better to stick with either a locally or statically declared array in your case.
What is the difference between defining an array of a length that is definied before runtime (depends on command line arguments) with array[size] and using malloc()? array[i] leads to the data put on the stack, malloc() uses the heap[see this stackoverflow]
So with large Data I can run into stackoverflows, but on a new machine a total of 30 chars and ints should not be problematic (Stack is around 1MB on windows according to this).
So I am probably missing something obvious.
As far as I understand, when defined in main(),the two should be the same:
Example 1
int size; // depends on some logic
char array[size];
Example 2
int size; // depends on some logic
array= (char *)malloc(size * sizeof(char));
free(array); // later after use
But when I use the array inside of functions and hand it over as a pointer (func(char* array)) or as an array (funct(char array[])), sometimes the gdb-debugger let's me know that the function gets handed corrupted data in #1 , using malloc() fixed the issue.
Is array[i], not okay to use when it is not determined at compile time? Is it some scoping issue? This answer has a comment suggesting such a thing, but I don't quite understand if that applies here.
I am using C99.
The main difference is that arrays declared with a fixed size are stack allocated (and references to it or its elements are valid only within its scope) while mallocd arrays are heap allocated.
Stack allocation means that variables are stored directly to the memory. Access to this memory is usually very fast as and it's allocation is done during compilation.
On the other hand, variables allocated on the heap have their memory allocated at runtime and - while accessing this memory is slower - your only limit is the size of virtual memory.
Have a look here:
https://gribblelab.org/CBootCamp/7_Memory_Stack_vs_Heap.html
Today I was helping a friend of mine with some C code, and I've found some strange behavior that I couldn't explain him why it was happening. We had TSV file with a list of integers, with an int each line. The first line was the number of lines the list had.
We also had a c file with a very simple "readfile". The first line was read to n, the number of lines, then there was an initialization of:
int list[n]
and finally a for loop of n with a fscanf.
For small n's (till ~100.000), everything was fine. However, we've found that when n was big (10^6), a segfault would occur.
Finally, we changed the list initialization to
int *list = malloc(n*sizeof(int))
and everything when well, even with very large n.
Can someone explain why this occurred? what was causing the segfault with int list[n], that was stopped when we start using list = malloc(n*sizeof(int))?
There are several different pieces at play here.
The first is the difference between declaring an array as
int array[n];
and
int* array = malloc(n * sizeof(int));
In the first version, you are declaring an object with automatic storage duration. This means that the array lives only as long as the function that calls it exists. In the second version, you are getting memory with dynamic storage duration, which means that it will exist until it is explicitly deallocated with free.
The reason that the second version works here is an implementation detail of how C is usually compiled. Typically, C memory is split into several regions, including the stack (for function calls and local variables) and the heap (for malloced objects). The stack typically has a much smaller size than the heap; usually it's something like 8MB. As a result, if you try to allocate a huge array with
int array[n];
Then you might exceed the stack's storage space, causing the segfault. On the other hand, the heap usually has a huge size (say, as much space as is free on the system), and so mallocing a large object won't cause an out-of-memory error.
In general, be careful with variable-length arrays in C. They can easily exceed stack size. Prefer malloc unless you know the size is small or that you really only do want the array for a short period of time.
int list[n]
Allocates space for n integers on the stack, which is usually pretty small. Using memory on the stack is much faster than the alternative, but it is quite small and it is easy to overflow the stack (i.e. allocate too much memory) if you do things like allocate huge arrays or do recursion too deeply. You do not have to manually deallocate memory allocated this way, it is done by the compiler when the array goes out of scope.
malloc on the other hand allocates space in the heap, which is usually very large compared to the stack. You will have to allocate a much larger amount of memory on the heap to exhaust it, but it is a lot slower to allocate memory on the heap than it is on the stack, and you must deallocate it manually via free when you are done using it.
int list[n] stores the data in the stack, while malloc stores it in the heap.
The stack is limited, and there is not much space, while the heap is much much bigger.
int list[n] is a VLA, which allocates on the stack instead of on the heap. You don't have to free it (it frees automatically at the end of the function call) and it allocates quickly but the storage space is very limited, as you have discovered. You must allocate larger values on the heap.
This declaration allocates memory on the stack
int list[n]
malloc allocates on the heap.
Stack size is usually smaller than heap, so if you allocate too much memory on the stack you get a stackoverflow.
See also this answer for further information
Assuming you have a typical implementation in your implementation it's most likely that:
int list[n]
allocated list on your stack, where as:
int *list = malloc(n*sizeof(int))
allocated memory on your heap.
In the case of a stack there is typically a limit to how large these can grow (if they can grow at all). In the case of a heap there is still a limit, but that tends to be much largely and (broadly) constrained by your RAM+swap+address space which is typically at least an order of magnitude larger, if not more.
If you are on linux, you can set ulimit -s to a larger value and this might work for stack allocation also.
When you allocate memory on stack, that memory remains till the end of your function's execution. If you allocate memory on heap(using malloc), you can free the memory any time you want(even before the end of your function's execution).
Generally, heap should be used for large memory allocations.
When you allocate using a malloc, memory is allocated from heap and not from stack, which is much more limited in size.
int array[n];
It is an example of statically allocated array and at the compile time the size of the array will be known. And the array will be allocated on the stack.
int *array(malloc(sizeof(int)*n);
It is an example of dynamically allocated array and the size of the array will be known to user at the run time. And the array will be allocated on the heap.
So,i was solving a problem on SPOJ.I have to handle large number of values of order 2^18.I am on a system with 4 GB ( 2**34 Bytes) ram. While allocating memory for my program , i observed something strange. I am using long long qualifier for storing the input in two arrays sum1[1<<17] and sum2[1<<17].So, total memory allocated by the two arrays is 2^17*8 + 2^17*8 = 2^21 Bytes.The part of code i want to refer is :
#include <stdio.h>
int main()
{
// some code
int a=17,b=17;
long long sum1[1<<a], sum2[1<<b];
// some more code
}
The problem is that whenever a+b >= 34 , the program stops working, else it works fine.I guess it is due to unavailability of large space. But if i make the two arrays global like that :
#include <stdio.h>
long long sum1[1<<18], sum2[1<<18];
int main()
{
// some code
}
It works great and doesn't bother about a+b <= 34 , as you can see it works fine for 1<<18.So what's happening under the hood.
Regards.
The local variables for your function go into the stack. Your stack size is not large enough to hold your arrays.
Additional info:
You can see your stack size with the following:
ulimit -s
Local variables are usually allocated on the stack, and most systems have a relatively small limit on the size of a stack frame. Large arrays should be either global or allocated dynamically with malloc().
Today I was helping a friend of mine with some C code, and I've found some strange behavior that I couldn't explain him why it was happening. We had TSV file with a list of integers, with an int each line. The first line was the number of lines the list had.
We also had a c file with a very simple "readfile". The first line was read to n, the number of lines, then there was an initialization of:
int list[n]
and finally a for loop of n with a fscanf.
For small n's (till ~100.000), everything was fine. However, we've found that when n was big (10^6), a segfault would occur.
Finally, we changed the list initialization to
int *list = malloc(n*sizeof(int))
and everything when well, even with very large n.
Can someone explain why this occurred? what was causing the segfault with int list[n], that was stopped when we start using list = malloc(n*sizeof(int))?
There are several different pieces at play here.
The first is the difference between declaring an array as
int array[n];
and
int* array = malloc(n * sizeof(int));
In the first version, you are declaring an object with automatic storage duration. This means that the array lives only as long as the function that calls it exists. In the second version, you are getting memory with dynamic storage duration, which means that it will exist until it is explicitly deallocated with free.
The reason that the second version works here is an implementation detail of how C is usually compiled. Typically, C memory is split into several regions, including the stack (for function calls and local variables) and the heap (for malloced objects). The stack typically has a much smaller size than the heap; usually it's something like 8MB. As a result, if you try to allocate a huge array with
int array[n];
Then you might exceed the stack's storage space, causing the segfault. On the other hand, the heap usually has a huge size (say, as much space as is free on the system), and so mallocing a large object won't cause an out-of-memory error.
In general, be careful with variable-length arrays in C. They can easily exceed stack size. Prefer malloc unless you know the size is small or that you really only do want the array for a short period of time.
int list[n]
Allocates space for n integers on the stack, which is usually pretty small. Using memory on the stack is much faster than the alternative, but it is quite small and it is easy to overflow the stack (i.e. allocate too much memory) if you do things like allocate huge arrays or do recursion too deeply. You do not have to manually deallocate memory allocated this way, it is done by the compiler when the array goes out of scope.
malloc on the other hand allocates space in the heap, which is usually very large compared to the stack. You will have to allocate a much larger amount of memory on the heap to exhaust it, but it is a lot slower to allocate memory on the heap than it is on the stack, and you must deallocate it manually via free when you are done using it.
int list[n] stores the data in the stack, while malloc stores it in the heap.
The stack is limited, and there is not much space, while the heap is much much bigger.
int list[n] is a VLA, which allocates on the stack instead of on the heap. You don't have to free it (it frees automatically at the end of the function call) and it allocates quickly but the storage space is very limited, as you have discovered. You must allocate larger values on the heap.
This declaration allocates memory on the stack
int list[n]
malloc allocates on the heap.
Stack size is usually smaller than heap, so if you allocate too much memory on the stack you get a stackoverflow.
See also this answer for further information
Assuming you have a typical implementation in your implementation it's most likely that:
int list[n]
allocated list on your stack, where as:
int *list = malloc(n*sizeof(int))
allocated memory on your heap.
In the case of a stack there is typically a limit to how large these can grow (if they can grow at all). In the case of a heap there is still a limit, but that tends to be much largely and (broadly) constrained by your RAM+swap+address space which is typically at least an order of magnitude larger, if not more.
If you are on linux, you can set ulimit -s to a larger value and this might work for stack allocation also.
When you allocate memory on stack, that memory remains till the end of your function's execution. If you allocate memory on heap(using malloc), you can free the memory any time you want(even before the end of your function's execution).
Generally, heap should be used for large memory allocations.
When you allocate using a malloc, memory is allocated from heap and not from stack, which is much more limited in size.
int array[n];
It is an example of statically allocated array and at the compile time the size of the array will be known. And the array will be allocated on the stack.
int *array(malloc(sizeof(int)*n);
It is an example of dynamically allocated array and the size of the array will be known to user at the run time. And the array will be allocated on the heap.