The below code is generated by parsed SIZE parameters which in this case anywhere you see '2512555' is a parsed SIZE parameter. The problem I have is the array is far too large and causes a Segmentation Fault.
Is there a way to overcome the Segmentation Fault without changing the SIZE parameter? This parameter can range from 0 - 400,000.
#include <stdio.h>
#include <stdlib.h>
double templ25_mem1[2512555][2512555];
int main()
{
int templ25_mem1_index1=0;
int templ25_mem1_index2=0;
for(templ25_mem1_index1; templ25_mem1_index1 < 2512555; templ25_mem1_index1++)
{
for(templ25_mem1_index2; templ25_mem1_index2 < 2512555; templ25_mem1_index2++)
{
int rndRow = rand() % 2512555;
int rndCol = rand() % 2512555;
templ25_mem1[rndRow][rndCol] = 0x7FFFFFFF;
templ25_mem1[rndRow][rndCol];
}
}
}
To overcome the Segmentation Fault without changing the SIZE parameter, when size can be that large, you need to redesign your data structure(s).
For erxample, you can have this huge array in a temporary file where you swap parts into your main memory. You may also need two-level addressing because you must make sure you can address the array on disk (45TB is larger than the available address space and probably larger than the largest int).
Best could be to go over the requirements again and see if they really require such a large array, or if they allow for other approaches to achieve their goal(s).
Related
I have already worked with the stack and the heap, and in the memory management topic generally, but there is a lot of thing's i can't understand
Like, if i'm allocating an array of integer using the heap with malloc and realloc how can i determine the exact size of the array i want to work with?
This Example:
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[]){
int *array = (int *)malloc(2);
array[0] = 2;
array[1] = 1;
//What? i have allocated just 2 to be the size
array[2] = 3;
array[3] = 4;
array[4] = 4;
array[5] = 6;
//there is no segmentation fault
for(int i = 0; i < sizeof(array); i++){
printf("%d\n",array[i]);
}
}
And the wierd result i'm getting is:
2
1
3
4
4
6
1041 // ???
0
So, can someone explain to me how can i use malloc in the 100% correct way?
To correctly allocate an array using malloc, use sizeof to determine the size of each element in your array, then multiply by the number of each that you need.
Your code is only allocating 2 bytes of memory in heap, so when you write these integers (which take 4 bytes each on my machine), you are overwriting the values of unrelated state within the heap located beyond those two bytes, thus corrupting the machine state and creating undefined (that's bad) behavior.
In addition, your for loop was looping on the size of array pointer, which is typically 8 bytes. So your for loop would have tried to walk over 8 int elements, in an array of 6 ints in which you had only allocated 2 byte instead of the 24 bytes needed. Lots of bad undefined behaviour to go around here!
You may or may not get a segmentation fault due to this. A segmentation fault means you are dereferencing a pointer to an invalid page (segment) of memory, and this is caught by the hardware.
When you corrupt memory, you may not see the result of your error immediately as a segmentation fault if the memory you are writing is valid memory. Worse, if you corrupt the stack or a pointer, it may take a long time to get an actual fault to help detect the corruption created. This makes it hard to connect the fault to the event that caused the exception since your code could run for a long time before getting a segmentation fault.
#include <stdio.h>
#include <stdlib.h>
#define NUM_INTS_WANTED 6
int main(int argc, char* argv[]){
// No more trampling memory since array allocated to correct size
int *array = malloc(sizeof(int)*NUM_INTS_WANTED);
// Always check if malloc succeeded by checking that pointer is not NULL
if (array != NULL) {
array[0] = 2;
array[1] = 1;
array[2] = 3;
array[3] = 4;
array[4] = 4;
array[5] = 6;
// No more seeing 8 ints since you stop after 6 ints now
for(int i = 0; i < NUM_INTS_WANTED; i++){
printf("%d\n",array[i]);
}
} else {
// malloc failed! Report it.
printf("malloc failed!\n");
}
}
You can't. It's impossible.
Once upon a time a certain heap manager exposed this information by providing another function that would indeed return the real size. (It's padded up to the next block size.) They took it out after it was discovered to have caused more bugs than it prevented.
On most heap managers, the real size in bytes can be found at a small negative offset, but not all of them. Don't write this code. You will regret it if you have to maintain it. The heap manager can be swapped out from under you and you won't know what went wrong.
I'm new to C programming and I couldn't find solution to my problem. Although the code works (I've been able to include it in other program), when it tries to free the memory assigned by calloc(), it returns the following error:
free(): invalid next size (normal):
followd by what appears to be a memory address. I'm using mpc libraries (for arbitrary precision complex numbers). This is the smallest program that repeats the error:
#include <stdio.h>
#include <stdlib.h>
#include <gmp.h>
#include <mpfr.h>
#include <mpc.h>
int N = 10;
int precision = 512;
int main(void) {
mpc_t *dets2;
dets2 = (mpc_t*)calloc(N-2,sizeof(mpc_t));
for (int i = 0; i<=N-2; i++) {
mpc_init2(dets2[i],512); //initialize all complex numbers
mpc_set_str(dets2[i],"1",0,MPFR_RNDN); //set all the numbers to one
}
free(dets2); //release the memory occupied by those numbers
return 0;
}
Thanks for your help!
Your for loop breaks after i == N-2, but it should break before. The condition in your for loop should be i<N-2 instead of i<=N-2.
So you try to access memory, which is out of bounds. This leads to undefined behaviour, so anything can happen, including a segmentation fault, a free run time error or nothing.
I started to learn C recently. I use Code::Blocks with MinGW and Cygwin GCC.
I made a very simple prime sieve for Project Euler problem 10, which prints primes below a certain limit to stdout. It works fine until roughly 500000 as limit, but above that my minGW-compiled .exe crashes and the GCC-compiled one throws a "STATUS_STACK_OVERFLOW" exception.
I'm puzzled as to why, since the code is totally non-recursive, consisting of simple for loops.
#include <stdio.h>
#include <math.h>
#define LIMIT 550000
int main()
{
int sieve[LIMIT+1] = {0};
int i, n;
for (i = 2; i <= (int)floor(sqrt(LIMIT)); i++){
if (!sieve[i]){
printf("%d\n", i);
for (n = 2; n <= LIMIT/i; n++){
sieve[n*i] = 1;
}
}
}
for (i; i <= LIMIT; i++){
if (!sieve[i]){
printf("%d\n", i);
}
}
return 0;
}
Seems like you cannot allocate 550000 ints on the stack, allocate them dynamically instead.
int * sieve;
sieve = malloc(sizeof(int) * (LIMIT+1));
Your basic options are to store variables in data segment when your memory chunk is bigger than stack:
allocating memory for array in heap with malloc (as #Binyamin explained)
storing array in Data/BSS segments by declaring array as static int sieve[SIZE_MACRO]
All the memory in that program is allocated on the stack. When you increase the size of the array you increase the amount of space required on the stack. Eventually the method cannot be called as there isn't enough space on the stack to accomodate it.
Either experiement with mallocing the array (so it's allocated on the heap). Or learn how to tell the compiler to allocate a larger stack.
I'm stunned, why does this code give me a segmentation fault?
#include <stdio.h>
#define LIMIT 1500000
typedef struct {
int p;
int a;
int b;
} triplet;
int main(int argc, char **argv) {
int i;
triplet triplets[LIMIT];
for (i = 0; i < LIMIT; i++) {
triplets[i].p = 9; // remove this line and everything works fine
}
printf("%d\n", triplets[15].p);
return 0;
}
EDIT: After changing LIMIT to 150 I no longer get a segmentation fault, it prints random numbers instead.
EDIT2: Now I know what the site name stands for :) I made the array global and everything works fine now.
Stack overflow! Allocating 1500000 records at 12 bytes per record (assuming 4-byte int), requires more than 17 MB of stack space. Make your triplets array global or dynamically allocate it.
As to your edit - shrinking the array will probably stop the stack overflow, but your printf() call will still print uninitialized data - triplets[15].p could be anything at the time you print it out.
When you do
triplet triplets[LIMIT];
you're allocating that on the stack. Which is apparently too big for your system.
If you do
triplet* triplets=(triplet*)malloc(LIMIT*sizeof(triplet));
you'll allocate it on the heap and everything should be fine. Be sure to free the memory when you're done with it
free(triplets);
I know it could be done using malloc, but I do not know how to use it yet.
For example, I wanted the user to input several numbers using an infinite loop with a sentinel to put a stop into it (i.e. -1), but since I do not know yet how many he/she will input, I have to declare an array with no initial size, but I'm also aware that it won't work like this int arr[]; at compile time since it has to have a definite number of elements.
Declaring it with an exaggerated size like int arr[1000]; would work but it feels dumb (and waste memory since it would allocate that 1000 integer bytes into the memory) and I would like to know a more elegant way to do this.
This can be done by using a pointer, and allocating memory on the heap using malloc.
Note that there is no way to later ask how big that memory block is. You have to keep track of the array size yourself.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char** argv)
{
/* declare a pointer do an integer */
int *data;
/* we also have to keep track of how big our array is - I use 50 as an example*/
const int datacount = 50;
data = malloc(sizeof(int) * datacount); /* allocate memory for 50 int's */
if (!data) { /* If data == 0 after the call to malloc, allocation failed for some reason */
perror("Error allocating memory");
abort();
}
/* at this point, we know that data points to a valid block of memory.
Remember, however, that this memory is not initialized in any way -- it contains garbage.
Let's start by clearing it. */
memset(data, 0, sizeof(int)*datacount);
/* now our array contains all zeroes. */
data[0] = 1;
data[2] = 15;
data[49] = 66; /* the last element in our array, since we start counting from 0 */
/* Loop through the array, printing out the values (mostly zeroes, but even so) */
for(int i = 0; i < datacount; ++i) {
printf("Element %d: %d\n", i, data[i]);
}
}
That's it. What follows is a more involved explanation of why this works :)
I don't know how well you know C pointers, but array access in C (like array[2]) is actually a shorthand for accessing memory via a pointer. To access the memory pointed to by data, you write *data. This is known as dereferencing the pointer. Since data is of type int *, then *data is of type int. Now to an important piece of information: (data + 2) means "add the byte size of 2 ints to the adress pointed to by data".
An array in C is just a sequence of values in adjacent memory. array[1] is just next to array[0]. So when we allocate a big block of memory and want to use it as an array, we need an easy way of getting the direct adress to every element inside. Luckily, C lets us use the array notation on pointers as well. data[0] means the same thing as *(data+0), namely "access the memory pointed to by data". data[2] means *(data+2), and accesses the third int in the memory block.
The way it's often done is as follows:
allocate an array of some initial (fairly small) size;
read into this array, keeping track of how many elements you've read;
once the array is full, reallocate it, doubling the size and preserving (i.e. copying) the contents;
repeat until done.
I find that this pattern comes up pretty frequently.
What's interesting about this method is that it allows one to insert N elements into an empty array one-by-one in amortized O(N) time without knowing N in advance.
Modern C, aka C99, has variable length arrays, VLA. Unfortunately, not all compilers support this but if yours does this would be an alternative.
Try to implement dynamic data structure such as a linked list
Here's a sample program that reads stdin into a memory buffer that grows as needed. It's simple enough that it should give some insight in how you might handle this kind of thing. One thing that's would probably be done differently in a real program is how must the array grows in each allocation - I kept it small here to help keep things simpler if you wanted to step through in a debugger. A real program would probably use a much larger allocation increment (often, the allocation size is doubled, but if you're going to do that you should probably 'cap' the increment at some reasonable size - it might not make sense to double the allocation when you get into the hundreds of megabytes).
Also, I used indexed access to the buffer here as an example, but in a real program I probably wouldn't do that.
#include <stdlib.h>
#include <stdio.h>
void fatal_error(void);
int main( int argc, char** argv)
{
int buf_size = 0;
int buf_used = 0;
char* buf = NULL;
char* tmp = NULL;
char c;
int i = 0;
while ((c = getchar()) != EOF) {
if (buf_used == buf_size) {
//need more space in the array
buf_size += 20;
tmp = realloc(buf, buf_size); // get a new larger array
if (!tmp) fatal_error();
buf = tmp;
}
buf[buf_used] = c; // pointer can be indexed like an array
++buf_used;
}
puts("\n\n*** Dump of stdin ***\n");
for (i = 0; i < buf_used; ++i) {
putchar(buf[i]);
}
free(buf);
return 0;
}
void fatal_error(void)
{
fputs("fatal error - out of memory\n", stderr);
exit(1);
}
This example combined with examples in other answers should give you an idea of how this kind of thing is handled at a low level.
One way I can imagine is to use a linked list to implement such a scenario, if you need all the numbers entered before the user enters something which indicates the loop termination. (posting as the first option, because have never done this for user input, it just seemed to be interesting. Wasteful but artistic)
Another way is to do buffered input. Allocate a buffer, fill it, re-allocate, if the loop continues (not elegant, but the most rational for the given use-case).
I don't consider the described to be elegant though. Probably, I would change the use-case (the most rational).