Why aren't the numbers after the first one of each line of the matrix random numbers? Why are they zeros? For example if I print a line of this matrix I get4 0 0 0 0 but I should get the numbers after 4 as random numbers instead.
void readfile(FILE *input,int **matrix){
int i=0, num;
while(fscanf(input, "%d ", &num) == 1){
matrix[i] = malloc((num+1)*sizeof(int));
matrix[i][0] = num;
i++;
}
}
Why aren't the numbers after the first one of each line of the matrix random numbers?
Why should they be?
Yes, malloc returns a newly allocated block of uninitialized memory, but nobody said that it had to be random.
Indeed, typically at process start you are going to get blank pages, just zeroed out by the operating system and provided to your process (the OS cannot recycle pages from other processes without blanking them out for security reasons), while later you are more likely to get back pages that your process has freed, so generally containing old data from your own program.
All this is strictly non-contractual, and is often violated - for example, so-called "debug heaps" generally fill in free pages with a known pattern (e.g. 0xCDCDCDCD on Visual C++) to spot usages of uninitialized memory.
Long story short: don't make any kind of assumption about the content of memory provided by malloc.
Related
I'm using a scientific simulation code that my supervisor wrote about 10 years ago to run some calculations. An intermittent issue keeps arising when running it in parallel on our cluster (which has hyperthreading enabled) using mpirun. The error it produces is very terse, and simply tells me that a memory assignment has failed.
program-name: malloc.c:4036: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned long) (nb)' failed.
[MKlabgroup:3448077] *** Process received signal ***
[MKlabgroup:3448077] Signal: Aborted (6)
[MKlabgroup:3448077] Signal code: (-6)
I've used the advice here to start the program and halt it so that I can attach a debugger session to one of the instances on a single core. The error occurs during the partitioning of the input mesh (using metis) when a custom matrix function is called for the first time, and requests space for ~4000 rows and 4 columns, with each element being an 8 byte integer. This particular function (below) uses an array of n pointers addressing m arrays of integer pointers:
int **matrix_int(int n, int m)
{
int i;
int **mat;
// First: Assign the rows [n]
mat = (int **) malloc(n*sizeof(int*));
if (!mat)
{
program_error("ERROR[0]: Memory allocation failure for the rows #[matrix_int()]");
}
// Second: Assign the columns [m]
for (i = 0; i <= n-1; i++)
{
mat[i] = (int *) malloc(m*sizeof(int));
if (!mat[i])
{
program_error("ERROR[0]: Memory allocation failure for the columns #[matrix_int()]");
}
}
return mat;
}
My supervisor thinks that the issue has to do with automatic resource allocation on the CPU. As such, I've recently tried using the -rf option in mpirun in conjunction with a rankfile specifying which cores to use, but this has produced similarly intermittent results; sometimes a single process crashes, sometimes several, and sometimes it runs fine. It always runs reliably in serial, but the calculations are extremely slow on a single core.
Does anyone know of a change to the server configuration or the code itself that I can make (aside from globally disabling hyperthreading) that would allow this to run for certain every time?
(Any general tips on debugging in parallel would also be greatly appreciated! I'm still pretty new to C/C++ and MPI, and have another bug to chase after this one which is probably related.)
After using the compiler flags suggested by n. 1.8e9-where's-my-share m. to diagnose memory access violations I've discovered that the memory corruption is indeed caused by a function that is called just before the one in my original question.
The offending function reads in data from a text file using sscanf, and would allocate a 3-element array for each line of the file (for the 3 numbers to be read in per line). The next part is conjecture, but I think that the problem arose because sscanf returns a NULL at the end of a sequence it reads. I'm surmising that this NULL was written to the next byte along from the 3 allocated, such that the next time malloc tried to allocate data the first thing it saw was a NULL, causing it to return without actually having allocated any space. Then the next function to try and use the allocated memory would come along and crash because it's trying to access unassigned memory that malloc had reported to be allocated.
I was able to fix the bug by changing the size of the allocated array in the read function from 3 to 4 elements. This would seem to allow the NULL character to be stored without it interfering with subsequent memory allocations.
I'm creating the below array:
int p[100];
int
main ()
{
int i = 0;
while (1)
{
p[i] = 148;
i++;
}
return (0);
}
The program aborts with a segmentation fault after writing 1000 positions of the array, instead of the 100. I know that C doesn't check if the program writes out of bounds, this is left to the OS. I'm running it on ubuntu, the size of the stack is 8MB (limit -s). Why is it aborting after 1000? How can I check how much memory my OS allocates for an array?
Sorry if it's been asked before, I've been googling this but can't seem to find a specific explanation for this.
Accessing an invalid memory location leads to Undefined Behavior which means that anything can happen. It is not necessary for a segmentation-fault to occur.
...the size of the stack is 8MB (limit -s)...
variable int p[100]; is not at the stack but in data area because it is defined as global. It is not initialized so it is placed into BSS area and filled with zeros. You can check that printing array values just at the beginning of main() function.
As other said, using p[i] = 148; you produced undefined behaviour. Filling 1000 position you most probably reached end of BSS area and got segmentation fault.
It appear that you clearly get over the 100 elements defined (int p[100];) since you make a loop without any limitation (while (1)).
I would suggest to you to use a for loop instead:
for (i = 0; i < 100; i++) {
// do your stuff...
}
Regarding you more specific question about the memory, consider that any outside range request (in your situation over the 100 elements of the array) can produce an error. The fact that you notice it was 1000 in your situation can change depending on memory usage by other program.
It will fail once the CPU says
HEY! that's not Your memory, leave it!
The fact that the memory is not inside of the array does not mean that it's not for the application to manipulate.
The program aborts with a segmentation fault after writing 1000 positions of the array, instead of the 100.
You do not reason out Undefined Behavior. Its like asking If 1000 people are under a coconut tree, will 700 hundred of them always fall unconscious if a Coconut smacks each of their heads?
I'am doing a homework on matrix multiplication. The problem that i want to find the largest matrix that i can handle (allocate). So i wrote the following code:
int n = 1;
while(1){
n++;
A=malloc(sizeof(double)*n*n);
B=malloc(sizeof(double)*n*n);
C=malloc(sizeof(double)*n*n);
if (C==NULL) {printf("Error No more size to allocate! n=%d\n",n); return 1;}
// and the rest of the code including freeing the allocate
the result:
Error No more size to allocate! n=21785
Now i want to use another method: using A as the result instead of C.
So that i only need 2(n**2)+n instead of 3(n**2).
So the new code should loke like this :
int n = 1;
while(1){
n++;
A=malloc(sizeof(double)*n*n);
B=malloc(sizeof(double)*n*n);
C=malloc(sizeof(double)*n);
if (C==NULL) {printf("Error No more size to allocate! n=%d\n",n); return 1;}
// and the rest of the code including freeing the allocated space.
The problem that when i run this code it wont stop incrementing n but if i change the condition from (C==NULL) to (A==NULL||B==NULL||C==NULL)
the result is:
Error No more size to allocate! n=21263
Any idea??
Edit
Do I cast the result of malloc?
PS: My professor tell us to always use cast in malloc!!
Your program fails to allocate A or B long before it fails to allocate C, which is much smaller. With n being approximately 21263, n*n is 21263 times larger than n. The loop will continue for about 10000 repetitions. If you free C after successful allocation, the loop will even continue for a few hundred million repetitions until n reaches about 21263*21263. You just have to wait long enough for your program to exit.
there are a few things to note
1) the main consideration is that memory, once allocated (by malloc and not free'd) means the amount of available heap to allocate goes down, even as 'n' goes up.
2) any call to malloc() can fail, including the first call
this means the failure could be any of the calls to malloc()
3) in C, the returned value from malloc (and family) should not be cast.
4) the returned value from malloc() should always be checked,
not just one in three calls to malloc()
regarding the posted code.
1) all the above considerations should be implemented
2) in the second example of posted code, just because a
larger memory request (n*n) fails does not mean a
smaller request (n) would fail.
that is why the modification catches the failure of 'A'
3) the heap address and the size of the heap are normally
available at compile time, so there is no need
for the kind of code you posted
You could try doing a single allocation, then assigning pointers for B and C as offsets from A. Rather than starting at a small value, start at a large value that should fail on the first loops, so that when the allocation does succeed, the heap will not be fragmented.
with the following code, I am trying to make an array of numbers and then sorting them. But if I set a high arraysize (MAX), the program stops at the last 'randomly' generated number and does not continue to the sorting at all. Could anyone please give me a hand with this?
#include <stdio.h>
#define MAX 2000000
int a[MAX];
int rand_seed=10;
/* from K&R
- returns random number between 0 and 62000.*/
int rand();
int bubble_sort();
int main()
{
int i;
/* fill array */
for (i=0; i < MAX; i++)
{
a[i]=rand();
printf(">%d= %d\n", i, a[i]);
}
bubble_sort();
/* print sorted array */
printf("--------------------\n");
for (i=0; i < MAX; i++)
printf("%d\n",a[i]);
return 0;
}
int rand()
{
rand_seed = rand_seed * 1103515245 +12345;
return (unsigned int)(rand_seed / 65536) % 62000;
}
int bubble_sort(void)
{
int t, x, y;
/* bubble sort the array */
for (x=0; x < MAX-1; x++)
for (y=0; y < MAX-x-1; y++)
if (a[y] > a[y+1])
{
t=a[y];
a[y]=a[y+1];
a[y+1]=t;
}
return 0;
}
The problem is that you are storing the array in global section, C doesn't give any guarantee about the maximum size of global section it can support, this is a function of OS, arch compiler.
So instead of creating a global array, create a global C pointer, allocated a large chunk using malloc. Now memory is saved in the heap which is much bigger and can grow at runtime.
Your array will land in BSS section for static vars. It will not be part of an image but program loader will allocate required space and fill it with zeros before your program starts 'real' execution. You can even control this process if using embedded compiler and fill your static data with anything you like. This array may occupy 2GB or your RAM and yet your exe file may be few kilobytes. I've just managed to use over 2GB array this way and my exe was 34KB. I can believe a compiler may warn you when you approach maybe 231-1 elements (if your int is 32bit) but static arrays with 2m elements are not a problem nowadays (unless it is embedded system but I bet it is not).
The problem might be that your bubble sort has 2 nested loops (as all bubble sorts) so trying to sort this array - having 2m elements - causes the program to loop 2*1012 times (arithmetic sequence):
inner loop:
1: 1999999 times
2: 1999998 times
...
2000000: 1 time
So you must swap elements
2000000 * (1999999+1) / 2 = (4 / 2) * 10000002 = 2*1012 times
(correct me if I am wrong above)
Your program simply remains too long in sort routine and you are not even aware of that. What you see it just last rand number printed and program not responding. Even on my really fast PC with 200K array it took around 1minute to sort it this way.
It is not related to your os, compiler, heaps etc. Your program is just stuck as your loop executes 2*1012 times if you have 2m elements.
To verify my words print "sort started" before sorting and "sort finished" after that. I bet the last thing you'll see is "sort started". In addition you may print current x value before your inner loop in bubble_sort - you'll see that it is working.
Dynamic Array
int *Array;
Array= malloc (sizeof(int) * Size);
The original C standard (ANSI 1989/ISO 1990) required that a compiler successfully translate at least one program containing at least one example of a set of environmental limits. One of those limits was being able to create an object of at least 32,767 bytes.
This minimum limit was raised in the 1999 update to the C standard to be at least 65,535 bytes.
No C implementation is required to provide for objects greater than that size, which means that they don't need to allow for an array of ints greater than
(int)(65535 / sizeof(int)).
In very practical terms, on modern computers, it is not possible to say in advance how large an array can be created. It can depend on things like the amount of physical memory installed in the computer, the amount of virtual memory provided by the OS, the number of other tasks, drivers, and programs already running and how much memory that are using. So your program may be able to use more or less memory running today than it could use yesterday or it will be able to use tomorrow.
Many platforms place their strictest limits on automatic objects, that is those defined inside of a function without the use of the 'static' keyword. On some platforms you can create larger arrays if they are static or by dynamic allocation.
ive got a C program that gets caught in a for loop that it shouldn't, running it with
valgrind --tool=memcheck --leak-check=yes a.out
doesnt return anything even up until the program gets caught. is there a way to change the settings of valgrind to help me find the leak? as many have pointed out, it wouldnt be considered a leak, apologies
thanks in advance
here is the loop in question
int clockstate=0;
int clocklength=0;
int datalength=0;
int datastate=0;
int dataloc = 9;
((((some other code that i don't think is important to this part))))
int dataerr[13] = {0};
int clockerr[13] = {0}; // assumes that spill does not change within an event.
int spill=0;
int k = 0;
spill = Getspill(d+4*255+1); // get spill bit from around the middle
//printf("got spill: %d \n", spill); // third breakpoint
for( k = 0; k < 512; k++)
{
// Discardheader(d); // doesnt actually do anything, since it's a header.f
int databit = Getexpecteddata(d+4*k+1);
printf("%d ",k);
int transmitted = Datasample(&datastate, &datalength, d+4*k+2,dataerr,dataloc, databit);
printf("%d ",k);
Clocksample(&clockstate, &clocklength, d+4*k+3,clockerr, transmitted);
printf("%d \n",k);
// assuming only one error per event (implying the possibility of multi-error "errors"
// we construct the final error at the very end of the (outside this loop)
}
and the loop repeats after printing
254 254 254
255 255 255
256 256 1 <- this is the problem
2 2 2
3 3 3
edit** so i've tracked down where it is happening, and at one point in
void Clocksample (int* state, int* length, char *d, int *type, int transbit);
i have code that says *length = 1; so it seems that this command is somehow writing onto int k. my question now is, how did this happen, why isnt it changing length back to one like i want, and how do i fix it. if you want, i can post the whole code to Clocksample
Similar to last time, something in one of those functions, Clocksample() this time, is writing to memory that doesn't belong to the data/arrays that the function should be using. Most likely an out of bounds array write. Note: this is not a memory leak, which is allocating then losing track of memory blocks that should be freed.
Set a breakpoint at the call to Clocksample() for when k is 256. Then step into Clocksample(), keeping a watch on k (or the memory used by k). You can probably also just set a hardware memory write breakpoint on the memory allocated to k. How you do any of this depends on the debugger you're using.
Now single-step (or just run to the return of Clocksample() if you have a hardware breakpoint set) and when k changes, you'll have the culprit.
Please note that Valgrind is exceedingly weak when it comes to detecting stack buffer overflows (which is what appears to be happening here).
Google address-sanitizer is much better at detecting stack overflows, and I suggest you try it instead.
So your debugging output indicates that k is being changed during the call to your function Clocksample. I see that you are passing the addresses of at least two variables, &clockstate and &clocklength into that call. It seems quite likely to me that you have an array overrun or some other wild pointer in Clocksample that ends up overwriting the memory location where k is stored.
It might be possible to narrow down the bug if you post the code where k is declared (and whatever other variables are declared nearby in the same scope). For example if clocklength is declared right before k then you probably have a bug in using the pointer value &clocklength that leads to writing past the end of clocklength and corrupting k. But it's hard to know for sure without having the actual layout of variables you're using.
valgrind doesn't catch this because if, say, clocklength and k are right next to each other on the stack, valgrind can't tell if you have a perfectly valid access to k or a buggy access past the end of clocklength, since all it checks is what memory you actually access.