I have created a linked list with the following code. As you can see, I have used malloc to created a list of size 3. But I ran the for loop over size 10 to initialize and print.
#include <stdio.h>
#include <malloc.h>
#include <stdlib.h>
struct node {
int value;
struct node *next;
};
int main() {
//code
struct node **head;
struct node *curr;
curr = (struct node *) malloc(sizeof(struct node)*3);
head = &curr;
printf("done 1\n");
(*head)->value = 0;
(*head)->next = NULL;
for(int i = 1; i < 10; i++) {
(*head+i-1)->next = (*head+i);
(*head+i)->value = i;
(*head+i)->next = NULL;
}
curr = *head;
printf("done 2\n");
for(int i = 0; i < 10; i++) {
printf("%d\t", (*head + i)->value);
//curr = curr->next;
}
printf("\ndone 3\n");
//free(curr);
return 0;
}
when I compile and run the code, the result obtained is,
done 1
done 2
0 1 2 3 154208560 842282289 876087600 154744882 808859448 875837236
done 3
Why am I able to assign values to the 4th node and access it when I actually created a list of size 3?
I see that garbage values were being printed from 5th to 10th node access. But how is the 4th node being created?
P.S:
I know that 10!=3. The code ran properly when I put the loop in its limits. I wanted to see what will happen when we go out of bounds. I see that the 4th node was also created as I was able to assign value when I actually created a list of size 3.
This is purely to see if I will get seg fault or not.
You're invoking undefined behavior. When you do this, the program may crash, it may appear to work properly, or it might behave in a seemingly random manner.
C doesn't perform any kind of bounds checks on arrays or allocated memory. It's one of the things that makes it fast. What that also means is that it will allow you to do things you're not supposed to do. It trusts the programmer to "do the right thing".
On your particular machine, you see random data after the third element. When I run the same code on my machine, I happen to get the expected output as though enough memory was allocated. Also, if I uncomment the call to free, the program crashes. That's undefined behavior.
Trying to understand undefined behavior is usually a futile effort. It all depends on the implementation details of your compiler and the machine it runs on. In this case, the memory being written to was probably unallocated memory in the heap after the memory that was properly allocated. Depending on how malloc is implemented, that part of memory may contain data needed for malloc and other functions to run properly. The point is that writing past the end of an array does not guarantee a crash, so you need to be careful.
This code will behave differently on the different operating systems.
You may get away even with assigning all 10 integers and print then correctly.
Your code puts the beginning of list on the heap with allocations for only 3 elements.
You may get lucky and use memory for all 10 element or your program will crash on the 4-th!
Related
I was trying to see the working of oom_kill by invoking manually.
I allocated memory dynamically and tried to use them infinitely with while loop at first and then with the for loop to test out of memory.
But in the first case where I used the while loop it threw segmentation fault without swap and became unresponsive with swap whereas with the for loop out of memory (oom_kill) was invoked.
Sample codes of both:
First case: while:
int main (void) {
char* p;
while (1) {
p=malloc(1<<20);
memset (p, 0, (1<<20));
}
}
Second case : for :
int main (void) {
int i, n = 0;
char *pp[N];
for (n = 0; n < N; n++) {
pp[n] = malloc(1<<20);
if (pp[n] == NULL)
break;
}
printf("malloc failure after %d MiB\n", n);
for (i = 0; i < n; i++) {
memset (pp[i], 0, (1<<20));
printf("%d\n", i+1);
}
where N is some very large number to invoke oom. Referred this https://www.win.tue.nl/~aeb/linux/lk/lk-9.html for 2nd case.
Why does it happen so? What is the mistake I'm making with the while loop?
Kernel version : 4.15
Why does it happen so?
To invoke the OOM killer, you need to have a situation where an access to memory cannot be fulfilled because there is not enough RAM available to fulfill the access. To do that, you want to first have large allocations (virtual memory mappings), then write to them.
The procedure to trigger the OOM killer is very simple:
Allocate lots of memory
Write to the allocated memory
You must have enough preallocated memory to cause everything evictable from RAM to be evicted (things like memory-mapped files), and all of swap to be used, before the kernel will evoke the OOM killer to provide more RAM/swap space to fullfill the backing to the virtual memory being written to.
What is the mistake I'm making with the while loop?
One bug, and one logical error.
The bug is, you do not check if malloc() returns NULL. malloc() returns NULL, when there is no more virtual memory available (or kernel refuses to provide more, for any reason) for the process. (In normal operation, the virtual memory available to each process is limited for non-privileged users; run e.g. ulimit -a to see the current limits.)
Because you access the memory immediately when allocated, the kernel simply refuses to allow your process more when it runs out of RAM and SWAP, and malloc() returns NULL. You then dereference the NULL pointer (by using memset(NULL, 0, 1<<20)), which causes the Segmentation fault.
The logical problem is that that scheme will not trigger the kernel OOM killer.
Remember, in order to trigger the kernel OOM killer, your process must have allocated memory that it has not accessed yet. The kernel evokes the OOM killer only when it has already provided the virtual memory, but cannot back it with actual RAM, because there is nothing evictable in RAM, and swap is already full.
In your case, the OOM killer will not get evoked, because when the kernel runs out of RAM and swap, it can simply refuse to provide more (virtual memory), leading to malloc() returning NULL.
(The Linux kernel memory subsystem is one that is actively developed, so the exact behaviour you see depends on both the kernel version, the amount of RAM and swap, and the memory manager tunables (e.g., those under /proc/sys/vm/). The above describes the most common, or typical cases and configurations.)
You don't need an external array, either. You can for example chain the allocations to a linked list:
#include <stdlib.h>
#include <stdio.h>
#ifndef SIZE
#define SIZE (2*1024*1024) /* 2 MiB */
#endif
struct list {
struct list *next;
size_t size;
char data[];
}
struct list *allocate_node(const size_t size)
{
struct list *new_node;
new_node = malloc(sizeof (struct list) + size);
if (!new_node)
return NULL;
new_node->next = NULL;
new_node->size = size;
}
int main(void)
{
size_t used = 0;
struct list *root = NULL, *curr;
/* Allocate as much memory as possible. */
while (1) {
curr = allocate_node(SIZE - sizeof (struct list));
if (!curr)
break;
/* Account for allocated total size */
used += SIZE;
/* Prepend to root list */
curr->next = root;
root = curr;
}
printf("Allocated %zu bytes.\n", used);
fflush(stdout);
/* Use all of the allocated memory. */
for (curr = root; curr != NULL; curr = curr->next)
if (curr->size > 0)
memset(curr->data, ~(unsigned char)0, curr->size);
printf("Wrote to %zu bytes of allocated memory. Done.\n", used);
fflush(stdout);
return EXIT_SUCCESS;
}
Note, the above code is untested, and even uncompiled, but the logic is sound. If you find a bug in it, or have some other issue with it, let me know in a comment so I can verify and fix.
The document you're reading is from 2003. The impossibly large number it chose to allocate was 10,000 MiB.
Today, in 2018, when new computers are likely to come with 16GiB of RAM, this kind of allocation could definitely succeed without issues.
What is the mistake I'm making with the while loop?
The segmentation fault is likely the result of passing a null pointer to memset(), since malloc() will return NULL on error.
Your second example avoids this error by always checking the return value from malloc().
I used the while loop it ... became unresponsive with swap ...
From the very document that you mentioned that you are reading:
Sometimes processes get a segfault when accessing memory that the kernel is unable to provide, sometimes they are killed, sometimes other processes are killed, sometimes the kernel hangs.
Other than mentioning the kernel version, you are very vague with the OS and system description. Presumably this is a 32-bit version?
There are actually two ways of running out of memory. Your program could exceed the amount of (virtual) memory that is allocated, or the system could actually run out of memory pages.
Note that availability of memory (pages) is a complex combination of physical memory size, swap space size, memory usage and process load.
Reference: When Linux Runs Out of Memory
by Mulyadi Santosa or here.
Background:
I'm working through a project whose goal is to build a memory management system that replicates malloc() and free(). The program begins by requesting a specified amount of memory from the operating system with mmap and then uses a linked list to keep track of which sections of memory are in use or free. See below for a general idea of what I'm doing.
Output after allocating one chunk of memory
Question
Why do I hit a segmentation fault when trying to create a new list node to represent free memory? This segmentation fault occurs only when trying to request more than two chunks of memory - the first works as expected.
Details
The segmentation fault occurs when trying to assign values to the attributes of the node just created (new in the code below) to represent the remaining free memory space. This assignment works the first time I split memory into used v. free segments, but the second time I try to split the free memory into used v. free segments, my program encounters a segmentation fault - I am assuming this occurs because the attributes of the new node_t struct are undefined (compiler doesn't know where they live in memory). What I don't know is why this is occurring, and why it doesn't occur the first time I use this code.
NOTE: The answer shouldn't be to use malloc when initializing the *new node because the point of the project is to do this without using malloc. Malloc wouldn't allocate from the memory I'm managing - instead, it'd allocate from the heap my program is actually maintaining (but not using) on the side.
Struct definitions:
typedef enum {false, true} bool;
typedef struct _node_t {
int size;
struct _node_t *next;
struct _node_t *prev;
bool used;
} node_t;
Problem code. ret is the memory allocation I'm going to return to the caller and new is the remainder of the free memory block I just allocated part of.
// Code above this steps through the list elements to find the first free
// element that is large enough. I've debugged enough to know that's not the
// source of the problem.
node_t *new = ret + sizeof(node_t) + size; // Pointer to new free slot
assert(new != NULL);
new->prev = ret;
new->next = ret->next;
new->size = ret->size - sizeof(node_t) - size;
new->used = false;
ret->used = true;
ret->next = new;
ret->size = size;
return (ret + sizeof(node_t));
Calling code:
int main(int argc, char* argv[]) {
int success = Mem_Init(5000, P_FIRSTFIT);
if (success == -1)
return -1;
int *slot1 = Mem_Alloc(sizeof(int));
// This is where segmentation fault occurs
int *slot2 = Mem_Alloc(sizeof(int));
Link to code you can compile and run.
Although you don't include a definition of ret and Dropbox is totally useless for viewing code on a smartphone, I think we can deduce the declaration
node_t* ret;
Otherwise ret->next won't compile.
In that case, ret + sizeof(node_t) does not make sense because of the nature of pointer arithmetic in C. Remember that p + k is exactly the same as &p[k]. In other words, p + 1 is the address of the next element of size sizeof*p. So ret + sizeof(node_t) is the address of the 16th or 32nd element after ret, depending on the sizeof (node_t).
What you probably wanted was ret + 1, whose simplicity illustrates the rationale behind C pointer arithmetic.
The answer: It's all in pointer math. The issue was this statement:
node_t *new = ret + sizeof(node_t) + size; // Pointer to new free slot
Any integer in the right hand side of this equation doesn't increment the memory address by one byte... it increments it to the next spot where a node_t struct would begin. So in the code above, the address of new is set (32 + size) * 32 bytes further along in memory (assuming sizeof(node_t) is 32, which is true on a 64 bit system). Do that enough and you run well outside of the 500 bytes of memory you actually requested from the OS.
The code that works is:
node_t *new = ret + 1 + (size / sizeof(node_t)) + 1;
This code ends up over-allocating memory a bit if size isn't exactly equal to sizeof(node_t), but that's a problem with a more complicated solution. For illustration, consider this example:
ret = address 1000
sizeof(node_t) = 32 (in a 64 bit system)
size = 50
The end of the header for ret is address 1032 in this example. The program then allocates 2x32 bytes to handle the 50 byte request (math is (50 / 32) + 1 = 2, or two node_t structs). So the memory address of new is actually 1096.
To make this work overall, I'll also have to update the size calculation to reflect the actual allocation instead of the requested allocation. This will be necessary to make a safeguard against running outside of the bounds of memory requested from the OS work properly.
I want to create a program that creates link list which contains arrays.
However, I want each link to have a smaller array than previous one. at this moment the program works, but I want to know whether logically the program is actually creating smaller arrays, so no free space is wasted.
int main()
{
int c=3;
int d=0;
typedef struct mylist {
struct mylist *link;
int info[c-d];
}Node;
Node *a = (Node*) malloc (sizeof (Node));
a -> link = NULL;
Node *b = a;
int i=0,j=0;
while (i!=4){
while ((j)<=(2-d)){
printf("link%d array%d ",i,j);
scanf("%d",&a->info[j]);
j++;
}
j=0;
if (i !=3 ){
a -> link = (Node*) malloc (sizeof (Node));
a = a -> link;
}
d++;
i++;
}
d=0;
a = b;
while (a != NULL){
while ((j)<=(2-d)){
printf("%d ->", a->info[j]);
j++;
}
j=0;
d++;
a = a -> link;
}
a=b;
printf(" the end");
return 0;
}
First of all, you're not free()ing memory which you allocated, make sure you take care of that.
Now, you're asking whether are you not using any additional memory - reading your code is pretty hard for us, as you can see in the comments. It is always a good idea to try it out yourself - find out the size of the memory occupied during the initialization of the structure, either with the sizeof() operator or with checking your values.
However, I don't think your code will work properly, due to the fact that you are statically initializing the arrays size. As the values are determined compile-time, this is completely fine. However, the problem is that you have already decided the size of the int array at the start.
If you want to use dynamic allocation of memory, you need to use the malloc() function. When creating a new node, you will have to allocate memory for an array in it as well (and calculate how much size you want to give it). This will also solve your problem of not knowing the size of the array or not being sure about it very well - as you are explicitly declaring how much memory you're allocating for the array.
Just a small addition though. When allocating for example 20 bytes of memory, the OS won't give you specifically 20 bytes - you'll get an entire page, which will often be more memory than necessary. So unless you're working with a device where controlling memory properly is really important or you're working with huge sizes of memory, you should not need to overthink the difference of the size of an array of 4 ints and 3 ints.
I have a problem with this code, I tried to understand what's going on, but I cannot understand why it just crushes. the functions works like I expect for the node->left.
But when the last call of the function ends in the case node->left; the function just crushes, I think because of the reallocated array, I don't know if I'm trying to access a forbidden memory.
I'll explain a little more where I think the problem comes from:we are in the last call of helper(node->left,table,len) before printing the array : consider len = N and node->left!=NULL ==> reallocating table len = N+1, assigning node->data to table[len] and everything is fine, node->left == NULL ==> printing the table and we are done with the helper(node->left,table,N) case; now we are in the helper(node->right,table,N) case; in this case the program just crushes ; it's supposed to reallocate table; and assign node->data to table[N]; and so one ...
By the way : this function tries to print all the Root-leaf paths in a binary tree.
struct node {
int data;
struct node* left;
struct node* right;
};
void helper(struct node* node,int *table,int len)
{
if(node == NULL){
return;
}
else{
table = realloc(table,(len+1)*sizeof(int));
table[len]=node->data;
len = len +1;
if(node->left == NULL && node->right == NULL)
{
int cmt=0;
for(cmt = 0; cmt < len ; cmt++ ){
printf("%d ->",table[cmt]);
}
}
else{
helper(node->left,table,len);
helper(node->right,table,len);
}
}
}
void print_rtl(struct node* node) {
if(NULL == node) return;
else{
int *t=malloc(sizeof(int));
t[0]=node->data;
int lenght = 1;
helper(node->left,t,1);
helper(node->right,t,1);
}
}
Here is the thing about realloc: it has the freedom to change not just the size of the allocated memory block, but also its location. In general, realloc can break any pointers you have that are pointing to objects in the block of memory that was realloced. When you try to use those pointers later, you will get undefined behavior.
You should either allocate enough memory at the beginning of your program that you won't need to call realloc, or you should redesign your program so you don't need to allocate that memory block in the first place, or you should make your table pointer be a global variable so that when you change its value, it gets updated for every function that is using the table.
Right now the problem with your code is that when you call helper, it might call realloc and change the location of your table, but it doesn't communicate that change to the caller in any way, so the caller doesn't know where the table is located.
There might be other problems with your code. If you continue to have trouble, I strongly urge you to produce an MCVE and post that in your next question. Doing that makes it much easier for someone else to reproduce your problem and find a solution to it that actually works.
It's hard to tell exactly what going on because it quite a big mess, but generally... The first thing helper() does (after validating node!=NULL) is reallocate table. That means when you get to the recursion point, helper(node->left.. will reallocate, and then immediately after helper(node->right.. will also try to do so, but on an invalid pointer. From there on, it's a wild ride to exceptions...
I'm getting a SIGTRAP signal when trying to free an dynamically created array, and have no idea on why.
I'm allocating the array like this:
int* visited = (int*) malloc( l.nodeCount * sizeof(int));
(l.nodeCount is an integer. In the instance of the program I get this error, it is set to 12.)
And when I try to free(visited), I get the SIGTRAP signal in the debugger.
The whole function is this one:
int Graph_GetSmallestPathCount(AdjacencyList l, int destination){
//One path if destination is root
if(destination == 0) return 1;
if(l.nodeCount == 0)
return 0;
Queue reading = Queue_NewQueue();
Queue storing = Queue_NewQueue();
/*Allocates visited array*/
int* visited = (int*) calloc( l.nodeCount, sizeof(int));
/*Visited array initialization*/
int i;
for(i = 0; i < l.nodeCount; i++)
visited[i] = 0;
/*Marks root node and enqueues it*/
visited[0] = 1;
Queue_Enqueue(&reading, 0);
//While there are nodes to read
while(!Queue_IsEmpty(reading))
{
//Dequeues a node
int v = Queue_Dequeue(&reading);
//Gets it's adjacency list
List* currentList = AdjacencyList_GetAdjacentNodes(l, v);
listCell* auxCell = currentList->head->next;
//While there are nodes in it's adjacency list
while(auxCell != NULL){
//Enqueues it if it has not been visited
if(visited[auxCell->data] == 0){
Queue_Enqueue(&storing, auxCell->data);
}
//Adds to the paths to that node
visited[auxCell->data] += visited[v];
auxCell = auxCell->next;
}
//When the queue ends
if(Queue_IsEmpty(reading)){
//If the destination has been reached, return
if(visited[destination] > 0){
Queue_Destroy(&reading);
Queue_Destroy(&storing);
return visited[destination];
}
else{
//Switch queues
Queue_Destroy(&reading);
reading = storing;
storing = Queue_NewQueue();
}
}
}
//Destination has not been reached before end of algorithms. Deallocate everything and return 0
free(visited);
Queue_Destroy(&reading);
Queue_Destroy(&storing);
return 0;
}
Sorry for the lack of comments, I did this on a run and didn't put any in. Also sorry for the printf overload, I put them there while trying to pinpoint the problem.
EDIT: I cleaned it up a little.
The weird thing is that the program works for certain inputs and doesn't for others.
Hope someone can help me out =D
I can't tell you why you get a SIGTRAP as you haven't published a minimal example.
However, I can tell you how to find out out yourself:
Make your program readable. Use one instruction per line. The indent tool is your friend. Sure, that won't fix the bug, but it will make it easier for you to find it.
Don't malloc like that. There is no need to cast the return value of malloc, and using calloc(l.nodeCount, sizeof (int)); or similar is more readable anyway.
What SIGTRAP actually means is you've hit a breakpoint instruction. No doubt what's actually happening is that you've jumped to something which is not your code, and might not be code at all, but contains the binary code for a breakpoint. Why did that happen? The normal cause would be memory corruption, particularly stack corruption. I'm guessing free() is corrupting its own stack. And I'd guess the reason for that is because you are (somewhere) writing to memory outside the memory you've allocated. To test this, run your program with the malloc()/calloc() immediately followed by the free() and an exit(0). If that works, you know the issue is something you are doing between.
We can't tell what you are doing between because you haven't (thankfully) posted the full program, but try running it under valgrind. When you get an out-of-range write, valgrind will normally pick it up. Fix every valgrind warning. That doesn't guarantee a solution, but will find one 95% of the time in my experience.
Also note that return visited[destination]; appears to exit the function without free()-ing visited, which is thus a memory leak.
First, don't call malloc() like that. l.nodeCount * sizeof(int) could potentially exceed INT_MAX, and you'll either have a security hole or if you're lucky, a crash.
Instead, use calloc(l.nodeCount, sizeof(int)).
You should also check the return value of malloc or calloc for NULL in the event your program runs out of memory to allocate.
Found the answer. There was indeed a chance that on specific cases the array would be created with one element less than it should. My bad.
Thanks to all who helped =D