I have a monte carlo simulation, that is supposed to run multiple times and then average over the individual results. After each run, I want to free the allocated memory for the stuff I need in the simulation. One piece of this stuff is an array of sites, where each site is a struct that has a linked list of type SLE as one member. So in order to free all of it, I first iterate through the linked list and free each node and then free the memory of the sites array.
SLE * neighbor, * tmp;
int i;
for(i = 0; i < args.nsites_arg; ++i)
{
// free neighbor memory
neighbor = sites[i].neighbors;
while(neighbor)
{
tmp = neighbor->next;
free(neighbor);
neighbor = tmp;
}
}
free((void *)sites);
There is, however, something very weird happening: The simulation runs fine once and then starts over again. I reallocate all the needed memory for sites and rewrite the mentioned linked list of SLEs. Then, in some sorting operation on the former array, it gives mit a segfault:
Program received signal SIGSEGV, Segmentation fault.
0x0000000000408468 in sortNeighbors (list=0xad0e00) at mc_init.c:485
485 while(temp && temp->next)
When I remove the free(neighbor) in the above snippet, it works fine. As I said, everything is reallocated just like before the first run. So what happens here? How could I gather more information on what is really freed and why this segfault is happening?
Edit: The other weird thing is, that the beginning of this sorting looks like this:
if (!list || !list->next)
return list;
SLE * temp = list;
// Find halfway through the list
while(temp && temp->next)
{
I explicitely check if list and list->next exist, so why is it giving a segfault in the while condition?
Edit2: The allocation
sites = (Site *) malloc(args.nsites_arg * sizeof (Site));
...
// s is now one element of the sites array
while(siteList)
{
neighbors = (SLE*) malloc(sizeof (SLE));
...
neighbors->next = s->neighbors;
s->neighbors = neighbors;
siteList = siteList->next;
}
If you are on linux, take a look at my answer here. It's about valgrind, a tool that helps debugging segfaults, memory leaks etc.
When you get SIGSEV by accessing freed memory (your case as I understand), it tells you where the memory was freed - it is not necessary that it was freed by the free call in your snippet.
Before searching for a bug, you might want to use an already implemented well tested single or double linked lists.
Or, if you don't want to bother with the mem bug, link against gc garbage collector (assuming the actual calculation of your simulation does what it should do)
Some more good commercial tools for Windows:
Purify
Insure++
Try running it through valgrind: valgind --tool=memcheck [COMMAND]
It works out of the box, nothing special needed.
For more info goto the Valgrind website.
Related
I'm trying to write a remove function for a singly linked list and got this error message:
Thread 1: EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
What does this message mean and how shall I fix it?
void fifoqueue_remove(Fifoqueue_Ptr queue_ptr, void * this_call){
Queue_Container_Ptr position = queue_ptr->front_ptr; //make position head
while (position->next_ptr != NULL){
if (position->next_ptr->content_ptr == this_call){
Queue_Container_Ptr next = position->next_ptr;
position->next_ptr = next->next_ptr;
free((char*) position); //remove position
}
else{
position = position->next_ptr;
}
}
}
Your loop is doing bad things with position which, if not the actual issue, is definitely undesirable.
As currently written, if you find the value to remove, you are calling free(position); and then continuing the loop. The very next thing that will happen is you access position->next_ptr, dereferencing a memory address that you returned to the system only moments ago.
This is likely to result in a crash if you're lucky. Possibly you're compiling in non-optimized mode where the memory manager is writing a byte pattern over any freed memory, so that attempts to use it are far more likely to result in bad pointer access.
I see you have actually prepared for the "correct approach" here but forgotten to do it:
// You remembered to save position->next_ptr
Queue_Container_Ptr next = position->next_ptr;
// ...
free(position);
// But you forgot to do this:
position = next;
It should also be mentioned that you are not handling the very last element in the list. Or perhaps that's not necessary because your list has a dummy tail node or something. It's hard to be sure, given what you've shown. But you should make sure that you test removal of a value that is the last position. You might discover a bug there.
I was trying to see the working of oom_kill by invoking manually.
I allocated memory dynamically and tried to use them infinitely with while loop at first and then with the for loop to test out of memory.
But in the first case where I used the while loop it threw segmentation fault without swap and became unresponsive with swap whereas with the for loop out of memory (oom_kill) was invoked.
Sample codes of both:
First case: while:
int main (void) {
char* p;
while (1) {
p=malloc(1<<20);
memset (p, 0, (1<<20));
}
}
Second case : for :
int main (void) {
int i, n = 0;
char *pp[N];
for (n = 0; n < N; n++) {
pp[n] = malloc(1<<20);
if (pp[n] == NULL)
break;
}
printf("malloc failure after %d MiB\n", n);
for (i = 0; i < n; i++) {
memset (pp[i], 0, (1<<20));
printf("%d\n", i+1);
}
where N is some very large number to invoke oom. Referred this https://www.win.tue.nl/~aeb/linux/lk/lk-9.html for 2nd case.
Why does it happen so? What is the mistake I'm making with the while loop?
Kernel version : 4.15
Why does it happen so?
To invoke the OOM killer, you need to have a situation where an access to memory cannot be fulfilled because there is not enough RAM available to fulfill the access. To do that, you want to first have large allocations (virtual memory mappings), then write to them.
The procedure to trigger the OOM killer is very simple:
Allocate lots of memory
Write to the allocated memory
You must have enough preallocated memory to cause everything evictable from RAM to be evicted (things like memory-mapped files), and all of swap to be used, before the kernel will evoke the OOM killer to provide more RAM/swap space to fullfill the backing to the virtual memory being written to.
What is the mistake I'm making with the while loop?
One bug, and one logical error.
The bug is, you do not check if malloc() returns NULL. malloc() returns NULL, when there is no more virtual memory available (or kernel refuses to provide more, for any reason) for the process. (In normal operation, the virtual memory available to each process is limited for non-privileged users; run e.g. ulimit -a to see the current limits.)
Because you access the memory immediately when allocated, the kernel simply refuses to allow your process more when it runs out of RAM and SWAP, and malloc() returns NULL. You then dereference the NULL pointer (by using memset(NULL, 0, 1<<20)), which causes the Segmentation fault.
The logical problem is that that scheme will not trigger the kernel OOM killer.
Remember, in order to trigger the kernel OOM killer, your process must have allocated memory that it has not accessed yet. The kernel evokes the OOM killer only when it has already provided the virtual memory, but cannot back it with actual RAM, because there is nothing evictable in RAM, and swap is already full.
In your case, the OOM killer will not get evoked, because when the kernel runs out of RAM and swap, it can simply refuse to provide more (virtual memory), leading to malloc() returning NULL.
(The Linux kernel memory subsystem is one that is actively developed, so the exact behaviour you see depends on both the kernel version, the amount of RAM and swap, and the memory manager tunables (e.g., those under /proc/sys/vm/). The above describes the most common, or typical cases and configurations.)
You don't need an external array, either. You can for example chain the allocations to a linked list:
#include <stdlib.h>
#include <stdio.h>
#ifndef SIZE
#define SIZE (2*1024*1024) /* 2 MiB */
#endif
struct list {
struct list *next;
size_t size;
char data[];
}
struct list *allocate_node(const size_t size)
{
struct list *new_node;
new_node = malloc(sizeof (struct list) + size);
if (!new_node)
return NULL;
new_node->next = NULL;
new_node->size = size;
}
int main(void)
{
size_t used = 0;
struct list *root = NULL, *curr;
/* Allocate as much memory as possible. */
while (1) {
curr = allocate_node(SIZE - sizeof (struct list));
if (!curr)
break;
/* Account for allocated total size */
used += SIZE;
/* Prepend to root list */
curr->next = root;
root = curr;
}
printf("Allocated %zu bytes.\n", used);
fflush(stdout);
/* Use all of the allocated memory. */
for (curr = root; curr != NULL; curr = curr->next)
if (curr->size > 0)
memset(curr->data, ~(unsigned char)0, curr->size);
printf("Wrote to %zu bytes of allocated memory. Done.\n", used);
fflush(stdout);
return EXIT_SUCCESS;
}
Note, the above code is untested, and even uncompiled, but the logic is sound. If you find a bug in it, or have some other issue with it, let me know in a comment so I can verify and fix.
The document you're reading is from 2003. The impossibly large number it chose to allocate was 10,000 MiB.
Today, in 2018, when new computers are likely to come with 16GiB of RAM, this kind of allocation could definitely succeed without issues.
What is the mistake I'm making with the while loop?
The segmentation fault is likely the result of passing a null pointer to memset(), since malloc() will return NULL on error.
Your second example avoids this error by always checking the return value from malloc().
I used the while loop it ... became unresponsive with swap ...
From the very document that you mentioned that you are reading:
Sometimes processes get a segfault when accessing memory that the kernel is unable to provide, sometimes they are killed, sometimes other processes are killed, sometimes the kernel hangs.
Other than mentioning the kernel version, you are very vague with the OS and system description. Presumably this is a 32-bit version?
There are actually two ways of running out of memory. Your program could exceed the amount of (virtual) memory that is allocated, or the system could actually run out of memory pages.
Note that availability of memory (pages) is a complex combination of physical memory size, swap space size, memory usage and process load.
Reference: When Linux Runs Out of Memory
by Mulyadi Santosa or here.
I have created a linked list with the following code. As you can see, I have used malloc to created a list of size 3. But I ran the for loop over size 10 to initialize and print.
#include <stdio.h>
#include <malloc.h>
#include <stdlib.h>
struct node {
int value;
struct node *next;
};
int main() {
//code
struct node **head;
struct node *curr;
curr = (struct node *) malloc(sizeof(struct node)*3);
head = &curr;
printf("done 1\n");
(*head)->value = 0;
(*head)->next = NULL;
for(int i = 1; i < 10; i++) {
(*head+i-1)->next = (*head+i);
(*head+i)->value = i;
(*head+i)->next = NULL;
}
curr = *head;
printf("done 2\n");
for(int i = 0; i < 10; i++) {
printf("%d\t", (*head + i)->value);
//curr = curr->next;
}
printf("\ndone 3\n");
//free(curr);
return 0;
}
when I compile and run the code, the result obtained is,
done 1
done 2
0 1 2 3 154208560 842282289 876087600 154744882 808859448 875837236
done 3
Why am I able to assign values to the 4th node and access it when I actually created a list of size 3?
I see that garbage values were being printed from 5th to 10th node access. But how is the 4th node being created?
P.S:
I know that 10!=3. The code ran properly when I put the loop in its limits. I wanted to see what will happen when we go out of bounds. I see that the 4th node was also created as I was able to assign value when I actually created a list of size 3.
This is purely to see if I will get seg fault or not.
You're invoking undefined behavior. When you do this, the program may crash, it may appear to work properly, or it might behave in a seemingly random manner.
C doesn't perform any kind of bounds checks on arrays or allocated memory. It's one of the things that makes it fast. What that also means is that it will allow you to do things you're not supposed to do. It trusts the programmer to "do the right thing".
On your particular machine, you see random data after the third element. When I run the same code on my machine, I happen to get the expected output as though enough memory was allocated. Also, if I uncomment the call to free, the program crashes. That's undefined behavior.
Trying to understand undefined behavior is usually a futile effort. It all depends on the implementation details of your compiler and the machine it runs on. In this case, the memory being written to was probably unallocated memory in the heap after the memory that was properly allocated. Depending on how malloc is implemented, that part of memory may contain data needed for malloc and other functions to run properly. The point is that writing past the end of an array does not guarantee a crash, so you need to be careful.
This code will behave differently on the different operating systems.
You may get away even with assigning all 10 integers and print then correctly.
Your code puts the beginning of list on the heap with allocations for only 3 elements.
You may get lucky and use memory for all 10 element or your program will crash on the 4-th!
I was implementing stacks using dynamically allocated arrays. Once the array is full I need to reallocate the array and make it twice as big as the initial array.
My code:
typedef int Item;
typedef struct stackImp *Stack;
struct stackImp{
Item * items;
int top;
int maxSize;
Stack createStack (void){
Stack s = malloc(sizeof(struct stackImp));
assert(s != NULL);
s->items = malloc(DEFAULT_SIZE * sizeof(Item));
assert(s->items != NULL);
s->top = 0;
s->maxSize = DEFAULT_SIZE;
return s;
void push (Stack stack, Item item)
.
.
.
if (stack->top < stack->maxSize) {
//Over here I'm checking if the top index is less than the
//maximum Items the array can store. If it's less then it pushes the
//item to the top of the array.
stack->items[stack->top] = item;
stack->top++;
}
else {
//If the index is greater than or equal to the maximum size then
//I realloc a new array which is twice the size of the initial array.
temp = realloc(stack->items, 2*(stack->maxSize) * sizeof(Item));
assert (temp != NULL);
stack->items = temp;
.
.
.
}
When I push items onto the stack it works perfectly fine but when I push on more than the initial maxSize it gives me this error:
It is clear I'm doing something wrong with my realloc function but I am unable to find what.
This is the valgrind output:
Your code is kind of confusing, since stack->top does not actually points to the index of the actual stack top. This might be causing some confusion on other parts of your code, since you have to use stack->items[stack->top-1] and not stack->items[stack->top] to access the top of your stack.
Please double check things like that. This error is probably caused by memory corruption. Learn how to use memory debuggers.
And I recommend that you try to implement stacks using lists instead of arrays.
Since the error is coming from realloc and the memory arena seems to be already corrupted, it's almost certainly something you're doing before this point.
Your best bet is to run it with a memory debugger like valgrind and check if you're overflowing buffers.
It may be the initial allocation of the array of Item objects (or lack thereof), it may be incorrect initialisation of the other fields like top or maxSize. Without the full code, it's a little hard for us to tell so, if you want a definitive answer, you should post all the relevant code.
But a memory debugger should help out greatly, should you not wish to do that.
I'm getting a SIGTRAP signal when trying to free an dynamically created array, and have no idea on why.
I'm allocating the array like this:
int* visited = (int*) malloc( l.nodeCount * sizeof(int));
(l.nodeCount is an integer. In the instance of the program I get this error, it is set to 12.)
And when I try to free(visited), I get the SIGTRAP signal in the debugger.
The whole function is this one:
int Graph_GetSmallestPathCount(AdjacencyList l, int destination){
//One path if destination is root
if(destination == 0) return 1;
if(l.nodeCount == 0)
return 0;
Queue reading = Queue_NewQueue();
Queue storing = Queue_NewQueue();
/*Allocates visited array*/
int* visited = (int*) calloc( l.nodeCount, sizeof(int));
/*Visited array initialization*/
int i;
for(i = 0; i < l.nodeCount; i++)
visited[i] = 0;
/*Marks root node and enqueues it*/
visited[0] = 1;
Queue_Enqueue(&reading, 0);
//While there are nodes to read
while(!Queue_IsEmpty(reading))
{
//Dequeues a node
int v = Queue_Dequeue(&reading);
//Gets it's adjacency list
List* currentList = AdjacencyList_GetAdjacentNodes(l, v);
listCell* auxCell = currentList->head->next;
//While there are nodes in it's adjacency list
while(auxCell != NULL){
//Enqueues it if it has not been visited
if(visited[auxCell->data] == 0){
Queue_Enqueue(&storing, auxCell->data);
}
//Adds to the paths to that node
visited[auxCell->data] += visited[v];
auxCell = auxCell->next;
}
//When the queue ends
if(Queue_IsEmpty(reading)){
//If the destination has been reached, return
if(visited[destination] > 0){
Queue_Destroy(&reading);
Queue_Destroy(&storing);
return visited[destination];
}
else{
//Switch queues
Queue_Destroy(&reading);
reading = storing;
storing = Queue_NewQueue();
}
}
}
//Destination has not been reached before end of algorithms. Deallocate everything and return 0
free(visited);
Queue_Destroy(&reading);
Queue_Destroy(&storing);
return 0;
}
Sorry for the lack of comments, I did this on a run and didn't put any in. Also sorry for the printf overload, I put them there while trying to pinpoint the problem.
EDIT: I cleaned it up a little.
The weird thing is that the program works for certain inputs and doesn't for others.
Hope someone can help me out =D
I can't tell you why you get a SIGTRAP as you haven't published a minimal example.
However, I can tell you how to find out out yourself:
Make your program readable. Use one instruction per line. The indent tool is your friend. Sure, that won't fix the bug, but it will make it easier for you to find it.
Don't malloc like that. There is no need to cast the return value of malloc, and using calloc(l.nodeCount, sizeof (int)); or similar is more readable anyway.
What SIGTRAP actually means is you've hit a breakpoint instruction. No doubt what's actually happening is that you've jumped to something which is not your code, and might not be code at all, but contains the binary code for a breakpoint. Why did that happen? The normal cause would be memory corruption, particularly stack corruption. I'm guessing free() is corrupting its own stack. And I'd guess the reason for that is because you are (somewhere) writing to memory outside the memory you've allocated. To test this, run your program with the malloc()/calloc() immediately followed by the free() and an exit(0). If that works, you know the issue is something you are doing between.
We can't tell what you are doing between because you haven't (thankfully) posted the full program, but try running it under valgrind. When you get an out-of-range write, valgrind will normally pick it up. Fix every valgrind warning. That doesn't guarantee a solution, but will find one 95% of the time in my experience.
Also note that return visited[destination]; appears to exit the function without free()-ing visited, which is thus a memory leak.
First, don't call malloc() like that. l.nodeCount * sizeof(int) could potentially exceed INT_MAX, and you'll either have a security hole or if you're lucky, a crash.
Instead, use calloc(l.nodeCount, sizeof(int)).
You should also check the return value of malloc or calloc for NULL in the event your program runs out of memory to allocate.
Found the answer. There was indeed a chance that on specific cases the array would be created with one element less than it should. My bad.
Thanks to all who helped =D