Linux- Out of Memory - c

Question

I was trying to see the working of oom_kill by invoking manually.
I allocated memory dynamically and tried to use them infinitely with while loop at first and then with the for loop to test out of memory.
But in the first case where I used the while loop it threw segmentation fault without swap and became unresponsive with swap whereas with the for loop out of memory (oom_kill) was invoked.
Sample codes of both:
First case: while:
int main (void) {
char* p;
while (1) {
p=malloc(1<<20);
memset (p, 0, (1<<20));
}
}
Second case : for :
int main (void) {
int i, n = 0;
char *pp[N];
for (n = 0; n < N; n++) {
pp[n] = malloc(1<<20);
if (pp[n] == NULL)
break;
}
printf("malloc failure after %d MiB\n", n);
for (i = 0; i < n; i++) {
memset (pp[i], 0, (1<<20));
printf("%d\n", i+1);
}
where N is some very large number to invoke oom. Referred this https://www.win.tue.nl/~aeb/linux/lk/lk-9.html for 2nd case.
Why does it happen so? What is the mistake I'm making with the while loop?
Kernel version : 4.15

Answer 1

Why does it happen so?
To invoke the OOM killer, you need to have a situation where an access to memory cannot be fulfilled because there is not enough RAM available to fulfill the access. To do that, you want to first have large allocations (virtual memory mappings), then write to them.
The procedure to trigger the OOM killer is very simple:
Allocate lots of memory
Write to the allocated memory
You must have enough preallocated memory to cause everything evictable from RAM to be evicted (things like memory-mapped files), and all of swap to be used, before the kernel will evoke the OOM killer to provide more RAM/swap space to fullfill the backing to the virtual memory being written to.
What is the mistake I'm making with the while loop?
One bug, and one logical error.
The bug is, you do not check if malloc() returns NULL. malloc() returns NULL, when there is no more virtual memory available (or kernel refuses to provide more, for any reason) for the process. (In normal operation, the virtual memory available to each process is limited for non-privileged users; run e.g. ulimit -a to see the current limits.)
Because you access the memory immediately when allocated, the kernel simply refuses to allow your process more when it runs out of RAM and SWAP, and malloc() returns NULL. You then dereference the NULL pointer (by using memset(NULL, 0, 1<<20)), which causes the Segmentation fault.
The logical problem is that that scheme will not trigger the kernel OOM killer.
Remember, in order to trigger the kernel OOM killer, your process must have allocated memory that it has not accessed yet. The kernel evokes the OOM killer only when it has already provided the virtual memory, but cannot back it with actual RAM, because there is nothing evictable in RAM, and swap is already full.
In your case, the OOM killer will not get evoked, because when the kernel runs out of RAM and swap, it can simply refuse to provide more (virtual memory), leading to malloc() returning NULL.
(The Linux kernel memory subsystem is one that is actively developed, so the exact behaviour you see depends on both the kernel version, the amount of RAM and swap, and the memory manager tunables (e.g., those under /proc/sys/vm/). The above describes the most common, or typical cases and configurations.)
You don't need an external array, either. You can for example chain the allocations to a linked list:
#include <stdlib.h>
#include <stdio.h>
#ifndef SIZE
#define SIZE (2*1024*1024) /* 2 MiB */
#endif
struct list {
struct list *next;
size_t size;
char data[];
}
struct list *allocate_node(const size_t size)
{
struct list *new_node;
new_node = malloc(sizeof (struct list) + size);
if (!new_node)
return NULL;
new_node->next = NULL;
new_node->size = size;
}
int main(void)
{
size_t used = 0;
struct list *root = NULL, *curr;
/* Allocate as much memory as possible. */
while (1) {
curr = allocate_node(SIZE - sizeof (struct list));
if (!curr)
break;
/* Account for allocated total size */
used += SIZE;
/* Prepend to root list */
curr->next = root;
root = curr;
}
printf("Allocated %zu bytes.\n", used);
fflush(stdout);
/* Use all of the allocated memory. */
for (curr = root; curr != NULL; curr = curr->next)
if (curr->size > 0)
memset(curr->data, ~(unsigned char)0, curr->size);
printf("Wrote to %zu bytes of allocated memory. Done.\n", used);
fflush(stdout);
return EXIT_SUCCESS;
}
Note, the above code is untested, and even uncompiled, but the logic is sound. If you find a bug in it, or have some other issue with it, let me know in a comment so I can verify and fix.

Answer 2

The document you're reading is from 2003. The impossibly large number it chose to allocate was 10,000 MiB.
Today, in 2018, when new computers are likely to come with 16GiB of RAM, this kind of allocation could definitely succeed without issues.

Answer 3

What is the mistake I'm making with the while loop?
The segmentation fault is likely the result of passing a null pointer to memset(), since malloc() will return NULL on error.
Your second example avoids this error by always checking the return value from malloc().
I used the while loop it ... became unresponsive with swap ...
From the very document that you mentioned that you are reading:
Sometimes processes get a segfault when accessing memory that the kernel is unable to provide, sometimes they are killed, sometimes other processes are killed, sometimes the kernel hangs.
Other than mentioning the kernel version, you are very vague with the OS and system description. Presumably this is a 32-bit version?
There are actually two ways of running out of memory. Your program could exceed the amount of (virtual) memory that is allocated, or the system could actually run out of memory pages.
Note that availability of memory (pages) is a complex combination of physical memory size, swap space size, memory usage and process load.
Reference: When Linux Runs Out of Memory
by Mulyadi Santosa or here.