how to use Dtrace to check malloc on Solaris 10? - c

I meet a troublesome bug about memory usage, so I want to use Dtrace to check malloc and free on Solaris 10.
I use the following command
dtrace -l | grep malloc
The output is:
7000 fbt unix prom_malloc entry
7001 fbt unix prom_malloc return
7141 fbt genunix cacl_malloc entry
7142 fbt genunix cacl_malloc return
12319 fbt genunix rmallocmap_wait entry
12320 fbt genunix rmallocmap_wait return
13078 fbt genunix rmalloc_wait entry
13079 fbt genunix rmalloc_wait return
13526 fbt genunix rmallocmap entry
13527 fbt genunix rmallocmap return
16846 fbt genunix rmalloc entry
16847 fbt genunix rmalloc return
25931 fbt tmpfs tmp_memalloc entry
25932 fbt tmpfs tmp_memalloc return
It seems there is no malloc.
I have checked Solaris Internal, and found the malloc calls sbrk. So I use the following command:
dtrace -l | grep sbrk
But there is nothing found.
So how can I use Dtrace to check malloc on Solaris 10?

There are various tools that already implement the logic required to identify memory leaks under Solaris,
libumem & mdb (UMEM_DEBUG=default UMEM_LOGGING=transaction LD_PRELOAD=libumem.so.1 then mdb's ::findleaks)
dbx (check -leaks)
Should you still want to go the dtrace way, you need to trace the process you suspect to leak memory using the pid provider. You searched the kernel probes with dtrace -l and found nothing but this is expected as the kernel does not implement malloc or brk. They are userland functions located in the C standard library.
This script will trace every malloc and free calls by a program:
dtrace -qn '
pid$target:libc:malloc:entry {
self->size=arg0;
}
pid$target:libc:malloc:return /self->size/ {
printf("malloc(%d)=%p\n",self->size,arg1);
self->size=0;
}
pid$target:libc:free:entry {
printf("free(%p)\n",arg0);
}
' -c program_to_trace
For more in-depth examples, have a look to http://ewaldertl.blogspot.fr/2010/09/debugging-memory-leaks-with-dtrace-and.html and http://www.joyent.com/blog/bruning-questions-debugging

For example, in order to trace all malloc calls with dtrace, by printing each allocation you would write (to a file named trace-malloc.d in this example) a script like below:
#!/usr/sbin/dtrace -s
pid$1::malloc:entry
{
self->trace = 1;
self->size = arg0;
}
pid$1::malloc:return
/self->trace == 1/
{
/* log the memory allocation */
printf("<__%i;%Y;%d;malloc;0x%x;%d;\n", i++, walltimestamp, tid, arg1, self->size);
ustack(50);
printf("__>\n\n");
self->trace = 0;
self->size = 0;
}
and then call it by passing the process id of the process you want to trace,
for example:
./trace-malloc.d 12345
In complex programs there are very frequent memory allocations and de-allocations so I've written a small program to help me identify memory leaks with dtrace. Each memory operation with malloc / calloc / realloc and free is traced and then an analysis program reads and processes all the traces and points out suspected memory leaks, also using various heuristics to point out strongly suspected memory leaks. If you are interested you can check it out here:
https://github.com/ppissias/DTLeakAnalyzer.

Related

mlock a program from a wrapper

Just a quick question (I hope). How would you allocate an address space via mlock and then launch an application within that space?
For instance I have a binary that launches from a wrapper program that configures the environment. I only have access to the wrapper code and would like to have the binary launch in a certain address space. Is it possible to do this from the wrapper?
Thanks!
If you have the sources for the program, add a command-line option so that the program calls mlockall(MCL_CURRENT | MCL_FUTURE) at some point. That locks it in memory.
If you want to control the address spaces the kernel loads the program into, you need to delve into kernel internals. Most likely, there is no reason to do so; only people with really funky hardware would.
If you don't have the sources, or don't want to recompile the program, then you can create a dynamic library that executes the command, and inject it into the process via LD_PRELOAD.
Save the following as lockall.c:
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
#include <errno.h>
static void wrerr(const char *p)
{
if (p) {
const char *q = p + strlen(p);
ssize_t n;
while (p < q) {
n = write(STDERR_FILENO, p, (size_t)(q - p));
if (n > 0)
p += n;
else
if (n != -1 || errno != EINTR)
return;
}
}
}
static void init(void) __attribute__((constructor));
static void init(void)
{
int saved_errno = errno;
if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
const char *errmsg = strerror(errno);
wrerr("Cannot lock all memory: ");
wrerr(errmsg);
wrerr(".\n");
exit(127);
} else
wrerr("All memory locked.\n");
errno = saved_errno;
}
Compile it to a dynamic library liblockall.so using
gcc -Wall -O2 -fPIC -shared lockall.c -Wl,-soname,liblockall.so -o liblockall.so
Install the library somewhere typical, for example
sudo install -o 0 -g 0 -m 0664 liblockall.so /usr/lib/
so you can run any binary, and lock it into memory, using
LD_PRELOAD=liblockall.so binary arguments..
If you install the library somewhere else (not listed in /etc/ld.so.conf), you'll need to specify path to the library, like
LD_PRELOAD=/usr/lib/liblockall.so binary arguments..
Typically, you'll see the message Cannot lock all memory: Cannot allocate memory. printed by the interposed library, when running commands as a normal user. (The superuser, or root, typically has no such limit.) This is because for obvious reasons, most Linux distributions limit the amount of memory an unprivileged user can lock into memory; this is the RLIMIT_MEMLOCK resource limit. Run ulimit -l to see the per-process resource limits currently set (for the current user, obviously).
I suggest you set a suitable limit of how much memory the process can run, running e.g. ulimit -l 16384 bash-built-in before executing the (to set the limit to 16384*1024 bytes, or 16 MiB), if running as superuser (root). If the process leaks memory, instead of crashing your machine (because it locked all available memory), the process will die (from SIGSEGV) if it exceeds the limit. That is, you'd start your process using
ulimit -l 16384
LD_PRELOAD=/usr/lib/liblockall.so binary arguments..
if using Bash or dash shell.
If running as a dedicated user, most distributions use the pam_limits.so PAM module to set the resource limits "automatically". The limits are listed either in the /etc/security/limits.conf file, or in a file in the /etc/security/limits.d/ subdirectory, using this format; the memlock item specifies the amount of memory each process can lock, in units of 1024 bytes. So, if your service runs as user mydev, and you wish to allow the user to lock up to 16 megabytes = 16384*1024 bytes per process, then add line mydev - memlock 16384 into /etc/security/limits.conf or /etc/security/limits.d/mydev.conf, whichever your Linux distribution prefers/suggests.
Prior to PAM, shadow-utils were used to control the resource limits. The memlock resource limit is specified in units of 1024 bytes; a limit of 16 megabytes would be set using M16384. So, if using shadow-utils instead of PAM, adding line mydev M16384 (followed by whatever the other limits you wish to specify) to /etc/limits should do the trick.

Trigger the invocations of system call in Linux, C program?

When the below C program is run in a Linux,the execution of which line must trigger invocation of system call, why? What is invocation of system call ?
void main()
{
double x=1;
double y;
double *z;
z=(double *)malloc(sizeof(double)); // line 1
scanf("%f", &y); // line 2
*z=sqrt(y); // line 3
y=y*2.0; // line 4
printf("y=%f, *z=%f\n", y, *z); // line 5
y=y/x; // line 6
printf("y=%f",y); // line 7
}
A call to malloc invokes a system call because the operating system manages the memory.
Calls to scanf and printf invoke system calls because the operating system manages i/o operations.
Invocation of a system call is a call for an operating system service.
The confusion above for people is syscall vs libc. Not every malloc will require a syscall - in fact, most shouldn't.
The Heap: LibC needs to manage the memory allocated to the heap(s). This is the area from which all of your individual malloc() allocations are sourced. When the heap is exhausted libc will invoke syscalls to kernel space to request more memory / reallocate / release pages / etc.
Malloc: LibC provides the user level code to assign memory blocks within the heap. It handles all the individual manner in which it allocates and frees this memory. If the allocation can't be done within the existing heap capacity it will trigger the Heap syscall.
If you look at the libc malloc code you'll see how this works between __libc_malloc() and _int_malloc(). In _int_malloc() it will fallback to sysmalloc() which will then perform a kernel memory heap change.

Windows Equivalent for sys/mman.h

I'm encountering issues when trying to compile my C code on Win64. More specifically, the compiler cannot find the sys/mman.h header, which I understand is found in Unix environments only.
I already know this is deals with memory allocation.
Is there an equivalent for Windows I can use in order to port the code (first time trying)?
Code in that causes issues:
/* Allocate memory required by processes */
buf = (int*) malloc (sizeof(int));
if (!buf)
{
perror("Error");
free (buf);
return -3;
}
/* Lock down pages mapped to processes */
puts("Locking down processes.");
if(mlockall (MCL_CURRENT | MCL_FUTURE) < 0)
{
perror("mlockall");
free (buf);
return -4;
}
You should look at the mman-win32 library. But as #Mgetz pointed out, a more simple way is to look at the VirtualAllocEx functions and try to adapt your code.
I was able to get around the issue by using g++ under cygwin, making sure the g++ came from the cygwin installation (the same version specified from the installer) and not the current compiler under windows.

Causing malloc() to return NULL on CentOS

I'm about to teach an introductory computer science course in C and I'd like to demonstrate to students why they should check whether malloc() returned a NULL. My plan was to use ulimit to restrict the amount of available memory such that I could exercise different code paths with different limits. Our prescribed environment is CentOS 6.5.
My first attempts to make this happened failed and the shell showed "Killed". This led to me discovering the Linux OOM killer. I have since tried to figure out the magic set of incantations that will cause the results I'm looking for. Apparently I need to mess with:
/etc/sysctl.conf
ulimit -m
ulimit -v
vm.overcommit_memory (which apparently should be set to 2, according to an Oracle article)
This far either I get "Killed" or a segmentation fault, neither of which is the expected outcome. The fact that I'm getting "Killed" with vm_overcommit_memory=2 means that I definitely don't understand what's going on.
If anyone can find a way to artificially and reliably create a constrained execution environment on CentOS so that students learn how to handle OOM (and other?) kinds of errors, many course instructors will thank you.
It is possible to [effectively] turn off overcommitting from kernel >= 2.5.30.
Following Linux Kernel Memory :
// save your work here and note your current overcommit_ratio value
# echo 2 > overcommit_memory
# echo 1 > overcommit_ratio
this sets the VM_OVERCOMMIT_MEMORY to 2 indicating not to overcommit past the overcommit_ratio, which is set to 1 (ie no overcommitting)
Null malloc demo
#include <stdio.h>
#include <stdlib.h>
int main(int argc,char *argv[])
{
void *page = 0; int index;
void *pages[256];
index = 0;
while(1)
{
page = malloc(1073741824); //1GB
if(!page)break;
pages[index] = page;
++index;
if(index >= 256)break;
}
if(index >= 256)
{
printf("allocated 256 pages\n");
}
else
{
printf("memory failed at %d\n",index);
}
while(index > 0)
{
--index;
free(pages[index]);
}
return 0;
}
Output
$ cat /proc/sys/vm/overcommit_memory
0
$ cat /proc/sys/vm/overcommit_ratio
50
$ ./code/stackoverflow/test-memory
allocated 256 pages
$ su
# echo 2 > /proc/sys/vm/overcommit_memory
# echo 1 > /proc/sys/vm/overcommit_ratio
# exit
exit
$ cat /proc/sys/vm/overcommit_memory
2
$ cat /proc/sys/vm/overcommit_ratio
1
$ ./code/stackoverflow/test-memory
memory failed at 0
remember to restore your overcommit_memory to 0 and overcommit_ratio as noted

Buffer Overflow not working

I was trying to do a buffer overflow (I'm using Linux) on a simple program that requires a password. Here's the program code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int check_authentication(char *password){
int auth_flag = 0;
char password_buffer[16];
strcpy(password_buffer, password);
if(strcmp(password_buffer, "pass1") == 0)
auth_flag = 1;
if(strcmp(password_buffer, "pass2") == 0)
auth_flag = 1;
return auth_flag;
}
int main(int argc, char **argv)
{
if(argc < 2){
printf("\t[!] Correct usage: %s <password>\n", argv[0]);
exit(0);
}
if(check_authentication(argv[1])){
printf("\n-=-=-=-=-=-=-=-=\n");
printf(" Access granted.\n");
printf("-=-=-=-=-=-=-=-=\n");
} else {
printf("\nAccess Denied.\n");
}
return 0;
}
OK, now I compiled it, no errors, and saved it as overflow.c.
Now I opened the Terminal, I moved into the file directory (Desktop) and then wrote:
./overflow.c AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
The Terminal said: "Stack smashing detected" (or something like that) and then quit the program execution.
Now, I'm reading a book, called "Hacking - The Art Of Exploitation" by Jon Erickson. In a chapter, he explains this type of exploit (I took the code from the book) and does the same command I've done. The memory overflows and the program prints "Access granted.". Now, why my OS is detecting I'm trying to exploit the program? I've done something wrong?
I also tried the exploit on Mac OS X. Same thing happened. Please, can someone help me? Thanks in advance.
In modern linux distributions buffer overflow is detected and the process is killed. In order to disable that mode simply compile your application with such flags (gcc):
-fno-stack-protector -fno-stack-protector-all
If compiling with gcc, add -fno-stack-protector flag. The message you received is meant to protect you from your bad code :)
The reason is stack smashing is actually a protection mechanism used by some compilers to detect buffer overflow attacks. You are trying to put the 29 A's into a shorter character array (16 bytes).
Most modern OS have protective mechanisms built in. Almost any good OS does not allow direct low level memory access to any program. It only allows programs to access the adress space allocated to them. Linux based OS automatically kill the processes that try to access beyond their allocated memory space.
Other than this, OS also have protective mechanisms that prevent a program from crashing the system by allocating large amounts of memory, in an attempt to severely deplete the resources available to the OS.

Resources