I have my program, which exec's another process (not mine, consider it a blackbox). Is there a way to detect operations, like open() and close(), for this child process?
Especially I'm interested in finding all newly created files, or existing files, that are opened with intention to be created (O_CREAT flag for open()).
The working approach is to redefine the open() within my own shared library and preload it inside exec()'ed process via LD_PRELOAD environment variable. Thanks to #alk for the approach.
The code for redefined open() looks like:
#include <fcntl.h>
#include <dlfcn.h>
#include <stdarg.h>
#include <sys/types.h>
extern "C" {
int open(const char *pathname, int flags, ...) {
bool has_mode = false;
mode_t mode = 0;
if (flags & O_CREAT) {
va_list ap;
va_start(ap, flags);
mode = va_arg(ap, mode_t);
has_mode = true;
va_end(ap);
}
using Fn = int (*)(const char * pathname, int flags, ...);
Fn new_open = reinterpret_cast<Fn>(dlsym(RTLD_NEXT, "open"));
// Do something useful.
if (has_mode) {
return new_open(pathname, flags, mode);
} else {
return new_open(pathname, flags);
}
}
} // extern "C"
The only problem is with fcntl.h - it may have some geeky declaration for the function open(). You need this file to get definition of the O_CREAT. Another way is to include the file with definition directly: in my case it's the file asm-generic/fcntl.h.
Related
If dlsym is available in dynamic linking setup, I can get access to the original impl pointers using dlsym with RTLD_NEXT and use them in my overrides, e.g. as follows:
// paste these in main.c
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <dlfcn.h>
int open(const char *path, int flags)
{
fprintf(stderr, "log_file_access_preload: open(\"%s\", %d)\n", path, flags);
typedef int (*orig_open_func_type)(const char *pathname, int flags);
orig_open_func_type orig_func = (orig_open_func_type)dlsym(RTLD_NEXT, "open");
return orig_func(path, flags);
}
FILE* fopen(const char *path, const char *mode)
{
fprintf(stderr, "log_file_access_preload: fopen(\"%s\", \"%s\")\n", path, mode);
typedef FILE* (*orig_fopen_func_type)(const char *path, const char *mode);
orig_fopen_func_type orig_func = (orig_fopen_func_type)dlsym(RTLD_NEXT, "fopen");
return orig_func(path, mode);
}
Is there a way to do static linking in such a way that doesn't hide the original libc/POSIX symbols and so that I can use them in my overrides? Should I create my own copy of musl *.a files with renamed original symbols? Should it work? Is there another way?
Usecase: implement redirection of file read/access functions for a custom LaTeX program (compilation process is controlled by me, statically built with musl) to read files from ISO or TAR archive (that contains a prepared TeX Directory Structure) without extraction to disk
I am trying to find somewhat elegant ways to mock and stub function calls to the standard C library functions.
While stubbing-off calls to C files of the project is easy by just linking other C files in the tests, stubbing the standard C functions is harder.
They are just there when linking.
Currently, my approach is to include the code-under-test from my test.cpp file, and placing defines like this:
#include <stdio.h>
#include <gtest/gtest.h>
#include "mymocks.h"
CMockFile MockFile;
#define open MockFile.open
#define close MockFile.close
#define read MockFile.read
#include "CodeUnderTestClass.cpp"
#undef open
#undef close
#undef read
// test-class here
This is cumbersome, and sometimes I run across code that uses 'open' as member names elsewhere or causes other collisions and issues with it. There are also cases of the code needing different defines and includes than the test-code.
So are there alternatives? Some link-time tricks or runtime tricks to override standard C functions? I thought about run-time hooking the functions but that might go too far as usually binary code is loaded read-only.
My unit-tests run only on Debian-Linux with gcc on amd64. So gcc, x64 or Linux specific tricks are also welcome.
I know that rewriting all the code-under-test to use an abstracted version of the C functions is an option, but that hint is not very useful for me.
Use library preloading to substitute system libraries with your own.
Consider following test program code, mytest.c:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main(void) {
char buf[256];
int fd = open("file", O_RDONLY);
if (fd >= 0) {
printf("fd == %d\n", fd);
int r = read(fd, buf, sizeof(buf));
write(0, buf, r);
close(fd);
} else {
printf("can't open file\n");
}
return 0;
}
It will open a file called file from the current directory, print it's descriptor number (usually 3), read its content and then print it on the standard output (descriptor 0).
Now here is your test library code, mock.c:
#include <string.h>
#include <unistd.h>
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
Compile it to a shared library called mock.so:
$ gcc -shared -fpic -o mock.so mock.c
If you compiled mytest.c to the mytest binary, run it with following command:
$ LD_PRELOAD=./mock.so ./mytest
You should see the output:
fd == 100
TEST!
Functions defined in mock.c were preloaded and used as a first match during the dynamic linking process, hence executing your code, and not the code from the system libraries.
Update:
If you want to use "original" functions, you should extract them "by hand" from the proper shared library, using dlopen, dlmap and dlclose functions. Because I don't want to clutter previous example, here's the new one, the same as previous mock.c plus dynamic symbol loading stuff:
#include <stdio.h>
#include <dlfcn.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <gnu/lib-names.h>
// this declares this function to run before main()
static void startup(void) __attribute__ ((constructor));
// this declares this function to run after main()
static void cleanup(void) __attribute__ ((destructor));
static void *sDlHandler = NULL;
ssize_t (*real_write)(int fd, const void *buf, size_t count) = NULL;
void startup(void) {
char *vError;
sDlHandler = dlopen(LIBC_SO, RTLD_LAZY);
if (sDlHandler == NULL) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
real_write = (ssize_t (*)(int, const void *, size_t))dlsym(sDlHandler, "write");
vError = dlerror();
if (vError != NULL) {
fprintf(stderr, "%s\n", vError);
exit(EXIT_FAILURE);
}
}
void cleanup(void) {
dlclose(sDlHandler);
}
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
ssize_t write(int fd, const void *buf, size_t count) {
if (fd == 0) {
real_write(fd, "mock: ", 6);
}
real_write(fd, buf, count);
return count;
}
Compile it with:
$ gcc -shared -fpic -o mock.so mock.c -ldl
Note the -ldl at the end of the command.
So: startup function will run before main (so you don't need to put any initialization code in your original program) and initialize real_write to be the original write function. cleanup function will run after main, so you don't need to add any "cleaning" code at the end of main function either.
All the rest works exactly the same as in the previous example, with the exception of newly implemented write function. For almost all the descriptors it will work as the original, and for file descriptor 0 it will write some extra data before the original content. In that case the output of the program will be:
$ LD_PRELOAD=./mock.so ./mytest
fd == 100
mock: TEST!
I want to implement a wrapper over the open system call, and I'm using the LD_PRELOAD trick to call my new open.
The problem is that open expects a variable number of arguments, and I cannot figure out how to call open with the same set of arguments, as open cannot take a va_list pointer as an argument(or I do not know of any such function).
How could I achieve this?
code so far:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
typedef int (*open_func_t)(const char*, int, ...)
int open(const char *pathname, int flags, ...){
// some custom code
// what args should I supply to dlsym?
return ((open_func_t)dlsym(RTLD_NEXT, "open"))(args);
}
So the problem is the following. The project needs to intercept all file IO
operations, like open() and close(). I am trying to add printf() before calling the corresponding open() or close(). I am not supposed to rewrite the source code by changing open() or close() to myOpen() or myClose() for example. I have been trying to use LD_PRELOAD environment variable. But the indefinite loop problem came up. My problem is like this one.
int open(char * path,int flags,int mode)
{
// print file name
printf("open :%s\n",path);
return __open(path,flags,mode);
}
Yes, you want LD_PRELOAD.
You need to create a shared library (.so) that has code for all functions that you want to intercept. And, you want to set LD_PRELOAD to use that shared library
Here is some sample code for the open function. You'll need to do something similar for each function you want to intercept:
#define _GNU_SOURCE
#include <dlfcn.h>
int
open(const char *file,int flags,int mode)
{
static int (*real_open)(const char *file,int flags,int mode) = NULL;
int fd;
if (real_open == NULL)
real_open = dlsym(RTLD_NEXT,"open");
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
I believe RTLD_NEXT is easiest and may be sufficient. Otherwise, you could add a constructor that does dlopen once on libc
UPDATE:
I am not familiar with C and I got the following problems with gcc. "error: 'NULL' undeclared (first use in this function)",
This is defined by several #include files, so try #include <stdio.h>. You'll need that if you want to call printf.
"error: 'RTLD_NEXT' undeclared (first use in this function)",
That is defined by doing #include <dlfcn.h> [as shown in my example]
and "symbol lookup error: ./hack_stackoverflow.so: undefined symbol: dlsym".
From man dlsym, it says: Link with -ldl So, add -ldl to the line that builds your .so.
Also, you have to be careful to prevent infinite recursion if the "special stuff" does something that loops back on your intercept function.
Notably, you want to call printf. If you intercept the write syscall, bad things may happen.
So, you need to keep track of when you're already in one of your intercept functions and not do anything special if already there. See the in_self variable.
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
ssize_t
write(int fd,const void *buf,size_t len)
{
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
static int in_self = 0;
ssize_t err;
if (real_write == NULL)
real_write = dlsym(RTLD_NEXT,"write");
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
err = real_write(fd,buf,len);
if (in_self == 1)
printf("mywrite: fd=%d buf=%p err=%ld\n",fd,buf,err);
--in_self;
return err;
}
The above works okay for single threaded programs/environments, but if you're intercepting an arbitrary one, it could be multithreaded.
So, we'd have to initialize all the real_* pointers in a constructor. This is a function with a special attribute that tells the dynamic loader to call the function ASAP automatically.
And, we have to put in_self into thread local storage. We do this by adding the __thread attribute.
You may need to link with -lpthread as well as -ldl for the multithreaded version.
Edit: We also have to preserve the correct errno value
Putting it all together:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <errno.h>
static int (*real_open)(const char *file,int flags,int mode) = NULL;
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
__attribute__((constructor))
void
my_lib_init(void)
{
real_open = dlsym(RTLD_NEXT,"open");
real_write = dlsym(RTLD_NEXT,"write");
}
int
open(const char *file,int flags,int mode)
{
int fd;
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
ssize_t
write(int fd,const void *buf,size_t len)
{
static int __thread in_self = 0;
int sverr;
ssize_t ret;
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
ret = real_write(fd,buf,len);
// preserve errno value for actual syscall -- otherwise, errno may
// be set by the following printf and _caller_ will get the _wrong_
// errno value
sverr = errno;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p ret=%ld\n",fd,buf,ret);
--in_self;
// restore correct errno value for write syscall
errno = sverr;
return ret;
}
I have an application that has both two external kernel modules and a userspace daemon. I want to load the modules from the daemon code, written in C, at startup, and unload them on clean exit. Can I load them in a cleaner way than doing system("modprobe module"); and unload them using the corresponding rmmod?
init_module / remove_module minimal runnable example
Tested on a QEMU + Buildroot VM and Ubuntu 16.04 host with this simple parameter printer module .
We use the init_module / finit_module and remove_module Linux system calls.
The Linux kernel offers two system calls for module insertion:
init_module
finit_module
and:
man init_module
documents that:
The finit_module() system call is like init_module(), but reads the module to be loaded from the file descriptor fd. It is useful when the authenticity of a kernel module can be determined from its location in the filesystem; in cases where that is possible, the overhead of using cryptographically signed modules to determine the authenticity of a module can be avoided. The param_values argument is as for init_module().
finit is newer and was added only in v3.8. More rationale: https://lwn.net/Articles/519010/
glibc does not seem to provide a C wrapper for them, so we just create our own with syscall.
insmod.c
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#define init_module(module_image, len, param_values) syscall(__NR_init_module, module_image, len, param_values)
#define finit_module(fd, param_values, flags) syscall(__NR_finit_module, fd, param_values, flags)
int main(int argc, char **argv) {
const char *params;
int fd, use_finit;
size_t image_size;
struct stat st;
void *image;
/* CLI handling. */
if (argc < 2) {
puts("Usage ./prog mymodule.ko [args="" [use_finit=0]");
return EXIT_FAILURE;
}
if (argc < 3) {
params = "";
} else {
params = argv[2];
}
if (argc < 4) {
use_finit = 0;
} else {
use_finit = (argv[3][0] != '0');
}
/* Action. */
fd = open(argv[1], O_RDONLY);
if (use_finit) {
puts("finit");
if (finit_module(fd, params, 0) != 0) {
perror("finit_module");
return EXIT_FAILURE;
}
close(fd);
} else {
puts("init");
fstat(fd, &st);
image_size = st.st_size;
image = malloc(image_size);
read(fd, image, image_size);
close(fd);
if (init_module(image, image_size, params) != 0) {
perror("init_module");
return EXIT_FAILURE;
}
free(image);
}
return EXIT_SUCCESS;
}
GitHub upstream.
rmmod.c
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#define delete_module(name, flags) syscall(__NR_delete_module, name, flags)
int main(int argc, char **argv) {
if (argc != 2) {
puts("Usage ./prog mymodule");
return EXIT_FAILURE;
}
if (delete_module(argv[1], O_NONBLOCK) != 0) {
perror("delete_module");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
GitHub upstream.
Busybox source interpretation
Busybox provides insmod, and since it is designed for minimalism, we can try to deduce how it is done from there.
On version 1.24.2, the entry point is at modutils/insmod.c function insmod_main.
The IF_FEATURE_2_4_MODULES is optional support for older Linux kernel 2.4 modules, so we can just ignore it for now.
That just forwards to modutils.c function bb_init_module.
bb_init_module attempts two things:
mmap the file to memory through try_to_mmap_module.
This always sets image_size to the size of the .ko file as a side effect.
if that fails, malloc the file to memory with xmalloc_open_zipped_read_close.
This function optionally unzips the file first if it is a zip, and just mallocs it otherwise.
I don't understand why this zipping business is done, since we can't even rely on it because the try_to_mmap_module does not seem to unzip things.
Finally comes the call:
init_module(image, image_size, options);
where image is the executable that was put into memory, and options are just "" if we call insmod file.elf without further arguments.
init_module is provided above by:
#ifdef __UCLIBC__
extern int init_module(void *module, unsigned long len, const char *options);
extern int delete_module(const char *module, unsigned int flags);
#else
# include <sys/syscall.h>
# define init_module(mod, len, opts) syscall(__NR_init_module, mod, len, opts)
# define delete_module(mod, flags) syscall(__NR_delete_module, mod, flags)
#endif
ulibc is an embedded libc implementation, and it seems to provide init_module.
If it is not present, I think glibc is assumed, but as man init_module says:
The init_module() system call is not supported by glibc. No declaration is provided in glibc headers, but, through a quirk of history, glibc does export an ABI for
this system call. Therefore, in order to employ this system call, it is sufficient to manually declare the interface in your code; alternatively, you can invoke
the system call using syscall(2).
BusyBox wisely follows that advice and uses syscall, which glibc provides, and which offers a C API for system calls.
insmod/rmmod use the functions init_module and delete_module to do this, which also have a man-page available. They both declare the functions as extern instead of including a header, but the man-page says they should be in <linux/module.h>.
I'd recommend against the use of system() in any daemon code that runs with root permissions as it's relatively easy to exploit from a security standpoint. modprobe and rmmod are, indeed, the right tools for the job. However, it'd be a bit cleaner and much more secure to use an explicit fork() + exec() to invoke them.
I'm not sure there's a cleaner way than system.
But for sure, if you want to load/unload the modules from your userspace daemon, then you force yourself to run the daemon as root*, which may not be considered as secure.
*: or you can add the explicit commands in the sudoers file, but this will be a nightmare to manage when deploying your application.
You can perform the same tasks that modprobe and Co. do, but I doubt that could be characterized as cleaner.