I have an application that has both two external kernel modules and a userspace daemon. I want to load the modules from the daemon code, written in C, at startup, and unload them on clean exit. Can I load them in a cleaner way than doing system("modprobe module"); and unload them using the corresponding rmmod?
init_module / remove_module minimal runnable example
Tested on a QEMU + Buildroot VM and Ubuntu 16.04 host with this simple parameter printer module .
We use the init_module / finit_module and remove_module Linux system calls.
The Linux kernel offers two system calls for module insertion:
init_module
finit_module
and:
man init_module
documents that:
The finit_module() system call is like init_module(), but reads the module to be loaded from the file descriptor fd. It is useful when the authenticity of a kernel module can be determined from its location in the filesystem; in cases where that is possible, the overhead of using cryptographically signed modules to determine the authenticity of a module can be avoided. The param_values argument is as for init_module().
finit is newer and was added only in v3.8. More rationale: https://lwn.net/Articles/519010/
glibc does not seem to provide a C wrapper for them, so we just create our own with syscall.
insmod.c
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#define init_module(module_image, len, param_values) syscall(__NR_init_module, module_image, len, param_values)
#define finit_module(fd, param_values, flags) syscall(__NR_finit_module, fd, param_values, flags)
int main(int argc, char **argv) {
const char *params;
int fd, use_finit;
size_t image_size;
struct stat st;
void *image;
/* CLI handling. */
if (argc < 2) {
puts("Usage ./prog mymodule.ko [args="" [use_finit=0]");
return EXIT_FAILURE;
}
if (argc < 3) {
params = "";
} else {
params = argv[2];
}
if (argc < 4) {
use_finit = 0;
} else {
use_finit = (argv[3][0] != '0');
}
/* Action. */
fd = open(argv[1], O_RDONLY);
if (use_finit) {
puts("finit");
if (finit_module(fd, params, 0) != 0) {
perror("finit_module");
return EXIT_FAILURE;
}
close(fd);
} else {
puts("init");
fstat(fd, &st);
image_size = st.st_size;
image = malloc(image_size);
read(fd, image, image_size);
close(fd);
if (init_module(image, image_size, params) != 0) {
perror("init_module");
return EXIT_FAILURE;
}
free(image);
}
return EXIT_SUCCESS;
}
GitHub upstream.
rmmod.c
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#define delete_module(name, flags) syscall(__NR_delete_module, name, flags)
int main(int argc, char **argv) {
if (argc != 2) {
puts("Usage ./prog mymodule");
return EXIT_FAILURE;
}
if (delete_module(argv[1], O_NONBLOCK) != 0) {
perror("delete_module");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
GitHub upstream.
Busybox source interpretation
Busybox provides insmod, and since it is designed for minimalism, we can try to deduce how it is done from there.
On version 1.24.2, the entry point is at modutils/insmod.c function insmod_main.
The IF_FEATURE_2_4_MODULES is optional support for older Linux kernel 2.4 modules, so we can just ignore it for now.
That just forwards to modutils.c function bb_init_module.
bb_init_module attempts two things:
mmap the file to memory through try_to_mmap_module.
This always sets image_size to the size of the .ko file as a side effect.
if that fails, malloc the file to memory with xmalloc_open_zipped_read_close.
This function optionally unzips the file first if it is a zip, and just mallocs it otherwise.
I don't understand why this zipping business is done, since we can't even rely on it because the try_to_mmap_module does not seem to unzip things.
Finally comes the call:
init_module(image, image_size, options);
where image is the executable that was put into memory, and options are just "" if we call insmod file.elf without further arguments.
init_module is provided above by:
#ifdef __UCLIBC__
extern int init_module(void *module, unsigned long len, const char *options);
extern int delete_module(const char *module, unsigned int flags);
#else
# include <sys/syscall.h>
# define init_module(mod, len, opts) syscall(__NR_init_module, mod, len, opts)
# define delete_module(mod, flags) syscall(__NR_delete_module, mod, flags)
#endif
ulibc is an embedded libc implementation, and it seems to provide init_module.
If it is not present, I think glibc is assumed, but as man init_module says:
The init_module() system call is not supported by glibc. No declaration is provided in glibc headers, but, through a quirk of history, glibc does export an ABI for
this system call. Therefore, in order to employ this system call, it is sufficient to manually declare the interface in your code; alternatively, you can invoke
the system call using syscall(2).
BusyBox wisely follows that advice and uses syscall, which glibc provides, and which offers a C API for system calls.
insmod/rmmod use the functions init_module and delete_module to do this, which also have a man-page available. They both declare the functions as extern instead of including a header, but the man-page says they should be in <linux/module.h>.
I'd recommend against the use of system() in any daemon code that runs with root permissions as it's relatively easy to exploit from a security standpoint. modprobe and rmmod are, indeed, the right tools for the job. However, it'd be a bit cleaner and much more secure to use an explicit fork() + exec() to invoke them.
I'm not sure there's a cleaner way than system.
But for sure, if you want to load/unload the modules from your userspace daemon, then you force yourself to run the daemon as root*, which may not be considered as secure.
*: or you can add the explicit commands in the sudoers file, but this will be a nightmare to manage when deploying your application.
You can perform the same tasks that modprobe and Co. do, but I doubt that could be characterized as cleaner.
Related
I am trying to find somewhat elegant ways to mock and stub function calls to the standard C library functions.
While stubbing-off calls to C files of the project is easy by just linking other C files in the tests, stubbing the standard C functions is harder.
They are just there when linking.
Currently, my approach is to include the code-under-test from my test.cpp file, and placing defines like this:
#include <stdio.h>
#include <gtest/gtest.h>
#include "mymocks.h"
CMockFile MockFile;
#define open MockFile.open
#define close MockFile.close
#define read MockFile.read
#include "CodeUnderTestClass.cpp"
#undef open
#undef close
#undef read
// test-class here
This is cumbersome, and sometimes I run across code that uses 'open' as member names elsewhere or causes other collisions and issues with it. There are also cases of the code needing different defines and includes than the test-code.
So are there alternatives? Some link-time tricks or runtime tricks to override standard C functions? I thought about run-time hooking the functions but that might go too far as usually binary code is loaded read-only.
My unit-tests run only on Debian-Linux with gcc on amd64. So gcc, x64 or Linux specific tricks are also welcome.
I know that rewriting all the code-under-test to use an abstracted version of the C functions is an option, but that hint is not very useful for me.
Use library preloading to substitute system libraries with your own.
Consider following test program code, mytest.c:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main(void) {
char buf[256];
int fd = open("file", O_RDONLY);
if (fd >= 0) {
printf("fd == %d\n", fd);
int r = read(fd, buf, sizeof(buf));
write(0, buf, r);
close(fd);
} else {
printf("can't open file\n");
}
return 0;
}
It will open a file called file from the current directory, print it's descriptor number (usually 3), read its content and then print it on the standard output (descriptor 0).
Now here is your test library code, mock.c:
#include <string.h>
#include <unistd.h>
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
Compile it to a shared library called mock.so:
$ gcc -shared -fpic -o mock.so mock.c
If you compiled mytest.c to the mytest binary, run it with following command:
$ LD_PRELOAD=./mock.so ./mytest
You should see the output:
fd == 100
TEST!
Functions defined in mock.c were preloaded and used as a first match during the dynamic linking process, hence executing your code, and not the code from the system libraries.
Update:
If you want to use "original" functions, you should extract them "by hand" from the proper shared library, using dlopen, dlmap and dlclose functions. Because I don't want to clutter previous example, here's the new one, the same as previous mock.c plus dynamic symbol loading stuff:
#include <stdio.h>
#include <dlfcn.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <gnu/lib-names.h>
// this declares this function to run before main()
static void startup(void) __attribute__ ((constructor));
// this declares this function to run after main()
static void cleanup(void) __attribute__ ((destructor));
static void *sDlHandler = NULL;
ssize_t (*real_write)(int fd, const void *buf, size_t count) = NULL;
void startup(void) {
char *vError;
sDlHandler = dlopen(LIBC_SO, RTLD_LAZY);
if (sDlHandler == NULL) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
real_write = (ssize_t (*)(int, const void *, size_t))dlsym(sDlHandler, "write");
vError = dlerror();
if (vError != NULL) {
fprintf(stderr, "%s\n", vError);
exit(EXIT_FAILURE);
}
}
void cleanup(void) {
dlclose(sDlHandler);
}
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
ssize_t write(int fd, const void *buf, size_t count) {
if (fd == 0) {
real_write(fd, "mock: ", 6);
}
real_write(fd, buf, count);
return count;
}
Compile it with:
$ gcc -shared -fpic -o mock.so mock.c -ldl
Note the -ldl at the end of the command.
So: startup function will run before main (so you don't need to put any initialization code in your original program) and initialize real_write to be the original write function. cleanup function will run after main, so you don't need to add any "cleaning" code at the end of main function either.
All the rest works exactly the same as in the previous example, with the exception of newly implemented write function. For almost all the descriptors it will work as the original, and for file descriptor 0 it will write some extra data before the original content. In that case the output of the program will be:
$ LD_PRELOAD=./mock.so ./mytest
fd == 100
mock: TEST!
So the problem is the following. The project needs to intercept all file IO
operations, like open() and close(). I am trying to add printf() before calling the corresponding open() or close(). I am not supposed to rewrite the source code by changing open() or close() to myOpen() or myClose() for example. I have been trying to use LD_PRELOAD environment variable. But the indefinite loop problem came up. My problem is like this one.
int open(char * path,int flags,int mode)
{
// print file name
printf("open :%s\n",path);
return __open(path,flags,mode);
}
Yes, you want LD_PRELOAD.
You need to create a shared library (.so) that has code for all functions that you want to intercept. And, you want to set LD_PRELOAD to use that shared library
Here is some sample code for the open function. You'll need to do something similar for each function you want to intercept:
#define _GNU_SOURCE
#include <dlfcn.h>
int
open(const char *file,int flags,int mode)
{
static int (*real_open)(const char *file,int flags,int mode) = NULL;
int fd;
if (real_open == NULL)
real_open = dlsym(RTLD_NEXT,"open");
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
I believe RTLD_NEXT is easiest and may be sufficient. Otherwise, you could add a constructor that does dlopen once on libc
UPDATE:
I am not familiar with C and I got the following problems with gcc. "error: 'NULL' undeclared (first use in this function)",
This is defined by several #include files, so try #include <stdio.h>. You'll need that if you want to call printf.
"error: 'RTLD_NEXT' undeclared (first use in this function)",
That is defined by doing #include <dlfcn.h> [as shown in my example]
and "symbol lookup error: ./hack_stackoverflow.so: undefined symbol: dlsym".
From man dlsym, it says: Link with -ldl So, add -ldl to the line that builds your .so.
Also, you have to be careful to prevent infinite recursion if the "special stuff" does something that loops back on your intercept function.
Notably, you want to call printf. If you intercept the write syscall, bad things may happen.
So, you need to keep track of when you're already in one of your intercept functions and not do anything special if already there. See the in_self variable.
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
ssize_t
write(int fd,const void *buf,size_t len)
{
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
static int in_self = 0;
ssize_t err;
if (real_write == NULL)
real_write = dlsym(RTLD_NEXT,"write");
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
err = real_write(fd,buf,len);
if (in_self == 1)
printf("mywrite: fd=%d buf=%p err=%ld\n",fd,buf,err);
--in_self;
return err;
}
The above works okay for single threaded programs/environments, but if you're intercepting an arbitrary one, it could be multithreaded.
So, we'd have to initialize all the real_* pointers in a constructor. This is a function with a special attribute that tells the dynamic loader to call the function ASAP automatically.
And, we have to put in_self into thread local storage. We do this by adding the __thread attribute.
You may need to link with -lpthread as well as -ldl for the multithreaded version.
Edit: We also have to preserve the correct errno value
Putting it all together:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <errno.h>
static int (*real_open)(const char *file,int flags,int mode) = NULL;
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
__attribute__((constructor))
void
my_lib_init(void)
{
real_open = dlsym(RTLD_NEXT,"open");
real_write = dlsym(RTLD_NEXT,"write");
}
int
open(const char *file,int flags,int mode)
{
int fd;
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
ssize_t
write(int fd,const void *buf,size_t len)
{
static int __thread in_self = 0;
int sverr;
ssize_t ret;
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
ret = real_write(fd,buf,len);
// preserve errno value for actual syscall -- otherwise, errno may
// be set by the following printf and _caller_ will get the _wrong_
// errno value
sverr = errno;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p ret=%ld\n",fd,buf,ret);
--in_self;
// restore correct errno value for write syscall
errno = sverr;
return ret;
}
I want to write a program using the new SCHED_DEADLINE scheduling policy available since Linux 3.14.
I start out with a simple program trying to use the sched_setattr function.
#include <sched.h>
int main(void)
{
// struct sched_attr attr;
// attr.size = sizeof(struct sched_attr);
// attr.sched_policy = SCHED_DEADLINE;
sched_setattr(0, (void*)0, 0);
return 0;
}
However when compiling I get the following error:
$gcc dead.c
dead.c: In function ‘main’:
dead.c:8:2: warning: implicit declaration of function ‘sched_setattr’ [-Wimplicit-function-declaration]
sched_setattr(0, (void*)0, 0);
^~~~~~~~~~~~~
/tmp/ccGxWxZE.o: In function `main':
dead.c:(.text+0x19): undefined reference to `sched_setattr'
collect2: error: ld returned 1 exit status
My system is running Ubuntu 16.10 Yakkety, with kernel 4.8.0-59-generic. The sched.h file included is found in /usr/include/sched.h and is provided by the package libc6-dev. This headerfile does not contain the function sched_setattr and friends that I am trying to use.
However the kernel (and kernel headers) I have installed comes with a sched.h header file containing the definitions I need. It is located at /usr/src/linux-headers-4.8.0-58/include/linux/sched.h, on my system.
So I naively think lets just build against the newer linux headers instead of the libc6-dev provided headers. My program will only run on this or newer kernels, but that is just fine.
I modify the first line to be: #include <linux/sched.h> and execute:
gcc -I/usr/src/linux-headers-$(uname -r)/include -I/usr/src/linux-headers-$(unam -r)/arch/x86/include dead.c
Now I am getting page after page of errors and warning. This does not seem the way to go.
What is the correct way to build a userspace program against a newer Linux headers than those that are provided by libc?
And subsequently how do I build the program above?
sched_setattr() is a syscall and doesn't seem to have one-to-one libc wrapper. You could do the wrapper yourself, something like this:
#define _GNU_SOURCE
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <unistd.h>
#include <linux/sched.h>
#include <sys/syscall.h>
#include <sys/types.h>
struct sched_attr {
uint32_t size; /* Size of this structure */
uint32_t sched_policy; /* Policy (SCHED_*) */
uint64_t sched_flags; /* Flags */
int32_t sched_nice; /* Nice value (SCHED_OTHER, SCHED_BATCH) */
uint32_t sched_priority; /* Static priority (SCHED_FIFO, SCHED_RR) */
/* Remaining fields are for SCHED_DEADLINE */
uint64_t sched_runtime;
uint64_t sched_deadline;
uint64_t sched_period;
};
static int sched_setattr (pid_t pid, const struct sched_attr *attr, unsigned int flags)
{
return syscall (SYS_sched_setattr, pid, attr, flags);
}
int main (int argc, char *argv[])
{
struct sched_attr attr;
int res;
memset (&attr, 0, sizeof (struct sched_attr));
attr.size = sizeof (struct sched_attr);
res = sched_setattr (getpid (), &attr, 0);
if (res < 0) {
perror ("sched_setattr");
return 1;
}
return 0;
}
Looking at the errors reported when trying to include kernel header files required to get the definition of struct sched_attr and reading the comments found by Googling "kernel headers in user space", I really can't suggest trying to include kernel header files just for this.
I'm trying to write a portable function in c that compares the last modified times of 2 files. The files are tiny and written one right after the other, so I need finer granularity than 1 second (milliseconds).
There seems to be a plethora of time/date functions...
You should look to the stat() function. It's available on *nix and on windows.
They will both return you a struct containing a field name st_msize. They are the finest functions I have heard of in order to get this kind of information from an Operating System.
Since you need portability, beware to take care of the various different types available on Windows. On *NIX, it's a classic time_t structure. If you include specific call, you can obtain nano seconds mtime: it was defined in POSIX.1-2008, according to the man page.
You can also take a look at how you can deal with 64/32 bit time_t
The C standard does not have any functions for this, but the Posix specification does. The 2008 edition even provides sub-second timestamps. #define _POSIX_C_SOURCE 200809L
The following code should give you an idea how to use it.
#define _POSIX_C_SOURCE 200809L
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h> // for printf
#include <stdlib.h> // for EXIT_FAILURE
int main(int argc, char **argv)
{
for (int i = 1; i < argc; ++i) {
struct stat st = {0};
int ret = lstat(argv[i], &st);
if (ret == -1) {
perror("lstat");
return EXIT_FAILURE;
}
printf("%s: mtime sec=%lld nsec=%lld\n", argv[i],
(long long) st.st_mtim.tv_sec,
(long long) st.st_mtim.tv_nsec);
}
return 0;
}
For POSIX UNIX, stat() is portable and gives struct stat st_mtime which is the modification time in epoch seconds. Windows stat returns windows time values, and has creation time rather than st_ctime.
For non-POSIX UNIX implementations, Windows and other OSes there is no portable concept of file modification time. So, depending on your idea of portable, this whole concept may or my not work for you.
Here's my code (created just to test fork()):
#include <stdio.h>
#include <ctype.h>
#include <limits.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
int main()
{
int pid;
pid=fork();
if (pid==0) {
printf("I am the child\n");
printf("my pid=%d\n", getpid());
}
return 0;
}
I get following warnings:
warning: implicit declaration of function 'fork'
undefined reference to 'fork'
What is wrong with it?
unistd.h and fork are part of the POSIX standard. They aren't available on windows (text.exe in your gcc command hints that's you're not on *nix).
It looks like you're using gcc as part of MinGW, which does provide the unistd.h header but does not implement functions like fork. Cygwin does provide implementations of functions like fork.
However, since this is homework you should already have instructions on how to obtain a working environment.
You have got #include <unistd.h> which is where fork() is declared.
So, you probably need to tell the system to show the POSIX definitions before you include the system headers:
#define _XOPEN_SOURCE 600
You can use 700 if you think your system is mostly POSIX 2008 compliant, or even 500 for an older system. Because fork() has been around forever, it will show up with any of those.
If you are compiling with -std=c99 --pedantic, then all the declarations for POSIX will be hidden unless you explicitly request them as shown.
You can also play with _POSIX_C_SOURCE, but using _XOPEN_SOURCE implies the correct corresponding _POSIX_C_SOURCE (and _POSIX_SOURCE, and so on).
As you've already noted, fork() should be defined in unistd.h - at least according to the man pages that come with Ubuntu 11.10. The minimal:
#include <unistd.h>
int main( int argc, char* argv[])
{
pid_t procID;
procID = fork();
return procID;
}
...builds with no warnings on 11.10.
Speaking of which, what UNIX/Linux distribution are you using? For instance, I've found several non-remarkable functions that should be defined in Ubuntu 11.10's headers aren't. Such as:
// string.h
char* strtok_r( char* str, const char* delim, char** saveptr);
char* strdup( const char* const qString);
// stdio.h
int fileno( FILE* stream);
// time.h
int nanosleep( const struct timespec* req, struct timespec* rem);
// unistd.h
int getopt( int argc, char* const argv[], const char* optstring);
extern int opterr;
int usleep( unsigned int usec);
As long as they're defined in your C library it won't be a huge problem. Just define your own prototypes in a compatibility header and report the standard header problems to whoever maintains your OS distribution.
I think that you have to do the following instead:
pid_t pid = fork();
To learn more about Linux API, go to this online manual page, or even go into your terminal right now and type,
man fork
Good luck!