I am trying to find somewhat elegant ways to mock and stub function calls to the standard C library functions.
While stubbing-off calls to C files of the project is easy by just linking other C files in the tests, stubbing the standard C functions is harder.
They are just there when linking.
Currently, my approach is to include the code-under-test from my test.cpp file, and placing defines like this:
#include <stdio.h>
#include <gtest/gtest.h>
#include "mymocks.h"
CMockFile MockFile;
#define open MockFile.open
#define close MockFile.close
#define read MockFile.read
#include "CodeUnderTestClass.cpp"
#undef open
#undef close
#undef read
// test-class here
This is cumbersome, and sometimes I run across code that uses 'open' as member names elsewhere or causes other collisions and issues with it. There are also cases of the code needing different defines and includes than the test-code.
So are there alternatives? Some link-time tricks or runtime tricks to override standard C functions? I thought about run-time hooking the functions but that might go too far as usually binary code is loaded read-only.
My unit-tests run only on Debian-Linux with gcc on amd64. So gcc, x64 or Linux specific tricks are also welcome.
I know that rewriting all the code-under-test to use an abstracted version of the C functions is an option, but that hint is not very useful for me.
Use library preloading to substitute system libraries with your own.
Consider following test program code, mytest.c:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main(void) {
char buf[256];
int fd = open("file", O_RDONLY);
if (fd >= 0) {
printf("fd == %d\n", fd);
int r = read(fd, buf, sizeof(buf));
write(0, buf, r);
close(fd);
} else {
printf("can't open file\n");
}
return 0;
}
It will open a file called file from the current directory, print it's descriptor number (usually 3), read its content and then print it on the standard output (descriptor 0).
Now here is your test library code, mock.c:
#include <string.h>
#include <unistd.h>
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
Compile it to a shared library called mock.so:
$ gcc -shared -fpic -o mock.so mock.c
If you compiled mytest.c to the mytest binary, run it with following command:
$ LD_PRELOAD=./mock.so ./mytest
You should see the output:
fd == 100
TEST!
Functions defined in mock.c were preloaded and used as a first match during the dynamic linking process, hence executing your code, and not the code from the system libraries.
Update:
If you want to use "original" functions, you should extract them "by hand" from the proper shared library, using dlopen, dlmap and dlclose functions. Because I don't want to clutter previous example, here's the new one, the same as previous mock.c plus dynamic symbol loading stuff:
#include <stdio.h>
#include <dlfcn.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <gnu/lib-names.h>
// this declares this function to run before main()
static void startup(void) __attribute__ ((constructor));
// this declares this function to run after main()
static void cleanup(void) __attribute__ ((destructor));
static void *sDlHandler = NULL;
ssize_t (*real_write)(int fd, const void *buf, size_t count) = NULL;
void startup(void) {
char *vError;
sDlHandler = dlopen(LIBC_SO, RTLD_LAZY);
if (sDlHandler == NULL) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
real_write = (ssize_t (*)(int, const void *, size_t))dlsym(sDlHandler, "write");
vError = dlerror();
if (vError != NULL) {
fprintf(stderr, "%s\n", vError);
exit(EXIT_FAILURE);
}
}
void cleanup(void) {
dlclose(sDlHandler);
}
int open(const char *pathname, int flags) {
return 100;
}
int close(int fd) {
return 0;
}
ssize_t read(int fd, void *buf, size_t count) {
strcpy(buf, "TEST!\n");
return 7;
}
ssize_t write(int fd, const void *buf, size_t count) {
if (fd == 0) {
real_write(fd, "mock: ", 6);
}
real_write(fd, buf, count);
return count;
}
Compile it with:
$ gcc -shared -fpic -o mock.so mock.c -ldl
Note the -ldl at the end of the command.
So: startup function will run before main (so you don't need to put any initialization code in your original program) and initialize real_write to be the original write function. cleanup function will run after main, so you don't need to add any "cleaning" code at the end of main function either.
All the rest works exactly the same as in the previous example, with the exception of newly implemented write function. For almost all the descriptors it will work as the original, and for file descriptor 0 it will write some extra data before the original content. In that case the output of the program will be:
$ LD_PRELOAD=./mock.so ./mytest
fd == 100
mock: TEST!
Related
So the problem is the following. The project needs to intercept all file IO
operations, like open() and close(). I am trying to add printf() before calling the corresponding open() or close(). I am not supposed to rewrite the source code by changing open() or close() to myOpen() or myClose() for example. I have been trying to use LD_PRELOAD environment variable. But the indefinite loop problem came up. My problem is like this one.
int open(char * path,int flags,int mode)
{
// print file name
printf("open :%s\n",path);
return __open(path,flags,mode);
}
Yes, you want LD_PRELOAD.
You need to create a shared library (.so) that has code for all functions that you want to intercept. And, you want to set LD_PRELOAD to use that shared library
Here is some sample code for the open function. You'll need to do something similar for each function you want to intercept:
#define _GNU_SOURCE
#include <dlfcn.h>
int
open(const char *file,int flags,int mode)
{
static int (*real_open)(const char *file,int flags,int mode) = NULL;
int fd;
if (real_open == NULL)
real_open = dlsym(RTLD_NEXT,"open");
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
I believe RTLD_NEXT is easiest and may be sufficient. Otherwise, you could add a constructor that does dlopen once on libc
UPDATE:
I am not familiar with C and I got the following problems with gcc. "error: 'NULL' undeclared (first use in this function)",
This is defined by several #include files, so try #include <stdio.h>. You'll need that if you want to call printf.
"error: 'RTLD_NEXT' undeclared (first use in this function)",
That is defined by doing #include <dlfcn.h> [as shown in my example]
and "symbol lookup error: ./hack_stackoverflow.so: undefined symbol: dlsym".
From man dlsym, it says: Link with -ldl So, add -ldl to the line that builds your .so.
Also, you have to be careful to prevent infinite recursion if the "special stuff" does something that loops back on your intercept function.
Notably, you want to call printf. If you intercept the write syscall, bad things may happen.
So, you need to keep track of when you're already in one of your intercept functions and not do anything special if already there. See the in_self variable.
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
ssize_t
write(int fd,const void *buf,size_t len)
{
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
static int in_self = 0;
ssize_t err;
if (real_write == NULL)
real_write = dlsym(RTLD_NEXT,"write");
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
err = real_write(fd,buf,len);
if (in_self == 1)
printf("mywrite: fd=%d buf=%p err=%ld\n",fd,buf,err);
--in_self;
return err;
}
The above works okay for single threaded programs/environments, but if you're intercepting an arbitrary one, it could be multithreaded.
So, we'd have to initialize all the real_* pointers in a constructor. This is a function with a special attribute that tells the dynamic loader to call the function ASAP automatically.
And, we have to put in_self into thread local storage. We do this by adding the __thread attribute.
You may need to link with -lpthread as well as -ldl for the multithreaded version.
Edit: We also have to preserve the correct errno value
Putting it all together:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <errno.h>
static int (*real_open)(const char *file,int flags,int mode) = NULL;
static ssize_t (*real_write)(int fd,const void *buf,size_t len) = NULL;
__attribute__((constructor))
void
my_lib_init(void)
{
real_open = dlsym(RTLD_NEXT,"open");
real_write = dlsym(RTLD_NEXT,"write");
}
int
open(const char *file,int flags,int mode)
{
int fd;
// do whatever special stuff ...
fd = real_open(file,flags,mode);
// do whatever special stuff ...
return fd;
}
ssize_t
write(int fd,const void *buf,size_t len)
{
static int __thread in_self = 0;
int sverr;
ssize_t ret;
++in_self;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p len=%ld\n",fd,buf,len);
ret = real_write(fd,buf,len);
// preserve errno value for actual syscall -- otherwise, errno may
// be set by the following printf and _caller_ will get the _wrong_
// errno value
sverr = errno;
if (in_self == 1)
printf("mywrite: fd=%d buf=%p ret=%ld\n",fd,buf,ret);
--in_self;
// restore correct errno value for write syscall
errno = sverr;
return ret;
}
I have been recoding some libC functions which I incorporated in my Shared Object Library. Some of these functions internally call themselves.
The problem now is when I use dlsym from another program, the function provided by dlsym (which internal use requires an internal function) will call libC's functions instead of those already present which I recoded in my library.
Here is a simple example:
lib.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void *memset(void *s, int c, size_t n)
{
printf("Calling LIB's Memset\n");
for (size_t i = 0; i < n; ++i)
((char *) s)[i] = c;
return s;
}
void *calloc(size_t mnemb, size_t size)
{
size_t sz = mnemb * size;
void *addr = malloc(sz);
printf("Calling LIB's Calloc\n");
memset(addr, 0, sz);
return addr;
}
gcc lib.c -fPIC -shared -o lib.so
main.c
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
int main(void)
{
void *handler = dlopen("./lib.so", RTLD_NOW);
void (*_calloc)(size_t, size_t);
if (!handler)
{
printf("Could not open lib\n");
return 1;
}
_calloc = dlsym(handler, "calloc");
if (!_calloc)
{
printf("Could not extract symbol\n");
return 1;
}
_calloc(10, 10);
return 0;
}
gcc main.c -ldl
$ ./a.out
Calling LIB's Calloc
As you can see, only the calloc function coming from the lib is called, the internal memset is not called.
How can I tell in my library to explicitly call its own functions ?
Note: My function have to be called the same way as libC's as it got to work with LD_PRELOAD too.
How can I tell in my library to explicitly call its own functions ?
The best way would probably be to give your own functions distinct names. For example, prefix their names with a consistent prefix: my_strlen, my_printf, etc.. Then use only those names when the library intends to make calls to other functions within. Stupid example:
size_t my_strlen(const char *s) {
return *s ? (1 + my_strlen(s + 1)) : 0;
}
To cause those to be called in place of their namesakes by external callers, interpose wrapper functions. For example:
size_t strlen(const char *s) {
return my_strlen(s);
}
But, again, do not rely on the wrappers inside the library. You could even split the wrappers into a separate library, if you wish.
The idea here is to minimize your exposure to linker games and dynamic linking effects. You cannot completely avoid those when the point is to substitute your own implementations of standard library functions, but in this way you can reduce the scope for them to cause you trouble.
I have this program that is supposed to mmap a file in read-write mode and be able to edit its contents. Also the file this is written for is about 40-50 GB, so I need mmap64. The problem is, while mmap64 does not return an error, the address it returns is not accessible.
#include <assert.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <unistd.h>
typedef unsigned long long u64;
void access_test(u64 p, u64 sz)
{
u64 i;
char tmp;
for (i=0; i<sz; i++) {
tmp = *(char*)(p+i);
}
}
int main(int argc, char *argv[])
{
int fd;
long long int sz, p;
struct stat buf;
fd = open(argv[1], O_RDWR, 0x0666);
if (fd == -1) {
perror("open");
return 1;
}
fstat64(fd, &buf);
sz = buf.st_size;
printf("File size: 0x%016llx\n", sz);
p = mmap64 (0, buf.st_size, PROT_READ | PROT_WRITE , MAP_SHARED, fd, 0);
if (p == -1) {
perror ("mmap");
return 1;
}
access_test(p,sz);
if (close (fd) == -1) {
perror ("close");
return 1;
}
if (munmap ((void*)p, buf.st_size) == -1) {
perror ("munmap");
return 1;
}
return 0;
}
The result of this is on a small file:
$ ./testmmap minicom.log
File size: 0x0000000000000023
[1] 8282 segmentation fault (core dumped) ./testmmap minicom.log
The same goes for the big one.
Always enable warnings when you compile
Here is the result with warnings enabled:
$ gcc mmp.c -Wall -g
mmp.c: In function ‘access_test’:
mmp.c:18:10: warning: variable ‘tmp’ set but not used [-Wunused-but-set-variable]
char tmp;
^
mmp.c: In function ‘main’:
mmp.c:36:5: warning: implicit declaration of function ‘fstat64’ [-Wimplicit-function-declaration]
fstat64(fd, &buf);
^
mmp.c:40:5: warning: implicit declaration of function ‘mmap64’ [-Wimplicit-function-declaration]
p = mmap64 (0, buf.st_size, PROT_READ | PROT_WRITE , MAP_SHARED, fd, 0);
The last two warnings here are extremely important. They say there is no prototype for mmap64. C therefore gives you a default prototype, and it is wrong, at least for the mmap64() call (since the prototype will return an int, which cannot represent a pointer on a 64-bit Linux host)
The argument to fstat64() is a struct stat64 too BTW, which is another issue.
Make the specific 64-bit functions available
If you want to make the fstat64()/mmap64() function available, you need to compile the code with the _LARGEFILE and LARGEFILE64_SOURCE #define, see information here, so you should compile this as e.g:
gcc -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE mmp.c -Wall -g
Or use #define _FILE_OFFSET_BITS=64
There is however no need to do this. Just call the normal fstat() and mmap() and #define _FILE_OFFSET_BITS=64 when compiling. e.g.:
gcc -D_FILE_OFFSET_BITS=64 mmp.c -Wall -g
This will enable support for large files, and e.g. translate the mmap() call to mmap64() if it is needed (e.g. if you're on a 32-bit host).
If you are trying to mmap() an 50 GB file, you anyway need to be on a 64-bit host, and on a 64-bit Linux host there's no need for any of this - mmap() and fstat() handles large files without any need to do anything.
Use pointers
The next issue is you're assigning the return value of mmap() to an integer. This might happen to work, but the code does look odd because of it. If you want to treat the thing as a char *, assign it to a char *. Don't play tricks with casting pointers around to a 64-bit integer type.
E.g. your access function should be:
void access_test(char *p, u64 sz)
{
u64 i;
char tmp;
for (i=0; i<sz; i++) {
tmp = p[i];
}
}
And p should be declared as char *p; in main(), or use uint8_t *p; if you intend to treat the data as binary data.
I am trying to hack the malloc function to call my malloc function first.Once my malloc function is executed within that, I want to invoke the standard malloc. But, I am getting a recursion, since it is just loading my defined malloc. How can i fix the below code?
#include <dlfcn.h>
#include "stdio.h"
//#include "stdlib.h"
void *handle;
void *handle_malloc;
int (*loadprg)(void);
void * (*malloc_sysm)(size_t);
void init()
{
handle = dlopen ("/export/home/joshis1/Foxtel/temp/libloadmap.so", RTLD_LAZY);
if( handle == NULL)
{
puts(dlerror());
}
handle_malloc = dlopen ("/lib/libc.so.6", RTLD_LAZY);
if( handle_malloc == NULL)
{
puts("handle malloc error\r\n");
puts(dlerror());
}
}
#include "stdio.h"
void *malloc(int size)
{
printf("called..my malloc\r\n");
malloc_sysm = dlsym(handle_malloc,"malloc");
if ( dlerror() != NULL)
{
puts("malloc symbol not found..");
exit(1);
}
printf("This should call actual malloc now..\r\n");
return malloc_sysm(size);
}
int main()
{
int *t;
init();
printf("call load program now\r\n");
loadprg = dlsym(handle, "loadprg");
if( dlerror()!= NULL)
{
puts("Symbol load errror");
}
(*loadprg)();
printf("Close the handle now..\r\n");
dlclose(handle);
t = (int *) malloc (100);
dlclose(handle_malloc);
return 0;
}
The output is recursion to my defined malloc(). how to fix this?
To override shared functions you need to compile your own shared library and preload it via the LD_PRELOAD environment variable.
#define _GNU_SOURCE
#include <stdlib.h>
#include <stdio.h>
#include <dlfcn.h>
void *malloc(size_t size) {
printf("called..my malloc\r\n");
void *(*original_malloc)(size_t size);
// Find original malloc function
original_malloc = dlsym(RTLD_NEXT, "malloc");
if ( dlerror() != NULL)
{
puts("malloc symbol not found..");
exit(1);
}
printf("This should call actual malloc now..\r\n");
return (*original_malloc)(size);
}
$ gcc -Wall -fPIC -shared -o mymalloc.so mymalloc.c -ldl
$ LD_PRELOAD=./mymalloc.so ./prog
Now your program will use malloc from preloaded library.
The standard way I'm always using is creating a macro called MALLOC (or MYMALLOC or whatever) which does what you want. All occurrences of malloc I have to replace by the use of the macro, of course, and I can understand when this is not what you want.
You also can achieve what you want by defining a macro called malloc (i. e. spelled like the original malloc) only when compiling the source you want to have your feature in. This malloc macro then would call a function called, say, wrappingMalloc which should be declared in a file which is compiled without defining the macro malloc and which then in turn can call the original function malloc. If this makefile fiddling is too much for you, you could also call the original function by calling (malloc) (this avoids running into the macro again):
#include <stdlib.h>
#include <stdio.h>
#define malloc(size) myMalloc(size)
void *myMalloc(size_t size) {
void *result;
printf("mallocing %ld bytes", size);
result = (malloc)(size);
printf(" at %p\n", result);
return result;
}
int main(int argc, char *argv[]) {
char *buffer;
buffer = malloc(10);
return 0;
}
In C++ you might get along by overloading the new operator for your classes.
I cannot see a problem in your code. But why not move malloc_sysm = dlsym(handle_malloc,"malloc"); into your init() function?
I have successfuly intercepted calls to read(),write(),open(),unlink(),rename(), creat() but somehow with exactly the same semantics intercepting stat() is not taking place. I have changed the execution environmnet using LD_PRELOAD.
Am I missing something?
The code is quite huge, which part of it will be most helpful to post so you can help?
Thanks.
Edit: I kept the interposed stat() wrapper simple to check if it works.
int stat(const char *path,struct stat *buff)
{
printf("client invoke: stat %s",path);
return 1;
}
Compile a function that calls stat(); see what reference(s) are generated (nm -g stat.o). Then you'll have a better idea of which function(s) to interpose. Hint: it probably isn't called stat().
If you are compiling with 64 bit file offsets, then stat() is either a macro or a redirected function declaration that resolves to stat64(), so you will have to interpose on that function too.
Well it was not very simple when running in linux. Gnu libc does some tricks. You need to intercept the __xstat and if you want to call the original save the call.
Here is how I got it to work
gcc -fPIC -shared -o stat.so stat.c -ldl
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
static int (*old_xstat)(int ver, const char *path, struct stat *buf) = NULL;
static int (*old_xstat64)(int ver, const char *path, struct stat64 *buf) = NULL;
int __xstat(int ver, const char *path, struct stat *buf)
{
if ( old_xstat == NULL ) {
old_xstat = dlsym(RTLD_NEXT, "__xstat");
}
printf("xstat %s\n",path);
return old_xstat(ver,path, buf);
}
int __xstat64(int ver, const char *path, struct stat64 *buf)
{
if ( old_xstat64 == NULL ) {
old_xstat64 = dlsym(RTLD_NEXT, "__xstat64");
}
printf("xstat64 %s\n",path);
return old_xstat64(ver,path, buf);
}