Im getting into kernel work for a bit of my summer research. We are looking to make modifications to the TCP, in specific RTT calculations. What I would like to do is replace the resolution of one of the functions in tcp_input.c to a function provided by a dynamically loaded kernel module. I think this would improve the pace at which we can develop and distribute the modification.
The function I'm interested in was declared as static, however I've recompiled the kernel with the function non-static and exported by EXPORT_SYMBOL. This means the function is now accessible to other modules/parts of the kernel. I have verified this by "cat /proc/kallsyms".
Now I'd like to be able to load a module that can rewrite the symbol address from the initial to my dynamically loaded function. Similarly, when the module is to be unloaded, it would restore the original address. Is this a feasible approach? Do you all have suggestions how this might be better implemented?
Thanks!
Same as Overriding functionality with modules in Linux kernel
Edit:
This was my eventual approach.
Given the following function (which I wanted to override, and is not exported):
static void internal_function(void)
{
// do something interesting
return;
}
modify like so:
static void internal_function_original(void)
{
// do something interesting
return;
}
static void (*internal_function)(void) = &internal_function_original;
EXPORT_SYMBOL(internal_function);
This redefines the expected function identifier instead as a function pointer (which can be called in a similar manner) pointing to the original implementation. EXPORT_SYMBOL() makes the address globally accessible, so we can modify it from a module (or other kernel location).
Now you can write a kernel module with the following form:
static void (*original_function_reference)(void);
extern void (*internal_function)(void);
static void new_function_implementation(void)
{
// do something new and interesting
// return
}
int init_module(void)
{
original_function_reference = internal_function;
internal_function = &new_function_implementation;
return 0;
}
void cleanup_module(void)
{
internal_function = original_function_reference;
}
This module replaces the original implementation with a dynamically loaded version. Upon unloading, the original reference (and implementation) is restored. In my specific case, I provided a new estimator for the RTT in TCP. By using a module, I am able to make small tweaks and restart testing, all without having to recompile and reboot the kernel.
I'm not sure that'll work - I believe the symbol resolution for the internal calls to the function you want to replace will have already been done by the time your module loads.
Instead, you could change the code by renaming the existing function, then creating a global function pointer with the original name of the function. Initialise the function pointer to the address of the internal function, so the existing code will work unmodified. Export the symbol of the global function pointer, then your module can just change its value by assignment at module load and unload time.
I once made a proof of concept of a hijack module that inserted it's own function in place of kernel function.
I just so happens that the new kernel tacing architecture uses a very similar system.
I injected my own function in the kernel by overwriting the first couple of bytes of code with a jump pointing to my custom function. As soon as the real function gets called, it jumps instead to my function that after it had done it's work called the original function.
#include <linux/module.h>
#include <linux/kernel.h>
#define CODESIZE 12
static unsigned char original_code[CODESIZE];
static unsigned char jump_code[CODESIZE] =
"\x48\xb8\x00\x00\x00\x00\x00\x00\x00\x00" /* movq $0, %rax */
"\xff\xe0" /* jump *%rax */
;
/* FILL THIS IN YOURSELF */
int (*real_printk)( char * fmt, ... ) = (int (*)(char *,...) )0xffffffff805e5f6e;
int hijack_start(void);
void hijack_stop(void);
void intercept_init(void);
void intercept_start(void);
void intercept_stop(void);
int fake_printk(char *, ... );
int hijack_start()
{
real_printk(KERN_INFO "I can haz hijack?\n" );
intercept_init();
intercept_start();
return 0;
}
void hijack_stop()
{
intercept_stop();
return;
}
void intercept_init()
{
*(long *)&jump_code[2] = (long)fake_printk;
memcpy( original_code, real_printk, CODESIZE );
return;
}
void intercept_start()
{
memcpy( real_printk, jump_code, CODESIZE );
}
void intercept_stop()
{
memcpy( real_printk, original_code, CODESIZE );
}
int fake_printk( char *fmt, ... )
{
int ret;
intercept_stop();
ret = real_printk(KERN_INFO "Someone called printk\n");
intercept_start();
return ret;
}
module_init( hijack_start );
module_exit( hijack_stop );
I'm warning you, when you're going to experiment with these kind of things, watch out for kernel panics and other disastrous events. I would advise you to do this in a virtualised environment. This is a proof-of-concept code I wrote a while ago, I'm not sure it still works.
It's a really easy principle, but very effective. Of course, a real solution would use locks to make sure nobody would call the function while you're overwriting it.
Have fun!
You can try using ksplice - you don't even need to make it non static.
I think what you want is Kprobe.
Another way that caf has mentioned is to add a hook to the original routine, and register/unregister hook in the module.
Related
Suppose there is a library function (can not modify) that accept a callback (function pointer) as its argument which will be called at some point in the future. My question: is there a way to store extra data along with the function pointer, so that when the callback is called, the extra data can be retrieved. The program is in c.
For example:
// callback's type, no argument
typedef void (*callback_t)();
// the library function
void regist_callback(callback_t cb);
// store data with the function pointer
callback_t store_data(callback_t cb, int data);
// retrieve data within the callback
int retrieve_data();
void my_callback() {
int a;
a = retrieve_data();
// do something with a ...
}
int my_func(...) {
// some variables that i want to pass to my_callback
int a;
// ... regist_callback may be called multiple times
regist_callback(store_data(my_callback, a));
// ...
}
The problem is because callback_t accept no argument. My idea is to generate a small piece of asm code each time to fill into regist_callback, when it is called, it can find the real callback and its data and store it on the stack (or some unused register), then jump to the real callback, and inside the callback, the data can be found.
pseudocode:
typedef struct {
// some asm code knows the following is the real callback
char trampoline_code[X];
callback_t real_callback;
int data;
} func_ptr_t;
callback_t store_data(callback_t cb, int data) {
// ... malloc a func_ptr_t
func_ptr_t * fpt = malloc(...);
// fill the trampoline_code, different machine and
// different calling conversion are different
// ...
fpt->real_callback = cb;
fpt->data = data;
return (callback_t)fpt;
}
int retrieve_data() {
// ... some asm code to retrive data on stack (or some register)
// and return
}
Is it reasonable? Is there any previous work done for such problem?
Unfortunately you're likely to be prohibited from executing your trampoline in more and more systems as time goes on, as executing data is a pretty common way of exploiting security vulnerabilities.
I'd start by reporting the bug to the author of the library. Everybody should know better than to offer a callback interface with no private data parameter.
Having such a limitation would make me think twice about how whether or not the library is reentrant. I would suggest ensuring you can only have one call outstanding at a time, and store the callback parameter in a global variable.
If you believe that the library is fit for use, then you could extend this by writing n different callback trampolines, each referring to their own global data, and wrap that up in some management API.
I have a project, and a case where I have a few often-changed preprocessor #defines that control how it works--ex:
void myfunction(int num, mystruct* content) {
doSomethingTo(content);
//...
#ifdef FEATURE_X
feature_x(content);
#endif
}
This works fine, although it does have to be recompiled each time, so it's in the "stuff that has to be recompiled each time" file. I would like to push it into a [static] library instead. I'm ok with changing how it's called (already have a function pointer for picking myFunction), so I'd like that to turn into
void myfunction(int num, mystruct* content) {
doSomethingTo(content);
//...
}
void myfunction_featureX(int num, mystruct* content) {
doSomethingTo(content);
//...
feature_x(content);
}
I need to do this in a couple places, so using a separate library (one with and one without -D FEATURE_X) for each isn't an acceptable option. I could do it with copy/paste, but that results in code reuse that carries a risk of fixing a bug in one copy but not the other.
Have the featureX versions of functions call the mainline functions. In your example myfunction_featureX would call myfunction and then do its own thing.
Surely, this is the point at which you change the activation of Feature X from a compile time issue into a run-time issue:
void myfunction(int num, mystruct* content)
{
doSomethingTo(content);
//...
if (FeatureX_Enabled())
feature_x(content);
}
The FeatureX_Enabled() test might be a full function, or it might be simply test an appropriately scoped variable that is defined outside the function — a static variable in the file, or an external variable. This avoids having to futz with the function pointers; it's the same function called as now. Changing a table of function pointers is equivalent to changing a single variable — it involves changing the value of something stored outside the function to change the behaviour of the function.
Would it help if you put myfeature_x in a function table instead?
#include <stdio.h>
#include <string.h>
typedef struct {
int x,y;
} mystruct;
typedef void (*fn_ptr)(mystruct* content);
fn_ptr vtable[10];
#define FEATURE_X_INDEX 0
void feature_x(mystruct *content)
{
printf("y: %d\n", content->y);
}
void myfunction(int num, mystruct* content) {
printf("x: %d\n", content->x);
//...
if (vtable[FEATURE_X_INDEX]) {
vtable[FEATURE_X_INDEX](content);
}
}
int main(void)
{
bzero(vtable, sizeof(vtable));
mystruct s;
s.x = 1;
s.y = 2;
myfunction(0, &s);
if (1) {
//Of course you'd use a more sensible condition.
vtable[FEATURE_X_INDEX] = feature_x;
}
myfunction(0, &s);
return 0;
}
Output:
x: 1
x: 1
y: 2
Then all you need to do is populate the virtual function table with NULLs if that feature is not to be used, and with function pointers if it is to be used. This you can do from wherever you want - your static library for example.. or you can compile feature_x into a dynamic library, load it at runtime and if the loading succeeded populate the function table, and clear the table when the dynamically linked library is unloaded.
I think the only benefit this really gives you over Jonathan Leffler's method is that the code for feature_x doesn't actually need to be linked into the same binary as your other code. If all you need is a runtime switch to turn the feature on or off, a simple if statement should do the trick, as Jonathan Leffler suggested. (Incidentally, there's an if here, too - it checks the function table's content :) )
I'm new at C, so sorry for my lack of knowledge (my C-book here is really massive :)
I would like to extend a shared library (libcustomer.so) with closed source, but public known api.
Is something like this possible?
rename libcustomer.so to liboldcustomer.so
create an extended shared library libcustomer.so (so others implicitly use the extended one)
link liboldcustomer.so into my extended libcustomer.so via -loldcustomer
forward any not extra-implemented methods directly to the old "liboldcustomer.so"
I don't think it would work that way (the name is compiled into the .so, isn't it?).
But what's the alternative?
For #4: is there a general way to do this, or do I have to write a method named like the old one and forward the call (how?)?
Because the original libcustomer.so (=liboldcustomer.so) can change from time to time, all that stuff should work dynamically.
For security reasons, our system has no LD_PRELOAD (otherwise I would take that :( ).
Think about extended validation-checks & some better NPE-handlings.
Thanks in advance for your help!
EDIT:
I'm just implementing my extension as shown in the answer, but I have one unhandled case at the moment:
How can I "proxy" the structs from the extended library?
For example I have this:
customer.h:
struct customer;
customer.c:
struct customer {
int children:1;
int age;
struct house *house_config;
};
Now, in my customer-extension.c I am writing all the public methods form customer.c, but how do I "pass-thru" the structs?
Many thanks for your time & help!
So you have OldLib with
void func1();
int func2();
... etc
The step 4 might look like creating another library with some static initialization.
Create NewLib with contents:
void your_func1();
void (*old_func1_ptr)() = NULL;
int (*old_func2_ptr)() = NULL;
void func1()
{
// in case you don't have static initializers, implement lazy loading
if(!old_func1_ptr)
{
void* lib = dlopen("OldLibFileName.so", RTLD_NOW);
old_func1_ptr = dlsym(lib, "func1");
}
old_func1_ptr();
}
int func2()
{
return old_func2_ptr();
}
// gcc extension, static initializer - will be called on .so's load
// If this is not supported, then you should call this function
// manually after loading the NewLib.so in your program.
// If the user of OldLib.so is not _your_ program,
// then implement lazy-loading in func1, func2 etc. - check function pointers for being NULL
// and do the dlopen/dlsym calls there.
__attribute__((constructor))
void static_global_init()
{
// use dlfcn.h
void* lib = dlopen("OldLibFileName.so", RTLD_NOW);
old_func1_ptr = dlsym(lib, "func1");
...
}
The static_global_init and all the func_ptr's can be autogenerated if you have some description of the old API. After the NewLib is created, you certainly can replace the OldLib.
Is there an mechanism or trick to run a function when a program loads?
What I'm trying to achieve...
void foo(void)
{
}
register_function(foo);
but obviously register_function won't run.
so a trick in C++ is to use initialization to make a function run
something like
int throwaway = register_function(foo);
but that doesn't work in C. So I'm looking for a way around this using standard C (nothing platform / compiler specific )
If you are using GCC, you can do this with a constructor function attribute, eg:
#include <stdio.h>
void foo() __attribute__((constructor));
void foo() {
printf("Hello, world!\n");
}
int main() { return 0; }
There is no portable way to do this in C, however.
If you don't mind messing with your build system, though, you have more options. For example, you can:
#define CONSTRUCTOR_METHOD(methodname) /* null definition */
CONSTRUCTOR_METHOD(foo)
Now write a build script to search for instances of CONSTRUCTOR_METHOD, and paste a sequence of calls to them into a function in a generated .c file. Invoke the generated function at the start of main().
Standard C does not support such an operation. If you don't wish to use compiler specific features to do this, then your next best bet might be to create a global static flag that is initialized to false. Then whenever someone invokes one of your operations that require the function pointer to be registered, you check that flag. If it is false you register the function then set the flag to true. Subsequent calls then won't have to perform the registration. This is similar to the lazy instantiation used in the OO Singleton design pattern.
There is no standard way of doing this although gcc provides a constructor attribute for functions.
The usual way of ensuring some pre-setup has been done (other than a simple variable initialization to a compile time value) is to make sure that all functions requiring that pre-setup. In other words, something like:
static int initialized = 0;
static int x;
int returnX (void) {
if (!initialized) {
x = complicatedFunction();
initialized = 1;
}
return x;
}
This is best done in a separate library since it insulates you from the implementation.
I am writing a large C program for embedded use. Every module in this program has an init() function (like a constructor) to set up its static variables.
The problem is that I have to remember to call all of these init functions from main(). I also have to remember to put them back if I have commented them out for some reason.
Is there anything clever I do to make sure that all of these functions are getting called? Something along the lines of putting a macro in each init function that, when you call a check_inited() function later, sends a warning to STDOUT if not all the functions are called.
I could increment a counter, but I'd have to maintain the correct number of init functions somewhere and that is also prone to error.
Thoughts?
The following is the solution I decided on, with input from several people in this thread
My goal is to make sure that all my init functions are actually being called. I want to do
this without maintaining lists or counts of modules across several files. I can't call
them automatically as Nick D suggested because they need to be called in a certain order.
To accomplish this, a macro included in every module uses the gcc constructor attribute to
add the init function name to a global list.
Another macro included in the body of the init function updates the global list to make a
note that the function was actually called.
Finally, a check function is called in main() after all of the inits are done.
Notes:
I chose to copy the strings into an array. This not strictly necessary because the
function names passed will always be static strings in normal usage. If memory was short
you could just store a pointer to the string that was passed in.
My reusable library of utility functions is called "nx_lib". Thus all the 'nxl' designations.
This isn't the most efficient code in the world but it's only called a boot time so that
doesn't matter for me.
There are two lines of code that need to be added to each module. If either is omitted,
the check function will let you know.
you might be able to make the constructor function static, which would avoid the need to give it a name that is unique across the project.
this code is only lightly tested and it's really late so please check carefully before trusting it.
Thank you to:
pierr who introduced me to the constructor attribute.
Nick D for demonstrating the ## preprocessor trick and giving me the framework.
tod frye for a clever linker-based approach that will work with many compilers.
Everyone else for helping out and sharing useful tidbits.
nx_lib_public.h
This is the relevant fragment of my library header file
#define NX_FUNC_RUN_CHECK_NAME_SIZE 20
typedef struct _nxl_function_element{
char func[NX_FUNC_RUN_CHECK_NAME_SIZE];
BOOL called;
} nxl_function_element;
void nxl_func_run_check_add(char *func_name);
BOOL nxl_func_run_check(void);
void nxl_func_run_check_hit(char *func_name);
#define NXL_FUNC_RUN_CHECK_ADD(function_name) \
void cons_ ## function_name() __attribute__((constructor)); \
void cons_ ## function_name() { nxl_func_run_check_add(#function_name); }
nxl_func_run_check.c
This is the libary code that is called to add function names and check them later.
#define MAX_CHECKED_FUNCTIONS 100
static nxl_function_element m_functions[MAX_CHECKED_FUNCTIONS];
static int m_func_cnt = 0;
// call automatically before main runs to register a function name.
void nxl_func_run_check_add(char *func_name)
{
// fail and complain if no more room.
if (m_func_cnt >= MAX_CHECKED_FUNCTIONS) {
print ("nxl_func_run_check_add failed, out of space\r\n");
return;
}
strncpy (m_functions[m_func_cnt].func, func_name,
NX_FUNC_RUN_CHECK_NAME_SIZE);
m_functions[m_func_cnt].func[NX_FUNC_RUN_CHECK_NAME_SIZE-1] = 0;
m_functions[m_func_cnt++].called = FALSE;
}
// call from inside the init function
void nxl_func_run_check_hit(char *func_name)
{
int i;
for (i=0; i< m_func_cnt; i++) {
if (! strncmp(m_functions[i].func, func_name,
NX_FUNC_RUN_CHECK_NAME_SIZE)) {
m_functions[i].called = TRUE;
return;
}
}
print("nxl_func_run_check_hit(): error, unregistered function was hit\r\n");
}
// checks that all registered functions were called
BOOL nxl_func_run_check(void) {
int i;
BOOL success=TRUE;
for (i=0; i< m_func_cnt; i++) {
if (m_functions[i].called == FALSE) {
success = FALSE;
xil_printf("nxl_func_run_check error: %s() not called\r\n",
m_functions[i].func);
}
}
return success;
}
solo.c
This is an example of a module that needs initialization
#include "nx_lib_public.h"
NXL_FUNC_RUN_CHECK_ADD(solo_init)
void solo_init(void)
{
nxl_func_run_check_hit((char *) __func__);
/* do module initialization here */
}
You can use gcc's extension __attribute__((constructor)) if gcc is ok for your project.
#include <stdio.h>
void func1() __attribute__((constructor));
void func2() __attribute__((constructor));
void func1()
{
printf("%s\n",__func__);
}
void func2()
{
printf("%s\n",__func__);
}
int main()
{
printf("main\n");
return 0;
}
//the output
func2
func1
main
I don't know how ugly the following looks but I post it anyway :-)
(The basic idea is to register function pointers, like what atexit function does.
Of course atexit implementation is different)
In the main module we can have something like this:
typedef int (*function_t)(void);
static function_t vfunctions[100]; // we can store max 100 function pointers
static int vcnt = 0; // count the registered function pointers
int add2init(function_t f)
{
// todo: error checks
vfunctions[vcnt++] = f;
return 0;
}
...
int main(void) {
...
// iterate vfunctions[] and call the functions
...
}
... and in some other module:
typedef int (*function_t)(void);
extern int add2init(function_t f);
#define M_add2init(function_name) static int int_ ## function_name = add2init(function_name)
int foo(void)
{
printf("foo\n");
return 0;
}
M_add2init(foo); // <--- register foo function
Why not write a post processing script to do the checking for you. Then run that script as part of your build process... Or better yet, make it one of your tests. You are writing tests, right? :)
For example, if each of your modules has a header file, modX.c. And if the signature of your init() function is "void init()"...
Have your script grep through all your .h files, and create a list of module names that need to be init()ed. Then have the script check that init() is indeed called on each module in main().
If your single module represents "class" entity and has instance constructor, you can use following construction:
static inline void init(void) { ... }
static int initialized = 0;
#define INIT if (__predict_false(!initialized)) { init(); initialized = 1; }
struct Foo *
foo_create(void)
{
INIT;
...
}
where "__predict_false" is your compiler's branch prediction hint. When first object is created, module is auto-initialized (for once).
Splint (and probably other Lint variants) can give a warning about functions that are defined but not called.
It's interesting that most compilers will warn you about unused variables, but not unused functions.
Larger running time is not a problem
You can conceivably implement a kind of "state-machine" for each module, wherein the actions of a function depend on the state the module is in. This state can be set to BEFORE_INIT or INITIALIZED.
For example, let's say we have module A with functions foo and bar.
The actual logic of the functions (i.e., what they actually do) would be declared like so:
void foo_logic();
void bar_logic();
Or whatever the signature is.
Then, the actual functions of the module (i.e., the actual function declared foo()) will perform a run-time check of the condition of the module, and decide what to do:
void foo() {
if (module_state == BEFORE_INIT) {
handle_not_initialized_error();
}
foo_logic();
}
This logic is repeated for all functions.
A few things to note:
This will obviously incur a huge penalty performance-wise, so is
probably not a good idea (I posted
anyway because you said runtime is
not a problem).
This is not a real state-machine, since there are only two states which are checked using a basic if, without some kind of smart general logic.
This kind of "design-pattern" works great when you're using separate threads/tasks, and the functions you're calling are actually called using some kind of IPC.
A state machine can be nicely implemented in C++, might be worth reading up on it. The same kind of idea can conceivably be coded in C with arrays of function pointers, but it's almost certainly not worth your time.
you can do something along these lines with a linker section. whenever you define an init function, place a pointer to it in a linker section just for init function pointers. then you can at least find out how many init functions have been compiled.
and if it does not matter what order the init functions are called, and the all have the same prototype, you can just call them all in a loop from main.
the exact details elude my memory, but it works soemthing like this::
in the module file...
//this is the syntax in GCC..(or would be if the underscores came through in this text editor)
initFuncPtr thisInit __attribute((section(.myinits)))__= &moduleInit;
void moduleInit(void)
{
// so init here
}
this places a pointer to the module init function in the .myinits section, but leaves the code in the .code section. so the .myinits section is nothing but pointers. you can think of this as a variable length array that module files can add to.
then you can access the section start and end address from the main. and go from there.
if the init functions all have the same protoytpe, you can just iterate over this section, calling them all.
this, in effect, is creating your own static constructor system in C.
if you are doing a large project and your linker is not at least this fully featured, you may have a problem...
Can I put up an answer to my question?
My idea was to have each function add it's name to a global list of functions, like Nick D's solution.
Then I would run through the symbol table produced by -gstab, and look for any functions named init_* that had not been called.
This is an embedded app so I have the elf image handy in flash memory.
However I don't like this idea because it means I always have to include debugging info in the binary.