I'm writing a kernel module which involves the tasklist_lock, __bss_start.
These symbols are not exported. I'm pretty sure even if not exported, we can access the symbols from text sections using kernsym_lookup_name()
Reference How my custom module on linux 3.2.28 can make a call to print_cpu_info?
$ vim System.map
...
80017be0 T register_undef_hook
80017c28 T unregister_undef_hook
80017c70 T do_unexp_fiq
...
806eb000 D mmlist_lock
806eb040 D tasklist_lock
806eb080 d softirq_vec
....
T represents text symbol.
D and d represents data segment symbol.
I'm able to access register_undef_hook() and unregister_undef_hook() using kallsyms_lookup_name().
But not tasklist_lock.
Please share your knowledge to access tasklist_lock from kernel module(LKM).
See this noble post
#include <linux/module.h>
#include <linux/kallsyms.h>
#include <linux/string.h>
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Access non-exported symbols");
MODULE_AUTHOR("Stephen Zhang");
static int __init lkm_init(void)
{
char *sym_name = "__bss_start";
unsigned long sym_addr = kallsyms_lookup_name(sym_name);
char filename[256];
strncpy(filename, (char *)sym_addr, 255);
printk(KERN_INFO "[%s] %s (0x%lx): %s\n", __this_module.name, sym_name, sym_addr, filename);
return 0;
}
static void __exit lkm_exit(void)
{
}
module_init(lkm_init);
module_exit(lkm_exit);
Related
I am really new to this Kernel stuff. I went here and I found this code that outputs process information like its ID.
main.c:
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched/signal.h>
#include <linux/sched.h>
struct task_struct *task; /* Structure defined in sched.h for tasks/processes */
struct task_struct *task_child; /* Structure needed to iterate through task children */
struct list_head *list; /* Structure needed to iterate through the list in each task->children struct */
int iterate_init(void) /* Init Module */
{
printk(KERN_INFO "%s","LOADING MODULE\n"); /* good practice to log when loading/removing modules */
for_each_process( task ){ /* for_each_process() MACRO for iterating through each task in the os located in linux\sched\signal.h */
printk(KERN_INFO "\nPARENT PID: %d PROCESS: %s STATE: %ld",task->pid, task->comm, task->state);/* log parent id/executable name/state */
list_for_each(list, &task->children){ /* list_for_each MACRO to iterate through task->children */
task_child = list_entry( list, struct task_struct, sibling ); /* using list_entry to declare all vars in task_child struct */
printk(KERN_INFO "\nCHILD OF %s[%d] PID: %d PROCESS: %s STATE: %ld",task->comm, task->pid, /* log child of and child pid/name/state */
task_child->pid, task_child->comm, task_child->state);
}
printk("-----------------------------------------------------"); /*for aesthetics*/
}
return 0;
} /* End of Init Module */
void cleanup_exit(void) /* Exit Module */
{
printk(KERN_INFO "%s","REMOVING MODULE\n");
} /* End of Exit Module */
module_init(iterate_init); /* Load Module MACRO */
module_exit(cleanup_exit); /* Remove Module MACRO */
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("ITERATE THROUGH ALL PROCESSES/CHILD PROCESSES IN THE OS");
MODULE_AUTHOR("Laerehte");
Makefile:
obj-m += main.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
Executed with ./ins (with chmod +x)
Code:
sudo insmod main.ko
sudo rmmod main
sudo dmesg -c
I looked up how to find how much memory a process uses, and I found this question: Memory usage of current process in C.
Correct me if I'm wrong here, but I'm thinking that you can read the current RAM usage of these processes by looking in /proc/[process_id]/status. I found out from another place(forgot where) that within this file, there is something called VmRSS that would hold the current RAM usage of the process.
You can apparently use:
ssize_t kernel_read(struct file *file, void *buf, size_t count, loff_t *pos);
to read a file, but I have not been able to modify main.c with this successfully. I need to find the size of the file, but I also have not been able to use vfs_stat correctly. When I just try some constant integer, I get all 0s in the buffer anyway. I don't know how to use these functions properly. I'm trying to modify main.c so that I will see the RAM usage of these processes along with the other information. Much of the information I found was outdated. Can anyone help?
While you can technically open and read files from kernel space, it's usually a bad idea for multiple reasons. Whatever information is provided under /proc is already available to the kernel, you just need to figure out where it is and how to obtain it, and you will be able to do everything in your module without reading any file.
You are interested in knowing how much RAM is a particular process using given its PID, and you are correct that this statistic is available in /proc/<PID>/status: in fact, it is labeled as "VmRSS" which means "virtual memory resident set size". If we take a look at the kernel source code, we can see exactly how that number is calculated:
The kernel function called when reading /proc/<PID>/status is proc_pid_status() in fs/proc/array.c, which then calls (among the other things) task_mem().
// ...
anon = get_mm_counter(mm, MM_ANONPAGES);
file = get_mm_counter(mm, MM_FILEPAGES);
shmem = get_mm_counter(mm, MM_SHMEMPAGES);
// ...
hiwater_rss = total_rss = anon + file + shmem;
// ...
SEQ_PUT_DEC(" kB\nVmHWM:\t", hiwater_rss);
SEQ_PUT_DEC(" kB\nVmRSS:\t", total_rss); // <===== here it is
SEQ_PUT_DEC(" kB\nRssAnon:\t", anon);
// ...
You can therefore obtain this information in the same way. First get ahold of the task_struct of the process you want to check, then inspect the ->mm field like get_mm_counter() does, or simply using get_mm_rss() from include/linux/mm.h, which does exactly what we want.
Note that:
The value obtained from get_mm_counter() or get_mm_rss() is the number of pages, you will have to multiply this by the page size (simply shift left by PAGE_SHIFT).
You will only be able to do this if your kernel was compiled with MMU support (CONFIG_MMU=y), you can check this in your module code with a simple #ifdef or #ifndef.
Here's a working example module to do this for all processes:
// SPDX-License-Identifier: GPL-3.0
#include <linux/kernel.h> // printk(), pr_*()
#include <linux/module.h> // THIS_MODULE, MODULE_VERSION, ...
#include <linux/init.h> // module_{init,exit}
#include <linux/sched/task.h> // struct task_struct, {get,put}_task_struct()
#include <linux/sched/signal.h> // for_each_process()
#include <linux/sched/mm.h> // get_task_mm(), mmput()
#include <linux/mm.h> // get_mm_rss()
/* Tested on kernel 5.6, qemu-system-aarch64
* Usage: sudo insmod task_rss
* sudo modprobe task_rss
*/
#ifdef pr_fmt
#undef pr_fmt
#endif
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
static int __init modinit(void)
{
struct task_struct *tsk;
struct mm_struct *mm;
unsigned long rss;
char comm[TASK_COMM_LEN];
#ifndef CONFIG_MMU
pr_err("No MMU, cannot calculate RSS.\n");
return -EINVAL;
#endif
for_each_process(tsk) {
get_task_struct(tsk);
get_task_comm(comm, tsk);
mm = get_task_mm(tsk);
// https://www.kernel.org/doc/Documentation/vm/active_mm.rst
if (mm) {
rss = get_mm_rss(mm) << PAGE_SHIFT;
mmput(mm);
pr_info("PID %d (\"%s\") VmRSS = %lu bytes\n", tsk->pid, comm, rss);
} else {
pr_info("PID %d (\"%s\") is an anonymous process\n", tsk->pid, comm);
}
put_task_struct(tsk);
}
return 0;
}
static void __exit modexit(void)
{
// This function is only needed to be able to unload the module.
}
module_init(modinit);
module_exit(modexit);
MODULE_VERSION("0.1");
MODULE_DESCRIPTION("Silly test module to calculare task RSS of all running tasks.");
MODULE_AUTHOR("Marco Bonelli");
MODULE_LICENSE("GPL");
Output on my machine (qemu-system-aarch64):
/ # insmod task_rss.ko
[ 7.306284] task_rss: loading out-of-tree module taints kernel.
[ 7.312912] task_rss: PID 1 ("init") VmRSS = 4096 bytes
[ 7.313295] task_rss: PID 2 ("kthreadd") is an anonymous process
[ 7.314039] task_rss: PID 3 ("rcu_gp") is an anonymous process
...
[ 7.330169] task_rss: PID 146 ("sh") VmRSS = 1363968 bytes
[ 7.330554] task_rss: PID 147 ("insmod") VmRSS = 1339392 bytes
I'm debugging vmlinux with gdb vmlinux /proc/kcore, and I want to add a symbol file, I installed a module and then check the location of .bss, .data and .init in /sys/module/simple_module/sections. When I try to use the add-symbol-file with the object file of my module and also the section's locations, gdb returns Unrecognized argument " -s". But from help add-symbol-file, I can see that -s actually a valid argument.
add-symbol-file simple_module.ko 0xc0ce9000 \ -s .bss 0xc0d833c0 \ -s .data 0xc0d83000
I don't think that the code of my module really necessary, but whatever
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("whoami");
MODULE_DESCRIPTION("Simple LKM");
int i; // non-init variable
char text[20] = "Hello, World!";
static int __init initialization_function(void)
{
pr_info("Module: my message!\n");
return 0;
}
static void __exit cleanup_funcion(void)
{
pr_info("Module: Cleanup done, exiting.\n");
}
module_init(initialization_function);
module_exit(cleanup_funcion);
I need to get base address of stack inside my running process. This would enable me to print raw stacktraces that will be understood by addr2line (running binary is stripped, but addr2line has access to symbols).
I managed to do this by examining elf header of argv[0]: I read entry point and substract it from &_start:
#include <stdio.h>
#include <execinfo.h>
#include <unistd.h>
#include <elf.h>
#include <stdio.h>
#include <string.h>
void* entry_point = NULL;
void* base_addr = NULL;
extern char _start;
/// given argv[0] will populate global entry_pont
void read_elf_header(const char* elfFile) {
// switch to Elf32_Ehdr for x86 architecture.
Elf64_Ehdr header;
FILE* file = fopen(elfFile, "rb");
if(file) {
fread(&header, 1, sizeof(header), file);
if (memcmp(header.e_ident, ELFMAG, SELFMAG) == 0) {
printf("Entry point from file: %p\n", (void *) header.e_entry);
entry_point = (void*)header.e_entry;
base_addr = (void*) ((long)&_start - (long)entry_point);
}
fclose(file);
}
}
/// print stacktrace
void bt() {
static const int MAX_STACK = 30;
void *array[MAX_STACK];
auto size = backtrace(array, MAX_STACK);
for (int i = 0; i < size; ++i) {
printf("%p ", (long)array[i]-(long)base_addr );
}
printf("\n");
}
int main(int argc, char* argv[])
{
read_elf_header(argv[0]);
printf("&_start = %p\n",&_start);
printf("base address is: %p\n", base_addr);
bt();
// elf header is also in memory, but to find it I have to already have base address
Elf64_Ehdr * ehdr_addr = (Elf64_Ehdr *) base_addr;
printf("Entry from memory: %p\n", (void *) ehdr_addr->e_entry);
return 0;
}
Sample output:
Entry point from file: 0x10c0
&_start = 0x5648eeb150c0
base address is: 0x5648eeb14000
0x1321 0x13ee 0x29540f8ed09b 0x10ea
Entry from memory: 0x10c0
And then I can
$ addr2line -e a.out 0x1321 0x13ee 0x29540f8ed09b 0x10ea
/tmp/elf2.c:30
/tmp/elf2.c:45
??:0
??:?
How can I get base address without access to argv? I may need to print traces before main() (initialization of globals). Turning of ASLR or PIE is not an option.
How can I get base address without access to argv? I may need to print traces before main()
There are a few ways:
If /proc is mounted (which it almost always is), you could read the ELF header from /proc/self/exe.
You could use dladdr1(), as Antti Haapala's answer shows.
You could use _r_debug.r_map, which points to the linked list of loaded ELF images. The first entry in that list corresponds to a.out, and its l_addr contains the relocation you are looking for. This solution is equivalent to dladdr1, but doesn't require linking against libdl.
Could you provide sample code for 3?
Sure:
#include <link.h>
#include <stdio.h>
extern char _start;
int main()
{
uintptr_t relocation = _r_debug.r_map->l_addr;
printf("relocation: %p, &_start: %p, &_start - relocation: %p\n",
(void*)relocation, &_start, &_start - relocation);
return 0;
}
gcc -Wall -fPIE -pie t.c && ./a.out
relocation: 0x555d4995e000, &_start: 0x555d4995e5b0, &_start - relocation: 0x5b0
Are both 2 and 3 equally portable?
I think they are about equally portable: dladdr1 is a GLIBC extension that is also present on Solaris. _r_debug predates Linux and would also work on Solaris (I haven't actually checked, but I believe it will). It may work on other ELF platforms as well.
This piece of code produces the same value as your base_addr on Linux:
#define _GNU_SOURCE
#include <dlfcn.h>
#include <link.h>
Dl_info info;
void *extra = NULL;
dladdr1(&_start, &info, &extra, RTLD_DL_LINKMAP);
struct link_map *map = extra;
printf("%#llx", (unsigned long long)map->l_addr);
The dladdr1 manual page says the following of RTLD_DL_LINKMAP:
RTLD_DL_LINKMAP
Obtain a pointer to the link map for the matched file. The
extra_info argument points to a pointer to a link_map structure (i.e., struct link_map **), defined in as:
struct link_map {
ElfW(Addr) l_addr; /* Difference between the
address in the ELF file and
the address in memory */
char *l_name; /* Absolute pathname where
object was found */
ElfW(Dyn) *l_ld; /* Dynamic section of the
shared object */
struct link_map *l_next, *l_prev;
/* Chain of loaded objects */
/* Plus additional fields private to the
implementation */
};
Notice that -ldl is required to link against the dynamic loading routines.
How can I find the library where the function open() is? Like, the name of the actual "xxxxxx.so" file that contains that function? Also, is there a place I could typically get this information for other functions?
I didn't know how to find the library a given function was in, which
was required for dlopen
Knowing the library is not required for dlopen():
If file is a null pointer, dlopen() shall return a
global symbol table handle for the currently running process image.
This symbol table handle shall provide access to the symbols from an
ordered set of executable object files consisting of the original
program image file, any executable object files loaded at program
start-up as specified by that process file (for example, shared
libraries), and the set of executable object files loaded using
dlopen() operations with the RTLD_GLOBAL flag. …
But if you really want to know:
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
int main()
{
// We don't need to know the library for dlopen():
void *handle = dlopen(NULL, RTLD_NOW);
if (!handle) puts(dlerror()), exit(1);
void *p = dlsym(handle, "open");
char *s = dlerror();
if (s) puts(s), exit(1);
printf("open = %p\n", p);
// But we can find the library with a Glibc extension:
Dl_info info;
if (dladdr(p, &info))
printf("%s contains %s\n", info.dli_fname, info.dli_sname);
}
When loaded a shared library is opened via the function dlopen(), is there a way for it to call functions in main program?
Code of dlo.c (the lib):
#include <stdio.h>
// function is defined in main program
void callb(void);
void test(void) {
printf("here, in lib\n");
callb();
}
Compile with
gcc -shared -olibdlo.so dlo.c
Here the code of the main program (copied from dlopen manpage, and adjusted):
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
void callb(void) {
printf("here, i'm back\n");
}
int
main(int argc, char **argv)
{
void *handle;
void (*test)(void);
char *error;
handle = dlopen("libdlo.so", RTLD_LAZY);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror(); /* Clear any existing error */
*(void **) (&test) = dlsym(handle, "test");
if ((error = dlerror()) != NULL) {
fprintf(stderr, "%s\n", error);
exit(EXIT_FAILURE);
}
(*test)();
dlclose(handle);
exit(EXIT_SUCCESS);
}
Build with
gcc -ldl -rdynamic main.c
Output:
[js#HOST2 dlopen]$ LD_LIBRARY_PATH=. ./a.out
here, in lib
here, i'm back
[js#HOST2 dlopen]$
The -rdynamic option puts all symbols in the dynamic symbol table (which is mapped into memory), not only the names of the used symbols. Read further about it here. Of course you can also provide function pointers (or a struct of function pointers) that define the interface between the library and your main program. It's actually the method what i would choose probably. I heard from other people that it's not so easy to do -rdynamic in windows, and it also would make for a cleaner communication between library and main program (you've got precise control on what can be called and not), but it also requires more house-keeping.
Yes, If you provide your library a pointer to that function, I'm sure the library will be able to run/execute the function in the main program.
Here is an example, haven't compiled it so beware ;)
/* in main app */
/* define your function */
int do_it( char arg1, char arg2);
int do_it( char arg1, char arg2){
/* do it! */
return 1;
}
/* some where else in main app (init maybe?) provide the pointer */
LIB_set_do_it(&do_it);
/** END MAIN CODE ***/
/* in LIBRARY */
int (*LIB_do_it_ptr)(char, char) = NULL;
void LIB_set_do_it( int (*do_it_ptr)(char, char) ){
LIB_do_it_ptr = do_it_ptr;
}
int LIB_do_it(){
char arg1, arg2;
/* do something to the args
...
... */
return LIB_do_it_ptr( arg1, arg2);
}
The dlopen() function, as discussed by #litb, is primarily provided on systems using ELF format object files. It is rather powerful and will let you control whether symbols referenced by the loaded library can be satisfied from the main program, and generally does let them be satisfied. Not all shared library loading systems are as flexible - be aware if it comes to porting your code.
The callback mechanism outlined by #hhafez works now that the kinks in that code are straightened out.