I am trying to invoke show_mem() from mm.h in a user defined kernel module. When I compile it shows show_mem undefined. I am running Ubuntu 14.04 and have a compiled linux kernel 3.19.
/*
* Author - [Deepak]
*/
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
#include <linux/init.h> /* Needed for the macros */
#include <linux/mm.h> /* Needed for show_mem */
#include <asm/cacheflush.h>
#include <linux/mm.h>
static char *user_data1 __initdata = "Hello World";
static int *user_data2 __initdata = 2;
static int __init starter(void)
{
printk(KERN_INFO "[ds494] Loading Hello2 module - %s %d \n",user_data1,user_data2);
show_mem(1);
return 0;
}
static void __exit ending(void)
{
printk(KERN_INFO "[ds494] Exiting Hello2 module - Goodbye World 2\n");
}
module_init(starter);
module_exit(ending);
And below is the make file -
obj-m += memmod.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
And I get the below error -
**MODPOST 1 modules
WARNING: "show_mem" [/home/deepak/cs/hw/homework_4/memmod.ko] undefined!
LD [M] /home/deepak/cs/hw/homework_4/memmod.ko
make[1]: Leaving directory '/home/deepak/Downloads/linux-3.19'**
Any suggestions, please.
Root cause.
You can't use show_mem() function in loadable modules, because it's not exported by EXPORT_SYMBOL.
Possible solutions.
Basically you have 3 options how to workaround this problem.
Modifying kernel sources
download kernel source (see this)
add code for exporting this symbol (like this, but this patch is intended for ARM architecture, for x86 see lib/show_mem.c)
build your module using those modified sources
If you also need to run your module -- you will need to build your customized kernel first and run it (instead of Ubuntu stock kernel).
It's not upstreamable solution though and quite frankly nobody will be able to use your module (you will need to provide modified kernel as well).
Compiling your module as built-in module.
It can be done inside of kernel tree, using obj-y instead of obj-m). In that case you will be able to use show_mem() function. Just like the first option, this option implies modifying of kernel sources.
Write your own show_mem() implementation.
I'm not sure about this one, though, because it may turn out that you can't use API needed for this task in loadable module at all. It's also can be quite difficult to implement this.
Conclusion.
If it's only educational task (which I guess it is), I'd say go with first option.
If you really need to implement that as a loadable module, and you can't modify kernel sources, I afraid you only have 3rd option, which is hardest one.
Related
Background
Trying to profile an executable, I experimented the profiler Intel VTune and I learn that there is an API library (ITT) that provide utility to start/stop profiling. Its basic functions __itt_resume() and __itt_pause(). What triggers me is that the library is optional, i.e. if the runtime library of ITT is not loaded, these functions are basically noops.
Optional library?
I want to know (first of all on Linux)
Does a process checks that the dynamic library he is linking to is loaded when he starts or when each symbol, or the first symbol of the library is called at runtime (i.e. lazy initialization)? I think on Windows it's at startup because of can't find XXX.dll messages, but I am not sure on Linux. Also, with the example, I don't get any compilation & execution issues even if the symbol is not defined in some_process.c.
How to implement this on Linux? Looking at the Github repo of ITT, among many macro trickery, I feel like the key is here:
#define ITTNOTIFY_VOID(n) (!ITTNOTIFY_NAME(n)) ? (void)0 : ITTNOTIFY_NAME(n)
Basically it wraps every function call with a function pointer call if its not NULL.
How to implement this in a cross-platform way (Windows, Mac, Linux) ?
I end up with a minimal example that looks like the code linked here, but it does not work as it should. In the linked version, my_api_hello_impl() is not called as it should. Also, there is no crash checking the value of the extern symbol api_hello_ptr() when the library is not linked.
my_api.c
#include "my_api.h"
#include <stdio.h>
void(*api_hello_ptr)();
void api_hello_impl()
{
printf("Hello\n");
}
__attribute__((constructor))
static void init()
{
printf("linked\n");
api_hello_ptr = api_hello_impl;
}
my_api.h
#pragma once
extern void(*api_hello_ptr)();
inline void api_hello() { if(api_hello_ptr) api_hello_ptr(); }
some_process.c
#include "my_api.h"
int main()
{
// NOOPS of not linked at runtime
api_hello();
}
Makefile
# my_api is not linked to some_process
some_process: some_process.c my_api.h
$(CC) -o $# $<
my_api.so: my_api.c my_api.h
$(CC) -shared -fPIC -o $# $<
test_linked: some_process my_api.so
LD_PRELOAD="$(shell pwd)/my_api.so" ./some_process
test_unlinked: some_process my_api.so
./some_process
.PHONY: test_linked test_unlinked
Output:
$ make test_linked
LD_PRELOAD="/tmp/tmp.EkrQbILrNg/my_api.so" ./some_process
linked
$ make test_unlinked
./some_process
Does a process checks that the dynamic library he is linking to is loaded when he starts
Yes, it does. If a dynamic library is linked, then it is a runtime requirement and the system loader will not start execution of a program without finding and loading the library first. There are mechanisms for delayed-loading, but it is not the norm on Linux, they are done manually or using custom libraries. By default, all dynamically linked objects need to be loaded before execution starts.
Note: I'm assuming we are talking about ELF executables here since we are on Linux.
How to implement this on Linux?
You can do it using macros or wrapper functions, plus libdl (link with -ldl), with dlopen() + dlsym(). Basically, in each one of those wrappers, the first thing you do is check if the library was already loaded, and if not, load it. Then, find and call the needed symbol.
Something like this:
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
static void *libfoo_handle = NULL;
static int (*libfoo_func_a)(int, int);
static void load_libfoo_if_needed(void) {
if (!libfoo_handle) {
// Without "/" in the path, this will look in all standard system
// dynamic library directories.
libfoo_handle = dlopen("libfoo.so", RTLD_LAZY | RTLD_GLOBAL);
if (!libfoo_handle) {
perror("failed to load libfoo.so");
_exit(1);
}
// Optionally use dlsym() here to initialize a set of global
// function pointers, so that you don't have to do it later.
void *tmp = dlsym(libfoo_handle, "func_a");
if (!tmp) {
perror("no symbol func_a in libfoo.so");
_exit(1);
}
*((void**)&libfoo_func_a) = tmp;
}
}
int wrapper_libfoo_func_a(int a, int b) {
load_libfoo_if_needed();
return libfoo_func_a(a, b);
}
// And so on for every function you need. You could use macros as well.
How to implement this in a cross-platform way (Windows, Mac, Linux)?
For macOS, you should have dlopen() and dlsym() just like in Linux.
Not sure how to exactly do this on Windows, but I know there is LoadLibrary() available in different flavors (e.g. one, two, etc.), which should be more or less the equivalent of dlopen() and GetProcAddress(), which should be the equivalent of dlsym().
See also: Loading a library dynamically in Linux or OSX?
I have build a Linux kernel for the beaglebone black using buildroot. Now I would like to develop a hello world Linux kernel module application:
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("Dual BSD/GPL");
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
The problem is I still keep missing some header files. After finally gathering them all, I get an error that the code is not compilable (many errors, I don't want to paste them all). What I was wondering is either I am really including the right files?
At the moment I have:
/home/lukasz/brl/Machine/beaglebone/build/linux-headers-a75d8e93056181d512f6c818e8627bd4554aaf92/include
/home/lukasz/brl/Machine/beaglebone/build/uboot-2018.01/arch/x86/include
/home/lukasz/brl/Machine/beaglebone/build/linux-headers-a75d8e93056181d512f6c818e8627bd4554aaf92/arch/arm/include/generated
/home/lukasz/brl/Machine/beaglebone/build/linux-headers-a75d8e93056181d512f6c818e8627bd4554aaf92/arch/arm/include
/home/lukasz/brl/Machine/beaglebone/build/linux-a75d8e93056181d512f6c818e8627bd4554aaf92/include
Its a bit odd to me that the C include files and asm files are so scattered around within the directory. Are there some mistakes in my understanding of the topic here?
My Linux version:
# uname -a
Linux buildroot 4.9.59 #1 SMP Fri Oct 5 11:55:54 CEST 2018 armv7l GNU/Linux
To compile a kernel module, you need the real kernel sources, not just the kernel header files. You have to build from the kernel source directory with M= pointing to the source of your modules. And together with your module source, you of course also need a working Makefile. These steps are explained in any of the dozens of how-to-write-a-kernel-module guides, e.g. this one.
For cross-compilation, you also need to pass the appropriate arguments so that the kernel knows for which architecture to build and which cross-compiler to use. At the very least, this means you have to give the ARCH= and CROSS_COMPILE= options when building. Sometimes you need additional options (e.g. to point to the appropriate depmod tool).
To simplify this, Buildroot offers kernel module infrastructure. In the simplest case, you can just create a Config.in file containing
config BR2_PACKAGE_HELLOMOD
bool "hellomod"
depends on BR2_LINUX_KERNEL
and a hellomod.mk file containing
HELLOMOD_SITE = /path/to/hellomod/source
$(eval $(kernel-module))
$(eval $(generic-package))
You also have to source the Config.in from package/Config.in in the Buildroot tree. Or better yet, use an external tree so you don't have to modify Buildroot itself.
I am trying to add a switch/GPIO interrupt. I want to write it as part of kernel source tree. After building the kernel image and deploying to my custom board it has to appear in proc/interrupts. I have already written the module and it is working if do insmod. Instead of compiling separately i want it to be a part of my kernel tree.
What are the steps to add the irq to kernel source.
Actually if you have written the module inside the kernel tree, it is pretty straightforward:
Lets say you put the source code in drivers directory, so the hierarchy looks as follows:
drivers/hello
Kconfig
Makefile
hello.c
In drivers/Makefile you should add the following:
obj-$(CONFIG-HELLO) += hello/
In drivers/Kconfig you should add the following:
source "drivers/hello/Kconfig"
Sample code for drivers/hello/Kconfig:
config HELLO
tristate "Hello world module"
default n
help
Enable Hello world module support
Sample code for drivers/hello/Makefile:
obj-$(CONFIG_HELLO) += hello.o
Sample code for drivers/hello.c:
#include <linux/module.h>
#include <linux/moduleparam.h>
...
...
static int __init hello_init(void)
{
...
}
static void __exit hello_exit(void)
{
...
}
module_init(hello_init);
module_exit(hello_exit);
MODULE_AUTHOR("Obi One Kenoby");
MODULE_DESCRIPTION("Hello Driver");
MODULE_LICENSE("GPL");
MODULE_VERSION("1.0");
Now you should be able to see the hello module in make menuconfig - select 'm' for module and '*' for built in. the module_init/module_exit macros works with both options.
Everywhere I search for Linux Kernel Development, I get answers for creating Linux Kernel modules.
Example
/*
* hello−1.c − The simplest kernel module.
*/
#include <linux/module.h> /* Needed by all modules */
#include <linux/kernel.h> /* Needed for KERN_INFO */
int init_module(void)
{
printk(KERN_INFO "Hello world 1.\n");
/*
* A non 0 return means init_module failed; module can't be loaded.
*/
return 0;
}
void cleanup_module(void)
{
printk(KERN_INFO "Goodbye world 1.\n");
}
Here, there is init_module and cleanup_module functions which i understand contains things to be executed when the kernel is initialized and cleaned up.
There are made by adding
obj-m += hello-1.c
to the makefile.
But I dont want this. I want to add a built in program, not a driver, basically a service to facilitate cloud uploading of some data from the kernel level. I dont event want the module option for the program when compiling the kernel.
I understand for just programs I should use obj-y not obj-m. But there is no manual to write these kind of programs. Why? Am I missing something? Does these programs also have the init_module and cleanup_module functions even though they are not modules?
For example consider that your source is under driver/new in the linux kernel source tree.
You need to modify Makefile's under drivers and new to build your module statically into linux kernel.
Under drivers/Makefile add the below line at the end.
obj-y += new/
Under drivers/new/Makefile add the below line at the end.
obj-y += hello.o
After build the linux kernel. And load to see that your module has printed the printk messages using dmesg command.
Note: When building module statically into linux, change
int init_module(void)
to
int __init init_module(void)
and change
void cleanup_module(void)
to
void __exit cleanup_module(void)
Look into kernel doc Makefiles
Refer:
"
--- 3.2 Built-in object goals - obj-y
The kbuild Makefile specifies object files for vmlinux
in the $(obj-y) lists. These lists depend on the kernel
configuration.
Kbuild compiles all the $(obj-y) files. It then calls
"$(LD) -r" to merge these files into one built-in.o file.
built-in.o is later linked into vmlinux by the parent Makefile.
The order of files in $(obj-y) is significant. Duplicates in
the lists are allowed: the first instance will be linked into
built-in.o and succeeding instances will be ignored.
Link order is significant, because certain functions
(module_init() / __initcall) will be called during boot in the
order they appear. So keep in mind that changing the link
order may e.g. change the order in which your SCSI
controllers are detected, and thus your disks are renumbered.
Example:
#drivers/isdn/i4l/Makefile
# Makefile for the kernel ISDN subsystem and device drivers.
# Each configuration option enables a list of files.
obj-$(CONFIG_ISDN_I4L) += isdn.o
obj-$(CONFIG_ISDN_PPP_BSDCOMP) += isdn_bsdcomp.o
"
I'm writing an API as a kernel module that provides device drivers with various functions. I wrote three functions in mycode.c. I then built and loaded the module, then copied mycode.h into < kernel >/include/linux. In a device driver, I have a #include < linux/mycode.h > and call those three functions. But when I build the driver module, I get three linker warnings saying that those functions are undefined.
Notes:
The functions are declared extern in mycode.h
The functions are exported using EXPORT_SYMBOL(func_name) in mycode.c
Running the command nm mycode.ko shows all three functions as being available in the symbol table (capital T next to them, meaning the symbols are found in the text (code) section)
After loading the module, the command grep func_name /proc/kallsyms shows all three functions as being loaded
So clearly the functions are being exported correctly and the kernel knows what and where they are. So why can't the driver see their definitions? Any idea what am I missing?
EDIT: I found some information about this here: http://www.kernel.org/doc/Documentation/kbuild/modules.txt
Sometimes, an external module uses exported symbols from another
external module. kbuild needs to have full knowledge of all symbols
to avoid spitting out warnings about undefined symbols. Three
solutions exist for this situation.
NOTE: The method with a top-level kbuild file is recommended but may
be impractical in certain situations.
Use a top-level kbuild file If you have two modules, foo.ko and
bar.ko, where foo.ko needs symbols from bar.ko, you can use a
common top-level kbuild file so both modules are compiled in the
same build. Consider the following directory layout:
./foo/ <= contains foo.ko
./bar/ <= contains bar.ko
The top-level kbuild file would then look like:
#./Kbuild (or ./Makefile):
obj-y := foo/ bar/
And executing
$ make -C $KDIR M=$PWD
will then do the expected and compile both modules with full
knowledge of symbols from either module.
Use an extra Module.symvers file When an external module is built,
a Module.symvers file is generated containing all exported symbols
which are not defined in the kernel. To get access to symbols from
bar.ko, copy the Module.symvers file from the compilation of bar.ko
to the directory where foo.ko is built. During the module build,
kbuild will read the Module.symvers file in the directory of the
external module, and when the build is finished, a new
Module.symvers file is created containing the sum of all symbols
defined and not part of the kernel.
Use "make" variable KBUILD_EXTRA_SYMBOLS If it is impractical to
copy Module.symvers from another module, you can assign a space
separated list of files to KBUILD_EXTRA_SYMBOLS in your build file.
These files will be loaded by modpost during the initialization of
its symbol tables.
But with all three of these solutions, in order for any driver to use my API, it would have to either create a new Makefile or have direct access to my Module.symvers file? That seems a bit inconvenient. I was hoping they'd just be able to #include my header file and be good to go. Do no other alternatives exist?
From my research, it seems that those are the only three ways to handle this situation, and I've gotten each of them to work, so I think I'll just pick my favorite out of those.
Minimal QEMU + Buildroot example
I have tested the following in a fully reproducible QEMU + Buildroot environment, so maybe having this working version version will help you find out what is wong with your code.
GitHub upstream is centered on the files:
dep.c
dep2.c
Makefile
dep.c
#include <linux/delay.h> /* usleep_range */
#include <linux/kernel.h>
#include <linux/kthread.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
int lkmc_dep = 0;
EXPORT_SYMBOL(lkmc_dep);
static struct task_struct *kthread;
static int work_func(void *data)
{
while (!kthread_should_stop()) {
printk(KERN_INFO "%d\n", lkmc_dep);
usleep_range(1000000, 1000001);
}
return 0;
}
static int myinit(void)
{
kthread = kthread_create(work_func, NULL, "mykthread");
wake_up_process(kthread);
return 0;
}
static void myexit(void)
{
kthread_stop(kthread);
}
module_init(myinit)
module_exit(myexit)
dep2.c
#include <linux/delay.h> /* usleep_range */
#include <linux/kernel.h>
#include <linux/kthread.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
extern int lkmc_dep;
static struct task_struct *kthread;
static int work_func(void *data)
{
while (!kthread_should_stop()) {
usleep_range(1000000, 1000001);
lkmc_dep++;
}
return 0;
}
static int myinit(void)
{
kthread = kthread_create(work_func, NULL, "mykthread");
wake_up_process(kthread);
return 0;
}
static void myexit(void)
{
kthread_stop(kthread);
}
module_init(myinit)
module_exit(myexit)
And now you can do:
insmod dep.ko
insmod dep2.ko
With that Buildroot setup, things are already configuring depmod /lib/module/*/depmod with the dependency, so just this is enough to load both:
modprobe dep
Also, if you built your kernel with CONFIG_KALLSYMS_ALL=y, then the exported symbol can be seen with:
grep lkmc_dep /proc/kallsyms
see also: Does kallsyms have all the symbol of kernel functions?
OK: You have one module where the function is and one place what wants to import it right?
You must use "EXPORT_SYMBOL("name of the function") such as foo in the place where the function is. So in the "c" file have the function "foo" defined and put in:
EXPORT_SYMBOL(foo)
Make sure you have the prototype for "foo" in a common header file (you can have it in separate places for each module and it will work but you are asking for trouble if the signatures change). So say: void foo(void *arg);
Then the other module that wants it just invoke "foo" and you are good.
Also: Make sure that you load the module with foo first. If you have cross dependencies like module2 needs foo from module1 and module1 needs bar from module2 you need to have one register functions with another. If you want to know please ask a separate Q.