While there were times that you want to know how kernel's specific code work, the best way is to write your test code by including specific header files, e.g., I want to know how kernel's kfifo working , the first I need is including <linux/kfifo.h>.
But this is not work for me, by only include <linux/kfifo.h>, we must tell GCC the header file search path, some predefined macro like __KERNEL__ and so on, is there any sample project did this before? or you can tell me how to debug these code without copy these code out and delete some useless feature.
The most straightforward approach to debugging kernel infrastructure code (basically, everything not including device drivers) is to compile a kernel for uml architecture (make config ARCH=um and the usual stuff after; make sure to retain the debug symbols in the build). Then you can use your favorite user space debugger to run and analyze the obtained uml kernel (as it is a normal linux executable).
http://lxr.linux.no/#linux+v3.12.1/Documentation/virtual/uml/UserModeLinux-HOWTO.txt
By the way, the linked howto is badly out of date - you don't need to download and patch anything; UML is part of the kernel for many years now. But it will get you started, especially if you google around a bit for additional details.
Related
I have a large code written in C, but I did not write all of it myself. I wish to create an overview of the call structure in the code for reference. That is: I wish to know what (non-standard) functions are called by the different functions in the code, and thus create a hierarchy or a tree of the different functions. Are there any free, Unix compatible programs (that means no Visual Studio, but a Vim plugin or such would be neat) that can do this, or will I have to write something that can do this myself?
Doxygen does that too, it has to be enabled though.
For an overview of available tools see
http://en.wikipedia.org/wiki/Call_graph
There is a Vim plugin C Call-Tree Explorer called CCTree
http://www.vim.org/scripts/script.php?script_id=2368
As you mentioned a Vim plug-in, check out http://sites.google.com/site/vimcctree/. It uses CScope to generate the tree, so you will need to first generate a CScope db of your source files.
Have a look at http://www.gson.org/egypt/ This uses GCC to process the code and extracts the interdependencies within the program from the AST it emits.
gprof will do that. It also generates an execution profile, but in doing so it creates a call tree.
I just downloaded SourceTrail (https://github.com/CoatiSoftware/Sourcetrail/releases) and it did what I wanted, which was pretty close to what I think you want.
(What I wanted was to find out what routines called the function I was considering changing, or needed to understand).
Note that it is no longer maintained, but it did exactly what I wanted. It runs under Windows and Linux, and made finding who calls a function pretty trivial (as well as following that function's call tree down as needed). If you care, it has a GUI (is a GUI? whatever).
It does the parsing itself, but it didn't take very long to run, perhaps about the same time or a little less than compiling the code.
But if you want text only, or don't want to use a gui, or don't want to have it scan the code, this isn't for you.
(Notes - in my case, I was hyper-focused on one or 2 functions, and didn't care what system functions were being called. I spent some time stubbing out all the include files that were needed (since I ran the parse on one machine (A Linux machine) that didn't have all the include files needed for the Windows program I was looking at, and then did the exploration on a different (Windows) machine. Which, I should mention, worked perfectly. I just copied the entire source tree from my Linux machine to my Windows machine (which included the Sourcetail project file), loaded Sourcetail and had it load the project - done.)
Okay, I'm reading about Linux kernel development and there are some code snippets using kernel's data structures and stuff. Let's say I'd like to experiment with them, e.g. there's a very simple snippet:
#include <../../linux-2.6.37.1/include/linux/sched.h>
struct task_struct *task;
for_each_process(task) {
printk("%s[%d]\n", task->comm, task->pid);
}
Seems pretty simple, eh? Now then, I can't possibly build the thing. I am using NetBeans. The sched.h is the correct file as if one can CTRL+clicks on it, one is brought to the right file.
Do I need to include somehow my sample file and build the whole kernel from the Makefile? I just wished to see that it builds and possibly that it would work. If I need to build the whole kernel how would I actually test my stuff?
I must be making something really stupid as I am very new to kernel development. I am quite a bit lost.
Thanks guys!
You do not need to compile the whole kernel, but you have to at least create a kernel module, which is far easier to compile. You should have a look at a tutorial, such as this, or even a full blown book like this.
Keep in mind that not all kernel code can be moved to a module - just those that use the public (exported) interfaces of the kernel. Code that is intrinsic to the kernel core parts (e.g. the VM or the scheduler) is probably inaccessible from the rest of the kernel.
Also keep in mind that trying out kernel code on your development machine is not advised - a
slight mistake can easily bring the whole system down. You should look at trying out your kernel code in a separate virtual machine e.g. in VirtualBox.
A detail that makes thing harder: in general you can only insert a module in the kernel that it was built for. A module compiled on the host system can be used on the testing VM if and only if the kernel is identical, i.e. the same kernel package version from the same distribution. Considering that you will want to upgrade your host distribution, in my opinion it is just simpler to build the module on the testing system.
Since you need a full development suite for C, you should probably install one of the popular Linux distrbutions. It should be more stable and you can have access to its user community. If you want to keep its size down, you can just install the base system without an X server or graphical applications.
BTW Netbeans is designed to develop userspace applications. You can probably adapt it for kernel code, but it will never be as suited as it is for userspace programming. As a matter of fact, no IDE is really suitable. Kernel code cannot be run from userspace (let alone using a separate VM), which breaks down the normal edit->compile->run->debug workflow cycle that IDEs automate.
Most kernel developers just use a souped-up editor with syntax highlighting for C, such as Vim or Emacs. Emacs is actually an IDE (and so much more) but, as I mentioned above, you cannot easily use an IDE-based workflow for kernel code development.
You can build a loadable kernel module if you don't want to build the whole kernel - e.g. see http://www.linux-tutorial.info/modules.php?name=Howto&pagename=Module-HOWTO.
All the code you write, compile and run as user programs run as ... well, user programs, in user mode. The kernel runs in kernel mode. Both modes are separated and cannot see each other directly. They communicate through defined interfaces. These interfaces are the C system calls (as opposed to the C library calls).
To be able to access the task_struct structures, your code has to be running in kernel mode. The best choice for this is to write a kernel module, and to load it in the kernel.
Very little kernel code can run outside the kernel in any form. Most kernel code is very 'intertwingled' (to use a phrase I learned from a coworker years ago to describe excessive coupling) with other portions of kernel code. Functions 'know' structure definitions for many many structures away from what they are working on. Typical software engineering people hate code like this:
if (unlikely(inode_init_always(sb, inode))) {
if (inode->i_sb->s_op->destroy_inode)
inode->i_sb->s_op->destroy_inode(inode);
else
kmem_cache_free(inode_cachep, inode);
return NULL;
}
This routine has to know how to destroy inodes through three structures and the calling convention of a function pointer on the other end of the chain. The kernel community knows all these functions very well, and are quite happy to modify member names in structures all throughout the kernel when changes are made, but this sort of tight coupling makes running portions of the kernel in userspace on their own extremely difficult. (And believe me, sometimes I wish I could write tests on my small portions of kernel code that would run in userspace.)
If you want to play around, it's not too hard to get a virtual system up and running these days with qemu+kvm or virtualbox or uml to try making modifications to the kernel. It is pretty hard to just "play" with structures on a live running system, but it is much more feasible than trying to compile portions of the kernel in userspace.
Good luck. :)
You might enjoy using systemtap as a wrapper for small bits of kernel module code:
# stap -g -e 'probe begin { your_function() exit() }
%{
#include <linux/whatever.h>
%}
function your_function() %{
... insert safe c code here ...
%}'
It can automatically cross-compile too (if you use stap --remote=VIRTMACHINE ...).
The term has several definition according to Wikipedia, however what I'm really interested in is creating a program that has all its needed dependencies included within the source folder, so the end user doesn't need to install additional libraries for the app to install. For example, how Mac apps has all its dependencies all within the program itself already...
or is there a function that autotools does this? I'm programming in the Linux environment...
Are you talking about the source code of your application, or about your application binary?
The answer I'd give for both the cases depends on what libraries you're using.
If you're using libraries that you can find anywhere, that are somehow standard and/or that are quite big, you shouldn't attach them to your application, just require them both to build and to run your application.
Anyway don't be much concerned about your source code: little people will build your application, and they probably know something about programming and how a Linux system works; it won't be a big deal to require many (also not-so-common) dependences to build your application.
For what concerns the binary version it could be a little more problematic, since it will be used by end users who often don't know anything about libraries and programming stuff: you could choose to statically link the smallest and most uncommon libraries to your binary, in order to have less dependences.
You could do it, if you link statically, but it'd be somewhat unusual, and depending on what your program is supposed to do, you might be limiting yourself.
The alternative, if this is not just a one-off project, is to create a Linux Standard Base compatible RPM package and restrict yourself to linking against the libraries and symbols that LSB defines.
Run ldd on your program to discover all dependencies, then copy these to your directory, and add a program-wrapper script that issues
#!/bin/sh
LD_LIBRARY_PATH="${0##*/}:$LD_LIBRARY_PATH" exec "${0##*/}/real-program" "$#";
Duplicating the Mac OS X .app behavior on a plain POSIX system is difficult because it is very hard to guarantee that a process can find it's own executable (there are several way that will almost always work...). Mac OS X provides a OS service for this, but Linux (for instance) does not.
Once you've accomplished that feat, this becomes possible. Though, as others have mentioned, it loses the ability to share resource demands (disk space, RAM space, cache space) with other programs that use the same libraries because you'd be using static copies, or dynamically loading your own copy from the .app-like bundle.
I just built an updated version of SDL.dll, an open-source C DLL that my Delphi project uses, with the Express edition of Visual C++ 2005. I dropped it in the folder with my EXE and tried to run it, but it won't load:
The procedure entry point SDL_RWFromFP could not be located in the dynamic
link library SDL.dll.
Now C never was my strong point, but I remember enough of it from college to try and track this one down. I went poking around in the source code to see what had happened to this function, and I found it grayed out, beneath a preprocessor directive:
#ifdef HAVE_STDIO_H
IIRC, STDIO is the standard C I/O library. I assume this means that it's not available. Anyone know why that would be and how to fix it? Is this a Visual C++ issue or an SDL one?
Most often in the Unix/Linux world, names like HAVE_STDIO_H indicate that the code has been 'autoconfiscated' (which is the official term used to describe the state of having been made to work with the 'autotools' such as 'autoconf'). In such a set up, the configure process would determine whether <stdio.h> was available and would set #define HAVE_STDIO_H 1 in the config.h file that it generates. The compilation would then discover that the platform has <stdio.h> and would compile the matching code (the stuff that is currently greyed out).
Adapting to your Windows environment, somewhat less than 100% confidently since there could be some other significance to HAVE_STDIO_H on Windows, you might decide that it would be OK to include -DHAVE_STDIO_H in the command line options when you run the compiler. Or you might create the config file by hand, and define -DHAVE_CONFIG_H (which is the normal way to indicate that configuration settings are in the file 'config.h'). In the 'config.h' file, you'd have #define HAVE_STDIO_H 1 as mentioned above.
Note: on Unix, you normally find a shell script called 'configure' which you run to create the config.h file. If you have Cygwin, there's an outside chance that you can use that script on Windows - I've just checked that an autoconfiscated package I created on Solaris was configurable on Windows under Cygwin and it mostly worked - all except some network handling. I'd not guarantee that it will always fail (but it's software - guaranteeing anything is pretty dangerous). I should add that the problem is in my auto-configuration code (the tests for the network functionality clearly aren't quite correct), and not in Cygwin per se. If I'd done the job properly, it would have worked. (Someone said "There is no portable code; there is only code that has been ported". That applies here.)
You do need a good simulation of a Unix environment. MingW might also work.
What do the preprocessor directives above it do?
Most likely, this is simply for determining some properties of your compiler, and it's greyed out because whatever the macro signifies, does not apply to your compiler.
Your problem is most likely elsewhere.
Since it can't find the function SDL_RWFromFP, you should probably see if that function is for some reason disabled by a preprocessor directive as well.
However, my guess is that the function exists, but does not get marked as __declspec( dllexport )
Without that, it won't get exposed so programs loading the DLL can call it. Most likely you have to #define something to specify that you want to create a DLL, which will enable the necessary preprocessor magic to insert the dllexport part in front of the function.
Normally, you should not have to add any HAVE_STDIO_H: those are not meant to be public anyway (although sadly many projects pollute the public namespace with those).
I would guess you did not build SDL correctly - or that Visual studio 2005 is not well supported by SDL (I don't know much about SDL, so those are really wild guesses without much information). Is there any test suite for SDL, such as you can test the built dll ?
I'm looking for a tool that, given a bit of C, will tell you what symbols (types, precompiler definitions, functions, etc) are used from a given header file. I'm doing a port of a large driver from Solaris to Windows and figuring out where things are coming from is getting to be difficult, so this would be a huge help. Any ideas?
Edit: Not an absolute requirement, but tools that work on Windows would be a plus.
Edit #2: To clarify what I'm trying to do, I have a codebase I'm trying to port, which brings in a large number of headers. What I'd like is a tool that, given foo.c, will tell me which symbols it uses from bar.h.
I like KScope, which copes with very large projects.
KScope http://img110.imageshack.us/img110/4605/99101zd3.png
I use on both Linux and Windows :
gvim + ctags + cscope.
Same environment will work on solaris as well, but this is of course force you to use vim as editor, i pretty sure that emacs can work with both ctags and cscope as well.
You might want give a try to vim, it's a bit hard at first, but soon you can't work another way. The most efficient editor (IMHO).
Comment replay:
Look into the cscope man:
...
Find functions called by this function:
Find functions calling this function:
...
I think it's exactly what are you looking for ... Please clarify if not.
Comment replay 2:
ok, now i understand you. The tools i suggested can help you understand code flow, and find there certain symbol is defined, but not what are you looking for.
Not what you asking for but since we are talking i have some experience with porting and drivers (feel free to ignore)
It seems like compiler is good enough for your task. You just starting with original file and let compiler find what missing part, it will be a lot of empty stubs and you will get you code compiled.
At least for beginning i suggest you to create a lot of stubs and modifying original code as less as possible, later on once you get it working you can optimize.
It's might be more complex depending on the type of driver your are porting (I'm assuming kernel driver), the Windows and Solaris subsystems are not so alike. We do have a driver working on both solaris and windows, but it was designed to be multi platform from the beginning.
emacs and etags.
And I leverage make to run the tag indexing for me---that way I can index a large project with one command. I've been thinking about building a master index and separate module indecies, but haven't gotten around to implementing this yet...
#Ilya: Would pistols at dawn be acceptable?
Try doxygen, it can produce graphs and/or HTML and highly customizable