Tracing code from user to hardware - c

Question

I'm curious if someone can point me in the right direction here. I'm learning about computer systems programming (the basics) and I'm trying to trace code through different levels to see how each interacts with the other. An example would be calling the fgets() function in C or getline() in C++ or similar. Both of those would make calls to the system right? Is there an easy way to look at the code that is called?
I'm working on Unix (Ubuntu). Is this something that is proprietary with Windows and Apple? Any good resources out there for this kind of thing? As always, thanks guys!

Answer 1

At least in the UNIX world, the answer is fairly easy: "Use the Source, Luke".
In your example, you would look at the sources for, say, fgetc(). That's in the C standard library, and the easiest way to find the source is google something like "C libraary fgets() source".
When you get that source, you'll see a bunch of code handling buffers etc, and a system call, probably to read(2). The "2" there tells you it is documented in Chapter 2 of the manual (eg, you can find it with man 2 read).
The system call is implemented in the kernel, so then you need to read the kernel source. Proceed from there.
Now, what you need to find this all without having to read randomly about in the sources (although that's the way a lot of people have learned it, it's not very efficient) is to get hold of a book on Linux like Kerrisk's The Linux Programming Interface, which explains some of these things at a somewhat higher level than just the source.

Answer 2

Something fgets is located within libc. That is, it's a userland library linked with most C binaries. Check out glibc, which is currently the most common implementation.
Eventually, libc will start making system calls to the kernel. You can get the source at kernel.org. Check out KGDB for kernel debugging. The simplest way to to do kernel debugging is to use a second machine connected via null model cable.

Answer 3

On Windows, you could get some insight with a few things. First you'd need something called symbol files that correspond to the binaries you'd like to investigate. Symbol files associate textual names with the global/stack/heap variables floating around a program. So to map the address in memory to the function fgets, and see fgets in certain programs you'd need to have the symobls for the version of Microsoft's implementation of the C std library. Lucky for you MS makes their symbols freely available
Second you would need to capture a callstack that dove deeper than fgets. The most obvious way to do this would be to be a Microsoft developer and introduce a crash into a deep MS dll, then analyse the crash dump with a debugger and symbols, but unfortunately we can't do that. What you can do is use whats called a sampling profiler, as in this one freely available from Microsoft. A sampling profiler profiles your code by taking periodic snapshots of the callstack of your program. Using symbol files from Microsoft, we can digest that callstack into something meaningful.
Given those 2 pieces of info, it wouldn't be hard to construct a program and get some insight into what fgets calls. You can then use the sampling profiler with Microsoft's symbols to get an idea of whats going on during your program.
Along these lines I constructed the following program to try this out:
int FgetSTest()
{
FILE* fp;
fp = fopen("C:/test.txt", "w");
char data[100];
int sum = 0;
for (int i = 0; i < 100; ++i)
{
fgets(data, 100, fp);
sum += data[0];
}
fclose(fp);
return sum;
}
int _tmain(int argc, _TCHAR* argv[])
{
int sum = 0;
for (int i = 0; i < 100; ++i)
{
sum += FgetSTest();
}
std::cout << sum;
return 0;
}
Assuming you've compiled this into a program (I've compiled it into one called perfPlay.exe) you can run MS's sampling profiler on the exe as follows:
C:\path\to\exe>vsperfcmd /start:sample /output:perfPlay.vsp
Microsoft (R) VSPerf Command Version 9.0.30729 x86
Copyright (C) Microsoft Corp. All rights reserved.
C:\path\to\exe\>vsperfcmd /launch:perfPlay.exe
Microsoft (R) VSPerf Command Version 9.0.30729 x86
Copyright (C) Microsoft Corp. All rights reserved.
Successfully launched process ID:3700 perfPlay.exe
sum is:40000
C:\path\to\exe>vsperfcmd /shutdown
Microsoft (R) VSPerf Command Version 9.0.30729 x86
Copyright (C) Microsoft Corp. All rights reserved.
Shutting down the Profile Monitor
------------------------------------------------------------
Get profiler output, notice the "symbolpath" switch to point the command to Microsoft's symbol server:
C:\path\to\exe>vsperfreport perfplay.vsp /summary:all /symbolpath:srv*c:\symbols*htt
p://msdl.microsoft.com/download/symbols
You can examine the csv directly of the caller-callee report, or find a good viewer, like the one I've been working on, and you can get an idea of where fgets spends most of its time:
Sadly, not terribly insightful. Unfortunately, one of the problems you'll run into with this approach is that many of the functions fgets calls in release mode could very well be inlined -- that is they are pretty much removed as functions from the final program and their contents directly "pasted" in to where they're used.
You could try repeat the above in debug mode to see what you get, as there's less chance of inlining.

Answer 4

First things first; this task will require good tools. I find etags, cscope, and gid (from GNU idutils) indispensable tools when navigating source. Figure out how to integrate one or more of these into your favorite editor or IDE. Switch editor or IDE to get these features, there's no excuse for poor tools. If you're looking for advice on one, I love vim, a vast many people argue for emacs, and there's some folks who love their Eclipse.
You'll want the sources locally; lxr is an amazing tool, but the latency involved in repeated web requests gets tiring for any serious work. On Debian-derived systems, this is pretty easy; change directory to wherever you wish to store the source and run apt-get source eglibc to download the glibc sources. I suggest getting the kernel sources via a tarball from http://www.kernel.org or cloning the master git repository (a better choice if you want to read changelogs or easily get updates -- though it does expand to 2.7 gigabytes as of June 2012, so it obviously isn't for everyone).
Once you've built tags files for the C library, you can just run: vim -t fgets and it will open libio/bits/stdio2.h directly to the source for the fgets() routine. (It is much less readable than you may hope.) Follow these around until you eventually get to a read() system call. (It may take a while.)
Now switch to the kernel sources. Look in fs/read_write.c for this this:
SYSCALL_DEFINE3(read, unsigned int, fd, char __user *, buf, size_t, count)
One downside to the way the kernel uses macros to define system calls is that it complicates searching for functions. vim -t can't find this directly. The easiest thing to do when looking for system calls is to run gid -s SYSCALL_DEFINE | grep read. (If you find a better tool, let me know.) Once you've found the system call entry point, it'll be far easier to read the rest of the kernel source. (I generally find it more legible than glibc sources, too -- though the days of being five or six function calls away from the block-level bread() call are long gone.)