I'm working on an integrated SDK for powerpc microprocessor which has some kernel modules.
The kernel(3.12.37) and toolchain(powerpc gcc 4.9.2) provided by Yocto project.
When i make SDK in native machine(x86_64bit Ubuntu 14) by mentioned toolchain and kernel source, I see these warnings during compilation of kernel module:
WARNING: "__udivdi3" [modules/foo.ko] undefined!
WARNING: "__umoddi3" [modules/foo.ko] undefined!
These warnings appear when i insmod kernel module.
I realized that this functions are used for 64bit operations and libgcc has these functions.
this is compilation flags:
"-Wall -Werror -gdwarf-3 -Wframe-larger-than=16384 -mcpu=powerpc"
also, i gave this flag to compile statically "-Wl,-static-libgcc" and didn't work.
my questions are:
why does linker is going to use 64 bit functions which powerpc arch($KERNELDIR/arch/powerpc) doesn't have? (Linker knows that ARCH is powerpc)
"in the kernel space, compiler does not use libgcc or other userspace functions and libs", is this right? if not, please explain to me.
and finally how can i solve this problem?
Thanks.
The compiler will generate code that calls functions in libgcc in order to perform certain operations. In this case, they are being called to perform the regular division (/) and modulo (%) operations on 64-bit quantities.
Correct, the kernel doesn't use libgcc. Kernel code needs to be written to prevent the compiler making calls to functions in libgcc (in this case for division and module operations on 64-bit quantities).
Any code that does division or modulo operations on 64-bit quantities (apart from the native signed and unsigned long types on 64-bit architectures) needs to be rewritten to use special 64-bit division functions and macros provided by the Linux kernel instead of the regular / and % operators.
In the old days, this was done by calling the do_div(n,base) macro defined by #include <asm/div64.h>, but that has an unusual, non-functionlike interface in that it modifies the caller's n parameter in place to become the dividend n / base in addition to returning the remainder n % base.
For new code, it is better to avoid the old do_div(n,base) macro and make use of the functions declared by #include <linux/math64.h>. There are various functions defined there for signed and unsigned arguments and for both 32-bit and 64-bit remainders. You need to pick the most appropriate function to match the requirements of your code.
Related
If you've ever linked a kernel with gcc you may know the parameter -lgcc.
Is this parameter important ? What does it do ?
If you do some driver/kernel dev, you may use the -nostdlib to remove your module from the bloated stdlib. However, you also remove all the internal hacks GCC has in order to have a consistent behaviour on a whole range of hardware.
http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Link-Options.html
-nostdlib
Do not use the standard system startup files or libraries when linking. No startup files and only the libraries you specify will be
passed to the linker, options specifying linkage of the system
libraries, such as -static-libgcc or -shared-libgcc, will be ignored.
The compiler may generate calls to memcmp, memset, memcpy and memmove.
These entries are usually resolved by entries in libc. These entry
points should be supplied through some other mechanism when this
option is specified.
One of the standard libraries bypassed by -nostdlib and -nodefaultlibs is libgcc.a, a library of internal subroutines that GCC uses to overcome shortcomings of particular machines, or special needs
for some languages. (See Interfacing to GCC Output, for more
discussion of libgcc.a.) In most cases, you need libgcc.a even when
you want to avoid other standard libraries. In other words, when you
specify -nostdlib or -nodefaultlibs you should usually specify -lgcc
as well. This ensures that you have no unresolved references to
internal GCC library subroutines. (For example, `__main', used to
ensure C++ constructors will be called; see collect2.)
https://gcc.gnu.org/onlinedocs/gcc-4.6.1/gccint/Interface.html#Interface
3 Interfacing to GCC Output
GCC is normally configured to use the same function calling convention
normally in use on the target system. This is done with the
machine-description macros described (see Target Macros).
However, returning of structure and union values is done differently
on some target machines. As a result, functions compiled with PCC
returning such types cannot be called from code compiled with GCC, and
vice versa. This does not cause trouble often because few Unix library
routines return structures or unions.
GCC code returns structures and unions that are 1, 2, 4 or 8 bytes
long in the same registers used for int or double return values. (GCC
typically allocates variables of such types in registers also.)
Structures and unions of other sizes are returned by storing them into
an address passed by the caller (usually in a register). The target
hook TARGET_STRUCT_VALUE_RTX tells GCC where to pass this address.
By contrast, PCC on most target machines returns structures and unions
of any size by copying the data into an area of static storage, and
then returning the address of that storage as if it were a pointer
value. The caller must copy the data from that memory area to the
place where the value is wanted. This is slower than the method used
by GCC, and fails to be reentrant.
On some target machines, such as RISC machines and the 80386, the
standard system convention is to pass to the subroutine the address of
where to return the value. On these machines, GCC has been configured
to be compatible with the standard compiler, when this method is used.
It may not be compatible for structures of 1, 2, 4 or 8 bytes.
GCC uses the system's standard convention for passing arguments. On
some machines, the first few arguments are passed in registers; in
others, all are passed on the stack. It would be possible to use
registers for argument passing on any machine, and this would probably
result in a significant speedup. But the result would be complete
incompatibility with code that follows the standard convention. So
this change is practical only if you are switching to GCC as the sole
C compiler for the system. We may implement register argument passing
on certain machines once we have a complete GNU system so that we can
compile the libraries with GCC.
On some machines (particularly the SPARC), certain types of arguments
are passed “by invisible reference”. This means that the value is
stored in memory, and the address of the memory location is passed to
the subroutine.
If you use longjmp, beware of automatic variables. ISO C says that
automatic variables that are not declared volatile have undefined
values after a longjmp. And this is all GCC promises to do, because it
is very difficult to restore register variables correctly, and one of
GCC's features is that it can put variables in registers without your
asking it to.
Is it possible to somehow determine whether an intrinsic function, such as __builtin_bswap16 is provided by the compiler? Preferably, I would like to be able to determine whether this function exists using just preprocessor.
In my particular case, I was using __builtin_bswap16 / 32 / 64 functions in my code which worked fine with GCC 4.x when compiling for 32-bit. Later I switched to a 64-bit Linux and noticed that __builtin_bswap16 suddenly disappeared - I received a linker error:
"undefined reference to `__builtin_bswap16'".
I guess this has something to do with the availability of certain ASM operations in 64-bit mode.
On a later occasion I was trying to compile this code on a different machine where unfortunately only an older version of GCC is installed and does not support these functions at all.
I would like to make this code compilable everywhere, using __builtin_bswap functions if provided, and fall back to hand-coded byteswap routine if not. Is it possible to achieve this somehow with just preprocessor?
My obvious attempt, e.g.:
...
#define MYBSWAP16(v) (v>>8)|(v<<8)
#ifdef __builtin_bswap16
printf("bswap16 is defined : %04x\n", __builtin_bswap16(0x1234));
#else
printf("bswap16 is not defined : %04x\n", MYBSWAP16(0x1234) );
#endif
...
was not successful, as __builtin_bswap16/32/64 are always evaluated to be undefined. Is there any way to make it work automatically within the C source, or is the only way to manually define constants in the Makefile, e.g. HAVE_BSWAP and pass them via -D option?
Please note that my question is not necessarily specific to __builtin_bswap, I'm looking for a general way to detect if the certain functions are available.
Unavailability of __builtin_bswap16 is a gcc bug which was fixed in gcc 4.8.
Sincw it is missing from some versions of gcc you can always add it to your code yourself :
static inline unsigned short __builtin_bswap16(unsigned short a)
{
return (a<<8)|(a>>8);
}
Optimizations such as constant propagation are possible across functions within the same compilation unit (ie. same file).
For example :
int f(int x)
{
return 3 + x;
}
int main(void)
{
printf("%d\n", 1 + f(4));
return 0;
}
In that example, I think that a sufficiently smart compiler can propagate the '4' constant to the function 'f', solving the integer arithmetic with
the other constant '3', and propagates back the result value thus folding everything to the final value '8'.
(Well, correct me if I'm wrong..)
However, what is happening if the function 'f' is in another compilation unit. Since they both units are compiled separately, the compiler can't
optimize that way.
Does it mean that optimizations are only possible within the same compilation unit, or is there some form late optimizations performed of link-time?
Both MSVC (since 8.0: VS2005) and GCC (since 4.5) support the concept.
MSVC uses a compiler switch /GL and linker switch /LTCG. Documentation
GCC must have it enabled and uses the -flto, -fwhole-program, -fwhopr, and/or -combine to the same effect. Documentation (search for the options in your browser)
The "problem" is that every compilation unit (source file) (and in the case of MSVC every library) needs to be compiled with this, so you can't use old binary object files compiled without it. It also makes debugging harder, because the optimizer is a lot more aggressive and unpredictable.
Clang compiles to LLVM IR, and the LLVM linker performs whole-program optimization when it produces a native binary.
Microsoft Visual Studio supports WPO (whole program optimization) enabled by ltcg switch (link-time code generation).
It causes several other problems which I don't remember right now, and is preferred off by many developers.
Yes, for the Visual C++ compiler in Visual Studio, this is known as Whole Program Optimization:
Whole program optimization allows the
compiler to perform optimizations with
information on all modules in the
program. Without whole program
optimization, optimizations are
performed on a per module (compiland)
basis
With information on all modules, the
compiler can:
Optimize the use of registers across
function boundaries.
Do a better job of tracking
modifications to global data, allowing
a reduction in the number of loads and
stores.
Do a better job of tracking the
possible set of items modified by a
pointer dereference, reducing the
numbers of loads and stores.
Inline a function in a module even
when the function is defined in
another module.
GCC 4.5 introduced link-time optimization. AFAIK, it only works on x86 and x64 targets.
Compiling a kernel module on 32-Bit Linux kernel results in
"__udivdi3" [mymodule.ko] undefined!
"__umoddi3" [mymodule.ko] undefined!
Everything is fine on 64-bit systems. As far as I know, the reason for this is that 64-bit integer division and modulo are not supported inside a 32-bit Linux kernel.
How can I find the code issuing the 64-bit operations. They are hard to find manually because I cannot easily check if an "/" is 32-bit wide or 64-bit wide. If "normal" functions are undefined, I can grep them, but this is not possible here. Is there another good way to search the references? Some kind of "machine code grep"?
The module consists of some thousand lines of code. I can really not check every line manually.
First, you can do 64 bit division by using the do_div macro. (note the prototype is uint32_t do_div(uint64_t dividend, uint32_t divisor) and that "dividend" may be evaluated multiple times.
{
unsigned long long int x = 6;
unsigned long int y = 4;
unsigned long int rem;
rem = do_div(x, y);
/* x now contains the result of x/y */
}
Additionally, you should be able to either find usage of long long int (or uint64_t) types in your code, or alternately, you can build your module with the -g flag and use objdump -S to get a source annotated disassembly.
note: this applies to 2.6 kernels, I have not checked usage for anything lower
Actually, 64-bit integer divison and modulo are supported within a 32-bit Linux kernel; however, you must use the correct macros to do so (which ones depend on your kernel version, since recently new better ones were created IIRC). The macros will do the correct thing in the most efficient way for whichever architecture you are compiling for.
The easiest way to find where they are being used is (as mentioned in #shodanex's answer) to generate the assembly code; IIRC, the way to do so is something like make directory/module.s (together with whatever parameters you already have to pass to make). The next easiest way is to disassemble the .o file (with something like objdump --disassemble). Both ways will give you the functions where the calls are being generated (and, if you know how to read assembly, a general idea of where within the function the division is taking place).
After compilation stage, you should be able to get some documented assembly, and see were those function are called. Try to mess with CFLAGS and add the -S flags.
Compilation should stop at the assembly stage. You can then grep for the offending function call in the assembly file.
If I include <stdlib.h> or <stdio.h> in a C program, I don't have to link these when compiling, but I do have to link to <math.h>, using -lm with GCC, for example:
gcc test.c -o test -lm
What is the reason for this? Why do I have to explicitly link the math library, but not the other libraries?
The functions in stdlib.h and stdio.h have implementations in libc.so (or libc.a for static linking), which is linked into your executable by default (as if -lc were specified). GCC can be instructed to avoid this automatic link with the -nostdlib or -nodefaultlibs options.
The math functions in math.h have implementations in libm.so (or libm.a for static linking), and libm is not linked in by default. There are historical reasons for this libm/libc split, none of them very convincing.
Interestingly, the C++ runtime libstdc++ requires libm, so if you compile a C++ program with GCC (g++), you will automatically get libm linked in.
Remember that C is an old language and that FPUs are a relatively recent phenomenon. I first saw C on 8-bit processors where it was a lot of work to do even 32-bit integer arithmetic. Many of these implementations didn't even have a floating point math library available!
Even on the first 68000 machines (Mac, Atari ST, Amiga), floating point coprocessors were often expensive add-ons.
To do all that floating point math, you needed a pretty sizable library. And the math was going to be slow. So you rarely used floats. You tried to do everything with integers or scaled integers. When you had to include math.h, you gritted your teeth. Often, you'd write your own approximations and lookup tables to avoid it.
Trade-offs existed for a long time. Sometimes there were competing math packages called "fastmath" or such. What's the best solution for math? Really accurate but slow stuff? Inaccurate but fast? Big tables for trig functions? It wasn't until coprocessors were guaranteed to be in the computer that most implementations became obvious. I imagine that there's some programmer out there somewhere right now, working on an embedded chip, trying to decide whether to bring in the math library to handle some math problem.
That's why math wasn't standard. Many or maybe most programs didn't use a single float. If FPUs had always been around and floats and doubles were always cheap to operate on, no doubt there would have been a "stdmath".
Because of ridiculous historical practice that nobody is willing to fix. Consolidating all of the functions required by C and POSIX into a single library file would not only avoid this question getting asked over and over, but would also save a significant amount of time and memory when dynamic linking, since each .so file linked requires the filesystem operations to locate and find it, and a few pages for its static variables, relocations, etc.
An implementation where all functions are in one library and the -lm, -lpthread, -lrt, etc. options are all no-ops (or link to empty .a files) is perfectly POSIX conformant and certainly preferable.
Note: I'm talking about POSIX because C itself does not specify anything about how the compiler is invoked. Thus you can just treat gcc -std=c99 -lm as the implementation-specific way the compiler must be invoked for conformant behavior.
Because time() and some other functions are builtin defined in the C library (libc) itself and GCC always links to libc unless you use the -ffreestanding compile option. However math functions live in libm which is not implicitly linked by gcc.
An explanation is given here:
So if your program is using math functions and including math.h, then you need to explicitly link the math library by passing the -lm flag. The reason for this particular separation is that mathematicians are very picky about the way their math is being computed and they may want to use their own implementation of the math functions instead of the standard implementation. If the math functions were lumped into libc.a it wouldn't be possible to do that.
[Edit]
I'm not sure I agree with this, though. If you have a library which provides, say, sqrt(), and you pass it before the standard library, a Unix linker will take your version, right?
There's a thorough discussion of linking to external libraries in An Introduction to GCC - Linking with external libraries. If a library is a member of the standard libraries (like stdio), then you don't need to specify to the compiler (really the linker) to link them.
After reading some of the other answers and comments, I think the libc.a reference and the libm reference that it links to both have a lot to say about why the two are separate.
Note that many of the functions in 'libm.a' (the math library) are defined in 'math.h' but are not present in libc.a. Some are, which may get confusing, but the rule of thumb is this--the C library contains those functions that ANSI dictates must exist, so that you don't need the -lm if you only use ANSI functions. In contrast, `libm.a' contains more functions and supports additional functionality such as the matherr call-back and compliance to several alternative standards of behavior in case of FP errors. See section libm, for more details.
As ephemient said, the C library libc is linked by default and this library contains the implementations of stdlib.h, stdio.h and several other standard header files. Just to add to it, according to "An Introduction to GCC" the linker command for a basic "Hello World" program in C is as below:
ld -dynamic-linker /lib/ld-linux.so.2 /usr/lib/crt1.o
/usr/lib/crti.o /usr/libgcc-lib /i686/3.3.1/crtbegin.o
-L/usr/lib/gcc-lib/i686/3.3.1 hello.o -lgcc -lgcc_eh -lc
-lgcc -lgcc_eh /usr/lib/gcc-lib/i686/3.3.1/crtend.o /usr/lib/crtn.o
Notice the option -lc in the third line that links the C library.
If I put stdlib.h or stdio.h, I don't have to link those but I have to link when I compile:
stdlib.h, stdio.h are the header files. You include them for your convenience. They only forecast what symbols will become available if you link in the proper library. The implementations are in the library files, that's where the functions really live.
Including math.h is only the first step to gaining access to all the math functions.
Also, you don't have to link against libm if you don't use it's functions, even if you do a #include <math.h> which is only an informational step for you, for the compiler about the symbols.
stdlib.h, stdio.h refer to functions available in libc, which happens to be always linked in so that the user doesn't have to do it himself.
It's a bug. You shouldn't have to explicitly specify -lm any more. Perhaps if enough people complain about it, it will be fixed. (I don't seriously believe this, as the maintainers who are perpetuating the distinction are evidently very stubborn, but I can hope.)
I think it's kind of arbitrary. You have to draw a line somewhere (which libraries are default and which need to be specified).
It gives you the opportunity to replace it with a different one that has the same functions, but I don't think it's very common to do so.
I think GCC does this to maintain backwards compatibility with the original cc executable. My guess for why cc does this is because of build time -- cc was written for machines with far less power than we have now. A lot of programs don't have any floating-point math, and they probably took every library that wasn't commonly used out of the default. I'm guessing that the build time of the Unix OS and the tools that go along with it were the driving force.
I would guess that it is a way to make applications which don't use it at all perform slightly better. Here's my thinking on this.
x86 OSes (and I imagine others) need to store FPU state on context switch. However, most OSes only bother to save/restore this state after the app attempts to use the FPU for the first time.
In addition to this, there is probably some basic code in the math library which will set the FPU to a sane base state when the library is loaded.
So, if you don't link in any math code at all, none of this will happen, therefore the OS doesn't have to save/restore any FPU state at all, making context switches slightly more efficient.
Just a guess though.
The same base premise still applies to non-FPU cases (the premise being that it was to make apps which didn't make use libm perform slightly better).
For example, if there is a soft-FPU which was likely in the early days of C. Then having libm separate could prevent a lot of large (and slow if it was used) code from unnecessarily being linked in.
In addition, if there is only static linking available, then a similar argument applies that it would keep executable sizes and compile times down.
stdio is part of the standard C library which, by default, GCC will link against.
The math function implementations are in a separate libm file that is not linked to by default, so you have to specify it -lm. By the way, there is no relation between those header files and library files.
All libraries like stdio.h and stdlib.h have their implementation in libc.so or libc.a and get linked by the linker by default. The libraries for libc.so are automatically linked while compiling and is included in the executable file.
But math.h has its implementations in libm.so or libm.a which is separate from libc.so. It does not get linked by default and you have to manually link it while compiling your program in GCC by using the -lm flag.
The GNU GCC team designed it to be separate from the other header files, while the other header files get linked by default, but math.h file doesn't.
Here read the item number 14.3, you could read it all if you wish:
Reason why math.h is needs to be linked
Look at this article: Why do we have to link math.h in GCC?
Have a look at the usage:
Using the library
Note that -lm may not always need to be specified even if you use some C math functions.
For example, the following simple program:
#include <stdio.h>
#include <math.h>
int main() {
printf("output: %f\n", sqrt(2.0));
return 0;
}
can be compiled and run successfully with the following command:
gcc test.c -o test
It was tested on GCC 7.5.0 (on Ubuntu 16.04) and GCC 4.8.0 (on CentOS 7).
The post here gives some explanations:
The math functions you call are implemented by compiler built-in functions
See also:
Other Built-in Functions Provided by GCC
How to get the gcc compiler to not optimize a standard library function call like printf?