I have used dlsym to create a malloc/calloc wrapper in the efence code as to able to able to access the libc malloc (occassionally apart from efence malloc/calloc). Now when i link it, and run, it gives following error: "RTLD_NEXT used in code not dynamically loaded"
bash-3.2# /tool/devel/usr/bin/gcc -g -L/tool/devel/usr/lib/ efence_time_interval_measurement_test.c -o dev.out -lefence -ldl -lpthread
bash-3.2# export LD_LIBRARY_PATH=/tool/devel/usr/lib/
bash-3.2# ./dev.out
eFence: could not resolve 'calloc' in 'libc.so': RTLD_NEXT used in code not dynamically loaded
Now, if i use "libefence.a" it is happening like this:
bash-3.2# /tool/devel/usr/bin/gcc -g -L/tool/devel/usr/lib/ -static
efence_time_interval_measurement_test.c -o dev.out -lefence -ldl -lpthread
/tool/devel/usr/lib//libefence.a(page.o): In function `stringErrorReport':
/home/raj/eFence/BUILD/electric-fence-2.1.13/page.c:50: warning: `sys_errlist' is deprecated; use `strerror' or `strerror_r' instead
/home/raj/eFence/BUILD/electric-fence-2.1.13/page.c:50: warning: `sys_nerr' is deprecated; use `strerror' or `strerror_r' instead
/tool/devel/usr/lib//libc.a(malloc.o): In function `__libc_free':
/home/rpmuser/rpmdir/BUILD/glibc-2.9/malloc/malloc.c:3595: multiple definition of `free'
/tool/devel/usr/lib//libefence.a(efence.o):/home/raj/eFence/BUILD/electric-fence-2.1.13/efence.c:790: first defined here
/tool/devel/usr/lib//libc.a(malloc.o): In function `__libc_malloc':
/home/rpmuser/rpmdir/BUILD/glibc-2.9/malloc/malloc.c:3551: multiple definition of `malloc'
/tool/devel/usr/lib//libefence.a(efence.o):/home/raj/eFence/BUILD/electric-fence-2.1.13/efence.c:994: first defined here
/tool/devel/usr/lib//libc.a(malloc.o): In function `__libc_realloc':
/home/rpmuser/rpmdir/BUILD/glibc-2.9/malloc/malloc.c:3647: multiple definition of `realloc'
/tool/devel/usr/lib//libefence.a(efence.o):/home/raj/eFence/BUILD/electric-fence-2.1.13/efence.c:916: first defined here
Please help me. Is there any problem in linking?
NO ONE IN STACK OVERFLOW WHO CAN RESOLVE THIS
The problem is with your question, not with us ;-)
First off, efence is most likely the wrong tool to use on a Linux system. For most bugs that efence can find, Valgrind can find them and describe them to you (so you could fix them) much more accurately. The only good reason for you to use efence is if your application runs for many hours, and Valgrind is too slow.
Second, efence is not intended to work with static linking, so the errors you get with -static flag are not at all surprising.
Last, you didn't tell us what libc is installed on your system (in /lib), and what libraries are present in /tool/devel/usr/lib/. It is exceedingly likely that there is libc.so.6 present in /usr/devel/usr/lib, and that its version does not match the one installed in /lib.
That would explain the RTLD_NEXT used in code not dynamically loaded error. The problem is that glibc consists of multiple binaries, which all must match exactly. If the system has e.g. libc-2.7 installed, then you are using /lib/ld-linux.so.2 from glibc-2.7 (the dynamic loader is hard-coded into every executable and is not affected by environment variables), and mixing it with libc.so.6 from glibc-2.9. The usual result of doing this is a SIGSEGV, weird unresolved symbol errors, and other errors that make no sense.
Related
So I am trying to identify some memory corruption issues in a large codebase. To begin with, I first built the codebase as it is with whatever existing makefile configuration was set up already. It worked fine and the binaries were generated. Now I added -g -fsanitize=address -fno-omit-frame-pointer -fno-common in compile flags and -fsanitize=address (also tried with -lasan if that makes any difference) in link flags to compile the codebase with ASan. But now I am getting multiple declaration errors at link time. I am clueless at this point. What could be the reason for this? If there were multiple definitions, then shouldn't the same error pop up when building without ASan flags too? I couldn't even find anything related to this in the ASan docs.
I cannot share the exact error trace, but it pretty much looks like this:
path/to/file/hdr.h:132: multiple definition of `myDataTable_type'
path/to/file/hdr.h:132: first defined here
path/to/obj/file/obj.o: multiple definition of `__odr_asan.myDataTable_type'
gcc --version
gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1)
Most likely you have something like
XXX myDataTable_type;
in one of the headers. When this header is included into multiple source files you'll get multiple definitions error under -fno-common.
To resolve this you could either remove -fno-common from CFLAGS but then Asan will not be able to detect buffer overflows in some static arrays. Better yet, just get rid of multiple definitions by changing declaration in header to
extern XXX myDataTable_type;
and adding
XXX myDataTable_type;
to one of the source files (to guarantee single definition).
I'm running OS X 10.12 and I'm developing a basic text-based operating system. I have developed a boot loader and that seems to be running fine. My only problem is that when I attempt to compile my kernel into pure binary, the linker won't work. I have done some research and I think that this is because of the fact OS X runs the Darwin linker and not the GNU linker. Because of this, I have downloaded and installed the GNU binutils. However, it still won't work...
Here is my kernel:
void main() {
// Create pointer to a character and point it to the first cell of video
// memory (i.e. the top-left)
char* video_memory = (char*) 0xb8000;
// At that address, put an x
*video_memory = 'x';
}
And this is when I attempt to compile it:
Hazims-MacBook-Pro:32 bit root# gcc -ffreestanding -c kernel.c -o kernel.o
Hazims-MacBook-Pro:32 bit root# ld -o kernel.bin -T text 0x1000 kernel.o --oformat binary
ld: unknown option: -T
Hazims-MacBook-Pro:32 bit root#
I would love to know how to solve this issue. Thank you for your time.
-T is a gcc compiler flag, not a linker flag. Have a look at this:
With these components you can now actually build the final kernel. We use the compiler as the linker as it allows it greater control over the link process. Note that if your kernel is written in C++, you should use the C++ compiler instead.
You can then link your kernel using:
i686-elf-gcc -T linker.ld -o myos.bin -ffreestanding -O2 -nostdlib boot.o kernel.o -lgcc
Note: Some tutorials suggest linking with i686-elf-ld rather than the compiler, however this prevents the compiler from performing various tasks during linking.
The file myos.bin is now your kernel (all other files are no longer needed). Note that we are linking against libgcc, which implements various runtime routines that your cross-compiler depends on. Leaving it out will give you problems in the future. If you did not build and install libgcc as part of your cross-compiler, you should go back now and build a cross-compiler with libgcc. The compiler depends on this library and will use it regardless of whether you provide it or not.
This is all taken directly from OSDev, which documents the entire process, including a bare-bones kernel, very clearly.
You're correct in that you probably want binutils for this especially if you're coding baremetal; while clang as is purports to be a cross compiler it's far from optimal or usable here, for various reasons. noticing you're developing on ARM I infer; you want this.
https://developer.arm.com/open-source/gnu-toolchain/gnu-rm
Aside from the fact that gcc does this thing better than clang markedly, there's also the issue that ld does not build on OS X from the binutils package; it in some configurations silently fails so you may in fact never have actually installed it despite watching libiberty etc build, it will even go through the motions of compiling the source of that target sometimes and just refuse to link it... to the fellow with the lousy tone blaming OP, if you had relevant experience ie ever had built this under this condition you would know that is patently obnoxious. it'd be nice if you'd refrain from discouraging people from asking legitimate questions.
In the CXXfilt package they mumble about apple-darwin not being a target; try changing FAKE_TARGET to instead of mn10003000-whatever or whatever they used, to apple-rhapsody some time.
You're still in way better shape just building them from current if you say need to strip relocations from something or want to work on restoring static linkage to the system. which is missing by default from that clang installation as well...anyhow it's not really that ld couldn't work with macho, it's all there, codewise in fact...that i am sure of
Regarding locating things in memory, you may want to refer to a linker script
http://svn.screwjackllc.com/?p=noid.git;a=blob_plain;f=new_mbed_bs.link_script.ld
As i have some code in there that will directly place things in memory, rather than doing it on command line it is more reproducible to go with the linker script. it's a little complex but what it is doing is setting up a couple of regions of memory to be used with my memory allocators, you can use malloc, but you should prefer not to use actual malloc; dynamic memory is fine when it isn't dynamic...heh...
The script also sets flags for the stack and heap locations, although they are just markers, not loaded til go time, they actually get placed, stack and heap, by the startup code, which is in assembly and rather readable and well commented (hard to believe, i know)... neat trick, you have some persistence to volatile memory, so i set aside a very tiny bit to flip and you can do things like have it control what bootloader to run on the next power cycle. again you are 100% correct regarding the linker; seems to be you are headed the right direction. incidentally another way you can modify objects prior to loading them , and preload things in memory, similar to this method, well there are a ton of ways, but, check out objcopy and objdump...you can use gdb to dump srecs of structures in memory, note the address, and then before linking but after assembly use dd to insert the records you extracted with gdb back in to extracted sections..is one of my favorite ways just because is smartass route :D also, if you are tight on memory ever and need to precalculate constants it's one way to optimize things...that way is actually closer to what ld is doing, just doing it by hand... probably path of least resistance on this now though is linker script.
new to using C
Header files for libraries like stdlib do not contain the actual implementation code for the functions they provide access to. I understand that the actual source text for libraries like this aren't needed to compile, but how does this work specifically? Are the implementation details for these libraries contained within the compiler?
When you use a function like printf(), including the header file essentially pastes in code for the declaration of the function, but normally the implementation code would need to be available as well.
What form is it stored in? (and where?) Is this compiler specific? Would it be possible to write custom code and reference it in this way without modifying the behavior of the compiler?
I've been searching around and found some info that is relevant but nothing specific. This could be related to not formulating the question well. Thanks.
When you link a program, the compiler will implicitly add some extra libraries to your program:
$ ls
main.c
$ cc -c main.c
$ cc main.o
$ ls
main.c main.o a.out
You can discover the extra libraries a program uses with ldd. Here, there are three libraries linked into the program, and I didn't ask for any of them:
$ ldd a.out
linux-vdso.so => (0x00...)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00...)
/lib64/ld-linux-x86-64.so.2 (0x00...)
So, what happens if we link without these libraries? That's easy enough, just use the linker (ld) directly, instead of calling it through cc. When you use ld, it doesn't give you these extra libraries, so you get an error:
$ ld main.o
Undefined symbols:
"_printf", referenced from:
_main in main.o
The implementation for printf() is stored in the standard C library, which is usually just another library on your system... the only difference is that it gets automatically included into your program when you compile C.
You can use nm to find out what symbols are in a library, so I can use it to find printf() in libc:
$ nm -D /lib/x86_64-linux-gnu/libc-2.13.so | grep printf
...
000000000004e4b0 T printf
...
So, now that we know that libc has printf(), we can use -lc to tell the linker to include libc, and that will get rid of the errors about printf() being missing:
$ ld main.o -lc
There might be some other bits missing, and that's why we use cc to link our programs instead of ld: cc gives us all the default libraries.
When you compile a file you only need to promise the compiler that you have certain functions and symbols. A function call is in the compiled into a call [some_address]
The compiler will compile each C-file into object files that just have place holders for calls to functions declared in the headers. That is [some_address] does not need to be known at this point.
A number of oject files can be collected into what is known as a library.
After that it is the linkers job to look through all object files and libraries it know of and find out what the real value of all unknown [some_address] is and translate the call to, e.g. call 0x1234 if the particular function you are calling starts at 0x1234 (or it might be a relative offset from the current program pointer.
Stdlib and other library functions are implemented in an object library. A library is a collection of code that is linked with your program. By default C programs are linked against the stdlib library, which is usually provided by the operating system. Most modern operating systems use a dynamical linker. That is, your program is not linked against the library until it is executed. When it is being loaded, the linker-loader combines your code and the library code in your program's address space. You code and then make a call to the printf() code that is located in that library.
Usually a header file contains only a function prototype while the implementation is either in a separate source file or a precompiled library in the case of stdlib (and other libraries, both shipped with a compiler or available separately) the precompiled library gets linked at the end of the compilation process. (There's also a distinction between static and dynamic libraries, but I won't go into detail about that)
The implementation of standard libraries (which are shipped with a compiler) are usually compiler specific (there is a standard describing which functions have to be in a library, but the compiler programmer can decide how exactly he implements them) and it is (in theory) possible to exchange these libraries with some of your own without modifying the behaviour of the compiler (though not recommended as you would have to rewrite the entire library in order to ensure that all functions are contained).
I'm running out of good ideas on how to crack this bug. I have 1000 lines of code that crashes every 2 or 3 runs. It is currently a prototype command line application written in C. An issue is that it's proprietary and I cannot give you the source, but I'd be happy to send a debug compiled executable to any brave soul on a Debian Squeeze x86_64 machine.
Here is what I got so far:
When I run it in GDB, it always complete successfully.
When I run it in Valgrind, it always complete successfully.
The issue seems to emanate from a recursive function call that is very basic. In an effort to pin point the error in this recursive function I wrote the same function in a separate application. It always completes successfully.
I built my own gcc 4.7.1 compiler, compiled my code with it and I'm still getting the same behavior.
FTped my application to another machine to eliminate the risk of HW issues and I still get the same behavior.
FTped my source code to another machine to eliminate the risk of a corrupt build environment and I still get the same behavior.
The application is single threaded and does no signal handling that might cause race conditions. I memset(,0,) all large objects
There are no exotic dependencies, the ldd follows below.
ldd gives me this:
ldd tst
linux-vdso.so.1 => (0x00007fff08bf0000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fe8c65cd000)
libm.so.6 => /lib/libm.so.6 (0x00007fe8c634b000)
libc.so.6 => /lib/libc.so.6 (0x00007fe8c5fe8000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe8c67fc000)
Are there any tools out there that could help me?
What would be your next step if you were in my position?
Thanks!
This is what got me in the right direction -Wextra I already used -Wall.
THANKS!!! This was really driving me crazy.
I suggested in comments :
to compile with -Wall -Wextra and improve the source code till no warnings are given;
to compile with both -g and -O; this is helpful to inspect dumped core files with gdb (you may want to set a big enough coredump size limit with e.g. ulimit bash builtin)
to show your code to a colleague and explain the issue?
to use ltrace or strace
Apparently -Wextra was helpful. It would be nice to understand why and how.
BTW, for larger programs, you could even add your own warnings to GCC by extending it with MELT; this may take days and is worthwhile mostly in big projects.
In this case, i think that you have some memory problems (see the output of valgrind carefully), cause GDB and valgrind change the original program by adding some memory tracking functions (so your original addresses are changed). You can compile with -ggdb option and set coredump (ulimit -c unlimited) and then trying to analyze what's going on. This link may help you:
http://en.wikipedia.org/wiki/Unusual_software_bug
Regards.
Background:
I am working on a project written in a mix of C and Fortran 77 and now need to link the LAPACK/BLAS libraries to the project (all in a Linux environment). The LAPACK in question is version 3.2.1 (including BLAS) from netlib.org. The libraries were compiled using the top level Makefile (make lapacklib and make blaslib).
Problem:
During linking, error messages claimed that certain (not all) BLAS-routines called from LAPACK-routines were undefined. This gave me some headache but the problem was eventually solved when (in the Makefile) the order of appearance of the libraries to be linked was changed.
Code:
In the following, (a) gives errors while (b) does not. The linking is performed by (c).
(a) LIBS = $(LAPACK)/blas_LINUX.a $(LAPACK)/lapack_LINUX.a
(b) LIBS = $(LAPACK)/lapack_LINUX.a $(LAPACK)/blas_LINUX.a
(c) gcc -Wall -O -o $# project.o project.a $(LIBS)
Question:
What could be the reason for the undefined references of only some routines and what makes the order of appearance relevant?
The LAPACK library needs stuff from BLAS, and the linker searches from left to right. So, putting BLAS after LAPACK (option (b)), worked.
If you want it to always work, regardless of the order, you can use linker groups:
-Wl,--start-group $(LAPACK)/blas_LINUX.a $(LAPACK)/lapack_LINUX.a -Wl,--end-group
That tells the linker to loop through the libraries until all symbols get resolved (or until it notices that looping again won't help).
Typically one always puts the "more fundamental/basic" library to the right of the "less fundamental/basic" - ie, the linker will look to the right of a file for the definition of a function appearing in said file. This is supposedly not necessary any more with modern linkers, but it's always a good idea (as in your case). I'm not sure why it only mattered with several routines.
Is clapack used as a LAPACK implementation? If no you can try to use it.