After a long sequence of debugging I've narrowed my problem down to one file. And the problem is that the file compiles differently in two different directories, when everything else is the same.
I'm using CodeSourcery's arm gcc compiler (gcc version 4.3.3, Sourcery G++ Lite 2009q1-161) to compile a simple file. I was using it in one module with no issues and then I copied it to another module to use there. When it compiles, the object file is significantly different. The command line to compile the two files is identical (I used the linux history to make sure), and the 3 include files are also identical copies (checked with diff).
I did a binary compare on the two object files and they have a lot of individual byte differences scattered around. I did an objdump -D of both and compared them and there are a lot of differences. Here is dump1, dump2, and the diff. The command line is "
arm-none-eabi-gcc --std=gnu99 -Wall -O3 -g3 -ggdb -Wextra -Wno-unused -c crc.c -o crc.o".
How is this possible? I've also compiled with -S instead of -c and looked at the assembler output and that's identical except for the directory path. So how can the object file be different?
My real problem is that when I try to link the object file for dump2 into my program, I get undefined reference errors, so something in the object is wrong, whereas the object for dump1 gets no such errors and links fine.
For large scale software, there are many implementations are doing hashing on pointers. This is one major reason that cause result randomization. Usually if the program logic is correct, the order of some internal data structures could be different which is not harmful in most cases.
And also, don't compare the 'objdump -D' output, since your are compiling the code from different directory, the string table, symbol table, DWARF or eh_frame should be different. You will certainly get lots of diff lines.
The only comparison that makes sense is to compare the output of 'objdump -d' which only takes care of the text section. If text section is same(similar) then it can be considered as identical.
Most likely your file picks up different include files. This this the most likely reason.
Check that your include paths are exactly the same, paths in the include statements. They may point to different directories. C and C++ has a feature that when you #include abcd.h it tries to load abcd.h from the directory of the calling file. Check this.
Related
Im a newcommer to Linux and the gcc commands. I was reading the
gcc documentation particularly about the -o flag where it mentions the following:
Though -o names only the primary output, it also affects the naming of
auxiliary and dump outputs. See the examples below. Unless overridden,
both auxiliary outputs and dump outputs are placed in the same
directory as the primary output. In auxiliary outputs, the suffix of
the input file is replaced with that of the ...
They mention it quite a lot following this paragraph but don't explain it. I've skimmed the document and also looked online but haven't found any satisfactory explanation. If someone could provide me some explanation or even link me to some resources where I can learn about these terms it would be greatly appreciated. Thanks!
-o file
Place the output in file. This applies regardless of the type of output produced, whether it is an executable file, an object file, an assembler file or preprocessed C code.
Since only one output file can be specified, it makes no sense to use -o when compiling more than one input file, unless you want to output an executable file.
If -o is not specified, the default behavior is to produce an executable file named a.out, an object file for source.suffix named source.o, its assembler file in source.s, and all C source code preprocessed on standard output.
source: http://www.linuxcertif.com/man/1/gcc/
hope it will be useful
I'm writing some code for a project that uses a Kinetis processor. These processors have a Flash Config Field which is stored in flash at a particular address. If the wrong value gets written to that field, you can lock yourself out of your chip for good.
My code for this consists of a packed struct, an instance that specifies the .flashConfig section, some static asserts to ensure the struct is the required size, and the #define that gets written to the FSEC field (the important one) is as expected. Then in the linker script I have that section stored in the correct bit of flash. Additionally I have an ASSERT to see if I have the correct amount of data in that section.
This is all pretty good, but I'm still nervous (I've seen these chips gets themselves locked up, on several occasions now). What I want to do is add an extra assert to the linker script, something like:
ASSERT(BYTE_AT(0x40C) == 0xBE);
Is that possible?
I considered using objdump / objcopy to get dump this from a .bin in a post build step. However I'm building this on windows, so no grep / awk which would be nice and easy. Other people will also have to build this, so I don't want to rely on cygwin being installed or what not. Plus this is a little more removed than the linker, and therefore could easily be missed if someone removes the post_build script.
I don't want to rely on cygwin being installed or what not.
Write a C program that performs the same check objdump and grep would have done.
Plus this is a little more removed than the linker, and therefore could easily be missed if someone removes the post_build script.
Make the verification program invoke the linker, and then verify the result. That is, instead of
${LD} -o foo.bin ${LDFLAGS} ${OBJS} && ./post_build foo.bin
do this:
./build_and_verify -o foo.bin ${LDFLAGS} ${OBJS}
Does the gcc output of the object file (C language) vary between compilations? There is no time-specific information, no change in compilation options or the source code. No change in linked libraries, environmental variables either. This is a VxWorks MIPS64 cross compiler, if that helps. I personally think it shouldn't change. But I observe that sometimes randomly, the instructions generated changes. I don't know what's the reason. Can anyone throw some light on this?
How is this built? For example, if I built the very same Linux kernel, it includes a counter that is incremented each build. GCC has options to use profiler information to guide code generation, if the profiling information changes, so will the code.
What did you analyze? The generated assembly, an objdump of object files or the executable? How did you compare the different versions? Are you sure you looked at executable code, not compiler/assembler/linker timestamps?
Did anything change in the environment? New libraries (and header files/declarations/macro definitions!)? New compiler, linker? New kernel (yes, some header files originate with the kernel source and are shipped with it)?
Any changes in environment variables (another user doing the compiling, different machine, different hookup to the net gives a different IP address that makes it's way into the build)?
I'd try tracing the build process in detail (run a build and capture the output in a file, and do so again; compare those).
Completely mystified...
I had a similar problem with g++. Pre 4.3 versions produced exactly the same object files each time. With 4.3 (and later?) some of the mangled symbol names are different for each run - even without -g or other recordings. Perhaps the use a time stamp or random number (I hope not). Obviously some of those symbols make it into the .o symbol table and you get a difference.
Stripping the object file(s) makes them equal again (wrt. binary comparison).
g++ -c file.C ; strip file.o; cmp file.o origfile.o
Why should it vary? It is the same result always. Try this:
for i in `seq 1000`; do gcc 1.c; md5sum a.out; done | sort | uniq | wc -l
The answer is always 1. Replace 1.c and a.out to suit your needs.
The above counts how many different executables are generated by gcc when compiling the same source for 1000 times.
I've found that in at least some environments, the same source may yield a different executable if the source tree for the subsequent build is located in a different directory. Example:
Checkout a pristine copy of your project to dir1. Do a full rebuild from scratch.
Then, with the same user on the same machine, checkout the same exact copy of your source code to dir2 (dir1 != dir2). Do another full rebuild from scratch.
These builds are minutes apart, with no change in the toolchain or any 3rd party libs or code. Binary comparison of source code is the same. However, the executable in dir1 has different md5sum than the executable in dir2.
If I compare the different executables in BeyondCompare's hex editor, the difference is not just some tiny section that could plausibly be a timestamp.
I do get the same executable if I build in dir1, then rebuild again in dir1. Same if I keep building the same source over and over from dir2.
My only guess is that some sort of absolute paths of the include hierarchy are embedded in the executable.
My gcc sometimes produces different code for exactly the same Input. The output object files differ in exactly one byte.
Sometimes this causes linker Errors, because one possible object file is invalid. Recompiling another version usually fixes the linker error.
The gcc Version is 4.3.4 on Suse Linux Enterprise.
The gcc Parameters are:
cc -std=c++0x -Wall -fno-builtin -march=native -g -I<path1> -I<path2> -I<path3> -o obj/file.o -c file.cpp
If someone experiences the same effect, then please let me know.
I'm trying to write a simple syntax checker for C code using the frontend available in libclang. Due to deployment concerns, I need to be able to statically link all the libraries in libclang, and not pass around the .so file that has all the libraries.
I'm building clang/llvm from source, and in llvm/Release+Asserts/lib I have a bunch of .a files that I think I should be able to use, but it never seems to work (the linker spews out thousands of errors about missing symbols). However, when I compile it using the libclang.so also present in that directory as follows:
clang main.c -o bin/dlc -I../llvm/tools/clang/include -L../llvm/Release+Asserts/lib/ -lclang
Everything seems to work well.
What is the minimum set of .a files I need to include to make this work? I've tried including absolutely all of the .a files in the build output directory, with them provided to clang/gcc in different orders, without any success. I only need the functions mentioned in libclang's Index.h, but there don't seem to be any resources or documentation on what the various libclang*.a files are for. It would be very helpful to know which files libclang.so pulls in.
The following is supposed to work, as long the whole project has all static libraries (I counted 116 in my Release/lib directory).
clang main.c -o bin/dlc -I../llvm/tools/clang/include ../llvm/Release/lib/*.a
[edit: clang main.c -o bin/dlc -I../llvm/tools/clang/include ../llvm/Release/lib/libclang.a ../llvm/Release/lib/*.a]
Note that the output binary is not static, so you don't need any -static flag for gcc or ld, if you're using this syntax.
If that doesn't work you might need to list the libraries in order: if some library requires a function available in another library, then it may be necessary to list it first in the command line. See comments about link order at:
http://gcc.gnu.org/onlinedocs/gcc-4.7.2/gcc/Link-Options.html#Link-Options
If one builds static libraries in one's build scripts and one wants to use those static libraries in linking the final executable, the order one mentions the .a files is important:
g++ main.o hw.a gui.a -o executable
If gui.a uses something defined in hw.a the link will fail, because at the time hw.a is processed, the linker doesn't yet know that the definition is needed later, and doesn't include it in the being.generated executable. Manually fiddling around with the linker line is not practical, so a solution is to use --start-group and --end-group which makes the linker run twice through the libraries until no undefined symbols are found anymore.
g++ main.o -Wl,--start-group hw.a gui.a -Wl,--end-group -o executable
However the GNU ld manual says
Using this option has a significant performance cost. It is best to use it only when there are unavoidable circular references between two or more archives.
So I thought that it may be better to take all .a files and put them together into one .a file with an index (-s option of GNU ar) which says in what order the files need to be linked. Then one gives only that one .a file to g++.
But I wonder whether that's faster or slower than using the group commands. And are there any problems with that approach? I also wonder, is there better way to solve these interdependency problems?
EDIT: I've written a program that takes a list of .a files and generates a merged .a file. Works with the GNU common ar format. Packing together all static libs of LLVM works like this
$ ./arcat -o combined.a ~/usr/llvm/lib/libLLVM*.a
I compared the speed against unpacking all .a files manually and then putting them into a new .a file using ar, recomputing the index. Using my arcat tool, I get consistent runtimes around 500ms. Using the manual way, time varies greatly, and takes around 2s. So I think it's worth it.
Code is here. I put it into the public domain :)
You can determine the order using the lorder and tsort utilities, for example
libs='/usr/lib/libncurses.a /usr/lib/libedit.a'
libs_ordered=$(lorder $libs | tsort)
resulting in /usr/lib/libedit.a /usr/lib/libncurses.a because libedit depends on libncurses.
This is probably only a benefit above --start-group if you do not run lorder and tsort again for each link command. Also, it does not allow mutual/cyclic dependencies like --start-group does.
Is there a third option where you just build a single library to begin with? I had a similar problem and I eventually decided to go with the third option.
In my experience, group is slower than just unifying the .a files. You can extract all files from the archive, then create a new .a file from from the smaller files
However, you have to be careful about a circumstance where both files happen to contain the same definition (you can explicitly check for this by using nm to see what definitions are contained in each library)