I am aiming at removing as much dead code as possible from my executable to reduce code-size. I followed a number of tips from some other questions on Stack Overflow and am using -fdata-sections -ffunction-sections... + ...--gc-sections in my makefile. However I have one question.
I am first compiling all my .o's using -fdata-sections -ffunction-sections as flags. Then I am archiving these into a static library (lib.a). Then I am building the executable linking this lib.a and using the --gc-sections flag in this linker. However this doesnt seem to do any improvement. Is this because I am archiving first? Does archiving remove the effect of splitting into sections? (Also, any form of "strip" I am using doesn't seem to affect the executable at all!)
Related
I am working on a task for the university, there is a webiste that checks my memory usage and it compiles the .c files with:
/usr/bin/gcc -DEVAL -std=c11 -O2 -pipe -static -s -o program programname.c -lm
and it says my program exceeds the memory limits of 4 Mib which is a lot i think. I was told this command makes it use more memory that the standard compilation I use on my pc, like this:
gcc myprog.c -o myprog
I launched the executable created by this one compilation with:
/usr/bin/time -v ./myprog
and under "maximum resident set size" it says 1708 kilobytes, which should be 1,6 Mibs. So how can it be that for the university checker my program goes over 4 Mibs? I have eliminated all the possible mallocs i have, I just left the essential ones but it still says it goes over the limit, what else should I improve? I'm almost thinking the wesite has an error or something...
From GNU GCC Manual, Page 197:
-static On systems that support dynamic linking, this overrides ‘-pie’ and prevents linking with the shared libraries. On other systems, this
option has no effect.
If you don't know about the pie flag quoted here, have a look at this section:
-pie Produce a dynamically linked position independent executable on targets that support it. For predictable results, you must also
specify the same set of options used for compilation (‘-fpie’,
‘-fPIE’, or model suboptions) when you specify this linker option.
To answer your question: yes is it possible this overhead generated by the static flag, because in that case, the compiler can not do the basic optimization by merging stdlib's code with the one you've produced.
As it was suggested in the comments you shall compile your code with the same flag of the website to have an idea of the real overhead of your program (be sure that your gcc version is the same of the website) and also you shall do some common manual optimization such constant folding, function inlining etc. A good reference to these optimizations could be this one
I've tried passing -ffunction-sections -fdata-sections for the compiler, but that doesn't seem to have the desired effect. As far as I understand, I also have to pass -Wl,--gc-sections to the linker, but I'm not linking the files at this point. I just want to have a .a library file as small as possible, with minimal redundant code/data.
The compiler performs optimization based on the knowledge it has of the program. Optimization levels -O2 and above, in particular, enable unit-at-a-time mode, which allows the compiler to consider information gained from later functions in the file when compiling a function. Compiling multiple files at once to a single output file in unit-at-a-time mode allows the compiler to use information gained from all of the files when compiling each of them.
Not all optimizations are controlled directly by a flag.
-ffunction-sections
-fdata-sections
Place each function or data item into its own section in the output file if the target supports arbitrary sections. The name of the function or the name of the data item determines the section's name in the output file.
Use these options on systems where the linker can perform optimizations to improve locality of reference in the instruction space. Most systems using the ELF object format and SPARC processors running Solaris 2 have linkers with such optimizations. AIX may have these optimizations in the future.
Only use these options when there are significant benefits from doing so. When you specify these options, the assembler and linker will create larger object and executable files and will also be slower. You will not be able to use gprof on all systems if you specify this option and you may have problems with debugging if you specify both this option and -g.
U can use the following link for more details..:-)
http://gcc.gnu.org/onlinedocs/gcc-4.0.4/gcc/Optimize-Options.html
The following will reduce the size of your compiled objects (and thus the static library)
-Os -g0 -fvisibility=hidden -fomit-frame-pointer -mno-accumulate-outgoing-args -finline-small-functions -fno-unwind-tables -fno-asynchronous-unwind-tables -s
The following will increase the size of objects (though they may make the binary smaller)
-ffunction-sections -fdata-sections -flto -g -g1 -g2 -g3
The only way to really make a static library smaller is to remove the unneeded code by compiling the static library with -ffunction-sections -fdata-sections and linking the final product with -Wl,--gc-sections,--print-gc-sections to find out what parts are cruft. Then go back and remove those functions/variables (this also works for making smaller shared libraries - see http://libraryopt.sf.net)
Compare the size of the library if you -g in the compiler flags, vs not having it. I can easily imagine you double the size of a static library if it includes debug information in. If that was what you saw, chances are you already stripped the library of debug symbols, and hence cannot significantly shrink the library file more. This could explain your memory of cutting the size in half at some point.
I am learning C and I just read the term multiple compilation.Till now I had a single file.c and I used the command gcc file.c to compile it and then ./a.out to execute it. But I got confused a little bit. When should I use the multiple compilation instead of the single and which would be the possible reasons that they will lead me to prefer a multiple compilation instead of the single? I searched it and I found some articles but they didn't cover fully my questions. 1 (this is for c++) , 2If i undestood well if I have some files.c in my project eg file1.c, file2.c and then i want to link them, i execute
gcc file1.c
gcc file.c
gcc file1.o file2.o //somehow i have to create the .o files..
Thank you..
Compilation takes time. There's no point in re-compiling C code that hasn't changed. So, for large projects, it makes sense to split the code into multiple files (typically not randomly of course, but into modules of different functionality) and compile them only when needed.
Linking is the process of taking a bunch of object code (what the .o files are called) and turning them into a single program.
There are many steps to compiling.
When you invoke gcc it will create by default an executable file i.e. all steps in one go:
.c -> .i preprocessor
.i -> .s compiler
.s -> .o assembler
*.o -> a.out linker
Generally the first two take up the most time. If you have a large project then recompiling the entire project may take a lot of time when you are developing. So the compiler allows you to stop at a certain point and reuse previous results of files that have not changed:
gcc -E for preprocess only (rarely used)
gcc -S compile, but don't assemble. Useful for debugging or optimising assembly
gcc -c compile, assemble, but don't link. This is the most commonly used one and produces object files. Those contain your assembled functions (object code), but it's not capable of running because not all functions may be present yet, library functions are missing and the executable header has not been linked in.
The final step gcc -o executable *.o will then take all those and link them together to create an executable. Optionally linking libraries into it.
Generally having all functions in one source file will allow the compiler to do the more optimisations (i.e. inlining), but at the cost of compile time.
Have a look at https://cs.senecac.on.ca/~btp200/pages/images/compile_link.png
For some rather complicated reason, I have a set of files which I would like to compile seperatly and then link, but so that the functions in one are placed inline in the second. This is because I would like them to be compiled with different flags in GCC. I know I could fix the problem by looking into how I could get around that, but I would like to know if this is possible.
EDIT 1:
If not, is it possible to compile the 'external' functions into a form of assembly that I could include in the other file. Yes crazy but also cool...
Having a quick look, this could well be an option. I guess it would be impossible to automatically compile it in, so could someone please give me a bit of information about assembly? I've only used basic ARM assembly. I've compiled to toy functions with the -S flag in GCC. How do I link registers with variables? Will they always be in the same order? The function will be highly optimised. When should I start and end the extract? Should I include .cfi_startproc at the start and .cfi_def_cfa 7, 8 at the end?#
EDIT 2:
This post details how gcc can do link-time optimisations like this with -flto. Sadly this is only available with version 4.5, which I do not have nor have the ability to install since I do not have root access of the machine I need to compile this on. Another possible solution would be to explain how I could install a different version of GCC into a folder on a unix machine.
As far as I know gcc doesn't do linktime optimizations (inlining in particular), at least with the standard ld linker (it could be that the new gold linker does it, but I really don't think so). Clang in principle should be capable of doing it, since it depends on LLVM, which supports link time optimizations (it seems that your question is gcc spacific, though).
From your question though, it seems you are looking for a a way to merge object files after compilation, not necessarily by inlining their contained functions. This can be done in multiple ways:
Archiving them into a static library with ar: e.g. ar libfoo.a obj1.o obj2.o.
Combining them together into a third relocatable object (ld's --relocatable option). gcc -Wl,--relocatable -o obj3.o obj1.o obj2.o
Putting them into a shared library (beware that this requires compiling the objects with -fPIC) e.g. gcc -shared -o libfoo.so obj1.o obj2.o
You could compile with the -c option to create a set of .o files, or even make a .so file. Then use the sequence you like in the linking phase of gcc.
We're facing an interesting topic. Lets say we have a special-functions.c file, basically a library.
We need to optimize the code as getting rid of all unused/unreferenced functions during the build process on-the-fly.
I'm not searching for generally unused (dead) code: some parts will be "dead" in case of compiling to one of the architectures, but it's going to be used in an other architecture build.
Does anybody knows of flags, tools, methods, tricks to do this?
The compiler is standard gcc with ansi 99 C code.
EDIT
I know, this is mainly the part of the linker, but using gcc, the process is not really split into two parts.
From http://embeddedfreak.wordpress.com/2009/02/10/removing-unused-functionsdead-codes-with-gccgnu-ld/ :
Compile with -fdata-sections to keep the data in separate data
sections and -ffunction-sections to keep functions in separate
sections, so they (data and functions) can be discarded if unused.
Link with --gc-sections to remove unused sections.
For example:
gcc -Os -fdata-sections -ffunction-sections test.c -o test -Wl,--gc-sections
I think that a recent GCC (i.e. 4.6) should do that if you compile and link with the -flto flag (link time optimization). I would imagine that having hidden or internal visibility should be relevant (at least for non-static functions).
To my knowledge, the GNU binary utils (ld, in this case) already remove unusesd references on static link