Setting environment variables and including linking directories in a Makefile - c

I have a C file that includes a header that itself includes "jni.h". I'm currently compiling the file using the command:
gcc hdfs_test.c -I/HDFS_HOME/hdfs/src/c++/libhdfs -I/usr/lib/jvm/default-java/include -L/HDFS_HOME/hdfs/src/c++/libhdfs -L/HDFS_HOME/build/c++/Linux-i386-32/lib -L/usr/lib/jvm/default-java/jre/lib/i386/server -ljvm -lhdfs -o hdfs_test
to run the resulting .o file, the environment variables for CLASSPATH and JAVAHOME.
Now, I want to use this file in an existing C project where they use make files. The project is Postgresql. In this project Makefiles are used. I want to add the needed instructions to the makefile in the folder where I'm adding the file, so that I can run it with the project.
The current makefile has the following:
subdir = src/backend/storage/smgr
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
OBJS = md.o smgr.o smgrtype.o
include $(top_srcdir)/src/backend/common.mk
What should I add to the makefile in order to compile my c file with the project?
Thanks
* Edit 1 *
I probably should have stated what I was trying to do. So far, I added multiple source files to the postgres SMGR module (source directory).
They all worked fine by just adding the fileName.o to the list of OBJS files. For example if I added A.c, I would add A.o to the OBJS list and the file will be compiled and added to the project.
Now the new file that I'm trying to add is hdfs_test.c
this is different than the other source files that I added previously. The difference is that it doesn't compile using simple gcc command, but needs the command that I've shown above.
I tried to manually generate the .o file manually by adding this to my Makefile:
OBJS = md.o smgr.o smgrtype.o hdfs_FD.o hdfsManager.o smgrWrapper.o hdfs_test.o
include $(top_srcdir)/src/backend/common.mk
hdfs_test.o : hdfs_test.c
gcc hdfs_test.c -I/HDFS_HOME/hdfs/src/c++/libhdfs -I/usr/lib/jvm/default-java/include -L/HDFS_HOME/hdfs/src/c++/libhdfs -L/HDFS_HOME/build/c++/Linux-i386-32/lib -L/usr/lib/jvm/default-java/jre/lib/i386/server -ljvm -lhdfs -o hdfs_test
When Make, I'm getting the following error:
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 0 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 1 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 4 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 5 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 6 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 7 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 11 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 12 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 13 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 14 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 15 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 16 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 17 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 18 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 19 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 20 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 21 has invalid symbol index 14
/usr/bin/ld: /usr/lib/debug/usr/lib/crt1.o(.debug_info): relocation 22 has invalid symbol index 22
/usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/crt1.o: In function _start':
(.text+0x18): undefined reference tomain'
collect2: ld returned 1 exit status
make: * [tati.o] Error 1
Hope this clarifies things.

postgress uses automake build environment from the looks of it.
Your knowledge of just Make seems to be limited, so i would suggest that you read up on the following.:
GNU make << start here >>,
GNU autoconf,
GNU automake,
and do the examples in there and them modify them for your hdfs_test. Once you understand how to do that, you'll be able to answer your question. When you stumble into an issue there, I'll be happy to help you resolve it.

This really depends on what exactly your code is trying to do, and why role the JVM and C++ play. It might not work at all, even if you get it to build. You should probably discuss the details with the PostgreSQL developers. You might also be better off building your code as a shared library and installing it as an extension.

Makefiles consisting of multiple files in multiple directories are typically very hard to understand, and rely on lots of black magic to work properly, so showing us just one small file is probably not very meaningful.
That said, I'd start by just adding a new .o file to OBJS, and see what happens. Probably it'll assume that there's a source file of the same basename, and Just Work. The file doesn't give any dependency information, so it will probably auto-generate that without any input from you.
On the other hand, the whole system might be rubbish ....

Related

With a new custom compiled, builded version of glibc/libc.so, what happens during a “relocation has invalid symbol index” error?

From the link, I would like to compile a new glibc by myself. I know that compiling a glic is difficult, so on my first step, I would like to comiple a new glibc that match the exact same version already running on my linux system. I can skip toolchain dependency checking and start to focus on the glic iteself.
My ubuntu info is like:
abbott#abbott-VirtualBox:/software/glibc/code$ uname -a
Linux abbott-VirtualBox 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
abbott#abbott-VirtualBox:/software/glibc/code$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
The already existed glibc is: 2.19. check the version like:
abbott#abbott-VirtualBox:/software/glibc/code$ ldd --version
ldd (Ubuntu EGLIBC 2.19-0ubuntu6.9) 2.19
I prepare a very simple c code for testing:
#include <stdio.h>
int main(){
long z; printf("Long int size is %i bytes long!\n", sizeof(z));
return 0;
}
I compile this code using the already existed gcc,
gcc simple.c
I goe the "a.out", it's ok. it's runnable it's wonderful:
abbott#abbott-VirtualBox:/software/glibc/code$ ldd a.out
linux-vdso.so.1 => (0x00007ffceaf0b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1a66f64000)
/lib64/ld-linux-x86-64.so.2 (0x000055b9118c7000)
So, I download from here, I choose the "glibc-2.19.tar.gz".
I put the source at
/software/glibc/glibc-2.19
compile at
/software/glibc/compile-2.19
compile result is OK:
abbott#abbott-VirtualBox:/software/glibc/compile-2.19$ ../glibc-2.19/configure -prefix=/usr
abbott#abbott-VirtualBox:/software/glibc/compile-2.19$ make
In the "compile-2.19" folder, there is a testrun.sh shell code,
I follow the "Compile normally, run under new glibc" section in the link, it's ok, it works.
abbott#abbott-VirtualBox:/software/glibc/compile-2.19$ ./testrun.sh ../code/a.out
I follow the "Compile against glibc build tree" section in the link,
GLIBC=/software/glibc/compile-2.19
gcc \
-Wl,-rpath=${GLIBC}:\
${GLIBC}/math:\
${GLIBC}/elf:\
${GLIBC}/dlfcn:\
${GLIBC}/nss:\
${GLIBC}/nis:\
${GLIBC}/rt:\
${GLIBC}/resolv:\
${GLIBC}/crypt:\
${GLIBC}/nptl:\
${GLIBC}/dfp \
-Wl,--dynamic-linker=${GLIBC}/elf/ld.so
-o myligcsimple simple.c
It output:
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 21
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
I googled for several days, I cound not find the answers. some says that the main func is missing. but I have the main func, and the code works fine with the already existed glibc.
Question 1:
Would anybody help to find out how to fix the problem?
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
I follow the "Compile against glibc build tree" section in the link, ...
/usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info)
Under no circumstances should your link use any objects from /usr/lib/debug/, and the command line arguments you show will certainly not cause that to happen. See also this answer.
Therefore, it is likely that you are not telling the whole story. Perhaps you've modified some environment variables, or the GCC driver, or you are not showing the actual command you used.

Linking errors from gcc -o0 optimization level

I have a project where I am trying to do some library and system call benchmarking. I want to make sure that the compiler does not optimize away my calls.
Here is my makefile:
CC=gcc
CFLAGS= -I ./
LIBFLAGS= -lm
RM=/bin/rm -f
all: osbench
osbench: osbench.c
$(CC) $(CFLAGS) -o $# osbench.c $(LIBFLAGS)
clean:
$(RM) osbench *~
When I set the flag to -o everything compiles and runs perfectly but I am not sure if the compiler has omitted my calls to the math.h library.
According to the gcc manual (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html) -o0 means that no optimization takes place - which is what I want. When I set the flag to -o0 I get all sorts of linking errors.
$(CC) $(CFLAGS) -o0 $# osbench.c $(LIBFLAGS)
Is there a another flag I am missing?
gcc -I ./ -o0 osbench osbench.c -lm
osbench: In function `_fini':
(.fini+0x0): multiple definition of `_fini'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crti.o:(.fini+0x0): first defined here
osbench: In function `__data_start':
(.data+0x0): multiple definition of `__data_start'
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 0 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 1 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 4 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 5 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 6 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 7 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 10 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 11 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 12 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 13 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 14 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 15 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 16 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 17 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 18 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 19 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): reloca tion 20 has invalid symbol index 19
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crt1.o:(.data+0x0): first defined here
osbench: In function `__data_start':
(.data+0x8): multiple definition of `__dso_handle'
/usr/lib/gcc/x86_64-linux-gnu/4.6/crtbegin.o:(.data+0x0): first defined here
osbench:(.rodata+0x0): multiple definition of `_IO_stdin_used'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crt1.o:(.rodata.cst4 +0x0): first defined here
osbench: In function `_start':
(.text+0x0): multiple definition of `_start'
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): reloca tion 0 has invalid symbol index 2
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crt1.o:/build/buildd /eglibc-2.15/csu/../sysdeps/x86_64/elf/start.S:109: first defined here
osbench: In function `_init':
(.init+0x0): multiple definition of `_init'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/crti.o:(.init+0x0): first defined here
/tmp/ccOMIYX2.o:(.rodata+0x0): multiple definition of `numOfCallsPerTrial'
osbench:(.rodata+0x8): first defined here
/tmp/ccOMIYX2.o:(.rodata+0x8): multiple definition of `numOfTrials'
osbench:(.rodata+0x10): first defined here
/tmp/ccOMIYX2.o: In function `runLibCallTrial':
osbench.c:(.text+0x0): multiple definition of `runLibCallTrial'
osbench:(.text+0xe4): first defined here
/tmp/ccOMIYX2.o: In function `calcLibCallTime':
osbench.c:(.text+0xab): multiple definition of `calcLibCallTime'
osbench:(.text+0x18f): first defined here
/tmp/ccOMIYX2.o: In function `main':
osbench.c:(.text+0x143): multiple definition of `main'
osbench:(.text+0x38c): first defined here
/usr/lib/gcc/x86_64-linux-gnu/4.6/crtend.o:(.dtors+0x0): multiple definition of `__DTOR_END__'
osbench:(.dtors+0x8): first defined here
/usr/bin/ld: error in osbench(.eh_frame); no .eh_frame_hdr table will be created .
collect2: ld returned 1 exit status
I can post my code if necessary, however the benchmarking code part is for a homework assignment so I'm not sure I am allowed to. It compiles fine otherwise so I don't think it's a problem in the .c
Thanks.
-O not -o. That is, upper case not lower case. Anyway, you can just leave it out altogether. Default is -O0.
In case it's not clear, -o has a different meaning. It determines the compilation output name. So you should keep that in your command line. Add -O to change the optimisation level (or leave it out for default). So in fact, your original build already has optimisations disabled.

What happens during a "relocation has invalid symbol index" error?

Here is a test reproducing the problem:
$ echo "void whatever() {}" > prog.c
$ gcc prog.c
This produces the following error on GCC 4.8.4:
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
... etc ...
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 21
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
Note that on GCC 6.2.0 the errors related to this question disappear, and it instead produces just:
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
This has been reported a number of times by a number of users, on Stack Overflow, and elsewhere.
I would like to understand this error, not solve it (it is already solved).
This error happens when doing gcc-4.8 prog.c without a main() function inside prog.c.
I have done a text search for this error on the binutils-source package. A tedious googling gave me only one useful link helping me better understanding the concept of relocation handling.
The number of errors does not seem to depend on the program, which suggests the relocations in consideration do not originate in this file, but as a direct result of the missing main() function. I have hypothesized that 3 of these relocations with wrong indexes might be for main(), argc and argv, but many remain, and this is just an unproven hypothesis.
This is quite above my head, and any information that helps me better understand it, or what changed in later versions of GCC, would be warmly welcome.
C Program Features (Unix-like)
every program is compiled separately into elf format
c program can use external variable/function reference, which is linked later
main is not the start of program as you originally thought, c lib has a starter program (crt1.o) which has a _start program which will invoke our main and do cleaning job after main
concludes above statement, we can know that even a very simple program as OP showed need to be linked
ELF Format
ELF has two headers, as following shows:
section header -- used to link multiple elf to make process image
program header -- used to load process image
Here we only focus on section header structure:
mapping<var_name, offset, size...>
// and special cases
mapping<external_var_name, offset, size...>
Every program is compiled separately, which means address allocation is similar (In the early version of linux, every compiled program start with same virtual address -- 0x08000000, and many attacks can make use of this, so it changes to adding some random delta to address to alleviate the problem), so there may exists some overlay area. This is why address relocation needed.
Relocation
The relocation info (offset, value etc) is stored in .rel.* section:
Relocation section '.rel.text' at offset 0x7a4 contains 2 entries:
Offset Info Type Sym.Value Sym. Name
0000000d 00000e02 R_386_PC32 00000000 main
00000015 00000f02 R_386_PC32 00000000 exit
Relocation section '.rel.debug_info' at offset 0x7b4 contains 43 entries:
Offset Info Type Sym.Value Sym. Name
00000006 00000601 R_386_32 00000000 .debug_abbrev
0000000c 00000901 R_386_32 00000000 .debug_str
When the linker want to set the address of main in the process of relocation, it can't find a symbol in your compiled elf file, so it complains that and stop the linking process.
Example
Here is the simplified version of os implementations, start.c corresponds to crt1.o's source code:
int entry(char *); // corresponds to main
void _start(char *args) {
entry(args);
exit();
}
Ref
ELF
Relocation
Instead of compiling the code directly, go through all the stages of compilation to figure out where the error is arising (as far as I know, such errors occur during linking). Following gcc arguments will be helpful:
-E Preprocess only; do not compile, assemble or link
-S Compile only; do not assemble or link
-c Compile and assemble, but do not link
Now:
gcc -E prog.c
gcc -S prog.c
gcc -c prog.c
With the program/code you have mentioned, all these steps are working perfect with gcc 4.8.4. But, during linking, when you compile using gcc prog.c, the compiler is unable to link with respective library, as it was not mentioned. Also, we have no main function in the prog.c file. So, we need to indicate -nostartfiles switch. Hence, you can compile prog.c as:
gcc prog.c -lc -nostartfiles
This produces the warning:
/usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 00000000004002a3
This is because of the sequence. i.e., init calls _start function and the _start function calls main function. This warning means that the _start function is unable to locate main function, where the init call is unable to locate _start. Please note that this is just a warning. In order to avoid this warning, we need to change the command to compile without warnings as follows.
gcc prog.c -lc --entry whatever -nostartfiles
With this command, we are instructing the kernel to compile prog.c using gcc by linking the libc.so library with the starting point as the function whatever, where this code contains no main function.
This is the context with gcc 4.8.4, which I've compiled on.
Coming to the case of gcc 6.2.0, I think all these linking stuff is taken care by the compiler itself. Hence, you can simply mention the compiling command as shown below.
gcc -c prog.c -nostartfiles
If it produces any other errors or warnings, you can use the switches mentioned above.
Also, note that crt0 through crtN (N depends on the ELF file) are executed before the init calls _start, which gives the metadata about the memory and other machine dependent parameters. The linking errors shown as
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
do not provide complete information for rectifying the issue, as machines are not as smart as human beings in identifying the point of error.
This produces a complete executable file. Please do notice that such code (without main function) is used when we are working on libraries/modules within a project.
All the data provided is done with step-by-step analysis. You can recreate all the steps mentioned. Hope this cleared your doubt. Good day!
We can break this down into two parts:
Undefined reference to `main'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status
This is simply because the C Runtime library (crt1.o) is trying to call your (missing) main() function. There is a good overview of the various C Runtime files here / here.
Relocation X has invalid symbol index Y
Note: This has been a bit of mission (learning opportunity) for me - I've taken a few days to research and understand in the limited free time that I have. It's something that I've wondered about for a long time but never looked into... Hopefully my understanding is correct (though clearly not complete - I'll update it if I can).
This is a little more complex, and hidden away.
Just by reading the message, we can gleen that it's related to debug information (the .debug_info and .debug_line sections of crt1.o's debug file is mentioned). Note the /usr/lib/debug/ path, which just contains debug information, the other crt1.o is a "stripped" file...
The format string is found in the binutils project, specifically bfd/elfcode.h. BFD being Binary File Descriptor - the GNU way to handle object files across a number of system architectures.
BFD is an intermediate format used for binary files - GCC will employ BFD before it finally writes an a.out, ELF, or other binary.
Looking into the manual, we can find some interesting snippets of knowledge:
[...] with each entry in the hash table the a.out linker keeps the index the symbol has in the final output file (this index number is used so that when doing a relocateable link the symbol index used in the output file can be quickly filled in when copying over a reloc).
[source]
The standard records contain only an address, a symbol index, and a type field. [source]
This means that these errors are issued due to relocations that are related to specific (missing?) 'symbols'. A 'symbol' in this context is any named 'thing' - e.g: functions and variables.
As these 'invalid symbols' appear to be resolved by simply declaring main(), I would guess that some (all?) of these symbol indexes are derived from main(), its debug information, and/or relations.
I can't tell you what is supposed to be at the symbol indexes mentioned (2, 11, 12, 13, 21), but it is interesting that my tests yielded the same list of symbol indexes.
Running ld with crt1.o alone gives us similar output:
$ ld /usr/lib/x86_64-linux-gnu/crt1.o
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 12
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 11
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 12
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 13
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 21
ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): undefined reference to `__libc_csu_fini'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x19): undefined reference to `__libc_csu_init'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x25): undefined reference to `__libc_start_main'

C compile error: ld returned 1 exit status [duplicate]

I have just dist-upgraded a Debian Weezy machine to run gcc-4.8 from gcc-4.7. Previously the build environment was sane and was compiling normally. Now it gives the following linker errors, with any program (even a trivial hello world):
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 13
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 20
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): undefined reference to `__libc_csu_fini'
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x19): undefined reference to `__libc_csu_init'
collect2: error: ld returned 1 exit status
I'm sure this is something simple connected with libc6, but I can't see what? I'm quite surprised to be seeing this error since I'd have assumed dpkg would have taken care of any dependencies, so I'm not sure if it's something wrong with this specific system or an issue with the debian package (which seems unlikely, since nobody else seems to have this problem!)
Any ideas? :)
Ah! As soon as I finished typing this, as a last ditch, I tried:
apt-get install libc6-dev --reinstall
(although I was convinced I'd already done it previously), and lo and behold, the issue went away!

Linker error on relocating a program above 2GB in x86_64 linux?

I have a user program which normally compiles to have an entry point at 0x400460 which I have to relocate to have an entry point starting at within 2GB of the shared libraries loaded in Linux. e.g
linux-vdso.so.1 => (0x00007fff109cd000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcd195e6000)
lib64/ld-linux-x86-64.so.2 (0x00007fcd199af000)
I am using gcc command line argument -Wl,-Ttext=0x80000000 to specify the start address for the .text segemnt.
The issue is that when I am giving an address above 2GB in this argument I am getting a linker error which is:
gcc test.c -ggdb -Wl,-Ttext=0x80000000 -o test1
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 1 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 2 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 3 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 4 has invalid symbol index 10
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 5 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 6 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 7 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 8 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 9 has invalid symbol index 2
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 10 has invalid symbol index 11
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 11 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 12 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 13 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 14 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 15 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 16 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 17 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 18 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 19 has invalid symbol index 12
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 20 has invalid symbol index 20
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_line): relocation 0 has invalid symbol index 2
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x12): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_fini' defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x19): relocation truncated to fit: R_X86_64_32S against symbol `__libc_csu_init' defined in .text section in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(elf-init.oS)
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in .text section in /tmp/ccFshK69.o
/var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux- gnu/4.8.2/crtbegin.o: In function `deregister_tm_clones':
crtstuff.c:(.text+0x8): relocation truncated to fit: R_X86_64_32S against `.tm_clone_table'
/var/services/homes/adabral/elider/gc/a1/lib/gcc/x86_64-unknown-linux- gnu/4.8.2/crtbegin.o: In function `register_tm_clones':
crtstuff.c:(.text+0x38): relocation truncated to fit: R_X86_64_32S against `.tm_clone_table'
collect2: error: ld returned 1 exit status
: The reason for this as far as I can make out after going through several SO questions and forums can be that some of the sections are still mapped to low 2GB address space.
This is the output of the readelf -a for a binary compiled with text segment just below 2GB (at 0x79990000).
Dynamic section at offset 0x190310 contains 24 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000c (INIT) 0x4003f0
0x000000000000000d (FINI) 0x79990204
0x0000000000000019 (INIT_ARRAY) 0x79b902f8
0x000000000000001b (INIT_ARRAYSZ) 8 (bytes)
0x000000000000001a (FINI_ARRAY) 0x79b90300
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x0000000000000004 (HASH) 0x400278
0x0000000000000005 (STRTAB) 0x400318
0x0000000000000006 (SYMTAB) 0x4002a0
0x000000000000000a (STRSZ) 72 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000015 (DEBUG) 0x0
0x0000000000000003 (PLTGOT) 0x79b904e8
0x0000000000000002 (PLTRELSZ) 72 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x4003a8
0x0000000000000007 (RELA) 0x400390
0x0000000000000008 (RELASZ) 24 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x000000006ffffffe (VERNEED) 0x400370
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x400360
0x0000000000000000 (NULL) 0x0
You can see that the INIT and some other sections still starts at the low 2GB address space. So the dynamic linker can not offset the relocation address at run time since the relocation type is R_X86_64_32.
So I tried compiling my code with gcc -mcmodel=large flag but I am still getting the same linker error. Using the large model should have rectified this error but it is not.
I am stuck at this point, any help is highly appreciated.
I am working on x86_64 ubuntu. gcc version 4.8.2
You should first understand that the ABI for x86_64 has several different "models": small, kernel, medium, and large. These are described under the GCC -mcmodel option here: http://gcc.gnu.org/onlinedocs/gcc/i386-and-x86-64-Options.html
What you're experiencing is that crt1.o, the startup code that gets linked into every program that's responsible for taking the initial ELF register/stack state and passing them into the libc startup code which eventually calls main, seems to be using the small model. You can see here:
/usr/lib/x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): relocation truncated to fit: R_X86_64_32S against symbol `main' defined in .text section in /tmp/ccFshK69.o
What's happening is that crt1.o has a relocation for the address of main that only allows for a 32-bit address to be filled in. (Note: Even if main were defined in a shared library rather than the executable, there would be a PLT entry in the executable, and the address of this PLT entry would be the official address of main to which the relocation would be resolved.)
To solve this problem, you would need a crt1.o that can handle full 64-bit addresses. One way to get this might be by using Scrt1.o, which is normally only used for PIE executables, instead of crt1.o. You could probably achieve this with -nostartfiles and manually specifying all the start files on the link command line. It might be worth filing a bug report against glibc requesting that the x86_64 crt1.o be converted to "large model" so that it works with main programs not linked in the 32-bit range.
Note that you'll probably also need -mcmodel=large (or perhaps -fPIE would work) for all of your own code to make it link and work correctly at high addresses. This could make it considerably larger and slower. You might want to rethink why you're doing this and whether you really need to.

Resources