I want to use libxml2 as parser for a c program on a system with ubuntu. I used the following command to install libxml2:
sudo apt-get install -y libxml2-dev
I try to compile the following file: http://xmlsoft.org/examples/reader1.c
My makefile looks like this:
xml_reader: xml_reader.o
gcc -o xml_reader xml_reader.o -lxml2 -lm
xml_reader.c: xml_reader.c
gcc -c xml_reader.c -I/usr/include/libxml2
But sadly I get the following response:
fatal error: libxml/xmlreader.h: No such file or directory
Did I miss something, which I had to do before compiling or am I even using the right -l argument?
The target of your second Makefile rule should be xml_reader.o and not xml_reader.c. Right now make is using a default rule instead which does not make use of -I/usr/include/libxml2 and thus gcc cannot find the required header.
I'm new to developing for embedded systems, but I have installed arm-linux-gnueabi-gcc via the Linux Mint package manager and managed to build a few programs successfully.
I'm currently struggling with getting a program to compile using libusb. I've done the following:
Downloaded and unpacked the libusb 1.0.20 sources from https://sourceforge.net/projects/libusb/.
Compiled and installed them using the following commands:
~/Downloads/libusb-1.0.20 $ ./configure --host=arm-linux-gnueabi --prefix=/opt/ --disable-udev
~/Downloads/libusb-1.0.20 $ sudo make
~/Downloads/libusb-1.0.20 $ sudo make install
(The reason for sudo-ing the make commands was because I encountered permission problems related to removing old files.)
Copied a small sample file from somewhere on the internet:
#include <libusb-1.0/libusb.h>
#include <stdio.h>
int main()
{
int i=0;
libusb_context **c = NULL;
i = libusb_init(c);
printf("\nusing libusb.h\n");
return 0;
}
Tried to build it and run it with gcc:
~/Desktop/libtest $ gcc libtest1.c -o libtest1 -lusb-1.0
~/Desktop/libtest $ ./libtest1
using libusb.h
However, when I try to do the same with arm-linux-gnueabi-gcc, it can't find the library:
~/Desktop/libtest $ arm-linux-gnueabi-gcc libtest1.c -o libtest1 -lusb-1.0
/usr/lib/gcc-cross/arm-linux-gnueabi/4.7/../../../../arm-linux-gnueabi/bin/ld: cannot find -lusb-1.0
collect2: error: ld returned 1 exit status
Where did I go wrong? Is there something else I need to do in order to use the library? Did I fail at compiling the library for the arm compiler? I didn't include the compiler output here since it's quite long, but there are no obvious errors. This might be a very stupid question, but I'm completely clueless.
I would like to link readline statically with my program and I found this page about readline compilation from source http://www.bioinf.org.uk/software/profit/doc/node17.html but I'm a bit confused about the process.
The page talks about a variable READLINELIB in the makefile but I don't find it.
Could someone show me the way to use readline statically in my program, what to put in my Makefile for compiling readline from source and link it with my program?
Thank you.
Finally I figured out the proper way to do it, I using the --prefix option of the configure file I can tell where to put/install the library. The problem about installation was that I don't have the right to access other directories than my $HOME, so no problem doing this:
configure --prefix=$HOME/libreadline && make && make install-static
Then in my program I include the file from $HOME/libreadline/include.
To compile the main program I link the program with the archive libraries $HOME/libreadline/lib/libreadline.a and $HOME/libreadline/lib/libhistory.a.
Also since readline files uses directive like #include <readline/readline.h> which doesn't correspond to the location of the files, I must tell the compiler where to look for included files. To do this, before running gcc, I set the variable C_INCLUDE_PATH to $HOME/libreadline/include.
Finally, since readline uses ncurses dynamic library I must tell the compiler to dynamically link it with my program. It might be the case of termcap too...
The overall process looks like:
configure --prefix=$HOME/libreadline && make && make install-static
export C_INCLUDE_PATH=$HOME/libreadline/include
gcc -o myprogram myprogram.c $HOME/libreadline/lib/libreadline.a $HOME/libreadline/libhistory.a -lncurses -ltermcap
I was confused about what make install do, it only copy files to the location provided by the configure, by default it installs in system directories like /usr/include, etc... but providing the --prefix option make install will copy all files in the specified directory.
Installation is just copying compiled program, libraries, doc, etc to a certain location, by default standart system directories, if you don't have access to those directories like me you could "install" it in your own directory and then do whatever you wan't with it.
I could have installed the dynamic library instead the static one, but then I would have to modify the LD_LIBRARY_PATH environment.
get readline source
wget http://git.savannah.gnu.org/cgit/readline.git/snapshot/readline-master.tar.gz
tar zxvf readline-master.tar.gz
cd readline-master/
examples folder does not have Makefile, which is generated using Makefile.in script.
following steps build static & dynamic libs & puts them inside /usr/local/bin
./configure
make
sudo make install
may have to install curses as "sudo apt-get install libncurses5-dev"
Use following make file (strip down version from examples folder)
(Make sure tab is honored otherwise makefile will not work)
RM = rm -f
CC = gcc
CFLAGS = -g -O
INCLUDES = -I/usr/local/include
LDFLAGS = -g -L/usr/local/lib
READLINE_LIB = -lreadline
TERMCAP_LIB = -ltermcap
.c.o:
${RM} $#
$(CC) $(CFLAGS) $(INCLUDES) -c $<
SOURCES = rlversion.c
EXECUTABLES = rlversion
OBJECTS = rlversion.o
all: $(EXECUTABLES)
everything: all
rlversion: rlversion.o
$(CC) $(LDFLAGS) -o $# rlversion.o $(READLINE_LIB) $(TERMCAP_LIB)
clean mostlyclean:
$(RM) $(OBJECTS) $(OTHEROBJ)
$(RM) $(EXECUTABLES)
rlversion.o: rlversion.c
I was in need of libraries libreadline.a, libhistory.a for both 64 and 32 bit versions.
The answer provided by Rajeev Kumar worked for me. ( Had a little trouble finding and installing libncurses).
For 32-bit versions, using https://packages.ubuntu.com/search?keywords=lib32readline-dev, the following command worked for me.
sudo apt install lib32readline-dev
So it is hoped that for 64 also, it works
sudo apt install libreadline-dev
I wrote a little Lua module in C, that generates a UUID leveraging libuuid. You can find the source at https://github.com/Mashape/lua-uuid
The library properly works on OSX and CentOS. I am currently having a problem on Ubuntu where although the library successfully compiles, executing it throws the following exception:
lua: error loading module 'lua_uuid' from file './lua_uuid.so':
./lua_uuid.so: undefined symbol: uuid_generate
stack traceback:
[C]: ?
[C]: in function 'require'
/test.lua:1: in main chunk
[C]: ?
It seems like the library can't find the libuuid dependency, although in the Makefile includes the -luuid flag (https://github.com/Mashape/lua-uuid/blob/master/Makefile#L4).
To replicate the problem, these are the dependencies required:
apt-get install lua5.1 luarocks unzip git make gcc uuid-dev
wget https://github.com/Mashape/lua-uuid/archive/0.1-7.zip -O /tmp/lua_uuid.zip
unzip /tmp/lua_uuid.zip -d /tmp
cd /tmp/lua-uuid-0.1-7/ && luarocks make lua_uuid-0.1-7.rockspec
Then you can run the following Lua script:
local uuid = require "lua_uuid"
local uuid_str = uuid()
print("New UUID: "..uuid_str)
I am not proficient with C and Makefiles, is there something obvious that I am missing?
Compiling this module using LuaRocks on Ubuntu results in the following compiler command lines:
gcc -c -O2 -fPIC -I/usr/include/lua5.1 lua_uuid.c -o lua_uuid.o
gcc -shared -luuid -o lua_uuid.so -L/usr/lib lua_uuid.o
The library libuuid is available as a static library, and it is listed before the object file that references its symbols. Since the GNU linker inspects libraries/object files from left to right, all symbolds in libuuid are deemed unnecessary and left out of the final build because they haven't been referenced yet. Moving -luuid to the end of the linker command line (to the right of lua_uuid.o) fixes the issue.
There are already some Stackoverflow answers that explain the particulars:
Why does the order in which libraries are linked sometimes cause errors in GCC?
Why does the order of '-l' option in gcc matter?
I have built glibc 2.14 and installed it in directory ~/GLIBC/glibc_install. Now I want to build and run programs using this C library instead of my system's default C library.
To be sure that I was using my custom glibc, I added a call to puts into glibc/stdio-common/printf.c:__printf to print a message.
Then I rebuilt and reinstalled glibc.
Then I wrote a "Hello, World" program and tried to compile and link it as follows:
gcc -nodefaultlibs -static -lgcc -L~/GLIBC/glibc_install/lib -o myprog myprog.c
But I get the following linker error report:
/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/crt1.o: In function `_start':
(.text+0x19): undefined reference to `__libc_csu_init'
/usr/lib/gcc/x86_64-linux-gnu/4.4.3/../../../../lib/crt1.o: In function `_start':
(.text+0x25): undefined reference to `__libc_start_main'
/tmp/ccACTQEp.o: In function `main':
c1.c:(.text+0xa): undefined reference to `puts'
collect2: ld returned 1 exit status
What am I doing wrong?
Following a couple of suggestions from the glibc help mailing list (libc-help#sourceware.org), I have a solution. It turns out that this task is a bit tricky because you have to tell the linker to omit everything it would normally include automatically (and silently), and then include back everything that it needs, including a bunch of start and end files. Some of the start and end files come from libc and some come from gcc, so the make rule is a bit complicated. Below is a general sample makefile to illustrate the approach. I will assume that you are building a program called prog from a source file called prog.c and that you have installed your custom glibc in directory /home/my_acct/glibc_install.
TARGET = prog
OBJ = $(TARGET).o
SRC = $(TARGET).c
CC = gcc
CFLAGS = -g
LDFLAGS = -nostdlib -nostartfiles -static
GLIBCDIR = /home/my_acct/glibc_install/lib
STARTFILES = $(GLIBCDIR)/crt1.o $(GLIBCDIR)/crti.o `gcc --print-file-name=crtbegin.o`
ENDFILES = `gcc --print-file-name=crtend.o` $(GLIBCDIR)/crtn.o
LIBGROUP = -Wl,--start-group $(GLIBCDIR)/libc.a -lgcc -lgcc_eh -Wl,--end-group
$(TARGET): $(OBJ)
$(CC) $(LDFLAGS) -o $# $(STARTFILES) $^ $(LIBGROUP) $(ENDFILES)
$(OBJ): $(SRC)
$(CC) $(CFLAGS) -c $^
clean:
rm -f *.o *.~ $(TARGET)
Setup 1: compile your own glibc without dedicated GCC and use it
Without static also working at: Multiple glibc libraries on a single host
This setup might work and is quick as it does not recompile the whole GCC toolchain, just glibc.
But it is not reliable as it uses host C runtime objects such as crt1.o, crti.o, and crtn.o provided by glibc. This is mentioned at: https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location Those objects do early setup that glibc relies on, so I wouldn't be surprised if things crashed in wonderful and awesomely subtle ways.
For a more reliable setup, see Setup 2 below.
Build glibc and install locally:
export glibc_install="$(pwd)/glibc/build/install"
git clone git://sourceware.org/git/glibc.git
cd glibc
git checkout glibc-2.28
mkdir build
cd build
../configure --prefix "$glibc_install"
make -j `nproc`
make install -j `nproc`
Setup 1: verify the build
test_glibc.c
#define _GNU_SOURCE
#include <assert.h>
#include <gnu/libc-version.h>
#include <stdatomic.h>
#include <stdio.h>
#include <threads.h>
atomic_int acnt;
int cnt;
int f(void* thr_data) {
for(int n = 0; n < 1000; ++n) {
++cnt;
++acnt;
}
return 0;
}
int main(int argc, char **argv) {
/* Basic library version check. */
printf("gnu_get_libc_version() = %s\n", gnu_get_libc_version());
/* Exercise thrd_create from -pthread,
* which is not present in glibc 2.27 in Ubuntu 18.04.
* https://stackoverflow.com/questions/56810/how-do-i-start-threads-in-plain-c/52453291#52453291 */
thrd_t thr[10];
for(int n = 0; n < 10; ++n)
thrd_create(&thr[n], f, NULL);
for(int n = 0; n < 10; ++n)
thrd_join(thr[n], NULL);
printf("The atomic counter is %u\n", acnt);
printf("The non-atomic counter is %u\n", cnt);
}
Compile and run with test_glibc.sh:
#!/usr/bin/env bash
set -eux
rm -rf tmp
mkdir tmp
gcc \
-L "${glibc_install}/lib" \
-I "${glibc_install}/include" \
-Wl,--rpath="${glibc_install}/lib" \
-Wl,--dynamic-linker="${glibc_install}/lib/ld-linux-x86-64.so.2" \
-static \
-std=c11 \
-o tmp/test_glibc.out \
-v \
test_glibc.c \
-pthread \
;
sudo chroot tmp /test_glibc.out
The program outputs the expected:
gnu_get_libc_version() = 2.28
The atomic counter is 10000
The non-atomic counter is 8674
even though we ran it on a clean chroot, so the -static must have worked.
Command adapted from https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location but --sysroot made it fail with:
cannot find /home/ciro/glibc/build/install/lib/libc.so.6 inside /home/ciro/glibc/build/install
so I removed it.
ldd output confirms that the ldd and libraries that we've just built are actually being used as expected:
+ ldd test_glibc.out
linux-vdso.so.1 (0x00007ffe4bfd3000)
libpthread.so.0 => /home/ciro/glibc/build/install/lib/libpthread.so.0 (0x00007fc12ed92000)
libc.so.6 => /home/ciro/glibc/build/install/lib/libc.so.6 (0x00007fc12e9dc000)
/home/ciro/glibc/build/install/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fc12f1b3000)
The gcc compilation debug output shows that my host runtime objects were used, which is bad as mentioned previously, but I don't know how to work around it, e.g. it contains:
COLLECT_GCC_OPTIONS=/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o
Setup 1: modify glibc
Now let's modify glibc with:
diff --git a/nptl/thrd_create.c b/nptl/thrd_create.c
index 113ba0d93e..b00f088abb 100644
--- a/nptl/thrd_create.c
+++ b/nptl/thrd_create.c
## -16,11 +16,14 ##
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#include <stdio.h>
+
#include "thrd_priv.h"
int
thrd_create (thrd_t *thr, thrd_start_t func, void *arg)
{
+ puts("hacked");
_Static_assert (sizeof (thr) == sizeof (pthread_t),
"sizeof (thr) != sizeof (pthread_t)");
Then recompile and re-install glibc, and recompile and re-run our program:
cd glibc/build
make -j `nproc`
make -j `nproc` install
./test_glibc.sh
and we see hacked printed a few times as expected.
This further confirms that we actually used the glibc that we compiled and not the host one.
Tested on Ubuntu 18.04.
Setup 2: crosstool-NG pristine setup
This is an alternative to setup 1, and it is the most correct setup I've achieved far: everything is correct as far as I can observe, including the C runtime objects such as crt1.o, crti.o, and crtn.o.
In this setup, we will compile a full dedicated GCC toolchain that uses the glibc that we want.
The only downside to this method is that the build will take longer. But I wouldn't risk a production setup with anything less.
crosstool-NG is a set of scripts that downloads and compiles everything from source for us, including GCC, glibc and binutils.
Yes the GCC build system is so bad that we need a separate project for that.
This setup is only not perfect because crosstool-NG does not support building the executables without extra -Wl flags, which feels weird since we've built GCC itself. But everything seems to work, so this is only an inconvenience.
Get crosstool-NG and configure it:
git clone https://github.com/crosstool-ng/crosstool-ng
cd crosstool-ng
git checkout a6580b8e8b55345a5a342b5bd96e42c83e640ac5
export CT_PREFIX="$(pwd)/.build/install"
export PATH="/usr/lib/ccache:${PATH}"
./bootstrap
./configure --enable-local
make -j `nproc`
./ct-ng x86_64-unknown-linux-gnu
./ct-ng menuconfig
The only mandatory option that I can see, is making it match your host kernel version to use the correct kernel headers. Find your host kernel version with:
uname -a
which shows me:
4.15.0-34-generic
so in menuconfig I do:
Operating System
Version of linux
so I select:
4.14.71
which is the first equal or older version. It has to be older since the kernel is backwards compatible.
Now you can build with:
env -u LD_LIBRARY_PATH time ./ct-ng build CT_JOBS=`nproc`
and now wait for about thirty minutes to two hours for compilation.
Setup 2: optional configurations
The .config that we generated with ./ct-ng x86_64-unknown-linux-gnu has:
CT_GLIBC_V_2_27=y
To change that, in menuconfig do:
C-library
Version of glibc
save the .config, and continue with the build.
Or, if you want to use your own glibc source, e.g. to use glibc from the latest git, proceed like this:
Paths and misc options
Try features marked as EXPERIMENTAL: set to true
C-library
Source of glibc
Custom location: say yes
Custom location
Custom source location: point to a directory containing your glibc source
where glibc was cloned as:
git clone git://sourceware.org/git/glibc.git
cd glibc
git checkout glibc-2.28
Setup 2: test it out
Once you have built he toolchain that you want, test it out with:
#!/usr/bin/env bash
set -eux
install_dir="${CT_PREFIX}/x86_64-unknown-linux-gnu"
rm -rf tmp
mkdir tmp
PATH="${PATH}:${install_dir}/bin" \
x86_64-unknown-linux-gnu-gcc \
-Wl,--dynamic-linker="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib/ld-linux-x86-64.so.2" \
-Wl,--rpath="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib" \
-static \
-v \
-o tmp/test_glibc.out \
test_glibc.c \
-pthread \
;
sudo chroot tmp /test_glibc.out
Everything seems to work as in Setup 1, except that now the correct runtime objects were used:
COLLECT_GCC_OPTIONS=/home/ciro/crosstool-ng/.build/install/x86_64-unknown-linux-gnu/bin/../x86_64-unknown-linux-gnu/sysroot/usr/lib/../lib64/crt1.o
Setup 2: failed efficient glibc recompilation attempt
It does not seem possible with crosstool-NG, as explained below.
If you just re-build;
env -u LD_LIBRARY_PATH time ./ct-ng build CT_JOBS=`nproc`
then your changes to the custom glibc source location are taken into account, but it builds everything from scratch, making it unusable for iterative development.
If we do:
./ct-ng list-steps
it gives a nice overview of the build steps:
Available build steps, in order:
- companion_tools_for_build
- companion_libs_for_build
- binutils_for_build
- companion_tools_for_host
- companion_libs_for_host
- binutils_for_host
- cc_core_pass_1
- kernel_headers
- libc_start_files
- cc_core_pass_2
- libc
- cc_for_build
- cc_for_host
- libc_post_cc
- companion_libs_for_target
- binutils_for_target
- debug
- test_suite
- finish
Use "<step>" as action to execute only that step.
Use "+<step>" as action to execute up to that step.
Use "<step>+" as action to execute from that step onward.
therefore, we see that there are glibc steps intertwined with several GCC steps, most notably libc_start_files comes before cc_core_pass_2, which is likely the most expensive step together with cc_core_pass_1.
In order to build just one step, you must first set the "Save intermediate steps" in .config option for the intial build:
Paths and misc options
Debug crosstool-NG
Save intermediate steps
and then you can try:
env -u LD_LIBRARY_PATH time ./ct-ng libc+ -j`nproc`
but unfortunately, the + required as mentioned at: https://github.com/crosstool-ng/crosstool-ng/issues/1033#issuecomment-424877536
Note however that restarting at an intermediate step resets the installation directory to the state it had during that step. I.e., you will have a rebuilt libc - but no final compiler built with this libc (and hence, no compiler libraries like libstdc++ either).
and basically still makes the rebuild too slow to be feasible for development, and I don't see how to overcome this without patching crosstool-NG.
Furthermore, starting from the libc step didn't seem to copy over the source again from Custom source location, further making this method unusable.
Bonus: stdlibc++
A bonus if you're also interested in the C++ standard library: How to edit and re-build the GCC libstdc++ C++ standard library source?
You command line is just bogus. Try:
gcc -nodefaultlibs -static -L~/GLIBC/glibc_install/lib -o myprog myprog.c -lgcc -lc -lgcc -lc
or similar. You omitted -lc, and also erroneously had your libraries before your input files.
And you were searching for a library called libibgcc rather than libgcc...