Make a C static library that only exposes the API - c

I have an old static library that includes some functions produced by a C auto-coder (Mathworks/Simulink). I have just made a new static library (that does totally different things from the old library) that uses a newer version of the auto-coder. I need to make an executable that links both these libraries. Unfortunately many of the helper functions generated by the auto-coder have the same name in the old and new libraries but their functionality has changed. These functions are not part of the API. Obviously when I try to link I get complaints that the same symbols appear in both libraries.
The old library is untouchable, but I can mess with the new one.
The libraries are built with
gcc -c *.c
ar -crs libwhatever.a *.o
and linked with
gcc main.o -L. -lold -lnew
My current solution is to look at all the symbol clashes given by gcc and rename the symbols in the new library using
objcopy --redefine-sym sym=sym_new libnew.a
This seems to work. My program links and behaves as expected. But I'm wondering if it is the right/best thing to do.
Ideally I'd like to make a libnew.a that only exposes the API functions and nothing else. That way I guarantee no symbol clashes when I try to link. Is this possible I cannot figure out how.
Please note that I have little previous experience in these matters. Also, if it's relevant/not obvious, I'm using Linux.
Edit
Below is a rather contrived minimal working example.
old/printOld.c:
#include "printOld.h"
#include <stdio.h>
void printOld()
{
printStr();
}
void printStr()
{
printf("Old\n");
}
old/printOld.h:
void printStr();
old/buildOld:
gcc -c printOld.c
ar -crs libold.a printOld.o
new/printNew.c:
#include "printNew.h"
#include <stdio.h>
void printNew()
{
printStr();
}
void printStr()
{
printf("New\n");
}
new/printNew.h:
void printStr();
new/buildNew:
gcc -c printNew.c
ar -crs libnew.a printNew.o
#objcopy --redefine-sym printStr=printStrNew libnew.a
main/main.c:
void printOld();
void printNew();
int main()
{
printOld();
printNew();
return 0;
}
main/buildMain:
gcc main.c -L../old -L../new -lold -lnew
The linker error when I leave objcopy commented out is:
/usr/bin/ld: ../new/libnew.a(printNew.o): in function `printStr':
printNew.c:(.text+0x11): multiple definition of `printStr'; ../old/libold.a(printOld.o):printOld.c:(.text+0x11): first defined here
collect2: error: ld returned 1 exit status
The above example has the same kind of structure as the auto-generated code. I know that I can fix my problem by declaring the functions in the header files as static, but I'd rather not have to mess with the auto-generated code. It is very large and very unpleasant.

Related

Can GCC warn about undefined functions in libraries?

Consider the following test project:
test.h:
#ifndef TEST_H
#define TEST_H
void test1(int);
void test2(int);
#endif /* TEST_H */
text.c:
#include "test.h"
void test1(int x) { (void) x; }
Oops, I forgot to define test2()! I would like some kind of feedback when I do this, preferably refusal to compile although a warning at least would be nice. However GCC 10.2 (on Ubuntu 20.10) compiles it fine with no warnings:
gcc -Wall -Wextra -Wpedantic -std=c11 -o libtest.o -c test.c
I think I understand why: what if test2() is actually meant to come from another library, maybe a system library? Make it the problem of whichever program ends up linking everything into an executable! But I want to know about it before then. In this case, it's not declared in any included header file. It's not called anywhere. Can that be detected?
I've tried:
--no-undefined which resulted in gcc: error: unrecognized command-line option ‘--no-undefined’; did you mean ‘-Wno-undef’?
-Wno-undef - accepted but no warning
-z,defs - accepted but no warning
-Wimplicit-function-declaration - accepted but no warning
-Werror=missing-declarations - I know this is for the opposite situation but I was getting desperate.
This isn't possible, as no linking is performed at the stage of assembling a static library.
I'd suggest having a "test container" for your library. Set up your build system to build the test executable any time you are building the library. It could even just be a single .c file in the same directory as the library sources, but obviously not in the list of objects that are part of the library.
The test executable calls all of the functions that you wish to be entry points for the library.
Probably that is something you should be doing anyway in order to test the library's functionality before doing a release.

Unable to access own custom global function of shared library in linux from other module in C

I have downloaded libgcrypt library source code and
added my own customize function inside one particular file.
Although compilation/build process of customized shared library is successful, and both nm and objdump show
the customized function is global, it nonetheless shows an error (undefined reference) at linking time.
Here is what I have done:
inside /src/visibility.c file, I have added my custom function,
void __attribute__((visibility("default"))) MyFunction(void)
{
printf("This is added just for testing purpose");
}
build process
./configure --prefix=/usr/local --disable-ld-version-script
sudo make install
nm and objdump command find this custom function as global inside shared library.
nm /usr/local/lib/libgcrypt.so | grep MyFunction
000000000000fbf0 T MyFunction
objdump -t /usr/local/lib/libgcrypt.so | grep MyFunction
000000000000fbf0 g F .text 0000000000000013 MyFunction
Here is my sample code to access my custom function.
//gcrypt_example_test.c
#include <stdio.h>
#include <gcrypt.h>
#include <assert.h>
int main()
{
MyFunction();
return 0;
}
export LD_RUN_PATH=/usr/local/lib
gcc gcrypt_example_test.c -o test -lgcrypt
/tmp/ccA0qgAB.o: In function `main':
gcrypt_example_test.c:(.text+0x3a2): undefined reference to `MyFunction'
collect2: error: ld returned 1 exit status
Edit 1:
I tried all possible way to include function prototype declaration inside header file (/src/gcrypt.h) as follows:
void __attribute__((visibility("default"))) MyFunction(void);
... or:
extern void __attribute__((visibility("default"))) MyFunction(void);
... or:
extern void MyFunction(void);
... or:
void MyFunction(void);
I am still getting the same error (undefined reference) although no build error results in all above cases.
Why is this happening, and what mistake am I making?
Although other global functions which are part of standard shared library and defined inside visibility.c (nm also shows T for those functions) are accessible, why is my customized global function (MyFunction) of the shared library still inaccessible? Thanks!
Any link or explanation to resolve this error will be highly appreciable.
From the GCC documentation (emphasis mine):
Some linkers allow you to specify the path to the library by setting LD_RUN_PATH in your environment when linking.
But, from the GNU ld man page:
-rpath=dir
Add a directory to the runtime library search path. This is used
when linking an ELF executable with shared objects. All -rpath
arguments are concatenated and passed to the runtime linker,
which uses them to locate shared objects at runtime. The -rpath
option is also used when locating shared objects which are needed
by shared objects explicitly included in the link; see the
description of the -rpath-link option. If -rpath is not used
when linking an ELF executable, the contents of the environment
variable "LD_RUN_PATH" will be used if it is defined.
Note that there is no mention at all of the link time library search path.
You need to compile/link with /usr/local/lib in the link time library search path:
gcc gcrypt_example_test.c -o test -L/usr/local/lib -lgcrypt
most likely cause of the problem:
The header file for the library has not been updated to include the prototype for the new function
I don't understand the reason behind why it is working now, but not before. Anyway, I found the way to make the code working after adding customized function inside standard library. This post may help others in future.
I first locate libgcrypt.so and then remove all versions of libgcrypt.so
locate libgcrypt.so
sudo rm /usr/local/lib/libgcrypt.so
sudo rm /usr/local/lib/libgcrypt.so.20
sudo rm /usr/local/lib/libgcrypt.so.20.2.2
then I delete the libgcrypt folder (which I had extracted for building library) to start fresh.
Again, I follow these steps
Step 0 : extract libgcrypt source code
Step 1 : add my custom function, inside /src/visibility.c file
void __attribute__((visibility("default"))) MyFunction(void)
{
printf("This is added just for testing purpose");
}
Step 2 : build library
export LD_RUN_PATH=/usr/local/lib
./configure --prefix=/usr/local --disable-ld-version-script
sudo make install
Step 3: Open another terminal to compile
export LD_RUN_PATH=/usr/local/lib
gcc gcrypt_example_test.c -o test -lgcrypt
Step 4 : run
./test
This is added just for testing purpose
This is working fine now as expected.
What I noticed that __attribute__((visibility("default"))) in function definition and --disable-ld-version-script during build process is very important to make the customized function global, elimination of any makes the customized function local inside shared library(.so) file.
Below changes are working at my end
visibility.h
#include <cstdio>
void __attribute__((visibility("default"))) MyFunction(void);
visibility.cpp
#include "visibility.h"
void MyFunction(void)
{
printf("This is added just for testing purpose");
}
library build command
gcc -shared -o libtest.so -Wall -Werror -fpic -I. visibility.cpp
test.cpp
#include <stdio.h>
#include <gcrypt.h>
#include <assert.h>
#include "visibility.h"
extern void MyFunction();
int main()
{
MyFunction();
return 0;
}
exe build command
gcc test.cpp -o test -I. -L. -ltest -lstdc++
My gcc version is 4.4.7
And of-course I did not try and install the lib under /usr/local/lib but kept it local for quick testing.

dlsym-like functionality for non-dynamically-loaded code?

I know how to use dlsym() to find symbols keyed by a string - when these symbols are exported by a shared library which I've dlopen()ed. But - what about other code? Just object code I've linked statically. Is it possible to somehow lookup symbols?
Notes:
If it helps, make any reasonable assumptions about the compilation and linking process (e.g. which compiler, presence of debug info, PIC code etc.)
I'm interested more in a non-OS-specific solution, but if it matters: Linux.
Solutions involving pre-registration of functions are not relevant. Or rather, maybe they are, but I would rather avoid that.
You can indeed just use dlsym() for that purpose.. You just have to export all symbols to the dynamic symbol table. Link the binary with gcc -rdynamic for that.
Example:
#include <stdio.h>
#include <dlfcn.h>
void foo (void) {
puts("foo");
}
int main (void) {
void (*foo)(void) = dlsym(NULL, "foo");
foo();
return 0;
}
Compile with: gcc -rdynamic -O2 dl.c -o dl -ldl
$ ./dl
foo
$

Can I re-compile a file with new code?

I have a question. I was wondering if you could re-compile code with another piece of code. For example (theoretical):
main.c:
#include <stdio.h>
void showme();
int main()
{
showme();
}
void showme()
{
fprintf(stderr, "errtest, show me");
}
Compile this file to main. (So the main is compiled)
After this I want to add a piece of code.
addthis.c:
void test()
{
test();
}
Now I want to use the (compiled) main and re-compile it with addthis.c.
When running it (./mainWithAddthis) should show the print 2 times.
I hope I explained it clear. Anybody an idea?
You need a forward declaration for your void test() like you have one for the void showme(). Compile each .c file with -c (compile only) option:
gcc -c addthis.c -o addthis.o
gcc -c main.c -o main.o
Then link the two object files with:
gcc main.o addthis.o -o main
Then enjoy ./main :-)
Your first code will not compile since there's not definition of test();.
As I understand, you want to take the compiled main and add it with the code generated on addthis.o to create a 2nd application named mainWithAddthis. This is not possible!
You are either confused or trying to do some hardcore trick.
Building an executable is a two step process.
For every source file you specify (in your project/makefile), your compiler will build an object file
For every object file you specify (in your project/makefile), your linker will link them together and make your executable
One way to re-compile would be simply to re-build your entire project. You'd get more or less the same result.
But it sounds like what you want to do is recompile only the source file, addthis.c, then re-link the old version of main.o (the object file compiled for main.c) with the new version of addthis.o. How to do this is completely dependent on the compiler and build system you use.
Also, that solution will only work if you have main.o, addthis.c, and have the exact same compiler binaries/install, and compiler flags used to generate main.o. If this is all on your box, then you're probably okay.
If you only have the files addthis.c and main.exe, then no there is no portable way to do what you want.
You can't do what you are talking about after the fact without some hardcore time with a hex editor.
However, if you plan ahead and build it into your software, you can use dynamic loading to achieve the same effect, which is how a lot of software provides plugin functionality. Check out glib modules for a common way to do this in C.
main.c
void f();
int main()
{
f();
return 0;
}
addon1.c
#include <stdio.h>
void f()
{
printf("I am the ONE.\n");
}
addon2.c
#include <stdio.h>
void f()
{
printf("I am the TWO.\n");
}
Compilation
gcc -c main.c -o main.o
gcc -c addon1.c -o addon1.o
gcc -c addon2.c -o addon2.o
gcc main.o addon1.o -o main1
gcc main.o addon2.o -o main2
You will have ./main1 and ./main2 programs which will print ...ONE. and ...TWO..

Including source files in C

So I get the point of headers vs source files. What I don't get is how the compiler knows to compile all the source files. Example:
example.h
#ifndef EXAMPLE_H
#define EXAMPLE_H
int example(int argument); // prototype
#endif
example.c
#include "example.h"
int example(int argument)
{
return argument + 1; // implementation
}
main.c
#include "example.h"
main()
{
int whatever;
whatever = example(whatever); // usage in program
}
How does the compiler, compiling main.c, know the implementation of example() when nothing includes example.c?
Is this some kind of an IDE thing, where you add files to projects and stuff? Is there any way to do it "manually" as I prefer a plain text editor to quirky IDEs?
Compiling in C or C++ is actually split up into 2 separate phases.
compiling
linking
The compiler doesn't know about the implementation of example(). It just knows that there's something called example() that will be defined at some point. So it just generated code with placeholders for example()
The linker then comes along and resolves these placeholders.
To compile your code using gcc you'd do the following
gcc -c example.c -o example.o
gcc -c main.c -o main.o
gcc example.o main.o -o myProgram
The first 2 invocations of gcc are the compilation steps. The third invocation is the linker step.
Yes, you have to tell the compiler (usually through a makefile if you're not using an IDE) which source files to compile into object files, and the compiler compiles each one individually. Then you give the linker the list of object files to combine into the executable. If the linker is looking for a function or class definition and can't find it, you'll get a link error.
It doesn't ... you have to tell it to.
For example, whe using gcc, first you would compile the files:
gcc file1.c -c -ofile1.o
gcc file2.c -c -ofile2.o
Then the compiler compiles those files, assuming that symbols that you've defined (like your example function) exist somewhere and will be linked in later.
Then you link the object files together:
gcc file1.o file2.o -oexecutable
At this point of time, the linker looks at those assumtions and "clarifies" them ie. checks whether they're present. This is how it basically works...
As for your IDE question, Google "makefiles"
The compiler does not know the implementation of example() when compiling main.c - the compiler only knows the signature (how to call it) which was included from the header file. The compiler produces .o object files which are later linked by a linker to create the executable binary. The build process can be controlled by an IDE, or if you prefer a Makefile. Makefiles have a unique syntax which takes a bit of learning to understand but will make the build process much clearer. There are lots of good references on the web if you search for Makefile.
The compiler doesn't. But your build tool does. IDE or make tool. The manual way is hand-crafted Makefiles.

Resources