Is there a way to produce a C static library from Go code, but without Go runtime function definitions?
Rationale:
Project A creates a C static library with go build -buildmode=c-archive, libA.a .
Works well: project B uses pure C and is able to easily create an executable, statically linking with libA.a, all is fine.
Problem 1: project C happens to also use Go, but would like to use libA.a as a regular C library. Now it has a link problem: the Go runtime functions such as e.g. _cgo_panic are now defined both in project C runtime (as it uses Go) and in libA.a.
Problem 2: project D uses pure C, same as B. Yet it wants to use two different libraries from project A, e.g. libA.a and some libA2.a. Sadly, it does not link either, because Go runtime functions are now defined in both libA.a and libA2.a.
The problems faced by project C and project D could be easily resolved if project A could produce its libraries without the Go runtime definitions inside. Project C could just link with libA.a. Project D would link with libA.a, libA2.a and some libGo.a that would contain definitions of all the Go runtime stuff.
What I tried:
Using linker flags at 'project C' level, such as -Wl,--allow-multiple-definition. Now its build fails with a cryptic message 'function symbol table not sorted by program counter'.
Manually removing go.o from 'libA.a' (as it's just an "ar" archive): didn't work as 'go.o' also contained implementations of my exported functions, so I've removed too much.
Using go build -buildmode=c-shared. As expected, it produces a dynamic library which uses another format, so I could not directly use it as a static library.
Any solution at the client side (such as finding a proper way to ignore duplicate definitions at the link stage for project C) would be also considered a valid answer.
I can also accept a negative answer (no solution) if it provides enough evidence.
Update: see a related question Is there a way to include multiple c-archive packages in a single binary
With the current implementation it's not going to work to use -buildmode=c-archive multiple times and put the results into multiple shared libraries, as you've discovered. The essential problem is that there has to be only one Go runtime, but you have multiple runtimes. When using -buildmode=c-archive there's no way to isolate the different runtimes.
The -buildmode=c-shared libraries differ from buildmode=c-archive in that they are built with -Bsymbolic which forces all their local references to be local. The effect is that we have multiple Go runtimes, but they don't refer to each other so there is no confusion.
You could try adding -Wl,-Bsymbolic to build each shared library that includes Go code in c-archive if your C code doesn't mind being linked with -Bsymbolic.
I wish you luck.
Related
I'm actually put in a big project. My first step to understand the code was to search the main function so that I have a vision of the architecture.
What I discovered is that there is more than one main function. It's true that they are in different folders, but I don't understand how this application succeed to build. What I know is that the linker expects one main function (Entry point).
I believe it's too hard to understand the build process of the application, so I'm asking because for sure some of you have encountered this.
1 - Should I have theoretical background to understand this? If so, please suggest me articles, books, what ever you want.
2 - When do we have to use several main functions in one application?
You can't have multiple main functions for a single executable. There are several possibilities.
If doing a build builds only a single application, then only one of the main functions will be compiled. (Or none, if there's an option to build a library rather than an executable.) There are probably options that determine which one to build, depending on which variant of the application you want, the target system, or something else.
Or perhaps the application consists of multiple executables, with one main function for each one.
If running the build doesn't take too long, a trick I've used to determine which of several source files is actually compiled is to temporarily add #error directives, like:
#error "TEMPORARY: This is /full/path/to/source.cpp"
The resulting error message will tell you which source file was actually compiled. (You can also use #warning directives if your compiler supports them.)
Should i have theorical background to undestand this, suggest me articles book what ever you want
You need to have some understanding of what a build means but, more importantly, you have to understand the build process in your specific environment. You need to have an understanding of:
The list of build targets (executables and shared libraries)
The compiler settings used by them (you could be building DLLs for VS 2010 as well as VS 2012).
When do we have to use several main functions in one application?
When your build system builds multiple executables, you will need multiple main functions.
When your build system chooses file1.c or file2.c for building an executable depending on some settings, you might find main functions in file1.c as well file2.c. This will be a rather poor way to organize code but it is possible.
Trying to link an executable with multiple definitions for the same identifier (aka main in this case) will fail as the linker cannot select one or the other.
There's one possibility, though that allows you to have multiple main entry points, but assuring you only select one in the building process.
I'll illustrate this with a simple example: suppose you have a library (a .a static library) that has a module that includes a main definition. As library modules are selected at link time depending on the actual need of the identifiers at the time the library is linked, you can have a main definition in a module to supply when you don't have one, or that module is not linked in case you have a proper definition of it. This is exploited in some standard libraries like -ll (flex has a definition of main that calls the yylex() function), -ly (bison has a main definition that call the yyparse() routine) These modules are included if you don't have done it before.
But beware, as if you link your main function after the library -ll, for example, it will be included (as main was not resolved at the time he library was included, and then you included the second, duplicated entry, making the ld linker to complaint)
I'm compiling my shared library, which is meant to be used in another (main) shared library of mine.
So, the whole set-up is compiled using cmake roughly as follows:
For the "main" shared library:
ADD_LIBRARY(lib_outer SHARED
....
)
TARGET_LINK_LIBRARIES(lib_outer
lib_inner
...
)
For the additional shared library:
ADD_LIBRARY(lib_inner SHARED
....
)
Now, the lib_inner uses some functions defined and implemented in the second lib_outer, Which leads to Undefined symbols for architecture x86_64 linking error.
Can i somehow tell the cmake to ignore those?
I shall i use something like 'externalwhen using those functions insidelib_inner` ?
I guess the solution is simple, but i never came across such a problem.
After some discussion with colleagues, it seems that the only good option is to compile the two libraries simultaneously as a whole. Effectively combining the CMakeLists.txt files which before were used to produce two separate, but dependent libraries.
Using TinyCC in my C program lets me use C as a sort of scripting language, reload C files on the fly, and do a lot of fairly neat things... But, one thing is really bothering me. Linking.
I do my normal tcc_new, and tcc_set_output_type with TCC_OUTPUT_MEMORY, but if I don't include a lot of these:
tcc_add_symbol(tcc_ctx, "printf", &printf);
tcc_add_symbol(tcc_ctx, "powf", &powf);
tcc_add_symbol(tcc_ctx, "sinf", &sinf);
everything is very limited.
I want a way to automatically bring in all symbols in the host program. I don't want to have to manually link in every last function in libc, and libm. What mechanisms exist to facilitate auto linking, or adding of symbols. How can I use libm in my code without manually dropping in every last component.
I'm currently using GCC, but on another platform use Visual Studio to compile my program. I could switch entirely to TCC.
TCC comes with a rudimentary runtime library libtcc1. It includes basic functions like those you mention. Therefore, in most cases you can replace all your calls with a single tcc_add_library(tcc_ctx, "libtcc1.a").
libtcc1 is not complete, so you might have to add manually some functions.
Say I have 2 static libs
ex1.a
ex2.a
In both libs I will define 10 same functions
When Compiling a sample test code say "test.c" , I link with both static libs ex1.a and ex2.a
In "test.c" I will call only 3 functions, then I will get the
linker error "same symbols deifned in both ex1.a and ex2.a libraries" This is Ok.
My Question here is :
1. Why this error only display 3 functions as multiple defined.. Why not it list all 10 functions
In VC8 How can I list all multiple defined symbols without actualy calling that function in test code ...
Thanks,
Thats because, linker tries to resovle a symbol name, when it compiles and links a code which has the function call. Only when the code has some function calls, linker would try to resolve it in either the test code or the libraries linked along and thats when it would find multiple definitions. If no function called, then I guess no problem.
What you experience is the optimizing part of the linker: By default it won't include code that isn't referenced. The compiler will create multiple object files with most likely unresolved dependencies (calls that couldn't be satisfied by the code included). So the linker takes all object files passed and tries to find solutions for the unresolved dependencies. If it fails, it will check the available library files. If there are multiple options with the same exact name/signature it will start complaining cause it won't be able to decide which one to pick (for identical code this won't matter but imagine different implementations using different "behind the scenes" work on memory, such as debug and release stuff).
The only (and possibly easiest way) I could think of to detect all these multiple definitions would be creating another static library project including all source files used in both static libs. When creating a library the linker will include everything called or exported - you won't need specific code calling the stuff for the linker to see/include everything as long as it's exported.
However I still don't understand what you're actually trying to accomplish as a whole. Trying to find code shared between two libraries?
I'm planning to release some compiled code that shall be linked by client applications on MacOSX.
The distribution is some kind of code library and a set of header files defining the public interface for the library.The code is internally C++ but its public interface (i.e what's being shown in the headers) is completely C.
These are my requirements or atleast what I hope I can accomplish:
I want my library to be as agnostic
as possible for what version of OSX
and GCC the user is running. Having
separate libraries for 64 bit and 32
bit is okay though.
I want my library
to be loadable from languages that
supports loading C libraries such as
python or similar.
I want my
libraries internal symbols to be
isolated from the code it's being
linked into. I don't want to have
duplicate symbol errors because we
happen to name an internal function
in the same way. My C++ code is properly namespaced so this may not be as big of an issue though, but some of the libraries I depend on is C and can be an issue (see next point).
I want my library
dependencies to be safe. My library
depends on some libraries such as
libpng, boost and stl and I don't
want issues because some users don't
necessarily have all of them installed
or get problems because they have
been compiled with other flags or
have different versions than I have.
On Windows I use a DLL with an export library and link all my dependencies statically into the dll. It fulfills all the criteria above and if I can get the same result on OSX it would be great, however I've heard that dynamic libraries tend not to isolate symbols on mac in the same way.
Is there some kind of best practice for this on OSX?
A normal OS X .dylib pretty much satisfies your requirements, with the note that you will want to have an exports file that the linker uses to determine exactly which symbols are exported (to prevent leaking your internal symbols).
In order to make your own library dependencies safe, you will probably need to either include those libraries with yours or link them statically into your library.
edit: To answer your follow-up question of how to apply an exports file to a link command, the man page for ld has the following to say:
-exported_symbols_list filename
The specified filename contains a list of global symbol names
that will remain as global symbols in the output file. All
other global symbols will be treated as if they were marked
as __private_extern__ (aka visibility=hidden) and will not be
global in the output file. The symbol names listed in file-
name must be one per line. Leading and trailing white space
are not part of the symbol name. Lines starting with # are
ignored, as are lines with only white space. Some wildcards
(similar to shell file matching) are supported. The *
matches zero or more characters. The ? matches one charac-
ter. [abc] matches one character which must be an 'a', 'b',
or 'c'. [a-z] matches any single lower case letter from 'a'
to 'z'.
So, if your library had only two functions that you wanted to be public, lets call them foo and bar, and they were C functions (so the symbol names aren't mangled), your exports file (let's call it myLibrary.exports) would contain these two lines:
_foo
_bar
and maybe some comments, etc. When you do the final link step to build the library, you would pass the -exported_symbols_list myLibrary.exports flag to the linker. This has the additional benefit that the link will fail if you don't provide one of the exported symbols; this can catch a lot of "oops, I forgot to include that file in the build" mistakes.
You don't need to use the command-line tools to do all this, of course. In the build settings for a dynamic library in XCode, you will find Exported Symbols File (undefined by default); set it to the path to your exports file there and it will be passed to the linker.
The key term you need is 'framework'. You need to create a 'universal' framework that is self-contained. ('Universal' is Apple-ease for 'compile several times and package into one library.) It's not as straightforward as on Windows in terms of encapsulation, but the necessary linker options are there.