I'm pretty new to working with libraries and I'm in the process of trying to understand some specifics regarding static libraries and object files.
Summary
The behavior I'm noticing is that I can link several objects to make an executable with no problem, but if I take an intermediate step of combining those objects into static libraries, I cannot link those static libraries to make an executable without additionally specifying the needed C Run-time library in the link command.
Also, or the record, I'm doing the compiling/linking with Visual Studio 2010 from the command line. More details of the process I'm following are below.
First, let's say I have four source files in a project: main.c, util1.c, util2.c, and util3.c.
What works
I can compile these sources with the following command:cl -c main.c util1.c util2.c util3.cAs a result, I now have four object files: main.obj, util1.obj, util2.obj, and util3.obj. These object files each contain a DEFAULTLIB statement intended to inform the linker that it should additionally check the static C Run-time library libcmt.lib for any unresolved external dependencies in these object files when linking them.
I can create an executable named "app_objs.exe" by linking these objects with the following command:
link -out:app_objs.exe main.obj util1.obj util2.obj util3.obj
As mentioned in step 1, the linker used the runtime library due to the compiler's step of adding a default library statement to the objects.
Where I'm confused
Let's say I want to have an intermediate step of combining these objects into static libraries, and then linking those resulting LIB files to create my executable. First, I can create these libraries with the following commands:
link -lib -out:main.lib main.obj
link -lib -out:util.lib util1.obj util2.obj util3.obj
Now, my original thought was that I could simply link these libraries and have the same executable that I created in step 2 of "What works". I tried the following command and received linker error LNK1561, which states that an entry point needs to be specified:
link -out:app_libs.exe main.lib util.lib
From Microsoft's documentation, it is evident that linking libraries without any object files may require entry points to be specified, so I modified the command to set the subsystem as "console" to specify that the executable in intended to be a console application (which seems to imply certain entry points, thereby resolving that error):link -out:app_libs.exe -subsystem:console main.lib util.libUnfortunately, now I get a linker error stating that mainCRTStartup is an unresolved external symbol. I understand that this is defined in the C runtime library, so I can resolve this issue by manually specifying that I want to link against libcmt.lib, and this gives me a functioning executable:link -out:app_libs.exe -subsystem:console main.lib util.lib libcmt.lib
What I'm not understanding is why the default library info that the compiler placed in each object file couldn't be used to resolve the dependency on libcmt.lib. If I can link object files without explicitly stating I want libcmt.lib, and I created static libraries that are containers for the object files, why can't I link those static libraries without having to explicitly state that I want libcmt.lib? Is this just the way things are, or is there some way I could create the static libraries so that the linker will know to check for unresolved symbols in the runtime library?
Thanks for your help. If I have some fundamentally incorrect ideas here, I'd love suggestions on good references to learn all of this correctly.
Well the answer to your misunderstanding is that .lib files are often a product in themselves, and the compiler can't make those assumptions safely. That's what "external" is for.
If I produce binaries for someone's platform because its users are totally helpless, and they want/need static linkage, I have to give them foo.h and libfoo.lib without tying them to a specific runtime entry point. They may very well have defined their own entry point already for their final product, whether DLL or EXE.
You either want the runtime, or you want your own .obj that contains your entry point. Be warned that declaring and defining mainCRTStartup on your own may mean you're not executing important instructions for the target platform.
Related
I did setup a C project with Eclipse Photon (4.8.0) for developing a program for the ESP-32. I did configure the IDE according to this official setup instructions.
Flashing the ESP-32 works fine. But as soon as I try to include header files from a sub folder, I run into troubles. I have set up a very simple project to illustrate the issue. The project consists of main.c, base/test.h and base/test.c, whereas the test.h and test.c files only contain one function with the signature void function1(void);.
When I try to call function1() in main.c, I get this error in main.c:
Undefined reference to function1()
Please compare to the attached screenshot, where everything is depicted.
How to solve this issue?
This is not a compiler, but rather a linker error.
Note, with #includeing a header file, you only make the external function known to the compiler. You also need to link to the external function during the linking stage. Make sure you include the compiled object file that contains function1 into the link.
Seems like you need to do proper linking.
If you are linking with a library, you need to specify:
The name of the library: Project\Settings\C C++ General\Paths and Symbols\Libraries
Location where the linker should search for this library:
Project\Settings\C C++ General\Paths and Symbols\Library Paths
Important: see Note.
If you are linking with object files, add those to:
Project\Settings\C C++ Build\Settings\Linker\Miscellaneous\Other objects
Note:
If your library name is, for example, libsomething.a, than you need to specify only something as the name; so omit lib prefix and .a suffix.
If your library is not prefixed with lib, then you need to add its name prefixed with :. For example, something.a should be added as :something.a.
Can somebody explain what the GNU ld option --undefined does?
Working on a LiteOS project. The app is linked with many -u options. For example -utask_shellcmd.
The GNU linker manual for --undefined=symbol simply says:
Force symbol to be entered in the output file as an undefined symbol. Doing this may, for example, trigger linking of additional modules from standard libraries.
So the symbol will be included in the output file as an undefined. What if the symbol is already defined in one of the linked obj files? If it is really undefined, when the linking of additional modules will happen and how does that happen?
The -u option is only relevant when archive (.a) libraries are involved (maybe also .so libraries with --as-needed in effect).
Unlike individual object files (.o) on the linking command line, which are all linked in the order in which they appear, object files from an archive library are only linked when they satisfy one or more undefined symbol references at the point they appear in the link command line order. Once once .o file from the archive is pulled into the link, the process is repeated recursively, so that if it introduces more undefined symbol references, other object files from the same (or later) archives will be pulled in to satisfy them.
Using -u allows you to cause a particular symbol (and, indirectly, all dependencies of the object file it was defined in) to be pulled into the link. Of course you could just put all .o files on the command line directly, without using any archive libraries, but by using libraries you can avoid linking unused object files (this is especially useful if large parts of the code may be unused depending on build-time-configurable settings in other files!) while getting the ones you need.
When I compile with Compaq Visual Fortran, I get these errors (when it starts the linker process) that should be located in a .lib file I thought I added to the workspace:
X30XFULL.OBJ : error LNK2001: unresolved external symbol _BCON#4
X30XFULL.OBJ : error LNK2001: unresolved external symbol _RCON#16
According to where I've googled about, it looks like Compaq Visual Fortran (Version 6, FYI) can't find the library files...
My main question is, how do I use them? Could there possibly be other missing files? Here is what I've tried:
Right Click->Adding the files in the FileView window
Going under Project->Settings, Clicking the Link tab, and under the input category, adding the library files under object/library modules (bprop.lib rprop.lib) and having the Additional Library Path point to where the files are. I also did this under the Resources tab and where it says "Additional Resource Include Directories," I put the directory of where these files were located.
To cover my bases, I also put these files in where the project workspace file, where the compiled executable file would be generated, and pretty much any place I could think of that CVF might possibly look to in order to find these files.
A little background:
I have this Fortran executable that was last compiled in the 90s. From my research, it's a 16-bit compiled one which won't work on a 64-bit machine.
The original code has, at least from what we can gather, 16 bit compiled libraries. Without the original compiler, we can't figure out how to look at or use them. We also have a (semi)equivalent library file that's actually a .FOR file. For all we know, the BPROP.FOR and BPROP.LIB could be the same file (they were found in the same source code area). If we use the BPROP.FOR file, the program can compile, but we are having issues with results that we've traced down to information that is used/gathered/processed in that file.
However, we do have 32-bit versions of (what we think) are the same .lib files. So, we're trying to use that, which is what is being used to compile the Fortran executable which results in the errors above.
Found the answer, at least for me. I don't know how easy it'll be to extrapolate if anyone else finds these answers, but this is how I solved it.
With the old Fortran libraries, all I had to do was add them to the FileList view that has all of the different fortran files (.FOR, etc). I did not have to add these libraries in the settings like I mentioned, but that will work as well. Other then that, I didn't need to add any extra declarations or anything similar.
What we did find out is that the function in question (BCON and RCON) that calls those .LIB files required an additional argument. The only way I found this out was examining other source code that used those libaries, so if anyone is stuck like I was, that would be a good place to start. Alternatively, if you can read the .lib file in a hex editor, you can kind of make out functions and their arguments.
Of course, if you have the original source code for said arguments, that's even better. :)
If I have two libraries, A.lib and B.lib, both of which export foo and bar, how do I tell the linker to use the symbol foo from A.lib and the symbol bar from B.lib?
You cannot. For your example you can have definitions from foo.lib or bar.lib but not both (especially if you cannot recompile the libraries and set symbol visibility so that only the symbols you want are exported). The order you link them against your application will depend on which library's definitions are used (you'll have to use the scientific method, I think the first one linked wins). Niklas Hansson's answer is a great way to do this dynamically but it seems you don't want to modify the original application, either, to dynamically pick/choose what symbols to take out of the libraries.
If you really wanted to you could mangle the symbol tables with a hex editor so that the symbols you don't want exported have different names (hacky, but it would work). I know on Linux there is a tool called objcopy that would let you do this (not sure about Windows).
You can use LIB.EXE /EXTRACT ... to extract only the object files you want to use, and link those files into your own application.
Or you may use LIB to create one new library containing the elements you need:
First, use /REMOVE on A.LIB to remove bar.obj:
LIB.EXE /OUT:ANOBAR.LIB /REMOVE:bar.obj A.LIB
Then combine A.LIB and B.LIB, and make sure to use ANOBAR.LIB as the last on the command line to ensure its foo.obj is used instead of B.LIB's:
LIB.EXE /OUT:COMBINED.LIB B.LIB ANOBAR.LIB
Details are found here: Managing a library, especially the paragraph:
You can use LIB [...] To replace a library member with a new object, specify the library containing the member object to be replaced and the file name for the new object (or the library that contains it). When an object that has the same name exists in more than one input file, LIB puts the last object specified in the LIB command into the output library. When you replace a library member, be sure to specify the new object or library after the library that contains the old object.
I didn't test the command lines given, but I've used similar ones extensively in the past.
If you are using dynamic libraries, you could use dynamic loading and pick foo from A and bar from B when loading.
Same source, all that, just want a static and shared version both. Easy to do?
Yes, it's moderately easy. Just use two "add_library" commands:
add_library(MyLib SHARED source1.c source2.c)
add_library(MyLibStatic STATIC source1.c source2.c)
Even if you have many source files, you can place the list of sources in a Cmake variable, so it's still easy to do.
On Windows you should probably give each library a different name, since there is a ".lib" file for both shared and static. But on Linux and Mac you can even give both libraries the same name (e.g. libMyLib.a and libMyLib.so):
set_target_properties(MyLibStatic PROPERTIES OUTPUT_NAME MyLib)
But I don't recommend giving both the static and dynamic versions of the library the same name. I prefer to use different names because that makes it easier to choose static vs. dynamic linkage on the compile line for tools that link to the library. Usually I choose names like libMyLib.so (shared) and libMyLib_static.a (static). (Those would be the names on linux.)
Since CMake version 2.8.8, you can use "object libraries" to avoid the duplicated compilation of the object files. Using Christopher Bruns' example of a library with two source files:
# list of source files
set(libsrc source1.c source2.c)
# this is the "object library" target: compiles the sources only once
add_library(objlib OBJECT ${libsrc})
# shared libraries need PIC
set_property(TARGET objlib PROPERTY POSITION_INDEPENDENT_CODE 1)
# shared and static libraries built from the same object files
add_library(MyLib_shared SHARED $<TARGET_OBJECTS:objlib>)
add_library(MyLib_static STATIC $<TARGET_OBJECTS:objlib>)
From the CMake docs:
An object library compiles source files but does not archive or link
their object files into a library. Instead other targets created by
add_library() or add_executable() may reference the objects using an
expression of the form $<TARGET_OBJECTS:objlib> as a source, where
objlib is the object library name.
Simply put, the add_library(objlib OBJECT ${libsrc}) command instructs CMake to compile the source files to *.o object files. This collection of *.o files is then referred to as $<TARGET_OBJECT:objlib> in the two add_library(...) commands that invoke the appropriate library creation commands that build the shared and static libraries from the same set of object files. If you have lots of source files, then compiling the *.o files can take quite long; with object libraries you compile them only once.
The price you pay is that the object files must be built as position-independent code because shared libraries need this (static libs don't care). Note that position-independent code may be less efficient, so if you aim for maximal performance then you'd go for static libraries. Furthermore, it is easier to distribute statically linked executables.
There is generally no need to duplicate ADD_LIBRARY calls for your purpose. Just make use of
$> man cmake | grep -A6 '^ *BUILD_SHARED_LIBS$'
BUILD_SHARED_LIBS
Global flag to cause add_library to create shared libraries if on.
If present and true, this will cause all libraries to be built shared unless the library was
explicitly added as a static library. This variable is often added to projects as an OPTION
so that each user of a project can decide if they want to build the project using shared or
static libraries.
while building, first (in one out-of-source directory) with -DBUILD_SHARED_LIBS:BOOL=ON, and with OFF in the other.
Please be aware that previous answers won't work with MSVC:
add_library(test SHARED ${SOURCES})
add_library(testStatic STATIC ${SOURCES})
set_target_properties(testStatic PROPERTIES OUTPUT_NAME test)
CMake will create test.dll together with test.lib and test.exp for shared target. Than it will create test.lib in the same directory for static target and replace previous one. If you will try to link some executable with shared target it will fail with error like:
error LNK2001: unresolved external symbol __impl_*.`.
Please use ARCHIVE_OUTPUT_DIRECTORY and use some unique output directory for static target:
add_library(test SHARED ${SOURCES})
add_library(testStatic STATIC ${SOURCES})
set_target_properties(
testStatic PROPERTIES
OUTPUT_NAME test
ARCHIVE_OUTPUT_DIRECTORY testStatic
)
test.lib will be created in testStatic directory and won't override test.lib from test target. It works perfect with MSVC.
It's possible to pack eveything in the same compilation breath, as suggested in the previous answers, but I would advise against it, because in the end it's a hack that works only for simple projects. For example, you may need at some point different flags for different versions of the library (esp. on Windows where flags are typically used to switch between exporting symbols or not). Or as mentionned above, you may want to put .lib files into different directories depending on whether they correspond to static or shared libraries. Each of those hurdles will require a new hack.
It may be obvious, but one alternative that has not been mentionned previously is to make the type of the library a parameter:
set( ${PROJECT_NAME}_LIBTYPE CACHE STRING "library type" )
set_property( CACHE ${PROJECT_NAME}_LIBTYPE PROPERTY STRINGS "SHARED;STATIC" )
add_library( ${PROJECT_NAME} ${PROJECT_NAME}_LIBTYPE ${SOURCE_FILES} )
Having shared and static versions of the library in two different binary trees makes it easier to handle different compilation options. I don't see any serious drawback in keeping compilation trees distinct, especially if your compilations are automated.
Note that even if you intend to mutualize compilations using an intermediate OBJECT library (with the caveats mentionned above, so you need a compelling reason to do so), you could still have end libraries put in two different projects.