Is there a (if possible standard) way to annotate a compiler output (eg object file.o, library file.a, or shared library libfile.so) with user defined attributes, for example with a json string (embedded in some section of the object file) that contains metadata for the object:
eg: as compiler (clang/gcc/dmd/ldc/gdc etc), version (eg: 2.072), whether c++11 was used, optimization and compiler flags, etc.
eg: {"compiler" : "dmd", "version" : "2.072", "flags" : "-O -debug"}
Use cases:
either the linker ld or some other user defined tool could use this metadata to give informative warnings or errors when attempting to link together several libraries whose attributes don't match (eg when linking libraries compiles with -std=c++11 with others compiled without this)
this would provide much more user friendly error messages compared to the usual hard to debug undefined symbol errors, eg: undefined reference to llvm::Twine::str[abi:cxx11]() https://github.com/ldc-developers/ldc/issues/1928 or other errors that are due to compilation flag mismatches (eg missing -debug from https://github.com/rejectedsoftware/vibe.d/issues/740#issuecomment-268039416)
make it possible for tools to read the embedded metadata and find the most appropriate matching libraries to link when multiple ones are available
NOTE: the only thing I could find was https://gcc.gnu.org/onlinedocs/gccint/LTO-object-file-layout.html but not sure if that's relevant or how I would write (and later read) such user defined attributes:
Command line options (.gnu.lto_.opts): This section contains the command line options used to generate the object files
Related
I see in some driver source code the line:
MODULE_INFO(retpoline, "Y")
What is does?
This macro defines key-value pairs that are stored in the compiled module files. You can use the modinfo command to display them. For example, when you compile this module, modinfo my_module.ko will include a line
retpoline: Y
You can find the definition of the macro and its documentation (such as it is) by searching on an LXR interface. It works by defining a symbol in the .modinfo section of the object file.
This mechanism is used for various metadata used by the module loaded in the kernel and by userland tools such as modprobe. Standard metadata include aliases, license, version, etc.
The retpoline tag indicates that the module is built with support for a mitigation against Spectre. I don't know why driver source code would declare this explicitly: as far as I understand, this should automatically be inserted (via a header) when compiling with retpoline support: this is a compiler feature, not a property of the source code.
I want to compile the sqlite amalgamation to create a database which is protected by a password via user authentication.
I followed this tutorial: https://www.sqlite.org/howtocompile.html
And also the documentation by SQLite for the user_authentication: https://www.sqlite.org/src/doc/trunk/ext/userauth/user-auth.txt
When I try to compile it without the extra compile-time option "-DSQLITE_USER_AUTHENTICATION" and without adding the other documents it works. When I try to compile it with I get the error C2129 at sqlite.c and error C1083 at userauth.c
In this directory are the following files:
shell.c
sqlite3.c
sqlite3.h
sqlite3ext.h
sqlite3userauth.h
userauth.c
cl -DSQLITE_USER_AUTHENTICATION shell.c sqlite3.c userauth.c -Fesqlite3.exe
Following output:
shell.c
sqlite3.c
sqlite3.c(222878): error C2129: static function 'void sqlite3CryptFunc(sqlite3_context *,int,sqlite3_value **)' declared but not defined
sqlite3.c(16263): note: see declaration of 'sqlite3CryptFunc'
userauth.c
userauth.c(26): fatal error C1083: Cannot open include file: 'sqliteInt.h': No such file or directory
Generating Code...
In case there is something like C#'s db.SetPassword("MyPW") available in c, that would be perfect!
I followed [...] the documentation by SQLite for the user_authentication: https://www.sqlite.org/src/doc/trunk/ext/userauth/user-auth.txt
Well no, it doesn't look like you did. Those docs say
Activate the user authentication logic by including the
ext/userauth/userauth.c source code file in the build and adding the
-DSQLITE_USER_AUTHENTICATION compile-time option. The ext/userauth/sqlite3userauth.h header file is available to
applications to define the interface.
When using the SQLite amalgamation, it is sufficient to append the
ext/userauth/userauth.c source file onto the end of the amalgamation.
You are using the amalgamation, so you should append [the contents of] userauth.c to the amalgamation. That is, copy its contents to the end of sqlite3.c. From your directory listing and command line, it appears that you are instead attempting to build it as a separate source file, to be linked to the main one at the end. That's not equivalent, and in particular, it differs with respect to the effect on the scope of static functions and variables, which is exactly what your compiler is complaining about.
It's unclear whether -DSQLITE_USER_AUTHENTICATION should also be used with the amalgamation. A literal reading of the SQLite docs suggests not, but I would be inclined to guess that it actually is required either way if you want to enable the feature.
The error about the missing header is a little concerning, and it is possible that you will see it again. If you do, it may be sufficient to simply remove or comment out the corresponding #include directive, as all the needed declarations from that header, which is among the main sources, should already be included in the amalgamation.
I am trying to create an executable with .o ,.so, .a files.
Here is my build command:-
pkgs/gcc/v4.8.3/bin/gcc -L/usr/X11R6/lib -O2 -DUSE_FLEX -Wall -Wno-char-subscripts -fPIC -DLINUX -DG_DISABLE_CONST_RETURNS -fno-strict-aliasing -o ../build/kl/Release/test/bin/pure.exe -L../build/kl/Release/test/modules ../build/kl/Release/test/maker/constrfunc.TCL.o ../../build/kl/Release/test/maker/pvdbprocs.TCL.o .. ../build/kl/Release/test/maker/maker/memmaker.TCL.o .. ../build/kl/Release/test/maker/modules/libdenbase.a .. ../build/kl/Release/test/maker/guibase.o -litk3.2 -litcl4.0.0 -ltk8.3 -lcdnviptcl8.4 -litclstub4.0.0 -ldenbase -lglib-2.0 -ldenbase -lX11 -ldl -lm -lviputil -lvippli -lcdsCommonMT_sh -lpthread -L/home/dlb/extlibs/arm/lib
I have libraries which have definitions of the functions at path "-L/home/dlb/extlibs/arm/lib" . Still it is throwing error below.
Error:
../build/kl/Release/test/maker/guibase.o: In function `decodeAddrList':
tree234.c:(.text+0xc): undefined reference to `ptritclStubsPtr'
tree234.c:(.text+0x20): undefined reference to `ptrlitclStubsPtr'
tree234.c:(.text+0x12c): undefined reference to `ptrlitclStubsPtr'
tree234.c:(.text+0x140): undefined reference to `ptrlitclStubsPtr'
I have the symbol in library which is at path /home/dlb/extlibs/arm/lib :-
Command:-
readelf -s libitcl4.0.0.so | grep ptrlitclStubsPtr
348: 0000000000060f10 8 OBJECT LOCAL DEFAULT 24 ptrlitclStubsPtr
Am I missing some thing here?
Note, OP has provided more info in a repost of the same question;
quote from comment by OP:
"Due to some privacy issues...I am renaming the symbols.. It was edited by me... It was typo...I just corrected it...:)"
I.e. the identifiers in the error message and the grep line and the output of the grep line have been manually altered.
It looks like the functions you're asking about are local to the file where they were defined. That is, it looks like they were explicitly intended not (and not even allowed) to be called externally.
That is, somewhere within the source for the shared library libitcl4.0.0.so probably appears a declaration like:
static tclStubs *ptrlitclStubsPtr;
That keyword static indicates that the visibility of the resulting symbol ptrlitclStubsPtr is confined to its own source file.
I infer all of this from the fact that your reported readelf output includes the line
348: 0000000000060f10 8 OBJECT LOCAL DEFAULT 24 ptrlitclStubsPtr
That flag LOCAL indicates that the symbol is local. If it were global, intended (and able) to be called externally, the flag GLOBAL would appear instead.
Why are variables made private (static) like this, so you can't use them? It's a software engineering thing, "information hiding", intended to reduce the "width" of the interface between you and a library like libitcl4. Symbols that are private are more intimately tied to the implementation decisions within libitcl4, decisions which are not supposed to be visible to, or any concern of, the callers. It's thought that, if callers were to have access to those symbols, callers would also have to know other implementation details, meaning that the author of libitcl4 would then be unable to change those implementation details without breaking (invalidating) the calling code. So to prevent that situation, the choice is usually made to make it impossible for the caller to become dependent in this way.
In this situation you basically have three paths forward:
Remove the static tags from the variable declarations in the sources to libitcl4.0.0.so. (This obviously requires that you have access to the sources to libitcl4.0.0.so, and the ability to rebuild it. It's also probably a very bad idea. As I've explained, those symbols were probably made static for a good reason.)
Add a new function within libitcl4.0.0.so which does whatever you need done, and which, by virtue of its placement within the same source file, does have access to those symbols. (This, too, requires that you have access to and the ability to rebuild ``libitcl4.0.0.so`.)
Find some other way of doing whatever you need done, using the existing public facilities of libitcl4.0.0.so.
Can somebody explain what the GNU ld option --undefined does?
Working on a LiteOS project. The app is linked with many -u options. For example -utask_shellcmd.
The GNU linker manual for --undefined=symbol simply says:
Force symbol to be entered in the output file as an undefined symbol. Doing this may, for example, trigger linking of additional modules from standard libraries.
So the symbol will be included in the output file as an undefined. What if the symbol is already defined in one of the linked obj files? If it is really undefined, when the linking of additional modules will happen and how does that happen?
The -u option is only relevant when archive (.a) libraries are involved (maybe also .so libraries with --as-needed in effect).
Unlike individual object files (.o) on the linking command line, which are all linked in the order in which they appear, object files from an archive library are only linked when they satisfy one or more undefined symbol references at the point they appear in the link command line order. Once once .o file from the archive is pulled into the link, the process is repeated recursively, so that if it introduces more undefined symbol references, other object files from the same (or later) archives will be pulled in to satisfy them.
Using -u allows you to cause a particular symbol (and, indirectly, all dependencies of the object file it was defined in) to be pulled into the link. Of course you could just put all .o files on the command line directly, without using any archive libraries, but by using libraries you can avoid linking unused object files (this is especially useful if large parts of the code may be unused depending on build-time-configurable settings in other files!) while getting the ones you need.
I'm pretty new to working with libraries and I'm in the process of trying to understand some specifics regarding static libraries and object files.
Summary
The behavior I'm noticing is that I can link several objects to make an executable with no problem, but if I take an intermediate step of combining those objects into static libraries, I cannot link those static libraries to make an executable without additionally specifying the needed C Run-time library in the link command.
Also, or the record, I'm doing the compiling/linking with Visual Studio 2010 from the command line. More details of the process I'm following are below.
First, let's say I have four source files in a project: main.c, util1.c, util2.c, and util3.c.
What works
I can compile these sources with the following command:cl -c main.c util1.c util2.c util3.cAs a result, I now have four object files: main.obj, util1.obj, util2.obj, and util3.obj. These object files each contain a DEFAULTLIB statement intended to inform the linker that it should additionally check the static C Run-time library libcmt.lib for any unresolved external dependencies in these object files when linking them.
I can create an executable named "app_objs.exe" by linking these objects with the following command:
link -out:app_objs.exe main.obj util1.obj util2.obj util3.obj
As mentioned in step 1, the linker used the runtime library due to the compiler's step of adding a default library statement to the objects.
Where I'm confused
Let's say I want to have an intermediate step of combining these objects into static libraries, and then linking those resulting LIB files to create my executable. First, I can create these libraries with the following commands:
link -lib -out:main.lib main.obj
link -lib -out:util.lib util1.obj util2.obj util3.obj
Now, my original thought was that I could simply link these libraries and have the same executable that I created in step 2 of "What works". I tried the following command and received linker error LNK1561, which states that an entry point needs to be specified:
link -out:app_libs.exe main.lib util.lib
From Microsoft's documentation, it is evident that linking libraries without any object files may require entry points to be specified, so I modified the command to set the subsystem as "console" to specify that the executable in intended to be a console application (which seems to imply certain entry points, thereby resolving that error):link -out:app_libs.exe -subsystem:console main.lib util.libUnfortunately, now I get a linker error stating that mainCRTStartup is an unresolved external symbol. I understand that this is defined in the C runtime library, so I can resolve this issue by manually specifying that I want to link against libcmt.lib, and this gives me a functioning executable:link -out:app_libs.exe -subsystem:console main.lib util.lib libcmt.lib
What I'm not understanding is why the default library info that the compiler placed in each object file couldn't be used to resolve the dependency on libcmt.lib. If I can link object files without explicitly stating I want libcmt.lib, and I created static libraries that are containers for the object files, why can't I link those static libraries without having to explicitly state that I want libcmt.lib? Is this just the way things are, or is there some way I could create the static libraries so that the linker will know to check for unresolved symbols in the runtime library?
Thanks for your help. If I have some fundamentally incorrect ideas here, I'd love suggestions on good references to learn all of this correctly.
Well the answer to your misunderstanding is that .lib files are often a product in themselves, and the compiler can't make those assumptions safely. That's what "external" is for.
If I produce binaries for someone's platform because its users are totally helpless, and they want/need static linkage, I have to give them foo.h and libfoo.lib without tying them to a specific runtime entry point. They may very well have defined their own entry point already for their final product, whether DLL or EXE.
You either want the runtime, or you want your own .obj that contains your entry point. Be warned that declaring and defining mainCRTStartup on your own may mean you're not executing important instructions for the target platform.