A library using off_t as a parameter for one function (seek). Library and application are compiled differently, one with large file support switched off, the other with large file support. This situation results in strange runtime errors, because both interpret off_t differently. How can the library check at runtime the size of off_t for the app? Or is there another solution, so that at least the user gets a meaningful error?
EDIT: The library (programmed in c and with autoconf) already exists and some third-party application use it. The library can be compiled with large file support (by default via AC_SYS_LARGEFILE). It is multiplatform, not only linux. How can be detected/prevented that installed applications will be broken by the change in LFS?
You could add an API to the library to return the sizeof(off_t) and then check it from the client. Alternatively the library could require every app to provide the API in order to successfully link:
library.c:
size_t lib_get_off_t_size (void)
{
return (sizeof(off_t));
}
client.c (init_function):
if (lib_get_off_t_size() != sizeof(off_t) {
printf("Oh no!\n");
exit();
}
If the library has an init function then you could put the check there, but then the client would have to supply the API to get the size of its off_t, which generally isn't how libraries work.
On Linux, when the library is compiled with large file support switched on, off_t is defined to be the same as off64_t. So, if the library is the one compiled with large file support, you could change its interface to always use off64_t instead of off_t (this might need _LARGEFILE64_SOURCE) and completely avoid the problem.
You can also check whether the application is being compiled with large file support or not (by seeing if _FILE_OFFSET_BITS is not defined or 32) and refuse compiling (with #error) if it's being compiled the wrong way; see /usr/include/features.h and Feature Test Macros.
As said before, the library will not be able to know how the application (being client to the library) is compiled, but the other way round has to work. Besides, I think you are talking about dynamic linking, since static linking certainly would not have different switches at same build time.
Similar to the already given answer by "Andrew Johnson", the library could provide a method for finding out whether it was compiled with large file support or not. Knowing that such build-time switches are mostly done with defines in C, this could be looking like this:
//in library:
BOOL isLargeFileSupport (void)
{
#ifdef LARGE_FILE_SUPPORT
return TRUE;
#else
return FALSE;
#endif
}
The application then knows how to handle file sizes reported by that lib, or can refuse to work when incompatible:
//in application
BOOL bLibLFS = lib_isLargeFileSupport();
BOOL bAppLFS = FALSE;
#ifdef LARGE_FILE_SUPPORT
bAppLFS = TRUE;
#endif
if (bLibLFS != bAppLFS)
//incompatible versions, bail out
exit(0);
Related
Does anyone know if there is a way to check at runtime or user compile time if a netcdf library build includes support for netcdf4?
There is a function nc_inq_libvers() but the version number itself is not useful because modern library versions can still be built with netcdf4 support turned off.
I simply want to be able to write code that checks if netcdf4 is supported, if it is then create a netcdf4 file, if not then create a netcdf3 file.
Ideal approach:
Investigate the library's source code for some #defines that you can check either at compile-time or at run-time (you may have a look also at the library's building process, in order to spot if such defines are made).
Then it's trivial to check for support; for instance, if the library defines NETCDF4_SUPPORT_ENABLED:
// compile-time check
#ifdef NETCDF4_SUPPORT_ENABLED
int main(void) {
// use netcdf4 format...
}
#else
int main(void) {
// use netcdf3 format...
}
#endif
More complex approach: (in case the library doesn't make such #defines)
You need to change your program's configuration/building procedure. When you'll build your program, a script written by you may check if the NETCDF library has been compiled with NETCDF4 support; in case support is enabled, your configuration script may add, for instance, parameter -DNETCDF4_SUPPORT_ENABLED to the command you use to compile your program, so this can be handled like in the previous solution.
Writing a script that detects if the NETCDF library has NETCDF4 support enabled is up to you, and different solutions work for different libraries.
A really (really) dirty hack, to only use in case of nuclear attack, is to write a small C program like this (that your configuration script will try to compile in order to detect NETCDF4 support):
#include <assert.h>
static_assert(sizeof( foo() ) != sizeof(int), "NETCDF4 support is not available");
// NOTE: replace `foo` with a library function that doesn't return `int` (nor another type with the same size of `int`)
// this code will fail to build if function `foo` is missing
Replace foo with the name of a library function that won't be included in the library if NETCDF4 support is disabled (also, the function's return type size must be different from int's size, e.g.: it may return long long, char, void, etc).
But I think a better solution may be found for the specific case of the NETCDF library (the previous hack is library-agnostic, but really dirty).
I have never been able to find any clear, concise and complete explanations on the creation and use of libraries in C; both dynamic/shared and static.
For a couple of reasons, I would like to implement some of my C header files for a project I am working on as libraries. Each library may have a global structure acting as a namespace for static functions in the headers, and an entry function to initialize this structure.
struct mylibrary {
int (*function1)(void);
int (*function2)(void);
} mylibrary;
static int my_library_function_1 (void) {
return 1;
}
static int my_library_function_2 (void) {
return 2;
}
void entry_mylibrary (void) {
mylibrary.function1 = my_library_function_1;
mylibrary.function2 = my_library_function_2;
}
As I understand it, this should work without a problem with a static library. All I have to do is link the library and call the function entry_mylibrary from elsewhere in my program. However, I would like my libraries to be self-initializing; that is, I want the entry function to be called when the code is loaded. As I understand it, that is the domain of shared libraries, which can declare entry points to be executed when the library is loaded.
So why don't I simply create dynamic libraries?
Well, I am worried about application security. If I am just linking a library at runtime, I don't see what is to prevent some end user from swapping my library for their own. Could I not create a new library declaring the same variables as the old one, and swap the definitions of these variables for my own? I don't see why the executable would care.
I suppose then that my questions are:
Is there some obscure method to self-initializing a static library?
How can I verify that the (shared) library I load is indeed MY library?
I am/will be working cross platform. 64 bit Windows and Linux. Apple products are to-be-determined, but not a priority.
I am using Visual Studio 2017 Community on Windows 10. I will be using GCC for linux. I will be implementing the same libraries on all platforms with platform compilers, e.g. VS 2017 on Windows and GCC on linux; I won't be cross-compiling platform-specific code.
Is there some obscure method to self-initializing a static library?
With GCC it is not obscure: just use __attribute__((constructor)). With MSVC it's a bit harder, but still possible: __attribute__((constructor)) equivalent in VC?
How can I verify that the (shared) library I load is indeed MY library?
With the above you won't need to use a shared library.
I know there are at least three popular methods to call the same function with multiple names. I haven't actually heard of someone using the fourth method for this purpose.
1). Could use #defines:
int my_function (int);
#define my_func my_function
OR
#define my_func(int (a)) my_function(int (a))
2). Embedded function calls are another possibility:
int my_func(int a) {
return my_function(a);
}
3). Use a weak alias in the linker:
int my_func(int a) __attribute__((weak, alias("my_function")));
4). Function pointers:
int (* const my_func)(int) = my_function;
The reason I need multiple names is for a mathematical library that has multiple implementations of the same method.
For example, I need an efficient method to calculate the square root of a scalar floating point number. So I could just use math.h's sqrt(). This is not very efficient. So I write one or two other methods, such as one using Newton's Method. The problem is each technique is better on certain processors (in my case microcontrollers). So I want the compilation process to choose the best method.
I think this means it would be best to use either the macros or the weak alias since those techniques could easily be grouped in a few #ifdef statements in the header files. This simplifies maintenance (relatively). It is also possible to do using the function pointers, but it would have to be in the source file with extern declarations of the general functions in the header file.
Which do you think is the better method?
Edit:
From the proposed solutions, there appears to be two important questions that I did not address.
Q. Are the users working primarily in C/C++?
A. All known development will be in C/C++ or assembly. I am designing this library for my own personal use, mostly for work on bare metal projects. There will be either no or minimal operating system features. There is a remote possibility of using this in full blown operating systems, which would require consideration of language bindings. Since this is for personal growth, it would be advantageous to learn library development on popular embedded operating systems.
Q. Are the users going to need/want an exposed library?
A. So far, yes. Since it is just me, I want to make direct modifications for each processor I use after testing. This is where the test suite would be useful. So an exposed library would help somewhat. Additionally, each "optimal implementation" for particular function may have a failing conditions. At this point, it has to be decided who fixes the problem: the user or the library designer. A user would need an exposed library to work around failing conditions. I am both the "user" and "library designer". It would almost be better to allow for both. Then non-realtime applications could let the library solve all of stability problems as they come up, but real-time applications would be empowered to consider algorithm speed/space vs. algorithm stability.
Another alternative would be to move the functionality into a separately compiled library optimised for each different architecture and then just link to this library during compilation. This would allow the project code to remain unchanged.
Depending on the intended audience for your library, I suggest you chose between 2 alternatives:
If the consumer of your library is guaranteed to be Cish, use #define sqrt newton_sqrt for optimal readability
If some consumers of your library are not of the C variety (think bindings to Dephi, .NET, whatever) try to avoid consumer-visible #defines. This is a major PITA for bindings, as macros are not visible on the binary - embedded function calls are the most binding-friendly.
What you can do is this. In header file (.h):
int function(void);
In the source file (.c):
static int function_implementation_a(void);
static int function_implementation_b(void);
static int function_implementation_c(void);
#if ARCH == ARCH_A
int function(void)
{
return function_implementation_a();
}
#elif ARCH == ARCH_B
int function(void)
{
return function_implementation_b();
}
#else
int function(void)
{
return function_implementation_c();
}
#endif // ARCH
Static functions called once are often inlined by the implementation. This is the case for example with gcc by default : -finline-functions-called-once is enabled even in -O0. The static functions that are not called are also usually not included in the final binary.
Note that I don't put the #if and #else in a single function body because I find the code more readable when #if directives are outside the functions body.
Note this way works better with embedded code where libraries are usually distributed in their source form.
I usually like to solve this with a single declaration in a header file with a different source file for each architecture/processor-type. Then I just have the build system (usually GNU make) choose the right source file.
I usually split the source tree into separate directories for common code and for target-specific code. For instance, my current project has a toplevel directory Project1 and underneath it are include, common, arm, and host directories. For arm and host, the Makefile looks for source in the proper directory based on the target.
I think this makes it easier to navigate the code since I don't have to look up weak symbols or preprocessor definitions to see what functions are actually getting called. It also avoids the ugliness of function wrappers and the potential performance hit of function pointers.
You might you create a test suite for all algorithms and run it on the target to determine which are the best performing, then have the test suite automatically generate the necessary linker aliases (method 3).
Beyond that a simple #define (method 1) probably the simplest, and will not and any potential overhead. It does however expose to the library user that there might be multiple implementations, which may be undesirable.
Personally, since only one implementation of each function is likley to be optimal on any specific target, I'd use the test suite to determine the required versions for each target and build a separate library for each target with only those one version of each function the correct function name directly.
Suppose I have an ELF binary that's dynamic linked, and I want to override/redirect certain library calls. I know I can do this with LD_PRELOAD, but I want a solution that's permanent in the binary, independent of the environment, and that works for setuid/setgid binaries, none of which LD_PRELOAD can achieve.
What I'd like to do is add code from additional object files (possibly in new sections, if necessary) and add the symbols from these object files to the binary's symbol table so that the newly added version of the code gets used in place of the shared library code. I believe this should be possible without actually performing any relocations in the existing code; even though they're in the same file, these should be able to be resolved at runtime in the usual PLT way (for what it's worth I only care about functions, not data).
Please don't give me answers along the line of "You don't want to do this!" or "That's not portable!" What I'm working on is a way of interfacing binaries with slightly-ABI-incompatible alternate shared-library implementations. The platform in question is i386-linux (i.e. 32-bit) if it matters. Unless I'm mistaken about what's possible, I could write some tools to parse the ELF files and perform my hacks, but I suspect there's a fancy way to use the GNU linker and other tools to accomplish this without writing new code.
I suggest the elfsh et al. tools from the ERESI (alternate) project, if you want to instrument the ELF files themselves. Compatibility with i386-linux is not a problem, as I've used it myself for the same purpose.
The relevant how-tos are here.
You could handle some of the dynamic linking in your program itself. Read the man page for dlsym(3) in particular, and dlopen(3), dlerror(3), and dlclose(3) for the rest of the dynamic linking interface.
A simple example -- say I want to override dup2(2) from libc. I could use the following code (let's call it "dltest.c"):
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <dlfcn.h>
int (*prev_dup2)(int oldfd, int newfd);
int dup2(int oldfd, int newfd) {
printf("DUP2: %d --> %d\n", oldfd, newfd);
return prev_dup2(oldfd, newfd);
}
int main(void) {
int i;
prev_dup2 = dlsym(RTLD_NEXT, "dup2");
if (!prev_dup2) {
printf("dlsym failed to find 'dup2' function!\n");
return 1;
}
if (prev_dup2 == dup2) {
printf("dlsym found our own 'dup2' function!\n");
return 1;
}
i = dup2(1,3);
if (i == -1) {
perror("dup2() failed");
}
return 0;
}
Compile with:
gcc -o dltest dltest.c -ldl
The statically linked dup2() function overrides the dup2() from the library. This works even if the function is in another .c file (and is compiled as a separate .o).
If your overriding functions are themselves dynamically linked, you may want to use dlopen() rather than trusting the linker to get the libraries in the correct order.
EDIT: I suspect that if a different function within the overridden library calls an overridden function, the original function gets called rather than the override. I don't know what will happen if one dynamic library calls another.
I don't seem to be able to just add comment to this question, so posting it as an "answer". Sorry about it, doing that just to hopefully help other folks who search an answer.
So, I seem to have similar usecase, but I explicitly find any modification to existing binaries unacceptable (for me), so I'm looking for standalone proxy approach: Proxy shared library (sharedlib, shlib, so) for ELF?
Windows provides only GetTickCount up to Windows Vista and starting from that OS also GetTickCount64. How can I make a C program compile with calls to different functions?
How can I make a C compiler check whether a function is declared in the included header files and compile different portions of code depending on whether that particular function is available or not?
#if ??????????????????????????????
unsigned long long get_tick_count(void) { return GetTickCount64(); }
#else
unsigned long long get_tick_count(void) { return GetTickCount(); }
#endif
Looking for a working sample file not just hints.
Edit: I tried the following using gcc 3.4.5 from MinGW on a (64-bit) Windows 7 RC but it didn't help. If this is a MinGW problem, how can I work around this issue?
#include <windows.h>
#if (WINVER >= 0x0600)
unsigned long long get_tick_count(void) { return 600/*GetTickCount64()*/; }
#else
unsigned long long get_tick_count(void) { return 0/*GetTickCount()*/; }
#endif
Compile time selection of an API based on the target Windows version locks the built executable to that version and newer. This is a common technique for open source, *nix targeted projects where it is assumed that the user will configure the source kit for his platform and compile clean to install.
On Windows, this is not the usual technique because it isn't generally safe to assume that an end user will have a compiler at all, let alone want to deal with the intricacies of getting a project to build.
Often, just using the older API that is present in all versions of Windows is a sufficient answer. This is also simple: you just ignore the existence of a new API.
When that isn't sufficient, you use LoadLibrary() and GetProcAddress() to attempt to resolve the new symbol at run time. If it can't be resolved, then you fall back to the older API.
Here's a possible implementation. It detects the first call, and at attempts to load the library and resolve the name "GetTickCount64". In all calls, if the pointer to resolved symbol is non-null, it calls it and returns the result. Otherwise, it falls back on the older API, casting its return value to match the wrapper's type.
unsigned long long get_tick_count(void) {
static int first = 1;
static ULONGLONG WINAPI (*pGetTickCount64)(void);
if (first) {
HMODULE hlib = LoadLibraryA("KERNEL32.DLL");
pGetTickCount64 = GetProcAddressA(hlib, "GetTickCount64");
first = 0;
}
if (pGetTickCount64)
return pGetTickCount64();
return (unsigned long long)GetTickCount();
}
Note that I used the ...A flavors of the API functions since it is known that the library name and the symbol name will only be ASCII... if using this technique to load symbols from an installed DLL that might be in a folder named with non-ASCII characters, then you will need to worry about using a Unicode build.
This is untested, your mileage will vary, etc...
You can achieve it using preprocessor definitions in Windows headers.
unsigned long long
get_tick_count(void)
{
#if WINVER >= 0x0600
return GetTickCount64();
#else
return GetTickCount();
#endif
}
The right way to deal with this kind of problems is to check whether the function is available, but this cannot be done reliably during the project compilation. You should add a configuration stage, which details depend on your build tool, both cmake and scons, two cross platforms build tools, provide the facilities. Basically, it goes like this:
/* config.h */
#define HAVE_GETTICKSCOUNT64_FUNC
And then in your project, you do:
#include "config.h"
#ifdef HAVE_GETTICKSCOUNT64_FUNC
....
#else
...
#endif
Although it looks similar to the obvious way, it is much more maintainable in the long term. In particular, you should avoid as much as possible to depend on versions, and check for capabilities instead. Checking for versions quickly leads to complicated, interleaved conditionals, whereas with the technique above, everything is controlled from one config.h, hopefully generated automatically.
In scons and cmake, they will have tests which are run automatically to check whether the function is available, and define the variable in the config.h or not depending on the check. The fundamental idea is to decouple the capability detection/setting from your code.
Note that this can handle cases where you need to build binaries which run on different platforms (say run on XP even if built on Vista). It is just a matter of changing the config.h. If dones poperly, that's just a matter of changing the config.h (you could have a script which generate the config.h on any platform, and then gather config.h for windows xp, Vista, etc...). I don't think it is specific to unix at all.
Previous answers have pointed out checking for the particular #define that would be present for your particular case. This answer is for a more general case of compiling different code whether a function is available or not.
Rather than trying to do everything in the C file itself, this is the sort of thing where configure scripts really shine. If you were running on linux, I would point you to the GNU Autotools without hesitation. I know there's ports available for Windows, at least if you're using Cygwin or MSYS, but I have no idea how effective they are.
A simple (and very very ugly) script that could work if you have sh handy (I don't have a Windows setup handy to test this on) would look something like this:
#!/bin/sh
# First, create a .c file that tests for the existance of GetTickCount64()
cat >conftest.c <<_CONFEOF
#include <windows.h>
int main() {
GetTickCount64();
return 0;
}
_CONFEOF
# Then, try to actually compile the above .c file
gcc conftest.c -o conftest.out
# Check gcc's return value to determine if it worked.
# If it returns 0, compilation worked so set CONF_HASGETTICKCOUNT64
# If it doesn't return 0, there was an error, so probably no GetTickCount64()
if [ $? -eq 0 ]
then
confdefs='-D CONF_HASGETTICKCOUNT64=1'
fi
# Now get rid of the temporary files we made.
rm conftest.c
rm conftest.out
# And compile your real program, passing CONF_HASGETTICKCOUNT64 if it exists.
gcc $confdefs yourfile.c
This should be easy enough to translate into your scripting language of choice. If your program requires extra include paths, compiler flags, or whatever, make sure to add the necessary flags to both the test compile and the real compile.
'yourfile.c' would look something like this:
#include <windows.h>
unsigned long long get_tick_count(void) {
#ifdef CONF_HASGETTICKCOUNT64
return GetTickCount64();
#else
return GetTickCount();
#endif
}
You're asking about C but the question is tagged C++ as well ...
In C++ you would use SFINAE technique, see similar questions:
Is it possible to write a template to check for a function's existence?
But use preprocessor directives in Windows when provided.
If your code is going to run on OSes berfore Vista, you can't just compile your calls down to GetTickCount64(), because GetTickCount64() doesn't exist on an XP machine.
You need to determine at runtime which operating system you are running and then call the correct function. In general both calls need to be in the code.
Now this may not be true in your case if you don't really need to be able to call either GetTickCount64() on Vista+ machines and GetTickCount() on XP- machines. You may be able to just call GetTickCount() no matter what OS you're running on. There is no indication in the docs that I have seen that they are removing GetTickCount() from the API.
I would also point out that maybe GetTickCount() isn't the right thing to use at all. The docs say it returns a number of milliseconds, but in reality the precision of the function isn't even close to 1 millisecond. Depending on the machine (and there's no way to know at runtime AFAIK) the precision could be 40 milliseconds or even more. If you need 1 millisecond precision you should be using QueryPerformanceCounter(). In fact, there's really no practical reason to not use QPC in all cases where you'd use GetTickCount() anyway.
G'day,
Isn't NTDDI_VERSION what you need to look for?
Update: You want to check if WINVER is 0x0600. If it is then you're running Vista.
Edit: For the semantic pecker head, I meant running a compiler in a Vista environment. The question only refers to compiling, the question only refers to header files which are only used at compile time. Most people understood that it was intended that you're compiling in a Vista env. The question made no reference to runtime behaviour.
Unless someone is running Vista, and compiling for windows XP maybe?
Sheesh!
HTH
cheers,
The Microsoft compiler will define _WIN64 when compiling for 64 bit machines.
http://msdn.microsoft.com/en-us/library/b0084kay%28VS.80%29.aspx
#if defined(_WIN64)
unsigned long long get_tick_count(void) { return GetTickCount64(); }
#else
unsigned long long get_tick_count(void) { return GetTickCount(); }
#endif
If you have to support pre-Vista, I would stick with only using GetTickCount(). Otherwise you have to implement runtime code to check the Windows version and to call GetTickCount() on pre-Vista versions of Windows and GetTickCount64() on Vista and later. Since they return different sized values (ULONGLONG v DWORD) you'll also need to have separate handling of what they return. Using only GetTickCount() (and checking for overflow) will work for both situations, whereas using GetTickCount64() when it's available increases your code complexity and doubles the amount of code you have to write.
Stick with using only GetTickCount() until you can be sure your app no longer has to run on pre-Vista machines.
Maybe it is a good replacement for GetTickCount()
double __stdcall
thetimer (int value)
{
static double freq = 0;
static LARGE_INTEGER first;
static LARGE_INTEGER second;
if (0 == value)
{
if (freq == 0)
{
QueryPerformanceFrequency (&first);
freq = (double) first.QuadPart;
}
QueryPerformanceCounter (&first);
return 0;
}
if (1 == value)
{
QueryPerformanceCounter (&second);
second.QuadPart = second.QuadPart - first.QuadPart;
return (double) second.QuadPart / freq;
}
return 0;
}