DLL dependency to static library - c

I currently build a purely static library MainLib for our customers that contains all symbols so that they can intrgrate it into their program. For several reasons, I now need to deliver a DLL version of MainLib that contains parts of the symbols alongside a static library FeatureLib that contains the remaining symbols. One reason is that we want to avoid bad guys using our software by simply stealing the DLL that is provided via the program of our customer. This wouldn't work if parts of the symbols are integrated within the calling software via a static library. The user of the package shall only be able to use the DLL if he added the symbols of FeatureLib into his application.
For Linux, I can make this work like a charm,i.e. the symbol doFeature() is not within libMainLib.so, but I don't succeed on this for Windows.
CMakeLists.txt:
cmake_minimum_required(VERSION 3.0)
project(MainLib)
add_library(FeatureLib STATIC src/FeatureLib.c)
target_include_directories(FeatureLib PUBLIC include
PRIVATE src)
add_library(MainLib SHARED src/MainLib.c)
target_include_directories(MainLib PUBLIC include
PRIVATE src)
# I don't want to include symbols from FeatureLib into shared MainLib
#target_link_libraries(MainLib PRIVATE FeatureLib)
add_executable(MainLibDemo src/demo.c)
target_link_libraries(MainLibDemo MainLib FeatureLib) #resolve symbol doFeature()
FeatureLib.h:
extern int doFeature(int input);
MainLib.h:
extern __declspec(dllexport) int MainLib(int input);
FeatureLib.c:
#include "FeatureLib.h"
int doFeature(int input) {return 4;}
MainLib.c:
#include "FeatureLib.h"
#include "MainLib.h"
__declspec(dllexport) int MainLib(int input)
{
if (input > 2) {
return doFeature(input);
} else {
return doFeature(0);
}
}
demo.c:
#include <stdio.h>
#include <stdlib.h>
#include "MainLib.h"
int main(int argc, char **argv)
{
if(argc > 1)
return MainLib(atoi(argv[1]));
else
return 0;
}
With this, I get the following compilation error:
"C:\Daten\tmp\DemoProject\simple\build\ALL_BUILD.vcxproj" (Standardziel) (1) ->
"C:\Daten\tmp\DemoProject\simple\build\MainLib.vcxproj" (Standardziel) (4) ->
(Link Ziel) ->
MainLib.obj : error LNK2019: unresolved external symbol _doFeature referenced in function _MainLib [C:\Daten\tmp\DemoProject\simple\build\MainLib.vcxproj]
C:\Daten\tmp\DemoProject\simple\build\Debug\MainLib.dll : fatal error LNK1120: 1 unresolved externals [C:\Daten\tmp\DemoProject\simple\build\MainLib.vcxproj]
0 Warnung(en)
2 Fehler
Is this even possible with Windows? What do I have to do to make it work and how can I verify it other than not linking FeatureLib to MainLibDemo. Any ideas are very welcome.
Kind regards,
Florian

The way you do it under Linux will not work under Windows
because dynamic linking works differently here.
Here is one strategy that could work.
In MainLib.dll code, instead of directly calling doFeature
you need to define a global pointer variable of proper function
pointer type and use it to call the function.
This will allow to build MainLib.dll without errors.
Now you need to set this pointer variable. One way would be:
Add exported function to MainLib.dll that takes pointers
to all functions that the DLL needs from the executable.
In FeatureLib.lib code add an initialisation function
that the application will need to call before using
your DLL which will pass pointers to its peers to the DLL.
This is basically the way most programs with plugins use to
give the plugins access to their facilities.
Another way would be to (Warning! I have not tested this specific
solution):
Declare the functions in FeatureLib.lib as exported
with __declspec(dllexport). This way they will be exported
from executable.
In MainLib.dll before first using the pointers use
GetModuleHandle and GetProcAddress to obtain the pointers.
It would best be done in some initialisation function for the
library. Otherwise you need to take care to avoid race conditions.
Hope this will help.
Though I do not think your copy protection scheme will work.
Andrew Henle is right in his comment: it is not hard
to extract the needed code from one executable and include it
in another.

Related

Optional dynamic library

Background
Trying to profile an executable, I experimented the profiler Intel VTune and I learn that there is an API library (ITT) that provide utility to start/stop profiling. Its basic functions __itt_resume() and __itt_pause(). What triggers me is that the library is optional, i.e. if the runtime library of ITT is not loaded, these functions are basically noops.
Optional library?
I want to know (first of all on Linux)
Does a process checks that the dynamic library he is linking to is loaded when he starts or when each symbol, or the first symbol of the library is called at runtime (i.e. lazy initialization)? I think on Windows it's at startup because of can't find XXX.dll messages, but I am not sure on Linux. Also, with the example, I don't get any compilation & execution issues even if the symbol is not defined in some_process.c.
How to implement this on Linux? Looking at the Github repo of ITT, among many macro trickery, I feel like the key is here:
#define ITTNOTIFY_VOID(n) (!ITTNOTIFY_NAME(n)) ? (void)0 : ITTNOTIFY_NAME(n)
Basically it wraps every function call with a function pointer call if its not NULL.
How to implement this in a cross-platform way (Windows, Mac, Linux) ?
I end up with a minimal example that looks like the code linked here, but it does not work as it should. In the linked version, my_api_hello_impl() is not called as it should. Also, there is no crash checking the value of the extern symbol api_hello_ptr() when the library is not linked.
my_api.c
#include "my_api.h"
#include <stdio.h>
void(*api_hello_ptr)();
void api_hello_impl()
{
printf("Hello\n");
}
__attribute__((constructor))
static void init()
{
printf("linked\n");
api_hello_ptr = api_hello_impl;
}
my_api.h
#pragma once
extern void(*api_hello_ptr)();
inline void api_hello() { if(api_hello_ptr) api_hello_ptr(); }
some_process.c
#include "my_api.h"
int main()
{
// NOOPS of not linked at runtime
api_hello();
}
Makefile
# my_api is not linked to some_process
some_process: some_process.c my_api.h
$(CC) -o $# $<
my_api.so: my_api.c my_api.h
$(CC) -shared -fPIC -o $# $<
test_linked: some_process my_api.so
LD_PRELOAD="$(shell pwd)/my_api.so" ./some_process
test_unlinked: some_process my_api.so
./some_process
.PHONY: test_linked test_unlinked
Output:
$ make test_linked
LD_PRELOAD="/tmp/tmp.EkrQbILrNg/my_api.so" ./some_process
linked
$ make test_unlinked
./some_process
Does a process checks that the dynamic library he is linking to is loaded when he starts
Yes, it does. If a dynamic library is linked, then it is a runtime requirement and the system loader will not start execution of a program without finding and loading the library first. There are mechanisms for delayed-loading, but it is not the norm on Linux, they are done manually or using custom libraries. By default, all dynamically linked objects need to be loaded before execution starts.
Note: I'm assuming we are talking about ELF executables here since we are on Linux.
How to implement this on Linux?
You can do it using macros or wrapper functions, plus libdl (link with -ldl), with dlopen() + dlsym(). Basically, in each one of those wrappers, the first thing you do is check if the library was already loaded, and if not, load it. Then, find and call the needed symbol.
Something like this:
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
static void *libfoo_handle = NULL;
static int (*libfoo_func_a)(int, int);
static void load_libfoo_if_needed(void) {
if (!libfoo_handle) {
// Without "/" in the path, this will look in all standard system
// dynamic library directories.
libfoo_handle = dlopen("libfoo.so", RTLD_LAZY | RTLD_GLOBAL);
if (!libfoo_handle) {
perror("failed to load libfoo.so");
_exit(1);
}
// Optionally use dlsym() here to initialize a set of global
// function pointers, so that you don't have to do it later.
void *tmp = dlsym(libfoo_handle, "func_a");
if (!tmp) {
perror("no symbol func_a in libfoo.so");
_exit(1);
}
*((void**)&libfoo_func_a) = tmp;
}
}
int wrapper_libfoo_func_a(int a, int b) {
load_libfoo_if_needed();
return libfoo_func_a(a, b);
}
// And so on for every function you need. You could use macros as well.
How to implement this in a cross-platform way (Windows, Mac, Linux)?
For macOS, you should have dlopen() and dlsym() just like in Linux.
Not sure how to exactly do this on Windows, but I know there is LoadLibrary() available in different flavors (e.g. one, two, etc.), which should be more or less the equivalent of dlopen() and GetProcAddress(), which should be the equivalent of dlsym().
See also: Loading a library dynamically in Linux or OSX?

esp-idf: Conditional inclusion of components with same functions

I'm working on a project which requires to develop the firmware for several esp32. All the microcontrollers share a common code that takes care of wifi and mqtt, however they all have a different behavior, which is defined in a specific component. The structure of my project is something like this:
- CMakeLists.txt
- Makefile
- sdkconfig
- main
- CMakeLists.txt
- main.c
- components
- wifi_fsm
- wifi_fsm.h
- wifi_fsm.c
- CMakeLists.txt
- mqtt_fsm
- mqtt_fsm.h
- mqtt_fsm.c
- CMakeLists.txt
- entity_1
- entity_1.h
- entity_1.c
- CMakeLists.txt
- entity2
- entity2.h
- entity2.c
- CMakeLists.txt
...
Each entity defines some functions with standard names, which implement specific logic for the entity itself and which are called within the shared code (main, wifi_fsm, mqtt_fsm).
void init_entity(); // called in main.c
void http_get(char *buf); // called in wifi_fsm
void http_put(char *buf);
void mqtt_msg_read(char *buf); // called in mqtt_fsm
void mqtt_msg_write(char *buf);
My idea was to have a conditional statement to include at will a specific behavior, so that depending on the entity included, the compiler would link the calls to the functions above to those found in the specific included library. Therefore, at the beginning of main.c I just added the following lines with the goal of having to change the only defined pre-processor symbol to compile for different enity behaviors.
#define ENTITY_1
#ifdef ENTITY_1
#include "entity_1.h"
#elif defined ENTITY_2
#include "entity_2.h"
#elif ...
#endif
#include "wifi_fsm.h"
#include "mqtt_fsm.h"
void app_main(void)
{
while(1){
...
}
}
On the one hand the compiler apparently works fine, giving successful compilation without errors or warnings, meaning that the include chain works correctlty otherwise a duplicate name error for the standard functions would be thrown. On the other hand, it always links against the first entity in alphabetical order, executing for instance the code included in the init_entity() of the component entity_1. If I rename the standard functions in entity_1, then it links against entity_2.
I can potentially use pointers to standard calls to be linked to specific functions in each entity if the approach above is wrong, but I would like to understand first what is wrong in my approach.
EDIT in response to Bodo's request (content of the CMakeFile.txt)
Project:
cmake_minimum_required(VERSION 3.5)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(proj)
Main:
set(COMPONENT_REQUIRES )
set(COMPONENT_PRIV_REQUIRES )
set(COMPONENT_SRCS "main.c")
set(COMPONENT_ADD_INCLUDEDIRS "")
register_component()
Component:
set(COMPONENT_SRCDIRS "src")
set(COMPONENT_ADD_INCLUDEDIRS "include")
set(COMPONENT_REQUIRES log freertos driver nvs_flash esp_http_server mqtt)
register_component()
This answer is based on guessing because I don't have enough information. For the same reason it is incomplete in some parts or may not fully match the use case of the question.
The details about how the project will be built seems to be hidden in a cmake include file like project.cmake or nested include files.
My guess is that the build system creates libraries from the source code of every individual component and then links the main object file with the libraries. In this case, the linker will find a symbol like init_entity in the first library that fulfills the dependency. This means the library (=component) listed first in the linker command line will be used.
If the linker command line would explicitly list the object files entity_1.o and entity_2.o, I would expect an error message about a duplicate symbol init_entity.
I can propose two ways to solve the problem:
Make sure only the selected entity is used to build the program.
Make the identifier names unique in all entities and use preprocessor macros to choose the right one depending on the selected entity.
For the first approach you can use conditionals in CMakeLists.txt. See https://stackoverflow.com/a/15212881/10622916 for an example. Maybe the register_component() is responsible for adding the component to the build. In this case you could wrap this in a condition.
BUT modifying the CMakeLists.txt might be wrong if the files are generated automatically.
For the second approach you should rename the identifiers in the entities to make them unique. The corresponding header files can define a macro to replace the common name intended for the identifier with the specific identifier of the selected entity.
In the code that uses the selected entity you will always use the common name, not the individual names.
Example:
entity_1.c:
#include "entity_1.h"
void init_entity_1(void)
{
}
entity_2.c:
#include "entity_2.h"
void init_entity_2(void)
{
}
entity_1.h:
void init_entity_1(void);
// This replaces the token/identifier "init_entity" with "init_entity_1" in subsequent source lines
#define init_entity init_entity_1
// or depending on the parameter list something like
// #define init_entity() init_entity_1()
// #define init_entity(x,y,z) init_entity_1(y,x,z)
entity_2.h:
void init_entity_2(void);
#define init_entity init_entity_2
main.c
#define ENTITY_1
#ifdef ENTITY_1
#include "entity_1.h"
#elif defined ENTITY_2
#include "entity_2.h"
#elif ...
#endif
void some_function(void)
{
init_entity();
}
In this example case with #define ENTITY_1, the preprocessor will change some_function to
void some_function(void)
{
init_entity_1();
}
before the compilation step and the linker will use init_entity_1 from entity_1.c. An optimizing linker may then omit the object file entity_2.o or the corresponding library because it is unused.

Link with custom function instead of crt one

I'm working on an application which defines it's own printf() to get around differences between the different CRTs out there or because some other platforms don't have it.
When building the application with gcc this automatically seems to work and the custom printf is used instead of libc's one; if I understand it correctly this is because of the order in which object files/libraries appear in the link command or maybe because object files are always searched before CRT libs, correct?
I'd like to do the same using msvc. Just building the project gives the expected 'LNK2005: _printf already defined in printf.obj' because printf is also in msvcrtd.lib. Fair enough. I know about /NODEFAULTLIB but that excludes everything resulting in unresolved references for everything but printf. I scanned through the other linker settings but couldn't find anything which allows this (apart from /FORCE maybe, but the 'might produce an invalid executable' comment doesn't make it sound like a good idea). Also nothing in the module definition file docs; the latter got me thinking it might be possible to create a stub library which has all exports from msvcrt.lib except printf but that seems a brittle solution even if it works.
In the end the question is simple: how do I tell msvc's linker it should skip msvcrt's printf definition and use the one from my printf.obj instead. Basically /NODEFAULTFUNCTION:printf or so. Just an answer for one single executable is ok, though I'd also be interested to know if and how it can be done when building a dll instead where the custom printf is exported: how to tell the linker it should use the export from my .lib instead of msvcrt.lib?
edit simplest repo I could find: create a file main.c:
#include <stdio.h>
int main(int argc, char** argv)
{
printf("Hello");
return 0;
}
and a file printf.c:
int printf(const char *fmt, ...)
{
write(1, "ok\n", 3);
return 3;
}
For VS2013 (though the other versions might work as well): create a new empty C++ project and add both files then build. (For gcc: just gcc main.c printf.c and the resulting a.out prints 'ok')
The culptrit for VS is #include : without that it works ok but I have yet to find out if the original code allows getting rid of it in some way. But even if it does I'd still want to know if this can be solved at the link level.

Mocking C functions in MSVC (Visual Studio)

I am reading several articles on mocking C functions (like CMock, or CMocka), but I am not sure how the actual functions are replaced with mocked functions in this process. For example, CMocka relies on automatic wrapping using a GNU compiler, which supports parameters like --wrap to append the __wrap prefix to function calls, or weak symbols which allow you to override any symbol you like.
But how do you do this in Visual Studio, for pretty much all other frameworks?
For example, CMock has an example similar to this (simplified a lot here):
// myfunc.c
#include <parsestuff.h>
// this is the function we would like to test
int MyFunc(char* Command)
{
// this is the call to the function we will mock
return ParseStuff(Command);
}
There is also the actual implementation, which contains the actual function the linker should find in the actual application:
// parsestuff.c
int ParseStuff(char* cmd)
{
// do some actual work
return 42;
}
Now, during testing the Ruby script creates mock functions like:
// MockParseStuff.c (auto created by cmock)
int ParseStuff(char* Cmd);
void ParseStuff_ExpectAndReturn(char* Cmd, int toReturn);
But if the VS project already includes parsestuff.c, how will it be possible that the call from myfunc.c ends up in MockParseStuff.c?
Does this mean I cannot have parsestuff.c included in the unit testing project? But if this is the case, then it's also impossible to mock, for example, MyFunc from myfunc.c in any tests, since I already had to include the file it in order to test it?
(Update) I am also aware that I can include the .c file instead of the .h file, and then do some preprocessor stuff to replace the original call, like:
// replace ParseStuff with ParseStuff_wrap
#define ParseStuff ParseStuff_wrap
// include the source instead of the header
#include <myfunc.c>
#undef ParseStuff
int ParseStuff_wrap(char* cmd)
{
// this will get called from MyFunc,
// which is now statically included
}
but this seems like a lot of plumbing, and I don't even see it mentioned anywhere.
Here's a simple and short solution with hippomocks:
I created an empty Win32 console application with
main.cpp
myfunc.c + myfunc.h
parsestuff.c, parsestuff.h
and added the code from your example.
With help of hippomocks, you can mock every C-Function. Here's how my main.cpp looks like:
#include "stdafx.h"
#include "myfunc.h"
#include "hippomocks.h"
extern "C" int ParseStuff(char* cmd);
int _tmain(int argc, _TCHAR* argv[])
{
MockRepository mocks;
mocks.ExpectCallFunc(ParseStuff).Return(4711);
char buf[10] = "";
int result = MyFunc(buf);
return result; //assert result is 4711
}
HippoMocks is a free, simple and very powerful one-header framework and can be downloaded on GitHub.
Hope I've earned the bounty :)
UPDATE, How it works:
HippoMocks gets the func pointer to ParseStuff
HippoMocks builds a replacement func pointer to a template function with same signature and own implementation.
Hippomocks patches the jmp opcode from the function call prologue in memory, so that it points to the replaced function.
Replacement and memory patch are released after call or in destructor.
Here's how it looks like on my machine:
#ILT+3080(_ParseStuff):
00D21C0D jmp HippoMocks::mockFuncs<char,int>::static_expectation1<0,char *> (0D21DB1h)
If you watch the memory address 00D21C0D (may differ from run to run) in memory window, you will see, that it gets patched after the call of ExpectCallFunc.
I have not dealt with the C mocking libraries or Visual Studio, but I have thought about this in my own project. The Feathers book suggests the preprocessor seam or the link seam as a tool for dealing with this. You already mentioned the preprocessor seam, so I'll focus on the link seam.
The link seam requires the mocked function to be in a library, and the mock function to be in a library. The test can link against the mock function library while the target application can link against the original library.
Of course, as you mention, to mock MyFunc() you will have to create another library and a separate test application to link against it (or dynamically load and unload libraries in the test application).
It sounds quite laborious which is why I am procrastinating adding tests in my own application!
Hope this helps!

How to link a dll, to an exe file in run-time?

I am new to programming and I created a dll project in which, i will just print a line. In the application project I have called the function defined in the dll project.
My question is, I am getting the dll file, soon after building the dll project. But when I build the main app, i.e., application project i am getting the following error.
--------------------Configuration: test_bench - Win32 Debug--------------------
Compiling...
main.c
Linking...
main.obj : error LNK2001: unresolved external symbol _print_dll
../../exec/test_bench.exe : fatal error LNK1120: 1 unresolved externals
Error executing link.exe.
test_bench.exe - 2 error(s), 0 warning(s)
If I link the obj before the build, it gets built. But if I change the code of dll project, again i have to re-build the main project, which should not be necessary while running a dll.
Please help me to achieve this
I wrote a little binary template to help with dlopen related calls. You'd need to tweak it for your specific use case (as it is, it only handles strings), but I have found it handy on many occasions.
Usage: dlopener /path/to/library.extension function_name [args...]
//gcc -rdynamic -o dlopener dlopener.c -ldl
#include <dlfcn.h> /*for dlopen,dlsym,dlclose*/
int main(int argc, char **argv){
/* get a "handle" for a shared library*/
void *handle = dlopen(argv[1], RTLD_LAZY);
/* make sure we got a handle before continuing*/
if (! handle) return 1;
/*undefined, but workable solution : POSIX.1-2003 (Technical Corrigendum 1) */
void* (*f)()=dlsym(handle, argv[2]);
/*now call the function f(argv[3],argv[4],...argv[argc]); */
//TODO convert args to unsigned char representations for other types
while (argc > 2) /*ugh, have to use asm to preserve stack*/
asm("push %0"::"r"(argv[argc--])); /*from right to left*/
asm("call *%0"::"r"(f)); //TODO "=a"(ret) where is uchar[XXX]
/*remember that shared library we opened?*/
dlclose(handle);
return 0;
}
Note: for some WIN32 functions (cdecl calling convention IIRC) you need to use a different asm mechanism than push to get the values in registers instead of on the stack
This error indicates that you didn't link the library (DLL) in your executable, that's why it can't find the function print_dll.
You can load libraries dynamically in Windows through LoadLibray()
Here is a sample on how you do it: http://msdn.microsoft.com/en-us/library/windows/desktop/ms686944(v=vs.85).aspx
For linking the dll project in run time, you need to implement the run time linking in your main file. If you implement the the run time linking means, no need to re-build the project every time.
For run time linking you need the following functions
#include <dlfcn.h>
void *dlopen(const char *filename, int flag);
char *dlerror(void);
void *dlsym(void *handle, const char *symbol); //symbol is your function name
int dlclose(void *handle);
Link with -ldl.
See the man page for more details.
but while changing your code, if you change the function name you should use the same name in the place of symbol.

Resources