Let's say the following architecture:
A static library is used/linked within a DLL
The DLL is loaded (either implicitly or explicitly) by an executable
Is it possible from the executable code to access code of the static library without relinking it explicitly nor inserting wrapper functions in the DLL?
In other terms, I am looking for a way to make a dll export of dependant static library code.
Given your constraints, the answer is no.
The reason is that the executable doesn't have any visibility into the dependencies or "call-ees" of the DLL. As far as the executable is concerned, he's just knows about the DLL itself: at link time, the executable knows only about those exports it is consuming from the DLL. He's going to LoadLibrary() against the DLL (which will fail if the dependencies of said DLL aren't resolvable), then call the exports of said DLL.
If you can't statically link with the library used by the DLL for some reason, another approach is to wrap the calls to said static library. This can be a pain of there are lots of calls, but there are automated tools which others have created to help. In particular I've used this before to create a wrapper for a DLL which exported hundreds of functions when I wanted to intercept a particular one: http://www.codeproject.com/KB/DLL/CreateYourProxyDLLs.aspx
The answer may easily be: Yes.
The only requirement is:
In your static LIB file you must define __declspec(dllexport) for all what you want to export.
When you then include this LIB file into your DLL project all the functions that you have declared as __declspec(dllexport) will be automatically DLL Exports and can be accessed from your Exe.
Related
How exactly do DLL files work? There seems to be an awful lot of them, but I don't know what they are or how they work.
So, what's the deal with them?
What is a DLL?
Dynamic Link Libraries (DLL)s are like EXEs but they are not directly executable. They are similar to .so files in Linux/Unix. That is to say, DLLs are MS's implementation of shared libraries.
DLLs are so much like an EXE that the file format itself is the same. Both EXE and DLLs are based on the Portable Executable (PE) file format. DLLs can also contain COM components and .NET libraries.
What does a DLL contain?
A DLL contains functions, classes, variables, UIs and resources (such as icons, images, files, ...) that an EXE, or other DLL uses.
Types of libraries:
On virtually all operating systems, there are 2 types of libraries. Static libraries and dynamic libraries. In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.
More on static and dynamic libraries:
You don't normally see static libraries though on your computer, because a static library is embedded directly inside of a module (EXE or DLL). A dynamic library is a stand-alone file.
A DLL can be changed at any time and is only loaded at runtime when an EXE explicitly loads the DLL. A static library cannot be changed once it is compiled within the EXE.
A DLL can be updated individually without updating the EXE itself.
Loading a DLL:
A program loads a DLL at startup, via the Win32 API LoadLibrary, or when it is a dependency of another DLL. A program uses the GetProcAddress to load a function or LoadResource to load a resource.
Further reading:
Please check MSDN or Wikipedia for further reading. Also the sources of this answer.
What is a DLL?
DLL files are binary files that can contain executable code and resources like images, etc. Unlike applications, these cannot be directly executed, but an application will load them as and when they are required (or all at once during startup).
Are they important?
Most applications will load the DLL files they require at startup. If any of these are not found the system will not be able to start the process at all.
DLL files might require other DLL files
In the same way that an application requires a DLL file, a DLL file might be dependent on other DLL files itself. If one of these DLL files in the chain of dependency is not found, the application will not load. This is debugged easily using any dependency walker tools, like Dependency Walker.
There are so many of them in the system folders
Most of the system functionality is exposed to a user program in the form of DLL files as they are a standard form of sharing code / resources. Each functionality is kept separately in different DLL files so that only the required DLL files will be loaded and thus reduce the memory constraints on the system.
Installed applications also use DLL files
DLL files also becomes a form of separating functionalities physically as explained above. Good applications also try to not load the DLL files until they are absolutely required, which reduces the memory requirements. This too causes applications to ship with a lot of DLL files.
DLL Hell
However, at times system upgrades often breaks other programs when there is a version mismatch between the shared DLL files and the program that requires them. System checkpoints and DLL cache, etc. have been the initiatives from M$ to solve this problem. The .NET platform might not face this issue at all.
How do we know what's inside a DLL file?
You have to use an external tool like DUMPBIN or Dependency Walker which will not only show what publicly visible functions (known as exports) are contained inside the DLL files and also what other DLL files it requires and which exports from those DLL files this DLL file is dependent upon.
How do we create / use them?
Refer the programming documentation from your vendor. For C++, refer to LoadLibrary in MSDN.
Let’s say you are making an executable that uses some functions found in a library.
If the library you are using is static, the linker will copy the object code for these functions directly from the library and insert them into the executable.
Now if this executable is run it has every thing it needs, so the executable loader just loads it into memory and runs it.
If the library is dynamic the linker will not insert object code but rather it will insert a stub which basically says this function is located in this DLL at this location.
Now if this executable is run, bits of the executable are missing (i.e the stubs) so the loader goes through the executable fixing up the missing stubs. Only after all the stubs have been resolved will the executable be allowed to run.
To see this in action delete or rename the DLL and watch how the loader will report a missing DLL error when you try to run the executable.
Hence the name Dynamic Link Library, parts of the linking process is being done dynamically at run time by the executable loader.
One a final note, if you don't link to the DLL then no stubs will be inserted by the linker, but Windows still provides the GetProcAddress API that allows you to load an execute the DLL function entry point long after the executable has started.
DLLs (dynamic link libraries) and SLs (shared libraries, equivalent under UNIX) are just libraries of executable code which can be dynamically linked into an executable at load time.
Static libraries are inserted into an executable at compile time and are fixed from that point. They increase the size of the executable and cannot be shared.
Dynamic libraries have the following advantages:
1/ They are loaded at run time rather than compile time so they can be updated independently of the executable (all those fancy windows and dialog boxes you see in Windows come from DLLs so the look-and-feel of your application can change without you having to rewrite it).
2/ Because they're independent, the code can be shared across multiple executables - this saves memory since, if you're running 100 apps with a single DLL, there may only be one copy of the DLL in memory.
Their main disadvantage is advantage #1 - having DLLs change independent your application may cause your application to stop working or start behaving in a bizarre manner. DLL versioning tend not to be managed very well under Windows and this leads to the quaintly-named "DLL Hell".
DLL files contain an Export Table which is a list of symbols which can be looked up by the calling program. The symbols are typically functions with the C calling convention (__stcall). The export table also contains the address of the function.
With this information, the calling program can then call the functions within the DLL even though it did not have access to the DLL at compile time.
Introducing Dynamic Link Libraries has some more information.
http://support.microsoft.com/kb/815065
A DLL is a library that contains code
and data that can be used by more than
one program at the same time. For
example, in Windows operating systems,
the Comdlg32 DLL performs common
dialog box related functions.
Therefore, each program can use the
functionality that is contained in
this DLL to implement an Open dialog
box. This helps promote code reuse and
efficient memory usage.
By using a DLL, a program can be
modularized into separate components.
For example, an accounting program may
be sold by module. Each module can be
loaded into the main program at run
time if that module is installed.
Because the modules are separate, the
load time of the program is faster,
and a module is only loaded when that
functionality is requested.
Additionally, updates are easier to
apply to each module without affecting
other parts of the program. For
example, you may have a payroll
program, and the tax rates change each
year. When these changes are isolated
to a DLL, you can apply an update
without needing to build or install
the whole program again.
http://en.wikipedia.org/wiki/Dynamic-link_library
DLL is a File Extension & Known As “dynamic link library” file format used for holding multiple codes and procedures for Windows programs. Software & Games runs on the bases of DLL Files; DLL files was created so that multiple applications could use their information at the same time.
IF you want to get more information about DLL Files or facing any error read the following post.
https://www.bouncegeek.com/fix-dll-errors-windows-586985/
DLLs (Dynamic Link Libraries) contain resources used by one or more applications or services. They can contain classes, icons, strings, objects, interfaces, and pretty much anything a developer would need to store except a UI.
According to Microsoft
(DLL) Dynamic link libraries are files that contain data, code, or resources needed for the running of applications. These are files that are created by the windows ecosystem and can be shared between two or more applications.
When a program or software runs on Windows, much of how the application works depends on the DLL files of the program. For instance, if a particular application had several modules, then how each module interacts with each other is determined by the Windows DLL files.
If you want detailed explanation, check these useful resources
What are dll files , About Dll files
I have a C-based DLL that I wrote years ago for a project and it exports a set of functions that define an API. Now I need to re-write this DLL's internals but keep the API exactly the same.
The user of the DLL used static linking and they do not want to or are unable to recompile their executable.
I've noticed that the RVAs of the exported functions are different. My understanding is that means the executable won't be able to find the functions unless it is re-linked with the updated lib file.
Is there a way in VS2017 to force an exported function to use a specific RVA? I checked the Microsoft LINK DEF file format and I didn't see an option in there.
Even if it is possible, is fixing the RVAs enough to ensure the old executable will be able to use the updated DLL or are there additional complications that make this a non-starter?
Thanks.
When you statically link an EXE module against a DLL, you do indeed link against the the DLL's import library (a .LIB) created alongside the DLL when the DLL was built. This is not the same thing as linking against a static library which is confusing because those are also .LIB files.
The first thing you should do is figure out if your EXE module has an import entry for said DLL using a tool like Dependency Walker, Dumpbin, pelook or your favorite PE analyzer tool. If there is no DLL import entry, you have have probably linked the EXE against a static library as described by #HAL9000 's answer. Short of reverse-engineering the EXE, your best bet would be to rebuild the module as suggested if possible.
Otherwise, if you do find an import for said DLL, then yes you can swap out a newly-built DLL provided you have the same export (function) names and/or ordinal values as the original. DLLs find function by export names and/or ordinal values, not RVAs which in this case are only an internal detail. Whether the DLL is implicitly loaded (from being statically-linked) during process (EXE) initialization (before the EXE's entry point is called) or explicitly loaded (via code using LoadLibrary, etc.) the whole point of being a DLL is that it is a module is designed to be dynamically replaced and Windows was designed around this concept. The internal RVAs both within the EXE (referencing the DLL) and the DLL itself do not need to match an old DLL's values; this bookkeeping is automatically handled by the Windows loader during a process also known as runtime linking.
In the event the EXE is linked against said DLL and ALSO specifies hard-coded addresses (RVAs) for the DLL's exported functions (a process known as static binding), Windows will still verify the addresses still internally reflect the correct values in the DLL that is actually loaded which may be a different, updated DLL. This is done via a timestamp check in the import section for the DLL. If there is a mismatch, the Windows loader tosses-out all of the static RVAs and updates them with the current values incurring a slight performance penalty, but the program will still load. FWIW the bind.exe tool to do this static binding no longer ships with the Visual C++ toolset as the performance gain in modern versions of Windows is minimal. This optimization used to be be common practice to speed up load times, especially in OS-supplied system DLLs, but shouldn't affect what you are trying to do one way or the other.
If the user has statically linked in your library, then it is not a DLL, and making a drop in replacement without relinking is not possible. At least not without some ugly hacks. The old library functions have been copied into the executable, so there are no way around editing the executable. If you can't recompile or relink, then it is probably easier to rewrite the executable from scratch.
Mucking around with adresses of functions in your new DLL, if possible, can't have any effect if the executable doesn't have any code to load a DLL at all.
I have learned that you can:
Convert a .DLL file into a .DEF file, which includes its exports
(Edit: This doesn't work with many conventions)
Convert a .DEF file into a .LIB file, which you can use to link to the DLL
Why can't (most) linkers link to a DLL given only a .DEF file, instead of a .LIB file?
Ultimately, the answer here is 'because noone wanted it badly enough and it doesn't really help anything'.
The DEF file is an input file that creates an import lib for the DLL. And then, later, when the DLL is being consumed by another link, the importlib is itself an input. The importlib looks like something special on the outside, but when you look at the inside it's really just a slightly special lib with objects in it.
It totally would be possible to modify the linker to take a def file (or a DLL, for that matter) directly.
But the design centre of the linker is that it takes objects as inputs and outputs a PE executable. So taking a DEF or DLL as an input goes outside the design pattern.
Beyond that it'd be rather pointless - allowing the linker to take a DEF file or DLL as input would neither enable any important new scenarios, nor does leaving this feature out block anything. Converting a DEF file you have (even without the actual DLL) into a usable importlib is the work of a few moments (simply create fake empty function for each DEF entry and link that). So there's no reason to add the ability to link a DEF file directly.
Martyn
In terms of MSVC, .lib files are always static libraries. They get linked in as a compilation unit along with all your compiled .c/.cpp files, so all of the library's code is included in your final executable.
Some .lib files, however, (in particular most of the Windows system ones) merely contain stubs which tell the OS to load the desired DLL at loadtime and then the stubs route function calls to the DLL. But, those stubs are statically linked into your executable. Your program will then use DLLs (and gain all the advantages and disadvantages thereof), but since the named DLL functions it requires are happily located in the .lib (and thus actually located in the executable itself), your code doesn't have to know it's using a DLL (specifically using declspec(dllimport)).
A .def file is merely used as a sort of "settings" or "configuration" file during the creation of a .dll to specify what functions the file should export. It cannot be linked to, as it doesn't really describe anything that the linker understands.
You do not convert a dll to a DEF file. The DEF just indicates which dll functions will be accessible from the outside, exported.
From the docs:
A DLL file has a layout very similar to an .exe file, with one
important difference — a DLL file contains an exports table. The
exports table contains the name of every function that the DLL exports
to other executables. These functions are the entry points into the
DLL; only the functions in the exports table can be accessed by other
executables. Any other functions in the DLL are private to the DLL.
The exports table of a DLL can be viewed by using the DUMPBIN tool
with the /EXPORTS option.
You can export functions from a DLL using two methods:
Create a module definition (.def) file and use the .def file when
building the DLL. Use this approach if you want to export functions
from your DLL by ordinal rather than by name.
Use the keyword __declspec(dllexport) in the function's definition.
When exporting functions with either method, make sure to use the
__stdcall calling convention.
Use the provided link to learn more about exporting from your dll's.
I think you got down voted because your point is not really clear, at least not to me. Also check this. It explains how to choose the export method.
Mehrdad, this isn't always a question of how to LINK to a DLL, as I, personally have NEVER linked a DLL using a .DEF file. What I HAVE done is take someone else's DLL, and very painstakingly constructed a header file, or rather, function prototypes that I could use with
LoadLibrary() in C, Declare Function ... Lib "Foo.dll" Alias "OrdinalName" in VB, and
[DllImport()] in C#.
Of course, this is RARELY done, as if you are using a DLL for something, normally you have permission to do so, and the authors provide the .lib's, and the headers to go with the binary DLL file.
I've never done the exact techniques you speak of, by converting a .DEF info a .LIB, etc... But, I suppose it would be easy to take a lib, or the DLL itself and export .DEF from it. Now, THAT I actually HAVE done, in a project where the DLL code was built with a vbScript that took code from the main project, and created an API out of all the existing, compiled, and tested code. This level of complication was only done because I had no idea what functions were going to BE in the DLL, as the main project could change at any time, so a static .DEF file would have never worked. So, I had to build the DLL once, capture the dimpbin /exports, undecorate the functions, and then build the .DEF file, and re-link the DLL.
If you find yourself in that type of situation, perhaps you need to re-think your original designs, and fix the problem from there...
As for .LIB files, USUALLY you'd only NEED those for static linkage, but they are also used when the .H file is available, often making debugging a just a little nicer...
I have c++ libary in wince 6.0, There is some global variable. that libary has been linked with DLL which is c++. when i exected application and called DLL. first its intilized all global variable of static libary but when control come back to DLL to static lib. Its reset all glbal value. Do anyone have idea about that.
If you have linked two DLLs with the static library, this means that you have two copies of all the code and variables in the library.
To solve this issue you should convert the static library into a DLL. This can be done with a simple project consisting of a module.def file, and a linker call.
Then instead of linking your DLLs with the static library, link them with your DLL.
Alternatively, instead of having two dlls, combine the project to have a single DLL.
Note that it can be OK to link two modules with the same static library, provided that you understand and accept that the data will NOT be shared.
I just downloaded curl and managed to build a programe with it.
What I don't understand is why it requires both curllib.lib and curllib.dll ?
In my opinion either curllib.lib or curllib.dll should be enough, why both?
Also I've searched through the source but doesn't find anywhere it uses dlopen to load curllib.dll, so why is curllib.dll required to run?
curllib.lib is the import library. It contains stubs for all the functions whose actual code is in the DLL, and it contains code to load the DLL. The .lib is automatically generated by the build system (e.g. Visual Studio), which is why you won't find any references in the code. You will find, however, some directives that tell the compiler/linker to export certain functions:
__declspec(dllexport) on a function declaration will export that function. This is convenient, but Microsoft-specific. For example:
void __declspec(dllexport) foobar(int qux);
More information here.
Alternatively, a .def file can be used, that explicitly lists all functions that are exported. For example:
LIBRARY MYFOO
EXPORTS
foobar #1
More information here.
You can do without the .lib if you really want to, but you'll have to manually use LoadLibrary, GetProcAddress and friends to call the function in the DLL.
The .dll is required to run. The .lib exists so the linker can resolve the ordinals and addresses from function names, and is only required at link time.
Other systems (such as Linux) maintain full linkage information in shared libraries and would not require this.
When you use a DLL on Windows, you typically use it with a .lib file that tells the linker enough to be able to create an executable. The executable then uses the .dll file at run time.
OTOH, dlopen is a UNIX/Linux/POSIX kind of thing -- the equivalent on Windows would be LoadLibrary. You'd normally use dlopen with a .so file.