I have learned that you can:
Convert a .DLL file into a .DEF file, which includes its exports
(Edit: This doesn't work with many conventions)
Convert a .DEF file into a .LIB file, which you can use to link to the DLL
Why can't (most) linkers link to a DLL given only a .DEF file, instead of a .LIB file?
Ultimately, the answer here is 'because noone wanted it badly enough and it doesn't really help anything'.
The DEF file is an input file that creates an import lib for the DLL. And then, later, when the DLL is being consumed by another link, the importlib is itself an input. The importlib looks like something special on the outside, but when you look at the inside it's really just a slightly special lib with objects in it.
It totally would be possible to modify the linker to take a def file (or a DLL, for that matter) directly.
But the design centre of the linker is that it takes objects as inputs and outputs a PE executable. So taking a DEF or DLL as an input goes outside the design pattern.
Beyond that it'd be rather pointless - allowing the linker to take a DEF file or DLL as input would neither enable any important new scenarios, nor does leaving this feature out block anything. Converting a DEF file you have (even without the actual DLL) into a usable importlib is the work of a few moments (simply create fake empty function for each DEF entry and link that). So there's no reason to add the ability to link a DEF file directly.
Martyn
In terms of MSVC, .lib files are always static libraries. They get linked in as a compilation unit along with all your compiled .c/.cpp files, so all of the library's code is included in your final executable.
Some .lib files, however, (in particular most of the Windows system ones) merely contain stubs which tell the OS to load the desired DLL at loadtime and then the stubs route function calls to the DLL. But, those stubs are statically linked into your executable. Your program will then use DLLs (and gain all the advantages and disadvantages thereof), but since the named DLL functions it requires are happily located in the .lib (and thus actually located in the executable itself), your code doesn't have to know it's using a DLL (specifically using declspec(dllimport)).
A .def file is merely used as a sort of "settings" or "configuration" file during the creation of a .dll to specify what functions the file should export. It cannot be linked to, as it doesn't really describe anything that the linker understands.
You do not convert a dll to a DEF file. The DEF just indicates which dll functions will be accessible from the outside, exported.
From the docs:
A DLL file has a layout very similar to an .exe file, with one
important difference — a DLL file contains an exports table. The
exports table contains the name of every function that the DLL exports
to other executables. These functions are the entry points into the
DLL; only the functions in the exports table can be accessed by other
executables. Any other functions in the DLL are private to the DLL.
The exports table of a DLL can be viewed by using the DUMPBIN tool
with the /EXPORTS option.
You can export functions from a DLL using two methods:
Create a module definition (.def) file and use the .def file when
building the DLL. Use this approach if you want to export functions
from your DLL by ordinal rather than by name.
Use the keyword __declspec(dllexport) in the function's definition.
When exporting functions with either method, make sure to use the
__stdcall calling convention.
Use the provided link to learn more about exporting from your dll's.
I think you got down voted because your point is not really clear, at least not to me. Also check this. It explains how to choose the export method.
Mehrdad, this isn't always a question of how to LINK to a DLL, as I, personally have NEVER linked a DLL using a .DEF file. What I HAVE done is take someone else's DLL, and very painstakingly constructed a header file, or rather, function prototypes that I could use with
LoadLibrary() in C, Declare Function ... Lib "Foo.dll" Alias "OrdinalName" in VB, and
[DllImport()] in C#.
Of course, this is RARELY done, as if you are using a DLL for something, normally you have permission to do so, and the authors provide the .lib's, and the headers to go with the binary DLL file.
I've never done the exact techniques you speak of, by converting a .DEF info a .LIB, etc... But, I suppose it would be easy to take a lib, or the DLL itself and export .DEF from it. Now, THAT I actually HAVE done, in a project where the DLL code was built with a vbScript that took code from the main project, and created an API out of all the existing, compiled, and tested code. This level of complication was only done because I had no idea what functions were going to BE in the DLL, as the main project could change at any time, so a static .DEF file would have never worked. So, I had to build the DLL once, capture the dimpbin /exports, undecorate the functions, and then build the .DEF file, and re-link the DLL.
If you find yourself in that type of situation, perhaps you need to re-think your original designs, and fix the problem from there...
As for .LIB files, USUALLY you'd only NEED those for static linkage, but they are also used when the .H file is available, often making debugging a just a little nicer...
Related
I have a C-based DLL that I wrote years ago for a project and it exports a set of functions that define an API. Now I need to re-write this DLL's internals but keep the API exactly the same.
The user of the DLL used static linking and they do not want to or are unable to recompile their executable.
I've noticed that the RVAs of the exported functions are different. My understanding is that means the executable won't be able to find the functions unless it is re-linked with the updated lib file.
Is there a way in VS2017 to force an exported function to use a specific RVA? I checked the Microsoft LINK DEF file format and I didn't see an option in there.
Even if it is possible, is fixing the RVAs enough to ensure the old executable will be able to use the updated DLL or are there additional complications that make this a non-starter?
Thanks.
When you statically link an EXE module against a DLL, you do indeed link against the the DLL's import library (a .LIB) created alongside the DLL when the DLL was built. This is not the same thing as linking against a static library which is confusing because those are also .LIB files.
The first thing you should do is figure out if your EXE module has an import entry for said DLL using a tool like Dependency Walker, Dumpbin, pelook or your favorite PE analyzer tool. If there is no DLL import entry, you have have probably linked the EXE against a static library as described by #HAL9000 's answer. Short of reverse-engineering the EXE, your best bet would be to rebuild the module as suggested if possible.
Otherwise, if you do find an import for said DLL, then yes you can swap out a newly-built DLL provided you have the same export (function) names and/or ordinal values as the original. DLLs find function by export names and/or ordinal values, not RVAs which in this case are only an internal detail. Whether the DLL is implicitly loaded (from being statically-linked) during process (EXE) initialization (before the EXE's entry point is called) or explicitly loaded (via code using LoadLibrary, etc.) the whole point of being a DLL is that it is a module is designed to be dynamically replaced and Windows was designed around this concept. The internal RVAs both within the EXE (referencing the DLL) and the DLL itself do not need to match an old DLL's values; this bookkeeping is automatically handled by the Windows loader during a process also known as runtime linking.
In the event the EXE is linked against said DLL and ALSO specifies hard-coded addresses (RVAs) for the DLL's exported functions (a process known as static binding), Windows will still verify the addresses still internally reflect the correct values in the DLL that is actually loaded which may be a different, updated DLL. This is done via a timestamp check in the import section for the DLL. If there is a mismatch, the Windows loader tosses-out all of the static RVAs and updates them with the current values incurring a slight performance penalty, but the program will still load. FWIW the bind.exe tool to do this static binding no longer ships with the Visual C++ toolset as the performance gain in modern versions of Windows is minimal. This optimization used to be be common practice to speed up load times, especially in OS-supplied system DLLs, but shouldn't affect what you are trying to do one way or the other.
If the user has statically linked in your library, then it is not a DLL, and making a drop in replacement without relinking is not possible. At least not without some ugly hacks. The old library functions have been copied into the executable, so there are no way around editing the executable. If you can't recompile or relink, then it is probably easier to rewrite the executable from scratch.
Mucking around with adresses of functions in your new DLL, if possible, can't have any effect if the executable doesn't have any code to load a DLL at all.
Let's say the following architecture:
A static library is used/linked within a DLL
The DLL is loaded (either implicitly or explicitly) by an executable
Is it possible from the executable code to access code of the static library without relinking it explicitly nor inserting wrapper functions in the DLL?
In other terms, I am looking for a way to make a dll export of dependant static library code.
Given your constraints, the answer is no.
The reason is that the executable doesn't have any visibility into the dependencies or "call-ees" of the DLL. As far as the executable is concerned, he's just knows about the DLL itself: at link time, the executable knows only about those exports it is consuming from the DLL. He's going to LoadLibrary() against the DLL (which will fail if the dependencies of said DLL aren't resolvable), then call the exports of said DLL.
If you can't statically link with the library used by the DLL for some reason, another approach is to wrap the calls to said static library. This can be a pain of there are lots of calls, but there are automated tools which others have created to help. In particular I've used this before to create a wrapper for a DLL which exported hundreds of functions when I wanted to intercept a particular one: http://www.codeproject.com/KB/DLL/CreateYourProxyDLLs.aspx
The answer may easily be: Yes.
The only requirement is:
In your static LIB file you must define __declspec(dllexport) for all what you want to export.
When you then include this LIB file into your DLL project all the functions that you have declared as __declspec(dllexport) will be automatically DLL Exports and can be accessed from your Exe.
I have two dll's a.dll and b.dll along with their library files a.lib and b.lib. I am trying to write a third dll that has functions that has to make use of functions in a.dll and b.dll.
Is this possible at all?
The output has to be a dll in itself - that is an absolute requirement.
I have the full C source code.
'Is this possible at all?'
In short, yes. DLLs can contain references to other DLLs in the same way that EXEs can. You don't say what compiler is being used, but it's likely that the 'lib' contains library references to the DLL. So, you would need to add these libraries into the linker configuration for your new DLL.
You'd typically reference any function prototypes from the other DLLs, via their header files (although you could prototype them again this seems unnecessary since you say you have the code), again this is the same as if you were using the DLL from an EXE.
You'd need to make sure all three DLLs are deployed together, since the new DLL references the existing ones, it does not take a copy of the functions into itself.
I just downloaded curl and managed to build a programe with it.
What I don't understand is why it requires both curllib.lib and curllib.dll ?
In my opinion either curllib.lib or curllib.dll should be enough, why both?
Also I've searched through the source but doesn't find anywhere it uses dlopen to load curllib.dll, so why is curllib.dll required to run?
curllib.lib is the import library. It contains stubs for all the functions whose actual code is in the DLL, and it contains code to load the DLL. The .lib is automatically generated by the build system (e.g. Visual Studio), which is why you won't find any references in the code. You will find, however, some directives that tell the compiler/linker to export certain functions:
__declspec(dllexport) on a function declaration will export that function. This is convenient, but Microsoft-specific. For example:
void __declspec(dllexport) foobar(int qux);
More information here.
Alternatively, a .def file can be used, that explicitly lists all functions that are exported. For example:
LIBRARY MYFOO
EXPORTS
foobar #1
More information here.
You can do without the .lib if you really want to, but you'll have to manually use LoadLibrary, GetProcAddress and friends to call the function in the DLL.
The .dll is required to run. The .lib exists so the linker can resolve the ordinals and addresses from function names, and is only required at link time.
Other systems (such as Linux) maintain full linkage information in shared libraries and would not require this.
When you use a DLL on Windows, you typically use it with a .lib file that tells the linker enough to be able to create an executable. The executable then uses the .dll file at run time.
OTOH, dlopen is a UNIX/Linux/POSIX kind of thing -- the equivalent on Windows would be LoadLibrary. You'd normally use dlopen with a .so file.
please, could someone explain to me a few basic things about working with languages like C? Especially on Windows?
If I want to use some other library, what do I need from the library? Header files .h and ..?
What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
To use another library, what is preferred way? LoadLibrary() or #include <>?
There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
How do I create one big .exe? Is this called "static linking"?
How to include some random file into .exe? Say a program icon or start-up song?
How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Well, so many question... Thanks for every info!
1: If I want to use some other library, what do I need from the library? Header files .h and ..?
... and, usually a *.lib file which you pass as an argument to your linker.
2: What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
This might be useful: Static libraries, dynamic libraries, DLLs, entry points, headers … how to get out of this alive?
3: Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
Yes, it matters. For interop between compilers, the normal way is to use a C-style (not C++-style) API, with well-defined parameter-passing conventions (e.g. __stdcall), or to use 'COM' interfaces.
4: To use another library, what is preferred way? LoadLibrary() or #include <>?
#include is for the compiler (e.g. so that it can compile calls to the library); and LoadLibrary (or, using a *.lib file) is for the run-time linker/loader (so that it can substitute the actual address of those library methods into your code): i.e. you need both.
5: There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
If it's only source then you can compile that source (once) into a library, and then (when you build your project) link to that library (without recompiling the library).
6: How do I create one big .exe? Is this called "static linking"?
Yes, compile everything and pass it all to the linker.
7: How to include some random file into .exe? Say a program icon or start-up song?
Define that in a Windows-specific 'resource file', which is compiled by the 'resource compiler'.
8: How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
Yes.
9: If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
I don't understand your question/example.
10: What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Yes.
If I want to use some other library, what do I need from the library? Header files .h and ..?
You need header .h or .hpp for C,C++ although some languages don't require header files. You'll also need .a, .so, .dll, .lib, .jar etc files. These files contain the machine code that you linker can link into your program. Goes without saying that the format of library is must be understood by you linker.
What is the difference between .dll and .dll.a.? .dll and .lib? .dll and .exe? What is .def?
dll and .a are library files, that contain code components that you can link into your own program. a .exe is your final program into which .a or .dll has already been linked.
Does it matter how was the library compiled? I mean, is it possible to use, on Windows, a C++ library compiled by VC from within my C code compiled by MinGW?
Yes, it is important that the library that you are using is compatible with your platform. Typically Unix libraries will not run on windows and vice versa, if you are using JAVA you are better off since a .jar files will usually work on any platform with JAVA enabled (though versions matter )
To use another library, what is preferred way? LoadLibrary() or #include <>?
include is not a way to use a library its just a preprocessor directive telling you preprocessor to include a external source file in your current source file. This file can be any file not just .h although usually it would be .h or a .hpp
You'll be better off my leaving the decision about when to load a library to you runtime environment or your linker, unless you know for sure that loading a library at a particular point of time is going to add some value to your code. The performance cost and exact method of doing this is platform dependent.
There are some libraries which only provide the source code or .dll - how to use such libraries? Do I have to recompile them every time I rebuild my project?
If you have source code you'll need to recompile it every time you make a change to it.
however if you have not changed the source of library in anyway there is no need to recompile it. The build tool like Make are intelligent enough to take this decision for you.
How do I create one big .exe? Is this called "static linking"?
Creating a static .exe is dependent on the build tool you are using.
with gcc this would usually mean that you have to you -static option
gcc -static -o my.exe my.c
How to include some random file into .exe? Say a program icon or start-up song?
Nothing in programming is random. If it were we would be in trouble. Again the way you can play a song or display an icon is dependent on the platform you are using on some platforms it may even be impossible to do so.
How do I split my huge .c into smaller ones? Do I need to create for every part a header file which then I include in the part with WinMain() or main()?
You'll need a header file with all your function prototypes and you can split you program into several .c files that contain one or more functions. You main files will include the header file. All source files need to be compiled individually and then linked into one executable. Typically you'll get a .o for every .c and then you link all the .o together to get a .exe
If there is a library which needs another library, is it possible to combine these two into one file? Say, python26.dll needs msvcr90.dll and Microsoft.VC90.CRT.manifest
Yes one library may require another library however its not advisable to package different libraries together, you may be violating the IPR and also for the fact that each library is usually a well define unit with a specific purpose and combining them into one usually doesn't make much sense.
What happens if I don't free previously allocated memory? Is this going to be cleaned up if the program (process) dies?
Again depends on the platform, usually on most OS the memory will be recovered after the program dies but on certain platforms like an embedded system it may be permanently lost.
It always a good idea to clean up the resources your program has used.
In all seriousness, the place to go to learn how to run your local environment is the documentation for your local environment. After all we on not even know exactly what your environment is, much less have it in front of us.
But here are some answers:
1. You need the headers, and a linkable object of some kind. Or you need the source so that you can build these.
3. It matters that the library is in a format that your linker understands. In c++ and perhaps other languages, it also needs to understand the name mangling that was used.
6. Forcing all the library code to be included in the executable is, indeed, called "static linking".
7. There is at least one StackOverflow question on "resource compilers".