Context
On Windows, using mingw32 and its x86_64 version. In order to use a third-party library (within Unity), I'am trying to build a wrapper library (dll) around the native one (to simplify the integration in Unity). I need this library to be 64bits.
The problem arises when linking against the created wrapper library in a test program: the program crash on a segfault, whereas the 32bits version runs just fine.
Procedure
I am using the 32 and 64bits version of the dll provided by the third-party (see Dependencies analysis on Picture) to compile wrapping libraries.
The wrapper library is simply wrapping some functions of the native library. The header is:
#include <stdlib.h>
#include <stdio.h>
#include "ATC3DG.h"
#pragma once
#ifdef __cplusplus
extern "C"
{
#endif
__declspec(dllexport) int trakSTAR_Init();
__declspec(dllexport) int trakSTAR_GetMeasurement(double S1_X[3], double S1_A[3], double *S1_T, unsigned short * S1_Q, double S2_X[3], double S2_A[3], double *S2_T, unsigned short * S2_Q, double S3_X[3], double S3_A[3], double *S3_T, unsigned short * S3_Q);
__declspec(dllexport) int trakSTAR_Close();
__declspec(dllexport) void trakSTAR_ErrorName(int errorcode, char *errortxt);
#ifdef __cplusplus
}
#endif
It is compiling fine for both the 32 and 64bits versions respectively with:
mingw32-gcc.exe -shared -Iinclude -o bin\Release\libWrapper.dll WrapperLib.c lib\win32\nativelib.lib
and
x86_64-w64-mingw32-gcc.exe -shared -Iinclude -o bin\Release\libWrapper64.dll WrapperLib.c lib\win64\nativelib64.lib
Then, compiling test codes linking this Wrapper dll using respectively:
mingw32-gcc.exe -I.\include -L.\test -llibWrapper .\test\MainTest.c -o test\Test32
and
x86_64-w64-mingw32-gcc.exe -g -I.\include -L.\test .\test\MainTest.c -o test\Test64 -llibWrapper64
also works fine and produce the two executables.
Problem
Now, while the 32bits version (Test32.exe) runs fine, the 64bits version produce a Segfault straight ahead (no even first console print). The necessary dll (native and wrapper for both 32 and 64bits version are in the execution folder).
When inspecting the two Wrapper dlls with Dependencies, I can't see any major difference (see picture).
What could be the cause of the problem with the 64bits library?
What are my options to understand what is going wrong and debug/inspect further?
Note: Compiling a test code against the native dll works fine for both in 32 and 64 bits.
Note2: Running GDB on the faulty test program does not provide more information.
Related
I am using some C library form Go, using cgo to handle the conversion.
I have a C header file that looks like:
#include <stdlib.h>
typedef struct {
k uint64_t;
} params;
This structure is used from Go with code like:
package test
/*
#cgo CFLAGS: -g -Wall
#cgo LDFLAGS: -L. -lmylib -lm -ldl
#include "mylib.h"
*/
import "C"
func NewParams(k uint64) Paramets {
return Params{
k: C.ulonglong(k),
}
}
The code compiles fine on OS X with Apple clang version 12.0.0, but fails on Linux with clang version 7.1.0, yielding the following error:
cannot use _Ctype_ulonglong(k) (type _Ctype_ulonglong) as type _Ctype_ulong in field value
If I change the call to C.ulonglong to C.ulong, it works fine on linux but then fails on OS X.
How can I ensure the same code works on both platform?
Using go build -work I was able to confirm the issue stems from differences in mapping:
On the linux side, a C unit64_t gets mapped to a C.ulong
On the Mac side, it gets mapped to C.ulonglong
Note that both C.ulong and C.ulonglong gets mapped to a go uint64 but using the Go type directly and removing the C.xxx(k) conversion fails.
Solution is to use C.uint64_t(k) which makes it explicit the size of the type we are talking about. This is just fine because the C type is uin64_t anyways.
I am trying to create a couple of Win32 64-bit DLLs (Windows 10) which have different implementations but consistent symbol exports. The aim for this is that one would link with whichever one at build time but have the option at deployment to install either DLL and correctly run with that. I have achieved this straightforwardly on Linux where I am much more comfortable and familiar with run-time linking. But on Windows, I have not yet managed this and I am wondering if this is possible at all. I am trying this using both VS2010 and VS2019.
Suppose I have two libraries blah_legacy.dll and blah_modern.dll. They both export 6 symbols which are the interface to using the library, e.g. blah_open, blah_read, blah_write, blah_close, blah_control, blah_status.
I can link with the import library for either blah implementation and a test program calling each symbol loads and executes correctly with the corresponding blah DLL.
However, I cannot yet switch the DLLs at run time. For example, should I actually be able to link with blah-legacy.lib and then run with blah-modern.dll if I rename it to blah-legacy.dll? (Or vice-versa.)
I already got around basic file-naming issues and ensured the DLL needed can actually be found. I still got the application failed to start (0x22).
I used "objdump -xs" on the DLLs and noticed the order of symbols and their ordinals are different. So I created a .def file and ensured that the exported symbols match in number, names and in ordinals. Still nothing - the same error occurs.
There's still something to this I clearly have not figured out and would appreciate some guidance. Is this actually possible? Where do I start to look (which tools) to figure out what step to take next.
Yes.
I don't use Visual Studio much, but this is the kind of thing that happens all the time if you use MSYS2, and install some MinGW packages, and update them.
Here's what I mean by that: MSYS2 is an open source software distribution for Windows that, among other things, provides a bunch of native Windows software packages. The package manager (pacman) let's you choose which packages to have in your system, and it downloads DLLs and EXEs that were created by the MSYS2 developers. When an MSYS2 developer updates a library, you can download the updated library package, and all the other packages using that library will automatically start using the new DLL. Usually there is no issue with that because the new library version will be ABI-compatible with the old library version.
You do not need to use LoadLibrary or otherwise mess up your source code; the linker and the operating system should be able to take care of this for you.
Example
Here is a minimal example I threw together with MSYS2 showing how this can work.
The file foo_legacy.c represents your legacy DLL. I added some extra symbols so it wouldn't be too similar to the modern DLL.
__declspec(dllexport) int eoo() {
return 0;
}
__declspec(dllexport) const char * foo_name() {
return "legacy";
}
__declspec(dllexport) int foo_version() {
return 1;
}
__declspec(dllexport) int goo() {
return 0;
}
The file foo_modern.c represents the modern implementation:
__declspec(dllexport) const char * foo_name(void);
__declspec(dllexport) int foo_version(void);
int foo_version() {
return 2;
}
const char * foo_name() {
return "modern";
}
The file main.c represents an application using the foo API:
#include <stdio.h>
__declspec(dllimport) const char * foo_name(void);
__declspec(dllimport) int foo_version(void);
int main()
{
printf("%s %d\n", foo_name(), foo_version());
}
My build.sh file is a Bash script that builds and tests everything:
#!/usr/bin/bash
set -uex
gcc -Wall foo_legacy.c -shared -o foo_legacy.dll
gcc -Wall foo_modern.c -shared -o foo_modern.dll
gcc -Wall -c main.c -I. -o main.o
gcc main.o foo_legacy.dll -o main.exe
./main.exe # output: "legacy 1"
mv foo_modern.dll foo_legacy.dll
./main.exe # output: "modern 2"
rm foo_legacy.dll
./main.exe # fails because foo_legacy.dll is not found
The build script runs main.exe three different times, showing that it can either use the legacy DLL, or use the modern DLL, or fail, depending on what was installed in foo_legacy.dll.
I am trying to call a C code from Julia v0.6.2, Windows 10.
Here is the C code I compiled into a DLL with Cygwin gcc.
int timesTwo(int x) {
return 2*x;
}
The compile commands:
gcc -c mydll.c
gcc -shared -o mydll.dll mydll.o
I tested the DLL with a test C program and it works. Then I tried to call it with Julia using an absolute path to the DLL.
using Base.Libdl
x = Int32(2)
mylib = dlopen("C:\\Users\\pedro\\Documents\\codigos\\exampleDLL\\C\\mydll.dll")
ccall((:timesTwo, mylib), Int32, (Int32,), x)
Then the Julia's Workspace is restarted (all variables become undefined as soon as I use the command dlopen).
What is happening and how can I solve it?
Update
That must be a windows issue. Could not reproduce in an Ubuntu distro.
I would try to use the same compiler that Julia uses (with the -fPIC flag) to build your DLL. On Windows, julia is actually build with the
Cygwin-to-MinGW cross-compiler:
https://github.com/JuliaLang/julia/blob/master/README.windows.md#cygwin-to-mingw-cross-compiling
Can you install the mingw64-x86_64-gcc-core package (for just the C compiler) and use the x86_64-w64-mingw32-gcc compiler command (instead of gcc) ?
Use the -fPIC switch to generate the position independent code. Plus, you might be lying to Julia unless your int is defined as int32 on the specific platform. To be sure, you need to change ccall((:timesTwo, mylib), Int32, (Int32,), x) to ccall((:timesTwo, mylib), Cint, (Cint,), x). Finally, to make the code work for any operating system, please consider replacing mylib = dlopen("C:\\Users\\pedro\\Documents\\codigos\\exampleDLL\\C\\mydll.dll") to const mylib = dlopen(joinpath(pwd(), "mydll.dll")). const is there for efficiency reasons, joinpath is OS-agnostic, meaning that it will add forward slash for *nix-based systems. Later on, when you want to change the library to .so, for instance, you should not deal with slashes, too.
I'm learning how to compile a C trigger to load on PostgreSQL
When compile the "trigf.c" (in the example at http://www.postgresql.org/docs/9.3/interactive/trigger-example.html), I get some issue related to int64 error (c.h header)
#ifdef HAVE_LONG_INT_64
/* Plain "long int" fits, use it */
#ifndef HAVE_INT64
typedef long int int64;
#endif
#ifndef HAVE_UINT64
typedef unsigned long int uint64;
#endif
#elif defined(HAVE_LONG_LONG_INT_64)
/* We have working support for "long long int", use that */
#ifndef HAVE_INT64
typedef long long int int64;
#endif
#ifndef HAVE_UINT64
typedef unsigned long long int uint64;
#endif
#else
/* neither HAVE_LONG_INT_64 nor HAVE_LONG_LONG_INT_64 */
#error must have a working 64-bit integer datatype
#endif
-> [Error] #error must have a working 64-bit integer datatype
I don't know how to solve that problem, because clearly that there is a working 64 bit integer datatype that I can use.
Edit: I installed pgsql from binary. The C compiler I used for compile the C function file is MinGW GCC 4.7.2. (Using the path of Dev-cpp mingw gcc).
The command line is :
gcc -fpic -c "D:\trigf.c"
At the first time, it showed an error that in c.h: not found libintl.h (no such file or directory). Then I download the Lib Intl - 0.14.4 (library for native language support). The installation create a folder: C:\Program Files (x86)\GnuWin32.
I edited the environment variable CPATH, added C:\Program Files (x86)\GnuWin32\include folder, which contained libintl.h.
I ran the command again, and I met with the above error.
Update: It turns out not to be too hard to build extensions stand-alone with MSVC on Windows. I wrote a blog post detailing the process today.
The usual way to build extensions on Windows is to do it inside a working PostgreSQL build tree.
See these instructions on the PostgreSQL wiki.
You might be able to do it using MinGW and PGXS using a suitable Makefile instead.
Just trying to compile a standalone .c file is unlikely to work as there are a variety of paths and preprocessor definitions required.
It doesn't help that the current PostgreSQL packages don't include headers for public dependencies, which is really rather frustrating. You can safely compile without ENABLE_NLS defined even if the target PostgreSQL was built with ENABLE_NLS, though, and in this case libintl.h won't be required.
I know how to link against libraries in Unix-ish contexts: If I'm working with .a or .so files, I specify the root search directory with -L/my/path/to/lib/ and for libMylib I add -lMyLib.
But what if I have
a .dll (e.g. in the Windows\System32 directory)?
a .dll (in Windows\System32) and a .lib (someplace else)?
These DLLs are by some other party; I don't have access to their sources - but do have access to the corresponding include files, against which I manage to compile.
If you can link against a .lib in Cygwin or MinGW, then you can (indirectly) link against a DLL.
In the MSVC world, it is not unusual to create an import library along with a DLL. It is a static library (.lib) that loads the DLL and wraps the interface of the DLL. You just call the wrapper functions in the (static) import library and let the import library do all the DLL-related things.
For the Windows API, there are import libraries in the WindowsSDK.
For your own MSVC DLLs, MSVC can automatically generate the import libraries when you build the DLL.
For a third party DLL, you can build a static wrapper library based on the corresponding header files.
Linking against the .lib file in Cygwin or MinGW is possible. Example:
g++ -o myprg myprg.o -lShlwapi
This links against Shlwapi.lib. (The library must be in the local directory or in the library path of the linker.)
Linking against import libraries of DLLs works the same way.
Note 1: Keep in mind the different ABIs and name mangeling. However, calling plain C functions in DLL or LIB files will work in most cases.
Note 2: Keep in mind that g++ requires the libraries to be specified in the correct order.
#einpoklum Converting my comment to an answer: #n.18e9 is correct in that you must use the full path name for the lib file without any -L or -l options.
g++ -o foo.exe foo.o c:\something\somethingelse\some.lib. You can also link directly to the Windows DLL file g++ -o foo.exe foo.o c:\something\somethingelse\some.dll.
Important - make sure you are linking to a lib file (and associated dll) generated for a 64-bit platform (on MSVC target X64, not Win32).
OK you wanted an example, well let's go.
Here are two examples using gcc/g++ to link to a Windows native DLL which exports plain C functions (using here x86_64-w64-mingw32/8.3.0 on Windows 10).
I'm using my own free xmlsq library as an example https://www.cryptosys.net/xmlsq.
You can download the core native DLL and all the source code quoted below. Make sure you use the 64-bit DLL.
The native Windows DLL diXmlsq.dll is written entirely in plain C code and exports simple C functions (extern "C").
In particular, for this example, it exports a XMLSQ_Gen_Version function that returns an integer value.
The DLL was compiled using MSVC 12.0 targetting the X64 platform. The associated library file generated by MSVC is diXmlsq.lib.
I should add that this DLL works exactly the same as a Windows "Win32 API" DLL, so the instructions here should work for the standard Windows libraries in Windows\System32 (again make sure you link against the 64-bit version).
Example 1. A plain C interface.
Both these commands compile without warning on my system:
> gcc -o test-ver test-ver.c "C:\fullpath\to\x64\diXmlsq.lib"
> gcc -o test-ver test-ver.c "C:\fullpath\to\x64\diXmlsq.dll"
diXmlsq.dll is compiled using the following definition file.
(You could alternatively use __declspec(dllexport))
Ref: https://learn.microsoft.com/en-us/cpp/build/exporting-from-a-dll?view=msvc-160
diXmlsq.def
LIBRARY "diXmlsq"
EXPORTS
XMLSQ_Gen_Version
diXmlsq.h - the C interface to diXmlsq.dll
#ifdef __cplusplus
extern "C" {
#endif
long __stdcall XMLSQ_Gen_Version(void);
#ifdef __cplusplus
}
#endif
To call the core function in a plain C program:
test-ver.c
#include <stdio.h>
#include "diXmlsq.h"
int main(void)
{
long n;
n = XMLSQ_Gen_Version();
printf("Version = %ld\n", n);
return 0;
}
Example 2. A C++ interface.
Both these commands compile without warning using g++ .
> g++ -o test-simple test-simple.cpp xmlsq.cpp "C:\fullpath\to\x64\diXmlsq.lib"
> g++ -o test-simple test-simple.cpp xmlsq.cpp "C:\fullpath\to\x64\diXmlsq.dll"
The idea of the C++ interface is to be an interface to the plain C library using the more convenient STL types like std::string and std::vector.
To keep things simple we'll just demonstrate the Gen::Version method.
Extracts of the C++ code follow:
test-simple.cpp - a test C++ program.
#include <iostream>
#include "xmlsq.hpp"
int main()
{
std::cout << "xmlsq::Gen::Version=" << xmlsq::Gen::Version() << std::endl;
}
xmlsq.hpp - the C++ interface
namespace xmlsq
{
class Gen {
private:
Gen() {} // Static methods only, so hide constructor.
public:
/** Get version number of core diXmlsq DLL. */
static int Version();
};
}
xmlsq.cpp - the C++ implementation.
#include "diXmlsq.h"
#include "xmlsq.hpp"
namespace xmlsq
{
int Gen::Version() {
int n = XMLSQ_Gen_Version();
return n;
}
}
Example 3. Attempting to link to the 32-bit library by mistake.
> gcc -o test-ver test-ver.c "C:\fullpath\to\Win32\diXmlsq.lib"
C:/Strawberry/c/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:\Users\user\AppData\Local\Temp\cce27Dhl.o:test-ver.c:(.text+0xe):
undefined reference to `XMLSQ_Gen_Version'
collect2.exe: error: ld returned 1 exit status