I am trying to call a C code from Julia v0.6.2, Windows 10.
Here is the C code I compiled into a DLL with Cygwin gcc.
int timesTwo(int x) {
return 2*x;
}
The compile commands:
gcc -c mydll.c
gcc -shared -o mydll.dll mydll.o
I tested the DLL with a test C program and it works. Then I tried to call it with Julia using an absolute path to the DLL.
using Base.Libdl
x = Int32(2)
mylib = dlopen("C:\\Users\\pedro\\Documents\\codigos\\exampleDLL\\C\\mydll.dll")
ccall((:timesTwo, mylib), Int32, (Int32,), x)
Then the Julia's Workspace is restarted (all variables become undefined as soon as I use the command dlopen).
What is happening and how can I solve it?
Update
That must be a windows issue. Could not reproduce in an Ubuntu distro.
I would try to use the same compiler that Julia uses (with the -fPIC flag) to build your DLL. On Windows, julia is actually build with the
Cygwin-to-MinGW cross-compiler:
https://github.com/JuliaLang/julia/blob/master/README.windows.md#cygwin-to-mingw-cross-compiling
Can you install the mingw64-x86_64-gcc-core package (for just the C compiler) and use the x86_64-w64-mingw32-gcc compiler command (instead of gcc) ?
Use the -fPIC switch to generate the position independent code. Plus, you might be lying to Julia unless your int is defined as int32 on the specific platform. To be sure, you need to change ccall((:timesTwo, mylib), Int32, (Int32,), x) to ccall((:timesTwo, mylib), Cint, (Cint,), x). Finally, to make the code work for any operating system, please consider replacing mylib = dlopen("C:\\Users\\pedro\\Documents\\codigos\\exampleDLL\\C\\mydll.dll") to const mylib = dlopen(joinpath(pwd(), "mydll.dll")). const is there for efficiency reasons, joinpath is OS-agnostic, meaning that it will add forward slash for *nix-based systems. Later on, when you want to change the library to .so, for instance, you should not deal with slashes, too.
Related
EDIT: For any poor soul that finds this, in search of a solution for the shared library from go conundrum: I was unable to find a solution that uses go and I would suggest, that until google go provides native c-shared support for AIX you should find an alternative for your project.
I did not go forward with gccgo because that felt like an entirely different can of worms that I was unwilling to delve further into. FWIW I myself am going forward switching to pure C implementation, because there I at least have a (somewhat) firm(er) understanding of it all.
Should anyone find a solution I'd love to hear from you and see how you got around this limitation.
Environment:
AIX 7.2
go1.16.12 aix/ppc64
gcc-8
I want to create a C shared object library (in usual unix vernacular a .so file) out of a golang project on AIX 7.2 so that it can be used by a C application.
I can compile it down to a final a.out binary in my example, but it can then not be executed because the shared object is apparently compiled the wrong way.
So far I have achieved the following:
Suppose my example go "library" sharedLibTest.go:
package main
import (
m "fmt"
)
import "C"
func main() {
fmt.Printf("%s\n", "Golang: main was called")
MyPackage_Init()
MyPackage_Create()
}
//export MyPackage_Init
func MyPackage_Init() {
fmt.Printf("%s\n", "Golang: MyPackage_Init was called")
}
//export MyPackage_Create
func MyPackage_Create() {
fmt.Printf("%s\n", "Golang: MyPackage_Create was called")
}
And some C application that calls these functions in main.c:
#include <stdio.h>
#include "sharedLibTest.h"
int main() {
printf("%s\n", "C: main() called");
MyPackage_Init();
MyPackage_Create();
}
m
Now, because AIX feels the need to do things differently the current golang toolchain does not support directly creating a c-shared object with -buildmode=c-shared. Instead I am trying to do the roundabout way by first creating a static lib with -buildmode=c-archive, compiling that into a shared object using gcc-8 and use that in my "target C application".
I can compile sharedLibTest.go this with
go build -v -buildmode=c-archive -mod vendor -o /home/myuser/workspace/go_proj/sharedLibTest/sharedLibTest.a /home/myuser/workspace/go_proj/sharedLibTest/sharedLibTest.go
Because the symbols MyPackage_Init and MyPackage_Create are not exported by default in AIX, I need to manually create an extra symbol file with
$ cat > file.exp << EOF
> MyPackage_Init
> MyPackage_Create
> EOF
Source
(If there are any ideas how i can omit this file.exp step I'd really appreciate it)
Now with that I can compile a shared object out of that by running
gcc -g -O2 -mcpu=power7 -maix64 -shared -lpthread -Wl,-bE:file.exp -o libsharedLibTest.so -Wl,-bnoobjreorder ./sharedLibTest.a
Now because AIX does not look for .so files but only .a files even if they are shared libraries, I rename the resulting libsharedLibTest.so into libsharedLibTest.a with
mv libsharedLibTest.so libsharedLibTest.a
Lastly I want to compile my C applications with
gcc -L/home/myuser/workspace/go_proj/sharedLibTest -g -O2 -mcpu=power7 -maix64 -Wl,-bnoobjreorder -lsharedLibTest -lpthreads main.c
This succeeds and I get my a.out file as a result.
However, when I try to run this with the following, I only get the error below
LD_LIBRARY_PATH=/home/myuser/workspace/go_proj/sharedLibTest ./a.out
$ ./a.out
exec(): 0509-036 Cannot load program ./a.out because of the following errors:
0509-150 Dependent module /home/myuser/workspace/go_proj/sharedLibTest/libsharedLibTest.a(libsharedLibTest.so) could not be loaded.
0509-187 The local-exec model was used for thread-local
storage, but the module is not the main program.
0509-193 Examine the .loader section header with the
'dump -Hv' command.
Some hours of googling so far have revealed that I might be missing the compile option -fPIC to create "emit position-independent code" however adding that flag to any of the above steps in various combinations has all resulted in the same error.
Clearly I need to add some compile option to tell the shared object not to be thread-local, however I am unclear how. Any ideas?
Few points... mv will not make an archieve, ar will. You need to use ar command to create .a file.
Second, use LIBPATH environment variable in place of LD_LIBRARY_PATH. Use of -fPIC option is irrelevant on AIX.
I am trying to create a couple of Win32 64-bit DLLs (Windows 10) which have different implementations but consistent symbol exports. The aim for this is that one would link with whichever one at build time but have the option at deployment to install either DLL and correctly run with that. I have achieved this straightforwardly on Linux where I am much more comfortable and familiar with run-time linking. But on Windows, I have not yet managed this and I am wondering if this is possible at all. I am trying this using both VS2010 and VS2019.
Suppose I have two libraries blah_legacy.dll and blah_modern.dll. They both export 6 symbols which are the interface to using the library, e.g. blah_open, blah_read, blah_write, blah_close, blah_control, blah_status.
I can link with the import library for either blah implementation and a test program calling each symbol loads and executes correctly with the corresponding blah DLL.
However, I cannot yet switch the DLLs at run time. For example, should I actually be able to link with blah-legacy.lib and then run with blah-modern.dll if I rename it to blah-legacy.dll? (Or vice-versa.)
I already got around basic file-naming issues and ensured the DLL needed can actually be found. I still got the application failed to start (0x22).
I used "objdump -xs" on the DLLs and noticed the order of symbols and their ordinals are different. So I created a .def file and ensured that the exported symbols match in number, names and in ordinals. Still nothing - the same error occurs.
There's still something to this I clearly have not figured out and would appreciate some guidance. Is this actually possible? Where do I start to look (which tools) to figure out what step to take next.
Yes.
I don't use Visual Studio much, but this is the kind of thing that happens all the time if you use MSYS2, and install some MinGW packages, and update them.
Here's what I mean by that: MSYS2 is an open source software distribution for Windows that, among other things, provides a bunch of native Windows software packages. The package manager (pacman) let's you choose which packages to have in your system, and it downloads DLLs and EXEs that were created by the MSYS2 developers. When an MSYS2 developer updates a library, you can download the updated library package, and all the other packages using that library will automatically start using the new DLL. Usually there is no issue with that because the new library version will be ABI-compatible with the old library version.
You do not need to use LoadLibrary or otherwise mess up your source code; the linker and the operating system should be able to take care of this for you.
Example
Here is a minimal example I threw together with MSYS2 showing how this can work.
The file foo_legacy.c represents your legacy DLL. I added some extra symbols so it wouldn't be too similar to the modern DLL.
__declspec(dllexport) int eoo() {
return 0;
}
__declspec(dllexport) const char * foo_name() {
return "legacy";
}
__declspec(dllexport) int foo_version() {
return 1;
}
__declspec(dllexport) int goo() {
return 0;
}
The file foo_modern.c represents the modern implementation:
__declspec(dllexport) const char * foo_name(void);
__declspec(dllexport) int foo_version(void);
int foo_version() {
return 2;
}
const char * foo_name() {
return "modern";
}
The file main.c represents an application using the foo API:
#include <stdio.h>
__declspec(dllimport) const char * foo_name(void);
__declspec(dllimport) int foo_version(void);
int main()
{
printf("%s %d\n", foo_name(), foo_version());
}
My build.sh file is a Bash script that builds and tests everything:
#!/usr/bin/bash
set -uex
gcc -Wall foo_legacy.c -shared -o foo_legacy.dll
gcc -Wall foo_modern.c -shared -o foo_modern.dll
gcc -Wall -c main.c -I. -o main.o
gcc main.o foo_legacy.dll -o main.exe
./main.exe # output: "legacy 1"
mv foo_modern.dll foo_legacy.dll
./main.exe # output: "modern 2"
rm foo_legacy.dll
./main.exe # fails because foo_legacy.dll is not found
The build script runs main.exe three different times, showing that it can either use the legacy DLL, or use the modern DLL, or fail, depending on what was installed in foo_legacy.dll.
I want to run serial commands from a Bealgebone to a 4Dsystems display. Therefore I copied the c library found here into a directory and created a test program main.c:
#include "Picaso_const4D.h"
#include "Picaso_Serial_4DLibrary.h"
int main(int argc,char *argv[])
{
OpenComm("/dev/ttyUSB0", B115200); // Matches with the display "Comms" rate
gfx_BGcolour(0xFFFF);
gfx_Cls();
gfx_CircleFilled(120,160,80,BLUE);
while (1) {}
}
Now when I do gcc -o main main.c its says
main.c:2:37: fatal error: Picaso_Serial_4DLibrary.h: No such file or
directory
So I try linking it:
gcc main.c -L. -lPICASO_SERIAL_4DLIBRARY
which gives me the same error. Then I tried to create a static library:
gcc -Wall -g -c -o PICASO_SERIAL_4DLIBRARY PICASO_SERIAL_4DLIBRARY.C
which gives me this:
PICASO_SERIAL_4DLIBRARY.C:1:21: fatal error: windows.h: No such file
or directory compilation terminated.
What am I doing wrong? the git page clearly says this library is created for people who do not run windows.
Thanks in advance!
You're not getting a linker error; you're getting a preprocessor error. Specifically, your preprocessor can't find Picaso_Serial_4DLibrary.h. Make sure that it's in your include path; you can add directories to your include path using the -I argument to gcc.
You've had two problems. First was the picaso_whatever.h file that couldn't be found. You fixed that with the -I you added. But, now, the picaso.h wants windows.h
What are you building on? WinX or BSD/Linux?
If you're compiling on WinX, you need to install the "platform sdk" for visual studio.
If you're using mingw or cygwin, you need to do something else.
If on WinX, cd to the C: directory. Do find . -type f -name windows.h and add a -I for the containing directory.
If under Linux, repeat the find at the source tree top level. Otherwise, there is probably some compatibility cross-build library that you need to install.
Or, you'll have to find WinX that has it as Picaso clearly includes it. You could try commenting out one or more of the #include's for it and see if things are better or worse.
If you can't find a real one, create an empty windows.h and add -I to it and see how bad [or good] things are.
You may need the mingw cross-compiler. See https://forums.wxwidgets.org/viewtopic.php?t=7729
UPDATE:
Okay ... Wow ... You are on the right track and close, but this is, IMO, ugly WinX stuff.
The primary need of Picaso is getting a serial comm port connection, so the need from within windows.h is [thankfully] minimal. It needs basic boilerplate definitions for WORD, DWORD, etc.
mingw or cygwin will provide their own copies of windows.h. These are "clean room" reimplementations, so no copyright issues.
mingw is a collection of compile/build tools that let you use gcc/ld/make build utilities.
cygwin is more like: I'd like a complete shell-like environment similar to BSD/Linux. You get bash, ls, gcc, tar, and just about any GNU utility you want.
Caveat: I use cygwin, but have never used mingw. The mingw version of windows.h [and a suite of .h files that it includes underneath], being open source, can be reused by other projects (e.g. cygwin, wine).
Under Linux, wine (windows emulator) is a program/suite that attempts to allow you to run WinX binaries under Linux (e.g. wine mywinpgm).
I git cloned the Picaso library and after some fiddling, I was able to get it to compile after pointing it to wine's version of windows.h
Picaso's OpenComm is doing CreateFile [a win32 API call]. So, you'll probably need cygwin. You're opening /dev/ttyUSB0. /dev/* implies cygwin. But, /dev/ttyUSB0 is a Linux-like name. You may need some WinX-style name like "COM:" or whatever. Under the cygwin terminal [which gives you a bash prompt], do ls /dev and see what's available.
You can get cygwin from: http://cygwin.com/ If you have a 64 bit system, be sure to use the 64 bit version of the installer: setup-x86_64.exe It's semi-graphical and will want two directories, one for the "root" FS and one to store packages. On my system, I use C:\cygwin64 and C:\cygwin64_packages--YMMV.
Note that the installer won't install gcc by default. You can [graphically] select which packages to install. You may also need some "devel" packages. They have libraries and .h files that a non-developer wouldn't need. As, docs mention, you can rerun the installer as often as you need. You can add packages that you forgot to specify or even remove ones that you installed that you don't need anymore.
Remember that you'll need to adjust makefile -I and/or -L option appropriately. Also, when building the picaso library, gcc generated a ton of warnings about overflow of a "large integer". The code was doing:
#define control_code -279
unsigned char buf[2];
buf[0] = control_code >> 8;
buf[1] = control_code;
The code is okay, and the warning is correct [because the code is sloppy]. If the code had done:
#define control_code -279
unsigned char buf[2];
buf[0] = (unsigned) control_code >> 8;
buf[1] = (unsigned) control_code;
it probably would have been silent. Use -Wno-overflow in your Makefile to get rid of the warnings rather that edit 50 or so lines
I am constructing an R package where I have few third party C functions which use GSL and OpenMP, and I then call these from R using wrappers made with Rcpp (which just modify some arguments and call the appropriate C function). Everything works fine in my Windows machine, but I am not sure how to define the Makevars.win and Makevars file in a portable way. My Makevars.win looks like this:
## This assumes that the LIB_GSL variable points to working GSL libraries
PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
PKG_LIBS=-L$(LIB_GSL)/lib/x64 -lgsl -lgslcblas $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS)
It is basically copied from various sources. Like said, this compiles in my computer (using toolchain of from Rtools), and if I remove PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS) I can also compile without OpenMP (for some reason I don't understand I get error if I remove OpenMP flag in PKG_LIBS).
My Makevars file looks identical, but I don't have access to Unix platforms so I am not sure how to deal with that side. My guess is that I need to replace LIB_GSL with something?
EDIT:
Okay, I think I finally understand how configure etc. works and was able to get everything working.
My Makevars.win:
## This assumes that the LIB_GSL variable points to working GSL libraries
PKG_CPPFLAGS=-I$(LIB_GSL)/include -I../inst/include
PKG_LIBS="-L$(LIB_GSL)/lib/$(R_ARCH)" -lgsl -lgslcblas $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(SHLIB_OPENMP_CFLAGS)
My Makevars.in:
GSL_CFLAGS = #GSL_CFLAGS#
GSL_LIBS = #GSL_LIBS#
PKG_LIBS=$(GSL_LIBS) $(SHLIB_OPENMP_CFLAGS)
PKG_CFLAGS=$(GSL_CFLAGS) $(SHLIB_OPENMP_CFLAGS)
And my configure.ac:
AC_INIT([testpackage], 0.0.1)
## Use gsl-config to find arguments for compiler and linker flags
##
## Check for non-standard programs: gsl-config(1)
AC_PATH_PROG([GSL_CONFIG], [gsl-config])
## If gsl-config was found, let's use it
if test "${GSL_CONFIG}" != ""; then
# Use gsl-config for header and linker arguments
GSL_CFLAGS=`${GSL_CONFIG} --cflags`
GSL_LIBS=`${GSL_CONFIG} --libs`
else
AC_MSG_ERROR([gsl-config not found, is GSL installed?])
fi
# Now substitute these variables in src/Makevars.in to create src/Makevars
AC_SUBST(GSL_CFLAGS)
AC_SUBST(GSL_LIBS)
AC_OUTPUT(src/Makevars)
I then run autoconf in testpackage directory in order to get configure file, which in turn converts Makevars.in to Makevars when running R CMD INSTALL.
There are a few CRAN packages using the GSL, and/or our RcppGSL bindings. Here is what I do in one of these (my RcppZiggurat package):
PKG_CPPFLAGS = -I. -I../inst/include
## Use the R_HOME indirection to support installations of multiple R version
PKG_LIBS = `$(R_HOME)/bin/Rscript -e "RcppGSL:::LdFlags()"`
That is the entire /src/Makevars. You can trivially add the same OpenMP variable from R. This does of course create a dependency on RcppGSL (as I already use it in RcppZiggurat). Else you can look into the R/init.R of RcppGSL and see how it tries to talk to gsl-config and store those values. You can do the same in src/Makevars -- it is just basic Make usage and nothing Rcpp specific.
Is it possible to load a DLL with C and use its functions?
I am new in C, and I am trying to search some good references on the internet for this; but I can't find any.
Any help would be appreciated!
I am using GNU GCC in Code::Blocks on Windows 7, 64 Bit.
HMODULE hModule = LoadLibrary(<dll file name>) followed by GetProcAddress(hModule, <function name>) will do this job using the WinAPI.
An example could be found here.
I think you should investigate the LoadLibrary function.
http://msdn.microsoft.com/en-us/library/ms684175.aspx
Loads the specified module into the address space of the calling process. The specified module may cause other modules to be loaded.
Building a DLL using MinGW, here are some instructions:
First, you need to mark your functions for export, so they can be used by callers of the DLL. To do this, modify them so they look like (for example)
__declspec( dllexport ) int add2(int num){
return num + 2;
}
then, assuming your functions are in a file called funcs.c, you can compile them:
gcc -shared -o mylib.dll funcs.c
The -shared flag tells gcc to create a DLL.
For a free IDE which will automate all the flags etc. needed to build DLLs, take a look at the excellent Code::Blocks, which works very well with MinGW.
Also, see the article Creating a MinGW DLL for Use with Visual Basic on the MinGW Wiki.