I finished working on my Checkers game engine and about to make it CheckerBoard (http://www.fierz.ch/cbdeveloper.php) compatible, and it states that.
CheckerBoard expects your engine to be compiled as a dll and to be in
the working directory of checkerboard.exe - or in the path. An engine
must support 2 required functions. ..., you must provide 2 more
functions for multi-version support. The calling convention for all
functions is __stdcall().
Required Functions
The current CheckerBoard API (version 2) requires the following 2
functions:
int WINAPI getmove(int board[8][8], int color, double maxtime, char str[1024], int *playnow, int info, int moreinfo, struct CBmove *move);
int WINAPI enginecommand(char command[256], char reply[1024]);
my engine includes the two above functions, i've got everything setup (the code) but i'm having troubles compiling it as dll, heres what i tried (using gcc)
gcc -c engine.c
gcc -shared -o engine.dll engine.o
it creates the dll as expected but the dll doesn't export any of the functions (as expected).
I used the Dependency walker program to scan the engine.dll file, and it shows that the dll exports the functions
_getmove#28
_enginecommand#20
i scanned one of the engines in the CheckerBoard directory and i found that they export the following functions:
getmove
enginecommand
i cant figure out why the dll i created exports differently, maybe its got
something to do with the WINAPI ?.
Any ideas ?
here's what it looks like
#define EXPORT __declspec(dllexport)
EXPORT int WINAPI getmove(int b[8][8], int color, double time, char str[1024], int *playnow, int info, int unused, struct CBmove *cbmove){
// ....
return 0;
}
EXPORT int WINAPI enginecommand (char str[256], char reply[1024]){
// ...
return 0;
}
i need both functions to be exported through the dll.
You probably need to annotate (the declaration of) these exported functions with Windows function attributes like __attribute__((dllexport)); I recommend having a declaration like
extern WINAPI getmove(int b[8][8], int color, double time,
char str[1024], int *playnow, int info,
int unused, struct CBmove *cbmove)
__attribute__((dllexport));
before that function's definition.
You may need to compile your source as position independent code (on Linux it is nearly required with -fPIC, for Windows DLL you need to check your documentation).
So try to compile with gcc -Wall -O -g -fPIC -shared engine.c -o engine.dll
See also this related question (and answers there).
PS. I never used Windows (only Unix since 1987 and Linux since 1993). I recommend reading Levine's Linkers and loaders book
Related
Background
Trying to profile an executable, I experimented the profiler Intel VTune and I learn that there is an API library (ITT) that provide utility to start/stop profiling. Its basic functions __itt_resume() and __itt_pause(). What triggers me is that the library is optional, i.e. if the runtime library of ITT is not loaded, these functions are basically noops.
Optional library?
I want to know (first of all on Linux)
Does a process checks that the dynamic library he is linking to is loaded when he starts or when each symbol, or the first symbol of the library is called at runtime (i.e. lazy initialization)? I think on Windows it's at startup because of can't find XXX.dll messages, but I am not sure on Linux. Also, with the example, I don't get any compilation & execution issues even if the symbol is not defined in some_process.c.
How to implement this on Linux? Looking at the Github repo of ITT, among many macro trickery, I feel like the key is here:
#define ITTNOTIFY_VOID(n) (!ITTNOTIFY_NAME(n)) ? (void)0 : ITTNOTIFY_NAME(n)
Basically it wraps every function call with a function pointer call if its not NULL.
How to implement this in a cross-platform way (Windows, Mac, Linux) ?
I end up with a minimal example that looks like the code linked here, but it does not work as it should. In the linked version, my_api_hello_impl() is not called as it should. Also, there is no crash checking the value of the extern symbol api_hello_ptr() when the library is not linked.
my_api.c
#include "my_api.h"
#include <stdio.h>
void(*api_hello_ptr)();
void api_hello_impl()
{
printf("Hello\n");
}
__attribute__((constructor))
static void init()
{
printf("linked\n");
api_hello_ptr = api_hello_impl;
}
my_api.h
#pragma once
extern void(*api_hello_ptr)();
inline void api_hello() { if(api_hello_ptr) api_hello_ptr(); }
some_process.c
#include "my_api.h"
int main()
{
// NOOPS of not linked at runtime
api_hello();
}
Makefile
# my_api is not linked to some_process
some_process: some_process.c my_api.h
$(CC) -o $# $<
my_api.so: my_api.c my_api.h
$(CC) -shared -fPIC -o $# $<
test_linked: some_process my_api.so
LD_PRELOAD="$(shell pwd)/my_api.so" ./some_process
test_unlinked: some_process my_api.so
./some_process
.PHONY: test_linked test_unlinked
Output:
$ make test_linked
LD_PRELOAD="/tmp/tmp.EkrQbILrNg/my_api.so" ./some_process
linked
$ make test_unlinked
./some_process
Does a process checks that the dynamic library he is linking to is loaded when he starts
Yes, it does. If a dynamic library is linked, then it is a runtime requirement and the system loader will not start execution of a program without finding and loading the library first. There are mechanisms for delayed-loading, but it is not the norm on Linux, they are done manually or using custom libraries. By default, all dynamically linked objects need to be loaded before execution starts.
Note: I'm assuming we are talking about ELF executables here since we are on Linux.
How to implement this on Linux?
You can do it using macros or wrapper functions, plus libdl (link with -ldl), with dlopen() + dlsym(). Basically, in each one of those wrappers, the first thing you do is check if the library was already loaded, and if not, load it. Then, find and call the needed symbol.
Something like this:
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
static void *libfoo_handle = NULL;
static int (*libfoo_func_a)(int, int);
static void load_libfoo_if_needed(void) {
if (!libfoo_handle) {
// Without "/" in the path, this will look in all standard system
// dynamic library directories.
libfoo_handle = dlopen("libfoo.so", RTLD_LAZY | RTLD_GLOBAL);
if (!libfoo_handle) {
perror("failed to load libfoo.so");
_exit(1);
}
// Optionally use dlsym() here to initialize a set of global
// function pointers, so that you don't have to do it later.
void *tmp = dlsym(libfoo_handle, "func_a");
if (!tmp) {
perror("no symbol func_a in libfoo.so");
_exit(1);
}
*((void**)&libfoo_func_a) = tmp;
}
}
int wrapper_libfoo_func_a(int a, int b) {
load_libfoo_if_needed();
return libfoo_func_a(a, b);
}
// And so on for every function you need. You could use macros as well.
How to implement this in a cross-platform way (Windows, Mac, Linux)?
For macOS, you should have dlopen() and dlsym() just like in Linux.
Not sure how to exactly do this on Windows, but I know there is LoadLibrary() available in different flavors (e.g. one, two, etc.), which should be more or less the equivalent of dlopen() and GetProcAddress(), which should be the equivalent of dlsym().
See also: Loading a library dynamically in Linux or OSX?
I'm working on an application which defines it's own printf() to get around differences between the different CRTs out there or because some other platforms don't have it.
When building the application with gcc this automatically seems to work and the custom printf is used instead of libc's one; if I understand it correctly this is because of the order in which object files/libraries appear in the link command or maybe because object files are always searched before CRT libs, correct?
I'd like to do the same using msvc. Just building the project gives the expected 'LNK2005: _printf already defined in printf.obj' because printf is also in msvcrtd.lib. Fair enough. I know about /NODEFAULTLIB but that excludes everything resulting in unresolved references for everything but printf. I scanned through the other linker settings but couldn't find anything which allows this (apart from /FORCE maybe, but the 'might produce an invalid executable' comment doesn't make it sound like a good idea). Also nothing in the module definition file docs; the latter got me thinking it might be possible to create a stub library which has all exports from msvcrt.lib except printf but that seems a brittle solution even if it works.
In the end the question is simple: how do I tell msvc's linker it should skip msvcrt's printf definition and use the one from my printf.obj instead. Basically /NODEFAULTFUNCTION:printf or so. Just an answer for one single executable is ok, though I'd also be interested to know if and how it can be done when building a dll instead where the custom printf is exported: how to tell the linker it should use the export from my .lib instead of msvcrt.lib?
edit simplest repo I could find: create a file main.c:
#include <stdio.h>
int main(int argc, char** argv)
{
printf("Hello");
return 0;
}
and a file printf.c:
int printf(const char *fmt, ...)
{
write(1, "ok\n", 3);
return 3;
}
For VS2013 (though the other versions might work as well): create a new empty C++ project and add both files then build. (For gcc: just gcc main.c printf.c and the resulting a.out prints 'ok')
The culptrit for VS is #include : without that it works ok but I have yet to find out if the original code allows getting rid of it in some way. But even if it does I'd still want to know if this can be solved at the link level.
Ever since Go 1.5 came out, I started taking another look at how I could integrate it into an existing project of mine.
The project's codebase is written entirely in C for low level access to to hardware and other fun stuff. However, some of the higher level things are tedious, and I would like to start writing them in a higher level language (Go)
Is there any way I can call Go code from a C program? I installed Go 1.5, which added -buildmode=c-archive (https://golang.org/s/execmodes) which I am trying to get working.
However, I can't seem to get Go to generate the appropriate header files to allow my project to actually compile. When I generate the archive, I see the function in the exported symbols (using objdump), but without the header files to include gcc complains about the function not existing (as expected)
I'm quite new to Go - however, I love the language and would like to make use of it. Is there any idiomatic way ("idiomatic" gets used a lot in the world of Go I see...) to get this to play nicely with each other?
The reason I asked this question and specifically mentioned Go 1.5 is that according to this document, https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ/edit?pli=1#heading=h.1gw5ytjfcoke
Go 1.5 added support for non-Go programs to call Go code. Specifically, mentioned under the section "Go code linked into, and called from, a non-Go program"
To build an archive callable from C, you will need to mark them as exported CGo symbols.
For example, if I create a file foo.go with the following contents:
package main
import (
"C"
"fmt"
)
//export PrintInt
func PrintInt(x int) {
fmt.Println(x)
}
func main() {}
The important things to note are:
The package needs to be called main
You need to have a main function, although it can be empty.
You need to import the package C
You need special //export comments to mark the functions you want callable from C.
I can compile it as a C callable static library with the following command:
go build -buildmode=c-archive foo.go
The results will be an archive foo.a and a header foo.h. In the header, we get the following (eliding irrelevant parts):
...
typedef long long GoInt64;
...
typedef GoInt64 GoInt;
...
extern void PrintInt(GoInt p0);
...
So that's enough to call the exported function. We can write a simple C program that calls it like so:
#include "foo.h"
int main(int argc, char **argv) {
PrintInt(42);
return 0;
}
We can compile it with a command like:
gcc -pthread foo.c foo.a -o foo
The -pthread option is needed because the Go runtime makes use of threads. When I run the resulting executable it prints 42.
The code above work just fine, but gcc will complain about functions and headers.
The includes should be:
#define _GNU_SOURCE
#include <stdio.h>
#include "mygopkg.h"
If you forget the #define _GNU_SOURCE, the gcc will complain:
warning: implicit declaration of function 'asprintf'; did you mean 'vsprintf'? [-Wimplicit-function-declaration]
If you forget the #include "mygopkg.h", the gcc will complain:
warning: implicit declaration of function 'PrintString' [-Wimplicit-function-declaration]
The last but not less important. The build command line I recommend for production code is:
go build -ldflags "-s -w" -buildmode c-archive -o mygopkg.a
It'll save you 53% size of final mygopkg.a.
I'm sort of a newbie to C coding but I've written a Matlab program for simulating neural networks and I wish to translate it to C code because our supercomputer cluster won't allow running more than a few Matlab simulations at once. To that end, I've found GotoBLAS to take care of the matrix math.
Unfortunately I'm not sure how to use it as I don't have a lot of experience in C and using external libraries. I'm assuming that 'dgemm' is a function in GotoBLAS from reading the BLAS guide pdf. I've been able to successfully compile GotoBLAS, but when I do:
gcc -o outputprog main.c -Wall -L -lgoto2.a
I get the messages:
undefined reference to 'dgemm'
As I understand it, I should be including some .h file (or maybe not) from GotoBLAS but I'm not sure which one (or if this is right at all).
Any help with this would be appreciated. Let me know if more information is needed.
One problem could be that the -L option expects a 'directory' name after it, and therefore gcc (or the linker invoked by gcc) is treating -lgoto2.a as a directory. The compiler does not complain about non-existent directories; it simply ignores them. Which directory did you expect to find the library in? (For the purposes of this answer, I'll assume it is in /usr/local/lib.)
Another problem could be that the library is not called libgoto2.a.a or libgoto2.a.so or something similar. You would not normally specify the .a suffix. (For the purposes of this answer, I'll assume that the library is either libgoto2.a or libgoto2.so.)
It appears that you don't need to specify where the headers are found; that means they're in a sufficiently conventional location that the compiler looks there anyway. If that's correct, the library too may be in a sufficiently conventional location too, and the -L option may be unnecessary.
So, you might be able to use:
gcc -Wall -o outputprog main.c -lgoto2
Or you might need to use:
gcc -Wall -o outputprog main.c -L/usr/local/lib -lgoto2
After some extensive discussion in the comments, and the information that the library is in the current directory and named libgoto2.a and that the symbol dgemm is still missing, I downloaded GotoBLAS2 version 1.13 and tried to compile it on a semi-supported platform (MacOS X, probably pretending to be Linux, with x86_64 architecture). The build was not completely successful - problems in some assembler code. However, poking around at the headers, there is one that looks like giving the solution to your problems:
cblas.h
In this, amongst many other function definitions, we find:
void cblas_dgemm(enum CBLAS_ORDER Order, enum CBLAS_TRANSPOSE TransA,
enum CBLAS_TRANSPOSE TransB, blasint M, blasint N, blasint K,
double alpha, double *A, blasint lda, double *B, blasint ldb,
double beta, double *C, blasint ldc);
All the function symbols in the header are prefixed with cblas_. Your code should be using:
#include "cblas.h"
You should be calling the functions using the Fortran name (in lower case) prefixed with cblas_:
cblas_dgemm(...);
And the correct link line to use is the first option listed above:
gcc -Wall -o outputprog main.c -lgoto2
At a pinch, you could define macros to map the regular (unprefixed) names to the correct C function names, but I'm not convinced it is worth it:
#define DGEMM cblas_dgemm
or (safer, because it checks the length of the argument list, but more verbose):
#define DGEMM(a,b,c,d,e,f,g,h,i,j,k,l,m,n) cblas_dgemm(a,b,c,d,e,f,g,h,i,j,k,l,m,n)
You can then write:
DGEMM(a, ..., n);
and the correct function would be called.
Experimentation with the partially successful build of GotoBLAS2 mentioned above shows that:
cblas.h is not self-contained (contrary to good coding standards).
common.h must be included before it.
common.h includes a lot of other headers:
config.h
common_x86_64.h
param.h
common_param.h
common_interface.h
common_macro.h
common_s.h
common_d.h
common_q.h
common_c.h
common_z.h
common_x.h
common_level1.h
common_level2.h
common_level3.h
common_lapack.h
The following code stands a chance of linking with a complete library:
#include "common.h"
#include "cblas.h"
void check_dgemm(void)
{
double A[33] = { 0.0 };
double B[33] = { 0.0 };
double C[33] = { 0.0 };
cblas_dgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
3, 3, 3, 2.0, A, 3, B, 3, 3.0, C, 3);
}
int main(void)
{
check_dgemm();
return 0;
}
(In my admittedly broken build of the library, the complaints went from being 'cblas_dgemm() not found' to a number of other functions missing. This is a vast improvement!)
Ok I was able to find the answer on the GotoBLAS mailing list https://lists.tacc.utexas.edu/mailman/listinfo/gotoblas (which is not listed on the website as far as I can see). Here's a quick step by step on using GotoBLAS2 with C and GCC compiler.
Build GotoBLAS2 libraries (.so and .a), there's good documentation on that included with the libraries so I won't post it here. Include BOTH of these files in the libs directory of your choice as set by -L. I was only including one because I thought they were just different versions of the same library which was not correct.
Also link to -lgfortran as well if you wish to compile with gcc. -lpthread might also be useful, although I'm not sure, I've seen examples with it but it compiles without. Your gcc should look something like this:
gcc -Wall -o outprog -L./GotoLIBSDIR -lgoto2 -lgfortran -lpthread(maybe) main.c
Finally, call function_() instead of function(), so for example, dgemm_() when using gfortran to compile the fortran interfaces.
Alternatively to the fortran interface the cblas interface can be used as cblas_dgemm(). You still need to link to -lgfortran for this as otherwise linking to libgoto2.so will fail, and you need to link to that file to be able to use cblas_dgemm() correctly.
There doesn't appear to be any need to include any of the .h files or anything else.
Hopefully someone else will find this useful. Thanks for all the help!
How do you have a header file for a function and the implementation of that function in different files? Also, how do you have main in yet another file and call this function?
The advantage is so that this function will then be an independent component which can be reused, right?
This is best illustrated by an example.
Say we want a function to find the cube of an integer.
You would have the definition (implementation) in, say, cube.c
int cube( int x ) {
return x * x * x;
}
Then we'll put the function declaration in another file. By convention, this is done in a header file, cube.h in this case.
int cube( int x );
We can now call the function from somewhere else, driver.c for instance, by using the #include directive (which is part of the C preprocessor) .
#include "cube.h"
int main() {
int c = cube( 10 );
...
}
Finally, you'll need to compile each of your source files into an object file, and then link those to obtain an executable.
Using gcc, for instance
$ gcc -c cube.c #this produces a file named 'cube.o'
$ gcc -c driver.c #idem for 'driver.o'
$ gcc -o driver driver.c cube.c #produces your executable, 'driver'
Actually you can implement any function in header files for better performance(when implementing libraries for example) as long are not referenced to a specific object(actually it won't compile that).
By the way even with that way, you have separate interface and implementation ;)
Of course you will have include gurads in you header files to avoid "multiple definition" errors.
In C/C++, non-inline functions should be defined only once. If you put function defination
in header files, you will get "multiple defination" link error when the header file is included more than once.