How do I make a struct for FFI that contains a nullable function pointer? - c

Question

I have an existing C program that loads shared library plugins. The main C program interacts with those plugins through a C struct containing integers, strings, function pointers, etc. How can I create such a plugin from Rust?
Note that the (real) C program cannot be changed, nor can the API be changed, those are fixed, existing things, so this is not a question about "how best to support plugins in Rust", it's how can Rust make *.so files which interoperate with an existing C program.
Here's a simplified example of a C program + C plugin:
/* gcc -g -Wall test.c -o test -ldl
./test ./test-api.so
*/
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <inttypes.h>
#include <dlfcn.h>
struct api {
uint64_t i64;
int i;
const char *name; /* can be NULL */
void (*load) (void); /* must not be NULL */
void (*hello) (const char *str); /* can be NULL */
};
int
main (int argc, char *argv[])
{
void *dl = dlopen (argv[1], RTLD_NOW);
if (!dl) { fprintf (stderr, "%s: %s\n", argv[1], dlerror ()); exit (1); }
struct api *(*get_api) (void) = dlsym (dl, "get_api");
printf ("calling get_api ...\n");
struct api *api = get_api ();
printf ("api->i64 = %" PRIi64 "\n", api->i64);
printf ("api->i = %d\n", api->i);
if (api->name)
printf ("api->name = %s\n", api->name);
printf ("calling api->load ...\n");
api->load ();
if (api->hello) {
printf ("calling api->hello ...\n");
api->hello ("world");
}
printf ("exiting\n");
exit (0);
}
/* gcc -g -shared -fPIC -Wall test-api.c -o test-api.so */
#include <stdio.h>
#include <stdint.h>
static void
load (void)
{
printf ("this is the load function in the plugin\n");
}
static void
hello (const char *str)
{
printf ("hello %s\n", str);
}
static struct api {
uint64_t i64;
int i;
const char *name;
void (*load) (void);
void (*hello) (const char *str);
} api = {
1042,
42,
"this is the plugin",
load,
hello,
};
struct api *
get_api (void)
{
return &api;
}
Here's what I wrote in Rust to try to get a plugin, but it doesn't compile:
extern crate libc;
use libc::*;
use std::ffi::*;
use std::ptr;
use std::os::raw::c_int;
#[repr(C)]
pub struct api {
i64: uint64_t,
i: c_int,
name: *const c_char,
load: extern fn (),
hello: extern fn (), // XXX
}
extern fn hello_load () {
println! ("hello this is the load method");
}
#[no_mangle]
pub extern fn get_api () -> *const api {
println! ("hello from the plugin");
let api = Box::new (api {
i64: 4201,
i: 24,
name: CString::new("hello").unwrap().into_raw(), // XXX memory leak?
load: hello_load,
hello: std::ptr::null_mut,
});
return Box::into_raw(api); // XXX memory leak?
}
This is compiled using Cargo.toml containing:
[package]
name = "embed"
version = "0.1.0"
[dependencies]
libc = "0.2"
[lib]
name = "embed"
crate-type = ["cdylib"]
The error is:
error[E0308]: mismatched types
--> src/lib.rs:32:16
|
32 | hello: std::ptr::null_mut,
| ^^^^^^^^^^^^^^^^^^ expected "C" fn, found "Rust" fn
|
= note: expected type `extern "C" fn()`
found type `fn() -> *mut _ {std::ptr::null_mut::<_>}`
error: aborting due to previous error
I didn't get to try loading the module but when I tried this before with the real program the fields were all wrong indicating something much more fundamental was wrong.

Answer 1

tl;dr Use Option to represent nullable function pointers and None for null.
The error message is confusing, first, because std::ptr::null_mut isn't a pointer; it's a generic function that returns a pointer, and you haven't called it. So Rust is seeing you pass a function that has the wrong signature and calling convention, and complaining about that.
But once you fix that, you'll get this error instead:
error[E0308]: mismatched types
--> src/lib.rs:29:16
|
29 | hello: std::ptr::null_mut(),
| ^^^^^^^^^^^^^^^^^^^^ expected fn pointer, found *-ptr
|
= note: expected type `extern "C" fn()`
found type `*mut _`
Function pointers and object pointers are not compatible (this is also the case in C), so you can't cast between them. null_mut returns an object pointer, so you need to find another way to create a null function pointer.
Function pointers (values of type fn(...) -> _) have another interesting property: unlike raw pointers (*const _ and *mut _), they can't be null. You don't need an unsafe block to call a function via pointer, and so creating a null function pointer is unsafe, like creating a null reference.
How do you make something nullable? Wrap it in Option:
#[repr(C)]
pub struct api {
// ...
load: Option<extern fn ()>,
hello: Option<extern fn ()>, // assuming hello can also be null
}
And populate it with Some(function) or None:
let api = Box::new (api {
// ...
load: Some(hello_load),
hello: None,
});
It's not usually a good idea to use enums, including Option, in a repr(C) struct, because C doesn't have an enum equivalent and so you don't know what you're going to get on the other side. But in the case of Option<T> where T is something non-nullable, None is represented by the null value, so it should be okay.
The use of Option to represent a nullable function pointer for FFI is documented in the Unsafe Code Guidelines:
null values are not supported by the Rust function pointer types -- just like references, the expectation is that you use Option to create nullable pointers. Option<fn(Args...) -> Ret> will have the exact same ABI as fn(Args...) -> Ret, but additionally allows null pointer values.