Golang, CGO, double pointers and macos Foundation framework - c

First, I have some basic C experience, but not too fluent with CGO and macos programming
I write a function, that removes a file in a system Trash. I have a working code, that I try to improve
The code first:
package trash
/*
#cgo CFLAGS: -x objective-c
#cgo LDFLAGS: -framework Foundation
#import "trash.h"
*/
import "C"
import (
"errors"
"unsafe"
)
// TrashItem puts items into system trash
func TrashItem(filePath string) error {
cString := C.CString(filePath)
defer C.free(unsafe.Pointer(cString))
var cErr = new(C.Error)
res := C.TrashItem(cString, cErr)
if res == 0 {
// convert cErr to go error
return errors.New("...")
}
return nil
}
#import <Foundation/Foundation.h>
typedef struct _Error {
long C;
const char *E;
} Error;
const BOOL TrashItem(const char *filePath, Error *error);
(I googled a lot to write this code, not sure, how good it is)
#import "trash.h"
const BOOL TrashItem(const char *filePath, Error *error) {
NSFileManager *manager = [NSFileManager defaultManager];
NSURL *fileURL = [[NSURL alloc] initFileURLWithPath:#(filePath)];
NSError *e;
BOOL success = [manager trashItemAtURL:fileURL resultingItemURL:nil error:&e];
if (!success) {
error->C = e.code;
error->E = [e.localizedDescription UTF8String];
}
return success;
}
First issue:
I don't wont to always initialize var cErr = new(C.Error). I think I could make trash function accept Error **error, but I struggle to pass double pointer from go to C (I have go pointer have go pointer error)
Second issue:
I would like to avoid custom Error struct and pass NSError * and access fields from go side.
I had Incompatible pointer types passing 'struct NSError *' to parameter of type 'NSError *, but https://stackoverflow.com/a/60715306/889530 solved it for me
But I think I still need NSError **, so I can read the data back, and again I struggle with void and double pointers here.
Can somebody explain me how to do that properly

Related

Swift: convert already allocated char array to char pointer

I am attempting to interface an existing C library to Swift 5.0.1 code on iOS. The C Header files have the following definitions:
char hostname[SFL_MAX_HOSTNAME_CHARS+1];
char os_release[SFL_MAX_OSRELEASE_CHARS+1];
int readHidCounters(HSP *sp, SFLHost_hid_counters *hid, char *hbuf, int hbufLen, char *rbuf, int rbufLen);
typedef struct _HSP {
[Many other elements omitted for brevity]
char hostname[SFL_MAX_HOSTNAME_CHARS+1];
char os_release[SFL_MAX_OSRELEASE_CHARS+1];
} HSP;
readHidCounters has an implementation (edited for brevity) like this:
int readHidCounters(HSP *sp, SFLHost_hid_counters *hid, char *hbuf, int hbufLen, char *rbuf, int rbufLen) {
int gotData = NO;
size_t len = hbufLen;
if(sysctlbyname("kern.hostname", hbuf, &len, NULL, 0) != 0) {
myLog(LOG_ERR, "sysctl(<kern.hostname>) failed : %s", strerror(errno));
}
else {
gotData = YES;
hid->hostname.str = hbuf;
hid->hostname.len = strlen(hbuf);
}
// UUID
memcpy(hid->uuid, sp->uuid, 16);
[...]
}
I have created an HSP struct and attempting to call readHidCounters in Swift like this
var sp = HSP()
[...]
readHidCounters(&sp,
&hidElem.counterBlock.host_hid,
&sp.hostname, // This is the error line
SFL_MAX_HOSTNAME_CHARS,
&sp.os_release,
SFL_MAX_OSRELEASE_CHARS)
I am attempting to pass in &sp.hostname results in the compiler error Cannot convert value of type '(Int8, Int8, Int8, [...], Int8)' to expected argument type 'Int8'. The problem is that hostname is a tuple of Int8's and I can't seem to get it properly converted to char *. I have tried various incarnations of UnsafeMutablePointer, withUnsafeMutablePointer and can't see how to get hostname properly recognized. Any suggestions greatly appreciated!
[Solved]
MartinR pretty much nailed it with his suggestion, but it did have a compiler error: Overlapping accesses to 'sp.hostname', but modification requires exclusive access; consider copying to a local variable. Updated code that compiles is
var myHostName = sp.hostname
var myOsRelease = sp.os_release
let _ = withUnsafeMutablePointer(to: &myHostName) {
$0.withMemoryRebound(to: Int8.self, capacity: MemoryLayout.size(ofValue: sp.hostname)) {
hostNamePtr in
withUnsafeMutablePointer(to: &myOsRelease) {
$0.withMemoryRebound(to: Int8.self, capacity: MemoryLayout.size(ofValue: sp.os_release)) {
osReleasePtr in
readHidCounters(&sp,
&hidElem.counterBlock.host_hid,
hostNamePtr, SFL_MAX_HOSTNAME_CHARS,
osReleasePtr, SFL_MAX_OSRELEASE_CHARS)
}
}
}
}
The “problem” is that C arrays are imported to Swift as tuples, and there is no simple method to treat a tuple as a Swift array, or get a pointer to the element storage (because a tuple can be inhomogeneous).
Similarly as in Converting a C char array to a String one can use the fact that Swift preserves the memory layout of structures imported from C, and with some pointer juggling and rebinding you'll get
let result = withUnsafeMutablePointer(to: &sp.hostname) {
$0.withMemoryRebound(to: Int8.self, capacity: MemoryLayout.size(ofValue: sp.hostname)) {
hostNamePtr in
withUnsafeMutablePointer(to: &sp.os_release) {
$0.withMemoryRebound(to: Int8.self, capacity: MemoryLayout.size(ofValue: sp.os_release)) {
osReleasePtr in
readHidCounters(&sp,
&hidElem.counterBlock.host_hid,
hostNamePtr, SFL_MAX_HOSTNAME_CHARS,
osReleasePtr, SFL_MAX_OSRELEASE_CHARS)
}
}
}
}
Another “trick” is to define C helper functions which return the array addresses as a pointers, and use the
SE-0044 Import as member
feature to make these helper functions available to Swift as computed properties. In the bridging header file you have to add
__attribute__((swift_name("getter:HSP.hostNamePtr(self:)")))
static inline char * _Nonnull hostNamePtr(HSP * _Nonnull hsp)
{
return hsp->hostname;
}
__attribute__((swift_name("getter:HSP.osReleasePtr(self:)")))
static inline char * _Nonnull osReleasePtr(HSP * _Nonnull hsp)
{
return hsp->os_release;
}
and then you can use these easily from Swift:
var sp = HSP()
let result = readHidCounters(&sp,
&hidElem.counterBlock.host_hid,
sp.hostNamePtr, SFL_MAX_HOSTNAME_CHARS,
sp.osReleasePtr, SFL_MAX_OSRELEASE_CHARS)
Try removing the & from the call, sp.hostname is already a pointer because it is an array, if it works you will have to remove the & from &sp.os_release too

Calling a C function with a double pointer output parameter using CGo

I am trying to figure out th right way to call this function:
size_t
fz_buffer_storage(fz_context *ctx, fz_buffer *buf, unsigned char **datap)
{
if (datap)
*datap = (buf ? buf->data : NULL);
return (buf ? buf->len : 0);
}
using CGo to get the underlying string and its length as a byte array in Go.
Is this the right way to do it?
var bufferContents *C.uchar
length := C.fz_buffer_storage(ctx, buf, &bufferContents)
bytes := C.GoBytes(unsafe.Pointer(bufferContents), C.int(length))
Since the C code overwrites *datap, I am not sure if the garbage collector will still do the right thing.
I saw an answer here suggesting something along the lines of
var tempUcharPtr *C.uchar
bufferContents := C.malloc(C.size_t(unsafe.Sizeof(tempUcharPtr)))
defer C.free(bufferContents)
length := C.fz_buffer_storage(ctx, buf, (**C.uchar)(bufferContents))
bytes := C.GoBytes(unsafe.Pointer(*(**C.uchar)(bufferContents)), C.int(length))
which also seems to work, but it's much more convoluted and I'm wondering if it's better / safer than the previous version.
Apparently, the first version is fine. Quoting the docs:
Go code may pass a Go pointer to C provided the Go memory to which it points does not contain any Go pointers.
From what I understand, since var bufferContents *C.uchar will be initialised to nil, it does not count as a "Go pointer" for the above rule. The following simplified code examples confirm this:
package main
// void F(char **p) {}
import "C"
func main() {
var p *C.char = new(C.char)
C.F(&p)
}
will trigger "panic: runtime error: cgo argument has Go pointer to Go pointer"
package main
// void F(char **p) {}
import "C"
func main() {
var p *C.char
C.F(&p)
}
works just fine, even when setting GODEBUG=cgocheck=2.
Thanks to the folks on the #cgo channel on the Gophers Slack community for helping me understand this!

Extension modules: marshalling void * to bytearray (and/or vice versa)

Doing some experiments using python (means python3) for preparing data (also as sending them to wire - SPI) shows it is slow (having limited system). So I was thinking of creation of extension module written in C to defer the critical stuff to. I would like either:
the python script would have access to memory block created by the malloc() in the extension module hopefully transparently convertible to bytearray
the extension module would get the pointer to bytearray object created in python hopefully transparently convertible to void *
The goal is to have zero copy also as zero conversion memory block accessible by both python (as bytearray) and extension module (as void *).
Is there any method, how to achieve this?
OK, it seems to be simpler than expected ;-)
bytearray provides direct support for accessing underlying memory block, which is exactly what is needed
there is a format specifier for extracting bytearray object from function call argument list
C extension module [test.c]:
#include <Python.h>
#include <stdint.h>
/* Forward prototype declaration */
static PyObject *transform(PyObject *self, PyObject *args);
/* Methods exported by this extension module */
static PyMethodDef test_methods[] =
{
{"transform", transform, METH_VARARGS, "testing buffer transformation"},
{NULL, NULL, 0, NULL}
};
/* Extension module definition */
static struct PyModuleDef test_module =
{
PyModuleDef_HEAD_INIT,
"BytearrayTest",
NULL,
-1,
test_methods,
};
/*
* The test function
*/
static PyObject *transform(PyObject *self, PyObject *args)
{
PyByteArrayObject *byte_array;
uint8_t *buff;
int buff_len = 0;
int i;
/* Get the bytearray object */
if (!PyArg_ParseTuple(args, "Y", &byte_array))
return NULL;
buff = (uint8_t *)(byte_array->ob_bytes); /* data */
buff_len = byte_array->ob_alloc; /* length */
/* Perform desired transformation */
for (i = 0; i < buff_len; ++i)
buff[i] += 65;
/* Return void */
Py_INCREF(Py_None);
return Py_None;
}
/* Mandatory extension module init function */
PyMODINIT_FUNC PyInit_BytearrayTest(void)
{
return PyModule_Create(&test_module);
}
C extension module build/deployment script [setup.py]:
#!/usr/bin/python3
from distutils.core import setup, Extension
module = Extension('BytearrayTest', sources = ['test.c'])
setup (name = 'BytearrayTest',
version = '1.0',
description = 'This is a bytearray test package',
ext_modules = [module])
Build/install the extension module:
# ./setup.py build
# ./setup.py install
Test it:
>>> import BytearrayTest
>>> a = bytearray(16); a
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
>>> BytearrayTest.transform(a); a
bytearray(b'AAAAAAAAAAAAAAAA')
>>>

Accessing a git_odb_writepack field in libgit2 gives error "dereferencing pointer to incomplete type"

I'm using libgit2 and I want to write a pack file to an odb created with git_repository_odb. So I call git_odb_write_pack and initialize a *git_odb_writepack. Then when I attempt to access a field of the writepack struct, I get a compiler error "dereferencing pointer to incomplete type". Here's the code:
#include <stdio.h>
#include <git2.h>
void check_error(int code, char *action) {
if (code) {
printf("Error %d, %s\n", code, action);
exit(1);
}
}
static int my_git_transfer_progress_callback(const git_transfer_progress *stats, void *payload) {
printf("Got transfer callback\n");
return 0;
}
int main(int argc, char **argv) {
int error;
const char *repo_path = "/path/to/repo";
git_repository *repo = NULL;
error = git_repository_open(&repo, repo_path);
check_error(error, "opening repo");
git_odb *odb = NULL;
error = git_repository_odb(&odb, repo);
check_error(error, "initializing odb");
git_odb_writepack *writepack = NULL;
char *payload = "here's my payload";
error = git_odb_write_pack(&writepack, odb, my_git_transfer_progress_callback, payload);
check_error(error, "opening pack writing stream");
printf("Address: %u\n", writepack->backend); // <-- Line generating the error.
return 0;
}
Then I compile and get the error:
$ gcc -lgit2 writepack_error.c && LD_LIBRARY_PATH=/usr/local/lib ./a.out
writepack_error.c: In function 'main':
writepack_error.c:33: error: dereferencing pointer to incomplete type
I'm using libgit2 version 0.21.0. I'm new to C and libgit2 so I may be doing something silly. My understanding is this "dereferencing" error means I failed to define or include a struct or typedef. However I thought libgit2 only requires one include, #include <git2.h>.
Normal usage is covered by git2.h. Some functionality is kept under the sys/ directory to indicate that it's considered more advanced usage.
This in particular looks like it might be a bug since git2.h does not include git2/odb_backend.h. For now you can simply include it manually.

Any good idioms for error handling in straight C programs?

Getting back in to some C work.
Many of my functions look like this:
int err = do_something(arg1, arg2, arg3, &result);
With the intent the result gets populated by the function, and the return value is the status of the call.
The darkside is you get something naive like this:
int err = func1(...);
if (!err) {
err = func2(...);
if (!err) {
err = func3(...);
}
}
return err;
I could macro it I suppose:
#define ERR(x) if (!err) { err = (x) }
int err = 0;
ERR(func1(...));
ERR(func2(...));
ERR(func3(...));
return err;
But that only works if I'm chaining function calls, vs doing other work.
Obviously Java, C#, C++ have exceptions that work very well for these kinds of things.
I'm just curious what other folks do and how other folks do error handling in their C programs nowadays.
If you have resources that need to be released at the end, then sometimes the old trusty goto can be handy!
int
major_func(size_t len)
{
int err;
char *buf;
buf = malloc(len);
if (err = minor_func1(buf))
goto major_func_end;
if (err = minor_func2(buf))
goto major_func_end;
if (err = minor_func3(buf))
goto major_func_end;
major_func_end:
free(buf);
return err;
}
Two typical patterns:
int major_func()
{
int err = 0;
if (err = minor_func1()) return err;
if (err = minor_func2()) return err;
if (err = minor_func3()) return err;
return 0;
}
int other_idea()
{
int err = minor_func1();
if (!err)
err = minor_func2();
if (!err)
err = minor_func3();
return err;
}
void main_func()
{
int err = major_func();
if (err)
{
show_err();
return;
}
happy_happy_joy_joy();
err = other_idea();
if (err)
{
show_err();
return;
}
happy_happy_joy_joy();
}
What are you doing in the else statements? If nothing, try this:
int err = func1(...);
if (err) {
return err;
}
err = func2(...);
if (err) {
return err;
}
err = func3(...);
return err;
This way you're short-circuiting the entire function, not even bothering with the following function calls.
EDIT
Going back and reading again, I realize that it doesn't matter what you do in your else statements. That sort of code can easily go immediately after the if blocks.
If error codes are boolean, then try the simpler code below:
return func1() && func2() && func3()
One approach which has been taken by OpenGL is to not return errors from functions at all but rather present an error state which can be examined after the function call. One nice thing about this approach is that when you have a function which you actually want to return something other than an error code, you can handle errors in the same way. Another thing which is nice about this is that if a user wants to call a number of functions and only succeed if all of them were successful, you can check for errors after the x amount of calls.
/* call a number of functions which may error.. */
glMatrixMode(GL_MODELVIEW);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnable(GL_TEXTURE_2D);
/* ...check for errors */
if ((error = glGetError()) != GL_NO_ERROR) {
if (error == GL_INVALID_VALUE)
printf("error: invalid value creating view");
else if (error == GL_INVALID_OPERATION)
printf("error: invalid operation creating view");
else if (error == GL_OUT_OF_MEMORY)
printf("error: out of memory creating view");
}
Others have suggested good ideas. Here're the idioms I've seen
int err;
...
err = foo(...);
if (err)
return err;
...
You could macro this out to something like
#define dERR int err=0
#define CALL err =
#define CHECK do { if (err) return err } while(0)
...
void my_func(void) {
dERR;
...
CALL foo(...);
CHECK;
or, if you're feeling really motivated, fiddle with CALL and CHECK so they can be used like
CALL foo(...) CHECK;
or
CALL( foo(...) );
--
Often, functions which need to do cleanup on exit (e.g. free memory) are written like this:
int do_something_complicated(...) {
...
err = first_thing();
if (err)
goto err_out;
buffer = malloc(...);
if (buffer == NULL)
goto err_out
err = another_complicated(...);
if (err)
goto err_out_free;
...
err_out_free:
free(buffer);
err_out:
return err; /* err might be zero */
}
You could use that pattern, or try to simplify it with macros.
--
Finally, if you're feeling /really/ motivated, you can use setjmp/longjmp.
int main(int argc, char *argv[]) {
jmp_buf on_error;
int err;
if (err = setjmp(on_error)) {
/* error occurred, error code in err */
return 1;
} else {
actual_code(..., on_error);
return 0;
}
}
void actual_code(..., jmp_buf on_error) {
...
if (err)
longjmp(on_error, err);
}
Essentially, a declaration of a new jmp_buf and a setjmp function as setting up a try block. The case where setjmp returns non-zero is your catch, and calling longjmp is your throw. I wrote this with passing the jmp_buf around in case you want nested handlers (e.g. if you need to free stuff before signaling an error); if you don't need that, feel free to declare err and the jmp_buf as globals.
Alternately, you could use macros to simply the argument passing around. I'd suggest the way Perl's implementation does it:
#define pERR jmp_buf _err_handler
#define aERR _err_handler
#define HANDLE_ERRORS do { jmp_buf _err_handler; int err = setjmp(_err_handler);
#define END_HANDLE while(0)
#define TRY if (! err)
#define CATCH else
#define THROW(e) longjmp(_err_handler, e)
void always_fails(pERR, int other_arg) {
THROW(42);
}
void does_some_stuff(pERR) {
normal_call(aERR);
HANDLE_ERRORS
TRY {
always_fails(aERR, 23);
} CATCH {
/* err is 42 */
}
END_HANDLE;
}
int main(int argc, char *argv[]) {
HANDLE_ERRORS
TRY {
does_some_stuff(aERR);
return 0;
} CATCH {
return err;
}
DONE_ERRORS;
}
--
Phew. I'm done. (Crazy examples untested. Some details might be off.)
And now for something completely different...
Another approach is to use a struct to contain your error information, e.g:
struct ErrorInfo
{
int errorCode;
char *errorMessage;
#if DEBUG
char *functionName;
int lineNumber;
#endif
}
The best way to use this is to return your method's results as the return code (e.g. "FALSE for failed", or "a file pointer or NULL if it fails", or "size of the buffer or 0 if it fails", etc) and pass in an ErrorInfo as a parameter that the called function will fill in if something fails.
This gives rich error reporting: if the method fails, you can fill in more than a simple error code (e.g. error message, code line and file of the failure, or whatever). The nice thing about it being a struct is that if you think of something, anything, useful later, you can just add it - for example, in my struct above I've allowed for a debug build to include the location of the error (file/line), but you could add a dump of the whole call stack in there at any time without having to change any of the client code.
You can use a global function to fill in an ErrorInfo so the error return can be managed cleanly, and you can update the struct to provide more info easily:
if (error)
{
Error(pErrorInfo, 123, "It failed");
return(FALSE);
}
...and you can have variants of this function that return FALSE, 0, or NULL, to allow most error returns to be phrased as a single line:
if (error)
return(ErrorNull(pErrorInfo, 123, "It failed"));
This gives you a lot of the advantages of an Exception class in other languages (although the caller still needs to handle the errors - callers have to check for error codes and may have to return early, but they can do nothing or next-to-nothing and allow the error to propagate back up a chain of calling methods until one of them wishes to handle it, much like an exception.
In addition, you can go further, to create a chain of error reports (like "InnerException"s):
struct ErrorInfo
{
int errorCode;
char *errorMessage;
...
ErrorInfo *pInnerError; // Pointer to previous error that may have led to this one
}
Then, if you "catch" an error from a function that you call, you can create a new, higher-level error description, and return a chain of these errors. e.g. "Mouse speed will revert to the default value" (because) "Preference block 'MousePrefs' could not be located" (because) "XML reader failed" (because) "File not found".
i.e.
FILE *OpenFile(char *filename, ErrorInfo *pErrorInfo)
{
FILE *fp = fopen(filename, "rb");
if (fp == NULL)
return(ChainedErrorNull(pErrorInfo, "Couldn't open file"));
return(fp);
}
XmlElement *ReadPreferenceXml(ErrorInfo *pErrorInfo)
{
if (OpenFile("prefs.xml", pErrorInfo) == NULL)
return(ChainedErrorNull(pErrorInfo, "Couldn't read pref"));
...
}
char *ReadPreference(char *prefName, ErrorInfo *pErrorInfo)
{
XmlElement *pXml = ReadPreferenceXml(pErrorInfo);
if (pXml == NULL)
return(ChainedErrorNull(pErrorInfo, "Couldn't read pref"));
...
}
You should check out what DirectX has done with the HRESULT - it's basically this. There's a reason that the exception came into being. Alternatively, if you run on Win32, they have SEH which runs in C programs.
You can get really silly and do continuations:
void step_1(int a, int b, int c, void (*step_2)(int), void (*err)(void *) ) {
if (!c) {
err("c was 0");
} else {
int r = a + b/c;
step_2(r);
}
}
This probably isn't actually what you want to do, but it is how many functional programming languages are used, and even more often how they model their code for optimization.
Something I've recently seen is this idom:
int err;
do
{
err = func1 (...);
if (!err) break;
err = func2 (...);
if (!err) break;
err = func3 (...);
if (!err) break;
/* add more calls here */
} while (0);
if (err)
{
/* handle the error here */
return E_ERROR; /* or something else */
}
else
{
return E_SUCCESS;
}
Pro arguments:
It avoids the goto (abuses the while(0) / break combination for that). Why would you want to do this? It keeps the cyclomatic complexity down and will still pass most static code analyzer checks (MISRA anyone?). For projects that get tested against cyclomatic complexity this is a god sent because it keeps all the initialization stuff together.
Contra arguments:
The meaning of the do/while loop construct is not obvious because a loop-construct is used as a cheap goto replacement, and this can only be seen at the loop tail. I'm sure for the first time this construct will cause lots of "WTF"-moments.
At least a comment is necessary to explain why the code is written the way it is required.
Here's a quite informative article & test file by IBM Unix article series:
Errors: errno in UNIX programs
Working with the standard error mechanism
https://www.ibm.com/developerworks/aix/library/au-errnovariable/
Another good example of how to implement exit codes is the source code of curl (man 1 curl).
Provided you are working with a specific context, I think the following pattern is very nice. The basic idea is that operations on an error-set state are no-ops, so error checking can be postponed to when it is convenient!
A concrete example: A deserialization context. Decoding of any element can fail, but the function may continue without error checking because all the decode_* functions are no-ops when the serialization record is in an error state. It's a matter of convenience or opportunity or optimization to insert decode_has_error. In the example below, there is no error check, the caller will take care of that.
void list_decode(struct serialization_record *rec,
struct list *list,
void *(*child_decode)(struct serialization_record *)) {
uint32_t length;
decode_begin(rec, TAG);
decode_uint32(rec, &length);
for (uint32_t i = 0; i < length; i++) {
list_append(list, child_decode(rec));
}
decode_end(rec, TAG);
}

Resources