Shared pointers and queues in FreeRTOS - shared-ptr

A C++ wapper around a FreeRTOS queue can be simplified into something like this:
template<typename T>
class Queue<T>
{
public:
bool push(const T& item)
{
return xQueueSendToBack(handle, &item, 0) == pdTRUE;
}
bool pop(T& target)
{
return xQueueReceive(handle, &target, 0) == pdTRUE;
}
private:
QueueHandle_t handle;
}
The documentation of xQueueSendToBack states:
The item is queued by copy, not by reference.
Unfortunately, it is literally by copy, because it all ends in a memcpy, which makes sense since it is a C API. While this works well for plain old data, more complex items such as the following event message give serious problems.
class ConnectionStatusEvent
{
public:
ConnectionStatusEvent() = default;
ConnectionStatusEvent(std::shared_ptr<ISocket> sock)
: sock(sock)
{
}
const std::shared_ptr<ISocket>& get_socket() const
{
return sock;
}
private:
const std::shared_ptr<ISocket> sock;
bool connected;
};
The problem is obviously the std::shared_ptr which doesn't work at all with a memcpy since its copy constructor/assignment operator isn't called when copied onto the queue, resulting in premature deletion of the held object when the event message, and thus the shared_ptr, goes out of scope.
I could solve this by using dynamically allocated T-instances and change the queues to only contain pointers to the instance, but I'd rather not do that since this shall run on an embedded system and I very much want to keep the memory static at run-time.
My current plan is to change the queue to contain pointers to a locally held memory area in the wrapper class in which I can implement full C++ object-copy, but as I'd also need to protect that memory area against multiple thread access, it essentially defeats the already thread-safe implementation of the FreeRTOS queues (which surely are more efficient than any implementation I can write myself) I might as well skip them entirely.
Finally, the question:
Before I implement my own queue, are there any tricks I can use to make the FreeRTOS queues function with C++ object instances, in particular std::shared_ptr?

The issue is what happens to the original once you put the pointer into the queue.
Copying seems trivial but not optimal.
To get around this issue i use a mailbox instead of a queue:
T* data = (T*) osMailAlloc(m_mail, osWaitForever);
...
osMailPut (m_mail, data);
Where you allocate the pointer explicitly to begin with. And just add the pointer to the mailbox.
And to retrieve:
osEvent ev = osMailGet(m_mail, osWaitForever);
...
osStatus freeStatus = osMailFree(m_mail, p);
All can be neatly warpend into c++ template methods.

Related

How do you avoid using global variables in inherently stateful programs?

I am currently writing a small game in C and feel like I can't get away from global variables.
For example I am storing the player position as a global variable because it's needed in other files. I have set myself some rules to keep the code clean.
Only use a global variable in the file it's defined in, if possible
Never directly change the value of a global from another file (reading from another file using extern is okay)
So for example graphics settings would be stored as file scope variables in graphics.c. If code in other files wants to change the graphics settings they would have to do so through a function in graphics.c like graphics_setFOV(float fov).
Do you think those rules are sufficient for avoiding global variable hell in the long term?
How bad are file scope variables?
Is it okay to read variables from other files using extern?
Typically, this kind of problem is handled by passing around a shared context:
graphics_api.h
#ifndef GRAPHICS_API
#define GRAPHICS_API
typedef void *HANDLE;
HANDLE init_graphics(void);
void destroy_graphics(HANDLE handle);
void use_graphics(HANDLE handle);
#endif
graphics.c
#include <stdio.h>
#include <stdlib.h>
#include "graphics_api.h"
typedef struct {
int width;
int height;
} CONTEXT;
HANDLE init_graphics(void) {
CONTEXT *result = malloc(sizeof(CONTEXT));
if (result) {
result->width = 640;
result->height = 480;
}
return (HANDLE) result;
}
void destroy_graphics(HANDLE handle) {
CONTEXT *context = (CONTEXT *) handle;
if (context) {
free(context);
}
}
void use_graphics(HANDLE handle) {
CONTEXT *context = (CONTEXT *) handle;
if (context) {
printf("width = %5d\n", context->width);
printf("height = %5d\n", context->height);
}
}
main.c
#include <stdio.h>
#include "graphics_api.h"
int main(void) {
HANDLE handle = init_graphics();
if (handle) {
use_graphics(handle);
destroy_graphics(handle);
}
return 0;
}
Output
width = 640
height = 480
Hiding the details of the context by using a void pointer prevents the user from changing the data contained within the memory to which it points.
How do you avoid using global variables in inherently stateful programs?
By passing arguments...
// state.h
/// state object:
struct state {
int some_value;
};
/// Initializes state
/// #return zero on success
int state_init(struct state *s);
/// Destroys state
/// #return zero on success
int state_fini(struct state *s);
/// Does some operation with state
/// #return zero on success
int state_set_value(struct state *s, int new_value);
/// Retrieves some operation from state
/// #return zero on success
int state_get_value(struct state *s, int *value);
// state.c
#include "state.h"
int state_init(struct state *s) {
s->some_value = -1;
return 0;
}
int state_fini(struct state *s) {
// add free() etc. if needed here
// call fini of other objects here
return 0;
}
int state_set_value(struct state *s, int value) {
if (value < 0) {
return -1; // ERROR - invalid argument
// you may return EINVAL here
}
s->some_value = value;
return 0; // success
}
int state_get_value(struct state *s, int *value) {
if (s->some_value < 0) { // value not set yet
return -1;
}
*value = s->some_value;
return 0;
}
// main.c
#include "state.h"
#include <stdlib.h>
#include <stdio.h>
int main() {
struct state state; // local variable
int err = state_init(&state);
if (err) abort();
int value;
err = state_get_value(&state, &value);
if (err != 0) {
printf("Getting value errored: %d\n", err);
}
err = state_set_value(&state, 50);
if (err) abort();
err = state_get_value(&state, &value);
if (err) abort();
printf("Current value is: %d\n", value);
err = state_fini(&state);
if (err) abort();
}
The only single case where global variables (preferably only a single pointer to some stack variable anyway) have to be used are signal handlers. The standard way would be to only increment a single global variable of type sig_atomic_t inside a signal handler and do nothing else - then execute all signal handling related logic from the normal flow in the rest of the code by checking the value of that variable. (On POSIX system) all other asynchronous communication from the kernel, like timer_create, that take sigevent structure, they can pass arguments to notified function by using members in union sigval.
Do you think those rules are sufficient for avoiding global variable hell in the long term?
Subjectively: no. I believe that a potentially uneducated programmer has too much freedom in creating global variables given the first rule. In complex programs I would use a hard rule: Do not use global variables. If finally after researching all other ways and all other possibilities have been exhausted and you have to use a global variables, make sure global variables leave the smallest possible memory footprint.
In simple short programs I wouldn't care much.
How bad are file scope variables?
This is opinion based - there are good cases where projects use many global variables. I believe that topic is exhausted in are global variables bad and numerous other internet resources.
Is it okay to read variables from other files using extern?
Yes, it's ok.
There are no "hard rules" and each project has it's own rules. I also recommend to read c2 wiki global variables are bad.
The first thing you have to ask yourself is: Just why did the programming world come to loath global variables? Obviously, as you noted, the way to model a global state is essentially a global (set of) variable(s). So what's the problem with that?
The Problem
All parts of the program have access to that state. The whole program becomes tightly coupled. Global variables violate the prime directive in programming, divide and conquer. Once all functions operate on the same data you can as well do away with the functions: They are no longer logical separations of concern but degrade to a notational convenience to avoid large files.
Write access is worse than read access: You'll have a hard time finding out just why on earth the state is unexpected at a certain point; the change can have happened anywhere. It is tempting to take shortcuts: "Ah, we can make the state change right here instead of passing a computation result back up three layers to the caller; that makes the code much smaller."
Even read access can be used to cheat and e.g. change behavior of some deep-down code depending on some global information: "Ah, we can skip rendering, there is no display yet!" A decision which should not be made in the rendering code but at top level. What if top level renders to a file!?
This creates both a debugging and a development/maintenance nightmare. If every piece of the code potentially relies on the presence and semantics of certain variables — and can change them! — it becomes exponentially harder to debug or change the program. The code agglomerating around the global data is like a cast, or perhaps a Boa Constrictor, which starts to immobilize and strangle your program.
Such programming can be avoided with (self-)discipline, but imagine a large project with many teams! It's much better to "physically" prevent access. Not coincidentally all programming languages after C, even if they are otherwise fundamentally different, come with improved modularization.
So what can we do?
The solution is indeed to pass parameters to functions, as KamilCuk said; but each function should only get the information they legitimately need. Of course it is best if the access is read-only and the result is a return value: Pure functions cannot change state at all and thus perfectly separate concerns.
But simply passing a pointer to the global state around does not cut the mustard: That's only a thinly veiled global variable.
Instead, the state should be separated into sub-states. Only top-level functions (which typically do not do much themselves but mostly delegate) have access to the overall state and hand sub-states to the functions they call. Third-tier functions get sub-sub states, etc. The corresponding implementation in C is a nested struct; pointers to the members — const whenever possible — are passed to functions which therefore cannot see, let alone alter, the rest of the global state. Separation of concerns is thus guaranteed.

why handle to an object frequently appears as pointer-to-pointer

What is the intention to set handle to an object as pointer-to pointer but not pointer? Like following code:
FT_Library library;
FT_Error error = FT_Init_FreeType( &library );
where
typedef struct FT_LibraryRec_ *FT_Library
so &library is a FT_LIBraryRec_ handle of type FT_LIBraryRec_**
It's a way to emulate pass by reference in C, which otherwise only have pass by value.
The 'C' library function FT_Init_FreeType has two outputs, the error code and/or the library handle (which is a pointer).
In C++ we'd more naturally either:
return an object which encapsulated the success or failure of the call and the library handle, or
return one output - the library handle, and throw an exception on failure.
C APIs are generally not implemented this way.
It is not unusual for a C Library function to return a success code, and to be passed the addresses of in/out variables to be conditionally mutated, as per the case above.
The approach hides implementation. It speeds up compilation of your code. It allows to upgrade data structures used by the library without breaking existing code that uses them. Finally, it makes sure the address of that object never changes, and that you don’t copy these objects.
Here’s how the version with a single pointer might be implemented:
struct FT_Struct
{
// Some fields/properties go here, e.g.
int field1;
char* field2;
}
FT_Error Init( FT_Struct* p )
{
p->field1 = 11;
p->field2 = malloc( 100 );
if( nullptr == p->field2 )
return E_OUTOFMEMORY;
return S_OK;
}
Or C++ equivalent, without any pointers:
class FT_Struct
{
int field1;
std::vector<char> field2;
public:
FT_Struct() :
field1( 11 )
{
field2.resize( 100 );
}
};
As a user of the library, you have to include struct/class FT_Struct definition. Libraries can be very complex so this will slow down compilation of your code.
If the library is dynamic i.e. *.dll on windows, *.so on linux or *.dylib on osx, you upgrade the library and if the new version changes memory layout of the struct/class, old applications will crash.
Because of the way C++ works, objects are passed by value, i.e. you normally expect them to be movable and copiable, which is not necessarily what library author wants to support.
Now consider the following function instead:
FT_Error Init( FT_Struct** pp )
{
try
{
*pp = new FT_Struct();
return S_OK;
}
catch( std::exception& ex )
{
return E_FAIL;
}
}
As a user of the library, you no longer need to know what’s inside FT_Struct or even what size it is. You don’t need to #include the implementation details, i.e. compilation will be faster.
This plays nicely with dynamic libraries, library author can change memory layout however they please, as long as the C API is stable, old apps will continue to work.
The API guarantees you won’t copy or move the values, you can’t copy structures of unknown lengths.

Linux Kernel - How to match a jprobe to kretprobe?

I am writing a kernel module to monitor a few syscalls wanting to return the function arguments to user-land (via netlink socket) if the call was successful.
jprobe.kp.symbol_name = "rename";
jprobe.entry = rename_handler;
kretprobe.kp.symbol_name = "rename";
kretprobe.handler = rename_ret_handler;
static rename_obj_t _g_cur_rename = NULL;
static void _rename_handler(const char *oldpath, const char *newpath)
{
_g_cur_rename = create_rename(oldpath, newpath);
jprobe_return();
}
static void _rename_ret_handler(struct kretprobe_instance *ri, struct pt_regs *regs)
{
/* Send only if successful */
if (regs_return_value(regs) == 0) {
add_send_queue(_g_cur_rename);
}
return 0;
}
I worry that another rename syscall may preempt[1] the current one after the jprobe and I will send incorrect return codes and arguments.
jprobe: rename(a, b)
jprobe rename(c, d)
kretprobe
kretprobe
Edit: This article[2] states that interrupts are disabled during a kprobe handler. But does that mean that interrupts are disable throughout the whole chain (jprobe -> kprobe -> kretprobe) or just for that single kprobe?
https://unix.stackexchange.com/questions/186355/few-questions-about-system-calls-and-kernel-modules-kernel-services-in-parallel
https://lwn.net/Articles/132196/
Interrupts are disabled for each jprobe call: not for the entire sequence.
How many calls are you expecting in the time it will take the application to process them? There are different approaches depending on how fast you expect the calls to come in. The simplest method, if you are only expecting maybe a few hundred calls before you can process them and you will dedicate the static memory to the purpose, is to implement a static array of rename_obj_t objects in memory and then use atomic_add from the kernel asm includes to point to the next entry (mod the size of your array).
This way you are returning a unique static reference each time, so long as the counter doesn't wrap around before you process the returned values. atomic_add is guaranteed to have the correct memory barriers in place so you don't have to worry about things like cache coherency.

transfer std::shared_ptr via mailbox

We have a Real Time Operating System which offers Inter-Task-Communication by so called Mailboxes.
A Mailbox is described by a Handle of type RTKMailbox.
The API looks like:
int RTKPut(RTKMailbox h, const void* data);
int RTKGet(RTKMailbox h, void* data);
The size of data is known by the Mailbox. Data transfer could be thought as doing a memcpy from sender to receiver.
Imagine I have a Producer-Task and a Consumer-Task; is it a good idea to send a shared_ptr by that system?
Since the Mailbox does not know a shared_ptr my idea is to wrap the shared_ptr in a transport structure.
The code could look like:
class MyData {
//...
};
struct TransportWrapper {
void BeforePut();
void AfterGet();
std::shared_ptr<MyData> Data;
TransportWrapper() {}
TransportWrapper(std::shared_ptr<MyData>& _data) : Data(_data)
{}
};
void Send(RTKMailbox mbHandle, std::shared_ptr<MyData>& data)
{
TransportWrapper wrap(data);
wrap.BeforePut();
RTKPut(mbHandle, &wrap);
}
std::shared_ptr<MyData> Receive(RTKMailbox mbHandle)
{
TransportWrapper wrap;
RTKGet(mbHandle, &wrap);
wrap.AfterGet();
return wrap.Data;
}
What do I have to do in BeforePut to prevent the shared_ptr to be deleted if the Lifetime of the wrapper ends?
What do I have to do in AfterGet to restore the shared_ptr to the state it had before Put?
Regards Andreas
Your example code won't work, you can't just memcpy a shared_ptr because all that does is copy the pointers it contains, it doesn't make a new copy of the shared_ptr and increase the reference count. You cannot use memcpy with objects that have non-trivial constructors or destructors.
Assuming the sender and receiver share an address space (because otherwise this is pretty much impossible to do via your mailbox API, you need shared memory), you need to increase the shared_ptr's reference count on the sender side, to ensure that the sender doesn't drop its last reference to the owned object and delete it before the receiver has received it. Then the receiver has to decrease the reference count, so they need to coordinate.
If delivery to a mailbox is asynchronous (i.e. the sender does not block until delivery is complete and the receiver has received the data) you can't do that with local variables in the Send function, because those variables will go out of scope as soon as the RTKPut call returns, which will decrease the reference count (and maybe destroy the data) before the receiver has got it.
The simplest way to solve that is to create a new shared_ptr on the heap and transfer its address.
void Send(RTKMailbox mbHandle, const std::shared_ptr<MyData>& data)
{
std::shared_ptr<MyData>* p = new std::shared_ptr<MyData>(data);
if (RTKPut(mbHandle, &p) != success)
{
delete p;
// deal with it
}
}
std::shared_ptr<MyData> Receive(RTKMailbox mbHandle)
{
std::shared_ptr<MyData>* p = nullptr;
if (RTKGet(mbHandle, &p) == success)
{
auto sp = *p;
delete p;
return sp;
}
// else deal with it
}
This assumes that if RTKPut returns successfully then delivery will not fail, otherwise you leak the shared_ptr created on the heap, and will never delete the object it owns.

How to create a Singleton in C?

What's the best way to create a singleton in C? A concurrent solution would be nice.
I am aware that C isn't the first language you would use for a singleton.
First, C is not suitable for OO programming. You'd be fighting all the way if you do. Secondly, singletons are just static variables with some encapsulation. So you can use a static global variable. However, global variables typically have far too many ills associated with them. You could otherwise use a function local static variable, like this:
int *SingletonInt() {
static int instance = 42;
return &instance;
}
or a smarter macro:
#define SINGLETON(t, inst, init) t* Singleton_##t() { \
static t inst = init; \
return &inst; \
}
#include <stdio.h>
/* actual definition */
SINGLETON(float, finst, 4.2);
int main() {
printf("%f\n", *(Singleton_float()));
return 0;
}
And finally, remember, that singletons are mostly abused. It is difficult to get them right, especially under multi-threaded environments...
You don't need to. C already has global variables, so you don't need a work-around to simulate them.
It's the same as the C++ version pretty much. Just have a function that returns an instance pointer. It can be a static variable inside the function. Wrap the function body with a critical section or pthread mutex, depending on platform.
#include <stdlib.h>
struct A
{
int a;
int b;
};
struct A* getObject()
{
static struct A *instance = NULL;
// do lock here
if(instance == NULL)
{
instance = malloc(sizeof(*instance));
instance->a = 1;
instance->b = 2;
}
// do unlock
return instance;
};
Note that you'd need a function to free up the singleton too. Especially if it grabs any system resources that aren't automatically released on process exit.
EDIT: My answer presumes the singleton you are creating is somewhat complex and has a multi-step creation process. If it's just static data, go with a global like others have suggested.
A singleton in C will be very weird . . . I've never seen an example of "object oriented C" that looked particularly elegant. If possible, consider using C++. C++ allows you to pick and choose which features you want to use, and many people just use it as a "better C".
Below is a pretty typical pattern for lock-free one-time initialization. The InterlockCompareExchangePtr atomically swaps in the new value if the previous is null. This protects if multiple threads try to create the singleton at the same time, only one will win. The others will delete their newly created object.
MyObj* g_singleton; // MyObj is some struct.
MyObj* GetMyObj()
{
MyObj* singleton;
if (g_singleton == NULL)
{
singleton = CreateNewObj();
// Only swap if the existing value is null. If not on Windows,
// use whatever compare and swap your platform provides.
if (InterlockCompareExchangePtr(&g_singleton, singleton, NULL) != NULL)
{
DeleteObj(singleton);
}
}
return g_singleton;
}
DoSomethingWithSingleton(GetMyObj());
Here's another perspective: every file in a C program is effectively a singleton class that is auto instantiated at runtime and cannot be subclassed.
Global static variables are your private class members.
Global non static are public (just declare them using extern in some header file).
Static functions are private methods
Non-static functions are the public ones.
Give everything a proper prefix and now you can use my_singleton_method() in lieu of my_singleton.method().
If your singleton is complex you can write a generate_singleton() method to initialize it before use, but then you need to make sure all the other public methods check if it was called and error out if not.
I think this solution might be the simplest and best for most use cases...
In this example, I am creating a single instance global dispatch queue, which you'd definitely do, say, if you were tracking dispatch source events from multiple objects; in that case, every object listening to the queue for events could be notified when a new task is added to the queue. Once the global queue is set (via queue_ref()), it can be referenced with the queue variable in any file in which the header file is included (examples are provided below).
In one of my implementations, I called queue_ref() in AppDelegate.m (main.c would work, too). That way, queue will be initialized before any other calling object attempts to access it. In the remaining objects, I simply called queue. Returning a value from a variable is much faster than calling a function, and then checking the value of the variable before returning it.
In GlobalQueue.h:
#ifndef GlobalQueue_h
#define GlobalQueue_h
#include <stdio.h>
#include <dispatch/dispatch.h>
extern dispatch_queue_t queue;
extern dispatch_queue_t queue_ref(void);
#endif /* GlobalQueue_h */
In GlobalQueue.c:
#include "GlobalQueue.h"
dispatch_queue_t queue;
dispatch_queue_t queue_ref(void) {
if (!queue) {
queue = dispatch_queue_create_with_target("GlobalDispatchQueue", DISPATCH_QUEUE_SERIAL, dispatch_get_main_queue());
}
return queue;
}
To use:
#include "GlobalQueue.h" in any Objective-C or C implementation source file.
Call queue_ref() to use the dispatch queue. Once queue_ref() has been called, the queue can be used via the queue variable in all source files
Examples:
Calling queue_ref():
dispatch_queue_t serial_queue_with_queue_target = dispatch_queue_create_with_target("serial_queue_with_queue_target", DISPATCH_QUEUE_SERIAL, **queue_ref()**);
Calling queue:
dispatch_queue_t serial_queue_with_queue_target = dispatch_queue_create_with_target("serial_queue_with_queue_target", DISPATCH_QUEUE_SERIAL, **queue**));]
Just do
void * getSingleTon() {
static Class object = (Class *)malloc( sizeof( Class ) );
return &object;
}
which works in a concurrent environment too.

Resources