I'm dynamically repeatedly creating a char[] of fixed length 32 and pass it to function that stores the pointer in a struct and passes it later on to a callback function.
Unforntunately I don't know the correct way to allocate the memory for this string data.
This is the function, where the nonceHex is created in a loop
static int polling_thread(struct pt *pt, uint16_t ticks)
{
static uint16_t timer;
timer += ticks;
PT_BEGIN(pt);
while (1)
{
uint8_t nonce[NONCE_LEN] = {};
char nonceHex[2 * NONCE_LEN] = "";
RNG(nonce, NONCE_LEN);
hex_encode(nonce, sizeof(nonce), nonceHex);
...
struct mg_connection *c = mg_connect_http(mgos_get_mgr(), http_cb, nonceHex, url, (const char *)sEtagHeader, NULL);
...
timer = 0;
PT_WAIT_UNTIL(pt, timer >= 500);
}
PT_END(pt);
}
And this is the callback function where the char[] is passed by the framework (mongoose os) as cb_arg
static void http_cb(struct mg_connection *nc, int ev, void *evd, void *cb_arg)
{
...
case MG_EV_HTTP_REPLY:
{
const char *nonceHex = (const char *)cb_arg;
LOG(LL_DEBUG, ("http_cb nonce: %s", nonceHex));
LOG(LL_DEBUG, ("http_cb nc->user_data: %s", (const char *)nc->user_data));
}
...
}
Unfortunately when I run this code I don't received the nonceHex but something that looks like unitialized char[]
http_cb nonce: Pa�?`B�?�a�?���?�c�?
http_cb nc->user_data: Pa�?`B�?�a�?���?�c�?
Do I need to dynamically malloc the char[]? If I use a static char[] it works, but it's not correct as nonceHex is overwritten by subsequent loop cycles and so cb_arg in the callback function contains invalid data.
Can anyone tell me how this would be programmed correctly?
#######################################################
Changed my function to
static int polling_thread(struct pt *pt, uint16_t ticks)
{
static uint16_t timer;
timer += ticks;
PT_BEGIN(pt);
while (1)
{
...
uint8_t nonce[NONCE_LEN] = {};
char *nonceHex = malloc(2 * NONCE_LEN + 1);
RNG(nonce, NONCE_LEN);
hex_encode(nonce, sizeof(nonce), nonceHex);
*(nonceHex+2 * NONCE_LEN) = '\0';
struct mg_connection *c = mg_connect_http(mgos_get_mgr(), http_cb, nonceHex, url, (const char *)sEtagHeader, NULL);
...
timer = 0;
PT_WAIT_UNTIL(pt, timer >= 500);
}
PT_END(pt);
}
Added call to free in callback function
static void http_cb(struct mg_connection *nc, int ev, void *evd, void *cb_arg)
{
...
case MG_EV_HTTP_REPLY:
{
char *nonceHex = (char *)cb_arg;
LOG(LL_DEBUG, ("http_cb nonce: %s", nonceHex));
LOG(LL_DEBUG, ("http_cb nc->user_data: %s", (const char *)nc->user_data));
free(nonceHex);
}
...
}
This results in a panic
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Can you see my fault?
I have a simple GTK-app. I created the design with glade (it is linked as RO-data into the binary) and load it with gtk_builder_new_from_string. In another c-file all the callback-functions are implemented.
Loading the glade xml-file:
builder = gtk_builder_new_from_string(GUI_CRAZYCART_GLADE_FILE, (gssize) GUI_CRAZYCART_GLADE_SIZE);
widgets->window = GTK_WINDOW(gtk_builder_get_object(builder, "CrazyCart"));
assert(widgets->window != NULL);
widgets->win_print = GTK_WINDOW(gtk_builder_get_object(builder, "win_print"));
assert(widgets->win_print != NULL);
... // more of the lines above, all the widgets I need
gtk_builder_connect_signals(builder, widgets);
g_object_unref(builder);
gtk_widget_show_all(GTK_WIDGET(widgets->window));
gtk_widget_hide(widgets->wdgt_email_pw);
gtk_window_maximize(widgets->window);
gtk_window_fullscreen(widgets->window);
// MAIN LOOP - APP WAITS HERE UNTIL EXIT
gtk_main();
// MAIN LOOP - APP WAITS HERE UNTIL EXIT
Similarly with the CSS-files, etc.
However, when I press just random buttons and watch the application with htop I can see the used memory (RES) rise up very quickly. The buttons essentially don't do anything.
I would like to run it on a raspberry pi3, but after some time executing it (it is controlling a racing game on karts) it crashes, because no memory is left.
I checked already with valgrind, but it outputs a logfile several MB large, it makes it very difficult to comb through it (using online available supp-files didn't help much) and I could only find "leaks" which are one time inits by external libraries, like gstreamer, libfontconfig, gtk, etc. Also, gcc's sanitizer outputs not much of significance (mostly libfontconfig because I am using the toy-functions).
It is approaching 1GB RES-memory after 30min of usage. It slowing down after that a little, however still rising. Also, repeatedly focusing and unfocusing the window rises the memory usage.
Sample-callback:
void VISIBILITY_DEFAULT on_kart_color_editing (
GtkTreeView *tree_view,
GtkTreePath *path,
GtkTreeViewColumn *column,
App_Widgets *widgets)
{
const char *title = gtk_tree_view_column_get_title(column);
if (hash(title) != hash(_("Colour"))) return;
GtkTreeIter iter;
GdkRGBA color;
guint cart_id;
guint id;
char *name;
gui_show_color_dialog(widgets->window, &color);
gtk_tree_model_get_iter(GTK_TREE_MODEL(widgets->tv_car_store), &iter, path);
gtk_list_store_set(widgets->tv_car_store, &iter, COL_CAR_COLOR, &color, -1);
gtk_tree_model_get(GTK_TREE_MODEL(widgets->tv_car_store), &iter, COL_CAR_DRIVER, &name, COL_KARTNUM, &cart_id, COL_CAR_ID, &id, -1);
gtk_tree_selection_unselect_all(gtk_tree_view_get_selection(tree_view));
game_change_player((uint8_t) id, (uint8_t) (cart_id - 1U), gdkrgba_to_argb(&color), name);
g_free(name);
}
I have also tried finding glib objects with reference-counter > 0 using this post however again most leaks were due to objects from external libs (and a huge file).
It seems as gstreamer/my sound playing code is one of the reasons for the increasing memory usage
sound.c, for gstreamer:
struct snd_data_t
{
GstElement *pipeline, *pitch;
GstElement *source;
guint bus_watch_id;
void (*cb_start)(void *);
void (*cb_end)(void *);
void *data_start;
void *data_end;
};
// return const, because user mustn't free or alter it!
const void *snd_play_mem(const char *buf, size_t len, void (*cb_start)(void *), void *data_start, void (*cb_end)(void *), void *data_end)
{
GstElement *source;
GstBuffer *gst_buffer;
char *buf_cloned;
struct snd_data_t *snd_data;
GstCaps *caps_ogg;
snd_data = emalloc(sizeof *snd_data);
source = gst_element_factory_make("appsrc", "mem-source");
caps_ogg = gst_caps_new_empty_simple("audio/ogg");
gst_app_src_set_caps((GstAppSrc*) source, caps_ogg);
gst_app_src_set_size((GstAppSrc*) source, (gint64) len);
gst_app_src_set_stream_type((GstAppSrc*) source, GST_APP_STREAM_TYPE_STREAM);
gst_app_src_set_max_bytes((GstAppSrc*) source, len);
gst_caps_unref(caps_ogg);
buf_cloned = g_memdup(buf, (guint) len);
gst_buffer = gst_buffer_new_wrapped(buf_cloned, len);
gst_app_src_push_buffer((GstAppSrc*) source, gst_buffer);
gst_app_src_end_of_stream((GstAppSrc*) source);
snd_data->source = source;
snd_data->cb_start = cb_start;
snd_data->cb_end = cb_end;
snd_data->data_start = data_start;
snd_data->data_end = data_end;
snd_thread_play(snd_data);
return snd_data;
}
// return const, because user mustn't free or alter it!
const void *snd_play_file(const char *path, void (*cb)(void *), void *data)
{
GstElement *source;
struct snd_data_t *snd_data;
snd_data = emalloc(sizeof *snd_data);
source = gst_element_factory_make("filesrc", "file-source");
g_object_set(G_OBJECT(source), "location", path, NULL);
snd_data->source = source;
snd_data->cb_end = cb;
snd_data->data_end = data;
snd_data->cb_start = NULL;
snd_data->data_start = NULL;
snd_thread_play(snd_data);
return snd_data;
}
void snd_stop(const void *data)
{
if (data == NULL) return;
// ugly hack, because const (user mustn't free or alter it)!
struct snd_data_t *snd_data = (struct snd_data_t *)(intptr_t) data;
gst_element_set_state(snd_data->pipeline, GST_STATE_NULL);
gst_object_unref(GST_OBJECT(snd_data->pipeline));
g_source_remove(snd_data->bus_watch_id);
free(snd_data);
}
void snd_set_speed(const void *data, double speed)
{
if (data == NULL) return;
// ugly hack, because const (user mustn't free or alter it)!
struct snd_data_t *snd_data = (struct snd_data_t *)(intptr_t) data;
g_object_set(snd_data->pitch, "pitch", speed, "tempo", speed, NULL);
}
static void *snd_thread_play(void *data)
{
GstElement *source, *decoder, *conv, *pitch, *sink;
GstBus *bus;
struct snd_data_t *snd_data;
snd_data = data;
snd_data->pipeline = gst_pipeline_new("CrazyCart");
source = snd_data->source;
decoder = gst_element_factory_make("decodebin", "decoder");
conv = gst_element_factory_make("audioconvert", "converter");
pitch = gst_element_factory_make("pitch", "pitcher");
sink = gst_element_factory_make("autoaudiosink", "audio-output");
if (!snd_data->pipeline || !source || !decoder || !conv || !sink || !pitch)
{
fprintf(stderr, "%s\n", _("'gst_element_factory_make' failed"));
exit(EXIT_FAILURE);
}
snd_data->pitch = pitch;
bus = gst_pipeline_get_bus(GST_PIPELINE(snd_data->pipeline));
snd_data->bus_watch_id = gst_bus_add_watch(bus, snd_bus_call, snd_data);
gst_object_unref(bus);
gst_bin_add_many(GST_BIN(snd_data->pipeline), source, decoder, conv, pitch, sink, NULL);
gst_element_link_many(source, decoder, NULL);
gst_element_link_many(conv, pitch, sink, NULL);
g_signal_connect(decoder, "pad-added", G_CALLBACK(snd_on_pad_added), conv);
gst_element_set_state(snd_data->pipeline, GST_STATE_PLAYING);
#ifdef DEBUG
GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(snd_data->pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "crazycart");
#endif
return NULL;
}
static void snd_on_pad_added(
__attribute__ ((unused)) GstElement *element,
GstPad *pad,
gpointer data
)
{
GstPad *sinkpad;
GstElement *conv = (GstElement *) data;
sinkpad = gst_element_get_static_pad(conv, "sink");
gst_pad_link(pad, sinkpad);
gst_object_unref(sinkpad);
}
static gboolean snd_bus_call(
UNUSED GstBus *bus,
GstMessage *msg,
gpointer data
)
{
struct snd_data_t *snd_data = data;
switch (GST_MESSAGE_TYPE(msg))
{
case GST_MESSAGE_EOS:
gst_element_set_state(snd_data->pipeline, GST_STATE_NULL);
gst_object_unref(GST_OBJECT(snd_data->pipeline));
g_source_remove(snd_data->bus_watch_id);
void (*cb_end)(void *) = snd_data->cb_end;
void *cb_data = snd_data->data_end;
free(snd_data);
if (cb_end != NULL)
cb_end(cb_data);
return false;
break;
case GST_MESSAGE_ERROR:
{
gchar *debug;
GError *error;
gst_message_parse_error(msg, &error, &debug);
g_free(debug);
g_printerr("Error: %s\n", error->message);
g_error_free(error);
exit(EXIT_FAILURE);
} break;
case GST_MESSAGE_STREAM_START:
if (snd_data->cb_start != NULL)
snd_data->cb_start(snd_data->data_start);
break;
default:
//DPRINT_VERB("GST_MESSAGE_TYPE %d\n", GST_MESSAGE_TYPE(msg));
break;
}
return true;
}
I have created a simple test app to test my gstreamer code, to be able to debug it seperately. Also, I followed this presentation, to get a few tipps how to find leaks. So this post doesn't get too long, I uploaded my log files to github.
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wcast-qual"
#pragma GCC diagnostic ignored "-Wcast-align"
#pragma GCC diagnostic ignored "-Wconversion"
#include <gst/gst.h>
#pragma GCC diagnostic pop
#include <glib.h>
#include "utils.h"
#include "sound.h"
#include "debug.h"
static char **argv_s;
void next_song(GMainLoop *loop)
{
snd_play_file(argv_s[2], (void (*)(void *)) &g_main_loop_quit, loop);
}
int main(int argc, char **argv)
{
argv_s = argv;
GMainLoop *loop = g_main_loop_new(NULL, FALSE);
gst_init(&argc, &argv);
char *mime;
get_mime_type(&mime, argv[1]);
printf("Mime [%s]: %s\n", argv[1], mime);
free(mime);
const void *p = snd_play_file(argv[1], (void (*)(void *)) &next_song, loop);
DPRINT_VERB("sleeping 10!\n");
sleep(10);
snd_set_speed(p, 1.5);
g_main_loop_run(loop);
DPRINT_VERB("deinit!\n");
gst_deinit();
g_main_loop_unref(loop);
return 0;
}
What am I missing here? Why is my (in my opinion) rather small and simple app using so much memory and how can I reduce it?
I am working on a custom I/O layer for libxml2 and have come across an oddity in the de-registration of Output callbacks. There appears to be no way to remove just one set of output callbacks. Am I missing something? I can provide a working example if the following pseudo-code does not clarify the question.
static int _in_matcher(const char* URI) {...}
static void* _in_opener(const char* URI) {...}
static int _in_reader(void *ctxt, char *in_buf, int buf_len) {...}
static int _in_closer(void *ctxt) {...}
int test_input_callbacks(const char *file_name)
{
int handler = xmlRegisterInputCallbacks(
_in_matcher,
_in_opener,
_in_reader,
_in_closer,
);
...
xmlDocPtr doc = xmlParseFile(file_name);
xmlFree(doc);
...
xmlPopInputCallbacks();
}
static int _out_matcher( const char *URI) {...}
static void* _out_opener( const char *URI) {...}
static int _out_writer( void *ctxt, const char *out_buf, int buf_len) {...}
static int _outcloser( void *ctxt) {...}
int test_output_callbacks(const char *file_name)
{
int handler = xmlRegisterOutputCallbacks(
_out_matcher,
_out_opener,
_out_writer,
_out_closer,
);
...
int result = xmlSaveFormatFileEnc(URI, doc, NULL, 0);
xmlFree(doc);
...
/* There does not appear to be a function that
* removes _only_ the last output callbacks
* registered!
*/
==> ?? xmlPopOutputCallbacks() ?? <==
}
As a work around, I added this function to my [local] version of libxml2 (2-2.9.10):
int
xmlPopOutputCallbacks(void)
{
if (!xmlOutputCallbackInitialized)
return(-1);
if (xmlOutputCallbackNr <= 0)
return(-1);
xmlOutputCallbackNr--;
xmlOutputCallbackTable[xmlOutputCallbackNr].matchcallback = NULL;
xmlOutputCallbackTable[xmlOutputCallbackNr].opencallback = NULL;
xmlOutputCallbackTable[xmlOutputCallbackNr].writecallback = NULL;
xmlOutputCallbackTable[xmlOutputCallbackNr].closecallback = NULL;
return(xmlOutputCallbackNr);
}
It works, but I get the nagging feeling there is either something I am missing; or there is a valid reason why libxml2 does not provide the ability to remove individual sets of output callbacks.
Any help/advice would be greatly appreciated!
Thanks,
Tom
First post, so will try and be brief until need to add more. Ported an app from macOS to NI LabWindows/CVI, in "C", then to port to GTK3 and trying to grasp the updating GUI from external thread concept. I've read the gnome documentation and searched on here, and everywhere, but not finding similar usage, or not grasping the updating the GUI from separate thread. I have experimented with g_idle_add() as follows.
int main(int argc, char* argv[])
{
gtk_init(&argc, &argv); // init Gtk
gtk_start_button = GTK_WIDGET(gtk_builder_get_object(builder,"start_button"));
gtk_main();
return EXIT_SUCCESS;
}
void start_button_clicked_cb(GtkWidget *widget, gpointer data)
{
printf("\nStart Button Pressed\n");
run_tests();
}
void run_tests( void )
{
GThread *start_testing_thread;
start_testing_thread = g_thread_new("", &start_testing, NULL);
}
void *start_testing (void *data)
{
pause(5);
}
void pause( double pause_time)
{
char string[33];
while( (double)pause_time > (double)0 )
{
sprintf( string, "Pausing %02.1f", pause_time );
printf(string);
//test_name( string );
g_idle_add(test_name_gui, string);
g_usleep(100000); // uSecs for 100 mSecs
pause_time -= 0.1;
}
}
void test_name_gui(gpointer user_data)
{
GtkTextBuffer* buffer = gtk_text_buffer_new(NULL);
char temp[99];
int error = -1;
buffer = gtk_text_view_get_buffer(GTK_TEXT_VIEW (gtktextview_test_name));
sprintf(temp,"\n%s gui\n",(char*)user_data);
printf(temp);
if(!g_utf8_validate(user_data,-1,NULL))
{
error = 3;
}
gtk_text_buffer_set_text (buffer, (char*)user_data, -1);
printf("\nTest Name gui\n");
g_object_unref(buffer);
return G_SOURCE_REMOVE;
}
I'm creating a simple thrift server/client program in C (g_lib).
This is how my thrift IDL file looks like:
namespace cpp tutorial
service Calculator {
void ping(),
binary getdata()
}
And the implementation for getdata on the thrift server looks like this:
static gboolean
tutorial_calculator_handler_getdata (CalculatorIf *iface,
GByteArray *_return,
GError **error)
{
THRIFT_UNUSED_VAR (iface);
THRIFT_UNUSED_VAR (error);
puts ("getdata()");
GByteArray *gbarray;
gint i;
gbarray = g_byte_array_new ();
for (i = 0; i < 100; i++)
g_byte_array_append (gbarray, (guint8*) &i, 1);
*_return = *gbarray;
return TRUE;
}
Now, on the client side, I'm calling the getdata as follows:
....
....
GByteArray *data;
....
....
if (!error && calculator_if_getdata (client, &data, &error)) {
puts ("getdata()");
}
Unfortunately, the client crashes with the following message in the calculator_if_getdata call:
*** Error in `./client': munmap_chunk(): invalid pointer: 0xb741742d ***
Aborted (core dumped)
Is this the correct way to send an array of integers from the server to client in thrift? What am i doing wrong here?
I see three issues with what you're doing here:
If a service method returns a complex type, the _return parameter passed to its handler function will point to a pre-allocated structure that should not be destroyed or re-created. Your code should be appending values to the GByteArray to which _return already points, not creating a GByteArray of its own.
Although you shouldn't be trying to modify the value of _return anyway, the way your code does this is incorrect and ends up clobbering the data structure to which _return points. This most likely explains the error message you're seeing.
Your code declares i a thirty-two bit integer but appends to the byte array only i's first byte. This will not have the effect you intend on every machine architecture.
I figured it out after spending some time on it, here's the working handler and the client side implementation code:
gboolean
tutorial_calculator_handler_getdata (CalculatorIf *iface,
GByteArray ** _return,
GError **error)
{
THRIFT_UNUSED_VAR (iface);
THRIFT_UNUSED_VAR (error);
GByteArray *thing = g_byte_array_new();
*_return = g_byte_array_new();
guint8 i;
for (i = 0; i < 10; i++){
g_byte_array_append (thing, (guint8*) &i, sizeof(guint8));
}
g_byte_array_append(*_return, (guint8*) thing->data, thing->len);
return TRUE;
}
And the client side:
GByteArray *data = g_byte_array_new();
if (!error && calculator_if_getdata (client, &data, &error)) {
puts ("getdata()");
printf ("Data : %d\n", data);
guint8 i;
guint8 size = sizeof(guint8);
for(i=0;i<10;i++)
printf ("Data : %d\n", data->data[size*i]);
}