In the example erlang port program
tuplep = erl_decode(buf);
fnp = erl_element(1, tuplep);
argp = erl_element(2, tuplep);
...
erl_free_compound(tuplep);
erl_free_term(fnp);
erl_free_term(argp);
Both erl_free_compound and erl_free_term are used for freeing term (and its sub-term) separately of the same ETERM*. From the documentation of erl_free_compund() it says
erl_free_compound() will recursively free all of the sub-terms associated with a given Erlang term
So, my question is, does erl_element() makes a copy of the element which, if not freed separately will leak memory or the above situation might lead to double free which is detected and handled by erl_free_term.
The erl_interface library indeed uses a sort of reference counting system to keep track
of allocated ETERM structs. So if you write:
ETERM *t_arr[2];
ETERM *t1;
t_arr[0] = erl_mk_atom("hello");
t_arr[1] = erl_mk_atom("world");
t1 = erl_mk_tuple(&t_arr[0],2);
you have created three (3) Erlang terms (ETERM). Now if you
call: erl_free_term(t1) you only free upp the tuple not the
other two ETERM's. To free all the allocated memory, you
would have to call:
erl_free_term(t_arr[0]);
erl_free_term(t_arr[1]);
erl_free_term(t1)
To avoid all these calls to erl_free_term() you can use:
erl_free_compund() instead. It does a "deep" free of
all ETERM's. So the above could be accomplished with:
erl_free_compund(t1)
Thus, this routine makes it possible for you to write
in a more compact way where you don't have to remember
the references to all sub-component ETERM's.
Example:
ETERM *list;
list = erl_cons(erl_mk_int(36),
erl_cons(erl_mk_atom("tobbe"),
erl_mk_empty_list()));
... /* do some work */
erl_free_compound(list);
Update: To check if you really free up all the create terms you can use this piece of code (original manual entry:
long allocated, freed;
erl_eterm_statistics(&allocated,&freed);
printf("currently allocated blocks: %ld\n",allocated);
printf("length of freelist: %ld\n",freed);
/* really free the freelist */
erl_eterm_release();
(answer adopted from here)
Related
The Objective-C runtime's hashtable2.mm file contains the following code:
static void bootstrap (void) {
free(malloc(8));
prototypes = ALLOCTABLE (DEFAULT_ZONE);
prototypes->prototype = &protoPrototype;
prototypes->count = 1;
prototypes->nbBuckets = 1; /* has to be 1 so that the right bucket is 0 */
prototypes->buckets = ALLOCBUCKETS(DEFAULT_ZONE, 1);
prototypes->info = NULL;
((HashBucket *) prototypes->buckets)[0].count = 1;
((HashBucket *) prototypes->buckets)[0].elements.one = &protoPrototype;
};
Why does it allocate and release the 8-bytes space immediately?
Another source of confusion is this method from objc-os.h:
static __inline void *malloc_zone_malloc(malloc_zone_t z, size_t size) { return malloc(size); }
While it uses only one parameter, does the signature ask for two?
For the first question I can only assume. My bet it was done to avoid/reduce memory churn, or segment the memory for some other reason. You can briefly find where it's discussed in the Changelog of bmalloc (which is not quite relevant, but i could not find a better reference):
2017-06-02 Geoffrey Garen <ggaren#apple.com>
...
Updated for new APIs. Note that we cache one free chunk per page
class. This avoids churn in the large allocator when you
free(malloc(X))
It's unclear however, if the memory churn is caused by this technique or it was supposed to address it.
For the second question, Objective-C runtime used to work with "zones" in order to destroy all allocated variables by just destroying the said zone, but it proved being error prone and later it was agreed to not use it anymore. The API, however still uses it for historical reasons (backward compatibility, i assume), but says that zones are ignored:
Zones are ignored on iOS and 64-bit runtime on OS X. You should not use zones in current development.
I called in Python one C function named decrypt, and decrypt returns one malloced memory buffer.
The code is as below:
sl = ctypes.cdll.LoadLibrary('some_crypto.so')
sl.decrypt.argtypes = [ctypes.c_char_p]
sl.decrypt.restype = ctypes.c_char_p
ret = sl.decrypt(ctypes.c_char_p(ciphertext))
At this stage, underlying C function decrypt malloced one buffer and returned to Python as ret.
Do I need to release the memory of ret variable to avoid memory leak ?
If yes, how can I release the underlying memory buffer?
More details added according to comments.
This case is different from How to free memory allocated by external C libraries interfacing with Cython module where the memory is ultimately returned to a Python process?, since no Cython is used here.
And as suggested here Memory leaks in Python when using an external C DLL, I use convert_and_free_char_p as the restype of C function decrypt, but still found memory leak through htop.
def convert_and_free_char_p(char_p):
if char_p == 0:
return None
try:
return ctypes.c_char_p(char_p).value
finally:
libc.free(char_p)
sl = ctypes.cdll.LoadLibrary('some_crypto.so')
sl.decrypt.argtypes = [ctypes.c_char_p]
sl.decrypt.restype = ctypes.convert_and_free_char_p
for i in range(0, 10000000000000):
ret = sl.decrypt(ctypes.c_char_p(ciphertext))
VIRT/RES column of this script in htop kept insreasing, indicating memory leak..
How comes the free not working properly?
I am working on a multithreaded client using C and the pthreads library, using a boss/worker arch design and am having issues understanding/debugging a stack-use-after-scope error that is causing my client to fail. (I am kinda new to C)
I have tried multiple things, including defining the variable globally, passing a double pointer reference, etc.
Boss logic within main:
for (i = 0; i < nrequests; i++)
{
struct Request_work_item *request_ctx = malloc(sizeof(*request_ctx));
request_ctx->server = server;
request_ctx->port = port;
request_ctx->nrequests = nrequests;
req_path = get_path(); //Gets a file path to work on
request_ctx->path = req_path;
steque_item work_item = &request_ctx; // steque_item is a void* so passing it a pointer to the Request_work_item
pthread_mutex_lock(&mutex);
while (steque_isempty(&work_queue) == 0) //Wait for the queue to be empty to add more work
{
pthread_cond_wait(&c_boss, &mutex);
}
steque_enqueue(&work_queue, work_item); //Queue the workItem in a workQueue (type steque_t, can hold any number of steque_items)
pthread_mutex_unlock(&mutex);
pthread_cond_signal(&c_worker);
}
Worker logic inside a defined function:
struct Request_work_item **wi;
while (1)
{
pthread_mutex_lock(&mutex);
while (steque_isempty(&work_queue) == 1) //Wait for work to be added to the queue
{
pthread_cond_wait(&c_worker, &mutex);
}
wi = steque_pop(&work_queue); //Pull the steque_item into a Request_work_item type
pthread_mutex_unlock(&mutex);
pthread_cond_signal(&c_boss);
char *path_to_file = (*wi)->path; //When executing, I get this error in this line: SUMMARY: AddressSanitizer: stack-use-after-scope
...
...
...
continues with additional worker logic
I expect the worker to pull the work_item from the queue, dereference the values and then perform some work. However, I keep getting AddressSanitizer: stack-use-after-scope, and the information for this error online is not very abundant so any pointers would be greatly appreciated.
The red flag here is that &request_ctx is the address of a local variable. It's not the pointer to the storage allocated with malloc, but the address of the variable which holds that storage. That variable is gone once this scope terminates, even though the malloc-ed block endures.
Maybe the fix is simply to delete the address-of & operator in this line?
steque_item work_item = &request_ctx; // steque_item is a void* so passing
// it a pointer to the Request_work_item
If we do that, then the comment actually tells the truth. Because otherwise we're making work_item a pointer to a pointer to the Request_work_item.
Since work_item has type void*, it compiles either way, unfortunately.
If the consumer of the item on the other end of the queue is extracting it as a Request_work_item *, then you not only have an access to an object that has gone out of scope, but also a type mismatch even if that object happens to still be in the producer's scope when the consumer uses it. The consumer ends up using a piece of the producer's stack as if it were a Request_work_item structure. Edit: I see that you are using a pointer-to-pointer when dequeuing the item and accessing it as (*wi)->path. Think about changing the design to avoid doing that. Or else, that wi pointer has to be dynamically allocated also, and freed. The producer has to do something like:
struct Request_work_item **p_request_ctx = malloc(sizeof *p_request_ctx);
struct Request_work_item *request_ctx = malloc(sizeof *request_ctx);
if (p_request_ctx && request_ctx) {
*p_request_ctx = request_ctx;
request_ctx->field = init_value;
// ... etc
// then p_request_ctx is enqueued.
The consumer then has to free the structure, and also free the pointer. That extra pointer just seems like pure overhead here; it doesn't provide any essential or useful level of indirection.
This seems like it should be simple but I wasn't able to find much related to it. I have structure which has different fields used to store data about the program operation. I want to log that data so that I can analyse it later. Attempting to continuously log data over the course of the programs operation eats up a lot of resources. Thus I would only like to call the logging function when the data has changed. I would love it if there was an efficient way to check whether the structure members have updated. Currently I am playing a shell game with 3 structures (old, current, and new) in order to detect when the data has changed. Thanks in advance.
You may track structures and its hashes in your log function.
Let you have a hash function:
int hash(void* ptr, size_t size);
Let you have a mapping from pointer to struct to struct's hash like:
/* Stores hash value for ptr*/
void ptr2hash_update_hash(void* ptr, int hash);
/* Remove ptr from mapping */
void ptr2hash_remove(void* ptr);
/* Returns 0 if ptr was not stored, or stored has otherwise*/
int ptr2hash_get_hash(void* ptr);
Then you may check if your object was changed between log calls like this:
int new_hash = hash(ptr, sizeof(TheStruct));
int old_hash = ptr2hash_get_hash(ptr);
if (old_hash == new_hash)
return;
ptr2hash_update_hash(ptr, new_hash);
/* Then do the logging */
Don't forget to remove ptr from mapping when you do free(ptr) :)
Here is simple hash table implementation, you will need it to implement ptr2hash mapping.
Simple hash functions are here.
If you're running on Linux (x86 or x86_64) then another possible approach is the following:
Install a segment descriptor for a non-writable segment in the local descriptor table using the modify_ldt system call. Place your data inside this segment (or install the segment such that your data structure is within it).
Upon write access, your process will receive a SIGSEGV (segmentation fault). Install a handler using sigaction to catch segmentation faults. Within that handler, first check that the fault occurred inside the previously set segment (si_addr member of the siginfo_t) and if so prepare to record a notification. Now, change the segment descriptor such that the segment becomes writable and return from the signal handler.
The write will now be performed, but you need a way to change the segment to be non-writable again and to actually check what was written and if your data actually changed.
A possible approach could be to send oneself (or a "delay" process and then back to the main process) another signal (SIGUSR1 for example), and doing the above in the handler for this signal.
Is this portable? No.
Is this relyable? No.
Is this easy to implement? No.
So if you can, and I really hope you do, use a interface like already suggested.
The easiest way what you can try is, You can just keep two structure pointers. Once you are receiving the new updated values that time you can just compare the new structure pointer with the old structure pointer, and if any difference is there you can detect it and then you can update to old structure pointer so that you can detect further changes in updated value in future.
typedef struct testStruct
{
int x;
float y;
}TESTSTRUCT;
TESTSTRUCT* getUpdatedValue()
{
TESTSTRUCT *ptr;
ptr->x = 5;
ptr->y = 6;
//You can put your code to update the value.
return ptr;
}
void updateTheChange(TESTSTRUCT* oldObj,TESTSTRUCT* newObj)
{
cout << "Change Detected\n";
oldObj = newObj;
}
int main()
{
TESTSTRUCT *oldObj = NULL;
TESTSTRUCT *newObj = NULL;
newObj = getUpdatedValue();
//each time a value is updated compae with the old structure
if(newObj == oldObj)
{
cout << "Same" << endl;
}
else
{
updateTheChange(oldObj,newObj);
}
return 0;
}
I am not sure, it gives you your exact answer or not.
Hope this Helps.
I am working on a fairly simple application written in C with GTK+ that is leaking memory badly. It has a few basic functions on timers that check the clock and poll an external networked device, parsing the string returned. The application runs on a small touch panel, and through TOP I can watch the available memory be eaten up as it runs.
I'm pretty new to C, so not surprised that I'm doing something wrong, I just can't seem to figure out what. I've been trying to use Valgrind to narrow it down, but honestly the output is a little over my head (10k+ line log file generated from running the application less than a minute). But in digging through that log I did find some functions repeatedly showing up with permanently lost blocks, all using some similar structure.
Example 1:
This is a short function that gets called when an option is selected. The last line with the g_strdup_printf is the one called out by Valgrind. select_next_show and select_show_five_displayed are both global variables.
static void show_box_five_clicked ()
{
g_timer_start(lock_timer);
gtk_image_set_from_file (GTK_IMAGE(select_show_1_cb_image), "./images/checkbox_clear.png");
gtk_image_set_from_file (GTK_IMAGE(select_show_2_cb_image), "./images/checkbox_clear.png");
gtk_image_set_from_file (GTK_IMAGE(select_show_3_cb_image), "./images/checkbox_clear.png");
gtk_image_set_from_file (GTK_IMAGE(select_show_4_cb_image), "./images/checkbox_clear.png");
gtk_image_set_from_file (GTK_IMAGE(select_show_5_cb_image), "./images/checkbox_checked.png");
select_next_show = g_strdup_printf("%i",select_show_five_displayed);
}
Example 2:
This is another function that gets called often and came up a lot in the Valgrind log. It takes the incoming response from the networked device, parses it into two strings, then returns one.
static gchar* parse_incoming_value(gchar* incoming_message)
{
gchar *ret;
GString *incoming = g_string_new(incoming_message);
gchar **messagePieces = g_strsplit((char *)incoming->str, "=", 2);
ret = g_strdup(messagePieces[1]);
g_strfreev(messagePieces);
g_string_free(incoming, TRUE);
return ret;
}
In all the cases like these which are causing problems I'm freeing everything I can without causing segmentation faults, but I must be missing something else or doing something wrong.
UPDATE:
To answer questions in comments, here is an example (trimmed down) of how I'm using the parse function and where the return is freed:
static void load_schedule ()
{
...other code...
gchar *holder;
gchar *holder2;
holder = read_a_line(schedListenSocket);
holder2 = parse_incoming_value(holder);
schedule_info->regShowNumber = holder2;
holder = read_a_line(schedListenSocket);
holder2 = parse_incoming_value(holder);
schedule_info->holidayShowNumber = holder2;
...other code....
g_free(holder);
g_free(holder2);
}
Any help is greatly appreciated!!
It looks like you free 'ret' once when calling g_free(holder2), but you've done multiple allocations for that one free - you call parse_incoming_value multiple times, each time causing an allocation, but you only free once right at the end.
As you copy the holder2 pointer into schedule_info elements each time, they actually have the "leaked" memory at the end.
If you do not free holder2 anywhere, but just free all the elements in schedule_info at the end of the code. I presume that shows no leak?
e.g.
holder2 = <result of dynamic alloc>;
schedule_info->a = holder2;
...
holder2 = <result of dynamic alloc>;
schedule_info->b = holder2;
...
// instead of g_free(holder2) at the end, do this...
g_free(schedule_info->a);
g_free(schedule_info->a);