Undefined behavior when deleting an element from dynamic array of structs - c

I have an n sized array of structs dynamically allocated, and each position of the array is an array too, with different sizes for each position (an array of arrays).
I created a function to delete a given array[index] but I'm facing some undefined behavior, for example:
If the array is of size 3, if I delete array[0],I can't access array[1]. This happens with other combinations of indexes too. The only way it works flawlessly is when I delete from end to start.
Here is the code I have:
Structures:
typedef struct point{
char id[5];
char type[5];
char color[10];
int x;
int y;
} Point;
typedef struct {
char lineID[5];
int nPoints;
Point *pt;
}railData;
typedef struct railway {
railData data;
}railway;
This is how the array was created:
headRail = (railway**)calloc(lineNum,sizeof(railway*));
And each Rail:
headRail[i] = (railway*)calloc(pointsNum,sizeof(railway));
These are the functions to delete a rail:
railway **delRail(railway **headRail, int j)
{
int nPts = 0;
if (!headRail)
{
puts(ERRORS[NULLPOINTER]);
return NULL;
}
// Number of rail points on jth rail
nPts = headRail[j]->data.nPoints;
// Free each rail point from jth rail
for (int i = 0; i < nPts; ++i)
{
free(headRail[j][i].data.pt);
}
// Free allocated memory for jth rail
free(headRail[j]);
return headRail;
}
And this is where I call the previous function:
railway **removeRail(railway **headRail)
{
char userID[20];
int index = 0;
// Quit if no rails
if (!headRail)
{
backToMenu("No rails available!");
return NULL;
}
// Get user input
getString("\nRail ID: ",userID,MINLEN,MAXLEN); // MINLEN = 2 MAXLEN = 4
// get index of the asked rail
getRailIndex(headRail,userID,&index);
if (index != NOTFOUND)
{
headRail = delRail(headRail, index);
// Update number of rails in the array (global var)
NUMOFRAILS--;
backToMenu("Rail deleted!\n");
}
else
backToMenu("Rail not found!");
return headRail;
}
So my question is how can I modify my code so that when position i is eliminated, all other indexes are shifted left and the last position, which would be empty, is discarded (something like realloc but for shrinking)
Is what I'm asking doable without changing the array's structure?

When removing element i, do memmove all the data from i+1 to i to the end of the array and then realloc with the size decremented by 1.
Note that arrays in C do not track their size in any way, so you need to pass the size by an external way.
Your data abstraction is strange. I would expect that headRail[j][0].data.nPoints is used to store the number of points inside the headRail[j][0].data structure, yet there you store the count of headRails in the j row headRail[j][<this count>]. I would advise to rewrite the abstraction, have one "object" for the railway and another for hadling two dimensional arrays of railways with dynamic sizes in all directions.
Like:
railway **delRail(railway **headRail, int j)
{
...
// this is strange, it's equal to
// nPts = headRail[j][0].data.nPoints;
// dunno if you mean that,
// or if [j][0].data.nPoints refers to the size of
// headRail[j][0].data.pt or to the size of the whole array
size_t nPts = headRail[j]->data.nPoints;
for (size_t i = 0; i < nPts; ++i) {
free(headRail[j][i].data.pt);
}
free(headRail[j]);
// note that arrays in C does not know how many elements are there in the array
// so you typically pass that along the arguments, like
// railway **delRail(railway **headRail, size_t railcount, int j);
size_t headRailCount = lineNum; // some external knowledge of the size
memmove(&headRail[j], &headRail[j + 1], (headRailCount - j - 1) * sizeof(*headRail));
void *pnt = realloc(headRail, (headRailCount - 1) * sizeof(*headRail));
if (pnt == NULL) return NULL; // that would be strange
headRail = pnt; // note that the previous headRail is no longer valid
--lineNum; // decrement that object where you store the size of the array
return headRail;
}
What about some encapsulation and more structs instead of 2d array? 2d arrays are really a bit of pain for C, what about:
typedef struct {
// stores a single row of rail datas
struct railData_row_s {
// stores a pointer to an array of rail datas
railData *data;
// stores the count of how many datas of rails are stored here
size_t datacnt;
// stores a pointer to an array of rows of rail datas
} *raildatas;
// stores the size of the pointer of rows of rail datas
size_t raildatascnt;
} railway;
The count of mallocs will stay the same, but thinking about data will get simpler. And each pointer that points to an array of data has it's own size tracking variable. An allocation might look like this:
railway *rail_new(size_t lineNum, size_t pointsNum) {
railway *r = calloc(1, sizeof(*r));
if (!r) { return NULL; }
// allocate the memory for rows of raildata
r->raildatascnt = lineNum;
r->raildatas = calloc(r->raildatascnt, sizeof(*r->raildatas));
if (!t->raildatas) { /* error hadnling */ free(r); abort(); }
// for each row of raildata
for (size_t i = 0; i < r->raildatascnt; ++i) {
struct railData_row_s * const row = &r->raildatas[i];
// allocate the memory for the column of raildata
// hah, looks similar to the above?
row->datacnt = pointsNum;
row->data = calloc(row->datacnt, sizeof(*row->data));
if (!row->data) { /* error ahdnling */ abort(); }
}
return r;
}

Related

How can I correctly allocate memory for this MergeSort implementation in C (with the DS I am using)?

My goal here is to perform MergeSort on a dynamic array-like data structure I called a dictionary used to store strings and their relative weights. Sorry if the implementation is dumb, I'm a student and still learning.
Anyway, based on the segfaults I'm getting, I'm incorrectly allocating memory for my structs of type item to be copied over into the temporary lists I'm making. Not sure how to fix this. Code for mergesort and data structure setup is below, any help is appreciated.
/////// DICTIONARY METHODS ////////
typedef struct {
char *item;
int weight;
} item;
typedef struct {
item **wordlist;
//track size of dictionary
int size;
} dict;
//dict constructor
dict* Dict(int count){
//allocate space for dictionary
dict* D = malloc(sizeof(dict));
//allocate space for words
D->wordlist = malloc(sizeof(item*) * count);
//initial size
D->size = 0;
return D;
}
//word constructor
item* Item(char str[]){
//allocate memory for struct
item* W = malloc(sizeof(item));
//allocate memory for string
W->item = malloc(sizeof(char) * strlen(str));
W->weight = 0;
return W;
}
void merge(dict* D, int start, int middle, int stop){
//create ints to track lengths of left and right of array
int leftlen = middle - start + 1;
int rightlen = stop - middle;
//create new temporary dicts to store the two sides of the array
dict* L = Dict(leftlen);
dict* R = Dict(rightlen);
int i, j, k;
//copy elements start through middle into left dict- this gives a segfault
for (int i = 0; i < leftlen; i++){
L->wordlist[i] = malloc(sizeof(item*));
L->wordlist[i] = D->wordlist[start + i];
}
//copy elements middle through end into right dict- this gives a segfault
for (int j = 0; j < rightlen; j++){
R->wordlist[j] = malloc(sizeof(item*));
R->wordlist[j]= D->wordlist[middle + 1 + k];
}
i = 0;
j = 0;
k = leftlen;
while ((i < leftlen) && (j < rightlen)){
if (strcmp(L->wordlist[i]->item, R->wordlist[j]->item) <= 0) {
D->wordlist[k] = L->wordlist[i];
i++;
k++;
}
else{
D->wordlist[k] = R->wordlist[j];
j++;
k++;
}
}
while (i < leftlen){
D->wordlist[k] = L->wordlist[i];
i++;
k++;
}
while (j < rightlen){
D->wordlist[k] = L->wordlist[j];
j++;
k++;
}
}
void mergeSort(dict* D, int start, int stop){
if (start < stop) {
int middle = start + (stop - start) / 2;
mergeSort(D, start, middle);
mergeSort(D, middle + 1, stop);
merge(D, start, middle, stop);
}
I put print statements everywhere and narrowed it down to the mallocs in the section where I copy the dictionary to be sorted into 2 separate dictionaries. Also tried writing that malloc as malloc(sizeof(D->wordlist[start + i])). Is there something else I need to do to be able to copy the item struct into the wordlist of the new struct?
Again, I'm new to this, so cut me some slack :)
There are numerous errors in the code:
In merge() when copying elements to the R list, the wrong (and uninitialized) index variable k is being used instead of j. R->wordlist[j]= D->wordlist[middle + 1 + k]; should be R->wordlist[j]= D->wordlist[middle + 1 + j];.
In merge() before merging the L and R lists back to D, the index variable k for the D list is being initialized to the wrong value. k = leftLen; should be k = start;.
In merge() in the loop that should copy the remaining elements of the "right" list to D, the elements are being copied from the "left" list instead of the "right" list. D->wordlist[k] = L->wordlist[j]; should be D->wordlist[k] = R->wordlist[j];.
In Item(), the malloc() call is not reserving space for the null terminator at the end of the string. W->item = malloc(sizeof(char) * strlen(str)); should be W->item = malloc(sizeof(char) * (strlen(str) + 1)); (and since sizeof(char) is 1 by definition it can be simplified to W->item = malloc(strlen(str) + 1);).
Item() is not copying the string to the allocated memory. Add strcpy(W->item, str);.
There are memory leaks in merge():
L->wordlist[i] = malloc(sizeof(item*)); is not required and can be removed since L->wordlist[i] is changed on the very next line: L->wordlist[i] = D->wordlist[start + i];.
Similarly, R->wordlist[j] = malloc(sizeof(item*)); is not required and can be removed since R->wordlist[j] is changed on the very next line.
L and R memory is created but never destroyed. Add these lines to the end of merge() to free them:
free(L->wordlist);
free(L);
free(R->wordlist);
free(R);
None of the malloc() calls are checked for success.
Allocate it all at once, before the merge sort even starts.
#include <stdlib.h>
#include <string.h>
// Weighted Word --------------------------------------------------------------
//
typedef struct {
char *word;
int weight;
} weighted_word;
// Create a weighted word
//
weighted_word* CreateWeightedWord(const char *str, int weight){
weighted_word* W = malloc(sizeof(weighted_word));
if (W){
W->word = malloc(strlen(str) + 1); // string length + nul terminator
if (W->word)
strcpy( W->word, str);
W->weight = weight;
}
return W;
}
// Free a weighted word
//
weighted_word *FreeWeightedWord(weighted_word *W){
if (W){
if (W->word)
free(W->word);
free(W);
}
return NULL;
}
// Dictionary (of Weighted Words) ---------------------------------------------
//
typedef struct {
weighted_word **wordlist; // this is a pointer to an array of (weighted_word *)s
int size; // current number of elements in use
int capacity; // maximum number of elements available to use
} dict;
// Create a dictionary with a fixed capacity
//
dict* CreateDict(int capacity){
dict* D = malloc(sizeof(dict));
if (D){
D->wordlist = malloc(sizeof(weighted_word*) * capacity);
D->size = 0;
D->capacity = capacity;
}
return D;
}
// Free a dictionary (and all weighted words)
//
dict *FreeDict(dict *D){
if (D){
for (int n = 0; n < D->size; n++)
FreeWeightedWord(D->wordlist[n]);
free(D->wordlist);
free(D);
}
return NULL;
}
// Add a new weighted word to the end of our dictionary
//
void DictAddWord(dict *D, const char *str, int weight){
if (!D) return;
if (D->size == D->capacity) return;
D->wordlist[D->size] = CreateWeightedWord(str, weight);
if (D->wordlist[D->size])
D->size += 1;
}
// Merge Sort the Dictionary --------------------------------------------------
// Merge two partitions of sorted words
// words • the partitioned weighted word list
// start • beginning of left partition
// middle • end of left partition, beginning of right partition
// stop • end of right partition
// buffer • temporary work buffer, at least as big as (middle-start)
//
void MergeWeightedWords(weighted_word **words, int start, int middle, int stop, weighted_word **buffer){
int Lstart = start; int Rstart = middle; // Left partition
int Lstop = middle; int Rstop = stop; // Right partition
int Bindex = 0; // temporary work buffer output index
// while (left partition has elements) AND (right partition has elements)
while ((Lstart < Lstop) && (Rstart < Rstop)){
if (strcmp( words[Rstart]->word, words[Lstart]->word ) < 0)
buffer[Bindex++] = words[Rstart++];
else
buffer[Bindex++] = words[Lstart++];
}
// if (left partition has any remaining elements)
while (Lstart < Lstop)
buffer[Bindex++] = words[Lstart++];
// We don't actually need this. Think about it. Why not?
// // if (right partition has any remaining elements)
// while (Rstart < Rstop)
// buffer[Bindex++] = words[Rstart++];
// Copy merged data from temporary buffer back into source word list
for (int n = 0; n < Bindex; n++)
words[start++] = buffer[n];
}
// Merge Sort an array of weighted words
// words • the array of (weighted_word*)s to sort
// start • index of first element to sort
// stop • index ONE PAST the last element to sort
// buffer • the temporary merge buffer, at least as big as (stop-start+1)/2
//
void MergeSortWeightedWords(weighted_word **words, int start, int stop, weighted_word **buffer){
if (start < stop-1){ // -1 because a singleton array is by definition sorted
int middle = start + (stop - start) / 2;
MergeSortWeightedWords(words, start, middle, buffer);
MergeSortWeightedWords(words, middle, stop, buffer);
MergeWeightedWords(words, start, middle, stop, buffer);
}
}
// Merge Sort a Dictionary
//
void MergeSortDict(dict *D){
if (D){
// We only need to allocate a single temporary work buffer, just once, right here.
dict * Temp = CreateDict(D->size);
if (Temp){
MergeSortWeightedWords(D->wordlist, 0, D->size, Temp->wordlist);
}
FreeDict(Temp);
}
}
// Main program ---------------------------------------------------------------
#include <stdio.h>
int main(int argc, char **argv){
// Command-line arguments --> dictionary
dict *a_dict = CreateDict(argc-1);
for (int n = 1; n < argc; n++)
DictAddWord(a_dict, argv[n], 0);
// Sort the dictionary
MergeSortDict(a_dict);
// Print the weighted words
for (int n = 0; n < a_dict->size; n++)
printf( "%d %s\n", a_dict->wordlist[n]->weight, a_dict->wordlist[n]->word );
// Clean up
FreeDict(a_dict);
}
Notes for you:
Be consistent. You were inconsistent with capitalization and * placement and, oddly, vertical spacing. (You are waaay better than most beginners, though.) I personally hate the Egyptian brace style, but to each his own.
I personally think there are far too many levels of malloc()s in this code too, but I will leave it at this one comment. It works as is.
Strings must be nul-terminated — that is, each string takes strlen() characters plus one for a '\0' character. There is a convenient library function that can copy a string for you too, called strdup(), which AFAIK exists on every system.
Always check that malloc() and friends succeed.
Don’t forget to free everything you allocate. Functions help.
“Item” was a terribly non-descript name, and it overlapped with the meaning of two different things in your code. I renamed them to separate things.
Your dictionary object should be expected to keep track of how many elements it can support. The above code simply refuses to add words after the capacity is filled, but you could easily make it realloc() a larger capacity if the need arises. The point is to prevent invalid array accesses by adding too many elements to a fixed-size array.
Printing the array could probably go in a function.
Notice how I set start as inclusive and stop as exclusive. This is a very C (and C++) way of looking at things, and it is a good one. It will help you with all kinds of algorithms.
Notice also how I split the Merge Sort up into two functions: one that takes a dictionary as argument, and a lower-level one that takes an array of the weighted words as argument that does all the work.
The higher-level merge sort a dictionary allocates all the temporary buffer the merge algorithm needs, just once.
The lower-level merge sort an array of (weighted_word*)s expects that temporary buffer to exist and doesn’t care (or know anything) about the dictionary object.
The merge algorithm likewise doesn't know much. It is simply given all the information it needs.
Right now the merge condition simply compares the weighted-word’s string value. But it doesn’t have to be so simple. For example, you could sort equal elements by weight. Create a function:
int CompareWeightedWords(const weighted_word *a, const weighted_word *b){
int rel = strcmp( a->word, b->word );
if (rel < 0) return -1;
if (rel > 0) return 1;
return a->weight < b->weight ? -1 : a->weight > b->weight;
}
And put it to use in the merge function:
if (CompareWeightedWords( words[Rstart], words[Lstart] ) < 0)
buffer[Bindex++] = words[Rstart++];
else
buffer[Bindex++] = words[Lstart++];
I don’t think I forgot anything.

Recovering elements of large array with multiple index ranges

This is a tricky problem that I have been thinking about for a long time and have yet to see a satisfactory answer anywhere. Lets say I have a large int array of size 10000. I can simply declare it in the following manner:
int main()
{
int foo[10000];
int i;
int n;
n = sizeof(foo) / sizeof(int);
for (i = 0; i < n; i++)
{
printf("Index %d is %d\n",i,foo[i] );
}
return 0;
}
It is pretty clear that each index in the array will hold a random assortment of numbers before I formally initialize them:
Index 0 is 0
Index 1 is 0
Index 2 is 0
Index 3 is 0
.
.
.
Index 6087 is 0
Index 6088 is 1377050464
Index 6089 is 32767
Index 6090 is 1680893034
.
.
.
Index 9996 is 0
Index 9997 is 0
Index 9998 is 0
Index 9999 is 0
Then lets say that I initialize select index ranges of my array with values that hold a specific value for the program as a whole and must be preserved, with the goal of passing in those values for subsequent operation to some function:
//Call this block 1
foo[0] = 0;
foo[1] = 7;
foo[2] = 99;
foo[3] = 0;
//Call this block 2
foo[9996] = 0;
foo[9997] = 444;
foo[9998] = 2;
foo[9999] = 0;
for (i = 0; i < (What goes here?); i++)
{
//I must pass in only those values initialized to select indices of foo[] (Blocks 1 and 2 uncorrupted)
//How to recover those values to pass into foo_func()?
foo_func(foo[]);
}
Some of those values that I initialized foo[] with overlap with pre-existing values in the array before formally initializing the array myself. How can I pass in just the indices of the array elements that I initialized, given that there are multiple index ranges? I just can't figure this out. Thanks for any and all help!
EDIT:
I should also mention that the array itself will be read from a .txt file. I just showed the initialization in the code for illustrative purposes.
There's a number of ways you can quickly zero out the memory in the array, either while initializing or after.
For an array on the stack, initialize it with zeros. {0} is shorthand for that.
int foo[10000] = {0};
For an array on the heap, use calloc to allocate memory and initialize it with 0's.
int *foo = calloc(10000, sizeof(int));
If the array already exists, use memset to quickly overwrite all the array's memory with zeros.
memset(foo, 0, sizeof(int) * 10000);
Now all elements are zero. You can set individual elements to whatever you like one by one. For example...
int main() {
int foo[10] = {0};
foo[1] = 7;
foo[2] = 99;
foo[7] = 444;
foo[8] = 2;
for( int i = 0; i < 10; i++ ) {
printf("%d - %d\n", i, foo[i]);
}
}
That will print...
0 - 0
1 - 7
2 - 99
3 - 0
4 - 0
5 - 0
6 - 0
7 - 444
8 - 2
9 - 0
As a side note, using only a few elements of a large array is a waste of memory. Instead, use a hash table, or if you need ordering, some type of tree. These can be difficult to implement correctly, but a library such as GLib can provide you with good implementations.
Introduction
I'm making a strong assumption on your problem, and it is sparsness (a majority of the elements in your array will remain zero).
Under this assumption I would build the array as a list. I'm including a sample code, that it is not complete and it is not intended to
be---you should do your own homework :)
The core object is a struct with a pointer to a begin element and the size:
typedef struct vector {
size_t size;
vector_element_t * begin;
} vector_t;
each element of the vector has its own index and value and a pointer to the next element in a list:
typedef struct vector_element vector_element_t;
struct vector_element {
int value;
size_t index;
vector_element_t *next;
};
on this basis we can build a dynamical vector as a list, by dropping a constraint on the ordering (it is not needed, you can modify this code
to maintain the ordering), using some simple custom methods:
vector_t * vector_init(); // Initialize an empty array
void vector_destroy(vector_t* v); // Destroy the content and the array itself
int vector_get(vector_t *v, size_t index); // Get an element from the array, by searching the index
size_t vector_set(vector_t *v, size_t index, int value); // Set an element at the index
void vector_delete(vector_t *v, size_t index); // Delete an element from the vector
void vector_each(vector_t *v, int(*f)(size_t index, int value)); // Executes a callback for each element of the list
// This last function may be the response to your question
Test it online
The main example
This is a main that uses all this methods and prints in console:
int callback(size_t index, int value) {
printf("Vector[%lu] = %d\n", index, value);
return value;
}
int main() {
vector_t * vec = vector_init();
vector_set(vec, 10, 5);
vector_set(vec, 23, 9);
vector_set(vec, 1000, 3);
printf("vector_get(vec, %d) = %d\n", 1000, vector_get(vec, 1000)); // This should print 3
printf("vector_get(vec, %d) = %d\n", 1, vector_get(vec, 1)); // this should print 0
printf("size(vec) = %lu\n", vec->size); // this should print 3 (the size of initialized elements)
vector_each(vec, callback); // Calling the callback on each element of the
// array that is initialized, as you asked.
vector_delete(vec, 23);
printf("size(vec) = %lu\n", vec->size);
vector_each(vec, callback); // Calling the callback on each element of the array
vector_destroy(vec);
return 0;
}
And the output:
vector_get(vec, 1000) = 3
vector_get(vec, 1) = 0
size(vec) = 3
Vector[10] = 5
Vector[23] = 9
Vector[1000] = 3
size(vec) = 3
Vector[10] = 5
Vector[1000] = 3
The callback with the function vector_each is something you really should look at.
Implementations
I'm giving you some trivial implementations for the functions in the introdution. They are not complete,
and some checks on pointers should be introduced. I'm leaving that to you. As it is, this code is not for production and under some circumstances can also overflow.
The particular part is the search of a specific element in the vector. Every time you tranverse the list,
and this is convenient only and only if you have sparsity (the majority of your index will always return zero).
In this implementation, if you access an index that is not enlisted, you get as a result 0. If you don't want this
you should define an error callback.
Initialization and destruction
When we initialize, we allocate the memory for our vector, but with no elements inside, thus begin points to NULL. When we destroy the vector we have not only to free the vector, but also each element contained.
vector_t * vector_init() {
vector_t * v = (vector_t*)malloc(sizeof(vector_t));
if (v) {
v->begin = NULL;
v->size = 0;
return v;
}
return NULL;
}
void vector_destroy(vector_t *v) {
if (v) {
vector_element_t * curr = v->begin;
if (curr) {
vector_element_t * next = curr->next;
while (next) {
curr = curr->next;
next = next->next;
if (curr)
free(curr);
}
if (next)
free(next);
}
free(v);
}
}
The get and set methods
In get you can see how the list works (and the same concept
is used also in set and delete): we start from the begin, and
we cross the list until we reach an element with an index equal
to the one requested. If we cannot find it we simply return 0.
If we need to "raise some sort of signal" when the value is
not found, it is easy to implement an "error callback".
As long as sparsness holds, searching in the whole array for an index is a good compromise in terms of memory requirements, and efficiency may be not an issue.
int vector_get(vector_t *v, size_t index) {
vector_element_t * el = v->begin;
while (el != NULL) {
if (el->index == index)
return el->value;
el = el->next;
}
return 0;
}
// Gosh, this set function is really a mess... I hope you can understand it...
// -.-'
size_t vector_set(vector_t *v, size_t index, int value) {
vector_element_t * el = v->begin;
// Case 1: Initialize the first element of the array
if (el == NULL) {
el = (vector_element_t *)malloc(sizeof(vector_element_t));
if (el != NULL) {
v->begin = el;
v->size += 1;
el->index = index;
el->value = value;
el->next = NULL;
return v->size;
} else {
return 0;
}
}
// Case 2: Search for the element in the array
while (el != NULL) {
if (el->index == index) {
el->value = value;
return v->size;
}
// Case 3: if there is no element with that index creates a new element
if (el->next == NULL) {
el->next = (vector_element_t *)malloc(sizeof(vector_element_t));
if (el->next != NULL) {
v->size += 1;
el->next->index = index;
el->next->value = value;
el->next->next = NULL;
return v->size;
}
return 0;
}
el = el->next;
}
}
Deleting an element
With this approach it is possible to delete an element quite easily, connecting
curr->next to curr->next->next. We must though free the previous curr->next...
void vector_delete(vector_t * v, size_t index) {
vector_element_t *curr = v->begin;
vector_element_t *next = curr->next;
while (next != NULL) {
if (next->index == index) {
curr->next = next->next;
free(next);
return;
} else {
curr = next;
next = next->next;
}
}
}
An iteration function
I think this is the answer to the last part of your question,
instead passing a sequence of indexes, you pass a callback to the vector.
The callback gets and sets value in a specific index. If you want to
operate only on some specific indexes, you may include a check in the
callback itself. If you need to pass more data to the callback, check
the very last section.
void vector_each(vector_t * v, int (*f)(size_t index, int value)) {
vector_element_t *el = v->begin;
while (el) {
el->value = f(el->index, el->value);
el = el->next;
}
}
Error callback
You may want to raise some out of bounds error or something else. One solution is to enrich your list with function pointer that represent a callback that should be called when your user sk for an undefined element:
typedef struct vector {
size_t size;
vector_element_t *begin;
void (*error_undefined)(vector *v, size_t index);
} vector_t
and maybe at the end of your vector_get function you may want to do something like:
int vector_get(vector_t *v, size_t index) {
// [ . . .]
// you know at index the element is undefined:
if (v->error_undefined)
v->error_undefined(v, index);
else {
// Do something to clean up the user mess... or simply
return 0;
}
}
usually it is nice to add also an helper function to set the callback...
Passing user data to "each" callback
If you want to pass more data to the user callback, you may add a void* as last argument:
void vector_each(vector_t * v, void * user_data, int (*f)(size_t index, int value, void * user_data));
void vector_each(vector_t * v, void * user_data, int (*f)(size_t index, int value, void * user_data)) {
[...]
el->value = f(el->index, el->value, user_data);
[...]
}
if the user do not need it, he can pass a wonderful NULL.

Fastest way to remove huge number of elements from an array in C

I have dynamic array that contains thousands of elements or even more, in order not to consume a large size of memory, I can remove unwanted elements from it (i.e elements have been used and no need for them any more) so from the beginning I can allocate a smaller memory size by estimating the maximum required size after removing the elements each time.
I use this way but it takes a very very long time to finish, sometime takes 30 minutes!
int x, y ;
for (x = 0 ; x<number_of_elements_to_remove ; x++){
for (y = 0 ; y<size_of_array; y++ ){
array[y] = array[y+1];
}
}
Is there a faster way than this?
Instead of removing elements one at a time, with two loops making for an O(n2) solution, you can make a single loop, with a single read and a single write index. Go through the array, copying items as you go:
int rd = 0, wr = 0;
while (rd != size_of_array) {
if (keep_element(array[rd])) {
array[wr++] = array[rd];
}
rd++;
}
At the end of the loop wr is the number of elements kept in the array.
as I noticed you want to only delete elements from the start of the array, try this:
int x;
for(x = 0 ; x< size_of_array - number_of_elements_to_remove; x++)
array[x] = array[number_of_elements_to_remove + x];
this way you're using one for loop which reduces the complexity alot
It seems you essentially do
int y;
for (y = 0; y<size_of_array; y++){
array[y] = array[y+numbre_of_elements_to_remove];
}
The above should be faster, but there are still some caveats / problems with your code (e.g., access beyond the end od the array).
Here is the code to defragment the array.
int sparse_to_compact(int*arr, int total, int*is_valid) {
int i = 0;
int last = total - 1;
// trim the last invalid elements
for(; last >= 0 && !is_valid[last]; last--); // trim invalid elements from last
// now we keep swapping the invalid with last valid element
for(i=0; i < last; i++) {
if(is_valid[i])
continue;
arr[i] = arr[last]; // swap invalid with the last valid
last--;
for(; last >= 0 && !is_valid[last]; last--); // trim invalid elements
}
return last+1; // return the compact length of the array
}
I copied the code from this answer.
I think more efficient way is to use a link-list of buckets. And the buckets are managed by bit-string memory manager. It is like the following,
struct elem {
uint32_t index; /* helper to locate it's position in the array */
int x; /* The content/object kept in the array */
}
Suppose, our array content is int and it is encapsulated in a structure named struct elem.
enum {
MAX_BUCKET_SIZE = 1024,
MAX_BITMASK_SIZE = (MAX_BUCKET_SIZE + 63) >> 6,
};
struct bucket {
struct bucket*next; /* link to the next bucket */
uint64_t usage[MAX_BITMASK_SIZE]; /* track memory usage */
struct elem[MAX_BUCKET_SIZE]; /* the array */
};
A bucket is defined as an array of struct elem and usage mask.
struct bucket_list {
struct bucket*head; /* dynamically allocated bucket */
}container;
And a bucket list is a linked list containing all the buckets.
So we need to write memory manager code.
At first we need new bucket to be allocated when needed.
struct bucket*bk = get_empty_bucket(&container);
if(!bk) { /* no empty bucket */
/* allocate a bucket */
struct bucket*bk = (struct bucket*)malloc(sizeof(struct bucket));
assert(bk);
/* cleanup the usage flag */
memset(bk->usage, 0, sizeof(bk->usage));
/* link the bucket */
bk->next = container.head;
container.head = bk;
}
Now as we have the bucket we need to set the value in the array when needed.
for(i = 0; i < MAX_BITMASK_SIZE; i++) {
uint64_t bits = ~bk.usage[i];
if(!bits) continue; /* no space */
/* get the next empty position */
int bit_index = _builtin_ctzl(bits);
int index = (i<<6)+bit_index;
/* set the array value */
bk->elem[index].index = index;
bk->elem[index].x = 34/* my value */;
bk.usage[i] |= 1<<bit_index; /* mark/flag the array element as used */
}
Deleting the array elements is easy as to mark them unused. Now when all the elements in a bucket is unused we can delete the bucket from the link-list.
We can sometimes defragment buckets or optimize them to fit in smaller space. Otherwise when we assign new elements we can select more crowded buckets over less crowded one. When we delete we can swap the element of less crowded one into more crowded one.
It is possible to delete elements of array in efficient way,
int remove_element(int*from, int total, int index) {
if(index != (total-1))
from[index] = from[total-1];
return total; // **DO NOT DECREASE** the total here
}
It is done by swapping the element with the last value.

Create vector char array of strings

I am trying to create an array of c string in C, which simulates a behavior similar to that of vector array in c++. The array doubles its capacity whenever the (currentSize + 1) is equal to (MAX_SIZE). This is how I am doing it:
void addLog(char ** dynamicArray, int* size, int *maxSize, int command){
if (*size < *maxSize){
dynamicArray[*size] = "User selects option 1 from main menu.";
(*size)++;
}
else{
//resizing the array here
int originalSize = *maxSize;
*maxSize = *maxSize * 2;
//copy elements of dynamic array in temporary array
char **tempArray = (char**)malloc(originalSize * sizeof(char*));
for (int i = 0; i < originalSize; ++i){
memcpy(&tempArray[i], &dynamicArray[i], sizeof(dynamicArray[i]));
}
//create new array of max * 2 size
dynamicArray = (char**)malloc(*maxSize * sizeof(char*));
//copy temp to dynamic
for (int i = 0; i < originalSize; ++i){
memcpy(&dynamicArray[i], &tempArray[i], strlen(tempArray[i]));
}
for (int i = 0; i < originalSize; i++) {
free(tempArray[i]); <---- this throws an exception on heap
}
free(tempArray);
//insert new element now
dynamicArray[*size] = "User selects option 1 from main menu.";
(*size)++;
}
}
I believe this is a trivial problem for a deep copy scenario. How to resize dynamic array to 2 * capacity and then free the temporary existing elements?
You could create a reusable implementation yourself by extending a struct.
This is a bit long, but it walks you through the entire process and should have everything you need to know:
http://eddmann.com/posts/implementing-a-dynamic-vector-array-in-c/
The structure will take advantage of a fixed-size array, with a counter invariant that keeps track of how many elements are currently present. If the underlying array becomes exhausted, the addition operation will re-allocate the contents to a larger size, by way of a copy."

XS Memory leak in this code?

Unable to find where the memory leak is happening in this code.
Basically I want to write a XS wrapper for a C-function which returns a two-dimensional array.
C-function:
int CW_returnArray(double** arrayDouble, int* count)
{
int number = 10;
int index, index1;
for(index = 0; index < number; index++)
{
for(index1 = 0; index1 < 10000; index1++)
{
arrayDouble[index][index1] = 12.51;
}
count[index] = 10000;
}
return number;
}
array -> output param to hold the two dimensional array
count -> output param to hold the number of element in each 1D array
XS wrapper:
void
returnArray()
PPCODE:
{
/** variable declaration **/
double** array;
int i = 0, j=0, status;
int* count;
int totalArrays;
SV** SVArrays; // to hold the references of 1D arrays
SV** SVtempArray; // temporary array to hold the elements of 1D array
/** allocate memory for C-type variables **/
New(0, array, 10, double*);
for(i = 0; i<10;i++)
{
New(0, array[i], 10000, double);
}
New(0, count, 10, int);
/** call C function **/
status = CW_returnArray(array, count);
/** check the status and retrieve the array to store it in stack **/
if(status > 0)
{
totalArrays = status;
New(0, SVArrays, totalArrays, SV*);
for(i = 0; i<totalArrays; i++)
{
/** allocate memory for temporary SV array **/
New(0, SVtempArray, count[i], SV*);
for(j = 0; j<count[i]; j++)
{
SVtempArray[j] = newSVnv(array[i][j]);
}
/** Make an array (AV) out of temporary SV array and store the reference in SVArrays **/
SVArrays[i] = newRV_noinc((SV*) av_make(count[i], SVtempArray));
/** free the memory allocated for temp SV array **/
for(j = 0; j<count[i]; j++)
{
sv_free(SVtempArray[j]);
}
Safefree(SVtempArray); SVtempArray = NULL;
}
}
else
{
totalArrays = 0;
}
/** push the return values to stack **/
EXTEND(SP, 2);
PUSHs(sv_2mortal(newSViv(status)));
PUSHs(sv_2mortal(newRV_noinc((SV*) av_make(totalArrays, SVArrays))));
/** clean up allocated memory for SV "array of array" , if needed **/
if(totalArrays > 0)
{
Safefree(SVArrays); SVArrays = NULL;
}
/** clean up allocated memory for C-type variables **/
for(i = 0; i<10;i++)
{
Safefree(array[i]);
}
Safefree(array); array = NULL;
Safefree(count); count = NULL;
}
An "array of array" is returned from XS.
testing in Perl script:
for(1..100)
{
my ($status, $arrayref) = returnArray();
undef $status;
$arrayref = [];
system('pause');
}
Every time the function returnArray() is called, the Commit size of Perl process is increasing.
But I would expect that the $arrayref variable should be garbage collected every time and the Memory usage should not increase.
I hope, I am freeing all the allocated memory in XS. But still there is a memory leak.
What is wrong with this XS code for memory leak?
Well, the pattern of "create a template array, do av_make(), then free the template" is not very good -- you'd be much better by simply creating your array with newAV(), av_extend()ing it to the right size, and then doing av_store(newSVnv(...)) for each element. That lets you avoid the intermediate SVtempArray allocations entirely.
However, that's not what you asked about. I think your problem is that you Safefree(SVArrays) without first sv_free()ing each element. Since av_make() duplicates the contents of the source array, AFAICT you're leaking the reference created by
SVArrays[i] = newRV_noinc((SV*) av_make(count[i], SVtempArray));
You'll need to iterate over SVArrays and call sv_free() on each element before you Safefree(SVArrays).

Resources