C malloc use case - realloc versus pre-computation - c

Question

I want to create an array of structs on the heap from another data structure. Say there are N total elements to traverse, and (N-x) pointers (computed_elements) will be added to the array.
My naive strategy for this is to create an array (temp_array) size N on the stack and traverse the data structure, keeping track of how many elements need to be added to the array, and adding them to temp_array when I encounter them. Once I've finished, I malloc(computed_elements) and populate this array with the temp_array.
This is suboptimal because the second loop is unnecessary. However, I am weighing this against the tradeoff of constantly reallocating memory every iteration. Some rough code to clarify:
void *temp_array[N];
int count = 0;
for (int i = 0; i < N; i++) {
if (check(arr[i])) {
temp_array[count] = arr[i];
count++;
}
}
void *results = malloc(count * sizeof(MyStruct));
for (int i = 0; i < count; i++) {
results[i] = temp_array[i];
}
return results;
Thoughts would be appreciated.

Answer 1

One common strategy is to try to estimate the number of elements you're going to need (not a close estimate, more of a "On the order of..." type estimate). Malloc that amount of memory, and when you get "close" to that limit ("close" also being up for interpretation), realloc some more. Personally, I typically double the array when I get close to filling it.
-EDIT-
Here is the "ten minute version". (I've ensured that it builds and doesn't segfault)
Obviously I've omitted things like checking for the success of malloc/realloc, zeroing memory, etc...
#include <stdlib.h>
#include <stdbool.h>
#include <string.h> /* for the "malloc only version" (see below) */
/* Assume 500 elements estimated*/
#define ESTIMATED_NUMBER_OF_RECORDS 500
/* "MAX" number of elements that the original question seems to be bound by */
#define N 10000
/* Included only to allow for test compilation */
typedef struct
{
int foo;
int bar;
} MyStruct;
/* Included only to allow for test compilation */
MyStruct arr[N] = { 0 };
/* Included only to allow for test compilation */
bool check(MyStruct valueToCheck)
{
bool baz = true;
/* ... */
return baz;
}
int main (int argc, char** argv)
{
int idx = 0;
int actualRecordCount = 0;
int allocatedSize = 0;
MyStruct *tempPointer = NULL;
MyStruct *results = malloc(ESTIMATED_NUMBER_OF_RECORDS * sizeof(MyStruct));
allocatedSize = ESTIMATED_NUMBER_OF_RECORDS;
for (idx = 0; idx < N; idx++)
{
/* Ensure that we're not about to walk off the current array */
if (actualRecordCount == (allocatedSize))
{
allocatedSize *= 2;
/* "malloc only version"
* If you want to avoid realloc and just malloc everything...
*/
/*
tempPointer = malloc(allocatedSize);
memcpy(tempPointer, results, allocatedSize);
free(results);
results = tempPointer;
*/
/* Using realloc... */
tempPointer = realloc(results, allocatedSize);
results = tempPointer;
}
/* Check validity or original array element */
if (check(arr[idx]))
{
results[actualRecordCount] = arr[idx];
actualRecordCount++;
}
}
if (results != NULL)
{
free(results);
}
return 0;
}

Answer 2

One possibility is malloc for the size N, then run your loop, then realloc for size N-x. Memory fragmentation may result for small x.

Answer 3

The best re-usable code very often is scalable. Unless obligated to use small sizes, assume code will grow in subsequent applications and need to be reasonable efficient for large N. You do want to re-use good code.
realloc() an array by a factor of 4 or so as the need arises. Arrays may also shrink - no need to have a bloated array laying around. I've used grow by factor of 4 at intervals 1,4,16,64... and shrink intervals at 2,8,32,128... By having grow/shrink intervals apart from each other, it avoid lots of actively should N waver around an interval.
Even in small ways, like using size_t vs. int. Sure, with sizes like 1000, it makes no difference, but with code re-use, an application may push the limit: size_t is better for array indexes.
void *temp_array[N];
for (int i = 0; i < N; i++) {
void *temp_array[N];
for (size_t i = 0; i < N; i++) {