I just wrote a nice library that handles nicely a dynamic array allocated on heap in C. It supports many operations, is relatively simple to use "feeling" almost like a regular good old array. It is also easy to simulate many data structures based on it (stacks, queues, heaps, etc...)
It can handle arrays of any type, but the problem is that there's only a single type per compilation. C doesn't have templates, so it's impossible to have for example dynamic arrays of ints and dynamic arrays of chars in the same program, which is a problem.
I haven't found any real solution, everything that I found involved void*, and NO, I do NOT want an array of void pointers. It's nice to be able to put pointers, but I also want to be able to have an array of a raw data type. (such that you can add for example, 3, and access it like : array.data[i]
Should I :
Copy/paste the library once for every type I want to use (horrible, but it'll work and be efficient)
Make it a giant macro, that I can expand with the type I want (so it'll have the same effect as 1, but be somewhat elegant and usable)
Have the size of pointed elements be a variable that is part of the dynamic array structure. Will work mostly but there will be a problem with functions taking and returning the dynamic array type directly. void* will not always be a viable option
Abandon the idea and use C++ instead whenever I need such advanced features
The library works like this : Usage
/* Variable length array library for C language
* Usage :
* Declare a variable length array like this :
*
* da my_array;
*
* Always initialize like this :
*
* da_init(&da); // Creates a clean empty array
*
* Set a length to an array :
*
* da_setlength(&da, n); // Note : if elements are added they'll be uninitialized
* // If elements are removed, they're permanently lost
*
* Always free memory before it goes out of scope (avoid mem leaks !)
*
* da_destroy(&da);
*
* Access elements much like a normal array :
* - No boundary checks : da.data[i]
* - With boundary checks (debug) : da_get(data, i)
*
* da.length; // Return the current length of the variable length array (do NOT set the length by affecting this !! Use da_setlength instead.)
*
* You can add single elements at the end and beginning of array with
*
* da_add(&da, value); // Add at the end
* da_push(&da, value); // Add at the front
*
* Retrieve values at the end and front of array (while removing them) with
*
* da_remove(&da); // From the end
* da_pop(&da); // From the front
*
* Concatenate it with a standard array or another variable length array of same type with
*
* da_append(&da, array, array_length); // Standard array
* da_append(&da, &db); // Another variable length array
*/
Implementation (I'm sorry it's huge, but I have to give it for completeness of the question)
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
// Increment by which blocks are reserved on the heap
#define ALLOC_BLOCK_SIZE 16
// The type that the variable length array will contain. In this case it's "int", but it can be anything really (including pointers, arrays, structs, etc...)
typedef int da_type;
// Commend this to disable all kinds of bounds and security checks (once you're sure your program is fully tested, gains efficiency)
#define DEBUG_RUNTIME_CHECK_BOUNDS
// Data structure for variable length array variables
typedef struct
{
da_type *start; // Points to start of memory allocated region
da_type *data; // Points to logical start of array
da_type *end; // Points to end of memory allocated region
size_t length; // Length of the array
}
da;
// Initialize variable length array, allocate 2 blocks and put start pointer at the beginning
void da_init(da *da)
{
da_type *ptr = malloc(ALLOC_BLOCK_SIZE * sizeof(da_type));
if(ptr == 0) exit(1);
da->start = ptr;
da->data = ptr;
da->end = da->start + ALLOC_BLOCK_SIZE;
da->length = 0;
}
// Set the da size directly
void da_setlength(da *da, size_t newsize)
{
if(newsize % ALLOC_BLOCK_SIZE != 0)
newsize += ALLOC_BLOCK_SIZE - newsize % ALLOC_BLOCK_SIZE;
ptrdiff_t offs = da->data - da->start;
da_type *ptr = realloc(da->start, newsize * sizeof(da_type));
if(!ptr) exit(1);
da->start = ptr;
da->data = ptr + offs;
da->end = ptr + newsize;
da->length = newsize;
}
// Destroy the variable length array (basically just frees memory)
void da_destroy(da* da)
{
free(da->start);
#ifdef DEBUG_RUNTIME_CHECK_BOUNDS
da->start = NULL;
da->data = NULL;
da->end = NULL;
da->length = 0;
#endif
}
// Get an element of the array with it's index
#ifdef DEBUG_RUNTIME_CHECK_BOUNDS
//Get an element of the array with bounds checking
da_type da_get(da *da, unsigned int index)
{
if(index >= da->length)
{
printf("da error : index %u is out of bounds\n", index);
exit(1);
}
return da->data[index];
}
//Set an element of the array with bounds checking
void da_set(da *da, unsigned int index, da_type data)
{
if(index >= da->length)
{
printf("da error : index %u is out of bounds\n", index);
exit(1);
}
da->data[index] = data;
}
#else
//Get an element of the array without bounds checking
#define da_get(da, index) ((da)->data[(index)])
//Set an element of the array without bounds checking
#define da_set(da, index, v) (da_get(da, index) = v)
#endif
// Add an element at the end of the array
void da_add(da *da, da_type i)
{ // If no more memory
if(da->data + da->length >= da->end)
{ // Increase size of allocated memory block
ptrdiff_t offset = da->data - da->start;
ptrdiff_t newsize = da->end - da->start + ALLOC_BLOCK_SIZE;
da_type *ptr = realloc(da->start, newsize * sizeof(da_type));
if(!ptr) exit(1);
da->data = ptr + offset;
da->end = ptr + newsize;
da->start = ptr;
}
da->data[da->length] = i;
da->length += 1;
}
// Remove element at the end of the array (and returns it)
da_type da_remove(da *da)
{
#ifdef DEBUG_RUNTIME_CHECK_BOUNDS
if(da->length == 0)
{
printf("Error - try to remove item from empty array");
exit(1);
}
#endif
//Read last element of the array
da->length -= 1;
da_type ret_value = da->data[da->length];
//Remove redundant memory if there is too much of it
if(da->end - (da->data + da->length) > ALLOC_BLOCK_SIZE)
{
ptrdiff_t offset = da->data - da->start;
ptrdiff_t newsize = da->end - da->start - ALLOC_BLOCK_SIZE;
da_type *ptr = realloc(da->start, newsize * sizeof(da_type));
if(!ptr) exit(1);
da->data = ptr + offset;
da->end = ptr + newsize;
da->start = ptr;
}
return ret_value;
}
// Add element at the start of array
void da_push(da *da, da_type i)
{ //If array reaches bottom of the allocated space, we need to allocate more
if(da->data == da->start)
{
ptrdiff_t newsize = da->end - da->start + ALLOC_BLOCK_SIZE;
da_type *ptr = realloc(da->start, newsize * sizeof(da_type));
if(!ptr) exit(1);
memmove(ptr + ALLOC_BLOCK_SIZE, ptr, da->length * sizeof(da_type));
da->data = ptr + ALLOC_BLOCK_SIZE;
da->start = ptr;
da->end = ptr + newsize;
}
// Store element at start of array
da->length += 1;
da->data -= 1;
da->data[0] = i;
}
//Remove 1st element of array (and return it)
da_type da_pop(da *da)
{
#ifdef DEBUG_RUNTIME_CHECK_BOUNDS
if(da->length == 0)
{
printf("Error - try to remove item from empty array");
exit(1);
}
#endif
da_type ret_value = da->data[0];
da->length -= 1;
da->data += 1;
ptrdiff_t offset = da->data - da->start;
if(offset > ALLOC_BLOCK_SIZE)
{
ptrdiff_t newsize = da->end - da->start - ALLOC_BLOCK_SIZE;
da_type *ptr = realloc(da->start, newsize * sizeof(da_type));
if(!ptr) exit(1);
memmove(ptr + offset - ALLOC_BLOCK_SIZE, ptr + offset, da->length * sizeof(da_type));
da->data = ptr + offset - ALLOC_BLOCK_SIZE;
da->start = ptr;
da->end = ptr + newsize;
}
return ret_value;
}
// Append array t to s
void da_array_append(da *s, const da_type *t, size_t t_len)
{
if((s->length + t_len) > (s->end - s->start))
{ // Should reserve more space in the heap
ptrdiff_t offset = s->data - s->start;
ptrdiff_t newsize = s->length + t_len;
// Guarantees that new size is multiple of alloc block size
if(t_len % ALLOC_BLOCK_SIZE != 0)
newsize += ALLOC_BLOCK_SIZE - (t_len % ALLOC_BLOCK_SIZE);
da_type *ptr = malloc(newsize * sizeof(da_type));
if(!ptr) exit(1);
memcpy(ptr, s->data, s->length * sizeof(da_type));
memcpy(ptr + s->length, t, t_len * sizeof(da_type));
free(s->start);
s->data = ptr;
s->start = ptr;
s->end = ptr + newsize;
}
else
// Enough space in heap buffer -> do it the simple way
memmove(s->data + s->length, t, t_len * sizeof(da_type));
s->length += t_len;
}
// Append a da is a particular case of appending an array
#define da_append(s, t) da_array_append(s, (t)->data, (t)->length)
What I would do is fall back to preprocessor hackage. You can definitely achieve something like templates in C++ by adding type information when necessary.
struct da_impl {
size_t len;
size_t elem_size;
size_t allocsize; // or whatever
};
void da_init_impl(struct da_impl *impl, size_t elem_size)
{
impl->len = 0;
impl->elem_size = elem_size;
impl->allocsize = 0;
}
#define DA_TEMPLATE(t) struct { da_impl impl; t *data; }
#define da_init(a) da_init_impl(&a.impl, sizeof(*a.data))
// etc.
Then you can use it like this:
DA_TEMPLATE(int) intArray;
da_init(intArray);
da_append(intArray, 42);
int foo = intArray.data[0];
One drawback is that this creates an anonymous structure which you can't really use outside of its scope, but maybe you can live with that...
Use a union for your generic data...
typedef union
{
int *pIntArr;
double *pDblArr;
struct *pStructArr;
... // Etc.
} genericData;
...and use a struct to hold this so you can also include what data your generic data union is holding and its length.
typedef struct
{
genericData data;
int dataType; // 0 == int, 1 == double, 2 == etc.
int dataLength; // Number of elements in array
} genericDataType;
Related
can you please check this code to see why my implementation of a realloc function without using memcpy doesn't work? I am trying to figure out a way to transfer the payload size and tags from one block to another block. I got an invalid pointer error when i tried to run this code.
How do I transfer payload from one block to another without using memcpy or any other native c memory copy functions.
void* realloc(void* ptr, size_t size) {
BlockInfo * oldBlockInfo;
BlockInfo* newBlockInfo;
size_t ptrSize;
void* newPtr;
if(ptr == NULL){
return malloc(size);
}
else{
if(size == 0){
free(ptr);
return NULL;
}
}
oldBlockInfo = (BlockInfo*) UNSCALED_POINTER_SUB(ptr, WORD_SIZE);
ptrSize = SIZE(oldBlockInfo->sizeAndTags); //getting the size of the old block
//checking size
if (ptrSize >= size){//if old size is greater or equal return old pointer
return ptr;
}
else{
newPtr = malloc(size);
newBlockInfo = (BlockInfo*)UNSCALED_POINTER_ADD(newPtr,WORD_SIZE);
ptrSize = SIZE(newBlockInfo->sizeAndTags);
for (int i = WORD_SIZE;i<ptrSize;i+=WORD_SIZE){
newBlockInfo ->sizeAndTags = oldBlockInfo ->sizeAndTags;
newBlockInfo = newBlockInfo ->next;
oldBlockInfo = oldBlockInfo ->next;
}
}
//examine_heap();
free(ptr);
return newPtr;
}
this is an implementation of realloc with memcpy, but I can't use memcpy
void *mm_realloc (void *ptr, size_t size) {
int minsize;
void *newptr;
// Allocate new block, returning NULL if not possible.
newptr = malloc (size);
if (newptr == NULL) return NULL;
// Don't copy/free original block if it was NULL.
if (ptr != NULL) {
// Get size to copy - mm_getsize must give you the size of the current block.
// But, if new size is smaller, only copy that much. Many implementations
// actually reserve the 16 bytes in front of the memory to store this info, e.g.,
// +--------+--------------------------------+
// | Header | Your data |
// +--------+--------------------------------+
// ^
// +--- this is your pointer.
// <- This is the memory actually allocated ->
minsize = mm_getsize (ptr);
if (size < minsize)
minsize = size;
// Copy the memory, free the old block and return the new block.
memcpy (newptr, ptr, minsize);
free (ptr)
}
return newptr;
}
You should copy just the payload of the old block into the new block. So don't overwrite ptrSize with the information about the new block, you need the old block's size. Subtract WORD_SIZE from it because you're not copying the header, since that was filled in properly by malloc().
newPtr = malloc(size);
ptrSize -= WORD_SIZE; // subtract the header size
char *tempPtr = newPtr;
for (int i = 0; i<ptrSize; i++){
tempPtr[i] = ptr[i];
}
Context: I have written a best-fit memory allocator. It allocates large blocks of memory, and serves best fitting chunks of it to programs upon request. If memory reserves are exhausted, its asks for OS for more. All free blocks are stored on a linked-list, ordered by increasing pointer value.
Problem:: When memory is released, the program is supposed to link it back into the free list of blocks for recycling, and more crucially merges the given block with its surrounding blocks if possible. Unfortunately, this only works well as long as I do not need to ask the OS for more then one super-block to serve. When this does occur, the new blocks I receive have nonsensical addressing and get inserted between other super-block addressing space. This leads to permanent fragmentation.
Tl;dr: I am being given new super-blocks of memory whose address is within the address space of another super-block, leading to fragmentation when sub-blocks are returned.
Illustration of problem:
Here is a diagram of the problem I described above.
Numbers: Memory addresses (from real execution).
Beige blocks: Free memory.
White blocks: Memory unlinked for use.
The diagram shows the progression of memory usage until the fragmentation catalyst occurs. You can see once the block gets inserted, merging will never be possible up for the busy blocks once they are checked back in.
Code: Reproducible Example:
The following includes the allocator and a small test program. It must be compiled with at least C99.
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
/*****************************************************************************/
/* SYMBOLIC CONSTANTS */
/*****************************************************************************/
#define NEXT(hp) ((hp)->key.next)
#define UNITS(hp) ((hp)->key.units)
#define UNIT_SIZE sizeof(Header)
#define MIN_UNITS_ALLOC 64
#define MAX(a,b) ((a) > (b) ? (a) : (b))
/*****************************************************************************/
/* TYPE DEFINITIONS */
/*****************************************************************************/
typedef union header {
intmax_t align;
struct {
union header *next;
unsigned units;
} key;
} Header;
/*****************************************************************************/
/* GLOBAL VARIABLES */
/*****************************************************************************/
Header base = {.key = { NULL, 0 }};
Header *list;
/*****************************************************************************/
/* PROTOTYPES */
/*****************************************************************************/
/* Allocates a 'bytes' size block of memory. On success, returns pointer to
* the block. On error, NULL is returned. */
void *alloc (size_t bytes);
/* Returns a 'bytes' size block of allocated memory for reuse */
void release (void *bytes);
/* Attempts to reserve memory from the operating system via a system call */
static Header *reserve (unsigned units);
/*****************************************************************************/
/* FUNCTION IMPLEMENTATIONS */
/*****************************************************************************/
void *alloc (size_t bytes) {
size_t units, diff;
Header *block, *lastBlock, *best, *lastBest;
if (bytes == 0) {
return NULL;
}
if (list == NULL) {
list = base.key.next = &base;
}
best = lastBest = NULL;
units = (bytes + UNIT_SIZE - 1) / UNIT_SIZE + 1;
diff = SIZE_MAX;
for (lastBlock = list, block = NEXT(list); ; lastBlock = block, block = NEXT(block)) {
/* Loop across list, find closest fitting block */
if (UNITS(block) >= units && UNITS(block) - units < diff) {
diff = UNITS(block) - units;
best = block;
lastBest = lastBlock;
}
/* Upon cycle completion */
if (block == list) {
/* If no block available, reserve some. */
if (best == NULL) {
if ((lastBest = reserve(units)) == NULL) {
return NULL;
} else {
fprintf(stderr, "\nalloc: Out of memory, linking new block %lld of size %u.\n\n", (long long)lastBest, UNITS(lastBest));
release((void *)(lastBest + 1));
}
/* If block of perfect size, return. Else slice and return */
} else {
if (diff == 0) {
NEXT(lastBest) = NEXT(best);
} else {
UNITS(best) = diff;
best += diff;
UNITS(best) = units;
}
fprintf(stderr, "alloc: Unlinked block %lld of %u units.\n", (long long)best, UNITS(best));
return (void *)(best + 1);
}
}
}
}
void release (void *bytes) {
Header *p, *block;
block = (Header *)bytes - 1;
/* Choose p such that: p -> block -> NEXT(p) */
for (p = list; !(p < block && NEXT(p) > block); p = NEXT(p)) {
if (p >= NEXT(p) && (block > p || block < NEXT(p))) {
break;
}
}
/* Merge block with NEXT(p) if adjacent */
if (block + UNITS(block) == NEXT(p)) {
NEXT(block) = NEXT(NEXT(p));
UNITS(block) += UNITS(NEXT(p));
} else {
NEXT(block) = NEXT(p);
}
/* Merge block with p if adjacent */
if (p + UNITS(p) == block) {
NEXT(p) = NEXT(block);
UNITS(p) += UNITS(block);
} else {
NEXT(p) = block;
}
}
static Header *reserve (unsigned units) {
char *bytes, *sbrk(int);
Header *block;
units = MAX(units, MIN_UNITS_ALLOC);
if ((bytes = sbrk(units)) == (char *)-1) {
return NULL;
} else {
block = (Header *)bytes;
UNITS(block) = units;
}
return block;
}
/*****************************************************************************/
/* TESTING FUNCTIONS (DELETE) */
/*****************************************************************************/
void printFreeList (unsigned byAddress) {
Header *lp = list;
if (lp == NULL) {
fprintf(stdout, "List is NULL\n");
return;
}
do {
fprintf(stdout, "[ %lld ] -> ", (byAddress ? (long long)lp : (long long)UNITS(lp)));
lp = NEXT(lp);
} while (lp != list);
putc('\n', stdout);
}
void releaseItemAtIndex (int i, int k, long long *p[]) {
fprintf(stderr, "Releasing block %d/%d\n", i, k);
release(p[i]);
for (int j = i; j < k; j++) {
if (j + 1 < k) {
p[j] = p[j + 1];
}
}
}
#define MAX_TEST_SIZE 5000
int main (int argc, const char *argv[]) {
/* Seed PRNG: For removed random deletion (now manual) */
srand(time(NULL));
/* Array of pointers to store allocated blocks, blockSize we want to allocate. */
long long *p[MAX_TEST_SIZE], blockSize = 256;
/* Number of blocks we choose to allocate */
int k = 6;
/* Allocate said blocks */
for (int i = 0; i < k; i++) {
p[i] = alloc(blockSize * sizeof(char));
printFreeList(0); printFreeList(1); putchar('\n');
}
fprintf(stderr, "\n\n");
int idx;
while (k > 0) {
fprintf(stderr, "Delete an index between 0 up to and including %d:\n", k - 1);
scanf("%d", &idx);
releaseItemAtIndex(idx, k, p);
printFreeList(0); printFreeList(1); putchar('\n');
k--;
}
return 0;
}
Miscellaneous Details:
I am running a 64 bit operating system.
I do not know if pointer comparison on the heap is guaranteed to be valid. This is not guaranteed by the standard according to K&R.
Well I eventually wrote one a couple years later that seems to work alright. Bonus: It's using static memory.
Header File
#if !defined(STATIC_ALLOCATOR_H)
#define STATIC_ALLOCATOR_H
/*
*******************************************************************************
* (C) Copyright 2020 *
* Created: 07/04/2020 *
* *
* Programmer(s): *
* - Jillian Oduber *
* - Charles Randolph *
* *
* Description: *
* Static first-fit memory allocator *
* *
*******************************************************************************
*/
#include <iostream>
#include <cstddef>
#include <cstdint>
#include "dx_types.h"
extern "C" {
#include "stddef.h"
}
/*
*******************************************************************************
* Type Definitions *
*******************************************************************************
*/
// Defines the header block used in the custom allocator
typedef union block_h {
struct {
union block_h *next;
size_t size; // Size is in units of sizeof(block_h)
} d;
max_align_t align;
} block_h;
/*
*******************************************************************************
* Class Definitions *
*******************************************************************************
*/
class Static_Allocator {
public:
/*\
* #brief Configures the class with the given static memory capacity
* #param memory Pointer to static array (must support address comparison)
* #param size Total number of bytes alloted for distribution
\*/
Static_Allocator(uint8_t *memory, size_t size);
/*\
* #brief Returns a memory block of the desired size
* #param size Number of bytes to allocate
* #return
* - valid pointer if memory is available
* - NULL otherwise
\*/
uint8_t *alloc (size_t size);
/*\
* #brief Releases the allocated block of memory
* #param ptr Pointer to allocated block of memory
* #return
* - STATUS_OK on success
* - STATUS_ERR if invalid block
* - STATUS_BAD_PARAM on NULL parameter
\*/
dx::status_t free (uint8_t *ptr);
/*\
* #brief Returns the amount of free memory
* #return size_t Bytes (of available free memory)
\*/
size_t free_memory_size ();
/*\
* #brief Debug utility which shows what is on the free-list
* #return None
\*/
void show ();
/*\
* #brief Debug utility which asserts there is only a single
* memory block
\*/
bool unified ();
private:
// Fixed allocator capacity
size_t d_capacity;
// Fixed memory
uint8_t *d_memory;
// Pointer to head of the linked list
block_h *d_free_list;
// Available memory
size_t d_free_memory_size;
// Unit size
static const size_t mem_unit_size = sizeof(block_h);
};
#endif
Source File
#include "static_allocator.h"
Static_Allocator::Static_Allocator (uint8_t *memory, size_t size):
d_capacity(0),
d_memory(NULL),
d_free_list(NULL),
d_free_memory_size(0)
{
// Assign memory array
d_memory = memory;
// Trim off two mem_unit_size for head + init-block + spillover
d_capacity = size - (size % mem_unit_size) - 2 * mem_unit_size;
// Ensure free memory is set to capacity
d_free_memory_size = d_capacity;
}
uint8_t * Static_Allocator::alloc (size_t size)
{
block_h *last, *curr;
// Number of blocks sized units to allocate
size_t nblocks = (size + mem_unit_size - 1) / mem_unit_size + 1;
// If larger than total capacity or invalid size; immediately return
if ((nblocks * sizeof(block_h)) > d_capacity || size == 0) {
return NULL;
}
// If uninitialized, create initial list units
if ((last = d_free_list) == NULL) {
block_h *head = reinterpret_cast<block_h *>(d_memory);
head->d.size = 0;
block_h *init = head + 1;
init->d.size = d_capacity / sizeof(block_h);
init->d.next = d_free_list = last = head;
head->d.next = init;
}
// Problem: You must not be allowed to merge the init block
// Look for space - stop if wrapped around
for (curr = last->d.next; ; last = curr, curr = curr->d.next) {
// If sufficient space available
if (curr->d.size >= nblocks) {
if (curr->d.size == nblocks) {
last->d.next = curr->d.next;
} else {
curr->d.size -= nblocks;
curr += curr->d.size; // Suspect
curr->d.size = nblocks;
}
// Reassign free list head
d_free_list = last;
// Update amount of free memory available
d_free_memory_size -= nblocks * sizeof(block_h);
return reinterpret_cast<uint8_t *>(curr + 1);
}
// Otherwise not sufficient space. If reached
// the head of the list again, there is no more
// memory left
if (curr == d_free_list) {
return NULL;
}
}
}
dx::status_t Static_Allocator::free (uint8_t *ptr)
{
block_h *b, *p;
// Check if parameter is valid
if (ptr == NULL) {
std::cerr << "Null pointer!";
return dx::STATUS_BAD_PARAM;
}
// Check if memory is in range
if (!(ptr >= (d_memory + 2 * mem_unit_size)
&& ptr < (d_memory + d_capacity + 2 * mem_unit_size))) {
std::cerr << "Pointer out of range!";
return dx::STATUS_ERR;
}
// Obtain block header (ptr - sizeof(block_h))
b = reinterpret_cast<block_h *>(ptr) - 1;
// Update available memory size
d_free_memory_size += b->d.size;
// Find insertion location for block
for (p = d_free_list; !(b >= p && b < p->d.next); p = p->d.next) {
// If the block comes at the end of the list - break
if (p >= p->d.next && b > p) {
break;
}
// If at the end of the list, but block comes before next link
if (p >= p->d.next && b < p->d.next) {
break;
}
}
// [p] <----b----> [p->next] ----- [X]
// Check if we can merge forwards
if (b + b->d.size == p->d.next) {
b->d.size += (p->d.next)->d.size;
b->d.next = (p->d.next)->d.next;
} else {
b->d.next = p->d.next;
}
// Check if we can merge backwards
if (p + p->d.size == b) {
p->d.size += b->d.size;
p->d.next = b->d.next;
} else {
p->d.next = b;
}
d_free_list = p;
return dx::STATUS_OK;
}
size_t Static_Allocator::free_memory_size ()
{
return d_free_memory_size;
}
void Static_Allocator::show ()
{
std::cout << "Block Size: " << sizeof(block_h) << "\n"
<< "Capacity: " << d_capacity << "\n"
<< "Free Size (B): " << d_free_memory_size << "\n";
if (d_free_list == NULL) {
std::cout << "<uninitialized>\n";
return;
}
// Show all blocks
block_h *p = d_free_list;
do {
std::cout << "-------------------------------\n";
std::cout << "Block Address: " << (uint64_t)(p) << "\n";
std::cout << "Blocks (32B): " << p->d.size << "\n";
std::cout << "Next: " << (uint64_t)(p->d.next) << "\n";
std::cout << "{The next block is " << (uint64_t)((p->d.next) - (p)) << " bytes away, but we claim the next " << p->d.size * mem_unit_size << " bytes\n";
p = p->d.next;
} while (p != d_free_list);
std::cout << "-------------------------------\n\n\n";
}
bool Static_Allocator::unified () {
if (d_free_list == NULL) {
return false;
}
return ((d_free_list->d.next)->d.next == d_free_list);
}
In implementation of list (Python 3.4) I saw the following:
static int
list_resize(PyListObject *self, Py_ssize_t newsize)
{
PyObject **items;
size_t new_allocated;
Py_ssize_t allocated = self->allocated;
/* Bypass realloc() when a previous overallocation is large enough
to accommodate the newsize. If the newsize falls lower than half
the allocated size, then proceed with the realloc() to shrink the list.
*/
if (allocated >= newsize && newsize >= (allocated >> 1)) {
assert(self->ob_item != NULL || newsize == 0);
Py_SIZE(self) = newsize;
return 0;
}
new_allocated = (newsize >> 3) + (newsize < 9 ? 3 : 6);
/* check for integer overflow */
if (new_allocated > PY_SIZE_MAX - newsize) {
PyErr_NoMemory();
return -1;
} else {
new_allocated += newsize;
}
if (newsize == 0)
new_allocated = 0;
items = self->ob_item;
if (new_allocated <= (PY_SIZE_MAX / sizeof(PyObject *)))
PyMem_RESIZE(items, PyObject *, new_allocated);
else
items = NULL;
if (items == NULL) {
PyErr_NoMemory();
return -1;
}
self->ob_item = items;
Py_SIZE(self) = newsize;
self->allocated = new_allocated;
return 0;
}
and concretely line:
PyMem_RESIZE(items, PyObject *, new_allocated);
How I know, realloc may return another pointer, different from the obtained. I don`t understand how it works stable.
This is the definition of PyMem_Resize:
#define PyMem_Resize(p, type, n) \
( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) )
So it modifies the pointer items in-place.
See also https://docs.python.org/3/c-api/memory.html#c.PyMem_Resize
I keep passing in and returning the dirs_later_array. When I get to "new_size=..." in the else block, I end up with new_size of 2 the second time around. So far so good. But when I do a realloc
dirs_later_array = realloc(dirs_later_array,
new_size * sizeof(struct dirs_later*));
the sizeof remains at 4, the size of the pointer, for dirs_later_array. I'm able to succesfully store at dirs_later_array[1] but that value keeps getting overwritten the next time I go into the function.
struct dirs_later** add_struct(const char *findme, struct dirent *dptr,
struct stat *this_lstat, char *relative_path, const char *type_str,
struct dirs_later **dirs_later_array) {
struct dirs_later *new_dir = malloc(sizeof(struct dirs_later));
check_realloc_dirs_error(new_dir);
if (strcmp(dptr->d_name, ".")) { //Dir and not same directory
//Copy the relative path to the struct
char *relative_path2;
relative_path2 = malloc(strlen(relative_path) + 1);
check_realloc_error(relative_path2);
strcpy(relative_path2, relative_path);
//if (strlen(relative_path) > 0)
// relative_path2[strlen(relative_path) - 1] = '\0';
if (NULL != new_dir) {
new_dir->findme = findme;
new_dir->dptr = dptr;
new_dir->st_mode = this_lstat->st_mode;
new_dir->relative_path = relative_path2;
new_dir->type_str = type_str;
}
int new_size = 0;
/*
//Check if this is the first element in the struct
if (sizeof(dirs_later_array) / sizeof(struct dirs_later*) == 1) {
new_size = 1;
}
*/
if (dirs_later_array == NULL) {
dirs_later_array = malloc(sizeof(struct dirs_later*)); //Store the directory structures or process later
check_realloc_arr_error(*dirs_later_array);
new_size = 1;
} else {
//Add directories to directories array
new_size = (((sizeof(dirs_later_array) + sizeof(struct dirs_later*)))/sizeof(struct dirs_later*));
//printf("new size: %d",new_size);
}
dirs_later_array = realloc(dirs_later_array,
new_size * sizeof(struct dirs_later*));
check_realloc_arr_error(dirs_later_array);
dirs_later_array[new_size - 1] = new_dir;
}
return dirs_later_array;
}
Operator sizeof is a compile time feature and it only checks the static size of an expression. So for pointer it only returns the size of that pointer which is 4 on your platform. sizeof does not measure the size of a dynamically allocated data. There is no standard feature in C to get the size of dynamically allocated data.
Your sizeof(struct dirs_later*) should be changed to sizeof(struct dirs_later) - as before!
Also the sizeof is a compile time feature. You need a structure like this to hold the size
struct my_dirs
struct dirs_later *dirs;
int size;
};
Initialise it like this
struct my_dirs directories;
directories.size = 0;
directories.dirs = NULL;
Then to add (note realloc can take NULL as a parameter
directories.dirs = realloc(directories.dirs,
(++directories.size) * sizeof(struct dirs_later));
This would also simplify your code.
I know how to build Dynamically allocated arrays, but not how to grow them.
for example I have the following interface..
void insertVertex( vertex p1, vertex out[], int *size);
This method takes a vertex and stores it into the out array. After storing the vertex I increase the count of length for future calls.
p1 - is the vertex I'm going to add.
out[] - is the array I need to store it in (which is always full)
length - the current length
Vertex is defined as..
typedef struct Vertex{
int x;
int y;
} Vertex;
This is what I'm using in Java..
Vertex tempOut = new Vertex[size +1];
//Code to deep copy each object over
tempOut[size] = p1;
out = tempOut;
This is what I believed I could use in c..
out = realloc(out, (*size + 1) * sizeof(Vertex));
out[(*size)] = p1;
However, I keep on receiving an error message that the object was not allocated dynamically.
I found a solution that will resolve this.. Instead of using Vertex* I was going to switch to Vertex** and store pointers vs. vertex. However, after switching everything over I found out that I over looked the fact that the unit test will be providing me a Vertex out[] that everything has to be stored in.
I have also tried the following with no luck.
Vertex* temp = (Vertex *)malloc((*size + 1) * sizeof(Vertex));
for(int i = 0; i < (*size); i++)
{
temp[i] = out[i];
}
out = temp;
However, no matter what I do when I test after both of these the array returned has not changed.
Update - Requested information
out - is defined as an array of Vertex (Vertex out[])
It is originally built with the number of vertex in my polygon. For example.
out = (Vertex *)malloc(vertexInPolygon * sizeof(Vertex))
Where vertexInPolygon is an integer of the number of vertex in the polygon.
length was a typo that should have been size.
Size is an integer pointer
int *size = 0;
Each time a vertex is in the clipping plane we add it to the array of vertex and increase the size by one.
Update
To better explain myself I came up with a short program to show what I'm trying to do.
#include <stdio.h>
#include <stdlib.h>
typedef struct Vertex {
int x, y;
} Vertex;
void addPointerToArray(Vertex v1, Vertex out[], int *size);
void addPointerToArray(Vertex v1, Vertex out[], int *size)
{
int newSize = *size;
newSize++;
out = realloc(out, newSize * sizeof(Vertex));
out[(*size)] = v1;
// Update Size
*size = newSize;
}
int main (int argc, const char * argv[])
{
// This would normally be provided by the polygon
int *size = malloc(sizeof(int)); *size = 3;
// Build and add initial vertex
Vertex *out = (Vertex *)malloc((*size) * sizeof(Vertex));
Vertex v1; v1.x = 1; v1.y =1;
Vertex v2; v2.x = 2; v2.y =2;
Vertex v3; v3.x = 3; v3.y =3;
out[0] = v1;
out[1] = v2;
out[2] = v3;
// Add vertex
// This should add the vertex to the last position of out
// Should also increase the size by 1;
Vertex vertexToAdd; vertexToAdd.x = 9; vertexToAdd.y = 9;
addPointerToArray(vertexToAdd, out, size);
for(int i =0; i < (*size); i++)
{
printf("Vertx: (%i, %i) Location: %i\n", out[i].x, out[i].y, i);
}
}
One long-term problem is that you are not returning the updated array pointer from the addPointerToArray() function:
void addPointerToArray(Vertex v1, Vertex out[], int *size)
{
int newSize = *size;
newSize++;
out = realloc(out, newSize * sizeof(Vertex));
out[(*size)] = v1;
// Update Size
*size = newSize;
}
When you reallocate space, it can move to a new location, so the return value from realloc() need not be the same as the input pointer. This might work while there is no other memory allocation going on while you add to the array because realloc() will extend an existing allocation while there is room to do so, but it will fail horribly once you start allocating other data while reading the vertices. There are a couple of ways to fix this:
Vertex *addPointerToArray(Vertex v1, Vertex out[], int *size)
{
int newSize = *size;
newSize++;
out = realloc(out, newSize * sizeof(Vertex));
out[(*size)] = v1;
// Update Size
*size = newSize;
return out;
}
and invocation:
out = addPointerToArray(vertexToAdd, out, size);
Alternatively, you can pass in a pointer to the array:
void addPointerToArray(Vertex v1, Vertex **out, int *size)
{
int newSize = *size;
newSize++;
*out = realloc(*out, newSize * sizeof(Vertex));
(*out)[(*size)] = v1;
// Update Size
*size = newSize;
}
and invocation:
out = addPointerToArray(vertexToAdd, &out, size);
Neither of these rewrites addresses the subtle memory leak. The trouble is, if you overwrite the value you pass into realloc() with the return value but realloc() fails, you lose the pointer to the (still) allocated array - leaking memory. When you use realloc(), use an idiom like:
Vertex *new_space = realloc(out, newSize * sizeof(Vertex));
if (new_space != 0)
out = new_space;
else
...deal with error...but out has not been destroyed!...
Note that using realloc() to add one new item at a time leads to (can lead to) quadratic behaviour. You would be better off allocating a big chunk of memory - for example, doubling the space allocated:
int newSize = *size * 2;
If you are worried about over-allocation, at the end of the reading loop, you can use realloc() to shrink the allocated space to the exact size of the array. However, there is then a bit more book-keeping to do; you need to values: the number of vertices allocated to the array, and the number of vertices actually in use.
Finally, for now at least, note that you should really be ruthlessly consistent and use addPointerToArray() to add the first three entries to the array. I'd probably use something similar to this (untested) code:
struct VertexList
{
size_t num_alloc;
size_t num_inuse;
Vertex *list;
};
void initVertexList(VertexList *array)
{
// C99: *array = (VertexList){ 0, 0, 0 };
// Verbose C99: *array = (VertexList){ .num_inuse = 0, .num_alloc = 0, .list = 0 };
array->num_inuse = 0;
array->num_alloc = 0;
array->list = 0;
}
void addPointerToArray(Vertex v1, VertexList *array)
{
if (array->num_inuse >= array->num_alloc)
{
assert(array->num_inuse == array->num_alloc);
size_t new_size = (array->num_alloc + 2) * 2;
Vertex *new_list = realloc(array->list, new_size * sizeof(Vertex));
if (new_list == 0)
...deal with out of memory condition...
array->num_alloc = new_size;
array->list = new_list;
}
array->list[array->num_inuse++] = v1;
}
This uses the counter-intuitive property of realloc() that it will do a malloc() if the pointer passed in is null. You can instead do a check on array->list == 0 and use malloc() then and realloc() otherwise.
You might notice that this structure simplifies the calling code too; you no longer have to deal with the separate int *size; in the main program (and its memory allocation); the size is effectively bundled into the VertexList structure as num_inuse. The main program might now start:
int main(void)
{
VertexList array;
initVertexList(&array);
addPointerToArray((Vertex){ 1, 1 }, &array); // C99 compound literal
addPointerToArray((Vertex){ 2, 2 }, &array);
addPointerToArray((Vertex){ 3, 3 }, &array);
addPointerToArray((Vertex){ 9, 9 }, &array);
for (int i = 0; i < array->num_inuse; i++)
printf("Vertex %d: (%d, %d)\n", i, array->list[i].x, array->list[i].y, i);
return 0;
}
(It is coincidental that this sequence will only invoke the memory allocation once because the new size (old_size + 2) * 2 allocates 4 elements to the array the first time. It is easy to exercise the reallocation by adding a new point, or by refining the formula to (old_size + 1) * 2, or ...
If you plan to recover from memory allocation failure (rather than just exiting if it happens), then you should modify addPointerToArray() to return a status (successful, not successful).
Also, the function name should probably be addPointToArray() or addVertexToArray() or even addVertexToList().
I have a few suggestions for your consideration:
1. Don't use the same input & output parameter while using realloc as it can return NULL in case memory allocation fails & the memory pointed previously is leaked. realloc may return new block of memory (Thanks to #Jonathan Leffler for pointing out, I had missed this out). You could change your code to something on these lines:
Vertex * new_out = realloc(out, newSize * sizeof(Vertex));
if( NULL != new_out )
{
out = new_out;
out[(*size)] = v1;
}
else
{
//Error handling & freeing memory
}
2. Add NULL checks for malloc calls & handle errors when memory fails.
3. Calls to free are missing.
4. Change the return type of addPointerToArray() from void to bool to indicate if the addition is successful. In case of realloc failure you can return failure say, false else you can return success say, true.
Other observations related to excessive copies etc, are already pointed out by #MatthewD.
And few good observations by #Jonathan Leffler (:
Hope this helps!
Your sample program works fine for me. I'm using gcc 4.1.1 on Linux.
However, if your actual program is anything like your sample program, it is rather inefficient!
For example, your program copies memory a lot: structure copies - initialising out, passing vertices to addPointerToArray(), memory copies via realloc().
Pass structures via a pointer rather than by copy.
If you need to increase the size of your list type a lot, you might be better off using a linked list, a tree, or some other structure (depending on what sort of access you require later).
If you simply have to have a vector type, a standard method of implementing dynamically-sized vectors is to allocate a block of memory (say, room for 16 vertices) and double its size everytime you run out of space. This will limit the number of required reallocs.
Try these changes , it should work.
void addPointerToArray(Vertex v1, Vertex (*out)[], int *size)
{
int newSize = *size;
newSize++;
*out = realloc(out, newSize * sizeof(Vertex));
*out[(*size)] = v1;
// Update Size
*size = newSize;
}
and call the function like
addPointerToArray(vertexToAdd, &out, size);
There is a simple way to fix these type of issue (you might already know this). When you pass a argument to a function, think what exactly goes on to the stack and then combine the fact that what ever changes you make to variables present on stack would vanish when come out the function. This thinking should solve most of the issues related to passing arguments.
Coming to the optimization part, picking the right data structure is critical to the success of any project. Like somebody pointed out above, link list is a better data structure for you than the array.