malloc implementation? - c

I'm trying to implement malloc and free for C, and I am not sure how to reuse memory. I currently have a struct that looks like this:
typedef struct _mem_dictionary {
void *addr;
size_t size;
int freed;
} mem_dictionary;
My malloc looks like this:
void *malloc(size_t size) {
void *return_ptr = sbrk(size);
if (dictionary == NULL)
dictionary = sbrk(1024 * sizeof(mem_dictionary));
dictionary[dictionary_ct].addr = return_ptr;
dictionary[dictionary_ct].size = size;
dictionary[dictionary_ct].freed = 1;
dictionary_ct++;
return return_ptr;
}
When I free memory, I would just mark the address as 0 (that would indicate that it is free). In my malloc, I would then use a for loop to look for any value in the array to equal 0 and then allocate memory to that address. I'm kind of confused how to implement this.

The easiest way to do it is to keep a linked list of free block. In malloc, if the list is not empty, you search for a block large enough to satisfy the request and return it. If the list is empty or if no such block can be found, you call sbrk to allocate some memory from the operating system. in free, you simply add the memory chunk to the list of free block. As bonus, you can try to merge contiguous freed block, and you can change the policy for choosing the block to return (first fit, best fit, ...). You can also choose to split the block if it is larger than the request.
Some sample implementation (it is not tested, and is obviously not thread-safe, use at your own risk):
typedef struct free_block {
size_t size;
struct free_block* next;
} free_block;
static free_block free_block_list_head = { 0, 0 };
static const size_t overhead = sizeof(size_t);
static const size_t align_to = 16;
void* malloc(size_t size) {
size = (size + sizeof(size_t) + (align_to - 1)) & ~ (align_to - 1);
free_block* block = free_block_list_head.next;
free_block** head = &(free_block_list_head.next);
while (block != 0) {
if (block->size >= size) {
*head = block->next;
return ((char*)block) + sizeof(size_t);
}
head = &(block->next);
block = block->next;
}
block = (free_block*)sbrk(size);
block->size = size;
return ((char*)block) + sizeof(size_t);
}
void free(void* ptr) {
free_block* block = (free_block*)(((char*)ptr) - sizeof(size_t));
block->next = free_block_list_head.next;
free_block_list_head.next = block;
}
Note: (n + align_to - 1) & ~ (align_to - 1) is a trick to round n to the nearest multiple of align_to that is larger than n. This only works when align_to is a power of two and depends on the binary representation of numbers.
When align_to is a power of two, it only has one bit set, and thus align_to - 1 has all the lowest bit sets (ie. align_to is of the form 000...010...0, and align_to - 1 is of the form 000...001...1). This means that ~ (align_to - 1) has all the high bit set, and the low bit unset (ie. it is of the form 111...110...0). So x & ~ (align_to - 1) will set to zero all the low bits of x and round it down to the nearest multiple of align_to.
Finally, adding align_to - 1 to size ensure that we round-up to the nearest multiple of align_to (unless size is already a multiple of align_to in which case we want to get size).

You don't want to set the size field of the dictionary entry to zero -- you will need that information for re-use. Instead, set freed=1 only when the block is freed.
You cannot coalesce adjacent blocks because there may have been intervening calls to sbrk(), so that makes this easier. You just need a for loop which searches for a large enough freed block:
typedef struct _mem_dictionary
{
void *addr;
size_t size;
int freed;
} mem_dictionary;
void *malloc(size_t size)
{
void *return_ptr = NULL;
int i;
if (dictionary == NULL) {
dictionary = sbrk(1024 * sizeof(mem_dictionary));
memset(dictionary, 0, 1024 * sizeof(mem_dictionary));
}
for (i = 0; i < dictionary_ct; i++)
if (dictionary[i].size >= size
&& dictionary[i].freed)
{
dictionary[i].freed = 0;
return dictionary[i].addr;
}
return_ptr = sbrk(size);
dictionary[dictionary_ct].addr = return_ptr;
dictionary[dictionary_ct].size = size;
dictionary[dictionary_ct].freed = 0;
dictionary_ct++;
return return_ptr;
}
void free(void *ptr)
{
int i;
if (!dictionary)
return;
for (i = 0; i < dictionary_ct; i++ )
{
if (dictionary[i].addr == ptr)
{
dictionary[i].freed = 1;
return;
}
}
}
This is not a great malloc() implementation. In fact, most malloc/free implementations will allocate a small header for each block returned by malloc. The header might start at the address eight (8) bytes less than the returned pointer, for example. In those bytes you can store a pointer to the mem_dictionary entry owning the block. This avoids the O(N) operation in free. You can avoid the O(N) in malloc() by implementing a priority queue of freed blocks. Consider using a binomial heap, with block size as the index.

I am borrowing code from Sylvain's response. He seems to have missed calculating the size of the free_block* ini calculating the overhead.
In overall the code works by prepending this free_block as a header to the allocated memory.
1. When user calls malloc, malloc returns the address of the payload, right after this header.
2. when free is called, the address of the starting of the header for the block is calculated (by subtracting the header size from the block address) and that is added to the free block pool.
typedef struct free_block {
size_t size;
struct free_block* next;
} free_block;
static free_block free_block_list_head = { 0, 0 };
// static const size_t overhead = sizeof(size_t);
static const size_t align_to = 16;
void* malloc(size_t size) {
size = (size + sizeof(free_block) + (align_to - 1)) & ~ (align_to - 1);
free_block* block = free_block_list_head.next;
free_block** head = &(free_block_list_head.next);
while (block != 0) {
if (block->size >= size) {
*head = block->next;
return ((char*)block) + sizeof(free_block);
}
head = &(block->next);
block = block->next;
}
block = (free_block*)sbrk(size);
block->size = size;
return ((char*)block) + sizeof(free_block);
}
void free(void* ptr) {
free_block* block = (free_block*)(((char*)ptr) - sizeof(free_block ));
block->next = free_block_list_head.next;
free_block_list_head.next = block;
}

Related

Why does this code segfault on one machine but run fine on another?

This code segfaults on line 97 (according to gdb) on one machine (Linode) yet runs just fine on a different machine (personal) and I haven't really been able to figure out why. I tried ensuring that the heap was extended properly via sbrk but that still didn't seem to fix the issue. If anyone wouldn't mind explaining what I did wrong, I'd really appreciate it.
`
#define _DEFAULT_SOURCE
#define _BSD_SOURCE
#include <stdio.h>
#include <math.h>
typedef struct block // This is a node in a linked list representing various sections of memory
{
int size; // size of the block of allocated memory
int free; // whether the block is free and can be reallocated
struct block* next; // pointer to the next block in the linked list
char end[1]; // end represents the end of the header block struct
} block_t;
#define STRUCT_SIZE sizeof(block_t)
// --- Global variables
block_t* head = NULL; // Pointer to head of the linked list
block_t* lastVisited = NULL; // Pointer to the last visited node
void* brkPoint = NULL; // This is a pointer to the empty space on heap
// findBlock: Iterates through all blocks of memory until it is able to return a block able to contain a node of size size.
// headptr: Head of the linked list of blocks
// size: Size of the memory section being claimed
block_t* findBlock(block_t* headptr, unsigned int size) {
block_t* ptr = headptr;
while (ptr != NULL) {
if (ptr->size >= (size + STRUCT_SIZE) && ptr->free == 1) {
return ptr;
}
lastVisited = ptr;
ptr = ptr->next;
}
return ptr;
}
// splitBlock: Given a block ptr, split it into two blocks of size of size and ptr->size - size
// ptr: Pointer to the block being split
// size: Size of the first one of the two blocks
void splitBlock(block_t* ptr, unsigned int size) {
block_t* newBlock;
newBlock = ptr->end + size;
newBlock->size = ptr->size - size - STRUCT_SIZE;
newBlock->free = 1;
newBlock->next = ptr->next;
ptr->size = size;
ptr->free = 0;
ptr->next = newBlock;
}
// Increase amount of memory the program uses from the heap
// lastVisitedPtr: Pointer to the beginning of free heap (end of the program heap)
// size: The amount that you want to increase
block_t* increaseAllocation(block_t* lastVisitedPtr, unsigned int size) {
brkPoint = sbrk(0);
block_t* curBreak = brkPoint; //Current breakpoint of the heap
if (sbrk(size + STRUCT_SIZE) == (void*)-1) {
return NULL;
}
curBreak->size = (size + STRUCT_SIZE) - STRUCT_SIZE;
curBreak->free = 0;
curBreak->next = NULL;
lastVisitedPtr->next = curBreak;
if (curBreak->size > size)
splitBlock(curBreak, size);
return curBreak;
}
// malloc: A custom implementation of malloc, behaves exactly as expected
// _size: the amount of memory to be allocated into a block
// returns void*, a pointer to the block
void* mymalloc(size_t _size) {
void* brkPoint1; // New heap breakpoint
unsigned int size = _size;
int memoryNeed = size + STRUCT_SIZE; // Total size needed, including metadata
block_t* ptr; // ptr to new block
brkPoint = sbrk(0); // Set breakpoint to heap
if (head == NULL) { // If being run for the first time
if (sbrk(memoryNeed) == (void*)-1) { // If you cannot allocate enough memory, return null
return NULL;
}
brkPoint1 = sbrk(0); // Set new breakpoint to heap
head = brkPoint; // Head is at heap
head->size = memoryNeed - STRUCT_SIZE;
head->free = 0; // Head is no longer free
head->next = NULL; // No next
ptr = head; // Return pointer is head
printf("Malloc %zu bytes\n", size);
return ptr->end; // Return end of the metadata pointer (AKA beginning of allocated memory)
}
else { // Head exists
block_t* freeBlock = NULL;
freeBlock = findBlock(head, size); // Find a block that can fit size
if (freeBlock == NULL) {
freeBlock = increaseAllocation(lastVisited, size); // Increase heap and create new block
if (freeBlock == NULL) {
return NULL;
}
printf("Malloc %zu bytes\n", size);
return freeBlock->end;
}
else { // Free block with size > _size exists, claim it
if (freeBlock->size > size) { // If block's size is > size, split it down to size
splitBlock(freeBlock, size);
}
}
printf("Malloc %zu bytes\n", size);
return freeBlock->end;
}
}
// myfree: Sets block referenced by pointer to be free and merges consecutive blocks
void myfree(void* ptr) {
block_t* toFree;
toFree = ptr - STRUCT_SIZE;
if (toFree >= head && toFree <= brkPoint) {
toFree->free = 1;
printf("Freed %zu bytes\n", toFree->size);
}
}
#define ARRAY_ELEMENTS 1024
int main() {
// Allocate some data
int *data = (int *) mymalloc(ARRAY_ELEMENTS * sizeof(int));
// Do something with the data
int i = 0;
for (i = 0; i < ARRAY_ELEMENTS; i++) {
data[i] = i;
}
// Free the data
myfree(data);
return 0;
}
`
As mentioned above, I tried debugging with gdb and expanding the heap with sbrk, but that didn't fix the issue. I have no idea why it would run fine on my personal machine but not on a machine hosted elsewhere. Thanks a lot for checking this out
There is a ton of warnings which you should fix.
This one in particular is likely to cause crashes:
t.c:61:16: warning: implicit declaration of function ‘sbrk’.
Why is this likely to cause a crash?
Without a prototype, the C compiler is required (by the standard) to assume that the function returns an int.
On 32-bit platforms, this typically doesn't cause a problem because sizeof(int) == sizeof(void*) == 4.
But on 64-bit platforms sizeof(int) == 4 and sizeof(void*) == 8. Thus assigning void *p = sbrk(0); without a prototype may result in the pointer having only the low 4 bytes of the returned address; and that is likely to produce a crash when that pointer is dereferenced.
When I add missing #include <unistd.h> (where the prototype for sbrk is), the crash goes away.
In general you should always compile with -Wall -Wextra and fix resulting warnings. The compiler will often tell you about bugs, and save you a lot of debugging time.

Problems with realloc, makes program crash

Hello i'implementing a smart vector in c, and i'm having problems with the reallocation of the buffer.
this is the struct that contains the array and its infos:
struct _vector
{
item* vec;
size_t elements;
size_t size;
};
item is just a typedef that in this case happens to be int.
I made several function to manage the array, but the one that should resize it, gives me problems.
(Vector is also a typedef for struct _vector* by the way)
This is the function:
void insertVector(const Vector vec,const int pos,const item a)
{
if(vec->elements==vec->size)
{
item* temp=realloc(vec->vec,(vec->size*2)*sizeof(item));
if(temp==NULL)
{
puts("Error: space unavailable");
return;
}
//vec->vec=realloc(vec->vec,(vec->size*2)*sizeof(item));
vec->vec=temp;
vec->size*=2;
}
int size=vec->elements;
if(pos>=0&&pos<=size)
{
for(int i=size;i>pos;i--)
{
vec->vec[i]=vec->vec[i-1];
}
vec->vec[pos]=a;
vec->elements+=1;
printf("size is %lu\nelements are %lu\n",vec->size,vec->elements);
}
}
I just shift the contents to make space for the new element, and it works fine, the problem is when the array is reallocated.
when the number of valid elements is equal to the actual size of the array,
i do a realloc to double the actual size.
As soon as that if activates though the realloc makes the program crash with this error:incorrect checksum for freed object.
The problem is in the if, because it only crashes when the size and elements are equal, if i comment out that section, everything works
I don't know what could it be.
EDIT:
The functions that i used to create and the initialise the instance i'm working with are:
Vector newVector(void)
{
Vector new=malloc(sizeof(*new));
new->vec=NULL;
new->elements=0;
new->size=0;
return new;
}
and
void initVector(const Vector vec,const size_t size)
{
vec->vec=calloc(size,sizeof(item));
vec->elements=size;
vec->size=size*2;
}
Based of your comment
I created a new vector setting to zero every field, then i used this function:
void initVector(const Vector vec,const size_t size)
{
vec->vec=calloc(size,sizeof(item));
vec->elements=size;
vec->size=size*2;
}
I think you are treating the size and the number of elements incorrectly. The
initVector function just allocates memory for the vec->vec array, so
vec->elements should be 0, not size. And vec->size should be size, not
size*2. So the correct function should be
// remove the const, you are modifying the data vec is pointing to
int initVector(Vector vec, size_t size)
{
if(vec == NULL)
return 0;
vec->vec = calloc(size, sizeof *vec->vec);
if(vec->vec == NULL)
return 0;
vec->elements = 0;
vec->size = size;
return 1;
}
Now the insertVector would only allocate new space, when all allocated spaces
are used.
And I suggest you use memmove to copy the memory:
// again, remove the const here
int insertVector(Vector vec, const size_t pos, const item a)
{
if(vec == NULL)
return 0;
if(vec->elements==vec->size)
{
item* temp=realloc(vec->vec,(vec->size*2)*sizeof *temp);
if(temp==NULL)
{
fprintf(stderr, "Error: space unavailable\n");
return 0;
}
vec->vec=temp;
vec->size*=2;
}
// I use vec->elements as upper limit,
// otherwise you could have "holes" in the array and
// you wouldn't realize it.
if(pos < 0 || pos > vec->elements)
{
fprintf(stderr, "invalid position\n");
return 0;
}
memmove(vec->vec + pos + 1, vec->vec + pos, (vec->elements - pos) * sizeof *vec->vec);
vec->vec[pos] = a;
vec->elements += 1;
printf("size is %lu\nelements are %lu\n",vec->size,vec->elements);
return 1;
}
In your initVector function, you set the size incorrectly, to two times what you allocated with calloc. This memory then gets overwritten as you are adding new elements and this is the reason the free fails when you finally invoke realloc. Change initVector to:
void initVector(const Vector vec,const size_t size)
{
vec->vec=calloc(size,sizeof(item));
vec->elements=size;
vec->size=size;
}

how to use a static struc into a static function ? ( like a global )

for the need of my project i need to handle a global (representing the heap ). It's a C project, i don't have any errors at the compilation.
but when i try to use a member of struct -> segfault.
if someone could tell me where is the point ?
thanks
static t_meta *init_get_meta()
{
static t_meta *allineed = NULL;
int i;
i = 0;
if (allineed == NULL)
{
//allineed->pagesize = getpagesize();
//allineed->pagesize = 4096;
allineed->pagesize = 0; --> segfault right here
printf("LOVE\n");
while (i < 8)
{
allineed->listfree[i++] = NULL;
}
allineed->last = extend_heap(allineed);
}
return (allineed);
}
You are de-referencing a NULL pointer.
Here in this line of code you check for NULL and go ahead and access that memory which is illegal.
if (allineed == NULL)
allineed->pagesize = 0; // incorrect at this time allineed is pointing to 0x0
What you need to do is malloc the structure and than check if malloc returned with not a NULL value. something on the lines of
static t_meta *allineed = malloc(sizeof(t_meta));
if (allineed)
{
//do something
}
else
//return error
You might want to look at these questions if you are trying to implement a basic malloc yourself
How do malloc() and free() work?
How is malloc() implemented internally?
A very basic malloc would do these basic steps
void * my_malloc(size_t size)
{
size_t headersize = 1; // 1 byte header
uint8_t alignment = 8; // 8 byte alignment
// the block should be 8 bytes align
size_t alloc_size = ((size+1)+(alignment-1))&~(alignment-1);
//use system call
void *head = sbrk(alloc_size );
if(head == (void *)(-1))
return NULL;
//update the header here to mark the size and other bits depending upon req
char *header_val = (char *)head;
*header_val = (alloc_size/2) | ( 1 << 7);//only support power 2 sizes
//return updated pointer location to point to ahead of header
// after changing the pointer to char type as pointer arithmetic is not allowed on void pointers
//printf("allocated size is %d with first byte %p\n",alloc_size,header_val);
//printf(" %02x\n",(unsigned char)*(char *)header_val);
return (char *)head + headersize;
}

Implementing malloc in multithreaded environment

I want to implement malloc in a multithreaded environment, and I got the code from here.
After adding in mutex:
typedef struct free_block {
size_t size;
struct free_block* next;
pthread_mutex_t lock;
} free_block;
void free_block_init(free_block *FB){
pthread_mutex_init(&FB->lock, NULL);
}
static free_block free_block_list_head = { 0, 0 };
static const size_t overhead = sizeof(size_t);
static const size_t align_to = 8;
void* mymalloc(unsigned int size) {
size = (size + sizeof(size_t) + (align_to - 1)) & ~ (align_to - 1);
free_block* block = free_block_list_head.next;
pthread_mutex_lock(&block->lock);
free_block** head = &(free_block_list_head.next);
while (block != 0) {
if (block->size >= size) {
*head = block->next;
pthread_mutex_unlock(&block->lock);
return ((char*)block) + sizeof(size_t);
}
head = &(block->next);
block = block->next;
}
block = (free_block*)sbrk(size);
block->size = size;
pthread_mutex_unlock(&block->lock);
return ((char*)block) + sizeof(size_t);
}
unsigned int myfree(void* ptr) {
free_block* block = (free_block*)(((char*)ptr) - sizeof(size_t));
pthread_mutex_lock(&block->lock);
block->next = free_block_list_head.next;
free_block_list_head.next = block;
pthread_mutex_unlock(&block->lock);
}
I'm only able to allocated memory to the first block, then a segmentation fault error. I don't know where my error is and I'm very new to threads and locks, so any sort of help would be great! Thanks.
In this line:
pthread_mutex_lock(&block->lock);
you haven't checked yet whether block is NULL. You would need to do the locking inside the loop.
Easier still, why not have just one mutex for your whole malloc - make it a static next to free_block_list_head then you can just lock it at the start of your function and unlock it after.
If you do stick with a per block mutex, remember to add space for the mutex to your calculations. You also need to make sure the pointer you pass back is pointing to memeory after your mutex in the data structure.
Edit: Also note: You also haven't called free_block_init anywhere.

C free function (not using malloc)

I'm writing my own memory allocation program (without using malloc) and now I'm stuck with the free function (asfree in my code). I believe the functionality for the allocation is all there, the only problem lays on the free function. So by running the code below I can allocate 32 blocks: each block has a size of 48 + 16 (size of header).
So how can I deallocate/free all of them just after I have allocated them? Could you have a look at my free function and point me at the right direction?
P.S.: This is for learning purposes. I'm trying to get my head around structs, linked lists, memory allocations.
Thanks in advance.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#define BUFFER_SIZE 2048
typedef struct blk_struct
{
size_t size_blk;
struct blk_struct *next;
char data[0];
}blk_struct;
struct blk_struct *first = NULL;
static char buffer[BUFFER_SIZE];
void *asalloc(size_t size)
{
int nunits = (size + sizeof(blk_struct));
static int val = 1;
blk_struct *block, *current;
//locate position for block
if(first == NULL)
{
block = (blk_struct *)&buffer[0];
// Sanity check size.
if(nunits > BUFFER_SIZE)
return NULL;
// Initialise structure contents.
block->size_blk = size;
block->next = NULL;
// Add to linked list.
first = block;
// Give user their pointer.
return block->data;
}
//create a free list and search for a free block
for(current = first; current != NULL; current = current->next)
{
// If this element is the last element.
if(current->next == NULL)
{
block = (blk_struct *) (current->data + current->size_blk);
// Check we have space left to do this allocation
if((blk_struct *) &block->data[size] >
(blk_struct *) &buffer[BUFFER_SIZE])
{
printf("No more space\n");
return NULL;
}
// Initialise structure contents.
block->size_blk = size;
block->next = NULL;
// Add to linked list.
current->next = block;
// Give user their pointer.
return block->data;
}
}
printf("List Error\n");
return NULL;
}
// 'Free' function. blk_ptr = pointer to the block of memory to be released
void asfree(void *blk_ptr)
{
struct blk_struct *ptr = first;
struct blk_struct *tmp = NULL;
while(ptr != NULL)
{
if(ptr == blk_ptr)
{
printf("Found your block\n");
free(blk_ptr);
break;
}
tmp = ptr;
ptr = ptr->next;
}
}
// Requests fixed size pointers
int test_asalloc(void)
{
void *ptr = NULL;
int size = 48;
int i = 1;
int total = 0;
do
{
ptr = asalloc(size);
if(ptr != NULL)
{
memset(ptr, 0xff, size);
printf("Pointer %d = %p%s", i, ptr, (i % 4 != 0) ? ", " : "\n");
// each header needs 16 bytes: (sizeof(blk_struct))
total += (size + sizeof(blk_struct));
i++;
}
asfree(ptr); // *** <--- Calls the 'free' function ***
}
while(ptr != NULL);
printf("%d expected %zu\nTotal size: %d\n", i - 1,
BUFFER_SIZE / (size + sizeof(blk_struct)), total);
}
int main(void)
{
test_asalloc();
return 0;
}
I can see some problems with your asfree.
You need to subtract sizeof(blk_struct) when looking for block head. As allocated data the user wants to free are right behind the header and you have pointer to that data, not the header.
The second problem is what to do when you get the header. You cannot just call free on the data. You need to have some flag in the header and mark the block as free. And next time you try to allocate a block you need to be able to reuse free blocks, not just creating new blocks at the end. It is also good to be able to split a large free block into two smaller. To avoid fragmentation is is needed to merge neighbour free blocks to one larger.
Currently I am writing an OS as a school project. I recommend you to use simple alocator working like this:
Blocks of memory have headers containing links to neighbour blocks and free flag.
At the beginning there is one large block with free flag covering whole memory.
When malloc is called, blocks are searched from first to last. When large enough block is found it is split into two(allocated one and free reminder). Pointer to allocated block data is returned.
When free is called, matching block is marked as free. If neighbour blocks are also free they are merged.
I think this is the basic to be able to do malloc and free without fragmentation and loosing memory.
This is how a structure of headers and footer can look like:
// Header of a heap block
typedef struct {
// Size of the block including header and footer
size_t size;
// Indication of a free block
bool free;
// A magic value to detect overwrite of heap header.
uint32_t magic;
} heap_block_head_t;
// Footer of a heap block
typedef struct {
// A magic value to detect overwrite of heap footer.
uint32_t magic;
// Size of the block
size_t size;
} heap_block_foot_t;
The heap full of blocks with headers and footers like the one above is much like a linked list. Blocks do not describe their neighbours explicitly, but as long as you now they are there you can find them easily. If you have a position of one header then you can add block size to that position and you have a position of a header of the next block.
// Get next block
heap_block_head_t *current = ....
heap_block_head_t *next = (heap_block_head_t*)(((void*) current) + current->size);
// Get previous block
heap_block_head_t *current = ....
heap_block_foot_t *prev_foot = (heap_block_foot_t*)(((void*) current) - sizeof(heap_block_foot_t));
heap_block_head_t *prev = (heap_block_head_t*)(((void*) prev_foot) + sizeof(heap_block_foot_t) - prev_foot->size);
// Not sure if this is correct. I just wanted to illustrate the idea behind.
// Some extra check for heap start and end are needed
Hope this helps.

Resources