In K&R(2nd) section 8.7, I think there is an unexpected infinite for loop in free() and its test part seems wrong.
I inserted four // comments. malloc() has one, morecore() has another and free() has the others.
typedef long Align;
union header {
struct {
union header* ptr;
unsigned size;
}s;
Align x;
};
typedef union header Header;
static Header base;
static Header *freep = NULL;
/* malloc: general-purpose storage allocator */
void *malloc(unsigned nbytes)
{
Header *p, *prevp;
Header *morecore(unsigned);
unsigned nunits;
nunits = (nbytes + sizeof(Header) - 1) / sizeof(Header) + 1;
if ((prevp = freep) == NULL) { /* no free list yet */
base.s.ptr = freep = prevp = &base;
base.s.size = 0;
}
for (p = prevp->s.ptr; ; prevp = p, p = p->s.ptr) {
if (p->s.size >= nunits) { /* big enough */
if (p->s.size == nunits) /* exactly */
prevp->s.ptr = p->s.ptr;
else { /* allocate tail end */
p->s.size -= nunits;
p += p->s.size;
p->s.size = nunits;
}
freep = prevp;
return (void *)(p + 1);
}
if (p == freep) /* wrapped around free list */
// base.s.ptr = &base, freep == &base
if ((p = morecore(nunits)) == NULL)
return NULL; /* none left */
}
}
#define NALLOC 1024 /* minimum #units to requst */
/* morecore: ask system for more memory */
static Header *morecore(unsigned nu)
{
char *cp, *sbrk(int);
Header *up;
if (nu < NALLOC)
nu = NALLOC;
cp = sbrk(nu * sizeof(Header));
if (cp == (char *) -1) /* no space at all */
return NULL;
up = (Header *) cp;
up->s.size = nu;
// base.s.ptr = &base, freep == &base
free((void *)(up+1));
return freep;
}
/* free: put block ap in free list */
void free(void *ap)
{
Header *bp, *p;
bp = (Header *)ap - 1; /* point to block header */
// base.s.ptr = &base, freep == &base
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
// for (p = freep; !(0); )\
if (p >= s.ptr && (0))\
break;
if (bp + bp->s.size == p->s.ptr) { /* join to upper nbr*/
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
} else
bp->s.ptr = p->s.ptr;
if (p + p->s.size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
freep = p;
}
What I want to say is
void free(void *ap) {
/* ... */
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
/* ... */
}
this for loop is infinite. Because bp > p && bp < p->s.ptr is the same as 0 because each value of p and p->s.ptr is &base. And p = p->s.ptr does nothing. p points to base, base.s.ptr points to itself. So the value of bp > p && bp < p->s.ptr won't be changed.
Moreover, the pointer bp points to the header of memory received by sbrk(). Is it possible to compare bp with p?
I think the function free() only makes sense when we already got a valid memory block using malloc() and call it to free it. The function call free((void *)(up+1)); in morecore() and the body of free() doesn't match(I think).
Q1. Why is the for loop infinite?
Q2. Is it valid to compare bp with p in that for loop? If so, how and where is bp(which points to the memory allocated by sbrk()) placed?
Q1. Why is the for loop infinite?
No, it's not infinite.
void free(void *ap) {
/* ... */
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
/* ... */
}
When base.s.ptr = &base and freep == &base, above code is the same as
void free(void *ap) {
/* ... */
for (p = freep; !(0); p = p->s.ptr)
if (1 && (1))
break; /* freed block at start or end of arena */
/* ... */
}
(I was confused with the meaning of || operator. I thought bp > p || bp < p->s.ptr should be the same as 0 when I asked the question.)
Q2. Is it valid to compare bp with p in that for loop? If so, how and where is bp(which points to the memory allocated by sbrk()) placed?
Yes, comparing those two is valid. sbrk() returns a previous program break value. So bp and p are comparable.
Related
I'm having a lot of trouble understanding what's going on inside the free function shown on chapter 8.7 from K&R, here's the full code and some information on how the program operates:
The blocks are kept in order of increasing storage address, and the last block (highest address) points to the first.
When a request is made, the free list is scanned until a big-enough block is found.
Freeing also causes a search of the free list, to find the proper
place to insert the block being freed. If the block being freed is adjacent to a free block on either side, it is coalesced with it
into a single bigger block, so storage does not become too
fragmented. Determining adjacency is easy because the free list is
maintained in order of increasing address.
A free block contains:
A pointer to the next block in the chain
A record of the size of the block
The free space itself
The control information at the beginning is called the "header". To simplify alignment, all blocks are multiples of the header size, and the header is aligned properly.
#include <unistd.h>
#define NULL ((void *)0)
typedef long Align; /* for alignment to long boundary */
union header { /* block header */
struct {
union header *ptr; /* next block if on free list */
unsigned size; /* size of this block */
} s;
Align x; /* force alignment of blocks */
};
typedef union header Header;
static Header base; /* empty list to get started */
static Header *freep = NULL; /* start of free list */
/* malloc: general-purpose storage allocator */
void *kr_malloc(unsigned nbytes)
{
Header *p, *prevp;
Header *morecore(unsigned);
unsigned nunits;
nunits = (nbytes+sizeof(Header)-1)/sizeof(Header) + 1;
if ((prevp = freep) == NULL) { /* no free list yet */
base.s.ptr = freep = prevp = &base;
base.s.size = 0;
}
for (p = prevp->s.ptr; ; prevp = p, p = p->s.ptr) {
if (p->s.size >= nunits) { /* big enough */
if (p->s.size == nunits) /* exactly */
prevp->s.ptr = p->s.ptr;
else {
p->s.size -= nunits;
p += p->s.size;
p->s.size = nunits;
}
freep = prevp;
return p+1;
}
if (p == freep) /* wrapped around free list */
if ((p = morecore(nunits)) == NULL)
return NULL; /* none left */
}
}
#define NALLOC 1024 /* minimum #units to request */
/* morecore: ask system for more memory */
Header *morecore(unsigned nu)
{
char *cp;
void kr_free(void *);
Header *up;
if (nu < NALLOC)
nu = NALLOC;
cp = sbrk(nu * sizeof(Header));
if (cp == (char *) -1) /* no space at all */
return NULL;
up = (Header *) cp;
up->s.size = nu;
kr_free(up+1);
return freep;
}
/* free: put block ap in free list */
void kr_free(void *ap)
{
Header *bp, *p;
bp = (Header *)ap - 1; /* point to block header */
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
if (bp + bp->s.size == p->s.ptr) { /* join to upper nbr */
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
} else
bp->s.ptr = p->s.ptr;
if (p + p->s.size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
freep = p;
}
/* free: put block ap in free list */
void kr_free(void *ap)
{
Header *bp, *p;
bp = (Header *)ap - 1; /* point to block header */
So.. bp is gonna point to the header of the block passed in as an argument (ap) and p is gonna be the variable used to iterate over the linked list, so far so good.
The first loop inside free is easy, just keep iterating over the linked list until bp is between p and p->s.ptr (p / bp / p->s.ptr) so you can insert bp where it belongs.
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
And here's where I'm starting to have trouble:
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
I understand that p > p->s.ptr is a condition that will only be true if p (the pointer that is being used to iterate over the linked list) is the last free block in the linked list, and
p == p->s.ptr will only happened if p is the only block of free memory available in the linked list, so when you say:
p >= p->s.ptr
you are checking if p is the last or only free block in the linked list why would you want to check for this case? And also what's the point of:
... && (bp > p || bp < p->s.ptr))? What is it that you are checking here? And why do you break out of the loop if the condition is true?
if (bp + bp->s.size == p->s.ptr) { /* join to upper nbr */
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
} else
bp->s.ptr = p->s.ptr;
No problem here, you are just coalescing the block passed in as an argument (ap) with p->s.ptr if ap and p->s.ptr are next to each other (ap / p->s.ptr).
if (p + p->s.size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
You are doing the same as above but this time coalescing ap with p.
Testing for (bp > p || bp < p->s.ptr) when p points to the last block in the free list is necessary to only break if the block to be freed is after the last block or before the first block of the free list.
This loop starts at freep, which is not necessarily the free block with the lowest address as malloc sets freep = prevp; when it returns a block carved from the free list, and free sets freep = p; the address of the last freed block.
Setting freep this way improves efficiency when blocks are freed in increasing order of addresses, which is the case when they are freed in the same order as allocation in a clean arena assuming no intervening calls to free.
The code for function malloc() in K&R section 8.7 is below
void *malloc(unsigned nbytes) {
Header *p, *prevp;
Header *moreroce(unsigned);
unsigned nunits;
nunits = (nbytes+sizeof(Header)-1)/sizeof(header) + 1;
if ((prevp = freep) == NULL) { /* no free list yet */
base.s.ptr = freeptr = prevptr = &base;
base.s.size = 0;
}
for (p = prevp->s.ptr; ; prevp = p, p = p->s.ptr) {
if (p->s.size >= nunits) { /* big enough */
if (p->s.size == nunits) { /* exactly */
prevp->s.ptr = p->s.ptr;
} else { /* allocate tail end */
p->s.size -= nunits;
p += p->s.size;
p->s.size = nunits;
}
freep = prevp;
return (void *)(p+1);
}
if (p == freep) /* wrapped around free list */
if ((p = morecore(nunits)) == NULL)
return NULL; /* none left */
}
}
I'm mainly confused by the "allocate tail end" part.
Suppose p->s.size = 5 and nunits = 2. According to the code, we first subtract 2 from p->s.size, advance p by 3, record allocated size at that address and return (void *)(p+1).
Let p' denote p after self-increment and * denote free space. The memory after above operations should look like this:
p * * p' * *
We've actually allocated 2 units of memory, but the remaining free space for p should be 2 instead of 3, since one unit is occupied by the header information for the allocated tail end.
So I think the line
p->s.size -= nunits;
should be replaced by
p->s.size -= nunits + 1;
Have I missed anything?
The answer is in this line
nunits = (nbytes+sizeof(Header)-1)/sizeof(header) + 1;
That line takes the number of bytes requested nbytes, adds sizeof(Header)-1 to round up, divides by sizeof(header) to get the number of units needed to hold nbytes. And finally, it adds 1 to make room for the header. So all the code after that assumes that you're reserving space for nbytes (plus padding if needed) plus the header.
https://github.com/joelpet/malloc
i want to learn how to create a more efficient malloc and I found this project. I can run make to compile it:
$ make tstmalloc
make: `tstmalloc' is up to date.
But how can I run and test it? I read the readme file but it is not detailed enough. I want to compile all the programs, test all the programs and understand what the programs do. But if I run just make then it complains in a manner that I don't understand:
$ make
gcc -g -Wall -ansi -DSTRATEGY=2 -c -o malloc.o malloc.c
malloc.c: In function ‘morecore’:
malloc.c:77:3: warning: implicit declaration of function ‘getpagesize’ [-Wimplicit-function-declaration]
noPages = ((nu*sizeof(Header))-1)/getpagesize() + 1;
^
malloc.c:78:84: error: ‘MAP_ANONYMOUS’ undeclared (first use in this function)
cp = mmap(__endHeap, noPages*getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
^
malloc.c:78:84: note: each undeclared identifier is reported only once for each function it appears in
malloc.c:85:5: warning: implicit declaration of function ‘perror’ [-Wimplicit-function-declaration]
perror("failed to get more memory");
^
make: *** [malloc.o] Error 1
And if I try to compile the programs individually I get another error msg that I don't understand:
$ gcc malloc.c
/usr/bin/ld: /usr/lib/debug/usr/lib/x86_64-linux-gnu/crt1.o(.debug_info): relocation 0 has invalid symbol index 11
The contents of my malloc.c is:
#include "brk.h"
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>
#define NALLOC 1024 /* minimum #units to request */
typedef long Align; /* for alignment to long boundary */
union header { /* block header */
struct {
union header *ptr; /* next block if on free list */
unsigned size; /* size of this block - what unit? */
} s;
Align x; /* force alignment of blocks */
};
typedef union header Header;
static Header base; /* empty list to get started */
static Header *freep = NULL; /* start of free list */
/* free: put block ap in the free list */
void free(void * ap)
{
Header *bp, *p;
if(ap == NULL) return; /* Nothing to do */
bp = (Header *) ap - 1; /* point to block header */
for(p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if(p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at atrt or end of arena */
if(bp + bp->s.size == p->s.ptr) { /* join to upper nb */
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
}
else
bp->s.ptr = p->s.ptr;
if(p + p->s.size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
freep = p;
}
/* morecore: ask system for more memory */
#ifdef MMAP
static void * __endHeap = 0;
void * endHeap(void)
{
if(__endHeap == 0) __endHeap = sbrk(0);
return __endHeap;
}
#endif
static Header *morecore(unsigned nu)
{
void *cp;
Header *up;
#ifdef MMAP
unsigned noPages;
if(__endHeap == 0) __endHeap = sbrk(0);
#endif
if(nu < NALLOC)
nu = NALLOC;
#ifdef MMAP
noPages = ((nu*sizeof(Header))-1)/getpagesize() + 1;
cp = mmap(__endHeap, noPages*getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
nu = (noPages*getpagesize())/sizeof(Header);
__endHeap += noPages*getpagesize();
#else
cp = sbrk(nu*sizeof(Header));
#endif
if(cp == (void *) -1){ /* no space at all */
perror("failed to get more memory");
return NULL;
}
up = (Header *) cp;
up->s.size = nu;
free((void *)(up+1));
return freep;
}
void * malloc(size_t nbytes)
{
Header *p, *prevp;
Header * morecore(unsigned);
unsigned nunits;
if(nbytes == 0) return NULL;
nunits = (nbytes+sizeof(Header)-1)/sizeof(Header) +1;
if((prevp = freep) == NULL) {
base.s.ptr = freep = prevp = &base;
base.s.size = 0;
}
for(p= prevp->s.ptr; ; prevp = p, p = p->s.ptr) {
if(p->s.size >= nunits) { /* big enough */
if (p->s.size == nunits) /* exactly */
prevp->s.ptr = p->s.ptr;
else { /* allocate tail end */
p->s.size -= nunits;
p += p->s.size;
p->s.size = nunits;
}
freep = prevp;
return (void *)(p+1);
}
if(p == freep) /* wrapped around free list */
if((p = morecore(nunits)) == NULL)
return NULL; /* none left */
}
}
66 static Header *morecore(
67 unsigned nu) /* the amount of memory to ask for (in Header-sized units) */
68 {
69 char *cp;
70 Header *up;
71 if (nu < NALLOC)
72 nu = NALLOC;
73 cp = sbrk(nu * sizeof(Header));
74 if (cp == (char *) -1) /* no space at all */
75 return NULL;
76 up = (Header *) cp;
77 up->s.size = nu;
78 free((void *)(up+1));
79 return freep;
80 }
Your error refers to line 77 of malloc.c being no_pages = ... but the current code on GitHub is not that. So post up what you have in your malloc.c or else get the latest code from GitHub.
This is an excerpt from malloc/free implementation on K&R book. I am having great difficulty in understanding the free function, especially the if statement in the for loop: if (p >= p->s.ptr && (bp > p || bp < p->s.ptr)). If p >= p->s.ptr is true, then p must be the last node in the free list, since it is a circular list, now p->s.ptr must point to the base header (Remember therer is a list header defined static Header base;). So how can bp < p->s.ptr be true? Can somebody please explain it?
/* free: put block ap in free list */
void free(void *ap)
{
Header *bp, *p;
bp = (Header *)ap - 1; /* point to block header */
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
if (bp + bp->s.size == p->s.ptr) { /* join to upper nbr */
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
} else
bp->s.ptr = p->s.ptr;
if (p + p->s.size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
freep = p;
}
The first loop is trying to find the free block just before the block that is about to be freed. It is only traversing through the blocks that have been freed, not all blocks.
There are two cases to consider. Either the next free block comes after the current free block (the usual case), or the next block wraps back around to the beginning.
If the next free block comes after the current free block, then we need to check to see if the block we are freeing is between the current block and the next. If, however, the next free block wraps around, then due to the circular nature of the list, we have to see whether the block we are freeing comes after the current block or before the next.
Note that the free list is the list of free blocks, so the if you are talking about is checking to see if the block that is about to be freed (but isn't freed yet!) is before the first block that has been freed, which is entirely possible.
You can find this code in K&R's The C Programming Language:
void *malloc(unsigned nbytes) {
Header *p, *prevp;
Header *moreroce(unsigned);
unsigned nunits;
nunits = (nbytes+sizeof(Header)-1)/sizeof(header) + 1;
if ((prevp = freep) == NULL) { /* no free list yet */
base.s.ptr = freeptr = prevptr = &base;
base.s.size = 0;
}
for (p = prevp->s.ptr; ; prevp = p, p = p->s.ptr) {
if (p->s.size >= nunits) { /* big enough */
if (p->s.size == nunits) { /* exactly */
prevp->s.ptr = p->s.ptr;
} else { /* allocate tail end */
p->s.size -= nunits;
p += p->s.size;
p->s.size = nunits; /* STRANGE LINE???????? */
}
freep = prevp;
return (void *)(p+1);
}
if (p == freep) /* wrapped around free list */
if ((p = morecore(nunits)) == NULL)
return NULL; /* none left */
}
}
(from http://pelusa.fis.cinvestav.mx/tmatos/LaSumA/LaSumA2_archivos/Supercomputo/The_C_Programming_Language.pdf)
There's basically a linked list, and each node in the list begins with a header indicating the next node and the amount of memory the node has allocated after the header (in multiples of the header size). This linked list tracks where there is free memory.
What I don't understand is why the line at "STRANGE LINE?????" is needed. I understand the first two. We want to give the end of the node to the user, so we shrink the size, and advance p with the new size and give that to the user (+1). But then the third line wants to set the size of the place p points at to the number of units. Why? After the second line p points at a spot in the free memory which doesn't seem to have any meaning. Is this some sort of tricksy optimization I'm unaware of, or is it actually necessary?
Thanks for any help!
The reason you did not understand this line is that it cannot be understood in isolation: from the point of view of malloc, it is useless. This assignment is done in order to support functionality of the corresponding free function, which needs to know the allocated size to add the memory chunk to the free list.
void free(void *ap)
{
Header *bp, *p;
bp = (Header *)ap - 1; /* point to block header */
for (p = freep; !(bp > p && bp < p->s.ptr); p = p->s.ptr)
if (p >= p->s.ptr && (bp > p || bp < p->s.ptr))
break; /* freed block at start or end of arena */
/* Here is where the saved size comes into play */
if (bp + bp->size == p->s.ptr) { /* join to upper nbr */
bp->s.size += p->s.ptr->s.size;
bp->s.ptr = p->s.ptr->s.ptr;
} else
bp->s.ptr = p->s.ptr;
if (p + p->size == bp) { /* join to lower nbr */
p->s.size += bp->s.size;
p->s.ptr = bp->s.ptr;
} else
p->s.ptr = bp;
freep = p;
}
As you can see, the size is squirreled away in the bytes just prior to what's returned to the caller. This is the header portion that malloc and free use for their "bookkeeping".