I am not understanding the 'aligning allocated memory' part from the mprotect usage.
I am referring to the code example given on http://linux.die.net/man/2/mprotect
char *p;
char c;
/* Allocate a buffer; it will have the default
protection of PROT_READ|PROT_WRITE. */
p = malloc(1024+PAGESIZE-1);
if (!p) {
perror("Couldn't malloc(1024)");
exit(errno);
}
/* Align to a multiple of PAGESIZE, assumed to be a power of two */
p = (char *)(((int) p + PAGESIZE-1) & ~(PAGESIZE-1));
c = p[666]; /* Read; ok */
p[666] = 42; /* Write; ok */
/* Mark the buffer read-only. */
if (mprotect(p, 1024, PROT_READ)) {
perror("Couldn't mprotect");
exit(errno);
}
For my understanding, I tried using a PAGESIZE of 16, and 0010 as address of p.
I ended up getting 0001 as the result of (((int) p + PAGESIZE-1) & ~(PAGESIZE-1)).
Could you please clarify how this whole 'alignment' works?
Thanks,
Assuming that PAGESIZE is a power of 2 (a requirement), an integral value x can be rounded down to a multiple of PAGESIZE with (x & ~(PAGESIZE-1)). Similarly, ((x + PAGESIZE-1) & ~(PAGESIZE-1)) will result in x rounded up to a multiple of PAGESIZE.
For example, if PAGESIZE is 16, then in binary with a 32-bit word:
00000000000000000000000000010000 PAGESIZE
00000000000000000000000000001111 PAGESIZE-1
11111111111111111111111111110000 ~(PAGESIZE-1)
A bitwise-and (&) with the above value will clear the low 4 bits of the value, making it a multiple of 16.
That said, the code quoted in the description is from an old version of the manual page, and is not good because it wastes memory and does not work on 64-bit systems. It is better to use posix_memalign() or memalign() to obtain memory that is already properly aligned. The example on the current version of the mprotect() manual page uses memalign(). The advantage of posix_memalign() is that it is part of the POSIX standard, and does not have different behavior on different systems like the older non-standard memalign().
Related
I came across following algorithm that aligns virtual address to immediate next page bounday.
VirtualAddr = (VirtualAddr & ~(PageSize-1));
Also, given a length of bytes aligns length (rounds it) to be on the page boundary
len = ((PageSize-1)&len) ? ((len+PageSize) & ~(PageSize-1)):len;
I am finding it hard to decipher how this works.
Can someone help me out to break it down?
Those calculations assume that the page size is a power of 2 (which is the case for
all systems that I know of), for example
PageSize = 4096 = 2^12 = 1000000000000 (binary)
Then (written as binary numbers)
PageSize-1 = 00...00111111111111
~(PageSize-1) = 11...11000000000000
which means that
(VirtualAddr & ~(PageSize-1))
is VirtualAddr with the lower 12 bits set to zero or, in other words,
VirtualAddr rounded down to the next multiple of 2^12 = PageSize.
Now you can (hopefully) see that in
len = ((PageSize-1)&len) ? ((len+PageSize) & ~(PageSize-1)):len;
the first expression
((PageSize-1)&len)
is zero exactly if len is a multiple of PageSize. In that case, len is left
unchanged. Otherwise (len + PageSize) is rounded down to the next multiple of
PageSize.
So in any case, len is rounded up to the next multiple of PageSize.
I think the first one should be
VirtualAddr = (VirtualAddr & ~(PageSize-1)) + PageSize;
This one-liner will do it - if it is already aligned aligned it will not skip to the next page boundary:
aligned = ((unsigned long) a & (getpagesize()-1)) ? (void *) (((unsigned long) a+getpagesize()) & ~(getpagesize()-1)) : a;
This one-liner will do it - if it is already aligned aligned it will not skip to the next page boundary:
if you really do want to skip to the next page boundary even if it's already aligned - just do:
aligned = (void *) (((unsigned long) a+getpagesize()) & ~(getpagesize()-1))
This should avoid all compiler warnings, too.
getpagesize() is a POSIX thing. #include <unistd.h> to avoid warnings.
I saw this interview question and wanted to know if my function is doing what it's supposed to or if there's a better way to do this.
Here's the exact quote of the question:
The operating system typically allocates memory in pages such that the base address of the page are 0, 4K, 8K etc. Given two addresses (pointers), write a function to find if two pointers are on the same page. Here's the function prototype: int AreOnSamePage (void * a, void * b);
Here's my implementation. I made it return 4 if it's between 4k and 8k. It returns 1 if it's between 0 and 4k and it returns -1 if it's over 8k away. Am I getting the right addresses? The interview question is worded vaguely. Is it correct to use long's since the addresses could be pretty big?
int AreOnSamePage(void* a, void* b){
long difference = abs(&a - &b);
printf("%ld %ld\n",(long)&a,(long)&b);
if(difference > 8000)
return -1;
if(difference >= 4000)
return 4;
return 1;
}
a and b are pointers, so the distance between them is:
ptrdiff_t difference = (ptrdiff_t) abs((char *)a - (char *) b)
But you don't need it.
Two pointers are on the same page, if
(uintptr_t)a / 4096 == ( uintptr_t ) b / 4096
Else they are on different pages.
So:
int AreOnSamePage(void* a, void* b) {
const size_t page_size = 4096;
if ( (uintptr_t) a / page_size == (uintptr_t) b / page_size)
return 1;
else
return 0;
}
There are many problems with your code.
You are comparing addresses of function parameters (they are side by side, on stack), not pointers
You for no reason compare the difference with 8000
4K != 4000
Imagine one address is 3K, other is 5K, according to your code, they are on the same page.
Bad choice of return values
The name AreOnSamePage() implies that the function returns either 0 or 1; I'd find it odd to have it return -1, 4 or other values.
If a page is 4KB, then it means you need 12 bits to index each byte inside a page (because 2^12 = 4096), so as long as the N-12 most significant bits of both pointer values compare equal, then you know they are on the same page (where N is the size of a pointer).
So you can do this:
#include <stdint.h>
static const uintptr_t PAGE_SIZE = 4096;
static const uintptr_t PAGE_MASK = ~(PAGE_SIZE-1);
int AreOnSamePage(void *a, void *b) {
return (((uintptr_t) a) & PAGE_MASK) == (((uintptr_t) b) & PAGE_MASK);
}
PAGE_MASK is a bit mask that has all N-12 most significant bits set to 1 and the 12 least significant bits set to 0. By doing the bitwise AND with an address, we effectively clear the least significant 12 bits (the offset into the page), so we can compare only the other bits that matter.
Note that uintptr_t is guaranteed to be wide enough to store pointer values, unlike long.
As already stated, you should use uintptr_t to proces the pointers. Your code is, however, wrong, as you test the distance, not the page. Also, you foget that computers use powers of two. 8000 is none; that would be 8192. similar for 4000.
The fastest approach for the test would be:
#include <stdbool.h>
#include <stdint.h>
// this should better be found in a system header:
#define PAGESIZE 4096U
bool samePage(void *a, void *b)
{
return ((uintptr_t)a ^ (uintptr_t)b) < PAGESIZE;
}
or:
return !(((uintptr_t)a ^ (uintptr_t)b) / PAGESIZE);
Note the result of the division will be converted to bool. If this is used as an inline, it will just tested for zero/not zero.
The XOR will zero all bits which are equal. So if any higher order bits differ, they will be set after XOR, and make the result >= PAGESIZE. This saves you one division or masking.
This requires PAGESIZE to be a power of two, of course.
Your aptempt to solve the interview's question is wrong.
You should be comparing a and b. Not &a and &b.
But even then it would still be wrong.
Consider pointer a points to last position of page 0 and pointer b points to first position of page 1. And page 1 is the one after page 0.
Their difference is 1. But they are in different pages.
In order to correctly implement it you should consider that a page is 4Kib long. 4Kib = 2^12 = 4096. So all the bits of a pair of pointers save for the last 12 will be equal if they are in the same page.
#include<stdint.h>
int AreOnSamePage(void* a, void* b){
return ((intptr_t)a & ~(intptr_t)0xFFF) ==
((intptr_t)b & ~(intptr_t)0xFFF);
}
A more concise but equivalent implementation :
int AreOnSamePage(void* a, void* b){
return ((intptr_t)a)>>12 == ((intptr_t)b)>>12;
}
I've been given the following c code for alignment
struct s *p, *new_p
p = (struct s*) malloc(sizeof(struct s) + BOUND -1);
new_p = (struct s*) (((int) p+BOUND-1) & ~(BOUND -1);
where BOUND represents 32 bytes. A line of cache is 32 bytes like in Pentium II and III but I cannot figure out the way p and new_p get aligned. Are both aligned or only new_p?
Also, I have this code for a line of cache of 64 B for a set associative cache with 8 blocks in each set and a size of 32 Kb:
int *tempA, *tempB;
...
pA= (int *) malloc (sizeof(int)*N + 63);
tempA = (int *)(((int)pA+63)&~(63));
tempB = (int *)((((int)pA+63)&~(63))+4096+64)
Accompanied with this remark: there will be a penalty if you access more than 8 address with a separation of 4 Kb.
The whole doesn't make much sense to me. Any ideas of what's going on?
Why not use _Alignas() (since C11)?
Casting a pointer to int is an invitation to disaster (aka undefined behaviour). Just think about a 64 bit machine with 32 bit (standard for most x86). If you need arithmetics on pointers, use uintptr_t (I would not recommend using intptr_t, though). However, even here, arithmetic on the value is still undefined (but very likely safe for platforms with single, linear address space).
Standard note: do not cast void * as returned by malloc().
Update:
Ok, lets take the code above and give it a proper formating and typing:
#include <stdint.h>
// align to this boundary (must be power of two!)
#define ALIGN_BOUNDARY 64U
Do not use magic numbers in your code! 2 months later you will wonder what that means.
int *tempA, *tempB;
How are those used?
int *pA = malloc (sizeof(int) * N + ALIGN_BOUNDARY - 1);
uintptr_t adjA = ((uintptr_t)pA + (ALIGN_BOUNDARY - 1)) & ~((uintptr_t) (ALIGN_BOUNDARY - 1);
This just rounds up the address to the next aligned boundary (here: 64 bytes).
tempA = (int *)adjA;
tempB = (int *)(adjA + 4096 + 64)
Not sure what the later is good for, but with the malloc given, that will result in disaster due to accessing beyond the allocated block if used with the same indexes (0..N) as *pA.
In any way, I would be very, very careful with this code. Not only it apparently is badly written/documented, but it seems also to contain errors.
Long time lurker, first time poster. I'm a student and haven't touched a programming course in two years. Now that I'm taking courses again, I'm having difficulty reading and understanding other people's code. I have a sample-code here from HMC containing a portion of a simple memory allocator in C.
void free_block(ptr p) {
*p = *p & ~1; /* clear allocated flag */
next = (unsigned*)((char*)p + *p); /* find next blk */
if ((*next & 1) == 0) /* if not allocated... */
*p += *next; /* add to this block */
}
Despite the comments, I'm still having confusion as to what's going on here exactly. I know what the code does but I would never be able to code this myself. If someone could possibly explain the mathy portions of this code, I would be extremely grateful.
Due to byte alignment, the last binary bit is not needed for allocation size,
so instead it is being used as a flag whether that block is allocated.
The size of the allocation, is represented by the value at the beginning of the block.
~1 is a bitwise inverse of 1, meaning instead of 0x01, it is 0xFE
*p = *p & ~1; /* clear allocated flag */
0xFFFFFFFE
(1111 1111 1111 1111 1111 1111 1111 1110)
A bitwise AND operation with the current value clears the last bit.
The result is the size of the original allocation.
Technically, it dereferences the value at the address at p, and performs a bitwise AND operation with 0xFFFFFFFE effectively keeping all of the value bits but the least significant
(ensuring the value no longer ends with a 1 if it originally did).
next = (unsigned*)((char*)p + *p); /* find next blk */
'next' is a pointer that point to the subsequent location from
p + [result value from the above statement]
if ((*next & 1) == 0) /* if not allocated... */
*p += *next; /* add to this block */
if the binary value at 'next' does not end with a one (bitwise AND operation),
take the value at 'p' and add the value at 'next', and assign the result to the location at 'p'.
So, in other words, if the next block is unallocated, then the original block includes it by adding that block size to itself (effectively removing it's existence).
Good luck. I hope this helps.
The operations are presumably clear, so I'll try to handle the actual interpretation in context.
*p = *p & ~1; /* clear allocated flag */
The ~ operator complements the bits. So, ~1 is all 1 bits except the last, therefore zeroing out the last bit.
We don't know why...yet.
next = (unsigned*)((char*)p + *p); /* find next blk */
Here, we have p cast to a pointer to a char ((char*)p), added to whatever the contents of p happen to be. That sum is treated as a pointer to an unsigned int. More or less, you're treating p as an offset from itself.
I'm pretty sure this is a bad idea without a lot more support code to make sure it's safe, but you're reading the code, not writing it...
if ((*next & 1) == 0) /* if not allocated... */
We now know why the least-significant bit was dropped in the first line. In this scheme, the code uses that bit to mark that the block has been allocated.
*p += *next; /* add to this block */
Here, we're just pushing p to beyond the thing we last allocated.
Without knowing what next points at, we can guess that it's moving through some sort of linked list-like structure. It's precarious, though, without knowing how p gets populated and how those pointers get structured.
But the upshot is that the code claims the next block and skips the pointer past it.
I was looking the implementation of memcpy.c, I found a different memcpy code. I couldnt understand why do they do (((ADDRESS) s) | ((ADDRESS) d) | c) & (sizeof(UINT) - 1)
#if !defined(__MACHDEP_MEMFUNC)
#ifdef _MSC_VER
#pragma function(memcpy)
#undef __MEMFUNC_ARE_INLINED
#endif
#if !defined(__MEMFUNC_ARE_INLINED)
/* Copy C bytes from S to D.
* Only works if non-overlapping, or if D < S.
*/
EXTERN_C void * __cdecl memcpy(void *d, const void *s, size_t c)
{
if ((((ADDRESS) s) | ((ADDRESS) d) | c) & (sizeof(UINT) - 1)) {
BYTE *pS = (BYTE *) s;
BYTE *pD = (BYTE *) d;
BYTE *pE = (BYTE *) (((ADDRESS) s) + c);
while (pS != pE)
*(pD++) = *(pS++);
}
else {
UINT *pS = (UINT *) s;
UINT *pD = (UINT *) d;
UINT *pE = (UINT *) (BYTE *) (((ADDRESS) s) + c);
while (pS != pE)
*(pD++) = *(pS++);
}
return d;
}
#endif /* ! __MEMFUNC_ARE_INLINED */
#endif /* ! __MACHDEP_MEMFUNC */
The code is testing whether the addresses are aligned suitably for a UINT. If so, the code copies using UINT objects. If not, the code copies using BYTE objects.
The test works by first performing a bitwise OR of the two addresses. Any bit that is on in either address will be on in the result. Then the test performs a bitwise AND with sizeof(UINT) - 1. It is expected the the size of a UINT is some power of two. Then the size minus one has all lower bits on. E.g., if the size is 4 or 8, then one less than that is, in binary 112 or 1112. If either address is not a multiple of the size of a UINT, then it will have one of these bits on, and the test will indicate it. (Usually, the best alignment for an integer object is the same as its size. This is not necessarily true. A modern implementation of this code should use _Alignof(UINT) - 1 instead of the size.)
Copying with UINT objects is faster, because, at the hardware level, one load or store instruction loads or stores all the bytes of a UINT (likely four bytes). Processors will typically copy faster when using these instructions than when using four times as many single-byte load or store instructions.
This code is of course implementation dependent; it requires support from the C implementation that is not part of the base C standard, and it depends on specific features of the processor it executes on.
A more advanced memcpy implementation could contain additional features, such as:
If one of the addresses is aligned but the other is not, use special load-unaligned instructions to load multiple bytes from one address, with regular store instructions to the other address.
If the processor has Single Instruction Multiple Data instructions, use those instructions to load or store many bytes (often 16, possibly more) in a single instruction.
The code
((((ADDRESS) s) | ((ADDRESS) d) | c) & (sizeof(UINT) - 1))
Checks to see if either s, d, or c are not aligned to the size of a UINT.
For example, if s = 0x7ff30b14, d = 0x7ffa81d8, c = 256, and sizeof(UINT) == 4, then:
s = 0b1111111111100110000101100010100
d = 0b1111111111110101000000111011000
c = 0b0000000000000000000000100000000
s | d | c = 0b1111111111110111000101111011100
(s | d | c) & 3 = 0b00
So both pointers are aligned. It is easier to copy memory between pointers that are both aligned, and this does it with only one branch.
On many architectures, *(UINT *) ptr is much faster if ptr is correctly aligned to the width of a UINT. On some architectures, *(UINT *) ptr will actually crash if ptr is not correctly aligned.