sizeof(long) returns 8 bytes but the &along (address of a long) is 12 hex digits (48-bits) or 6 bytes. On OS X 64-bit compiled with clang. Is there a discrepancy here or is this a genuine 64-bit address space?
In effect modern systems use 64 bit for addresses but not all of these are used by the system since 2 to the 64 is an amount of memory no one will use in a near future. Under linux you can find out what your machine actually uses by doing cat /proc/cpuinfo. On my machine I have e.g
address sizes : 36 bits physical, 48 bits virtual
so this would allow for 64 GiB of physical memory and 256 TiB of virtual memory. This is largely enough for what this machine will ever encounter.
I think you're conflating two different concepts. Sizeof tells you how long a long is, not now long an address is. A long takes up six bytes of memory, but the pointer to it (apparently) takes only 6 (or more likely, takes 8, but the first two are all-zero because of how the memory is laid out)
Let me expand some explanation about address spaces. While pointers are 8 bytes long in a 64 bit processor, the address space rarely needs that much. That allows 2^64 bytes to be addressed, much more than we need. So, for simplicity, many processors and compilers only use 48 of those bits. See this wikipedia link for a few useful diagrams:
http://en.wikipedia.org/wiki/X86-64#Canonical_form_addresses
An address us a memory location. On a 32-bit system it will be 32-bits long, on a 64-bit system 64-bits, and so on.
The sizeof of variable is how much memory it occupies. They are completely unrelated numbers.
sizeof(long) returns 8 bytes but the &along (address of a long) is 12 hex digits (48-bits) or 6 bytes.
Yes, because when you print a number the leading 0's aren't printed. For example:
int x = 0x0000000F;
printf( "%X", x );
// prints "F"
I'm not sure if this is the exact reason why you're seeing this, but current 64-bit processors typically don't actually have a 64-bit address space. The extra hardware involved would be a waste, since 48 bits of address space is more than I expect to need in my lifetime. It could be that you're seeing a truncated version of the address (i.e., the address is not being front-padded with zeros).
Related
I know that each two letters in a hexdecimal address represents a byte, meaning that 0xFFFF is a 16bit address and can represent 65,536 bytes of memory, if the system is byte-addressable. However, if talking about a number of bits that is not a multiple of 8 (such as 14bit address), how can the operating system represnt these addresses?
0xFFFF -> 16 bit (e.g. Virtual memory address)
14 bit -> 0xFFF ? (Physical memory address)
One might say that the system has to be not byte addressable to access a not multiple of 8 address.. Then what will it be? I want the 16 bit of the virtual address to be byte addressable so we can easily access the data stored at that address, and I want to represent it in C code, but I have trouble representing the 14 bit physical memory addresses.
#define MAX_VIRT_ADDR ((0xffff) - 1) /* 65,536 */
#define MAX_PHYS_ADDR ((?) - 1) /* Max of 14bit physical memory space */
The addresses are still just numbers. On a system with a 14-bit address bus, they go from 00000000000000 (binary) up to 11111111111111 (binary).
We can also write those numbers in decimal: they go from 0 up to 16383.
Or we can write them in hexadecimal: they go from 0 up to 3FFF (or 0000 up to 3FFF).
Or in octal: they go from 0 up to 37777 (or 00000 up to 37777).
Or in any other system we like.
Typically a system based on 8-bit bytes will allow 2 bytes to be used to access a memory address. If the system has some kind of memory protection unit, and it's configured appropriately, then addresses above 3FFF may cause some kind of invalid address exception (i.e. a segfault). Otherwise, usually the extra bits are just ignored, so that no matter whether the program accesses "address" 0x0005, 0x4005, 0x8005, or 0xC005, the actual address sent to the address bus is 5 (binary: 00000000000101).
Your maximum "virtual" 16-bit address is 0xFFFF. Your maximum "physical" 14-bit address is 0x3FFF.
There's no rule that says address sizes have to be powers of 2, or that they have to be the same size as the words being addressed. It's massively more convenient to do things that way, but not required.
The old Motorola 68K had 32-bit words but a 24-bit address bus - address values were stored in 32-bit words with the upper 8 bits left unused.
As for mapping your 16-bit "virtual" address space onto a 14-bit "physical" address space, treat the upper two bits in the virtual address as a page number, treat the lower 14 bits as the offset into the page, map them directly to "physical" addresses. Store in a 16-bit type like uint16_t, then use macros to extract page number and address like so:
#define PAGENO(vaddr) ((0xC000 & vaddr) >> 14)
#define PHADDR(vaddr) (0x3FFF & vaddr)
It's simple: your 14bit address is
struct {
unsigned addr : 14;
};
You simply ignore 2 bits of the 16bit value.
Another way is to sign extend the 14bit value to 16bit. That's what 64bit systems like AMD64 or ARM64 do. They have 43-56 bits address space depending on the CPU and that is sign extended to 64bit. Addresses where the top bits aren't all 0 or all 1 are illegal.
Exotic systems where bytes have more than 8 bits still (as far as I know) use the same addressing on byte level. Only the size of a byte changes. Certain digital signal processors like this do exist in the real world, although those would not run on an OS but get coded as "bare metal". Traditionally, DSP programming is also most often done in assembler rather than C.
So for any system with 16 bit wide address bus, you'll have:
#define MAX_VIRT_ADDR 0xffffu /* 65,535 */
#define MAX_PHYS_ADDR 0xffffu /* 65,535 */
And the size of one byte is irrelevant. If you'd design the system in any other way, you'd probably lose the advantage of having a larger byte size.
Below is an excerpt from the red dragon book.
Example 7.3. Figure 7.9 is a simplification of the data layout used by C compilers for two machines that we call Machine 1 and Machine 2.
Machine 1 : The memory of Machine 1 is organized into bytes consisting of 8 bits each. Even though every byte has an address, the instruction set favors short integers being positioned at bytes whose addresses are even, and integers being positioned at addresses that are divisible by 4. The compiler places short integers at even addresses, even if it has to skip a byte as padding in the process. Thus, four bytes, consisting of 32 bits, may be allocated for a character followed by a short integer.
Machine 2: each word consists of 64 bits, and 24 bits are allowed for the address of a word. There are 64 possibilities for the individual bits inside a word, so 6 additional bits are needed to distinguish between them. By design, a pointer to a character on Machine 2 takes 30 bits — 24 to find the word and 6 for the position of the character inside the word. The strong word orientation of the instruction set of Machine 2 has led the compiler to allocate a complete word at a time, even when fewer bits would suffice to represent all possible values of that type; e.g., only 8 bits are needed to represent a character. Hence, under alignment, Fig. 7.9 shows 64 bits for each type. Within each word, the bits for each basic type are in specified positions. Two words consisting of 128 bits would be allocated for a character followed by a short integer, with the character using only 8 of the bits in the first word and the short integer using only 24 of the bits in the second word. □
I found about the concept of alignment here ,here and here. What I could understand from them is as follows: In word addressable CPUs (where size is more than a byte), there certain paddings are introduced in the data objects, such that CPU can efficiently retrieve data from the memory with minimum no. of memory cycles.
Now the Machine 1 here is actually a byte address one. And the conditions in the Machine 1 specification are probably more difficult than a simple word addressable machine having word size of say 4 bytes. In such a 64 bit machine, we need to make sure that our data items are just word aligned ,no more difficulty. But how to find the alignment in systems like Machine 1 (as given in the table above) where the simple concept of word alignment does not work, because it is byte addressable and has much more difficult specifications.
Moreover I find it quite weird that in the row for double the size of the type is more than what is given in the alignment field. Shouldn't alignment(in bits) ≥ size (in bits) ? Because alignment refers to the memory actually allocated for the data object (?).
"each word consists of 64 bits, and 24 bits are allowed for the address of a word. There are 64 possibilities for the individual bits inside a word, so 6 additional bits are needed to distinguish between them. By design, a pointer to a character on Machine 2 takes 30 bits — 24 to find the word and 6 for the position of the character inside the word." - Moreover how should this statement about the concept of the pointers, based on alignment is to be visualized (2^6 = 64, it is fine but how is this 6 bits correlating with the alignment concept)
First of all, the machine 1 is not special at all - it is exactly like a x86-32 or 32-bit ARM.
Moreover I find it quite weird that in the row for double the size of the type is more than what is given in the alignment field. Shouldn't alignment(in bits) ≥ size (in bits) ? Because alignment refers to the memory actually allocated for the data object (?).
No, this isn't true. Alignment means that the address of the lowest addressable byte in the object must be divisible by the given number of bytes.
Additionally, with C, it is also true that within arrays sizeof (ElementType) will need to be greater than or equal to the alignment of each member and sizeof (ElementType) be divisible by alignment, thus the footnote a. Therefore on the latter computer:
struct { char a, b; }
might have sizeof 16 because the characters are in distinct addressable words, whereas
struct { char a[2]; }
could be squeezed into 8 bytes.
how should this statement about the concept of the pointers, based on alignment is to be visualized (2^6 = 64, it is fine but how is this 6 bits correlating with the alignment concept)
As for the character pointers, the 6 bits is bogus. 3 bits are needed to choose one of the 8 bytes within the 8-byte words, so this is an error in the book. An ordinary byte would select just a word with 24 bits, and a character (a byte) pointer would select the word with 24 bits, and one of the 8-bit bytes inside the word with 3 bits.
I was reading through a presentation on the implementation of malloc, and on slide 7 it suggests storing a regions size and availability in a single word to save space. The alternative is to use two words, which is wasteful as the availability bit only needs to be 0 or 1.
This is the given explanation:
If blocks are aligned, low-order address bits are always 0
Why store an always-0 bit?
Use it as allocated/free flag! When reading size word, must mask out this bit
http://courses.engr.illinois.edu/cs241/sp2012/lectures/09-malloc.pdf
But I'm not really understanding how this works and how it could be implemented in C. Why is one bit of the size integer always 0?
If blocks are aligned, low-order address bits are always 0
This is the key to understanding what it going on. Many CPUs require that multibyte primitive values be stored at addresses divisible by the number of bytes in the primitive: 16-bit primitives need to be stored at even addresses; 32-bit ints need to be stored at addresses divisible by four, and so on. An attempt to access an int through a pointer that corresponds to an odd address results in a bus error.
In systems like that malloc must always return an address suitable for storing any primitive supported by the given CPU. Therefore, if CPU supports 32-bit integers, all addresses returned by malloc must be divisible by 4. Such addresses are said to be aligned. To comply, malloc implementations pad sizes blocks requested by the program by 0 to 3 bytes at the end to have length divisible by 4. As a consequence of this decision, the last two bits of an address of an aligned block will always be zero. An implementation of malloc can use these bits for its own purposes, as long as they are "masked out" before returning the result to callers.
malloc(3) (as specified by Posix) should
return a fresh block of memory; or NULL on failure; the returned pointer is not an alias of any other pointer in the program
return a suitably aligned block of memory. Alignment constraints are compiler, ABI, and processor specific. (Often, the alignment should be two words).
The size is not always zero. (actually, it is never zero). You could round it up to a multiple of two words, and use the last bit as a used/free bit.
However, pointers returned by malloc should be suitably aligned, e.g. to 8 bytes. So their bottom 3 bits are zero, and the allocated size in bytes of the malloc-ed zone is a multiple of 8 bytes (above the requested size passed to malloc), so the last 3 bits are zero (and you could use the last bit for other purposes, e.g. a used/free bit).
Why is one bit of the size integer always 0?
I see why this is confusing, but I don't think that's what they're saying on slide 7. They're saying that the low-order address bits are always 0.
Memory addresses of objects are aligned to specific boundaries, which means objects are aligned to a memory address that is a multiple of their size.
So a 64-bit integer is aligned to an eight-byte boundary;
0x7fff315470d8
If a pointer is always aligned to an eight-byte boundary, then the low-order three bits are always zero. ie: 0x816 = 10002
Basically, you can stick whatever you want in those low-order bits, so long as you take them out before dereferencing the pointer. In the 64-bit case you have 3 bits that are always 0, so you can store 3 "flags". In the case of this power point, they're saying take that lowest bit and use it for an "allocated" flag. Stick a 1 in it as long as the memory is allocated, mask it out when you send the pointer to the user.
#include <stdio.h>
int main(void){
int *ptr;
printf("the value of ptr is %p",ptr);
}
This gives me 0x7fffbd8ce900, which is only 6 bytes. Should it be 8 bytes (64bit)?
Although a pointer is 64 bits, current processors actually only support 48 bits, so the upper two bytes of an address are always either 0000 or (due to sign-extension) FFFF.
In the future, if 48 bits is no longer enough, new processors can add support for 56-bit or 64-bit virtual addresses, and existing programs will be able to utilize the additional space since they're already using 64-bit pointers.
That just means the first two bytes are zero (which, incidentally, is currently guaranteed for x86-64 chips—but that doesn't mean anything in this case, since your pointer is not initialized). %p is allowed to truncate leading zeroes, just like any other numeric type. %016p, however, is not. This should work fine:
printf("the value of ptr is %016p", ptr);
Because the 6-bytes address is just the virtual address(offset of the actual physical address). In physical architecture(X86 for instance), memory is divided into portions that may be addressed by a single index register without changing a 16-bit segment selector. In real mode of X86-CPU, a segment is always using 16-bit(2-bytes) segment-selector, which will dynamically decided by the Operating-System at the very beginning when your program started to run(i.e. creating actual running process).
Hence, if your variable have the 48-bit address 0x7fffbd8ce900, and your program have the segment-selector offset 08af, and the real address of the variable is (0x08af<<48)+0x7fffbd8ce900 = 0x08af7fffbd8ce900, which is 64-bit.
further reading pls turn to:
x86 memory segmentation
Are pointers on a 64-bit system still 4 byte aligned (similar to a double on a 32 bit system)? Or are they note 8 byte aligned?
For example, on a 64-bit system how big is the following data structure:
struct a {
void* ptr;
char myChar;
}
Would the pointer by 8 byte aligned, causing 7 bytes of padding for the character (total = 8 + 8 = 16)? Or would the pointer be 4 byte aligned (4 bytes + 4 bytes) causing 3 bytes of padding (total = 4 + 4 + 4 = 12)?
Thanks,
Ryan
Data alignment and packing are implementation specific, and can be usually changed from compiler settings (or even with pragmas).
However assuming you're using default settings, on most (if not all) compilers the structure should end up being 16 bytes total. The reason is because computers reads a data chunk with size of its native word size (which is 8 bytes in 64-bit system). If it were to pad it to 4 byte offsets, the next structure would not be properly padded to 64-bit boundary. For example in case of a arr[2], the second element of the array would start at 12-byte offset, which isn't at the native byte boundary of the machine.
I don't think you can rely on any hard-and-fast rules. I think it's a function of the compiler you use and the compilation options you choose.
Your best bet is to write a program that tests this and spits out a header file that codifies the alignment rules as #defines. You might also be able to just calculate what you're interested in right in the macros, too.
Generally on a 64-bit system:
struct a {
void* ptr; // size is 8 bytes, alignment is 8
char myChar; // size is 1 byte, alignment is 1
// padding of 7 bytes so array elements will be properly aligned
}
For a total size of 16 bytes.
But this is all implementation defined - I'm just giving an example that likely to be true for many (most?) 64-bit systems.
You would need to consult the documentation for the particular ABI you are interested in. For example, here is the System V ABI x86-64 architeture supplement - you can see on page 12 that pointers on this ABI are 8-byte aligned (so yes, the structure you show would be padded out to 16 bytes).
The language standard makes no statements about padding. The alignment rules are platform-specific (i.e., you have to align differently on e.g. a PowerPC CPU than on a x86_64 CPU), and they are implementation-defined, meaning your compiler can do whatever works (and might change that behaviour with different command-line options or after a version update).
I strongly believe that any recommendation along the lines of "this is usually this or that" is misleading, and possibly dangerous.
You could write a test program that executes a couple of sizeof() and/or offsetof()statements and writes a header for you containing some #defines stating the paddings used.
You can use autoconf to do that for you.
In the very least, you should add assert( sizeof( ... ) ) statements at the beginning of your main() function so you get informed when your assumptions are wrong.