This code for example:
int x = 75;
int *p = &x;
printf("%llx\n",p);
Writes a 64-bit number. What I'm asking is, what exactly is this number? Yes, it is an address.
But is it an absolute address in virtual memory where the value 75 is stored? Or is it possibly offset from some page marker, or an offset from the "start point" of the program's memory block?
If it matters, I'm asking about Windows 10, 64 bit, on a typical x64 intel chip.
Yes, it is the absolute address in your program's virtual address space.
It is not an offset.
In 16-bit Windows (which was common 30 years ago), a segmented memory model was used, in which pointers were segmented and consisted of a 16-bit segment pointer and a 16-bit offset (32 bits in total).
However, 32-bit and 64-bit Windows both use a flat memory model, which uses absolute addresses.
It is a virtual address, which is a virtual page number and an offset from the beginning of the page. The translation mechanism looks up in the page tables of the process to determine the corresponding physical page number and combines it with the offset to come up with the physical address.
Related
I was reading Intel 64 and IA-32 Architectures Software Developer's Manual.
I understand that before going through paging, the logical address will first have to be converted to linear address, and from linear address it will goes through paging table in order to generate the ultimate physical address.
My question is that, if we run the C code below, the address of variable a that is printed, is it a logical address or a linear address?
I know that Windows 10 64-bit is currently using long mode and so the logical address and linear address are the same, but the question I have in mind is:
Is the address we are seeing in user mode program is logical address or it's the linear address that has gone through the global descriptor table translation?
#include <stdio.h>
int main(void)
{
int a = 50;
printf("%p", &a);
return 0;
}
Windows has not used segmented memory since it stopped being 16-bit. Or to put it a different way, the GDT just spans from 0 to the end of linear space in 32-bit Windows.
All this is irrelevant when asking about C because it has no knowledge of such details. To answer we must ignore the C abstract machine and look directly at Windows on x86.
If we imagine your code running in 16-bit Windows, taking the address of a local variable is going to give you the offset into the segment. This is 16-bits. A FAR address on the other hand is 32-bits of information, the segment and the offset. The Windows function lstrlen (l for long?) takes a FAR address and can compute the length of a string coming from anywhere. The strlen C function might just use a plain char* pointer; just the segment offset. Your C compiler might support different memory models (tiny, small, compact, medium, large), giving you access to more memory, perhaps even FAR pointers. Classic DOS .com files use the tiny model, there are no segments, just a maximum of 64 KiB for all code and data. Other models might separate code and data in different segments.
In 32 and 64 bit Windows the logical and linear addresses are the same but if you have to think of it in terms of your graphic, %p is going to print the logical address. print is not going to ask the CPU/OS to translate the address in any way.
I am a beginner in Computer Science and I started learning the C language.
(So, I apologize at first if my question doesn't make any sense)
I am facing a doubt in the chapter called Pointers.
I am thinking that when we declare a variable, let's say an integer variable i.
A memory is allocated for the variable in the RAM.
In my book nothing was written about how the computer selects the memory address which is to be allocated for the variable.
I am asking this because, I was thinking a computer has 8GB RAM and a 32-bit processor.
I learnt that a 32-bit processor consists of a 32-bit register, which can allow the processor to access atmost up to 4GB of the RAM.
So, is it possible in that computer, when we declare the integer variable i, the computer allocated a memory space with address which can't be accessed by the 32-bit processor?
And if this happens what will be shown on the screen if I want to print the address of i using the address of operator?
A computer with 8 GB of RAM will have a 64 bit processor.
Many 64 bit CPU's can still run 32 bit programs, in which case that both use 4 GB. You will see the address &i as a 32 bit offset in your program's memory. But even if you did compile for 64 bits, you'd still see the 64 bit offset in your program - the OS will instruct the CPU how different programs use different parts of memory.
So I knew that a 32-bits OS can support 232 different values, which is approximately 4x109.
I would imagine that the internal representation of each value is like this:
0000 0000 0000 0000 0000 0000 0000 0000
.....
1111 1111 1111 1111 1111 1111 1111 1111
So we have approximately 4x109 different patterns here.
But Since each address consists of 4 bytes (32/8=4), shouldn't the RAM be
4x4x109?
Each address addresses one byte, in typical modern systems.
Even if the hardware can only transfer four bytes or eight bytes at a time, each byte within such a unit is given its own address. The processor may only interact with the memory hardware using 28 or 29 or some other number of bits, but it uses the additional bits to distinguish bytes within words.
When a program accesses a particular address, the processor uses the low bits to isolate bytes. When reading, it gets an entire unit from memory and then uses the low bits to isolate the requested byte or bytes. When writing, it uses the low bits to merge the selected byte or bytes into a unit of data, and then it writes the complete unit to memory.
So, with 32 bits in an address, 232 = 4,294,967,296 addresses are available, and 4,294,967,296 things can be addressed. In typical modern hardware, each of those things is one byte. Often, not all of them are available to user programs, as some addresses are reserved for special purposes.
x86 is byte addressed, which means that each byte has a unique address. This makes (under normal circumstances) the total amount of addressable memory to be 2^32 bytes (roughly 4 GB). And non addressable memory (memory that you can't have an address for) is utterly unusable.
Out of the whole address space, not all addresses are to main memory. A chunk of it is reserved for IO, so the amount of maximum RAM is even lower than 4 GB.
I will try to address your confusion
So we have approximately 4x109 different patterns here.
correct
But Since each address consists of 4 bytes (32/8=4)
irrelevant
shouldn't the RAM be 4x4x109?
no
The number of bits of the address determines how many different values can be. We already determined that 32 bits give us exactly 2^32 ≈ 4^10 different addresses. We can therefore have ≈4^10 different objects that have unique addresses. Bear with me, let's think a bit outside of the box. If the addresses were addresses of cities, then we could have a maximum of ≈4^10 cities. If the addresses were addresses of streets, we could have a maximum of ≈4^10 streets. If the addresses were addresses of apples, we could have a maximum of ≈4^10 apples. With me? If the addresses were addresses of 64-bit QWORDS then we could have a maximum of ≈4^10 64-bit qwords. If the addresses were addresses of 32-bit DWORDS then we could have a maximum of ≈4^10 32-bit qwords. But in x86 an address is none of the above. In x86 an address is an address of a byte (8 bits) so we can have a maximum of ≈4^10 bytes aka ≈4 GiB.
As you can see the width of the address only gives the number of different objects we can address. In x86 those objects are bytes, so we can have a maximum of 2^32 addressable bytes.
The normal limit of 2^32 bytes can be overcome with Physical Address Extension. This requires both OS and hardware (CPU, chipset and motherboard) support and even when these are met each program can still only work with 32 bit addresses.
For a modern OS, typically there's virtual addresses that are translated into physical addresses.
For a 32-bit OS, the virtual addresses are often (but not necessarily) 32-bit. With byte addressing, this means you can have 1234 processes where each process has 4 GiB of virtual space (or a total of 4936 GiB of virtual space). However, typically each virtual address space is split with "user-space" in one part and "kernel-space" in another part; so it might be more like 2 GiB for each process plus 2 GiB for the kernel (or a total of 2470 GiB for 1234 processes).
However, because virtual addresses are converted into physical addresses the size of a virtual address doesn't need to be the same as the size of a physical address. This means that even if virtual addresses are 32-bit, a physical address can be larger (or smaller) than 32-bit. For example, for most older 80x86 CPUs there's a "Physical Address Extensions" (PAE) feature that extends the physical address size to 36 bits (giving you a physical address space of 16 GiB), and for modern 80x86 CPUs (that are capable of running a 64-bit OS) PAE was enhanced to allow a 32-bit OS to use physical addresses up to (a "current architectural maximum" of) 52-bits, giving a physical address space size up to 4096 TiB for a 32-bit OS (in theory).
Of course the physical address space contains RAM, some ROM, some areas for devices, etc. For example, with 16 GiB of physical address space, 1.5 GiB might be reserved for things that aren't RAM, so the maximum RAM you might be able to have (and the maximum a 32-bit OS could use) might be 14.5 GiB.
Sadly(?) most motherboards don't support the maximum amount of RAM that the CPU is capable of using. For example, a lot of modern CPUs support 48-bit physical addresses (256 TiB of physical address space) but I've never seen a motherboard that's able to support more than 8 TiB of RAM and most modern motherboards don't even support 1 TIB of RAM.
In the same way, various operating systems have their own limits. For example, most 32-bit versions of Windows didn't support PAE (because of device driver compatibility issues initially, and then because everyone adopted 64-bit anyway so nobody cared); so if you had a computer with (e.g.) 8 GiB of RAM the OS can't use most of the RAM (and would probably only be able to use 3 GiB of the RAM because 1 GiB of space is probably reserved/used by ROMs, devices, etc).
Note that for 64-bit operating systems on 80x86; virtual addresses are 48-bit (not 64 bit) and physical addresses are anything from 32-bit (Atom) to 52-bit (and also not 64 bit); and Intel has been thinking of a "5-level paging" extension to allow 57-bit virtual addresses (which still won't be 64-bit).
In general (if you ignore specific CPUs); the size of a general purpose register, the size of a virtual address and the size of a physical address can all be completely different; and for a 32-bit OS (using 32-bit general purpose registers) the virtual address space size can be anything and the physical address space size can be anything; and the maximum amount of RAM you could have in the physical address space can be anything.
[It's likely I misunderstood the question. I might delete this answer later.]
For a system with 32-bit (4-byte -- assuming 8-bit bytes) addresses, there are 232 distinct address values. If the memory space is fully populated with RAM, then you can use a 32-bit address to refer to any of the 232 bytes of memory on the system.
But you can't store all those 232 address values in memory simultaneously. As you say, you would need 4 * 232 bytes to store all those address values.
Fortunately, there's no need to store all those distinct memory address values in memory. You only need to store those address values that you're actually using.
(I'm ignoring issues of virtual vs. physical memory.)
I know normally in a 32-bit machine the size of pointers used in regular C programs is 32-bit. What about in a x86 system with PAE?
It's still 32 bits.
PAE increases the size of physical memory addresses, which lets the operating system use more than 4GB RAM for running applications. To run an application the operating system maps the larger physical addresses to 32 bit virtual addresses. This means that the address space in each application is still limited to 4GB.
PAE changes the structure of page tables to allow them to address more than 32 bits worth of physical memory. Virtual memory addressing remains unchanged — pointers in userspace applications are still 32 bits.
Note that this means that 32-bit applications can be used unmodified on PAE systems, but can still each only use 4 GB of memory.
It is 32 bit only.Because,
PAE is a feature to allow 32-bit central processing units (CPUs) to access a physical address space (including random access memory and memory mapped devices) larger than 4 gigabytes.
see this http://en.wikipedia.org/wiki/Physical_Address_Extension
You can access the memory through a window (address range). Each time you have to use something outside that range, you should use a system call to map another range there. Consider using multiple heaps, with an offset within the window (pointer) - then the full pointer would be the heap identifier and a window offset (structure), totally 64 bits, each time you have to go outside the current heap, you have to switch them though.
char *ptr = (char*) malloc(40);
printf("%u",ptr);
56737856 (some output)
Now, if i am not incorrect the output that we see above is not a physical address but from the virtual address space. Am i correct?
Is there any way to see the actual physical address? Or vice versa (if my above assumption is wrong), and does all the internal implementation of malloc necessarily use the jemalloc algorithm?
All addresses you see in user space applications are virtual addresses.
Physical addresses are only of concern to the kernel. The mapping from virtual to physical addresses is complex, given that:
Not all virtual addresses have physical addresses at all. (For instance, pages that are unmapped, lazily zero-filled, or swapped out have no physical address.)
Physical addresses may change without warning (e.g, if a page is swapped out and back in, or if a shared page is copied).
Outside of some very unusual situations (mostly having to do with messing around with hardware), you should not care about physical addresses.
Yes, on a platform with virtual memory it is an address in process address space, i.e. it is an address in virtual memory. In such systems, at the typical application level the actual physical address in RAM makes no meaningful sense whatsoever - even if it is already known at that moment, it can change at any moment anyway. The physical RAM address is beyond your control. So, at the typical application level when people speak about "physical addresses", they normally refer to what you printed - the address in the process address space, i.e. the virtual address.
Just don't use %u to printf pointers. Use %p. Or at least convert your pointer to an appropriately sized unsigned integer type and use format specifier for that type.
Is there any way to see the actual physical address?
On an x86 architecture in real mode, there's no memory protection and you get back the actual physical address, so you can do moronic things such as overwriting the 0x0 address.
Here's code snippet from 'Memory Management: Algorithms and Implementations in C/C++' that can crash a DOS running computer:
void main()
{
unsigned char* ptr;
int i;
ptr = (unsigned char *)0x0;
for(i = 0; i < 1024; i++)
{
ptr[i]=0x0;
}
return;
}
If I may quote Wikipedia:
Real mode provides no support for memory
protection, multitasking, or code privilege levels. Before the release
of the 80286, which introduced Protected mode, real mode was the only
available mode for x86 CPUs. In the interests of backwards
compatibility, all x86 CPUs start in real mode when reset.