As asked in "How does pointer incrementation work?", I have a follow-up question.
How does a pointer know the underlying size of the data it points to? Do pointers store a size of the underlying type so they can know how to increment?
I'd expect that the following code would move a pointer forward one byte:
int intarr[] = { ... };
int *intptr = intarr;
intptr = intptr + 1;
printf("intarr[1] = %d\n", *intptr);
According to the accepted answer on the linked site, having a pointer increment by bytes and not by the underlying sizeof the pointed element would cause mass hysteria, confusion, and chaos.
While I understand that this would probably be an inevitable outcome, I still don't understand how pointers work in this regard. Couldn't I declare a void pointer to some struct[] type array, and if I did so, how would the void pointer know to increment by sizeof(struct mytype)?
Edit: I believe that I've worked most of the difficulties out that I'm having, but I'm not quite there as far as demonstrating it in code.
See here: http://codepad.org/0d8veP4K
#include <stdio.h>
int main(int argc, char *argv[])
{
int intarr[] = { 0, 5, 10 };
int *intptr = intarr;
// get the value where the pointer points
printf("intptr(%p): %d\n", intptr, *intptr);
printf("intptr(%p): %d\n", intptr + 1, *(intptr + 1));
printf("intptr(%p): %d\n", intptr + 2, *(intptr + 2));
// the difference between the pointer value should be same as sizeof(int)
printf("intptr[0]: %p | intptr[1]: %p | difference: %d | expected: %d",
intptr, intptr + 1, (intptr + 1) - intptr, sizeof(int));
return 0;
}
It is in the type declaration. p1 knows the size of the type because it is sizeof(*p1) or sizeof(int). p2 does not know as sizeof(void) is not defined.
int *p1;
void *p2;
p1++; // OK
p2++; // Not defined behavior in C
Do pointers store a size of the underlying type so they can know how to increment?
This question suggests that type information needs to be kept with the object at runtime to make correct decisions on how to perform the correct operations for the type. That's not true. Type information becomes part of the code.
It may be easier to understand if we add a third type into the mix: floating point.
Consider this sample program:
int a,b,c;
float x,y,z;
void f(void)
{
c = a+b*3;
z = x+y*3;
}
(I ask you to think about the float vs. int case first not because it's simpler but because it's more complex. The extra complexity prevents you from taking shortcuts that are tempting but wrong.)
The compiler must translate f into some assembly code that performs two different kinds of addition and multiplication. Although the same operators (+ and *) appear twice in the C code, the assembly code won't look so symmetric. The first half will use the processor's integer registers, integer addition instruction, and integer multiplication instruction, and the second half will use floating point registers, floating point addition, and floating point multiplication. Even the constant 3 will be represented differently in the two places it appears.
At the assembly level, the memory where a, b, c, x, y, and z are stored doesn't need to be tagged because the type information is implicit in the instructions that access that memory. The loads and stores of the integer registers will only be targeted at the memory locations holding a, b, and c.
The C arithmetic operators are overloaded. When translating from a language with an overloaded operator to a language without a corresponding overloaded operator, the type information from the first language becomes part of the name of the operator in the second language. ("Name mangling" when translating from C++ to C is the same thing happening at another level. You could say that assembly language "ADD" (integer) and "FADD" (floating point) instructions are name-mangled + operators.)
Now, about pointer arithmetic. Pointers are just another type to overload. If the expression a=a+1 can generate two different varieties of assembly code depending on whether a is int or float, why not a third variety when a is int *, another when a is struct tm *, and so on?
In the C code, type information is contained in the variable declarations. In the compiler's intermediate representation, the type of every expression is known. In the compiler's output, the necessary pieces of type information are implicit in the machine instructions.
Kind of a crude answer, but it's worth noting at the machine level that data types, as we know them in C, don't exist. We might have arithmetical instructions that operate on integers stored in some general-purpose register, e.g., but there's nothing stored to identify that the contents of some register is actually an int. All the machine sees is a bunch of bits and bytes in various types of memory.
So you might even wonder how it's possible for a compiler to know how to do this:
int z = x + y;
How can it know to do an integer addition here if there's nothing stored when the program is running to identify that the memory regions storing the contents of x and y and z are ints?
And the short/crude answer is that the machine doesn't know once the program is running. Yet it had this information available when it generated the instructions that would be used to run the program.
It's the same case with pointers:
int intarr[] = { ... };
int *intptr = intarr;
Doing something like intptr + 1 here can be done to increment the pointer address by sizeof(int). The compiler knows to do this based on the information provided by you, the programmer, in this C code. If you did this instead:
int intarr[] = { ... };
void *voidptr = intarr;
... then trying to perform any arithmetic on voidptr would result in an error, since we aren't giving the information necessary for the compiler to know what machine instructions to generate.
Couldn't I declare a void pointer to some struct[] type array, and if
I did so, how would the void pointer know to increment by
sizeof(struct mytype)?
It can't. The void pointer would equate to a loss of compile-time information that would prevent the compiler from being able to generate the appropriate instructions. If you don't provide the info, the compiler doesn't know how to do the pointer arithmetic. And this is why functions which accept a void pointer like memcpy need a byte size to be specified. The pointee contents don't provide that kind of info, only the programmer can provide it since this kind of information is not stored in the memory used by the program at runtime.
in your example:
sizeof(pointer) is 4 bytes
sizeof(int) is 4 bytes
and in your program
Output:
intptr(0xffcbf5dc): 0
intptr(0xffcbf5e0): 5
intptr(0xffcbf5e4): 10
intptr[0]: 0xffcbf5dc | intptr[1]: 0xffcbf5e0 | difference: 1 | expected: 4
and if you try: 0xffcbf5e0 - 0xffcbf5dc = 4 (hex sub)and this is the sizeof(int).
about using void*: you can use the void*
about your structure :you can make sizeof(yourStructre)
Related
I understand that I can reassign a variable to a bigger type if it fits, ad its ok to do it. For example:
short s = 2;
int i = s;
long l = i;
long long ll = l;
When I try to do it with pointers it fails and I don't understand why. I have integers that I pass as arguments to functions expecting a pointer to a long long. And it hasn't failed, yet..
The other day I was going from short to int, and something weird happens, I hope someone can I explain it to me. This would be the minimal code to reproduce.
short s = 2;
int* ptr_i = &s; // here ptr_i is the pointer to s, ok , but *ptr_i is definitely not 2
When I try to do it with pointers it fails and I don't understand why.
A major purpose of the type system in C is to reduce programming mistakes. A default conversion may be disallowed or diagnosed because it is symptomatic of a mistake, not because the value cannot be converted.
In int *ptr_i = &s;, &s is the address of a short, typically a 16-bit integer. If ptr_i is set to point to the same memory and *ptr_i is used, it attempts to refer to an int at that address, typically a 32-bit integer. This is generally an error; loading a 32-bit integer from a place where there is a 16-bit integer, and we do not know what is beyond it, is not usually a desired operation. The C standard does not define the behavior when this is attempted.
In fact, there are multiple things that can go wrong with this:
As described above, using *ptr_i when we only know there is a short there may produce undesired results.
The short object may have alignment that is not suitable for an int, which can cause a problem either with the pointer conversion or with using the converted pointer.
The C standard does not define the result of converting short * to int * except that, if it is properly aligned for int, the result can be converted back to short * to produce a value equal to the original pointer.
Even if short and int are the same width, say 32 bits, and the alignment is good, the C standard has rules about aliasing that allow the compiler to assume that an int * never accesses an object that was defined as short. In consequence, optimization of your program may transform it in unexpected ways.
I have integers that I pass as arguments to functions expecting a pointer to a long long.
C does allow default conversions of integers to integers that are the same width or wider, because these are not usually mistakes.
I've been doing some pointers testing in C, and I was just curious if the addresses of a function's parameters are always in a difference of 4 bytes from one another.
I've tries to run the following code:
#include <stdio.h>
void func(long a, long b);
int main(void)
{
func(1, 2);
getchar();
return 0;
}
void func(long a, long b)
{
printf("%d\n", (int)&b - (int)&a);
}
This code seems to always print 4, no matter what is the type of func's parameters.
I was just wondering if it's ALWAYS 4, because if so it can be useful for something I'm trying to do (but if it isn't necessarily 4 I guess I could just use va_list for my function or something).
So: Is it necessarily 4 bytes?
Absolutely not, in so many ways that it would be hard to count them all.
First and foremost, the memory layout of arguments is simply not specified by the C language. Full stop. It is not specified. Thus the answer is "no" immediately.
va_list exists because there was a need to be able to navigate a list of varadic arguments because it wasn't specified other than that. va_list is intentionally very limited, so that it works on platforms where the shape of the stack does not match your intuition.
Other reasons it can't always be 4:
What if you pass an object of length 8?
What if the compiler optimizes a reference to actually point at the object in another frame?
What if the compiler adds padding, perhaps to align a 64-bit number on a 64-bit boundary?
What if the stack is built in the opposite direction (such that the difference would be -4 instead of +4)
The list goes on and on. C does not specify the relative addresses between arguments.
As the other answers correctly say:
No.
Furthermore, even trying to determine whether the addresses differ by 4 bytes, depending on how you do it, probably has undefined behavior, which means the C standard says nothing about what your program does.
void func(long a, long b)
{
printf("%d\n", (int)&b - (int)&a);
}
&a and &b are expression of type long*. Converting a pointer to int is legal, but the result is implementation-defined, and "If the result cannot be represented in the integer type, the behavior is undefined. The result need not be in the range of values of any integer type."
It's very likely that pointers are 64 bits and int is 32 bits, so the conversions could lose information.
Most likely the conversions will give you values of type int, but they don't necessarily have any meaning, nor does their difference.
Now you can subtract pointer values directly, with a result of the signed integer type ptrdiff_t (which, unlike int, is probably big enough to hold the result).
printf("%td\n", &b - &a);
But "When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements." Pointers to distinct object cannot be meaningfully compared or subtracted.
Having said all that, it's likely that the implementation you're using has a memory model that's reasonably straightforward, and that pointer values are in effect represented as indices into a monolithic memory space. Comparing &b vs. &a is not permitted by the C language, but examining the values can provide some insight about what's going on behind the curtain -- which can be especially useful if you're tracking down a bug.
Here's something you can do portably to examine the addresses:
printf("&a = %p\n", (void*)&a);
printf("&b = %p\n", (void*)&b);
The result you're seeing for the subtraction (4) suggests that type long is probably 4 bytes (32 bits) on your system. I'd guess you're on Windows. It also suggests something about the way function parameters are allocated -- something that you as a programmer should almost never have to care about, but is worth understanding anyway.
[...] I was just curious if the addresses of a function's parameters are always in a difference of 4 bytes from one another."
The greatest error in your reasoning is to think that the parameters exist in memory at all.
I am running this program on x86-64:
#include <stdio.h>
#include <stdint.h>
void func(long a, long b)
{
printf("%d\n", (int)((intptr_t)&b - (intptr_t)&a));
}
int main(void)
{
func(1, 2);
}
and compile it with gcc -O3 it prints 8, proving that your guess is absolutely wrong. Except... when I compile it without optimization it prints out -8.
X86-64 SYSV calling convention says that the arguments are passed in registers instead of being passed in memory. a and b do not have an address until you take their address with & - then the compiler is caught with its pants down from cheating the as-if rule and it quickly pulls up its pants and stuffs them into some memory location so that they can have their address taken, but it is in no way consistent in where they're stored.
As I understand it, all of the cases where C has to handle an address involve the use of a pointer. For example, the & operand creates a pointer to the program, instead of just giving the bare address as data (i.e it never gives the address without using a pointer first):
scanf("%d", &foo)
Or when using the & operand
int i; //a variable
int *p; //a variable that store adress
p = &i; //The & operator returns a pointer to its operand, and equals p to that pointer.
My question is: Is there a reason why C programs always have to use a pointer to manage addresses? Is there a case where C can handle a bare address (the numerical value of the address) on its own or with another method? Or is that completely impossible? (Being because of system architecture, memory allocation changing during and in each runtime, etc). And finally, would that be useful being that addresses change because of memory management? If that was the case, it would be a reason why pointers are always needed.
I'm trying to figure out if the use pointers is a must in C standardized languages. Not because I want to use something else, but because I want to know for sure that the only way to use addresses is with pointers, and just forget about everything else.
Edit: Since part of the question was answered in the comments of Eric Postpischil, Michał Marszałek, user3386109, Mike Holt and Gecko; I'll group those bits here: Yes, using bare adresses bear little to no use because of different factors (Pointers allow a number of operations, adresses may change each time the program is run, etc). As Michał Marszałek pointed out (No pun intended) scanf() uses a pointer because C can only work with copies, so a pointer is needed to change the variable used. i.e
int foo;
scanf("%d", foo) //Does nothing, since value can't be changed
scanf("%d", &foo) //Now foo can be changed, since we use it's address.
Finally, as Gecko mentioned, pointers are there to represent indirection, so that the compiler can make the difference between data and address.
John Bode covers most of those topics in it's answer, so I'll mark that one.
A pointer is an address (or, more properly, it’s an abstraction of an address). Pointers are how we deal with address values in C.
Outside of a few domains, a “bare address” value simply isn’t useful on its own. We’re less interested in the address than the object at that address. C requires us to use pointers in two situations:
When we want a function to write to a parameter
When we need to track dynamically allocated memory
In these cases, we don’t really care what the address value actually is; we just need it to access the object we’re interested in.
Yes, in the embedded world specific address values are meaningful. But you still use pointers to access those locations. Like I said above, a pointer is an address for our purposes.
C allows you to convert pointers to integers. The <stdint.h> header provides a uintptr_t type with the property that any pointer to void can be converted to uintptr_t and back, and the result will compare equal to the original pointer.
Per C 2018 6.3.2.3 6, the result of converting a pointer to an integer is implementation-defined. Non-normative note 69 says “The mapping functions for converting a pointer to an integer or an integer to a pointer are intended to be consistent with the addressing structure of the execution environment.”
Thus, on a machine where addresses are a simple numbering scheme, converting a pointer to a uintptr_t ought to give you the natural machine address, even though the standard does not require it. There are, however, environments where addresses are more complicated, and the result of converting a pointer to an integer may not be straightforward.
int i; //a variable
int *p; //a variable that store adres
i = 10; //now i is set to 10
p = &i; //now p is set to i address
*p = 20; //we set to 20 the given address
int tab[10]; // a table
p = tab; //set address
p++; //operate on address and move it to next element tab[1]
We can operate on address by pointers move forward or backwards. We can set and read from given address.
In C if we want get return values from functions we must use pointers. Or use return value from functions, but that way we can only get one value.
In C we don't have references therefore we must use pointers.
void fun(int j){
j = 10;
}
void fun2(int *j){
*j = 10;
}
int i;
i = 5; // now I is set to 5
fun(i);
//printf i will print 5
fun2(&i);
//printf I will print 10
This is taken from C, and is based on that.
Let's imagine we have a 32 bit pointer
char* charPointer;
It points into some place in memory that contains some data. It knows that increments of this pointer are in 1 byte, etc.
On the other hand,
int* intPointer;
also points into some place in memory and if we increase it it knows that it should go up by 4 bytes if we add 1 to it.
Question is, how are we able to address full 32 bits of addressable space (2^32) - 4 gigabytes with those pointers, if obviously they contain some information in them that allows them to be separated one from another, for example char* or int*, so this leaves us with not 32 bytes, but with less.
When typing this question I came to thinking, maybe it is all syntatic sugar and really for compiler? Maybe raw pointer is just 32 bit and it doesn't care of the type? Is it the case?
You might be confused by compile time versus run time.
During compilation, gcc (or any C compiler) knows the type of a pointer, in particular knows the type of the data pointed by that pointer variable. So gcccan emit the right machine code. So an increment of a int * variable (on a 32 bits machine having 32 bits int) is translated to an increment of 4 (bytes), while an increment of a char* variable is translated to an increment of 1.
During runtime, the compiled executable (it does not care or need gcc) is only dealing with machine pointers, usually addresses of bytes (or of the start of some word).
Types (in C programs) are not known during runtime.
Some other languages (Lisp, Python, Javascript, ....) require the types to be known at runtime. In recent C++ (but not C) some objects (those having virtual functions) may have RTTI.
It is indeed syntactic sugar. Consider the following code fragment:
int t[2];
int a = t[1];
The second line is equivalent to:
int a = *(t + 1); // pointer addition
which itself is equivalent to:
int a = *(int*)((char*)t + 1 * sizeof(int)); // integer addition
After the compiler has checked the types it drops the casts and works only with addresses, lengths and integer addition.
Yes. Raw pointer is 32 bits of data (or 16 or 64 bits, depending on architecture), and does not contain anything else. Whether it's int *, char *, struct sockaddr_in * is just information for compiler, to know what is the number to actually add when incrementing, and for the type it's going to have when you dereference it.
Your hypothesis is correct: to see how different kinds of pointer are handled, try running this program:
int main()
{
char * pc = 0;
int * pi = 0;
printf("%p\n", pc + 1);
printf("%p\n", pi + 1);
return 0;
}
You will note that adding one to a char* increased its numeric value by 1, while doing the same to the int* increased by 4 (which is the size of an int on my machine).
It's exactly as you say in the end - types in C are just a compile-time concept that tells to the compiler how to generate the code for the various operations you can perform on variables.
In the end pointers just boil down to the address they point to, the semantic information doesn't exist anymore once the code is compiled.
Incrementing an int* pointer is different from a incrementing char* solely because the pointer variable is declared as int*. You can cast an int* to char* and then it will increment with 1 byte.
So, yes, it is all just syntactic sugar. It makes some kinds of array processing easier and confuses void* users.
With a 32-bit OS, we know that the pointer size is 4 bytes, so sizeof(char*) is 4 and sizeof(int*) is 4, etc. We also know that when you increment a char*, the byte address (offset) changes by sizeof(char); when you increment an int*, the byte address changes by sizeof(int).
My question is:
How does the OS know how much to increment the byte address for sizeof(YourType)?
The compiler only knows how to increment a pointer of type YourType * if it knows the size of YourType, which is the case if and only if the complete definition of YourType is known to the compiler at this point.
For example, if we have:
struct YourType *a;
struct YourOtherType *b;
struct YourType {
int x;
char y;
};
Then you are allowed to do this:
a++;
but you are not allowed to do this:
b++;
..since struct YourType is a complete type, but struct YourOtherType is an incomplete type.
The error given by gcc for the line b++; is:
error: arithmetic on pointer to an incomplete type
The OS doesn't really have anything to do with that - it's the compiler's job (as #zneak mentioned).
The compiler knows because it just compiled that struct or class - the size is, in the struct case, pretty much the sum of the sizes of all the struct's contents.
It is primarily an issue for the C (or C++) compiler, and not primarily an issue for the OS per se.
The compiler knows its alignment rules for the basic types, and applies those rules to any type you create. It can therefore establish the alignment requirement and size of YourType, and it will ensure that it increments any YourType* variable by the correct value. The alignment rules vary by hardware (CPU), and the compiler is responsible for knowing which rules to apply.
One key point is that the size of YourType must be such that when you have an array:
YourType array[20];
then &array[1] == &array[0] + 1. The byte address of &array[1] must be incremented by sizeof(YourType), and (assuming YourType is a structure), each of the elements of array[1] must be properly aligned, just as the elements of array[0] must be properly aligned.
Also remember types are defined in your compiled code to match the hardware you are working on. It is entirely up to the source code that is used to work this out.
So a low end chipset 16 bit targeted C program might have need to define types differently to a 32 bit system.
The programming language and compiler are what govern your types. Not the OS or hardware.
Although of course trying to stick a 32 bit number into a 16 bit register could be a problem!
C pointers are typed, unlike some old languages like PL/1. This not only allows the size of the object to be known, but so widening operations and formatting can be carried out. For example getting the data at *p, is that a float, a double, or a char? The compiler needs to know (think divisions, for example).
Of course we do have a typeless pointer, a void *, which you cannot do any arithmetic with simply because the compiler has no idea how much to add to the address.