I got surprised when the following program did not crash.
typedef struct _x {
int a;
char b;
int c;
} x;
main() {
x *ptr = 0;
char *d = &ptr->b;
}
As per my understanding the -> operator has higher precedence over & operator. So I expected the program to crash at the below statement when we try to dereference the NULL pointer tr.
char *d = &ptr->b;
But the statement &ptr->b evaluates to a valid address. Could somebody please explain where I'm wrong?
Your expectations were unfounded. C programs don't necessarily "crash" when you dereference null pointers. C programs exhibit so called undefined behavior when you attempt to do something like that. Undefined behavior can manifest itself in many different ways. It can result in a crash. Or it can produce something that even resembles a "working" program. The latter is what apparently happened in your case.
But in any case, your program's behavior is undefined. And no, it does not produce a "valid address" as you seem to mistakingly believe. A numerical address that corresponds to a location in memory where no object exists is not valid (with the exception of null pointer value, of course).
The reason that your code doesn't crash is that you didn't actually dereference the pointer. Notice that the expression
&ptr->b
doesn't actually try loading the contents of ptr or ptr->b. Instead, it just stores the address of where this is located in memory. What you'll end up getting is a pointer to where the b field of the object pointed at by ptr should be. This will be a few bytes past address 0, so dereferencing the pointer you just created will cause a segfault.
&ptr->b == sizeof(int), that means the offset of b within _x after _x.a (which is of type int) relative to the address *((x*)0). The offset of 4 (typical for 32bit architecture) is saved within the d pointer. You have to access to d in order to get an seg-fault.
Computing an address does not require accessing memory. &ptr->b means "give me the address of the b field of the structure pointed to by ptr." Doing so does not require looking at whatever may be stored in that memory location.
It may be helpful to think about indexing an array instead of a structure. C defines ptr[5] as equivalent to *(ptr + 5) , which means that &(ptr[5]) is the same as &(*(ptr + 5)). Now it's easy to see that the & and the * "cancel out" and leave you with (ptr + 5), which involves only a pointer increment, and not a load from memory.
C makes this slightly cloudy because it distinguishes lvalues from rvalues. That is, an expression that refers to memory is treated differently on the left hand side of an expression than it is on the right. Given a statement like x = y;, a C compiler will load a value from the address of y, and store it in the address of x. This is the distinction: y is implicitly dereferenced, but x is not.
Related
I have seen that there are many questions related to this topic, but I could not infer an answer, so I decided to ask my first question here on stack overflow. Currently, my question is regarding the bytes and addresses, does each address actually represent one address, meaning that if I would initialize one address e.g. 0x55555555d156 but if I were to initialize an int, it would take 4 addresses, meaning that it will range from e.g. 0x55555555d156 to 0x55555555d160 ? So what confuses me is that, a pointer will hold an address, right?
Let's say the pointer holds the address e.g. 0x55555555d156 and if I were to deference that address I would get the value of that int, right? what about the other 3 addresses, if I deference them? I could not manage to acquire that information by writing a C program.
if I were to deference that address I would get the value of that int, right?
Yes.
what about the other 3 addresses, if I deference them?
If you have int *p = &some_integer;, then *(int *)((char *)p + 1) (dereferencing p "shifted" by one byte) would attempt to read 4 bytes from that new address and interpret them as an integer. Whether your program has permission to read that last byte that's right next to some_integer in memory, is another story: if it doesn't, you'll get a segmentation fault or other memory access issues.
Or you may get no errors and read garbage data.
Example
#include <stdio.h>
int main(void) {
int my_int = 0x12345678;
int *ptr = &my_int;
printf("%x\n", *ptr);
printf("%x\n", *(int *)((char *)ptr + 1));
}
Output:
~/test $ clang so.c && ./a.out
12345678
80123456
^^
|-- This "random" byte was read as part of
--- the "new" int shifted by 1 byte
Different microprocessors have different addressable units of memory. Most, including the x86 series and ARM, are addressable in units of one byte. So, for example, a 32-bit int will be stored in four consecutive memory addresses as you say (LSB first, unless the ARM is set to "Big Endian" mode).
Other processors, like PIC, may have one address point to a 16-bit memory word.
Your C code should probably not make assumptions either way, unless you're sure what the code will be run on.
You can't "dereference an address" -- you can dereference a pointer. A pointer value is an address, but the pointer also has a type. The result of the dereference depends on the pointer's type.
The result of derefencing a pointer is NOT the value stored at the memory location being pointed to. It is an expression that designates an object. This is known as an lvalue in C .
If all of this is unclear; first check that you understand what is happening in the code:
int x = 0;
x = 5;
In the second line, the use of x does not retrieve the value 0. The expression x is an lvalue which means that it designates a region of memory consisting of several bytes. Each byte has its own address. If you output &x you will likely see the same result as if you output the address of the first byte of x (although this is not a Standard requirement), but the types are different.
Whether or not the stored value is retrieved when an lvalue expression appears in the code, depends on the context of the expression. For example if it appears on the left-hand side of the assignment operator, the value is not retrieved.
Once you have understood x = 5; , then *p = 5; behaves identically; the meaning of *p is exactly as if the label x existed for the memory region that p points to.
I followed the discussion on One-byte-off pointer still valid in C?.
The gist of that discussion, as far as I could gather, was that if you have:
char *p = malloc(4);
Then it is OK to get pointers up to p+4 by using pointer arithmetic. If you get a pointer by using p+5, then the behavior is undefined.
I can see why dereferencing p+5 could cause undefined behavior. But undefined behavior using just pointer arithmetic?
Why would the arithmetic operators + and - not be valid operations? I don’t see any harm by adding or subtracting a number from a pointer. After all, a pointer is a represented by a number that captures the address of an object.
Of course, I was not in the standardization committee :) I am not privy to the discussions they had before codifying the standard. I am just curious. Any insight will be useful.
The simplest answer is that it is conceivable that a machine traps integer overflow. If that were the case, then any pointer arithmetic which wasn't confined to a single storage region might cause overflow, which would cause a trap, disrupting execution of the program. C shouldn't be obliged to check for possible overflow before attempting pointer arithmetic, so the standard allows a C implementation on such a machine to just allow the trap to happen, even if chaos ensues.
Another case is an architecture where memory is segmented, so that a pointer consists of a segment address (with implicit trailing 0s) and an offset. Any given object must fit in a single segment, which means that valid pointer arithmetic can work only on the offset. Again, overflowing the offset in the course of pointer arithmetic might produce random results, and the C implementation is under no obligation to check for that.
Finally, there may well be optimizations which the compiler can produce on the assumption that all pointer arithmetic is valid. As a simple motivating case:
if (iter - 1 < object.end()) {...}
Here the test can be omitted because it must be true for any pointer iter whose value is a valid position in (or just after) object. The UB for invalid pointer arithmetic means that the compiler is not under any obligation to attempt to prove that iter is valid (although it might need to ensure that it is based on a pointer into object), so it can just drop the comparison and proceed to generate unconditional code. Some compilers may do this sort of thing, so watch out :)
Here, by the way, is the important difference between unspecified behaviour and undefined behaviour. Comparing two pointers (of the same type) with == is defined regardless of whether they are pointers into the same object. In particular, if a and b are two different objects of the same type, end_a is a pointer to one-past-the-end of a and begin_b is a pointer to b, then
end_a == begin_b
is unspecified; it will be 1 if and only if b happens to be just after a in memory, and otherwise 0. Since you can't normally rely on knowing that (unless a and b are array elements of the same array), the comparison is normally meaningless; but it is not undefined behaviour and the compiler needs to arrange for either 0 or 1 to be produced (and moreover, for the same comparison to consistently have the same value, since you can rely on objects not moving around in memory.)
One case I can think of where the result of a + or - might give unexpected results is in the case of overflow or underflow.
The question you refer to points out that for p = malloc(4) you can do p+4 for comparison. One thing this needs to guarantee is that p+4 will not overflow. It doesn't guarantee that p+5 wont overflow.
That is to say that the + or - themselves wont cause any problems, but there is a chance, however small, that they will return a value that is unsuitable for comparison.
Performing basic +/- arithmetic on a pointer will not cause a problem. The order of pointer values is sequential: &p[0] < &p[1] < ... &p[n] for a type n objects long. But pointer arithmetic outside this range is not defined. &p[-1] may be less or greater than &p[0].
int *p = malloc(80 * sizeof *p);
int *q = p + 1000;
printf("p:%p q:%p\n", p, q);
Dereferencing pointers outside their range or even inside the memory range, but unaligned is a problem.
printf("*p:%d\n", *p); // OK
printf("*p:%d\n", p[79]); // OK
printf("*p:%d\n", p[80]); // Bad, but &p[80] will be greater than &p[79]
printf("*p:%d\n", p[-1]); // Bad, order of p, p[-1] is not defined
printf("*p:%d\n", p[81]); // Bad, order of p[80], p[81] is not defined
char *r = (char*) p;
printf("*p:%d\n", *((int*) (r + 1)) ); // Bad
printf("*p:%d\n", *q); // Bad
Q: Why is p[81] undefined behavior?
A: Example: memory runs 0 to N-1. char *p has the value N-81. p[0] to p[79] is well defined. p[80] is also well defined. p[81] would need to the value N to be consistent, but that overflows so p[81] may have the value 0, N or who knows.
A couple of things here, the reason p+4 would be valid in such a case is because iteration to one past the last position is valid.
p+5 would not be a problem theoretically, but according to me the problem will be when you will try to dereference (p+5) or maybe you will try to overwrite that address.
In the Following example, I expect that foo((&i)++) will evaluate to foo(4 + address of (i)) assuming that the int size is 4 Byte however it gives a compilation error at this line
anyone has an explanation ?
void foo(int*);
int main()
{
int i = 10;
foo((&i)++);
}
void foo(int *p)
{
printf("%d\n", *p);
}
The error message is "lvalue required for increment operator". The problem is that ++ needs to operate on a variable - you increment, AND STORE THE RESULT.
You cannot store the result of the increment operation in (&i).
To get foo to operate on an integer that is stored at the address you appear to want, you can do one of the following (I'm sure you can think of others):
foo(&i);
int *p = &i; foo(p++);
The second option will correctly call foo with a pointer to i, but will increment that pointer for the next time (which seems to be what you were trying to do with your code - except you had nowhere to put that value. By declaring a separate pointer p, I created that storage space. But realize that p is now pointing "nowhere" - if you access it again, you will get undefined behavior).
If you wanted to point to the next location after the address of i you would have to do
foo(++p);
but that would be undefined behavior (since there is no way of knowing what is stored in the next location after i; most likely it will be p but that is not guaranteed.)
Pointers. Powerful, dangerous, and slightly mysterious.
The operand of ++ must be an lvalue -- a variable or other location in memory that can be modified -- since it adds 1 to it and replaces it with the result. &i is not an lvalue, it's just an expression that yields the address of i.
Also,
foo(4 + address of (i))
is wrong since you're using postfix ++ rather than prefix ++. The value of EXPR++ is EXPR (with the side effect of changing the variable that expr refers to).
To get the value you want, just use
foo(&i + 1)
Note, however, that this is likely to result in undefined behavior ... depending on just what foo does with its argument.
You're incrementing and trying to store back the address of i. How and where does the result get stored back? It can't, because &i doesn't exist in memory.
I think you want to do this instead:
foo((&i)+1);
I was browsing through a webpage which had some c FAQ's, I found this statement made.
Similarly, if a has 10 elements and ip
points to a[3], you can't compute or
access ip + 10 or ip - 5. (There is
one special case: you can, in this
case, compute, but not access, a
pointer to the nonexistent element
just beyond the end of the array,
which in this case is &a[10].
I was confused by the statement
you can't compute ip + 10
I can understand accessing the element out of bounds is undefined, but computing!!!.
I wrote the following snippet which computes (let me know if this is what the website meant by computing) a pointer out-of-bounds.
#include <stdio.h>
int main()
{
int a[10], i;
int *p;
for (i = 0; i<10; i++)
a[i] = i;
p = &a[3];
printf("p = %p and p+10 = %p\n", p, p+10);
return 0;
}
$ ./a.out
p = 0xbfa53bbc and p+10 = 0xbfa53be4
We can see that p + 10 is pointing to 10 elements(40 bytes) past p. So what exactly does the statement made in the webpage mean. Did I mis-interpret something.
Even in K&R (A.7.7) this statement is made:
The result of the + operator is the
sum of the operands. A pointer to an
object in an array and a value of any
integral type may be added. ... The
sum is a pointer of the same type as
the original pointer, and points to
another object in the same array,
appropriately offset from the original
object. Thus if P is a pointer to an
object in an array, the expression P+1
is a pointer to the next object in the
array. If the sum pointer points
outside the bounds of the array,
except at the first location beyond
the high end, the result is
undefined.
What does being "undefined" mean. Does this mean the sum will be undefined, or does it only mean when we dereference it the behavior is undefined. Is the operation undefined even when we do not dereference it and just calculate the pointer to element out-of-bounds.
Undefined behavior means exactly that: absolutely anything could happen. It could succeed silently, it could fail silently, it could crash your program, it could blue screen your OS, or it could erase your hard drive. Some of these are not very likely, but all of them are permissible behaviors as far as the C language standard is concerned.
In this particular case, yes, the C standard is saying that even computing the address of a pointer outside of valid array bounds, without dereferencing it, is undefined behavior. The reason it says this is that there are some arcane systems where doing such a calculation could result in a fault of some sort. For example, you might have an array at the very end of addressable memory, and constructing a pointer beyond that would cause an overflow in a special address register which generates a trap or fault. The C standard wants to permit this behavior in order to be as portable as possible.
In reality, though, you'll find that constructing such an invalid address without dereferencing it has well-defined behavior on the vast majority of systems you'll come across in common usage. Creating an invalid memory address will have no ill effects unless you attempt to dereference it. But of course, it's better to avoid creating those invalid addresses so that your code will work perfectly even on those arcane systems.
The web page wording is confusing, but technically correct. The C99 language specification (section 6.5.6) discusses additive expressions, including pointer arithmetic. Subitem 8 specifically states that computing a pointer one past the end of an array shall not cause an overflow, but beyond that the behavior is undefined.
In a more practical sense, C compilers will generally let you get away with it, but what you do with the resulting value is up to you. If you try to dereference the resulting pointer to a value, as K&R states, the behavior is undefined.
Undefined, in programming terms, means "Don't do that." Basically, it means the specification that defines how the language works does not define an appropriate behavior in that situation. As a result, theoretically anything can happen. Generally all that happens is you have a silent or noisy (segfault) bug in your program, but many programmers like to joke about other possible results from causing undefined behavior, like deleting all of your files.
The behaviour would be undefined in the following case
int a[3];
(a + 10) ; // this is UB too as you are computing &a[10]
*(a+10) = 10; // Ewwww!!!!
Actually, memcpy works just fine when I use pointers to characters, but stops working when I use pointers to pointers to characters.
Can somebody please help me understand why memcpy fails here, or better yet, how I could have figured it out myself. I am finding it very difficult to understand the problems arising in my c/c++ code.
char *pc = "abcd";
char **ppc = &pc;
char **ppc2 = &pc;
setStaticAndDynamicPointers(ppc, ppc2);
char c;
c = (*ppc)[1];
assert(c == 'b'); // assertion doesn't fail.
memcpy(&c,&(*ppc[1]),1);
if(c!='b')
puts("memcpy didn't work."); // this gets printed out.
c = (*ppc2)[3];
assert(c=='d'); // assertion doesn't fail.
memcpy(&c, &(*ppc2[3]), 1);
if(c != 'd')
puts("memcpy didn't work again.");
memcpy(&c, pc, 1);
assert(c == 'a'); // assertion doesn't fail, even though used memcpy
void setStaticAndDynamicPointers(char **charIn, char **charIn2)
{
// sets the first arg to a pointer to static memory.
// sets the second arg to a pointer to dynamic memory.
char stat[5];
memcpy(stat, "abcd", 5);
*charIn = stat;
char *dyn = new char[5];
memcpy(dyn, "abcd", 5);
*charIn2 = dyn;
}
your comment implies that char stat[5] should be static, but it isn't. As a result charIn points to a block that is allocated on the stack, and when you return from the function, it is out of scope. Did you mean static char stat[5]?
char stat[5];
is a stack variable which goes out of scope, it's not // sets the first arg to a pointer to static memory.. You need to malloc/new some memory that gets the abcd put into it. Like you do for charIn2
Just like what Preet said, I don't think the problem is with memcpy. In your function "setStaticAndDynamicPointers", you are setting a pointer to an automatic variable created on the stack of that function call. By the time the function exits, the memory pointed to by "stat" variable will no longer exist. As a result, the first argument **charIn will point to something that's non-existent. Perhaps you can read in greater detail about stack frame (or activation record) here: link text
You have effectively created a dangling pointer to a stack variable in that code. If you want to test copying values into a stack var, make sure it's created in the caller function, not within the called function.
In addition to the definition of 'stat', the main problem in my eyes is that *ppc[3] is not the same as (*ppc)[3]. What you want is the latter (the fourth character from the string pointed to by ppc), but in your memcpy()s you use the former, the first character of the fourth string in the "string array" ppc (obviously ppc is not an array of char*, but you force the compiler to treat it as such).
When debugging such problems, I usually find it helpful to print the memory addresses and contents involved.
Note that the parenthesis in the expressions in your assignment statements are in different locations from the parenthesis in the memcpy expressions. So its not too suprising that they do different things.
When dealing with pointers, you have to keep the following two points firmly in the front of your mind:
#1 The pointer itself is separate from the data it points to. The pointer is just a number. The number tells us where, in memory, we can find the beginning of some other chunk of data. A pointer can be used to access the data it points to, but we can also manipulate the value of the pointer itself. When we increase (or decrease) the value of the pointer itself, we are moving the "destination" of the pointer forward (or backward) from the spot it originally pointed to. This brings us to the second point...
#2 Every pointer variable has a type that indicates what kind of data is being pointed to. A char * points to a char; a int * points to an int; and so on. A pointer can even point to another pointer (char **). The type is important, because when the compiler applies arithmetic operations to a pointer value, it automatically accounts for the size of the data type being pointed to. This allows us to deal with arrays using simple pointer arithmetic:
int *ip = {1,2,3,4};
assert( *ip == 1 ); // true
ip += 2; // adds 4-bytes to the original value of ip
// (2*sizeof(int)) => (2*2) => (4 bytes)
assert(*ip == 3); // true
This works because the array is just a list of identical elements (in this case ints), laid out sequentially in a single contiguous block of memory. The pointer starts out pointing to the first element in the array. Pointer arithmetic then allows us to advance the pointer through the array, element-by-element. This works for pointers of any type (except arithmetic is not allowed on void *).
In fact, this is exactly how the compiler translates the use of the array indexer operator []. It is literally shorthand for a pointer addition with a dereference operator.
assert( ip[2] == *(ip+2) ); // true
So, How is all this related to your question?
Here's your setup...
char *pc = "abcd";
char **ppc = &pc;
char **ppc2 = &pc;
for now, I've simplified by removing the call to setStaticAndDynamicPointers. (There's a problem in that function too—so please see #Nim's answer, and my comment there, for additional details about the function).
char c;
c = (*ppc)[1];
assert(c == 'b'); // assertion doesn't fail.
This works, because (*ppc) says "give me whatever ppc points to". That's the equivalent of, ppc[0]. It's all perfectly valid.
memcpy(&c,&(*ppc[1]),1);
if(c!='b')
puts("memcpy didn't work."); // this gets printed out.
The problematic part —as others have pointed out— is &(*ppc[1]), which taken literally means "give me a pointer to whatever ppc[1] points to."
First of all, let's simplify... operator precedence says that: &(*ppc[1]) is the same as &*ppc[1]. Then & and * are inverses and cancel each other out. So &(*ppc[1]) simplifies to ppc[1].
Now, given the above discussion, we're now equipped to understand why this doesn't work: In short, we're treating ppc as though it points to an array of pointers, when in fact it only points to a single pointer.
When the compiler encounters ppc[1], it applies the pointer arithmetic described above, and comes up with a pointer to the memory that immediately follows the variable pc -- whatever that memory may contain. (The behavior here is always undefined).
So the problem isn't with memcopy() at all. Your call to memcpy(&c,&(*ppc[1]),1) is dutifully copying 1-byte (as requested) from the memory that's pointed to by the bogus pointer ppc[1], and writing it into the character variable c.
As others have pointed out, you can fix this by moving your parenthesis around:
memcpy(&c,&((*ppc)[1]),1)
I hope the explanation was helpful. Good luck!
Although the previous answers raise valid points, I think the other thing you need to look at is your operator precedence rules when you memcpy:
memcpy(&c, &(*ppc2[3]), 1);
What happens here? It might not be what you're intending. The array notation takes higher precedence than the dereference operator, so you first attempt perform pointer arithmetic equivalent to ppc2++. You then dereference that value and pass the address into memcpy. This is not the same as (*ppc2)[1]. The result on my machine is an access violation error (XP/VS2005), but in general this is undefined behaviour. However, if you dereference the same way you did previously:
memcpy(&c, &((*ppc2)[3]), 1);
Then that access violation goes away and I get proper results.