Following an hot comment thread in another question, I came to debate of what is and what is not defined in C99 standard about C arrays.
Basically when I define a 2D array like int a[5][5], does the standard C99 garantee or not that it will be a contiguous block of ints, can I cast it to (int *)a and be sure I will have a valid 1D array of 25 ints.
As I understand the standard the above property is implicit in the sizeof definition and in pointer arithmetic, but others seems to disagree and says casting to (int*) the above structure give an undefined behavior (even if they agree that all existing implementations actually allocate contiguous values).
More specifically, if we think an implementation that would instrument arrays to check array boundaries for all dimensions and return some kind of error when accessing 1D array, or does not give correct access to elements above 1st row. Could such implementation be standard compilant ? And in this case what parts of the C99 standard are relevant.
We should begin with inspecting what int a[5][5] really is. The types involved are:
int
array[5] of ints
array[5] of arrays
There is no array[25] of ints involved.
It is correct that the sizeof semantics imply that the array as a whole is contiguous. The array[5] of ints must have 5*sizeof(int), and recursively applied, a[5][5] must have 5*5*sizeof(int). There is no room for additional padding.
Additionally, the array as a whole must be working when given to memset, memmove or memcpy with the sizeof. It must also be possible to iterate over the whole array with a (char *). So a valid iteration is:
int a[5][5], i, *pi;
char *pc;
pc = (char *)(&a[0][0]);
for (i = 0; i < 25; i++)
{
pi = (int *)pc;
DoSomething(pi);
pc += sizeof(int);
}
Doing the same with an (int *) would be undefined behaviour, because, as said, there is no array[25] of int involved. Using a union as in Christoph's answer should be valid, too. But there is another point complicating this further, the equality operator:
6.5.9.6
Two pointers compare equal if and only if both are null pointers, both are pointers to the same object (including a pointer to an object and a subobject at its beginning) or function, both are pointers to one past the last element of the same array object, or one is a pointer to one past the end of one array object and the other is a pointer to the start of a different array object that happens to immediately follow the first array object in the address space. 91)
91) Two objects may be adjacent in memory because they are adjacent elements of a larger array or adjacent members of a structure with no padding between them, or because the implementation chose to place them so, even though they are unrelated. If prior invalid pointer operations (such as accesses outside array bounds) produced undefined behavior, subsequent comparisons also produce undefined behavior.
This means for this:
int a[5][5], *i1, *i2;
i1 = &a[0][0] + 5;
i2 = &a[1][0];
i1 compares as equal to i2. But when iterating over the array with an (int *), it is still undefined behaviour, because it is originally derived from the first subarray. It doesn't magically convert to a pointer into the second subarray.
Even when doing this
char *c = (char *)(&a[0][0]) + 5*sizeof(int);
int *i3 = (int *)c;
won't help. It compares equal to i1 and i2, but it isn't derived from any of the subarrays; it is a pointer to a single int or an array[1] of int at best.
I don't consider this a bug in the standard. It is the other way around: Allowing this would introduce a special case that violates either the type system for arrays or the rules for pointer arithmetic or both. It may be considered a missing definition, but not a bug.
So even if the memory layout for a[5][5] is identical to the layout of a[25], and the very same loop using a (char *) can be used to iterate over both, an implementation is allowed to blow up if one is used as the other. I don't know why it should or know any implementation that would, and maybe there is a single fact in the Standard not mentioned till now that makes it well defined behaviour. Until then, I would consider it to be undefined and stay on the safe side.
I've added some more comments to our original discussion.
sizeof semantics imply that int a[5][5] is contiguous, but visiting all 25 integers via incrementing a pointer like int *p = *a is undefined behaviour: pointer arithmetics is only defined as long as all pointers invoved lie within (or one element past the last element of) the same array, as eg &a[2][1] and &a[3][1] do not (see C99 section 6.5.6).
In principle, you can work around this by casting &a - which has type int (*)[5][5] - to int (*)[25]. This is legal according to 6.3.2.3 §7, as it doesn't violate any alignment requirements. The problem is that accessing the integers through this new pointer is illegal as it violates the aliasing rules in 6.5 §7. You can work around this by using a union for type punning (see footnote 82 in TC3):
int *p = ((union { int multi[5][5]; int flat[25]; } *)&a)->flat;
This is, as far as I can tell, standards compliant C99.
If the array is static, like your int a[5][5] array, it's guaranteed to be contiguous.
Related
Some code flattens multidimensional arrays like this:
int array[10][10];
int* flattened_array = (int*)array;
for (int i = 0; i < 10*10; ++i)
flattened_array[i] = 42;
This is, as far as I know, undefined behaviour.
I am trying to detect cases like this with gcc sanitizers, however, neither -fsanitize=address nor -fsanitize=undefined work.
Is there a sanitizer option that I'm missing, or perhaps a different way to detect this at run time? Or maybe I am mistaken and the code is legal?
Edit: the sanitizers detect this access as an error:
array[0][11] = 42;
but do not detect this:
int* first_element = array[0];
first_element[11] = 42;
Furthermore, clang detects the first access statically, and gives out a warning
warning: array index 11 is past the end of the array (which contains 10 elements) [-Warray-bounds]
Edit: the above does not change if int in the declaration is replaced with char.
Edit: There are two potential sources of UB.
Accessing an object (of type int[10]) through an lvalue of an incompatible type (int).
Out-of-bounds access with a pointer of type int* and an index >=10 where the size of the underlying array is 10 (rather than 100).
Sanitizers don't seem to detect the first kind of violation. There's a debate whether this is a violation at all. After all, there's also an object of type int at the same address.
As for the second potential UB, the UB sanitizer does detect such access, but only if it is done directly via the 2D array itself and not via another variable that points to its first element, as shown above. I don't think the two accesses should differ in legality. They should be either both legal (and then ubsan has a false positive) or both illegal (and then ubsan has a false negative).
Edit: Appendix J2 says array[0][11] should be UB, even though it is only informative.
From a language lawyer point of view, this is generally seen as invalid code because the integers arrays are only of size 10 and the code does access past the declared array size. Yet it used to be a common idiom, and I know no compiler that would not accept it. Still with all real world compilers I know, the resulting program will have the expected behaviour.
After a second (in reality much more) reading of the C11 standard draft (n1570) the intent of the standard is still not clear. 6.2.5 Types § 20 says:
An array type describes a contiguously allocated nonempty set of objects with a
particular member object type, called the element type.
It makes clear that an array contains contiguously allocated objects. But IMHO is unclear about whether a contiguously allocated set of objects is an array.
If you answer no, then the shown code does invoke UB by accessing an array past it last element
But if you answer yes, then a set of 10 contiguous sets of 10 contiguous integers gives 100 contiguous integers and can be seen as an array of 100 integers. Then the shown code would be legal.
That latter acception seems to be common in the real word because it is consistent with dynamic array allocation: you allocate enough memory for a number of objects, and you can access that as if it had been declared as an array - and the allocation function ensures no alignment problem.
My conclusion so far is:
is it nice and clean code: certainly not and I would avoid it in production code
does it invokes UB: I really do not know and my personal opinion is probably no
Let us look at the code added in the edit:
array[0][11] = 42;
The compiler knows that array is declared as int[10][10]. So it knows that both indexes must be less than 10, and it can raise a warning.
int* first_element = array[0];
first_element[11] = 42;
first_element is declared as a mere pointer. Statically, the compiler has to assume that it can point inside an array of unknown size, so outside of a specific context, it is much harder to raise a warning. Of course for a human programmer it is evident that both way should be seen the same, but as a compiler is not required to emit any diagnostic for out of bounds array, efforts to detect them are left to the minimum and only trivial cases are detected.
In addition, when a compiler internally codes pointer arithmetics on common platforms, it just computes a memory address which is the original address and a byte offset. So it could emit the same code as:
char *addr = (char *) first_element; // (1)
addr += 11 * sizeof(int); // (2)
*((int *) addr) = 42; // (3)
(1) is legal because a pointer to any objet (here an int) can be converter to a pointer to char, which is required to point to the first byte of the representation of the object
(2) the trick here is that (char *) first_element is the same as (char *) array because the first byte of the 10*10 array is the first byte of the first int of the first row, and an single byte can only have one single address. As the size of array is 10 * 10 * sizeof(int), 11 * sizeof(int) is a valid offset in it.
(3) for the very same reason, (char *) &array[1][1] is addr because elements in an array are contiguous so their byte representation are also contiguous. And as a forth and back conversion between 2 types is legal and required to give back the original pointer, (int *) addr is (int*) ((char*) &array[1][1]). That means that dereferencing (int *) addr is legal and shall have the same effect as array[1][1] = 42.
This does not mean that first_element[11] does not involve UB. array[0] has a declared size which is 10. It just explains why all known compilers accepts it (in addition to not wanting to break legacy code).
The sanitizers are not especially good at catching out-of-bounds access unless the array in question is a complete object.
For example, they do not catch out-of-bounds access in this case:
struct {
int inner[10];
char tail[sizeof(int)];
} outer;
int* p = outer.inner;
p[10] = 42;
which is clearly illegal. But they do catch access to p[11].
Array flattening is not really different in spirit from this kind of access. Code generated by the compiler, and the way it is instrumented by sanitizers, should be pretty similar. So there's little hope that array flattening can be detected by these tools.
Multidimensional arrays are required to be contiguously allocated (C uses row-major). And there can't be any padding between elements of an array - though not stated explicitly in the standard, this can be inferred with array definition that says "contiguously allocated nonempty set of objects" and the definition of sizeof operator.
So the "flattening" should be legal.
Re. accessing array[0][11]: although, Annex J2 directly gives an example, what exactly is the violation in the normative isn't obvious. Nevertheless, it's still possible to make it legal an intermediate cast to char*:
*((int*)((char*)array + 11 * sizeof(int))) = 42;
(writing such code is obviously not advised ;)
The problem here is that there Standard describes as equivalent two operations, one of which clearly should be defined and one of which the Standard expressly says is not defined.
The cleanest way to resolve this, which seems to coincide with what clang and gcc already do, which is to say that applying [] operator to an array lvalue or non-l value does not cause it to decay, but instead looks up an element directly, yielding an lvalue if the array operand was an lvalue, and a non-l value otherwise.
Recognizing the use of [] with an array as being a distinct operator would clean up a number of corner cases in the semantics, including accessing an array within a structure returned by a function, register-qualified arrays, arrays of bitfields, etc. It would also make clear what the inner-array-subscript limitations are supposed to mean. Given foo[x][y], a compiler would be entitled to assume that y would be within the bounds of the inner array, but given *(foo[x]+y) it would not be entitled to make such an assumption.
Here consider the following sample of code:
int *a = malloc(sizeof(int) * n);
Can this code be used to define an array a containing n integers?
int *a = malloc(sizeof(int) * n);
Can this code be used to define an array a containing n integers?
That depends on what you mean by "define an array".
A declaration like:
int arr[10];
defines a named array object. Your pointer declaration and initialization does not.
However, the malloc call (if it succeeds and returns a non-NULL result, and if n > 0) will create an anonymous array object at run time.
But it does not "define an array a". a is the name of a pointer object. Given that the malloc call succeeds, a will point to the initial element of an array object, but it is not itself an array.
Note that, since the array object is anonymous, there's nothing to which you can apply sizeof, and no way to retrieve the size of the array object from the pointer. If you need to know how big the array is, you'll need to keep track of it yourself.
(Some of the comments suggest that the malloc call allocates memory that can hold n integer objects, but not an array. If that were the case, then you wouldn't be able to access the elements of the created array object. See N1570 6.5.6p8 for the definition of pointer addition, and 7.22.3p1 for the description of how a malloc call can create an accessible array.)
int *a = malloc(sizeof(int) * n);
Assuming malloc() call succeeds, you can use the pointer a like an array using the array notation (e.g. a[0] = 5;). But a is not an array itself; it's just a pointer to an int (and it may be a block of memory which can store multiple ints).
Your comment
But I can use an array a in my program with no declaration otherwise
suggests this is what you are mainly asking about.
In C language,
p[i] == *(p + i) == *(i + p) == i[p]
as long as one of i or p is of pointer type (p can an array as well -- as it'd be converted into a pointer in any expression). Hence, you'd able to index a like you'd access an array. But a is actually a pointer.
Yes. That is exactly what malloc() does.
The important distinction is that
int array[10];
declares array as an array object with enough room for 10 integers. In contrast, the following:
int *pointer;
declares pointer as a single pointer object.
It is important to distiguinsh that one of them is a pointer and that the other as an actual array, and that arrays and pointers are closely related but are different things. However, saying that there is no array in the following is also incorrect:
pointer = malloc(sizeof (int) * 10);
Because what this piece of code does is precisely to allocate an array object with room for 10 integers. The pointer pointer contains the address of the first element of that array.(C99 draft, section 7.20.3 "Memory management functions")
Interpreting your question very literally, the answer is No: To "define an array" means something quite specific; an array definition looks something like:
int a[10];
Whereas what you have posted is a memory allocation. It allocates a space suitable for holding an array of 10 int values, and stores a pointer to the first element within this space - but it doesn't define an array; it allocates one.
With that said, you can use the array element access operator, [], in either case. For instance the following code snippets are legal:
int a[10];
for (int i = 0; i < 10; i++) a[i] = 0;
and
int *a = malloc(sizeof(int) * n);
for (int i = 0; i < n; i++) a[i] = 0;
There is a subtle difference between what they do however. The first defines an array, and sets all its elements to 0. The second allocates storage which can hold an equivalently-typed array value, and uses it for this purpose by initialising each element to 0.
It is worth pointing out that the second example does not check for an allocation error, which is generally considered bad practice. Also, it constitutes a potential memory leak if the allocated storage is not later freed.
In the language the Standard was written to describe (as distinct from the language that would be described by a pedantic literal reading of it), the intention was that malloc(n) would return a pointer that would, if cast to a T*, could be treated as a pointer to the first element of a T[n/sizeof T*]. Per N1570 7.22.3:
The
pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to
a pointer to any type of object with a fundamental alignment requirement and then used
to access such an object or an array of such objects in the space allocated (until the space
is explicitly deallocated).
The definition of pointer addition and subtraction, however, do not speak of acting upon pointers that are "suitably aligned" to allow access to arrays of objects, but rather speak of pointers to elements of actual array objects. If a program accesses space for 20 int objects, I don't think the Standard does actually says that the resulting pointer would behave in all respects as though it were a pointer to element [0] of an int[20], as distinct from e.g. a pointer to element [0][0] of an int[4][5]. An implementation would have to be really obtuse not to allow it to be used as either, of course, but I don't think the Standard actually requires such treatment.
While doing some research on multi-dimensional arrays in C and how they're stored in memory I came across this: "Does C99 guarantee that arrays are contiguous?". The top-voted answer states that "It must also be possible to iterate over the whole array with a (char *)," then provides the following "valid" code:
int a[5][5], i, *pi;
char *pc;
pc = (char *)(&a[0][0]);
for (i = 0; i < 25; i++)
{
pi = (int *)pc;
DoSomething(pi);
pc += sizeof(int);
}
The poster then goes on to say that "Doing the same with an (int *) would be undefined behavior, because, as said, there is no array[25] of int involved."
That line confuses me.
Why does using a char pointer constitute as valid / defined behavior while substituting it with an integer pointer doesn't?
Sorry if the answer to my question should be obvious. :(
The difference between using a char* and an int* is strict aliasing rules: If you access (&a[0][0])[6] (i. e. via an int*), the compiler is free to assumes that the access [6] does not leave the array at a[0]. As such, it is free to assumes that (&a[0][0]) + 6 and a[1] + 1 point to different memory locations, even though they don't, and reorder their accesses accordingly.
The char* is a difference because it is explicitly exempted from strict aliasing rules: You can cast anything to a char* and manipulate its bits through this pointer without invoking undefined behavior.
The standard is very clear that if you have:
int a[5];
int* p = &a[0];
Then
p += 6;
is cause for undefined behavior.
We also know that memory allocated for a 2D array such as
int a[5][5];
must be contiguous. Given that, if we use:
int* p1 = &a[0][0];
int* p2 = &a[1][0];
p1+5 is a legal expression and given the layout of a, it is equal to p2. Hence, if we use:
int* p3 = p1 + 6;
why should that not be equivalent to
int* p3 = p2 + 1;
If p2 + 1 is legal expression, why should p1 + 6 not be a legal expression?
From a purely pedantic interpretation of the standard, using p1 + 6 is cause for undefined behavior. However, it is possible that the standard does not adequately address the issue when it comes to 2D arrays.
In conclusion
From all practical points of view, there is no problem in using p1 + 6.
From a purely pedantic point of view, using p1 + 6 is undefined behavior.
Either an int pointer or a char pointer should work, but the operation should differ slightly in these two cases. Assuming sizeof(int) is 4. pc += sizeof(int) moves the pointer 4 bytes forward, but pi += sizeof(int) would move 4 times 4 bytes forward. If you want to use an int pointer, you should use pi ++.
EDIT: sorry about the answer above, using an int pointer does not comply with C99 (although it usually practically works). The reason is explained well in the original question: pointer goes across an array is not well defined in the standard. If you use an int pointer, you would start from a[0], which is a different array from a[1]. In this case, an a[0] int pointer cannot legally (well-defined) point to a[1] element.
SECOND EDIT: Using a char pointer is valid, because the following reason given by the original answer:
the array as a whole must be working when given to memset, memmove or memcpy with the sizeof. It must also be possible to iterate over the whole array with a (char *).
From section 6.5.6 "Additive Operators"
For the purposes of these operators, a pointer to an object that is not an element of an
array behaves the same as a pointer to the first element of an array of length one with the
type of the object as its element type.
So it is reasonable.
Sanity-check questions:
I did a bit of googling and discovered the correct way to return a one-dimensional integer array in C is
int * function(args);
If I did this, the function would return a pointer, right? And if the return value is r, I could find the nth element of the array by typing r[n]?
If I had the function return the number "3", would that be interpreted as a pointer to the address "3?"
Say my function was something like
int * function(int * a);
Would this be a legal function body?
int * b;
b = a;
return b;
Are we allowed to just assign arrays to other arrays like that?
If pointers and arrays are actually the same thing, can I just declare a pointer without specifying the size of the array? I feel like
int a[10];
conveys more information than
int * a;
but aren't they both ways of declaring an array? If I use the latter declaration, can I assign values to a[10000000]?
Main question:
How can I return a two-dimensional array in C? I don't think I could just return a pointer to the start of the array, because I don't know what dimensions the array has.
Thanks for all your help!
Yes
Yes but it would require a cast: return (int *)3;
Yes but you are not assigning an array to another array, you are assigning a pointer to a pointer.
Pointers and arrays are not the same thing. int a[10] reserves space for ten ints. int *a is an uninitialized variable pointing to who knows what. Accessing a[10000000] will most likely crash your program as you are trying to access memory you don't have access to or doesn't exist.
To return a 2d array return a pointer-to-pointer: int ** f() {}
Yes; array indexing is done in terms of pointer arithmetic: a[i] is defined as *(a + i); we find the address of the i'th element after a and dereference the result. So a could be declared as either a pointer or an array.
It would be interpreted as an address, yes (most likely an invalid address). You would need to cast the literal 3 as a pointer, because values of type int and int * are not compatible.
Yes, it would be legal. Pointless, but legal.
Pointers and arrays are not the same thing; in most circumstances, an expression of array type will be converted ("decay") to an expression of pointer type and its value will be the address of the first element of the array. Declaring a pointer by itself is not sufficient, because unless you initialize it to point to a block of memory (either the result of a malloc call or another array) its value will be indeterminate, and may not point to valid memory.
You really don't want to return arrays; remember that an array expression is converted to a pointer expression, so you're returning the address of the first element. However, when the function exits, that array no longer exists and the pointer value is no longer valid. It's better to pass the array you want to modify as an argument to the function, such as
void foo (int *a, size_t asize)
{
size_t i;
for (i = 0; i < asize; i++)
a[i] = some_value();
}
Pointers contain no metadata about the number of elements they point to, so you must pass that as a separate parameter.
For a 2D array, you'd do something like
void foo(size_t rows, size_t columns, int (*a)[columns])
{
size_t i, j;
for (i = 0; i < rows; i++)
for (j = 0; j < columns; j++)
a[i][j] = some_value;
}
This assumes you're using a C99 compiler or a C2011 compiler that supports variable length arrays; otherwise the number of columns must be a constant expression (i.e., known at compile time).
These answers certainly call for a bit more depth. The better you understand pointers, the less bad code you will write.
An array and a pointer are not the same, EXCEPT when they are. Off the top of my head:
int a[2][2] = { 1, 2, 3, 4 };
int (* p)[2] = a;
ASSERT (p[1][1] == a[1][1]);
Array "a" functions exactly the same way as pointer "p." And the compiler knows just as much from each, specifically an address, and how to calculate indexed addresses. But note that array a can't take on new values at run time, whereas p can. So the "pointer" aspect of a is gone by the time the program runs, and only the array is left. Conversely, p itself is only a pointer, it can point to anything or nothing at run time.
Note that the syntax for the pointer declaration is complicated. (That is why I came to stackoverflow in the first place today.) But the need is simple. You need to tell the compiler how to calculate addresses for elements past the first column. (I'm using "column" for the rightmost index.) In this case, we might assume it needs to increment the address ((2*1) + 1) to index [1][1].
However, there are a couple of more things the compiler knows (hopefully), that you might not.
The compiler knows two things: 1) whether the elements are stored sequentially in memory, and 2) whether there really are additional arrays of pointers, or just one pointer/address to the start of the array.
In general, a compile time array is stored sequentially, regardless of dimension(s), with no extra pointers. But to be sure, check the compiler documentation. Thus if the compiler allows you to index a[0][2] it is actually a[1][0], etc. A run time array is however you make it. You can make one dimensional arrays of whatever length you choose, and put their addresses into other arrays, also of whatever length you choose.
And, of course, one reason to muck with any of these is because you are choosing from using run time multiplies, or shifts, or pointer dereferences to index the array. If pointer dereferences are the cheapest, you might need to make arrays of pointers so there is no need to do arithmetic to calculate row addresses. One downside is it requires memory to store the addtional pointers. And note that if the column length is a power of two, the address can be calculated with a shift instead of a multiply. So this might be a good reason to pad the length up--and the compiler could, at least theoretically, do this without telling you! And it might depend on whether you select optimization for speed or space.
Any architecture that is described as "modern" and "powerful" probably does multiplies as fast as dereferences, and these issues go away completely--except for whether your code is correct.
Is subtraction of pointers not pointing to different elements of same array valid in C?
Is something such as below guaranteed to work according to C Standards? I vaguely remember reading that this is not valid?
int * a;
int * b;
a = (int*) 100;
b = (int*) 200;
printf("%d\n", b-a);
Will this give me 25.
From the C spec, Appendix J.2 Undefined behaviour:
Pointers that do not point into, or just beyond, the same array object are subtracted (6.5.6).
6.5.6 Additive operators, paragraph 9 says:
When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements.
There you have it - your example causes undefined behaviour. That said, on most systems it will work just fine. You probably do want to change your printf format to %td to indicate that you're printing a ptrdiff_t type.
This is undefined behavior.
For one, those pointers don't point to memory that you own.
You can only substract pointers that point inside the same array (or one position after the end of the array).
Of course, it will most likely work on most compilers, and you get 25 because sizeof(int) == 4 on your platform. If they were char *, you'd get 100. (possibly, or it could crash, that's the beauty of UB).
Even the standard does not promise a defined behavior, the result is correct.
An integer needs in your architecture 4 bytes. Thus, the difference give you the number of integer values the both pointer are apart.
You can use the difference as an index or offset in an array.
For the same reason
int *p = (int*) 100;
p++;
will result in p=104.
Of course it's undefined. Subtracting two arbitrary pointers (viewed as integers) is not even guaranteed to be a multiple of your object's size.