In the following code...
#include <stdlib.h>
#include <stdint.h>
extern void get_buffer_from_HW_driver(volatile uint32_t **p);
void getBuffer(volatile uint32_t **pp)
{
// Write an address into pp, that is obtained from a driver
// The underlying HW will be DMA-ing into this address,
// so the data pointed-to by the pointer returned by this
// call are volatile.
get_buffer_from_HW_driver(pp);
}
void work()
{
uint32_t *p = NULL;
getBuffer((volatile uint32_t **)&p);
}
...the compiler rightfully detects that any potential accesses to the data pointed to by p inside work are dangerous accesses. As-is, the code instructs the compiler that it is safe to emit code that optimizes away repeated read accesses to *p - which is indeed wrong.
But the weird thing is, that the warning emitted by compiling this code...
$ gcc -c -Wall -Wextra -Wcast-qual constqual.c
...doesn't complain about the loss of volatile - it instead recommends using const:
constqual.c: In function ‘work’:
constqual.c:20:15: warning: to be safe all intermediate pointers in cast from
‘uint32_t ** {aka unsigned int **}’ to ‘volatile uint32_t **
{aka volatile unsigned int **}’ must be ‘const’ qualified
[-Wcast-qual]
getBuffer((volatile uint32_t **)&p);
^
I cannot see how const makes sense here.
P.S. Note that adding volatile in front of the uint32_t *p, as expected, fixes the issue. My question is why GCC recommends const instead of volatile.
Well, I raised a ticket in GCC's Bugzilla about this... and Joseph Myers has answered with a laconic answer:
No, GCC is not confused. It's saying that it's type-safe to convert
uint32_t ** to volatile uint32_t *const *, but not to convert it to
volatile uint32_t *.
...and he also added a reference to this part of the C FAQ.
I have to admit that my first reaction to this was a "say what?". I quickly tested the suggestion, changing the code to make it use the proposed declaration (and cast) instead...
#include <stdlib.h>
#include <stdint.h>
extern void get_buffer_from_HW_driver(volatile uint32_t * const *p);
void getBuffer(volatile uint32_t * const *pp)
{
// Write an address into pp, that is obtained from a driver
// The underlying HW will be DMA-ing into this address,
// so the data pointed-to by the pointer returned by this
// call are volatile.
get_buffer_from_HW_driver(pp);
}
void work()
{
uint32_t *p = NULL;
getBuffer((volatile uint32_t * const *)&p);
}
$ gcc -c -Wall -Wextra -Wcast-qual constqual.c
$
...and indeed, no warning anymore.
So I went ahead and read the relevant FAQ - and I think I understand a bit more of what is happening. By adding the const modifier, the parameter we are passing is (reading from right to left, as we're supposed to do in this kind of C syntax):
a pointer to a constant pointer (that will never change) that points to volatile data
This indeed maps very well to what is happening here: I am getting a pointer that points to volatile data, that is a driver-provided buffer - i.e. one that I indeed am not allowed to change, since it comes from pre-allocated lists of buffers that the driver itself allocated. Modifying the pointer that get_buffer_from_HW_driver returned would make no sense; it's not mine to modify, I can only use it as-is.
I confess I am really surprised that C's typesystem (augmented with the really strong static-analysis checks of -Wcast-qual) can actually help in guaranteeing these semantics.
Many thanks to Joseph - and I'll leave the question open for a few weeks, in case someone else wants to elaborate more.
P.S. Adding a mental note: from now on, when anyone claims that C is a simple language, I think I'll point them here.
Related
struct details_state {
struct details_status D1;
struct details_status D2;
};
struct details {
struct details_state details_states[2];
} __attribute__((packed));
struct details *p;
void print_details_status(struct details_status *s)
print_details_status(&(p->details_states[i].D1));
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
warning: taking address of packed member of 'struct details' may result in an unaligned pointer value [-Waddress-of-packed-member]
GCC gives this warning with >9 version. How to get rid of this warning without using [-Wno-address-of-packed-member]
In the comments to your post you have the explanation of the why it warns and why it can be a bad idea to take a pointer of a non-aligned member. The issue is not so much about assigning an unaligned value to a pointer (which seems to be permitted by bullet 5 of 6.3.2.3 here, but only about dereferencing it. Dereferencing it seems to be even explicitly permitted on some architectures, albeit with a performance penalty (e.g.: on x86 when not using SSE load/store), but it may cause severe issues depending on type and generated assembly on other architectures, typically a BUS ERROR if you are in user space or a fault if you are in kernel space.
For bad that it may be to do something like that, there is code around that takes pointers of packed struct's elements
and even dereferences them without shame:
https://github.com/abperiasamy/rtl8812AU_8821AU_linux/blob/4d1726146fd96552c9fa5af05c75187027d6885b/core/rtw_cmd.c#L1481
https://github.com/abperiasamy/rtl8812AU_8821AU_linux/blob/4d1726146fd96552c9fa5af05c75187027d6885b/core/rtw_mlme.c#L3689
I can imagine that someone may be in a situation where they are stuck with someone else's code with a packed struct (note that it would be bad to have a packed struct in a library's header file, as packing is a non-standard extension to the language and its results are compiler dependent). Or maybe they have a function that is guaranteed to deal properly with an unaligned pointer; as the C standard provides no way to communicate that in the code, one would have to rely on the function's documentation (and/or implementation) to figure that.
Now that you understand what's wrong with it but also that it can sometimes be useful to suppress that warning locally without setting a global compiler option; here's an answer to:
How to get rid of this warning without using [-Wno-address-of-packed-member]
You can use the base address to the struct and add to it the relevant offset, obtained via offsetof(). This requires some casting and, even turned into a macro, it is not particularly clean as it requires several arguments:
#define member_nowarn(structType, elementType, structPtr, memberName) \
((elementType*)(((char*)(structPtr)) + offsetof(structType, memberName)))
This can be simplified so you don't have to pass the elementType, but then the return value of the macro will be a void* which means compiler warnings may get suppressed when you mistakenly pass it to a function that expects a pointer to something else than the type of the memberName element, so I wouldn't recommend this:
#define member_nowarn_unsafe(structType, structPtr, memberName) \
((void*)((((char*)(structPtr)) + offsetof(structType, memberName))))
If your compiler supports the non-standard typeof() (e.g.: this is provided by stddef.h in gcc), you can simplify the macro so that no type has to be passed, the returning pointer's type is still correct and the code using it becomes simpler and less error-prone:
#define member_nowarn_nostd(structPtr, memberName) \
((typeof((structPtr)->memberName)*)(((char*)(structPtr)) + offsetof(typeof(*(structPtr)), memberName)))
Example code:
#include <stdio.h>
#include <stddef.h>
#include <string.h>
typedef float MYTYPE;
struct details {
char mychar;
MYTYPE mymember;
} __attribute__((packed));
struct details d;
void myfunc(MYTYPE*);
#define member_nowarn(structType, elementType, structPtr, memberName) \
((elementType*)(((char*)(structPtr)) + offsetof(structType, memberName)))
#define member_nowarn_unsafe(structType, structPtr, memberName) \
((void*)((((char*)(structPtr)) + offsetof(structType, memberName))))
#define member_nowarn_nostd(structPtr, memberName) \
((typeof((structPtr)->memberName)*)(((char*)(structPtr)) + offsetof(typeof(*(structPtr)), memberName)))
int main()
{
d.mymember = 123;
// warns
myfunc(&d.mymember);
//warns
myfunc((MYTYPE*)&d.mymember);
// ugly, but it does't warn
myfunc(((MYTYPE*)((char*)(&d) + offsetof(struct details, mymember))));
// same, turned into a macro
myfunc(member_nowarn(struct details, MYTYPE, &d, mymember));
// simpler, but the return value of the macro is void* so you won't get a
// warning when passing to a function that requires a different type of pointer
myfunc(member_nowarn_unsafe(struct details, &d, mymember));
// simpler macro, but uses non-standard typeof()
myfunc(member_nowarn_nostd(&d, mymember));
return 0;
}
void myfunc(MYTYPE* arg)
{
MYTYPE val;
// do not dereference the pointer: it may crash
// depending on MYTYPE, architecture and the generated
// assembly.
// val = *arg; // don't do this
// instead use memcpy to get the data it points to
memcpy(&val, arg, sizeof(val));
printf("%p %f\n", arg, (double)val);
}
EDIT: problem explained more in depth here (thank you #Eric Postpischil). It seems to be a bug in GCC.
First, let me start with some context: the code I'm writing is using an API I can't change, on a GCC version I can't change, with compilation flags I'm not allowed to remove, and when I'm done with it must have precisely zero warnings or #pragmas.
EDIT: no unions either.
EDIT2: assume the build system also uses -Wall -ansi -pedantic and every other warnings under the sun.
I'll confirm the GCC version tomorrow but I'm fairly certain it's not above GCC 7. In the meantime I'm testing with GCC 6.3.
EDIT3: I'm marking the issue as 'answered'. For completeness' sake, I'm adding some more information below:
I've checked the compiler version being used, and it's not pretty. We're using Mingw and a gcc.exe --version tells me it's GCC 3.4.5.
Furthermore, compilation flags include wall wextra wcast-qual wpointer-arith wconversion wsign-conversion along with others that are not relevant to the problem at hand.
The problem
Consider the following code:
#include "stdio.h"
#include "stdint.h"
typedef uint32_t MyType[4];
const MyType* foo(const uint8_t* a)
{
return (const MyType*) a;
}
void myapi_foo(const MyType* d) {}
int main()
{
uint8_t a[4*sizeof(uint32_t)];
const MyType* b = foo((const uint8_t*) a);
myapi_foo(b);
return 0;
}
Compiled with GCC and the -Wcast-qual flag, this code will throw the following warning:
warning: cast discards ‘const’ qualifier from pointer target type [-Wcast-qual]
return (const MyType*) a;
EDIT: to clarify, the error is on this line:
return (const MyType*) a;
The cause of the problem
I know the root cause of the problem is the typedef type MyType which is in fact an array. Sadly, I do not have the luxury of modifying this typedef, nor the API function myapi_foo and its dubious choice of parameter type.
To be honest, I don't really understand why is the compiler so unhappy about this cast, so clarifications are more than welcome.
The question
What would be the cleanest way of indicating to the compiler everything should be treated as a pointer to const data?
Discarded and potential solutions
Here are a few 'solutions' that I have found but left me unsatisfied:
Remove the -Wcast-qual flag. I cannot do that due to code quality rules.
Add a #pragma to turn off the warning around that part of the code (as shown here). Similarly I'm not allowed to do that.
Cast the pointer to an integer, then cast back to a pointer (as shown here) return (const MyType*) (uint32_t) a;. It's very crude, but using uint32_t as memory addresses has precedent in this project so I might have to use it as a last ditch effort.
EDIT: #bruno suggested using an union to side-step the problem. This is a portable and fairly elegant solution. However, the aforementioned code quality rules downright bans the use of unions.
EDIT: #Eric Postpischil and #M.M suggested using a (const void*) cast return (const void*) a;, which would work regardless of the value of sizeof(MyType*). Sadly it doesn't work on the target.
Thank you for your time.
This is GCC bug 81631. GCC fails to recognize the cast to const MyType * retains the const qualifier. This may be because, in this “pointer to array of four const uint32_t”, GCC performs a test of whether the array is const whether than of whether the array elements are const.
In some GCC versions, including 8.2, a workaround is to change:
return (const MyType*) a;
to:
return (const void *) a;
A more drastic change that is likely to work in more versions is to use:
return (const MyType *) (uintptr_t) a;
Note About Conversion and Aliasing:
It may be a problem that this code passes a to a function that casts it to const MyType *:
uint8_t a[4*sizeof(uint32_t)];
const MyType* b = foo((const uint8_t*) a);
In many C implementations, MyType, being an array of uint32_t, will require four-byte alignment, but a will only require one-byte alignment. Per C 2018 6.3.2.3 6, if a is not correctly aligned for MyType, the result of the conversion is not defined.
Additionally, this code suggests that the uint_t array a may be used as an array of four uint32_t. That would violate C aliasing rules. The code you show in the question appear to be a sample, not the actual code, so we cannot be sure, but you should consider this.
You can do that :
const MyType* foo(const uint8_t* a)
{
union {
const uint8_t* a;
const MyType* b;
} v;
v.a = a;
return v.b;
}
w.c being your modified file :
pi#raspberrypi:/tmp $ gcc -pedantic -Wall -Wcast-qual w.c
pi#raspberrypi:/tmp $
That works whatever the compiler (no #pragma) or the respective size of int and pointer(no cast between int and pointer), but I am not sure this is very elegant ;-)
It is strange to have that foo function and at the same time compile with Wcast-qual, it's contradictory
Edit, If you cannot use union you can also do that
const MyType* foo(const uint8_t* a)
{
const MyType* r;
memcpy(&r, &a, sizeof(a));
return r;
}
Compilation :
pi#raspberrypi:/tmp $ gcc -pedantic -Wall -Wcast-qual w.c
pi#raspberrypi:/tmp $
If nothing works, you might like to use the uintptr_t hammer, if the implmentation provides it. It is optional by the C11 Standard:
const MyType* foo(const uint8_t* a)
{
uintptr_t uip = (uintptr_t) a;
return (const MyType*) uip;
}
Some C compilers can sometimes deduce a casted pointer as still a lvalue, but gcc defaults won't compile it. Instead of having to port (by human-error-prone way)legacy code base to:
p=(int *)p+1 ; /* increment by sizeof(int) */
can gcc be made to allow this code below (even if not technically correct)?
void f() {
void *p ; /* type to be cast later for size */
((int *)p)++ ; /* gcc -c f.c => lvalue required error */ }
Edit: even if technically incorrect, I assume the only programmer intent for such code is for p to remain lvalue and "increment" it, generating same code as my long form, right? (perreal flagged as "lvalue cast")
Edit2: We all agree refactoring to std C is best, but if something like Mateo's -fpermissive worked, it might not catch future programming errors, but hoping initial gcc porting effort will be less faulty... Any other similar suggestions?
If you want to clean up your code(remove excessive casts) , you can concentrate the ugly casts inside a macro or, even better, an inlined function.
The below fragment only makes use of implicit casts to/from void*
You are ,of course, still responsible for the proper alignment.
#include <stdio.h>
static void *increment(void * p,size_t offset)
{
char *tmp = p;
return tmp+offset;
}
int main(void)
{
void *ptr = "Hell0 world!\n";
ptr = increment( ptr, sizeof(int) );
printf("%s", (char*) ptr);
return 0;
}
Consider the following structs:
typedef struct {
uint32_t foo;
uint32_t bar;
} first_struct_t;
typedef struct {
first_struct_t f;
uint8_t *p;
uint8_t buf[];
} second_struct_t;
However, later in my code, the following assignment happens:
typedef struct {
first_struct_t *f;
// ...
} some_struct;
int function_before_foo(some_struct *p) {
p->f = (second_struct_t *) malloc(sizeof(second_struct_t));
// ...
}
int function_foo(some_struct *p) {
second_struct_t *s = (second_struct_t *) p->f;
// ...
}
And it generates the following error due to -Wcast-align:
'second_struct_t *' increases required alignment from 4 to 8 [-Werror,-Wcast-align]
second_struct_t *s = (second_struct_t *) p->f;
A solution is to cast it to a void *, but that seems to only mask the problem. What is the cleanest solution to this?
EDIT: This only happens with clang and not GCC, regardless of the fact that both compilers are given the -Wcast-align flag.
I'm going to assume when you have struct_second_t you actually meant second_struct_t, otherwise your code does not compile.
The problem is that f in some_struct is a pointer to a first_struct_t not a pointer to a second_struct_t.
I should add that the cast in struct_second_t *s = (struct_second_t *) p->f; is hiding the warning message. In general if you have to cast one pointer to another you are more often than not going to cause undefined behavior. This is not always true but is a pretty good guideline.
In response to comment.
First it appears you will not get that warning with gcc for x86 (32 and 64 bit) as there is no alignment requirement for general purpose registers although alignment can improve performance (see this SO post for more info). As to why clang emits that warning, perhaps because of performance or they do not have the exception as gcc does for x86.
Second what you are trying to accomplish is similar to what the container_of macro does in the Linux kernel. container_of is typically defined as:
#define container_of(ptr, type, member) ({ \
const typeof( ((type *)0)->member ) *__mptr = (ptr); \
(type *)( (char *)__mptr - offsetof(type,member) );})
In searching the only thing I have found that addresses your issue is this commit to tup that changed their version of container_of to include a cast to void*.
Basically, I believe the issue is while you know that p->f actually points to a second_struct_t your compiler does not and therefore issues the warning. So you can either not do that or cast to void*.
Additional:
It seems mozilla address the issue by casting to void* as well:
bug report
patch
Finally, I would recommend looking at container_of and using that instead of relying on the fact that the first element of the struct is the other struct. This way your code is more resilient, if someone else changes the order of the struct members it will still work. You will have to add a void* cast to container_of to avoid the warning. Note that on architectures with alignment issues, you should not have an issue at runtime assuming you always do your type change between child and parent correctly.
I've used this interesting approach to make opaque and private structs:
struct s_vector_private
{
size_t item_size;
uint32_t used_slots;
uint32_t buffer_total_slots;
uint8_t * buffer;
};
typedef uint8_t vector_t[sizeof(struct s_vector_private)];
/* This code can be found at [1] */
This way, vector_t contents are private and vector_t can be allocated on heap and stack.
Recently I've wondered if allocating vector_t, either in stack or heap, can cause alignment problems since compiler sees it is an array instead of a struct.
For further enlightenment, there are big libraries that use this kind of approach, like zmq declaration of the zmq_msg_t type:
typedef struct zmq_msg_t {unsigned char _ [40];} zmq_msg_t;
/* This code can be found at [2] */
In the zmq example, is it possible to get misaligned structures as well? I couldn't avoid noting that zmq_msg_t has declared the array inside of a struct, would this be a safe way to implement the technique in question here?
[1] http://github.com/felipe-lavratti/chelper/blob/master/include/chelper/helper_types.h, line 48.
[2] http://github.com/zeromq/libzmq/blob/master/include/zmq.h, line 202.
It is necessary to use either C11 or compiler extensions to align vector_t. For example, for a truly opaque type with flexible size and alignment, that means a declarator like this:
#define stack_alloc_opaque(identifier) \
alignas(alignof(max_align_t)) char (identifier)[sizeof_opaque_type()]
Where sizeof_opaque_type() is a function returning the true size at runtime.
If you don't need the full flexibility of an opaque type you must at least align by alignof(struct s_vector_private), but this limits your ability to alter the type behind the scenes.
While you could probably get away with the alignment mismatch on x86, it is undefined behavior to cast between pointers with different alignment requirements, and accessing such an object is definitely not portable.
Speaking of which, there's an elephant in the room here.
Doing the above will make the types perfectly compatible at the byte level, but it's skirting strict aliasing rules. The exception that that allows char* to alias any other type is one-way only -- vector_t being declared as uint8_t[] (or even char[]) makes that its effective type, so it can't be punned to something else. That leaves you with a few options:
Use a compiler extension like __attribute__((may_alias)) on s_vector_private
Disable strict aliasing optimizations altogether (-fno-strict-aliasing)
Leave them on, but test carefully with awareness that problems may occur in the future or in other configurations
Don't use opaque types on the stack
Unfortunately the only really portable and safe one is the last. I like this approach too, but I don't know of any other way around that particular issue.
The technique won't cause problems with heap allocations. The pointer returned by malloc() et al is aligned sufficiently for any use — the malloc() function cannot tell whether it is allocating a 40-byte string that could be on any byte boundary or an array of a type that needs to be aligned on an 8-byte boundary, so it takes the conservative view that it is always allocating for the most stringent alignment that the implementation requires.
The technique could in theory cause problems with stack allocation. If you have:
char c1;
vector_t v1;
char c2;
then you could have v1 misaligned. It probably won't be a problem, but if you wish to be safe, you avoid the possibility. However, GCC 4.8.2 running on Mac OS X 10.9.1 seems to go to some lengths to avoid trouble:
$ gcc -O3 -g -std=c11 -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Werror alx.c -o alx
$ alx
24
&c1 = 0x7fff54b9552d
&v1 = 0x7fff54b95530
&c2 = 0x7fff54b9552e
&p1 = 0x7fff54b95550
&c3 = 0x7fff54b9552f
$ gcc -g -std=c11 -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Werror alx.c -o alx
$ alx
24
&c1 = 0x7fff579a756f
&v1 = 0x7fff579a7550
&c2 = 0x7fff579a754f
&p1 = 0x7fff579a7530
&c3 = 0x7fff579a752f
$
#include <stdint.h>
#include <stdio.h>
struct s_vector_private
{
size_t item_size;
uint32_t used_slots;
uint32_t buffer_total_slots;
uint8_t *buffer;
};
typedef uint8_t vector_t[sizeof(struct s_vector_private)];
int main(void)
{
char c1;
vector_t v1;
char c2;
struct s_vector_private p1;
char c3;
printf("%zu\n", sizeof(vector_t));
printf("&c1 = %p\n", (void *)&c1);
printf("&v1 = %p\n", (void *)&v1);
printf("&c2 = %p\n", (void *)&c2);
printf("&p1 = %p\n", (void *)&p1);
printf("&c3 = %p\n", (void *)&c3);
return 0;
}
Also, the technique is a trifle odd. It hasn't really hidden the underlying type (the code can still see it); it just isn't called vector_t.
Posix c library uses the zmq mentioned technique to hide struct contents. To deal with alignment they do a union with the array. See ptread_mutexattr_t type below.
typedef union
{
char __size[__SIZEOF_PTHREAD_MUTEXATTR_T];
int __align;
} pthread_mutexattr_t;
This is an excellent approach to hide struct members, but there is one main drawback that may be significant if your code is not a library with slow releases. the size of the array has got to be a typed constant:
#ifdef __x86_64__
# if __WORDSIZE == 64
# define __SIZEOF_PTHREAD_ATTR_T 56
# define __SIZEOF_PTHREAD_MUTEX_T 40
# define __SIZEOF_PTHREAD_MUTEXATTR_T 4
...
This is a considerable drawback in rapid changing modules, or code under development, since each time the struct internals are changed, this number must be manually update. Forgetting to do it will lead to undefined behaviour hard to track down.
There is not a problem here. To understand why, you need to see how sizeof works with structs that do not have __attribute__ ((__packed__)) or explicit alignment.
Try the following program:
#include <stdio.h>
#include <stdlib.h>
struct st
{
void *p;
char c;
};
struct st arr[5];
int
main (int argc, char **argv)
{
printf ("Size of struct: %lu\nSize of array: %lu\n",
sizeof (struct st),
sizeof (arr));
exit (0);
}
The output is as follows (on x86_64):
Size of struct: 16
Size of array: 80
This marginally surprising result is because while struct st appears to need only 9 bytes (8 for the pointer, and one for the char), it actually takes 16 for precisely the reason that an array of them must have proper alignment. sizeof thus includes the padding bytes. Hence 5 of them take 80 bytes, and each of the pointers is correctly aligned.
Even within the structure, without __attribute__ ((__packed__)), padding bytes will be inserted to align the types properly (preventing this is precisely what __attribute__ ((__packed__)) is for).
Re the ZMQ example, I am not familiar with the code, but I am guessing they are defining the wire format for a message, which is of constant byte size. At some point they will cast part or all of it to a structure, probably with __attribute__ ((__packed__)), being careful about what falls where. Why do this rather than just define a structure for the entire message? This may be so that if someone changes the struct, they get an immediate failure as opposed to a working but incompatible protocol. But that bit is just speculation.