I've encountered a similiar problem as described in another thread (perf_event_open - how to monitoring multiple events). I was able to solve it and the code is working, but I want to understand why this part actually works and how this is not a violation of any kind:
char buf[4096];
struct read_format* rf = (struct read_format*) buf;
struct read_format is defined as followed:
struct read_format {
uint64_t nr;
struct {
uint64_t value;
uint64_t id;
} values[/*2*/]; };
How does the compiler know to which value uint64_t nr should be initialized? Or how to initialize the inner struct right?
This code is incorrect in Standard C:
char buf[4096];
read(fd1, buf, 4096); // Assume error handling, omitted for brevity
struct read_format* rf = (struct read_format*) buf;
printf("%llu\n", rf->nr);
There are two issues -- and these are distinct issues which should not be conflated -- :
buf might not be correctly aligned for struct read_format. If it isn't, the behaviour is undefined.
Accessing rf->nr violates the strict aliasing rule and the behaviour is undefined. An object with declared type char cannot be read of written by an expression of type . unsigned long long. Note that the converse is not true.
Why does it appear to work? Well, "undefined" does not mean "must explode". It means the C Standard no longer specifies the program's behaviour. This sort of code is somewhat common in real code bases. The major compiler vendors -- for now -- include logic so that this code will behave as "expected", otherwise too many people would complain.
The "expected" behaviour is that accessing *rf should behave as if there exists a struct read_format object at the address, and the bytes of that object are the same as the bytes of buf . Similar to if the two were in a union.
The code could be made compliant with a union:
union
{
char buf[4096];
struct read_format rf;
} u;
read(fd1, u.buf, sizeof u.buf);
printf("%llu\n", u.rf->nr);
The strict aliasing rule is "disabled" for union members accessed by name; and this also addresses the alignment problem since the union will be aligned for all members.
It's up to you whether to be compliant, or trust that compilers will continue put practicality ahead of maximal optimization within the constraints permitted by the Standard.
It doesn't The buffer is zero-initialized and the struct pointer is initialized with a pointer to the buffer.
It looks completely whack; however it really isn't. The read function is going to read as many structures into the buffer as fit.
The outer structure is variable-length. The advance loop looks like this:
struct read_format *current = rf;
if (readstructs(..., ¤t, 4096)) {
for (;current;current=current->nr?((struct read_format *)((char *)current + current->nr)):NULL) {
}
}
These things appear in system-level OS calls to decrease the complexity of copying memory across security boundaries. The read side is easy and well-taught. The writer performs the operations necessary in filling the buffer to ensure this simple reader does not violate any system-level constraints. The code will work despite looking like it violates types left and right because the writer has set it up to work. In particular, the pointer will be aligned.
I've seen a similar method used in old file formats. Unfortunately that only follows the rules of the platform that wrote it (usually something ancient and far more permissive than a modern system) and leads to having to write a byte-at-a-time reader because the host doing the reading doesn't correspond.
Related
Consider the following struct:
struct s {
int a, b;
};
Typically1, this struct will have size 8 and alignment 4.
What if we create two struct s objects (more precisely, we write into allocated storage two such objects), with the second object overlapping the first?
char *storage = malloc(3 * sizeof(struct s));
struct s *o1 = (struct s *)storage; // offset 0
struct s *o2 = (struct s *)(storage + alignof(struct s)); // offset 4
// now, o2 points half way into o1
*o1 = (struct s){1, 2};
*o2 = (struct s){3, 4};
printf("o2.a=%d\n", o2->a);
printf("o2.b=%d\n", o2->b);
printf("o1.a=%d\n", o1->a);
printf("o1.b=%d\n", o1->b);
Is anything about this program undefined behavior? If so, where does it become undefined? If it is not UB, is it guaranteed to always print the following:
o2.a=3
o2.b=4
o1.a=1
o1.b=3
In particular, I want to know what happens to the object pointed to by o1 when o2, which overlaps it, is written. Is it still allowed to access the unclobbered part (o1->a)? Is accessing the clobbered part o1->b simply the same as accessing o2->a?
How does effective type apply here? The rules are clear enough when you are talking about non-overlapping objects and pointers that point to the same location as the last store, but when you start talking about the effective type of portions of objects or overlapping objects it is less clear.
Would anything change if the the second write was of a different type? If the members were say int and short rather than two ints?
Here's a godbolt if you want to play with it there.
1 This answer applies to platforms where this isn't the case too: e.g., some might have size 4 and alignment 2. On a platform where the size and alignment were the same, this question wouldn't apply since aligned, overlapping objects would be impossible, but I'm not sure if there is any platform like that.
Basically this is all grey area in the standard; the strict aliasing rule specifies basic cases and leaves the reader (and compiler vendors) to fill in the details.
There have been efforts to write a better rule but so far they haven't resulted in any normative text and I'm not sure what the status of this is for C2x.
As mentioned in my answer to your previous question, the most common interpretation is that p->q means (*p).q and the effective type applies to all of *p, even though we then go on to apply .q .
Under this interpretation, printf("o1.a=%d\n", o1->a); would cause undefined behaviour as the effective type of the location *o1 is not s (since part of it has been overwritten).
The rationale for this interpretation can be seen in a function like:
void f(s* s1, s* s2)
{
s2->a = 5;
s1->b = 6;
printf("%d\n", s2->a);
}
With this interpretation the last line could be optimised to puts("5"); , but without it, the compiler would have to consider that the function call may have been f(o1, o2); and therefore lose all benefits that are purportedly provided by the strict aliasing rule.
A similar argument applies to two unrelated struct types that both happen to have an int member at different offset.
Context:
I was reviewing some code that receives data from an IO descriptor into a character buffer, does some control on it and then use part of the received buffer to populate a struct, and suddenly wondered whether a strict aliasing rule violation could be involved.
Here is a simplified version
#define BFSZ 1024
struct Elt {
int id;
...
};
unsigned char buffer[BFSZ];
int sz = read(fd, buffer, sizeof(buffer)); // correctness control omitted for brievety
// search the beginning of struct data in the buffer, and process crc control
unsigned char *addr = locate_and_valid(buffer, sz);
struct Elt elt;
memcpy(&elt, addr, sizeof(elt)); // populates the struct
// and use it
int id = elt.id;
...
So far, so good. Provide the buffer did contain a valid representation of the struct - say it has been produced on same platform, so without endianness or padding problem - the memcpy call has populated the struct and it can safely be used.
Problem:
If the struct is dynamically allocated, it has no declared type. Let us replace last lines with:
struct Elt *elt = malloc(sizeof(struct Element)); // no declared type here
memcpy(elt, addr, sizeof(*elt)); // populates the newly allocated memory and copies the effective type
// and use it
int id = elt->id; // strict aliasing rule violation?
...
Draft n1570 for C language says in 6.5 Expressions ยง6
The effective type of an object for an access to its stored value is the declared type of the
object, if any.87) If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value. If a value is copied into an object having no declared type using
memcpy or memmove, or is copied as an array of character type, then the effective type
of the modified object for that access and for subsequent accesses that do not modify the
value is the effective type of the object from which the value is copied, if it has one.
buffer does have an effective type and even a declared type: it is an array of unsigned char. That is the reason why the code uses a memcpy instead of a mere aliasing like:
struct Elt *elt = (struct Elt *) addr;
which would indeed be a strict aliasing rule violation (and could additionaly come with alignment problems). But if memcpy has given an effective type of an unsigned char array to the zone pointed by elt, everything is lost.
Question:
Does memcpy from an array of character type to a object with no declared type give an effective type of array of character?
Disclaimer:
I know that it works without a warning with all common compilers. I just want to know whether my understanding of standard is correct
In order to better show my problem, let us considere a different structure Elt2 with sizeof(struct Elt2)<= sizeof(struct Elt), and
struct Elt2 actual_elt2 = {...};
For static or automatic storage, I cannot reuse object memory:
struct Elt elt;
struct Elt2 *elt2 = &elt;
memcpy(elt2, &actual_elt2, sizeof(*elt2));
elt2->member = ... // strict aliasing violation!
While it is fine for dynamic one (question about it there):
struct Elt *elt = malloc(sizeof(*elt));
// use elt
...
struct Elt2 *elt2 = elt;
memcpy(elt2, &actual_elt2, sizeof(*elt2));
// ok, memory now have struct Elt2 effective type, and using elt would violate strict aliasing rule
elt2->member = ...; // fine
elt->id = ...; // strict aliasing rule violation!
What could make copying from a char array different?
The code is fine, no strict aliasing violation. The pointed-at data has an effective type, so the bold cited text does not apply. What applies here is the part you left out, last sentence of 6.5/6:
For all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.
So the effective type of the pointed-at object becomes struct Elt. The returned pointer of malloc does indeed point to an object with no delcared type, but as soon as you point at it, the effective type becomes that of the struct pointer. Otherwise C programs would not be able to use malloc at all.
What makes the code safe is also that you are copying data into that struct. Had you instead just assigned a struct Elt* to point at the same memory location as addr, then you would have a strict aliasing violation and UB.
Lundin's answer is correct; what you are doing is fine (so long as the data is aligned and of same endianness).
I want to note this is not so much a result of the C language specification as it is a result of how the hardware works. As such, there's not a single authoritative answer. The C language specification defines how the language works, not how the language is compiled or implemented on different systems.
Here is an interesting article about memory alignment and strict aliasing on a SPARC versus Intel processor (notice the exact same C code performs differently, and gives errors on one platform while working on another):
https://askldjd.com/2009/12/07/memory-alignment-problems/
Fundamentally, two identical structs, on the same system with the same endian and memory alignment, must work via memcpy. If it didn't then the computer wouldn't be able to do much of anything.
Finally, the following question explains more about memory alignment on systems, and the answer by joshperry should help explain why this is a hardware issue, not a language issue:
Purpose of memory alignment
I'd like to know if casting a pointer to uint32_t to a pointer of a union containing a uint32_t will lead to defined behavior in C, i.e.
typedef union
{
uint8_t u8[4];
uint32_t u32;
} T32;
void change_value(T32 *t32)
{
t32->u32 = 5678;
}
int main()
{
uint32_t value = 1234;
change_value((T32 *)&value); // value is 5678 afterwards
return EXIT_SUCCESS;
}
Is this valid C? Many thanks in advance.
The general answer to your question is, no, this is in general not defined. If the union contains a field that has larger alignment than uint32_t such a union must have the largest alignment and accessing that pointer would then lead to UB. This could e.g happen if you replace uint8_t in your example by double.
In your particular case, though, the behavior is well defined. uint8_t, if it exists, is most likely nothing other than unsigned char and all character types always have the least alignment requirement.
Edit:
As R.. mentions in his comments there are other issues with your approach. First, theoretically, uint8_t could be different from unsigned char if there is an unsigned "extended integer type" of that width. This is very unlikely, I never heard of such an architecture. Second, your approach is subject to aliasing issues, so you should be extremely careful.
At the risk of incurring downvotes... Conceptually, there is nothing wrong with what you are trying to do. That is, define a piece of storage that can be viewed as four bytes an a 32 bit integer, and then reference and modify that storage using a pointer.
However, I would ask why you would want to write code where its intent is obscured. What you are really doing is forcing the next programmer who reads your code to think for minutes and maybe even try a little test program. Thus, this programming style is "expensive".
You could have just as easily defined, value as:
T32 value;
// etc.
change_value(&value);
and then avoid the cast and subsequent angst.
Since all union members are guaranteed to start at the same memory address, your program as written does not lead to undefined behavior.
I have a structure like this
struct packet
{
int seqnum;
char type[1];
float time1;
float pri;
float time2;
unsigned char data[512];
}
I am receiving packet in an array
char buf[529];
I want to take the seqnum,data everything separately.Does the following typecast work.. It is giving junk value for me.
struct packet *pkt;
pkt=(struct packet *)buf;
printf(" %d",pkt->seqnum)
No, that likely won't work and is generally a bad and broken way of doing this.
You must use compiler-specific extensions to make sure there's no invisible padding between your struct members, for something like that to work. With gcc, for instance, you do this using the __attribute__() syntax.
It is, thus, not a portable idea.
It's much better to be explicit about it, and unpack each field. This also gives you a chance to have a well-defined endianness in your network protocol, which is generally a good idea for interoperability's sake.
No, that isn't generally valid code. You should make the struct first and then memcopy stuff into it:
packet p;
memcpy(&p.seqnum, buf + 0, 4);
memcpy(&p.type[0], buf + 4, 1);
memcpy(&p.time1, buf + 5, 4);
And so forth.
You must take great care to get the type sizes and endianness right.
First of all, you cannot know in advance where the compiler will insert padding bytes in your structure for performance optimization (cache line alignment, integer alignment etc) since this is platform-dependent. Except, of course, if you are considering building the app only on your platform.
Anyway, in your case it seems like you are getting data from somewhere (network ?) and it is highly probable that the data has been compacted (no padding bytes between fields).
If you really want to typecast your array to a struct pointer, you can still tell the compiler to remove the padding bytes it might add. Note that this depends on the compiler you use and is not a standard C implementation. With gcc, you might add this statement at the end of your structure definition :
struct my_struct {
int blah;
/* Blah ... */
} __attribute__((packed));
Note that it will affect the performance for member access, copy etc ...
Unless you have a very good reason to do so, don't ever use the __attribute__((packed)) thing !
The other solution, which is much more advisable is to make the parsing on your own. You just allocate an appropriate structure and fill its fields by seeking the good information from your buffer. A sequence of memcpy instructions is likely to do the trick here (see Kerrek's answer)
I know a pointer to one type may be converted to a pointer of another type. I have three questions:
What should kept in mind while typecasting pointers?
What are the exceptions/error may come in resulting pointer?
What are best practices to avoid exceptions/errors?
A program well written usually does not use much pointer typecasting. There could be a need to use ptr typecast for malloc for instance (declared (void *)malloc(...)), but it is not even necessary in C (while a few compilers may complain).
int *p = malloc(sizeof(int)); // no need of (int *)malloc(...)
However in system applications, sometimes you want to use a trick to perform binary or specific operation - and C, a language close to the machine structure, is convenient for that. For instance say you want to analyze the binary structure of a double (that follows thee IEEE 754 implementation), and working with binary elements is simpler, you may declare
typedef unsigned char byte;
double d = 0.9;
byte *p = (byte *)&d;
int i;
for (i=0 ; i<sizeof(double) ; i++) { ... work with b ... }
You may also use an union, this is an exemple.
A more complex utilisation could be the simulation of the C++ polymorphism, that requires to store the "classes" (structures) hierarchy somewhere to remember what is what, and perform pointer typecasting to have, for instance, a parent "class" pointer variable to point at some time to a derived class (see the C++ link also)
CRectangle rect;
CPolygon *p = (CPolygon *)▭
p->whatami = POLY_RECTANGLE; // a way to simulate polymorphism ...
process_poly ( p );
But in this case, maybe it's better to directly use C++!
Pointer typecast is to be used carefully for well determined situations that are part of the program analysis - before development starts.
Pointer typecast potential dangers
use them when it's not necessary - that is error prone and complexifies the program
pointing to an object of different size that may lead to an access overflow, wrong result...
pointer to two different structures like s1 *p = (s1 *)&s2; : relying on their size and alignment may lead to an error
(But to be fair, a skilled C programmer wouldn't commit the above mistakes...)
Best practice
use them only if you do need them, and comment the part well that explains why it is necessary
know what you are doing - again a skilled programmer may use tons of pointer typecasts without fail, i.e. don't try and see, it may work on such system / version / OS, and may not work on another one
In plain C you can cast any pointer type to any other pointer type. If you cast a pointer to or from an uncompatible type, and incorrectly write the memory, you may get a segmentation fault or unexpected results from your application.
Here is a sample code of casting structure pointers:
struct Entity {
int type;
}
struct DetailedEntity1 {
int type;
short val1;
}
struct DetailedEntity2 {
int type;
long val;
long val2;
}
// random code:
struct Entity* ent = (struct Entity*)ptr;
//bad:
struct DetailedEntity1* ent1 = (struct DetailedEntity1*)ent;
int a = ent->val; // may be an error here, invalid read
ent->val = 117; // possible invali write
//OK:
if (ent->type == DETAILED_ENTITY_1) {
((struct DetailedEntity1*)ent)->val1;
} else if (ent->type == DETAILED_ENTITY_2) {
((struct DetailedEntity2*)ent)->val2;
}
As for function pointers - you should always use functions which exactly fit the declaration. Otherwise you may get unexpected results or segfaults.
When casting from pointer to pointer (structure or not) you must ensure that the memory is aligned in the exact same way. When casting entire structures the best way to ensure it is to use the same order of the same variables at the start, and differentiating structures only after the "common header". Also remember, that memory alignment may differ from machine to machine, so you can't just send a struct pointer as a byte array and receive it as byte array. You may experience unexpected behaviour or even segfaults.
When casting smaller to larger variable pointers, you must be very careful. Consider this code:
char* ptr = malloc (16);
ptr++;
uint64_t* uintPtr = ptr; // may cause an error, memory is not properly aligned
And also, there is the strict aliasing rule that you should follow.
You probably need a look at ... the C-faq maintained by Steve Summit (which used to be posted in the newsgroups, which means it was read and updated by a lot of the best programmers at the time, sometimes the conceptors of the langage itself).
There is an abridged version too, which is maybe more palatable and still very, very, very, very useful. Reading the whole abridged is, I believe, mandatory if you use C.