This is in regards to some old (pre-C89) code that I'm working on. Part of that code is a small library that defines these structures in the header file:
// lib.h
struct data_node {
const struct data_node *next;
const struct data_node *prev;
void *data;
};
struct trace_node {
const struct trace_node *next;
const struct trace_node *prev;
unsigned int id;
const char *file;
int line;
};
const struct trace_node *get_trace(void);
The source file redefines those same structures, like so:
// lib.c
// does *not* include "lib.h"
struct data_node {
struct data_node *next;
struct data_node *prev;
void *data;
};
struct trace_node {
struct trace_node *next;
struct trace_node *prev;
unsigned int id;
const char *file;
int line;
struct data_node *syncData; /* not included in header file version */
};
It works like you would expect: the syncData field is not visible to client code that includes the "lib.h" header.
Background
The library maintains 2 internal lists: the trace list, and the data list. The syncData field keeps the 2 lists in sync (go figure).
If client code had access to the syncData field, it could disrupt the synchronization between the lists. But, the trace list can get pretty large, so rather than copying every node into a smaller version of the struct, it just returns the address of the sentinel node for the internal list.
Question
I've compiled this with -Wall, -Wpedantic, and -Wextra, and I can't get gcc to complain about it, both with -std=c99 and -std=c11. A hex dump of the memory shows the bytes for the hidden field, right where they ought to be.
The relevant section of the standard (6.2.7.1) says:
Two types have compatible type if their types are the same. Additional rules for
determining whether two types are compatible are described in 6.7.2 for type specifiers,
in 6.7.3 for type qualifiers, and in 6.7.5 for declarators.46) Moreover, two structure,
union, or enumerated types declared in separate translation units are compatible if their
tags and members satisfy the following requirements: If one is declared with a tag, the
other shall be declared with the same tag. If both are complete types, then the following
additional requirements apply: there shall be a one-to-one correspondence between their
members such that each pair of corresponding members are declared with compatible
types, and such that if one member of a corresponding pair is declared with a name, the
other member is declared with the same name. For two structures, corresponding
members shall be declared in the same order. For two structures or unions, corresponding
bit-fields shall have the same widths. For two enumerations, corresponding members
shall have the same values.
Which, depending on how you want to read it, could be taken to say that compatible struct definitions are restricted to ONLY having corresponding pairs of members (and no others), or that struct definitions are compatible if, where they do have corresponding pairs of members, those pairs meet the requirements.
I don't think this is Undefined Behavior. At worst, I think it may be unspecified. Should I refactor this to use 2 distinct struct definitions? Doing this would require a performance hit to allocate a new public node for each node in the internal list and copy over the public data.
This has undefined behavior, and for good reasons.
First, the text clearly states that compatible struct must have a one-to-one correspondance between fields. So the behavior is undefined if client and library access the same object. A compiler can't detect that undefined behavior, because this rule is about stiching together knowledge from two different translation units that are compiled separately. This is why you don't see any diagnostic.
The reason that your example is particularly bad practise is that not even the sizes of the two struct types aggree and maybe not even their alignment. So a client that accesses such an object will make false assumptions about optimization opportunities.
If lib.c does not include lib.h then definitions from there are not visible to lib.c so no conflicts.
C does not use function overloading and so it does not use name mangling and so if you have this declaration:
struct trace_node *get_trace(void);
in one place but function is implemented as
struct foo_trace_node *get_trace(void);
then the linker will happily link your code with that get_trace()
What you do is directly violating standards. A "one-to-one correspondance" would expect the pointer to be seen by client code as well. Your code violates the first part of that sentence.
Imagine the client code to unlink one structure from the list and create and link in one of "his" structures with the wrong size - Lib would crash sooner or later when dereferencing that pointer.
If you don't want to expose certain fields of structures, don't expose the structure at all. Hand the client code an anonymous structure pointer and expose accessor functions in the lib that return field values the client is allowed to see. Or pack the "allowed part" into an embedded structure within the larger one and hand that to client code if you want to avoid field-by-field access. It's probably also not a good idea to have client code see the link pointers of your list structure.
I found some additional information in
The New C Standard
An Economic and Cultural Commentary
Derek M. Jones
derek#knosof.co.uk
http://www.coding-guidelines.com/cbook/cbook1_2.pdf
This particular version covers the C99 standard, but since the text for the relevant section is identical in both versions of the standard, it's a wash.
Pertinent commentary:
633
Moreover, two structure, union, or enumerated types declared in
separate translation units are compatible if their tags and members
satisfy the following requirements:
Commentary
These requirements apply if the structure or union type was declared
via a typedef or through any other means. Because there can be more
than one declaration of a type in the same translation unit, these
requirements really apply to the composite type in each translation
unit. In the following list of requirements, those that only apply to
structures, unions, enumerations, or a combination thereof are
explicitly called out as such. Two types are compatible if they obey
both of the following requirements:
• Tag compatibility.
If both types have tags, both shall be the same.
If one, or neither, type has a tag, there is no requirement to be obeyed.
• Member compatibility.
Here the requirement is that for every member in both types there is a
corresponding member in the other type that has the following
properties:
The corresponding members have a compatible type.
The corresponding members either have the same name or are unnamed.
For structure types, the corresponding members are defined in the same order in their respective definitions.
For structure and union types, the corresponding members shall either both be bit-fields having the same width, or neither shall be
bit-fields.
For enumerated types, the corresponding members (the enumeration constants) have the same value.
So the verdict is unanimous. The wording in Mr. Jones' commentary is perhaps a little bit clearer (to me, at any rate).
I failed to mention in the OP that the original header file comments for the get_trace() function clearly state that the trace list is to be considered strictly read-only, so the points raised about an object of the abbreviated structure in client code finding its way back into the library code are - while still valid in the general sense - not exactly applicable in this specific case.
However, the question of compiler optimizations is bang on, especially considering how much more aggressive compiler optimizations are now, versus 35 years ago. So, I will refactor.
Related
The C 11 standard defines struct compatibility as follows (6.2.7):
Moreover, two structure, union, or enumerated types declared in separate translation units are compatible if their tags and members satisfy the following requirements: If one is declared with a tag, the other shall be declared with the same tag. If both are completed anywhere within their respective translation units, then the following additional requirements apply: there shall be a one-to-one correspondence between their members such that each pair of corresponding members are declared with compatible types…
That means I can have 2 files like this:
foo.c:
struct struc {
int x;
};
int foo(struct struc *s)
{
return s->x;
}
main.c:
struct struc {
float x;
};
int foo(struct struc *s);
int main(void)
{
return foo(&(struct struc){1.2f});
}
Smells like undefined behavior (as it is for types like int and float). But if I am understanding the standard correctly (maybe I am misinterpreting the second sentence), this is allowed. If so, what is the rationale behind this? Why not also specify that structs in separate translation units must also be structurally equivalent?
Smells like undefined behavior
Because it is.
But if I am understanding the standard correctly
This doesn't seem to be the case in this particular instance.
this is allowed.
Nope. I do not see (and you do not explain) how the standard language could be interpreted this way.
The standard says
If both are completed anywhere within their respective translation units
This condition holds in your your example.
then the following additional requirements apply: there shall be a one-to-one correspondence between their members such that each pair of corresponding members are declared with compatible types
This requirement is not satisfied, so the types are not compatible.
Why not also specify that structs in separate translation units must also be structurally equivalent?
The standard specifies exactly that. "[o]ne-to-one correspondence between their members such that each pair of corresponding members are declared with compatible types" is precisely the definition of structural equivalence.
I'm designing an application and came across an implementation issue. I have the following struct definition:
app.h:
struct application_t{
void (*run_application)(struct application_t*);
void (*stop_application)(struct application_t*);
}
struct application_t* create();
The problem came when I tried to "implement" this application_t. I tend to define another struct:
app.c:
struct tcp_application_impl_t{
void (*run_application)(struct application_t*);
void (*stop_application)(struct application_t*);
int client_fd;
int socket_fd;
}
struct application_t* create(){
struct tcp_application_impl_t * app_ptr = malloc(sizeof(struct tcp_application_impl_t));
//do init
return (struct application_t*) app_ptr;
}
So if I use this as follows:
#include "app.h"
int main(){
struct application_t *app_ptr = create();
(app_ptr -> run_application)(app_ptr); //Is this behavior well-defined?
(app_ptr -> stop_application)(app_ptr); //Is this behavior well-defined?
}
The problem confusing me is if I this calling to (app_ptr -> run_application)(app_ptr); yeilds UB.
The "static type" of app_ptr if struct application_t*, but the "dynamic type" is struct tcp_application_impl_t*. The struct application_t and struct tcp_application_t are not compatible by N1570 6.2.7(p1):
there shall be a one-to-one correspondence between their members such
that each pair of corresponding members are declared with compatible
types
which obviously is not true in this case.
Can you please provide a reference to the Standard explaining the behavior?
Your two structs aren't compatible since they are different types. You have already found the chapter "compatible types" that defines what makes two structs compatible. The UB comes later when you access these structs with a pointer to the wrong type, strict aliasing violation as per 6.5/7.
The obvious way to solve this would have been this:
struct tcp_application_impl_t{
struct application_t app;
int client_fd;
int socket_fd;
}
Now the types may alias, since tcp_application_impl_t is an aggregate containing a application_t among its members.
An alternative to make this well-defined, is to use a sneaky special rule of "union common initial sequence", found hidden in C17 6.5.2.3/6:
One special guarantee is made in order to simplify the use of unions: if a union contains
several structures that share a common initial sequence (see below), and if the union
object currently contains one of these structures, it is permitted to inspect the common
initial part of any of them anywhere that a declaration of the completed type of the union
is visible. Two structures share a common initial sequence if corresponding members
have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.
This would allow you to use your original types as you declared them. But somewhere in the same translation unit, you will have to add a dummy union typedef to utilize the above rule:
typedef union
{
struct application_t app;
struct tcp_application_impl_t impl;
} initial_sequence_t;
You don't need to actually use any instance of this union, it just needs to sit there visible. This tells the compiler that these two types are allowed to alias, as far as their common initial sequence goes. In your case, it means the function pointers but not the trailing variables in tcp_application_impl_t.
Edit:
Disclaimer. The common initial sequence trick is apparently a bit controversial, with compilers doing other things with it than the committee intended. And possibly works differently in C and C++. See union 'punning' structs w/ "common initial sequence": Why does C (99+), but not C++, stipulate a 'visible declaration of the union type'?
If the "strict aliasing rule" (N1570 6.5p7) is interpreted merely as specifying the circumstances under which things may alias (which would seem to be what the authors intended, given Footnote 88, which says "The intent of this list is to specify those circumstances in which an object may or may not be aliased") code like yours should pose no problem provided that in all contexts where an object is accessed using lvalues of two different types, one of the involved lvalues is visibly freshly derived from the other.
The only way 6.5p7 can make any sense is if operations involving objects that are freshly visibly derived from other objects are recognized as operations on the originals. The question of when to recognize such derivation is left as a quality-of-implementation issue, however, and thought the marketplace would be better able to judge than the Committee what was necessary for something to be a "quality" implementation suitable for some particular purpose.
If the goal is to write code that will work on implementations which are configured to honor the clear intention of footnote 88, one should be safe provided that objects don't alias. Upholding this requirement may require that one ensure that the compiler can see either that pointers are related to each other, or that they are each freshly derived from a common object at point of use. Given, e.g.
thing1 *p1 = unionArray[i].member1;
int v1 = p1->x;
thing2 *p2 = unionArray[j].member2;
p2->x = 31;
thing1 *p3 = unionArray[i].member1;
int v2 = p3->x;
each pointer would be used in a context where it was freshly derived from unionArray, and thus there would be no aliasing even if i==j. A compiler like "icc" will have no problems with such code, even with -fstrict-aliasing enabled, but because both gcc and clang impose the requirements of 6.5p7 upon programmers even in cases not involving aliasing, they will not process it correctly.
Note that if the code had been:
thing1 *p1 = unionArray[i].member1;
int v1 = p1->x;
thing2 *p2 = unionArray[j].member2;
p2->x = 31;
int v2 = p1->x;
then the second use of p1 would alias p2 in cases where i==j because p2 would access the storage associated with p1, via means not involving p1, between the time p1 is formed and the last time it is used (thus aliasing p1).
According to the authors of the Standard, the Spirit of C includes the principles "Trust the programmer" and "Don't prevent the programmer from doing what needs to be done". Unless there is a particular need to cope with the limitations of an implementation that is not particularly well suited to what one is doing, one should target implementations that uphold the Spirit of C in a fashion appropriate to one's purposes. The -fstrict-aliasing dialect processed by icc, or the -fno-strict-aliasing dialects processed by icc, gcc, and clang, should be suitable for your purposes. The -fstrict-aliasing dialects of gcc and clang should be recognized as simply unsuitable for your purposes, and not worth targeting.
I ran into some weird issues (at compile time) with my embedded C code when I increased the size of the array from size [2] to [27] which contained structured metadata. So it is obvious that this is a nested structured style program and NOT my original work!
I remember in assembly code instructions, there used to be a JMP FAR, JMP NEAR/SHORT.. Is there any such limitations in structures (obviously there is no jump in structures). Could there be a condition that the compiler cannot assign a contiguous memory block for a structure because there is a limit to it and fails during compilation.
SOME CLUE: The fix as i recall was to move the member (array member)from original structure to another nested structure. That's the reason why i deduced that there maybe a limitation.
//STRUCTURES
typedef UINT8 P_Name_t[5];
typedef UINT8 ChipSN_t[3];
typedef struct
{
ChipSN_t ChipSN;
<other members>
} ChipIdent_t;
typedef struct Data_t
{
ChipIdent_t ReadOnlyMemID;
<other members>
} Data_t;
typedef struct
{
P_Name_t NameOfPart;
<other members>
} Log_t;
Data_t Data_Src;
typedef struct
{
P_Name_t NameOfPart;
ChipSN_t ChipSN;
}PartNum_ID_t;
typedef struct
{
PartNum_ID_t PN_ChipID[27];
<other members>;
}
According to standard, there is a limitation; but the limitation is much more than the size you mentioned here. C89 and C99 standard has the limit of 1023 structure members. See below text from the standard.
5.2.4.1 Translation limits
— 1023 members in a single structure or union
So looks like if your code is correct and limited to 1023 structure members the compiler you are using may have the limitation.
There may indeed be a limit to the number of structure elements your compiler can handle for a single structure, but as MCG quoted from the C Standard, this limit must be at least 1023 for compliant compilers and most compilers have higher limits or no practical limits at all.
I suspect the problem is elsewhere:
The Makefile for your project may have incomplete dependencies on the header file with the structure definitions: modules compiled with different definitions will be incompatible and produce undefined behavior. Remove all object files and recompile the project.
Some assembly modules may rely on the original structure layout: changing the definition will only affect the recompiled C modules.
Some library modules or OS system calls may rely on the original layout. Recompiling all libraries or even patching the OS may be required.
Some functions may rely on the original layout and use casts to access some of the structures via another type. Changing the layout may break such code, which is brittle anyway as even changing compilers may produce similar problems.
It is also possible that some of the type be multiply defined in different modules. This is poor and very error prone coding style as changing only one of the definitions will lead to undefined behavior when the object modules are linked together.
I'm quite rusty in C, but I think I'm having issues understanding the proper usage of using typedefs in headers, defining the actual struct's structure in the implementation file, and then using the struct in a third file.
I wrote a queue data-structure that has a type defined as such:
typedef struct queue
{
int count;
qnode_t *head;
qnode_t *tail;
} queue_t;
Where qnode_t is a struct that's only used within the queue's implementation file.
Within my header I have this:
typedef struct queue queue_t;
When using this queue_t type within another file I'm attempting to get the queue's length like this:
queue_t *work_queue;
...
int length = work_queue->count;
However, on that line where I'm looking up count I get the compiler error:
dereferencing pointer to incomplete type
I've been doing a lot of research about how to properly define types in C, but I think I just keep confusing myself more and more instead of getting clarity since many examples are either conflicting with other resources or are too simplified for me to put to practical use.
Would I be getting this error because the 'count' variable within the struct isn't defined there? If this is the case, then can I define the struct in BOTH the implementation and header? If so, can the header only have the count variable defined since the head & tail should be hidden/private? (I miss OOP) Should I just be making another function that takes a queue_t* and returns its length as size_t?
you can only dereference defined types, not declared types.
Type declarations are useful to type-check opaque pointers, but
object fields are not visible, cannot be accessed. You need to move the typedef into the header
to access fields of your queue object.
Edit: from the questions/answers below:
Yes, two identical struct definitions are seen as the same typedef. You could omit fields if you never had both definitions in the same source file, but don't do it, that leads to bugs and maintenance confusion. Better to use a naming convention eg names starting with underscores are internal.
The convention is to define the struct in the header then include the same header in the implementation file. This keeps the published layout in sync with the implementation
It is not possible in C to dereference any pointer unless the compiler has access to complete information about the type of what is pointed at. For struct pointers, that means a complete struct definition is needed.
So, when compiling the code that is complaining about an incomplete type, the compiler needs to have visibility of the complete definition of the struct type, not just the typedef of the pointer.
if you want to cut to the chase, please skip down to the last two paragraphs. If you're interested in my predicament and the steps I've taken to solve it, continue reading directly below.
I am currently developing portions of a C library as part of my internship. So naturally, there are some parts of code which should not be accessible to the user while others should be. I am basically developing several architecture-optimized random number generators (RNG's)(uniform, Gaussian, and exponential distributed numbers). The latter two RNG's depend on the uniform generator , which is in a different kernel (project). So, in the case that the user wants to use more than one RNG, I want to make sure I'm not duplicating code needlessly since we are constrained with memory (no point in having the same function defined multiple times at different addresses in the code segment).
Now here's where the problem arises. The convention for all other kernels in the library is that we have a two header files and two C files (one each for the natural C implementation and the optimized C version (which may use some intrinsic functions and assembly and/or have some restrictions to make it faster and better for our architecture). This is followed by another C file (a testbench) where our main function is located and it tests both implementations and compares the results. With that said, we cannot really add an additional header file for private or protected items nor can we add a global header file for all these generators.
To combat this restriction, I used extern functions and extern const int's in the C files which depend on the uniform RNG rather than #define's at the top of each C file in order to make the code more portable and easily modified in one place. This worked for the most part.
However, the tricky bit is that we are using an internal type within these kernels (which should not be seen by the user and should not be placed in the header file). Again, for portability, I would like to be able to change the definition of this typedef in one place rather than in multiple places in multiple kernels since the library may be used for another platform later on and for the algorithms to work it is critical that I use 32-bit types.
So basically I'm wondering if there's any way I can make a typedef "protected" in C. That is, I need it to be visible among all C files which need it, but invisible to the user. It can be in one of the header files, but must not be visible to the user who will be including that header file in his/her project, whatever that may be.
============================Edit================================
I should also note that the typedef I am using is an unsigned int. so
typedef unsigned int myType
No structures involved.
============================Super Edit==========================
The use of stdint.h is also forbidden :(
I am expanding on Jens Gustedt’s answer since the OP still has questions.
First, it is unclear why you have separate header files for the two implementations (“natural C” and “optimized C”). If they implement the same API, one header should serve for either.
Jens Gustedt’s recommendation is that you declare a struct foo in the header but define it only in the C source file for the implementation and not in the header. A struct declared in this way is an incomplete type, and source code that can only see the declaration, and not the definition, cannot see what is in the type. It can, however, use pointers to the type.
The declaration of an incomplete struct may be as simple as struct foo. You can also define a type, such as typedef struct foo foo; or typedef struct foo Mytype;, and you can define a type that is a pointer to the struct, such as typedef struct foo *FooPointer;. However, these are merely for convenience. They do not alter the basic notion, that there is a struct foo that API users cannot see into but that they can have pointers to.
Inside the implementation, you would fully define the struct. If you want an unsigned int in the struct, you would use:
struct foo
{
unsigned int x;
};
In general, you define the struct foo to contain whatever data you like.
Since the API user cannot define struct foo, you must provide functions to create and destroy objects of this type as necessary. Thus, you would likely have a function declared as extern struct foo *FooAlloc(some parameters);. The function creates a struct foo object (likely by calling malloc or a related function), initializes it with data from the parameters, and returns a pointer to the object (or NULL if the creation or initialization fails). You would also have a function extern void FooFree(struct foo *p); that frees a struct foo object. You might also have functions to reset, set, or alter the state of a foo object, functions to copy foo objects, and functions to report about foo objects.
Your implementations could also define some global struct foo objects that could be visible (essentially by address only) to API users. As a matter of good design, this should be done only for certain special purposes, such as to provide instances of struct foo objects with special meanings, such as a constant object with a permanent “initial state” for copying.
Your two implementations, the “natural C” and the “optimized C” implementations may have different definitions for the struct foo, provided they are not both used in a program together. (That is, each entire program is compiled with one implementation or the other, not both. If necessary, you could mangle both into a program by using a union, but it is preferable to avoid that.)
This is not a singleton approach.
Just do
typedef struct foo foo;
These are two declarations, a forward declaration of a struct and a type alias with the same name. Forward declared struct can be used to nothing else than to define pointers to them. This should give you enough abstraction and type safety.
In all your interfaces you'd have
extern void proc(foo* a);
and you'd have to provide functions
extern foo* foo_alloc(size_t n);
extern void foo_free(foo* a);
This would bind your users as well as your library to always use the same struct. Thereby the implementation of foo is completely hidden to the API users. You could even one day to decide to use something different than a struct since users should use foo without the struct keyword.
Edit: Just a typedef to some kind of integer wouldn't help you much, because these are only aliases for types. All your types aliased to unsigned could be used interchangeably. One way around this would be to encapsulate them inside a struct. This would make your internal code a bit ugly, but the generated object code should be exactly the same with a good modern compiler.