According to C11 § 6.5.2.3
Accessing a member of an atomic structure or union object results
in undefined behavior.
This makes sense since you cannot access a whole structure in general. Still why is _Atomic then also a type qualifier and not only a type specifier?
In other words what is the purpose of a structure which is qualified as _Atomic? I'm not allowed to either read or write to any element of it.
#include <stdatomic.h>
struct {
int x;
} _Atomic foo;
int main(void) {
foo.x = 42; // write error
return foo.x; // read error
}
Both accesses of foo.x result in a warning/error in GCC/Clang---which is perfectly fine w.r.t. the C11 standard. Why would I want to qualifier then a structure as _Atomic?
You can not access individual members, but always the structure as a whole. Accessing parts of something that is meant to be an atomic unit, makes not much sense, I think.
A typical use case for an atomic structure is the combination of two pointers, e.g. the head and tail of a list. To manipulate such a struct you would have to copy the current value from the atomic to a temporary, modify that, and then copy back. By that you'd always have a guarantee that the stored value is consistent at any time.
Per default all such operations on a whole atomic struct have sequential consistency.
Some implementations might be able to guarantee atomic behavior for structures meeting certain criteria. I don't know whether any implementations that did so actually existed before the publication of the C11 Standard, but it would certainly seems plausible. If an implementation usefully supported such structures, it would have been rather annoying for the Standard to demand that they be treated as a constraint violation.
It would have been helpful if the Standard mandated, or at least suggested, that implementations treat as constraint violations any attempts to define atomic structures for which they cannot offer proper semantics, but for whatever reason the authors of the Standard seem extremely loath to recognize a category of programs that would be rejected by some implementations but have defined behavior on implementations that don't reject it. A good standard should seek to maximize the number of useful programs that would fall into that category, but the Standard instead tends to regard constructs which some implementations will support and others won't as Undefined Behavior, letting implementations support them or not at their leisure, but not offering any safe way to determine whether they are supported.
Related
First, I apologize if this appears to be a duplicate, but I couldn't find exactly this question elsewhere
I was reading through N1570, specifically §6.5¶7, which reads:
An object shall have its stored value accessed only by an lvalue expression that has one of the following types:
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the object,
— a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
— a character type.
This reminded me of a common idiom I had seen in (BSD-like) socket programming, especially in the connect() call. Though the second argument to connect() is a struct sockaddr *, I have often seen passed to it a struct sockaddr_in *, which appears to work because they share a similar initial element. My question is:
To which contingency detailed in the above rule does this situation apply and why, or is it now undefined behavior that's an artifact of previous standard(s)?
This behavior is not defined by the C standard.
The behavior is defined by The Single Unix Specification and/or other documents relating to the software you are using, albeit in part implicitly.
The phrasing that “An object shall have its stored value accessed only by…” is misleading. The C standard cannot compel you to do anything; you are not obligated to obey its “shall” requirements. In terms of the C standard, the only consequence of not obeying its requirements is that the C standard does not define the behavior. This does not prohibit other documents from defining the behavior.
In the netinet/in.h documentation, we see “The sockaddr_in structure is used to store addresses for the Internet protocol family. Values of this type must be cast to struct sockaddr for use with the socket interfaces defined in this document.” So the documentation tells us not only that we should, but that we must, convert a sockaddr_in to a sockaddr. The fact that we must do so implies that the software supports it and that it will work. (Note that the phrasing is imprecise here; we do not actually cast a sockaddr_in to a sockaddr but actually convert the pointer, causing the sockaddr_in object in memory to be treated as a sockaddr.)
Thus there is an implied promise that the operating system, libraries, and developer tools provided for a Unix implementation support this.
This is an extension to the C language: Where behavior is not defined by the C standard, other documents may provide definitions and allow you to write software that cannot be written using the C standard alone. Behavior that the C standard says is undefined is not behavior that is prohibited but rather is an empty space that may be filled in by other specifications.
The rules about common initial sequences goes back to 1974. The earliest rules about "strict aliasing" only go back to 1989. The intention of the latter was not that they trump everything else, but merely that compilers be allowed to perform optimizations that their customers would find useful without being branded non-conforming. The Standard makes clear that in situations where one part of the Standard and/or an implementation's documentation would describe the behavior of some action but another part of the Standard would characterize it as Undefined Behavior, implementations may opt to give priority to the first, and the Rationale makes clear that the authors thought "the marketplace" would be better placed than the Committee to determine when implementations should do so.
Under a sufficiently pedantic reading of the N1570 6.5p7 constraints, almost all programs violate them, but in ways that won't matter unless an implementation is being sufficiently obtuse. The Standard makes no attempt to list all the situations in which an object of one type may be accessed by an lvalue of another, but rather those where a compiler must allow for an object of one type to be accessed by a seemingly unrelated lvalue of another. Given the code sequence:
int x;
int *p[10];
p[2] = &someStruct.intMember;
...
*p[2] = 23;
x = someStruct.intMember;
In the absence of the rules in 6.5p7, unless a compiler kept track of where p[2] came from, it would have no reason to recognize that the read of someStruct.member might be targeting storage that was just written using *p[2]. On the other hand, given the code:
int x;
int *p[10];
...
someStruct.intMember = 12;
p[2] = &someStruct.intMember;
x = *p[2];
Here, there is no rule that would actually allow the storage associated with a structure to be accessed by an lvalue of that member type, but unless a compiler is being deliberately blind, it would be able to see that after the first assignment to someStruct.intMember, the address of that member is being taken, and should either:
Account for all actions that will ever be done with the resulting pointer, if it is able to do so, or
Refrain from assuming that the structure's storage will not be accessed between the previous and succeeding actions using the structure's type.
I don't think it ever occurred to the people who were writing the rules that would later be renumbered as N1570 6.5p7 that they would be construed so as to disallow common patterns that exploited the Common Initial Sequence rule. As noted, most programs violate the constraints of 6.5p7, but do so in ways that would be processed predictably by any compiler that isn't being obtuse; those using the Common Initial Sequence guarantees would have fallen into that category. Since the authors of the Standard recognized the possibility of a "conforming" compiler that was only capable of meaningfully processing one contrived and useless program, the fact that an obtuse compiler could abuse the "aliasing rules" wasn't seen as a defect.
Is it OK do do something like this?
struct MyStruct {
int x;
const char y; // notice the const
unsigned short z;
};
struct MyStruct AStruct;
fread(&MyStruct, sizeof (MyStruct), 1,
SomeFileThatWasDefinedEarlierButIsntIncludedInThisCodeSnippet);
I am changing the constant struct member by writing to the entire struct from a file. How is that supposed to be handled? Is this undefined behavior, to write to a non-constant struct, if one or more of the struct members is constant? If so, what is the accepted practice to handle constant struct members?
It's undefined behavior.
The C11 draft n1570 says:
6.7.3 Type qualifiers
...
...
If an attempt is made to modify an object defined with a const-qualified type through use of an lvalue with non-const-qualified type, the behavior is undefined.
My interpretation of this is: To be compliant with the standard, you are only allowed to set the value of the const member during object creation (aka initialization) like:
struct MyStruct AStruct = {1, 'a', 2}; // Fine
Doing
AStruct.y = 'b'; // Error
should give a compiler error.
You can trick the compiler with code like:
memcpy(&AStruct, &AnotherStruct, sizeof AStruct);
It will probably work fine on most systems but it's still undefined behavior according to the C11 standard.
Also see memcpy with destination pointer to const data
How are constant struct members handled in C?
Read the C11 standard n1570 and its §6.7.3 related to the const qualifier.
If so, what is the accepted practice to handle constant struct members?
It depends if you care more about strict conformance to the C standard, or about practical implementations. See this draft report (work in progress in June 2020) discussing these concerns. Such considerations depend on the development efforts allocated on your project, and on portability of your software (to other platforms).
It is likely that you won't spend the same efforts on the embedded software of a Covid respirator (or inside some ICBM) and on the web server (like lighttpd or a library such as libonion or some FastCGI application) inside a cheap consumer appliance or running on some cheap rented Linux VPS.
Consider also using static analysis tools such as Frama-C or the Clang static analyzer on your code.
Regarding undefined behavior, be sure to read this blog.
See also this answer to a related question.
I am changing the constant struct member by writing to the entire struct from a file.
Then endianness issues and file system issues are important. Consider perhaps using libraries related to JSON, to YAML, perhaps mixed to sqlite or PostGreSQL or TokyoCabinet (and the source code of all these open source libraries, or from the Linux kernel, could be inspirational).
The Standard is a bit sloppy in its definition and use of the term "object". For a statement like "All X must be Y" or "No X may be Z" to be meaningful, the definition of X must have criteria that are not only satisfied by all X, but that would unambiguously exclude all objects that aren't required to be Y or are allowed to be Z.
The definition of "object", however, is simply "region of data storage in the execution environment, the contents of which can represent values". Such a definition, however, fails to make clear whether every possible range of consecutive addresses is always an "object", or when various possible ranges of addresses are subject to the constraints that apply to "objects" and when they are not.
In order for the Standard to unambiguously classify a corner case as defined or undefined, the Committee would have to reach a consensus as to whether it should be defined or undefined. If the Committee members fundamentally disagree about whether some cases should be defined or undefined, the only way to pass a rule by consensus will be if the rule is written ambiguously in a way that allows people with contradictory views about what should be defined to each think the rule supports their viewpoint. While I don't think the Committee members explicitly wanted to make their rules ambiguous, I don't think the Committee could have been consensus for rules that weren't.
Given that situation, many actions, including updating structures that have constant members, most likely falls in the realm of actions which the Standard doesn't require implementations to process meaningfully, but which the authors of the Standard would have expected that implementations would process meaningfully anyhow.
This is in reference to this question
How to implement the c malloc/realloc functions properly?
where the accepted answer mentions that a char array can be used to model a pool of arbitrary memory.
However one of the comments to that accepted answer states
A char array that doesn't have allocated storage duration can only be
aliased by a character type. In other words it cannot and it should
not be used as arbitrary memory
Is this correct? If so, then what could be used? I'd like to avoid using alloc or any specific OS calls - so I'm symmetrical with that question.
There are different problems around. First as shown by #Magisch's answer the related question was returning a dangling pointer causing Undefined Behaviour and generally execution errors.
The second one is related to the #^*# (censorship here) strict aliasing rule. Common compilers produce correct code when you use a character array as a large buffer to allocate any type from it, provided you ensure correct alignment. After all, this is the way they have to implement the malloc, realloc, and free routines. And as they are part of the hosted environment (C standard library), the compiler developpers are not masochist enough to fordib that usage.
But the C standard is a bit more strict here. You should read my own answer here to a similar question and particularly #EOF's comment to it:
You cannot parcel of parts of an object declared as char [] into objects of other types (except character types), because they do have a declared type... which means technically you can't implement malloc() in pure C
I believe the issue he pointed out was that if you allocate a char type array statically and then compile your library with a modern, desktop-like C compiler like gcc, you cannot easily convert the contents of that area to another type. Because then the compiler would perform optimizations based on pointer aliasing and screw everything up, see "the strict aliasing rule".
Simply assure that your compiler does not use strict aliasing and you'll be fine. For example, none of the common embedded systems compilers on the market does this.
With gcc you'd compile as -fno-strict-aliasing. Might be good to always enable warnings for code that would cause such problems -Wstrict-aliasing.
As a side note, uint8_t makes far more sense to use as generic type, because unlike char, it is completely unambiguous: it has no signedness and the size is well-known.
Aliasing refers to the strict aliasing rule that governs how the compiler is allowed to use registers. If you have pointers of different type referring to the same memory location, then writing done through one pointer type may not be noticed when reading through the other pointer type because the compiler is allowed to cache the data in registers.
When you implement a memory pool, this problem is usually moot, because the pool implementation doesn't read/write to the memory.
If you want arbitrary types, then the safest bet is a union. Not only will it "beat" the strict aliasing rule, it will also ensure correct alignment. Remember that malloc and friends ensure alignment suitable for any type while auto doesn't.
The issue is alignment. On processors that have alignment restrictions, a char array may not start at an address suitable for storing larger objects, like int or double.
So to be safe, you need to make sure the char array is aligned properly for any type.
If you're using a C11 compiler, then you can force alignment like this
#include <stddef.h>
#include <stdalign.h>
_Alignas(max_align_t) char buffer[SIZE];
For older compilers, __attribute__((aligned(SIZE))) may be a solution. Otherwise, you need to look for a #pragma that forces alignment.
And as discussed in various comments/answers, you should definitely disable the strict aliasing optimization with the -fno-strict-aliasing option. If that option (or the equivalent) doesn't exist, then you need to determine the optimization level that relies on the strict aliasing rule, and only use a lower optimization level.
Are there features / semantics introduced, or removed, in C99 which would make a well defined program written in C89 either
invalid (i.e not compiling anymore, according to the C99 standard)
compiling, but having different semantics.
My findings so far, concerning plainly invalid programs:
implicit int (C89 §3.5.2)
implicit function declaration (C89 §3.3.2.2)
not returning from a function expecting a return value (C89 §3.6.6.4)
using new keywords as identifier (for example restrict, inline, etc)
hacks involving //, which are now treated as comments. However, nearly never encountered in production code.
Subtle changes, making the same code having different semantics:
Integer division has been made well defined, for example -3 / 2 now has to truncate towards zero (C99 §6.5.5/6), instead of being implementation defined (C89 §3.3.5/6)
strtod gained the ability to parse hexadecimal numbers in C99, by parsing 0x or 0X
What have I missed?
There are a lot of programs which would have been considered valid under C89, prior to the publication of C99, which some people insist were never valid. C89 includes a rule that requires that an object of any type may only be accessed using a pointer of that type, a related type, or a character type. Prior to the publication of C99, this rule was generally interpreted as applying only to "named" objects (variables of static or automatic duration which are accessed directly by name), and only in situations where the object in question didn't have its address taken immediately before it was used as a different pointer type. Such interpretation was motivated by a number of factors:
One of the stated goals of the Standard was to fit with what existing compilers and programs were doing, and while it would have been rare for existing programs to access discrete named variables using pointers of different types other than in cases where the variable's address was taken immediately before such use, many other usages of pointer type punning were quite common.
The rationale for the Standard includes as its sole example a function which receives a pointer of one primitive type to write a global variable of another primitive type in such a way that a compiler would have no particular reason to expect aliasing. Being able to keep global variables in registers is clearly a useful optimization, and the stated purpose of the rule is to allow such optimizations in cases where a compiler would have no reason to expect aliasing to occur. Outlawing constructs like like (int*)&foo=23; does nothing to aid such optimizations, since the fact that code is taking foo's address and dereferencing it should make it abundantly clear to any compiler that isn't being deliberately obtuse that the code is going to modify foo.
There are many kinds of code which require semantically the ability to use memory bits as various types, and nothing in the Standard indicate that the rules were intended to make programmers jump through hoops (e.g. by using memcpy) to achieve semantics that could have been easily obtained in the absence of the rules, especially considering that using memcpy would prevent the compiler from keeping global variables in registers across the pointer accesses (thus defeating the purpose for which the rules were written in the first place).
If structure types V and W have a common initial sequence, U is any union type containing both, and p is a V* which identifies the V within a U, then (W*)(U*)p may be used to access those common members, and will be equivalent to (W*)p. Unless a compiler could show that p couldn't possibly be a pointer to a member of some union containing W, it would be required to allow (W*)p to access the common members; it was more helpful to simply treat such common member access as being legitimate regardless of whether or where U might exist than to search for excuses to deny it.
Nothing in the C89 rules makes clear how the "type" of a region of allocated storage is defined, or how storage which holds things of one type that are no longer needed might be re-purposed to hold things of another.
Keeping track of registers allocated to named variables was easier than keeping track of registers allocated to other pointer exceptions, and code which was interested in minimizing the number of loads and stores via pointers would often copy things to named variables and work on them there.
C99 added "effective type" rules which are explicitly applicable to allocated storage. Some people insist those were merely "clarifications" of rules which already existed in C89, but for the above reasons I find that viewpoint untenable. It's fashionable to claim that the only reasons compilers didn't apply aliasing rules to unnamed objects are #5 and #6, but objections #1-#4 are equally significant (and continue to apply to C99 just as much as C89). Still, since C99 added the effective type rules, many constructs which would have been treated as legitimate by most common interpretations of the C89 rules are clearly forbidden.
As an element of contrast and comparison, the git/git codebase remains strictly conform to C89 and does not use C99 initializers, or features from newer C standard.
This is detailed in Git 2.23 (Q3 2019) in Git Coding Guidelines.
This answer illustrates post-C89 feature that might be compatible with C89.
See commit cc0c429 (16 Jul 2019) by Junio C Hamano (gitster).
(Merged by Junio C Hamano -- gitster -- in commit fe9dc6b, 25 Jul 2019)
CodingGuidelines: spell out post-C89 rules
Even though we have been sticking to C89, there are a few handy features we borrow from more recent C language in our codebase after trying them in weather balloons and saw that nobody screamed.
Spell them out.
While at it, extend the existing variable declaration rule a bit to
read better with the newly spelled out rule for the for loop.
The coding guidelines now include:
You should not use features from newer C standard, even if your compiler groks them.
There are a few exceptions to this guideline:
since early 2012 with e1327023ea (Git v1.7.9.2), we have been using an enum definition whose last element is followed by a comma.
This, like an array initializer that ends with a trailing comma, can be used to reduce the patch noise when adding a new identifer at the end.
since mid 2017 with cbc0f81d (Git v2.15.0-rc0), we have been using designated
initializers for struct (e.g. "struct t v = { .val = 'a' };")
There are certain C99 features that might be nice to use in our code base, but we've hesitated to do so in order to avoid breaking compatibility with older compilers.
But we don't actually know if people are even using pre-C99 compilers these days.
If this patch can survive a few releases without complaint, then we can feel more confident that designated initializers are widely supported by our user base.
It also is an indication that other C99 features may be supported, but not a guarantee (e.g., gcc had designated initializers before C99 existed).
since mid 2017 with 512f41cf (Git v2.15.0-rc0), we have been using designated initializers for array (e.g. "int array[10] = { [5] = 2 }").
This is another test balloon to see if we get complaints from people
whose compilers do not support designated initializer for arrays.
These used to be forbidden, but we have not heard any breakage report, and they are assumed to be safe.
Variables have to be declared at the beginning of the block, before the first statement (i.e. -Wdeclaration-after-statement).
Declaring a variable in the for loop "for (int i = 0; i < 10; i++)" is still not allowed in this codebase.
Is the behavior mentioned the LWN article “Betrayed by a bitfield” forbidden by C11? Assume that spinlock_t is a type referring to a struct which is accessed only through means of C11 atomic primitives on its members (Actually it uses Linux (the kernel) memory barriers, but assume it doesn't so that it isn't undefined because of using an implementation-specific interface). In addition to the question in the title, is it an example of an “out of thin air store”?.
Thanks in advance.
No, as it is presented here it is not forbidden in C11. C11 only makes atomicity guarantees for _Atomic types, that is types that you declare within the language as being atomic. Since the kernel uses its own homegrown primitives to ensure atomicity you are on your own, C11 can't help you here.
The thing that changes with C11 _Atomic compared to homegrown assembler hacks, is that _Atomic may change size and alignment properties of the data type. In the given example, it could for example force the trailing bit field over the edge of the next word boundary.