C lexer: token concatenation of unterminated string literals - c

Question

Consider the following c code:
#include <stdio.h>
#define PRE(a) " ## a
int main() {
printf("%s\n", PRE("));
return 0;
}
If we adhere strictly to the tokenization rules of c99, I would expect it to break up as:
...
[#] [define] [PRE] [(] [a] [)] ["]* [##] [a]
...
[printf] [(] ["%s\n"] [,] [PRE] [(] ["]* [)] [)] [;]
...
* A single non-whitespace character that does not match any preprocessing-token pattern
And thus, after running preprocessing directives, the printf line should become:
printf("%s\n", "");
And parse normally. But instead, it throws an error when compiled with gcc, even when using the flags -std=c99 -pedantic. What am I missing?

Answer 1

From C11 6.4. Lexical elements:
preprocessing-token:
header-name
identifier
pp-number
character-constant
string-literal
punctuator
each non-white-space character that cannot be one of the above
3 [...] The categories of preprocessing tokens are: header names,
identifiers, preprocessing numbers, character constants, string
literals, punctuators, and single non-white-space characters that do
not lexically match the other preprocessing token categories.69) If a
' or a " character matches the last category, the behavior is
undefined. [...]
So if " is not part of a string-literal, but is a non-white-space character, the behavior is undefined. I do not know why it's undefined and not a hard error - I think it's to allow compilers to parse multiline string literals.
it throws an error
But on godbolt:
<source>:3:16: warning: missing terminating " character
3 | #define PRE(a) " ## a
| ^
<source>:6:24: warning: missing terminating " character
6 | printf("%s\n", PRE("));
| ^
<source>:8: error: unterminated argument list invoking macro "PRE"
8 | }
|
<source>: In function 'int main()':
<source>:6:20: error: 'PRE' was not declared in this scope
6 | printf("%s\n", PRE("));
| ^~~
<source>:6:20: error: expected '}' at end of input
<source>:5:12: note: to match this '{'
5 | int main() {
| ^
it throws an error not on #define PRE line (it could), but on PRE(") line. Tokens are recognized before macro substitutions (phase 3 vs phase 4), so whatever you do you can't like "create" new lexical string literals as a result of macro substitution by for example gluing two macros or like you want to do. Note that -pedantic will not change the warning into error - -pedantic throws errors where standard tells to throw error, but the standard tells the behavior is undefined, so no error is needed there.