See following example:
$ cat foo.c
int main()
{
char *p = "foo\\
bar";
return 0;
}
$ gcc -E foo.c
# 1 "foo.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "foo.c"
int main()
{
char *p = "foo\bar";
return 0;
}
$
From my understanding the 2nd \ is escaped by the 1st \ so the 2nd \ should not be combined with the following <NEWLINE> to form the line continuation.
The rules are quite explicit in ISO/IEC 9899:2011 §5.1.1.2 Translation Phases:
Each instance of a backslash character (\) immediately followed by a new-line
character is deleted, splicing physical source lines to form logical source lines.
Only the last backslash on any physical source line shall be eligible for being part
of such a splice.
The character preceding the final backslash is not consulted. Phase 1 converts trigraphs into regular characters. That matters because ??/ is the trigraph for \.
The preprocessor removes all occurrences of backslash-newline before even trying to tokenize the input; there is no escape mechanism for this. It's not limited to string literals either:
#inclu\
de <st\
dio.h>
int m\
ain(void) {
/\
* yes, this is a comment */
put\
s("Hello,\
world!");
return 0;
}
This is valid code.
Using \\ to get a single \ only applies to string and character literals and happens much later in processing.
Related
I wanted to demonstrate that the preprocessor is totally independant of the build process. It is another grammar and another lexer than the C language. In fact I wanted to show that the preprocessor could be applied to any type of file.
So I have this arbitrary file:
#define FOO
#ifdef FOO
I am foo
#endif
#
#
#
Something
#pragma Hello World
And I thought this would work:
$ gcc -E test.txt -o-
gcc: warning: test.txt: linker input file unused because linking not done
Unfortunately it only work with this:
$ cat test.txt | gcc -E -
Why is this error with GCC?
You need to tell gcc it's a C file.
gcc -xc -E test.txt
The C compiler uses the file name suffix as an indicator of the files that have to be compiled (ending in .c) files that have only to be linked (ending in .o or .so) For the files ending in .s it calls the assembler as(1) and for files ending in .f it call the fortran compiler, and for .cc it switches to C++ compiling.
Indeed, normally, C compilers take everything they don't match as a linker file, so once you pass it a linker file, it tries to link it, calling the linker ld(1). This is what happens with your .txt file. The linker has some similar way to recognise ld(1) scripts against object or shared object files.
BTW, the CPP language is indeed a macro language, but there's some similarities with C that cannot be avoided. It has, at least, to recognise C identifiers, as macro names have the same syntax as C identifiers, and it has to check that an identifier matches a macro name or not. In other side... It has to recognise C comments, and C strings (it indeed eliminates comments for the compiler), as macro expansion doesn't enter to expand inside them, and it has also to recognize parenthesis (they are considered for macro parameter detection and the , symbol, used to separate parameters). It also recognizes (inside the macro string) the tokens # (to stringify a parameter) and ## (to catenate and merge two symbols into one) (this last operator must force cpp to recognise almost any C token, as it must check for errors if you try to merge something like +##+ into ++, which is an error)
So, the conclussion is: the cpp doesn't have to implement the whole C syntax as a language, but the tokens of the C language must be recognised almost completely. The standard for the C language forces the c preprocessor to tokenize the input, so the ## operator can be used to merge tokens (and to check for validity) This means that, if you define a macro like:
#define M(p) +p
and then you call it like:
a = +M(-c);
you will get a string similar to:
a = + +-c;
in the output (it will insert a space in between the two + signs, so they don't get merged into ++ operator. The symbols + and - are together, because they will never be scanned as one token) See the next example (input is preceded by > symbol)
$ cpp - <<EOF
> #define M(p) +p
> a = +M(p);
> b = -M(p);
> p = +M(+p);
> p = +M(-p);
> EOF
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 346 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2
a = + +p;
b = -+p;
p = + + +p;
p = + +-p;
Another example will show more difficulties in parsing the tokens (input is delimited with >, stderr with >> and stdout is unquoted):
$ cpp - <<EOF
#define M(a,b) a##b
> a = M(a+,+b)
> a = M(a+,-b)
> a = M(a,+b)
> a = M(a,b)
> a = M(a,300)
> a = M(a,300.2)
> EOF
>> <stdin>:3:5: error: pasting formed '+-', an invalid preprocessing token
>> a = M(a+,-b)
>> ^
>> <stdin>:1:17: note: expanded from macro 'M'
>> #define M(a,b) a##b
>> ^
>> <stdin>:4:5: error: pasting formed 'a+', an invalid preprocessing token
>> a = M(a,+b)
>> ^
>> <stdin>:1:17: note: expanded from macro 'M'
>> #define M(a,b) a##b
>> ^
>> <stdin>:7:5: error: pasting formed 'a300.2', an invalid preprocessing token
>> a = M(a,300.2)
>> ^
>> <stdin>:1:17: note: expanded from macro 'M'
>> #define M(a,b) a##b
>> ^
>> 3 errors generated.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 346 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2
a = a++b
a = a+-b
a = a+b
a = ab
a = a300
a = a 300.2
As you can see in this example, merging a and 300 goes fine, as one token makes an identifier, which is valid and cpp(1) doesn't complain, but when merging a and 300.2 the resulting token a300.2 is not a valid token in C, so it is rejected (it is also not joined and the tool inserts a space, to make the compiler see both tokens as separate ---should it joined both together, they would have been scanned as the tokens a300 and .2).
If you want to use a language independent macro preprocesor, consider using m4(1) as a macro language. It's far more powerful than cpp in many ways. But beware, it's difficult to learn due to the complexity of macro expansions it allows.
You can use the C preprocessor, cpp (or the more traditional form, /lib/cpp):
cpp test.txt
or
/lib/cpp test.txt
I started coding C in vim and I have some problems.
The backslash is intended to join lines but when I try to write:
ret\
urn 0;
I get
return
0;
and when I add spaces before urn; it stay like that without join.
ret\
urn 0;
it stay like that.
why in the second case I don't get return 0; but
ret
urn 0;
code:
CPP output:
command:
gcc -E -Wall -Wextra -Wimplicit -pedantic -std=c99 main.c -o output.i
GCC 5.4,
Vim 7.4
-E output is not officially specified by the standard. It's an engineering tradeoff among several different design constraints, of which the relevant two are:
whitespace must be inserted or deleted as necessary so that the "compiler proper" (imagine feeding the -E output back into gcc -fpreprocessed — which is what -save-temps does) sees the same sequence of pp-tokens that it would normally (without -E). (See C99 section 6.4 for the definition of a pp-token.)
to the maximum extent possible, tokens should appear at the same line and column position that they did in the original source code, so that error messages and debugging information are as accurate as possible.
Here's how this applies to your examples:
ret\
urn 0;
The backslash-newline combines ret and urn into a single pp-token, which must therefore appear all together on one line in the output. The 0 and ;, however, should continue to be on their original line and column so that diagnostics are accurate. So you get
return
0;
with spaces inserted to keep the 0 in its original column.
ret\
urn 0;
Here the backslash-newline is immediately followed by whitespace, so ret and urn do not have to be combined, so, again, the diagnostics are most accurate if everything stays where it originally was, and the output is
ret
urn 0;
which looks like the backslash-newline had no effect at all.
You might find the output of gcc -E -P less surprising. -P tells the preprocessor not to bother trying to preserve token position (and also turns off all those lines beginning with # in the output). Your examples produce return 0; and ret urn 0;, both all on one line, in -P mode.
Finally, a word of advice: everyone who ever has to read your code (and that includes yourself six months later) will appreciate it if you never split a token in the middle with backslash-newline, except for very long string literals. It's a legacy misfeature that wouldn't be included in the language if it were designed from scratch today.
The white space is a token separator. Just because you split the line doesn't mean a white space will be ignored.
What the compiler sees is something like ret urn;. Which is not valid C, since it's two tokens which probably weren't defined before, nor are they in a valid expression.
Keywords must be written as a single token with no spaces.
Now, when you do :
ret\
urn;
The backslash followed by a newline is removed in the early translation phases, and the subsequent line is appended. If the line has no white spaces at the beginning, the result is a valid token that the compiler understands as the keyword return.
Long story short, you seem to be asking about specific behavior for GCC. It seems like a compiler bug. Since clang does the expected thing (although the line count remains the same):
clang -E -Wall -Wextra -Wimplicit -pedantic -std=c99 -x c main.cpp
# 1 "main.cpp"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 316 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "main.cpp" 2
int main(void) {
ret urn 0;
}
It doesn't seem crucial however, since in this particular case the code will be invalid either way.
The behavior of the C preprocessor on \ followed by a newline is to remove both bytes from the input. This is done in a very early phase of the parsing. Yet the preprocessor retains the original line number for each token it sees and tries to output tokens on separate lines for the compiler to issue correct diagnostics for later phases of compilation.
For the input:
ret\
urn 1;
it may produce:
#line 1 "myfile.c"
return
#line 2 "myfile.c"
1;
Which it may shorten as
return
1;
Note that you can split any input line at any position with an escaped newline:
#inclu\
de <st\
dio.h>\
"Hello word\\
n"
for (i = 0; i < n; i+\
+)
ret\
\
\
urn;
\
r\
et\
urn\
123;\
I wrote this code that creates identifiers containing universal character names via token concatenation.
//#include <stdio.h>
int printf(const char*, ...);
#define CAT(a, b) a ## b
int main(void) {
//int \u306d\u3053 = 10;
int CAT(\u306d, \u3053) = 10;
printf("%d\n", \u306d\u3053);
//printf("%d\n", CAT(\u306d, \u3053));
return 0;
}
This code worked well with gcc 4.8.2 with -fextended-identifiers option and gcc 5.3.1, but didn't work with clang 3.3 with error message:
prog.c:10:17: error: use of undeclared identifier 'ねこ'
printf("%d\n", \u306d\u3053);
^
1 error generated.
and local clang (Apple LLVM version 7.0.2 (clang-700.1.81)) with error message:
$ clang -std=c11 -Wall -Wextra -o uctest1 uctest1.c
warning: format specifies type 'int' but the argument has type
'<dependent type>' [-Wformat]
uctest1.c:10:17: error: use of undeclared identifier 'ねこ'
printf("%d\n", \u306d\u3053);
^
1 warning and 1 error generated.
When I used -E option to have the compilers output code with macro expanded, gcc 5.3.1 emitted this:
# 1 "main.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 1 "<command-line>" 2
# 1 "main.c"
int printf(const char*, ...);
int main(void) {
int \U0000306d\U00003053 = 10;
printf("%d\n", \U0000306d\U00003053);
return 0;
}
local clang emitted this:
# 1 "uctest1.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 326 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "uctest1.c" 2
int printf(const char*, ...);
int main(void) {
int \u306d\u3053 = 10;
printf("%d\n", ねこ);
return 0;
}
As you see, the identifiers declared and used in printf() matches in gcc's output, but they don't match in clang's output.
I know that creating universal character names via token concatenation invokes undefined behavior.
Quote from N1570 5.1.1.2 Translation phases:
If a character sequence that
matches the syntax of a universal character name is produced by token
concatenation (6.10.3.3), the behavior is undefined.
I thought that this character sequence \u306d\u3053 may "match the syntax of a universal character name" because it contains universal character names as its substring.
I also thought that "match" may mean that the entire token produced via concatenation stands for one universal character name, and that therefore this undefined behavior isn't invoked in this code.
Reading PRE30-C. Do not create a universal character name through concatenation, I found a comment saying this kind of concatenation is allowed:
What is forbidden, to create a new UCN via concatenation. Like doing
assign(\u0001,0401,a,b,4)
just concatenating stuff that happens to contain UCNs anywhere is okay.
And a log that shows that a code example like this case (but with 4 characters) is replaced with another code example.
Does my code example invoke some undefined behaviors (not limited to ones invoked by producing universal character names via token concatenation)?
Or is this a bug in clang?
Your code is not triggering the undefined behavior you mention, as universal character name (6.4.3) not being produced by token concatenation.
And, according to 6.10.3.3, as both the left side and the right side of operator ## is an identifier, and the produced token is also a valid preprocessing token (an identifier too), the ## operator itself not trigger an undefined behavior.
After reading description about identifier (6.4.2, D.1, D.2), universal character names (6.4.3), I'm pretty sure that it is more like a bug in clang preprocessor, which treats identifier produced by token concatenation and normal identifier differently.
This question already has answers here:
What is the meaning of lines starting with a hash sign and number like '# 1 "a.c"' in the gcc preprocessor output?
(3 answers)
Closed 7 years ago.
These are the first few lines of the pre-processor output of a simple C program. What do they mean?
# 1 "test.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 325 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "test.c" 2
# 1 "some_path/stdio.h" 1 3 4
# 64 "some_path/stdio.h" 3 4
Here's my program:
#include <stdio.h>
int main()
{
printf("Hello, World!\n");
return 0;
}
# linenum filename flags
These are called linemarkers. They are inserted as needed into the output (but never within a string or character constant). They mean that the following line originated in file filename at line linenum. filename will never contain any non-printing characters; they are replaced with octal escape sequences.
After the file name comes zero or more flags, which are ‘1’, ‘2’, ‘3’, or ‘4’. If there are multiple flags, spaces separate them. Here is what the flags mean:
‘1’ This indicates the start of a new file.
‘2’ This indicates returning to a file (after having included another file).
‘3’ This indicates that the following text comes from a system header file, so certain warnings should be suppressed.
‘4’ This indicates that the following text should be treated as being wrapped in an implicit extern "C" block.
Source: GCC Manual
alright, i understand that the title of this topic sounds a bit gibberish... so i'll try to explain it as clearly as i can...
this is related to this previous post (an approach that's been verified to work):
multipass a source code to cpp
-- which basically asks the cpp to preprocess the code once before starting the gcc compile build process
take the previous post's sample code:
#include <stdio.h>
#define DEF_X #define X 22
int main(void)
{
DEF_X
printf("%u", X);
return 1;
}
now, to be able to freely insert the DEF_X anywhere, we need to add a newline
this doesn't work:
#define DEF_X \
#define X 22
this still doesn't work, but is more likely to:
#define DEF_X \n \
#define X 22
if we get the latter above to work, thanks to C's free form syntax and constant string multiline concatenation, it works anywhere as far as C/C++ is concerned:
"literal_str0" DEF_X "literal_str1"
now when cpp preprocesses this:
# 1 "d:/Projects/Research/tests/test.c"
# 1 "<command-line>"
# 1 "d:/Projects/Research/test/test.c"
# 1 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h" 1 3
# 19 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h" 3
# 1 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/_mingw.h" 1 3
# 32 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/_mingw.h" 3=
# 33 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/_mingw.h" 3
# 20 "c:\\mingw\\bin\\../lib/gcc/mingw32/4.7.2/../../../../include/stdio.h" 2 3
ETC_ETC_ETC_IGNORED_FOR_BREVITY_BUT_LOTS_OF_DECLARATIONS
int main(void)
{
\n #define X 22
printf("%u", X);
return 1;
}
we have a stray \n in our preprocessed file. so now the problem is to get rid of it....
now, the unix system commands aren't really my strongest suit. i've compiled dozens of packages in linux and written simple bash scripts that simply enter multiline commands (so i don't have to type them every time or keep pressing the up arrow and choose the correct command successions). so i don`t know the finer points of stream piping and their arguments.
having said that, i tried these commands:
cpp $MY_DIR/test.c | perl -p -e 's/\\n/\n/g' > $MY_DIR/test0.c
gcc $MY_DIR/test0.c -o test.exe
it works, it removes that stray \n.
ohh, as to using perl rather than sed, i'm just more familiar with perl's variant to regex... it's more consistent in my eyes.
anyways, this has the nasty side effect of eating up any \n in the file (even in string literals)... so i need a script or a series of commands to:
remove a \n if:
if it is not inside a quote -- so this won't be modified: "hell0_there\n"
not passed to a function call (inside the argument list)
this is safe as one can never pass a single \n, which is neither a keyword nor an identifier.
if i need to "stringify" an expression with \n, i can simply call a function macro QUOTE_VAR(token). so that encapsulates all instances that \n would have to be treated as a string.
this should cover all cases that \n should be substituted... at least for my own coding conventions.
really, i would do this if i could manage it on my own... but my skills in regex is extremely lacking, only using it in for simple substitutions.
The better way is to replace \n if it occurs in the beginning of line.
The following command should do the work:
sed -e 's/\s*\\n/\n/g'
or occurs before #
sed -e 's/\\n\s*#/\n#/g'
or you can reverse the order of preprocessing and substitute DEF_X with your own tool before C preprocessor.