I'm trying to make a code-section reusable. My comment snippet below isn't doing what I want it to:
#define NAME ABC
#define LOG_SIZE NAME##_LEN
I would like LOG_SIZE to resolve to ABC_LEN. I've tried playing around with the #'s, but haven't been able to get this to work. LOG_SIZE is used all over the code, so I don't want to change the macro to:
#define LOG_SIZE(name) name##_LEN
Is there a way to do this?
The problem is that macro arguments aren't automatically expanded if they would be stringified or concatenated to another token.
C99 6.10.3.1/1:
After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list, unless preceded by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file; no other preprocessing tokens are available.
You can get around this by adding another macro in between the one that passes NAME and the one that concatenates it with _LEN.
#define NAME ABC
#define AFTERX(x) x##_LEN
#define XAFTERX(x) AFTERX(x)
#define LOG_SIZE XAFTERX(NAME)
LOG_SIZE
//evaluates to ABC_LEN
The gcc manual goes into further detail if you're curious, in Section 3.10.6: Argument Prescan
Related
You can iterate over a preprocessor sequence using the following construct:
#define A() B
#define B() A
A()()()()()
Expands to B on most compilers/preprocessor: clang, gcc, tcc, chibicc, SDCC (I couldn't test msvc, because it didn't work on godbolt, but if you want to test it make sure to use the /Zc:preprocessor flag, because otherwise the preprocessor will be non conforment).
Reading 6.10.3.4 seems to suggest, that the expansion of B happens inside A, which would cause the second expansion of A not to happen, rather it would be painted blue, and the expansion would stop.
6.10.3.4 Rescanning and further replacement
After all parameters in the replacement list have been substituted and # and ## processing has taken place, all placemarker preprocessing tokens are removed. The resulting preprocessing token sequence is then rescanned, along with all subsequent preprocessing tokens of the source file, for more macro names to replace.
But Annex J.1 says that whether this is done using nesting or not is unspecified behavior:
When a fully expanded macro replacement list contains a function-like macro name as its last preprocessing token and the next preprocessing token from the source file is a (, and the fully expanded replacement of that macro ends with the name of the first macro and the next preprocessing token from the source file is again a (, whether that is considered a nested replacement (6.10.3).
Ok, fair, so most preprocessor use the non nesting approach, but what allows the following to work?
#define A() B(
#define B() A(
A()))))
Now granted the former will give you an error, for a "unterminated argument list invoking macro 'B'", but wouldn't you expect this to expand to A())), where A is now painted blue, which shouldn't give an error?
And further, you can get rid of the error by detecting the last closing parentheses, showing that this does also not seem to use nesting, which is weird, because where does the standard suggest that this is valid?
There is already a similar question on SO, but I don't see how the answer has anything to do with the question, since the passage quoted is only talking about argument substitution:
6.10.3.1 Argument substitution
After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter
in the replacement list, unless preceded by a # or ## preprocessing
token or followed by a ## preprocessing token (see below), is replaced
by the corresponding argument after all macros contained therein have
been expanded. Before being substituted, each argument’s preprocessing
tokens are completely macro replaced as if they formed the rest of the
preprocessing file; no other preprocessing tokens are available.
Which makes sense, so e.g. in #define A(x) x x x the argument x passed to A would only need to be expanded once in insolation and afterwards the resulting tokens are inserted in place of the occurrences of x in the expansion list.
This also explains the following behavior:
#define STR(x) #x
#define f(x) x
#define F(x) STR(x(23))
F(f) // expands to "f(23)"
So the in isolation part refers to the arguments them self and not what happens in the rescanned, that is detailed in 6.10.3.4, which is my initial standard quote.
So what is going on here, how should I think about the macro expansion process?
From my reading of DR17, if the ) is joined with the result of expansion on the left of it has been intentionally left unspecified in the standard. The behavior is undefined. Strictly conforming programs shouldn't use this.
Why is the macro-name not painted blue? ,
https://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_017.html ,
https://port70.net/%7Ensz/c/c11/n1570.html#6.10.3.4p4
I'm looking to replace some tokens within a called macro but can't seem to determine the right ordering of expansion and\or deferral. For example:
#define EXPAND(...) __VA_ARGS__
#define REPLACE(hello,y) EXPAND(y)
REPLACE(goodbye, hello world)
In my mind the REPLACE macro would call the EXPAND macro, making it functionally identical to:
#define REPLACE(hello,y) hello world
Allowing the hello world to be transformed into goodbye world.
My compiler (MSVC 2017) doesn't seem to be doing that, so I suspect that I'm in the wrong here. I've read up on expansion and deferral and have tried many different combinations of DEFER() and EXPAND(), but none seem to give the result I'm after.
Does anyone have any insight into what I'm doing wrong?
That is not how macro parameters are handled, and for a reason. If the use of a macro parameter name in the arguments to a macro could be replaced, then it would be impossible to write safe macros: accidentally using the name of a macro parameter would cause chaos, and there is no reason why a macro caller needs to know what the names of the parameters are. Macro parameters are local to the macro expansion, similar to the way that function parameters are local to the body of the function.
Here's the actual substitution algorithm, from §6.10.3.1/1 [Argument Substitution] of the C standard:
After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list… is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file; no other preprocessing tokens are available.
Note that the arguments are macro replaced before being placed into the macro expansion. Once that is done, the parameter names in the replacement list are no longer relevant, and are not part of the replaced text.
Once the macro invocation has been replaced with its expansion, the resulting tokens are then scanned again (§6.10.3.4: "The resulting preprocessing token sequence is then rescanned, along with all subsequent preprocessing tokens of the source file, for more macro names to replace."). However, since the macro invocation has been completely replaced prior to this rescan, the parameter tokens no longer appear.
So this particular solution to your problem is a dead-end. I recommend that you back up a step and focus on the problem you actually wish to solve.
I want to use a variadic macro but it appears to be designed to only treat the first parameter specially. I want the first two parameters to be named and the rest not, like so:
#define FOO(AA,BB,...) AA->BB(AA,##...)
FOO(mystruct,funcname,123)
However this is not working with LLVM. Am I doing something wrong, or is there a limitation to how the variadic macro works?
UPDATE
The correct answer is, use ##VA_ARGS instead of ##...
There are some webpages that claim that "..." is valid but at least with the MacOS llvm it is not.
The macro arguments are not expanded with ... in the macro expansion - how could they, because then you couldn't have a macro that used ellipsis in the expansion. Instead it will be available as a special parameter __VA_ARGS__.
With this, the following program
#define FOO(AA,BB,...) AA->BB(AA, __VA_ARGS__)
FOO(mystruct,funcname,123)
FOO(mystruct,funcname,123,456)
will be preprocessed to
The ## is a token-pasting operator. It will make a single preprocessing token out of 2 parts. , ## ... attempts to make a preprocessing token ,.... It is not a valid C token, and that is why Clang will report
<source>:3:1: error: pasting formed ',...', an invalid preprocessing token
... macro arguments are pasted into macro bodies with __VA_ARGS__.
The problem is how to allow for it to be empty.
If it is empty, you'll usually want to comma before it erased and
you can use the GNU ##__VA_ARGS__ extension to achieve that.
#define FOO(AA,BB,...) AA->BB(AA,##__VA_ARGS__) /*GNU extension*/
FOO(mystruct,funcname) //warning with -pedantic
FOO(mystruct,funcname,123)
The above, however, will trigger warnings if compiled with -pedantic.
If you want your macro usable without warnings at -pedantic, you could perhaps achieve that by swapping the first two arguments in the macro definition.
#define FIRST(...) FIRST_(__VA_ARGS__,)
#define FIRST_(X,...) X
#define BAR_(CallExpr,...) CallExpr(__VA_ARGS__)
#define BAR(BB,/*AA,*/...) BAR_(FIRST(__VA_ARGS__)->BB,__VA_ARGS__)
BAR(funcname,mystruct) //no warning
BAR(funcname,mystruct,123)
The code I'm working on uses some very convoluted macro voodoo in order to generate code, but in the end there is a construct that looks like this
#define ARGS 1,2,3
#define MACROFUNC_OUTER(PARAMS) MACROFUNC_INNER(PARAMS)
#define MACROFUNC_INNER(A,B,C) A + B + C
int a = MACROFUNC_OUTER(ARGS);
What is expected is to get
int a = 1 + 2 + 3;
This works well for the compiler it has originally been written for (GHS) and also for GCC, but MSVC (2008) considers PARAMS as a single preprocessing token that it won't expand, setting then A to the whole PARAM and B and C to nothing. The result is this
int a = 1,2,3 + + ;
while MSVC warns that not enough actual parameters for macro 'MACROFUNC_INNER'.
Is it possible to get MSVC do the expansion with some tricks (another layer of macro to force a second expansion, some well placed ## or #, ...). Admitting that changing the way the construct work is not an option. (i.e.: can I solve the problem myself?)
What does the C standard say about such corner case? I couldn't find in the C11 norm anything that explicitly tells how to handle arguments that contains a list of arguments. (i.e.: can I argue with the author of the code that he has to write it again, or is just MVSC non-conform?)
MSVC is non-conformant. The standard is actually clear on the point, although it does not feel the need to mention this particular case, which is not exceptional.
When a function-like macro invocation is encountered, the preprocessor:
§6.10.3/11 identifies the arguments, which are possibly empty sequences of tokens separated by non-protected commas , (a comma is protected if it is inside parentheses ()).
§6.10.3.1/1 does a first pass over the macro body, substituting each parameter which is not used in a # or ## operation with the corresponding fully macro-expanded argument. (It does no other substitutions in the macro body in this step.)
§6.10.3.4/1 rescans the substituted replacement token sequence, performing more macro replacements as necessary.
(The above mostly ignores stringification (#) and token concatenation (##), which are not relevant to this question.)
This order of operations unambiguously leads to the behaviour expected by whoever wrote the software.
Apparently (according to #dxiv, and verified here) the following standards-compliant workaround works on some versions of MS Visual Studio:
#define CALL(A,B) A B
#define OUTER(PARAM) CALL(INNER,(PARAM))
#define INNER(A,B,C) whatever
For reference, the actual language from the C11 standard, skipping over the references to # and ## handling:
§6.10.3 11 The sequence of preprocessing tokens bounded by the outside-most matching parentheses forms the list of arguments for the function-like macro. The individual arguments within the list are separated by comma preprocessing tokens, but comma preprocessing tokens between matching inner parentheses do not separate arguments.…
§6.10.3.1 1 After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list… is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file…
§6.10.3.4 1 After all parameters in the replacement list have been substituted… [t]he resulting preprocessing token sequence is then rescanned, along with all subsequent preprocessing tokens of the source file, for more macro names to replace.
C11 says that each appearance of an object-like macro's name
[is] replaced by the replacement list of preprocessing tokens that constitute the remainder of the directive. The replacement list is then rescanned for more macro names as specified below.
[6.10.3/9]
Of function-like macros it says this:
If the identifier-list in the macro definition does not end with an ellipsis, the number of arguments [...] in an invocation of a function-like macro shall equal the number of parameters in the macro definition.
[6.10.3/4]
and this:
The sequence of preprocessing tokens bounded by the outside-most matching parentheses forms the list of arguments for the function-like macro.
[6.10.3/11]
and this:
After the arguments for the invocation of a function-like macro have been identified, argument substitution takes place. A parameter in the replacement list [...] is replaced by the corresponding argument after all macros contained therein have been expanded. Before being substituted, each argument’s preprocessing tokens are completely macro replaced as if they formed the rest of the preprocessing file; no other preprocessing tokens are available.
[6.10.3.1/1]
Of macros in general it also says this:
After all parameters in the replacement list have been substituted [... t]he resulting preprocessing token sequence is then rescanned, along with all subsequent preprocessing tokens of the source file, for more macro names to replace.
[6.10.3.4/1]
MSVC++ does not properly expand the arguments to function-like macros before rescanning the expansion of such macros. It seems unlikely that there is any easy workaround.
UPDATE:
In light of #dxiv's answer, however, it may be that there is a solution after all. The problem with his solution with respect to standard-conforming behavior is that there needs to be one more expansion than is actually performed. That can easily enough be supplied. This variation on his approach works with GCC, as it should, and inasmuch as it is based on code that dxiv claims works with MSVC++, it seems likely to work there, too:
#define EXPAND(x) x
#define PAREN(...) (__VA_ARGS__)
#define EXPAND_F(m, ...) EXPAND(m PAREN(__VA_ARGS__))
#define SUM3(a,b,c) a + b + c
#define ARGS 1,2,3
int sum = EXPAND_F(SUM3, ARGS);
I have of course made it a little more generic than perhaps it needs to be, but that may serve you well if you have a lot of these to deal with..
Curiuosly enough, the following appears to work in MSVC (tested with 2010 and 2015).
#define ARGS 1,2,3
#define OUTER(...) INNER PARAN(__VA_ARGS__)
#define PARAN(...) (__VA_ARGS__)
#define INNER(A,B,C) A + B + C
int a = OUTER(ARGS);
I don't know that it's supposed to work by the letter of the standard, in fact I have a hunch it's not. Could still be conditionally compiled just for MSVC, as a workaround.
[EDIT] P.S. As pointed out in the comments, the above is (another) non-standard MSVC behavior. Instead, the alternative workarounds posted by #rici and #JohnBollinger in the respective replies are compliant, thus recommended.
What is the correct output of preprocessing the following 3 lines under the C99 rules?
#define y(x) x
#define x(a) y(a
x(1) x(2)))
BTW cpp under linux produces an error message, but I can't see why the answer isn't simply
1 2
Assuming cpp is correct and I'm wrong, I'd be very grateful for an explanation.
When a macro is found, the preprocessor gathers up the arguments to the macro and then scans each macro argument in isolation for other macros to expand within the argument BEFORE the first macro is expanded:
6.10.3.1 Argument substitution
After the arguments for the invocation of a function-like macro have been identified,
argument substitution takes place. A parameter in the replacement list, unless preceded
by a # or ## preprocessing token or followed by a ## preprocessing token (see below), is
replaced by the corresponding argument after all macros contained therein have been
expanded. Before being substituted, each argument’s preprocessing tokens are
completely macro replaced as if they formed the rest of the preprocessing file; no other
preprocessing tokens are available.
So in this specific example, it sees x(1) and expands that, giving
y(1 x(2)))
It then identifies the macro call y(1 x(2)), with the argument 1 x(2) and prescans that for macros to expand. Within that it finds x(2) which expands to y(2 and then triggers the error due to there not being a ) for the y macro. Note at this point its still looking to expand the argument of the first y macro, so its looking at it in isolation WITHOUT considering the rest of the input file, unlike the expansion that takes place for 6.10.3.4
Now there's some question as to whether this should actually be an error, or if the preprocessor should treat this y(2 sequence as not being a macro invocation at all, as there is no ')'. If it does the latter then it will expand that y call to 1 y(2 which will then be combined with the rest of the input ()) and ultimately expand to 1 2
After a macro is expanded, attempts to expand macros in the resulting text occur in isolation before it is combined with the surrounding text. Thus the attempt to expand y(1 gives this error. It would actually be very difficult to specify macro expansion that works the way you want, while still meeting lots of the other required behaviors (such as lack of infinite recursion).