I stumbled on some C++ code like this:
int $T$S;
First I thought that it was some sort of PHP code or something wrongly pasted in there but it compiles and runs nicely (on MSVC 2008).
What kind of characters are valid for variables in C++ and are there any other weird characters you can use?
The only legal characters according to the standard are alphanumerics
and the underscore. The standard does require that just about anything
Unicode considers alphabetic is acceptable (but only as single
code-point characters). In practice, implementations offer extensions
(i.e. some do accept a $) and restrictions (most don't accept all of the
required Unicode characters). If you want your code to be portable,
restrict symbols to the 26 unaccented letters, upper or lower case, the
ten digits, and the '_'.
It's an extension of some compilers and not in the C standard
MSVC:
Microsoft Specific
Only the first 2048 characters of Microsoft C++ identifiers are significant. Names for user-defined types are "decorated" by the compiler to preserve type information. The resultant name, including the type information, cannot be longer than 2048 characters. (See Decorated Names for more information.) Factors that can influence the length of a decorated identifier are:
Whether the identifier denotes an object of user-defined type or a type derived from a user-defined type.
Whether the identifier denotes a function or a type derived from a function.
The number of arguments to a function.
The dollar sign is also a valid identifier in Visual C++.
// dollar_sign_identifier.cpp
struct $Y1$ {
void $Test$() {}
};
int main() {
$Y1$ $x$;
$x$.$Test$();
}
https://web.archive.org/web/20100216114436/http://msdn.microsoft.com/en-us/library/565w213d.aspx
Newest version: https://learn.microsoft.com/en-us/cpp/cpp/identifiers-cpp?redirectedfrom=MSDN&view=vs-2019
GCC:
6.42 Dollar Signs in Identifier Names
In GNU C, you may normally use dollar signs in identifier names. This is because many traditional C implementations allow such identifiers. However, dollar signs in identifiers are not supported on a few target machines, typically because the target assembler does not allow them.
http://gcc.gnu.org/onlinedocs/gcc/Dollar-Signs.html#Dollar-Signs
In my knowledge only letters (capital and small), numbers (0 to 9) and _ are valid for variable names according to standard (note: the variable name should not start with a number though).
All other characters should be compiler extensions.
This is not good practice. Generally, you should only use alphanumeric characters and underscores in identifiers ([a-z][A-Z][0-9]_).
Surface Level
Unlike in other languages (bash, perl), C does not use $ to denote the usage of a variable. As such, it is technically valid. In C it most likely falls under C11, 6.4.2. This means that it does seem to be supported by modern compilers.
As for your C++ question, lets test it!
int main(void) {
int $ = 0;
return $;
}
On GCC/G++/Clang/Clang++, this indeed compiles, and runs just fine.
Deeper Level
Compilers take source code, lex it into a token stream, put that into an abstract syntax tree (AST), and then use that to generate code (e.g. assembly/LLVM IR). Your question really only revolves around the first part (e.g. lexing).
The grammar (thus the lexer implementation) of C/C++ does not treat $ as special, unlike commas, periods, skinny arrows, etc... As such, you may get an output from the lexer like this from the below c code:
int i_love_$ = 0;
After the lexer, this becomes a token steam like such:
["int", "i_love_$", "=", "0"]
If you where to take this code:
int i_love_$,_and_.s = 0;
The lexer would output a token steam like:
["int", "i_love_$", ",", "_and_", ".", "s", "=", "0"]
As you can see, because C/C++ doesn't treat characters like $ as special, it is processed differently than other characters like periods.
It's common in C (and other languages) to use prefixes and suffixes for names of variables and functions. Particularly, one occasionally sees the use of underscores, before or after a "proper" identifier, e.g. _x and _y variables, or _print etc. But then, there's also the common wisdom of avoiding names starting with underscore, so as to not clash with the C standard library implementation.
So, where and where is it ok to use underscores?
Good-enough rule of thumb
Don't start your identifier with an underscore.
That's it. You might still have a conflict with some file-specific definitions (see below), but those will just get you an error message which you can take care of.
Safe, slightly restrictive, rule of thumb
Don't start your identifier with:
An underscore.
Any 1-3 letter prefix, followed by an underscore, which isn't a proper word (e.g. a_, st_)
memory_ or atomic_.
and don't end your identifier with either _MIN or _MAX.
These rules forbid a bit more than what is actually reserved, but are relatively easy to remember.
More detailed rules
This is based on the C2x standard draft (and thus covers previous standards' reservations) and the glibc documentation.
Don't use:
The prefix __ (two underscores).
A prefix of one underscore followed by a capital letter (e.g. _D).
For identifiers visible at file scope - the prefix _.
The following prefixes with underscores, when followed by a lowercase letter: atomic_, memory_, memory_order_, cnd_, mtx_, thrd_, tss_
The following prefixes with underscores, when followed by an uppercase ltter : LC_, SIG_, ATOMIC, TIME_
The suffix _t (that's a POSIX restriction; for C proper, you can use this suffix unless your identifier begins with int or uint)
Additional restrictions are per-library-header-file rather than universal (some of these are POSIX restrictions):
If you use header file...
You can't use identifiers with ...
dirent.h
Prefix d_
fcntl.h
Prefixes l_, F_, O_, and S_
grp.h
Prefix gr_
limits.h
Suffix _MAX (also probably _MIN)
pwd.h
Prefix pw_
signal.h
Prefixes sa_ and SA_
sys/stat.h
Prefixes st_ and S_
sys/times.h
Prefix tms_
termios.h
Prefix c_
And there are additional restrictions not involving underscores of course.
The C standard, library chapter, reserves certain identifiers (emphasis mine):
C17 7.1.3 Reserved identifiers
— All identifiers that begin with an underscore and either an uppercase letter or another
underscore are always reserved for any use.
— All identifiers that begin with an underscore are always reserved for use as identifiers
with file scope in both the ordinary and tag name spaces.
— Each macro name in any of the following subclauses (including the future library
directions) is reserved for use as specified if any of its associated headers is included;
unless explicitly stated otherwise (see 7.1.4).
— All identifiers with external linkage in any of the following subclauses (including the
future library directions) and errno are always reserved for use as identifiers with
external linkage.184)
— Each identifier with file scope listed in any of the following subclauses (including the
future library directions) is reserved for use as a macro name and as an identifier with
file scope in the same name space if any of its associated headers is included.
Where "reserved for any use" means reserved for the compiler/standard library, see What's the meaning of "reserved for any use"? "Reserved for the implementation" also means reserved for the compiler/standard library.
Furthermore, Future library directions C17.31 reserve a lot of identifiers - it's a big chapter, I'll only quote the most notable parts:
7.31.10 Integer types <stdint.h>
Typedef names beginning with int or uint and ending with _t may be added to the
types defined in the <stdint.h> header. Macro names beginning with INT or UINT
and ending with _MAX, _MIN, or _C may be added to the macros defined in the
<stdint.h> header.
7.31.12 General utilities <stdlib.h>
Function names that begin with str and a lowercase letter may be added to the
declarations in the <stdlib.h> header.
7.31.13 String handling <string.h>
Function names that begin with str, mem, or wcs and a lowercase letter may be added to the declarations in the <string.h> header.
To answer your question directly:
So, where and where is it ok to use underscores?
Strictly speaking: nowhere. You should never declare identifiers starting with underscore, since they may clash with the standard library or language keywords etc. Though as is hinted from the bold text above, you may use one underscore followed by lower case in a local namespace.
At #Zaibis suggestion (and related to my own answer to What are the valid characters for macro names?, as well as 😃 (and other unicode characters) in identifiers not allowed by g++))...
clang allows a lot of "crazy" characters.. although I have struggled to find much rhyme or reason - as to why some are allowed (🔴 ϟ ツ ⌘ ☁ ½), and others are not (▶︎ ∀ ★ ©).
For example, the following all compile A-OK (clang-700.1.76)
#define 💩 ?: // OK (Pile of poo)
#define ￭ #end // OK (HALFWIDTH BLACK SQUARE)
#define 🅺 #interface // OK (NEGATIVE SQUARED LATIN CAPITAL LETTER K)
#define Ｐ #protocol // OK (FULLWIDTH LATIN CAPITAL LETTER P)
yet the following all result in the same compiler error...
Macro name must be an identifier.
#define ☎ TEL
#define ❌ NO
#define ⇧ UP
#define 〓 ==
#define 🍎 APPLE
clang's docs refer to the issue, stating only...
... support for extended identifiers in C99 and C++. This feature allows identifiers to contain certain Unicode characters, as specified by the active language standard; these characters can be written directly in the source file using the UTF-8 encoding, or referred to using universal character names (\u00E0, \U000000E0).
So, I guess I'm asking.. what IS the "active language standard", and how can I find an authoritative source for what identifiers are legal.
I created the following code just to see what clang would do with it. Out of about 63488 possible identifiers tested, 23 issued warnings and 9506 generated errors. That leaves almost 54,000 valid characters to use in identifiers. Certainly enough, but who got cut? And why?
As others have mentioned, Annex D of ISO/IEC 9899:2011 lists the hexadecimal values of characters valid for universal character names in C11. (I won't bother repeating it here.) I have been searching for an answer as to "why" this list was chosen.
Character set standards
First, there are two relevant standards defining a set of characters: ISO/IEC 10646 (defining UCS) and Unicode. To further confuse (or simplify) things, they both define the same characters since the ISO and Unicode keep them synchronized. UCS is essentially just a character map associating values to a set of characters ("repertoire"), while Unicode also gives further definitions such how to compare strings in an alphabetical sorting order (collation), which code points represent "canonically equivalent" characters (normalization), and a bidirectional algorithm for how to process characters in languages written right to left, and more.
Universal character names in C
Universal character names (UCN) was a feature newly added in C99 (ISO/IEC 9899:1999). In the "Rationale for International Standard---Programming Languages---C" (Rev. 2, Oct. 1999), the purpose was "to enable the use of any 'native' character in identifiers, string literals and character constants, while retaining the portability objective of C" (sec. 5.2.1). This section continues on about issues of how to encode these characters in C (the \U and \u forms versus multibyte characters or native encodings) and policy models of how to deal with it (p.14, see PDF page 22).
Rationale
I was hoping that the same "rationale" document from 1999 would give a reason of why each extended character range was selected as acceptable for C99's UCNs. The entirety of the rationale's Annex I is:
Annex I Universal character names for identifiers (normative)
A new feature of C9X.
This is not much of a rationale. They didn't even know what year the C standard would be published, so it's just called "C9X". A later rationale document from 2003 is slightly more enlightening:
Annex D Universal character names for identifiers (normative)
New feature for C99.
The intention is to keep current with ISO/IEC TR 10176.
ISO/IEC TR 10176 is "Guidelines for the preparation of programming language standards." It a basically a guidebook for people who write programming language standards. It includes guidelines for the use of character sets in programming languages as well as a "recommended extended repertoire for user-defined identifiers" (Annex A). But this quote from the 2003 rationale document is only an "intention to keep current," not a pledge of strict adherence to TR 10176.
There is a publicly available ISO/IEC TR 10176:2003 table of characters. The character values refer to ISO 10646. The table classifies ranges of characters from numerous languages as being "uppercase" Lu; "lowercase" Ll; "number, decimal digit" Nd, "punctuation, connector" Pc; etc. It should be clear what use such classifications have to a programming language.
An important reminder is that TR 10176 is a Technical Report, and not a standard. I have found several passing references to it on forums and in documents related to other programming languages, such as Ada, COBOL, and D language. Much of the discussion was about how closely standards of those languages should follow TR 10176 (not being a standard) and complaints that TR 10176 was lagging behind updates to ISO 10646.
Perhaps most enlightening is document WG21/N3146: "Recommendations for extended identifier characters for C and C++." It starts with a comment in 2010 to the standards body recommending restrictions on the initial characters of identifiers. It mentions similar complaints about C referencing TR 10176, and makes suggestions about what characters should be allowed as initial characters of an identifier based on restrictions from Unicode's Identifier and Pattern Syntax and XML's Common Syntactic Constructs. WG21/N3146 gives the proposed wording that later appeared in the C11 standard ISO/IEC 9899:2011. There is a table at the end of the document that helps shed light on the character ranges selected.
Characters allowed and not allowed in C11
Below is a compiled list of ranges for extended identifier characters. The boldface ranges are those given in C11 (ISO/IEC 9899:2011 Annex D). Some comments are added about the italicized ranges not listed in C11 (i.e. not allowed). They are either marked in WG21/N3146 as disallowed by Unicode's UAX#31 or XML's Common Syntactic Constructs, or prohibited by some other comment.
00A8, 00AA, 00AD, 00AF, 00B2-00B5, 00C0-00D6, 00D8-00F6, 00F8-00FF: (Various characters, such as feminine ª and masculine º ordinal indicators, vowels with diacritics, numeric characters such as superscript numbers, fractions, etc.)
(previous gaps): All disallowed by UAX31 and/or XML. (Generally punctuation type marks like «», monetary symbols ¥£, mathematical operators ×÷, etc.)
0100-167F: (Latin, Greek, Cyrillic, Arabic, Thai, Ethiopic, etc.---many others)
1680: "The Ogham block contains a script-specific space: "
1681-180D: (Ogham, Tagalog, Mongolian, etc.)
180E: "The Mongolian block contains a script-specific space"
180F-1FFF: (More languages... phonetics, extended Latin & Greek, etc.)
2000: starts the "General Punctuation" block, but some are allowed:
200B−200D, 202A−202E, 203F−2040, 2054, 2060−206F: (selections from "General Punctuation" block)
2070−218F: "Superscripts and Subscripts, Currency Symbols, Combining Diacritical Marks for Symbols, Letterlike Symbols, Number Forms"
2190-245F: "Arrows, Mathematical Operators, Miscellaneous Technical, Control Pictures, Optical Character Recognition"
2460-24FF: "Enclosed Alphanumerics"
2500: starts "Box Drawing, Block Elements, Geometric Shapes", etc.
2776-2793: (some dingbats and circled dingbats)
2794-2BFF: (a different dingbat set, mathematical symbols, arrows, Braille patterns, etc.)
2C00-2DFF, 2E80-2FFF: "Glagolitic, Latin Extended-C, Coptic, Georgian Supplement, Tifinagh, Ethiopic Extended, Cyrillic Extended-A" (also CJK radical supplement)
3000: (start of "CJK Symbols and Punctuation", some selections allowed)
3004-3007, 3021-302F, 3031-303F: (allowed "CJK Symbols and Punctuation")
3040-D7FF: "Hiragana, Katakana," more CJK ideograms, radicals, etc.
D800-F8FF: (This starts the High and Low Surrogate Areas (number space needed for encodings), and Private Use)
F900-FD3D, FD40-FDCF, FDF0-FE44, FE47-FFFD: selections from "CJK Compatibility Ideographs," "Arabic Presentation Forms," etc.
10000−1FFFD, 20000−2FFFD, 30000−3FFFD, 40000−4FFFD, 50000−5FFFD,
60000−6FFFD, 70000−7FFFD, 80000−8FFFD, 90000−9FFFD, A0000−AFFFD,
B0000−BFFFD, C0000−CFFFD, D0000−DFFFD, E0000−EFFFD: WG21/N3146 gives the rationale for these final ranges:
The Supplementary Private Use Area extends from F0000 through 10FFFF; both [AltId] and [XML2008] disallow characters in that range.
In addition, [AltId] disallows, as non-characters, the last two code positions of each plane, i.e. every position of the form PFFFE or PFFFF, for any value of P.
The "Ranges of characters disallowed initially" from C11 Annex D.2 are 0300−036F, 1DC0−1DFF, 20D0−20FF, FE20−FE2F.
With this WG21/N3146 placed next to the Annex D of the C11 standard, much can be inferred about how they line up. For example, mathematical operators and punctuation seem to be not allowed. I hope this sheds some light on "why" or "how" the allowed characters were chosen.
TLDR; version
Authoritative source for legal identifier characters is the C11 standard ISO/IEC 9899:2011 (See Annex D).
This list is based on a technical report, ISO/IEC TR 10176, but with modifications.
C 2011 standard
6.4.2 Identifiers
6.4.2.1 General
...
3 Each universal character name in an identifier shall designate a character whose encoding
in ISO/IEC 10646 falls into one of the ranges specified in D.1.71) The initial character
shall not be a universal character name designating a character whose encoding falls into
one of the ranges specified in D.2. An implementation may allow multibyte characters
that are not part of the basic source character set to appear in identifiers; which characters
and their correspondence to universal character names is implementation-defined.
...
71) On systems in which linkers cannot accept extended characters, an encoding of the universal character
name may be used in forming valid external identifiers. For example, some otherwise unused
character or sequence of characters may be used to encode the \u in a universal character name.
Extended characters may produce a long external identifier.
...
Annex D
(normative)
Universal character names for identifiers
1 This clause lists the hexadecimal code values that are valid in universal character names
in identifiers.
D.1 Ranges of characters allowed
1 00A8, 00AA, 00AD, 00AF, 00B2−00B5, 00B7−00BA, 00BC−00BE, 00C0−00D6,
00D8−00F6, 00F8−00FF
2 0100−167F, 1681−180D, 180F−1FFF
3 200B−200D, 202A−202E, 203F−2040, 2054, 2060−206F
4 2070−218F, 2460−24FF, 2776−2793, 2C00−2DFF, 2E80−2FFF
5 3004−3007, 3021−302F, 3031−303F
6 3040−D7FF
7 F900−FD3D, FD40−FDCF, FDF0−FE44, FE47−FFFD
8 10000−1FFFD, 20000−2FFFD, 30000−3FFFD, 40000−4FFFD, 50000−5FFFD,
60000−6FFFD, 70000−7FFFD, 80000−8FFFD, 90000−9FFFD, A0000−AFFFD,
B0000−BFFFD, C0000−CFFFD, D0000−DFFFD, E0000−EFFFD
D.2 Ranges of characters disallowed initially
1 0300−036F, 1DC0−1DFF, 20D0−20FF, FE20−FE2F
The syntax for identifiers, which include macro names, is presented in section 6.4.2 of the C2011 standard, as interpreted in light of appendix D.1. These provisions hold that every identifier may contain underscores, upper- and lower-case Latin letters, decimal digits, sequences of characters constituting "universal character names" (subject to limitations), and any other character defined by the implementation.
Universal character names (UCNs) are Unicode escape sequences similar to those provided by Java, Python, and some other languages: they start with a backslash (\), which is followed by a u or U, and either four or eight hexadecimal digits, respectively. There are some limitations on the specific hex digit sequences that may be used, some general, others specific to identifier context. Note, however, that syntactically, the only additional character that the provision for UCNs allows to appear in identifiers is the backslash; all the other characters that can appear in a UCN are allowed in identifiers outside of UCN context, too.
Thus, speaking syntactically and restricting the discussion to the characters that the standard requires to be allowed in identifiers, the underscore, (unaccented) Latin letters, decimal digits, and the backslash are the only characters that C requires must be supported in identifiers. Support for the backslash is required only in the context of UCNs, and not all valid UCNs are allowed in identifiers. Additionally, the standard does not require support for digits as the first characters of identifiers.
On the other hand, the standard is quite liberal in allowing "other implementation-defined characters" in identifiers, including as the first character. Even decimal digits, which otherwise cannot be the first character in an identifier, could, in principle, be allowed at that position under this provision, at the discretion of the implementation. If you want your code to be portable among implementations then you will avoid relying on this provision anywhere. If you want to know which characters your particular implementation allows then you must consult its documentation.
Every standard-conforming implementation must document its behavior with respect to every detail the standard declares to be implementation defined. For example, GCC's documentation specifies that the dollar sign ($) is allowed in identifiers on most target architectures. You yourself linked to and quoted Clang's documentation of the same implementation-defined detail, which is more liberal -- it allows all the characters that can be represented in identifiers via UCNs to also be representable by UTF-8 byte sequences. In many cases, if you display or print source code containing such byte sequences, they will be rendered as a single display character.
As already mentioned, the C11 Standard defines several allowed Ranges of Unicode characters.
00A8, 00AA, 00AD, 00AF, 00B2−00B5, 00B7−00BA, 00BC−00BE, 00C0−00D6, 00D8−00F6, 00F8−00FF
0100−167F, 1681−180D, 180F−1FFF
200B−200D, 202A−202E, 203F−2040, 2054, 2060−206F
2070−218F, 2460−24FF, 2776−2793, 2C00−2DFF, 2E80−2FFF
3004−3007, 3021−302F, 3031−303F
3040−D7FF
F900−FD3D, FD40−FDCF, FDF0−FE44, FE47−FFFD
10000−1FFFD, 20000−2FFFD, 30000−3FFFD, 40000−4FFFD, 50000−5FFFD, 60000−6FFFD, 70000−7FFFD, 80000−8FFFD, 90000−9FFFD, A0000−AFFFD, B0000−BFFFD, C0000−CFFFD, D0000−DFFFD, E0000−EFFFD
This also means there are several ranges of characters excluded from usage.
From your examples:
☎ is 260E and from the "Miscellaneous Symbols" block: 2600-26FF which means youre missing out on all of these
❌ is 274C and from the "Dingbats" block: 2700-27BF which is all of these but some of them are allowed (2776−2793)
⇧ is 21E7 and from the "Arrows " block: 2190-21FF which means youre missing out on all of these
〓 is 3013 and from the "CJK Symbols and Punctuation" block: 3000-303F which is all these but some of them are allowed.
🍎 is 1F34E and from the "Miscellaneous Symbols and Pictographs" block: 1F300-1F5FF which is all these and actually should work (maybe a clangproblem? btw this is not displayed on my home computer (Ubuntu) but on my work PC (Win7))
How can I match a word (1-n characters) in ANSI C? (in addition: What is the pattern to match a constant in C-sourcecode?)
I tried reading the file and passing it to regexec() (regex.h).
Problem: The tool I'm writing should be able to read sourcecode and find
all used constants (#define) to check if they're defined.
The pattern used for testing is: [a-zA-Z_0-9]{1,}. But this would match words such as the "h" in "test.h".
Identifiers must start with a letter or underscore, so the pattern is
[A-Za-z_][A-Za-z0-9_]*
I know of no syntactic difference between C and preprocessor identifiers. There is a convention to use upper case for preprocessor and lowercase for C identifiers, but no actual requirement. Unless defines are guaranteed to use a distinct naming convention you would basically have to find every identifier in the source file and any included files and sort them into preprocessor identifiers, C identifiers and undeclared identifiers.
From the GCC manual:
Preprocessing tokens fall into five broad classes: identifiers, preprocessing numbers, string literals, punctuators, and other. An identifier is the same as an identifier in C: any sequence of letters, digits, or underscores, which begins with a letter or underscore. Keywords of C have no significance to the preprocessor; they are ordinary identifiers. You can define a macro whose name is a keyword, for instance. The only identifier which can be considered a preprocessing keyword is defined.
Another option besides doing regex searches over C source code would be to use a preprocessor library like Boost Wave or perhaps something like Coan instead of starting from scratch.
Here is the Lexer grammar and the Parser grammar (in flex and bison format, respectively) for the entire c language. In particular, the part relevant to identifiers is:
D [0-9]
L [a-zA-Z_]
{L}({L}|{D})* { count(); return(check_type()); }
So the id can start with any uppercase or lowercase letter or an underscore, and then have more uppercase or lowercase letters, underscores, and numbers. I believe it doesn't match parts of file names because they're quoted and it handles quotes separately.
I'm looking for how can I write identifiers name with characters like [ ' " or #.
Everytime that I try to do that, I give the error:
error: macro names must be identifiers
But learning about gcc, I found this option:
-fextended-identifiers
But it seems not working like I wanted, please, somebody know how to accomplish that?
Identifiers can't include such characters. It is defined that way in the language syntax, identifiers are letters, digits or underline (and mustn't begin with a digit to avoid ambiguity with litteral numbers).
If it was possible this would conflict with the C compiler (that uses [ for arrays) and C preprocessor syntax (that uses #). Extended identifiers extension only allow using characters non forbidden by the language syntax inside identifiers (basically unicode foreign letters, etc.).
But if you really, really want to do this, nothings forbids you to preprocess your source files with your own "extended macro preprocessor", practically creating a new "C like" language. That looks like a terrible idea, but it's not really hard to do. Then you'll see soon enough by yourself why it's not a good idea...
According to this link, -fextended-identifiers only enables UTF-8 support for identifiers, so it won't help in your case.
So, answer is: You can't use such characters in macro identifiers.
Even if the extended identifier characters support was fully enabled, it wouldn't help you get characters such as:
[ ' " #
enabled for identifiers. The standard allows 'universal character names' or 'other implementation-defined characters' to be part of an identifier, but they cannot be part of the basic character set. Out of the basic character set, only _, letters and digits can be part of an identifier name (6.4.2.1 Identifiers/General).