In C (and similar languages), a string is declared for example as "abc". Another example is "ab\"c". I have a file which contains these strings. That is, the file contents is "abc" or "ab\c" etc. Any literal string that can be defined in a .c file can be defined in the file I'm reading.
These strings can be malformed. E.g. "abc (no closing quotes). What is the best way to write a parser to make sure the string in the file is a valid C literal string? (so that if I copy the file contents and paste them after char* str =, the resulting expression will be accepted by the compiler when at the top of a function)
The strings are each in a separate line.
Alternatively, you can think of this as wanting to parse lines that declare literal string variables. Imagine I'm grepping a big file and use char\* .* = (.*);$ and want to make sure the part in the parenthesis will not cause compilation errors;
The grammar for C string literals is given in C 2018 6.4.5. Supposing you want to parse only plain strings, not those with encoding prefixes such as u in u"xyz", then the grammar for a string-literal is " s-char-sequenceopt ", where “opt” means optional and s-char-sequence is one or more s-char tokens. An s-char is any member of the source character set except ", \ or the new-line character or is an escape-sequence.
The source character set includes at least the Latin alphabet (26 letters A-Z) in uppercase and lowercase, the ten digits, space, horizontal tab, vertical tab, form feed, and these characters:
"#%&’()*+,-./:;?[\]^_{|}~
However, a C implementation may include other characters in its source character set. Therefore, any character found in the string other than ", \, or the new-line character must be accepted as potentially valid in some C implementation.
An escape-sequence is defined in 6.4.4.4 1 to be one of:
\ followed by ', ", ?, \, a, b, f, n, r, t, v,
\ followed by one to three octal digits, or
\x followed by one or more hexadecimal digits, or
a universal-character-name.
Paragraph 7 says:
Each octal or hexadecimal escape sequence is the longest sequence of characters that can constitute the escape sequence.
A universal-character-name is defined in 6.4.3 to be \u followed by four hexadecimal digits or \U followed by eight hexadecimal digits. Paragraph 2 limits these:
A universal character name shall not specify a character whose short identifier is less than 00A0 other than 0024 ($), 0040 (#), or 0060 (‘), nor one in the range D800 through DFFF inclusive.
This part of the C grammar looks fairly simple to parse:
A string literal must start with a ".
If the next character is anything other than ", \, or a new-line character, then accept it.
If the next character is \ and it is followed by one of the single characters listed above, accept it and the following character.
If the next character is \ and it is followed by one to three octal digits, accept it and up to three octal digits.
If the next two characters are \x and are followed by a hexadecimal digit, accept them and all the hexadecimal digits that follow.
If the next two characters are \u and are followed by four hexadecimal digits, accept those six characters. However, if the value is one of those prohibited in the constraint above, this is not a valid C string literal.
If the next two characters are \U and are followed by eight hexadecimal digits, accept those ten characters. However, if the value is one of those prohibited in the constraint above, this is not a valid C string literal.
Repeat the above until the next character is not accepted.
If the next character is not ", this is not a valid C string literal.
If the next character is ", accept it.
If that is the end of the line read from the file, it is a valid C string literal. Otherwise, it is not.
Related
I have the following C code:
#include <stdio.h>
#include <strings.h>
int main(void){
char * str = "\012\0345";
char testArr[8] = {'\0','1','2','\0','3','4','5','\0'};
printf("%s\n",str);
printf("**%s**",testArr);
return 0;
}
See live code here
I'm having trouble understanding the results and I have googled but am unsure that I understand why a null character at the start of a string and why one in the middle would cause only the string "5" to display. Also, when I assign each string character to array testArr and then attempt to display that array of characters the result is different despite the string and the array having the same characters. So, I'm struck by the confounding results, especially their disparity. With the string str, does the code display "5" because the null characters overwrite what is in memory?
Also, with the array I created using the same characters, nothing displays of the data contained in array testArr. Is it that once the first null is encountered for some reason everything else is ignored? If so, why doesn't the same behavior occur with string str which contains the same characters?
An octal escape sequence is \ followed by one to three octal digits, per C 2018 6.4.4.4 1. Per 6.4.4.4 7: “Each octal or hexadecimal escape sequence is the longest sequence of characters that can constitute the escape sequence.” So, when the compiler sees "\012\0345", it interprets it as the sequence \012 (which is ten), the sequence \034 (which is twenty-eight), and the character 5.
To represent the string you intended, you could use "\00012\000345". Since an octal escape sequence stops at three digits, this is interpreted as the sequence \000, the characters 1 and 2, the sequence \000, and the characters 3, 4, and 5. (A null terminating character will also be appended automatically.)
When you printed "\012\0345", the characters with codes ten and twenty-eight were printed but had no visible effect. (Your C implementation likely uses ASCII, in which case they are control characters. \012 is new-line, so it should have caused a line advance, but you probably did not notice that. \034 is a file-separator control character, which likely has no effect when printed to a regular terminal display.)
When you printed testArr, the null character in the first position ended the string.
In C, on mentioning three backslashes, like so:
#include <stdio.h>
int main() {
printf(" \\\ ");
}
prints out just one backslash in the output. Why and how does this work?
That sequence is:
A space
A double backslash, which encodes a single backslash in the runtime string
A backslash followed by a space, which is not a standard escape sequence and should give you a diagnostic
The C11 draft says (in note 77):
The semantics of these characters were discussed in 5.2.2. If any other
character follows a backslash, the result is not a token and a diagnostic is
required.
On godbolt.org I got:
<source>:8:14: warning: unknown escape sequence '\ ' [-Wunknown-escape-sequence]
So you seem to be using a non-conforming compiler, which chooses to implement undefined backslash sequences by just letting the character through.
That is printing:
space
slash
escaped space
The 3rd slash is being interpreted as "slash space"
C11; 6.4.4.4 Character constants:
The double-quote " and question-mark ? are representable either by themselves or by the
escape sequences \" and \?, respectively, but the single-quote ' and the backslash \
shall be represented, respectively, by the escape sequences \' and \\.
So, To represent a single backslash, it’s necessary to place double backslashes \\ in the source code. To print two \\ you need four backslash \\\\. In your code extra \ is a space character, which isn't valid.
It is indeed a very simple operation in c. A \ is just a escape sequences. Hence below statement will print two slash.
printf(" \\\\ ");
For example some characters in c are represented with a slash like end of a line character \n or end of a string character \0 etc. But if you want to print such a character as it is what will you do? Hence you need to add a escape sequence character in front of it:
printf("\\n"); // will print \n
But
printf("\n"); // will print end of character hence you don't see anything in output
If I write putchar('\\t'); while trying to print "\t" instead of an actual tab, I get the multi character constant warning. On the other hand, if I write putchar('\\'); I get no warning. Upon looking in the ASCII table, there is no character '\\', only '\'. So why is there no warning? Why is '\\' one character but '\\t' is more than one? Can a backslash only be used to escape one following character?
You cannot print \ and t with one putchar invocation, since putchar puts one and exactly only one character into the standard output. Use 2:
putchar('\\');
putchar('t');
Another option would be to use the string "\\t" with fputs:
fputs("\\t", stdout);
There is no warning for '\\' because that is one way how you enter the character literal for the character \. On ASCII this is synonymous with '\134' and '\x5c'.
From C11 6.4.4.4 paragraphs 2 and 4:
2
An integer character constant is a sequence of one or more multibyte characters enclosed in single-quotes, as in 'x'. [...] With a few exceptions detailed later, the elements of the sequence are any members of the source character set; they are mapped in an implementation-defined manner to members of the execution character set.
[...]
4
The double-quote " and question-mark ? are representable either by themselves or by the escape sequences \" and \?, respectively, but the single-quote ' and the backslash \ shall be represented, respectively, by the escape sequences \' and \\.
The reason why you get a warning for this is that the behaviour is wholly implementation-defined. In C11 J.3.4 the following is listed as implementation-defined behaviour:
The value of an integer character constant containing more than one character or containing a character or escape sequence that does not map to a single-byte execution character (6.4.4.4).
Since '\\' contains an escape sequence that maps to a single-byte execution character \, there is no implementation-defined pitfalls, and nothing to warn about; but \\t contains 2 characters: \ and t, and it wouldn't do what you want portably.
\\ is one character, t is one character, so that is clearly two characters.
\\ is an escape sequence, just like \t; it means \.
If you want to print the two characters \ and t, you clearly need either two calls to putch() or a function that takes a string argument "\\t".
https://en.wikipedia.org/wiki/Escape_sequences_in_C#Table_of_escape_sequences
I've been going through strlen and sizeof for strings (char arrays) and I don't quite get one thing.
I have the following code:
int main() {
char str[]="gdb\0eahr";
printf("sizeof=%u\n",sizeof(str));
printf("strlen=%u\n",strlen(str));
return 0;
}
the output of the code is:
sizeof=9
strlen=3
At first I was pretty sure that 2 separate characters \ followed by 0 wouldn't actually act as a NUL (\0) but I managed to figure that it does.
The thing is that I have no idea why sizeof shows 9 and not 10.
Since sizeof counts the amount of used bytes by the data type why doesn't it count the byte for the \?
In a following example:
char str[]="abc";
printf("sizeof=%u\n",sizeof(str));
that would print out "4" because of the NUL value terminating the array so why is \ being not counted?
In a character or string constant, the \ character marks the beginning of an escape sequence, used to represent character values for which there isn't a symbol in the source character set. For example, the escape sequence \n represents the newline character, \b represents the backspace character, \0 represents the zero-valued character (which is also the string terminator), etc.
In the string literal "gdb\0eahr", the escape sequence \0 maps to a single 0-valued character; the actual contents of str are {'g', 'd', 'b', 0, 'e', 'a', 'h', 'r', 0}.
It seems you already have the answer:
At first I was pretty sure that 2 separate characters "\" followed by "0" wouldn't actually act as a NULL "\0" but I managed to figure that it does.
The sequence \0 is an octal escape sequence for the byte 0. So while there are two characters in the code to denote this, it translated to a single byte in the string.
So you have 7 alphabetic characters, a null byte in the middle, and a null byte at the end. That's 9 bytes.
Why should char str[]="gdb\0eahr"; be 10 bytes with sizeof operator? It is 9 bytes because there are 8 string elements + trailing zero.
\0 is only 1 character, not 2. \'s purpose is to escape characters, therefore you might see some of these: \t, \n, \\ and others.
Strlen returns 3 because you have string termination at position str[3].
Single sequence of \ acts as escape character and is not part of string size. If you want to literally use \ in your string, you have to write it twice in sequence like \\, then this is single char of \ printable char.
The C compiler scans text strings as a part of compiling the source code and during the scan any special, escape sequences of characters are turned into a single character. The symbol backslash (\) is used to indicate the start of an escape sequence.
There are several formats for escape sequences. The most basic is a backslash followed by one of several special letters. These two characters are then translated into a single character. Some of these are:
'\n' is turned into a line feed character (0x0A or decimal 10)
'\t' is turned into a tab character (0x09 or decimal 9)
'\r' is turned into a carriage return character (0x0D or decimal 13)
'\\' is turned into a backslash character (0x5C)
This escape sequence idea was used back in the old days so that when a line of text was printed to a teletype machine or printer or a CRT terminal, the programmer could use these and other special command code characters to set where the next character would be printed or to cause the device to do some physical action like ring a bell or feed the paper to the next line.
The escape character also allowed you to embed a double quote (") or a single quote (') into a text string so that you could print text that contain quote marks.
In addition to the above special sequences of backslash followed by a letter there was also a way to specify any character by using a backslash followed by one up to three octal digits (0 through 7). So you could specify a line feed character by either using '\n' or you could use '\12' where 12 is the octal representation of the hexadecimal value A or the decimal value 10.
Then the ability to use a hexadecimal escape sequence was introduced with the backslash followed by the letter x followed by one or more hexadecimal digits. So you can write a line feed character with '\n' or '\12' or '\xa'.
See also Escape sequences in C in Wikipedia.
A[50][5000];
for(i=0;i<50;++i)
scanf("%[\n]",A[i]);
%[^\n]
usage and meaning of it
and can i use that struct like
%[\t]
%[\a]
scanf()'s "%[" conversion specifier starts what's called a "scanset". It's has some similarities to the regex construct that looks the same (but it still is quite different) Here's what the standard says:
Matches a nonempty sequence of characters from a set of expected characters (the scanset).
...
The conversion specifier includes all subsequent characters in the format string, up to and including the matching right bracket (]). The characters between the brackets (the scanlist) compose the scanset, unless the character after the left bracket is a circumflex (^), in which case the scanset contains all characters that do not appear in the scanlist between the circumflex and the right bracket. If the conversion specifier begins with [] or [^], the right bracket character is in the scanlist and the next following right bracket character is the matching right bracket that ends the specification; otherwise the first following right bracket character is the one that ends the specification. If a - character is in the scanlist and is not the first, nor the second where the first character is a ^, nor the last character, the behavior is implementation-defined.
So the scanf() conversion "%[\n]" will match a newline character, while "%[^\n]" will match all characters up to a newline.
Here's what P.J. Plauger has to say about scansets in "The Standard C Library":
A scan set behaves much like the s conversion specifier. It stores up to w characters (default is the rest of the input) in the char array pointed at by ptr. It always stores a null character after any input. It does not skip leading white-space. It also lets you specify what characters to consider as part of the field. You can specify all the characters that match, as in %[0123456789abcdefABCDEF], which matches an arbitrary sequence of hexadecimal digits. Or you can specify all the characters that do not match, as in %[^0123456789] which matches any characters other than digits.
If you want to include the right bracket (]) in the set of characters you specify, write it immediately after the opening [ (or [^), as in %[][] which scans for square brackets. You cannot include the null character in the set of characters you specify. Some implementations may let you specify a range of characters by using a minus sign (-). The list of hexadecimal digits, for example, can be written as %[0-9abcdefABCDEF] or even, in some cases, as %[0-9a-fA-F]. Please note, however, that such usage is not universal. Avoid it in a program that you wish to keep maximally portable.
Yes, it's pretty much like a set in a regular expression -- you can specify a set of character to be accepted, or a set of characters to end the scan, so "%[^ \r\n\t]" would read until it encountered a space, carriage return, new-line or tab. Like with an RE, the leading "^" means "not" -- you can omit it to specify the characters that will be accepted instead of those that will end the conversion. With most compilers (though it's not technically required) you can specify ranges, such as "%[a-z]" to specify any lower-case letter (in this case, where the '-' isn't the first or last character, the behavior is implementation defined).
Though not widely used (or even known) this conversion has been part of C almost forever, and is supported in C89/90.
copies a string up to a newline from standard input to element i of A. as written, this acts almost like gets().