I'm writing a small language, and I'm really stuck on expression parsing. I've written a LR Recursive Descent Parser, it works, but now I need to parse expressions I'm finding it really difficult. I do not have a grammar defined, but if it helps, I kind of have an idea on how it works even without a grammar. Currently, my expression struct looks like this:
typedef struct s_ExpressionNode {
Token *value;
char expressionType;
struct *s_ExpressionNode lhand;
char operand;
struct *s_ExpressionNode rhand;
} ExpressionNode;
I'm trying to get it to parse something like:
5 + 5 + 2 * (-3 / 2) * age
I was reading this article on how to parse expressions. The first grammar I tried to implement but it didn't work out too well, then I noticed the second grammar, which appears to remove left recursion. However, I'm stuck trying to implement it since I don't understand what P, B means, and also U is a - but the - is also for a B? Also I'm not sure what expect(end) is supposed to mean either.
In the "Recursive-descent recognition" section of the article you linked, the E, P, B, and U are the non-terminal symbols in the expression grammar presented. From their definitions in the text, I infer that "E" is chosen as a mnemonic for "expression", "P" as mnemonic for "primary", "B" for "binary (operator)", and "U" for "unary (operator)". Given those characterizations, it should be clear that the terminal symbol "-" can be reduced either to a U or to a B, depending on context:
unary: -1
binary: x-1
The expect() function described in the article is used to consume the next token if it happens to be of the specified type, or otherwise to throw an error. The end token is defined to be a synthetic token representing the end of the input. Thus
expect(end)
expresses the expectation that there are no more tokens to process in the expression, and its given implementation throws an error if that expectation is not met.
All of this is in the text, except the reason for choosing the particular symbols E, P, B, and U. If you're having trouble following the text then you probably need to search out something simpler.
In my search for an example of a software phase lock loop I came across the following question
Software Phase Locked Loop example code needed
In the answer by Adam Davis a site is given that is broken and I have tried the new link that is given in a comment but I cant get that to work either.
The answer from Kragen Javier Sitaker gave the following code as a simple example of a software phase locked loop.
main(a,b){for(;;)a+=((b+=16+a/1024)&256?1:-1)*getchar()-a/512,putchar(b);}
Also included in his answer was a link to what should be a much more readable example but this link is also broken. Thus I have been trying to translate the above code into simpler and more readable code.
I have come this far:
main(a,b){
for(;;){
// here I need to break up the code somehow into a if() statement('s).
if(here I get lost){
a = a+1;
if(here i get lost some more){
b = b+1;
}
}
}
Thanks to the SO question What does y -= m < 3 mean?
I know it is possible to break up the a+= and b+= into if statements.
But the (&256? 1 : -1)*getchar()-a/512,putchar(b); part in the code is killing me. I have been looking on Google and on SO to the meaning of the symbols and functions that are used.
I know the & sign indicates an address in memory.
I know that the : sign declares a bit field OR can be used in combination with the ? sign which is a conditional operator. The combination of the two I can use like sirgeorge answer in what does the colon do in c?
Theory Behind getchar() and putchar() Functions
I know that getchar() reads one character
I know that putchar() displays the character
But the combination of all these in the example code is not readable for me and . I can not make it readable for my self even do I know what they all separately do.
So my question: How do I read this software phase lock loop code?
main(a,b){for(;;)a+=((b+=16+a/1024)&256?1:-1)*getchar()-a/512,putchar(b);}
What I get is:
main (a, b)
{
char c;
for (;;)
{
c = getchar();
b = (b + 16 + (a / 1024));
if(!(b & 256))
{
c = c * -1;
}
a = a + c - (a/512);
putchar(b);
}
}
I had to add a c variable to not get lost.
What the program does:
Take a and b.
Infinite loop:
get a char input in c
calculate b
If (b bitwise AND 256)
c = -c
Calculate a
Print b
It seems it translate input into something else, I have to see the code in action to understand better myself.
Hope it helped!
Hint:
https://en.wikipedia.org/wiki/Operators_in_C_and_C%2B%2B
a+= => a = a +
a?b:c => if(a){return b;} else {return c;} (As a function itself, it don t truly return)
Add parentheses, it help.
a & b is bitwise AND:
a/b |0|1|
0|0|0|
1|0|1|
How to get and evaluate the Expressions from a string in C
char *str = "2*8-5+6";
This should give the result as 17 after evaluation.
Try by yourself. you can Use stack data structure to evaluate this string here is reference to implement (its in c++)
stack data structre for string calcualtion
You have to do it yourself, C does not provide any way to do this. C is a very low level language. Simplest way to do it would be to find a library that does it, or if that does not exist use lex + yacc to create your own interpreter.
A quick google suggests the following:
http://www.gnu.org/software/libmatheval/
http://expreval.sourceforge.net/
You should try TinyExpr. It's a single C source code file (with no dependencies) that you can add to your project.
Using it to solve your problem is just:
#include <stdio.h>
#include "tinyexpr.h"
int main()
{
double result = te_interp("2*8-5+6", 0);
printf("Result: %f\n", result);
return 0;
}
That will print out: Result: 17
C does not have a standard eval() function.
There are lots of libraries and other tools out there that can do this.
But if you'd like to learn how to write an expression evaluator yourself, it can be surprisingly easy. It is not trivial: it is actually a pretty deeply theoretical problem, because you're basically writing a miniature parser, perhaps built on a miniature lexical analyzer, just like a real compiler.
One straightforward way of writing a parser involves a technique called recursive descent. Writing a recursive descent parser has a lot in common with another great technique for solving big or hard problems, namely by breaking the big, hard problem up into smaller and hopefully easier subproblems.
So let's see what we can come up with. We're going to write a function int eval(const char * expr) that takes a string containing an expression, and returns the int result of evaluating it. But first let's write a tiny main program to test it with. We'll read a line of text typed by the user using fgets, pass it to our expr() function, and print the result.
#include <stdio.h>
int eval(const char *expr);
int main()
{
char line[100];
while(1) {
printf("Expression? ");
if(fgets(line, sizeof line, stdin) == NULL) break;
printf(" -> %d\n", eval(line));
}
}
So now we start writing eval(). The first question is, how will we keep track of how how far we've read through the string as we parse it? A simple (although mildly cryptic) way of doing this will be to pass around a pointer to a pointer to the next character. That way any function can move forwards (or occasionally backwards) through the string. So our eval() function is going to do almost nothing, except take the address of the pointer to the string to be parsed, resulting in the char ** we just decided we need, and calling a function evalexpr() to do the work. (But don't worry, I'm not procrastinating; in just a second we'll start doing something interesting.)
int evalexpr(const char **);
int eval(const char *expr)
{
return evalexpr(&expr);
}
So now it's time to write evalexpr(), which is going to start doing some actual work. Its job is to do the first, top-level parse of the expression. It's going to look for a series of "terms" being added or subtracted. So it wants to get one or more subexpressions, with + or - operators between them. That is, it's going to handle expressions like
1 + 2
or
1 + 2 - 3
or
1 + 2 - 3 + 4
Or it can read a single expression like
1
Or any of the terms being added or subtracted can be a more-complicated subexpression, so it will also be able to (indirectly) handle things like
2*3 + 4*5 - 9/3
But the bottom line is that it wants to take an expression, then maybe a + or - followed by another subexpression, then maybe a + or - followed by another subexpression, and so on, as long as it keeps seeing a + or -. Here is the code. Since it's adding up the additive "terms" of the expression, it gets subexpressions by calling a function evalterm(). It also needs to look for the + and - operators, and it does this by calling a function gettok(). Sometimes it will see an operator other than + or -, but those are not its job to handle, so if it sees one of those it "ungets" it, and returns, because it's done. All of these functions pass the pointer-to-pointer p around, because as I said earlier, that's how all of these functions keep track of how they're moving through the string as they parse it.
int evalterm(const char **);
int gettok(const char **, int *);
void ungettok(int, const char **);
int evalexpr(const char **p)
{
int r = evalterm(p);
while(1) {
int op = gettok(p, NULL);
switch(op) {
case '+': r += evalterm(p); break;
case '-': r -= evalterm(p); break;
default: ungettok(op, p); return r;
}
}
}
This is some pretty dense code, Stare at it carefully and convince yourself that it's doing what I described. It calls evalterm() once, to get the first subexpression, and assigns the result to the local variable r. Then it enters a potentially infinite loop, because it can handle an arbitrary number of added or subtracted terms. Inside the loop, it gets the next operator in the expression, and uses it to decide what to do. (Don't worry about the second, NULL argument to gettok; we'll get to that in a minute.)
If it sees a +, it gets another subexpression (another term) and adds it to the running sum. Similarly, if it sees a -, it gets another term and subtracts it from the running sum. If it gets anything else, this means it's done, so it "ungets" the operator it doesn't want to deal with, and returns the running sum -- which is literally the value it has evaluated. (The return statement also breaks the "infinite" loop.)
At this point you're probably feeling a mixture of "Okay, this is starting to make sense" but also "Wait a minute, you're playing pretty fast and loose here, this is never going to work, is it?" But it is going to work, as we'll see.
The next function we need is the one that collects the "terms" or subexpressions to be added (and subtracted) together by evalexpr(). That function is evalterm(), and it ends up being very similar -- very similar -- to evalexpr. Its job is to collect a series of one or more subexpressions joined by * and/or /, and multiply and divide them. At this point, we're going to call those subexpressions "primaries". Here is the code:
int evalpri(const char **);
int evalterm(const char **p)
{
int r = evalpri(p);
while(1) {
int op = gettok(p, NULL);
switch(op) {
case '*': r *= evalpri(p); break;
case '/': r /= evalpri(p); break;
default: ungettok(op, p); return r;
}
}
}
There's actually nothing more to say here, because the structure of evalterm ends up being exactly like evalexpr, except that it does things with * and /, and it calls evalpri to get/evaluate its subexpressions.
So now let's look at evalpri. Its job is to evaluate the three lowest-level, but highest-precedence elements of an expression: actual numbers, and parenthesized subexpressions, and the unary - operator.
int evalpri(const char **p)
{
int v;
int op = gettok(p, &v);
switch(op) {
case '1': return v;
case '-': return -evalpri(p);
case '(':
v = evalexpr(p);
op = gettok(p, NULL);
if(op != ')') {
fprintf(stderr, "missing ')'\n");
ungettok(op, p);
}
return v;
}
}
The first thing to do is call the same gettok function we used in evalexpr and evalterm. But now it's time to say a little more about it. It is actually the lexical analyzer used by our little parser. A lexical analyzer returns primitive "tokens". Tokens are the basic syntactic elements of a programming language. Tokens can be single characters, like + or -, or they can also be multi-character entities. An integer constant like 123 is considered a single token. In C, other tokens are keywords like while, and identifiers like printf, and multi-character operators like <= and ++. (Our little expression evaluator doesn't have any of those, though.)
So gettok has to return two things. First it has to return a code for what kind of token it found. For single-character tokens like + and - we're going to say that the code is just the character. For numeric constants (that is, any numeric constant), we're going to say that gettok is going to return the character 1. But we're going to need some way of knowing what the value of the numeric constant was, and that, as you may have guessed, is what the second, pointer argument to the gettok function is for. When gettok returns 1 to indicate a numeric constant, and if the caller passes a pointer to an int value, gettok will fill in the integer value there. (We'll see the definition of the gettok function in a moment.)
At any rate, with that explanation of gettok out of the way, we can understand evalpri. It gets one token, passing the address of a local variable v in which the value of the token can be returned, if necessary. If the token is a 1 indicating an integer constant, we simply return the value of that integer constant. If the token is a -, this is a unary minus sign, so we get another subexpression, negate it, and return it. Finally, if the token is a (, we get another whole expression, and return its value, checking to make sure that there's another ) token after it. And, as you may notice, inside the parentheses we make a recursive call back to the top-level evalexpr function to get the subexpression, because obviously we want to allow any subexpression, even one containing lower-precedence operators like + and -, inside the parentheses.
And we're almost done. Next we can look at gettok. As I mentioned, gettok is the lexical analyzer, inspecting individual characters and constructing full tokens from them. We're now, finally, starting to see how the passed-around pointer-to-pointer p is used.
#include <stdlib.h>
#include <ctype.h>
void skipwhite(const char **);
int gettok(const char **p, int *vp)
{
skipwhite(p);
char c = **p;
if(isdigit(c)) {
char *p2;
int v = strtoul(*p, &p2, 0);
*p = p2;
if(vp) *vp = v;
return '1';
}
(*p)++;
return c;
}
Expressions can contain arbitrary whitespace, which is ignored, so we skip over that with an auxiliary function skipwhite. And now we look at the next character. p is a pointer to pointer to that character, so the character itself is **p. If it's a digit, we call strtoul to convert it. strtoul helpfully returns a pointer to the character following the number it scans, so we use that to update p. We fill in the passed pointer vp with the actual value strtoul computed for us, and we return the code 1 indicating an integer constant.
Otherwise -- if the next character isn't a digit -- it's an ordinary character, presumably an operator like + or - or punctuation like ( ), so we simply return the character, after incrementing *p to record the fact that we've consumed it. Properly "incrementing" p is mildly tricky: it's a pointer to a pointer, and we want to increment the pointed-to pointer. If we wrote p++ or *p++ it would increment the pointer p, so we need (*p)++ to say that it's the pointed-to pointer that we want to increment. (See also C FAQ 4.3.)
Two more utility functions, and then we're done. Here's skipwhite:
void skipwhite(const char **p)
{
while(isspace(**p))
(*p)++;
}
This simply skips over zero or more whitespace characters, as determined by the isspace function from <ctype.h>. (Again, we're taking care to remember that p is a pointer-to-pointer.)
Finally, we come to ungettok. It's a hallmark of a recursive descent parser (or, indeed, almost any parser) that it has to "look ahead" in the input, making a decision based on the next token. Sometimes, however, it decides that it's not ready to deal with the next token after all, so it wants to leave it there on the input for some other part of the parser to deal with later.
Stuffing input "back on the input stream", so to speak, can be tricky. This implementation of ungettok is simple, but it's decidedly imperfect:
void ungettok(int op, const char **p)
{
(*p)--;
}
It doesn't even look at the token it's been asked to put back; it just backs the pointer up by 1. This will work if (but only if) the token it's being asked to unget is in fact the most recent token that was gotten, and if it's not an integer constant token. In fact, for the program as written, and as long as the expression it's parsing is well-formed, this will always be the case. It would be possible to write a more complicated version of gettok that explicitly checked for these assumptions, and that would be able to back up over multi-character tokens (such as integer constants) if necessary, but this post has gotten much longer than I had intended, so I'm not going to worry about that for now.
But if you're still with me, we're done! And if you haven't already, I encourage you to copy all the code I've presented into your friendly neighborhood C compiler, and try it out. You'll find, for example, that 1 + 2 * 3 gives 7 (not 9), because the parser "knows" that * and / have higher precedence than + and -. Just like in a real compiler, you can override the default precedence using parentheses: (1 + 2) * 3 gives 9. Left-to-right associativity works, too: 1 - 2 - 3 is -4, not +2. It handles plenty of complicated, and perhaps surprising (but legal) cases, too: (((((5))))) evaluates to just 5, and ----4 evaluates to just 4 (it's parsed as "negative negative negative negative four", since our simplified parser doesn't have C's -- operator).
This parser does have a pretty big limitation, however: its error handling is terrible. It will handle legal expressions, but for illegal expressions, it will either do something bizarre, or just ignore the problem. For example, it simply ignores any trailing garbage it doesn't recognize or wasn't expecting -- the expressions 4 + 5 x, 4 + 5 %, and 4 + 5 ) all evaluate to 9.
Despite being somewhat of a "toy", this is also a very real parser, and as we've seen it can parse a lot of real expressions. You can learn a lot about how expressions are parsed (and about how compilers can be written) by studying this code. (One footnote: recursive descent is not the only way to write a parser, and in fact real compilers will usually use considerably more sophisticated techniques.)
You might even want to try extending this code, to handle other operators or other "primaries" (such as settable variables). Once upon a time, in fact, I started with something like this and extended it all the way into an actual C interpreter.
I have found two different things in two well known books in c,
first one is
"Formal parameters are not replaced in quoted string in macro expansion" - by K&R c language page 76
second one is a code,
#define PRINT(var,format) printf("variable is %format\n",var)
PRINT(x_var,f);
later macro invocation would be expanded as
printf("x_var is %f\n",x_var);
this is by programming in ansi c - E. balagurusamy at page 448.
Surely two citations are contradictory one with another. as far I know first one is true and my compiler giving me result so. But second book is also well known and popular. I want to know if there was such things in previous versions of c or the second citation is a false one.
The second book is wrong: it is easy to check that the macro will not be expanded like that. However, you can get the effect that they describe by stringizing tokens using the # preprocessor operator:
#define PRINT(var,format) printf(#var" is %"#format"\n",var)
Now you can print your variable as follows:
int xyz = 123;
PRINT(xyz, d);
Here is a link to a working sample on ideone.
Note the addition of the double quotes before and after the '#format', and the '#' before 'var' and 'format'. The '#' operator causes the value of the variable to be made into a quoted string -- with double quotes of its own. This makes the replaced strings be four quoted strings in a row, that the C compiler recognizes as a request to concatenate into one string. Thus the strings: "xyz", " is %", "d" and "\n" are concatenated into: "xyz is %d\n"
(Note this example is different from the example in the original question in that the orignal example had "variable is..." where the answer replaced 'variable' with an instance of the 'var' macro argument)
The book is correct. For it's time. I wrote a little test program to verify it (you don't appreciate text editors until you have programmed in ed):
#define PRINT(fmt,val) printf("val = %fmt\n", (val))
main()
{
int x;
x = 5;
PRINT(d, x);
}
I compiled it on a PDP-11 running Unix V6. Running the program produces this output:
x = 5
This is even pre K&R C. The "feature" was removed in one of the later iterations of C, and made official in ISO C90.