Flex - identfiy float/ int/ id tokens - c

Question

I'm trying to create a flex file that will recognize float/ integers and id:
Valid int - not allowed to start with 0.
Valid float-its presentation must include exponent whose value is an integer number with or without sign 2.78e+10.
Valid id- can only start with a lower-case letter and several underscores can not appear one after another
I am not sure where I'm wrong, if I have only float number I getting back float also int and id, but when everything combined in one file it's not working.
this the file that I create:
%option noyywrap
%{
#include "Token.h"
#include <stdio.h>
#include <stdlib.h>
static int skip_single_line_comment(int num); //function for one line comment
static int skip_multiple_line_comment(int num);//function for multiple lines of comments
int line_num=0;
%}
ALPHA ([a-zA-Z])
DIGIT ([0-9])
Sign ([+|-])
Expo ([e]{Sign}?)
float_num ([1-9]+(\.({DIGIT}+{Expo}{DIGIT}+)))
int_num [1-9]{DIGIT}+
id ([a-z]+({ALPHA}|{DIGIT}|(\_({ALPHA}|{DIGIT})))*)
%%
{float_num} {
create_and_store_token(TOKEN_FLOAT, yytext, line_num);
fprintf(yyout,"Line %d : found token of type TOKEN_FLOAT, lexeme %s.\n", line_num, yytext);
}
\n {line_num++;}
{int_num} {
create_and_store_token(TOKEN_INTEGER, yytext, line_num);
fprintf(yyout,"Line %d : found token of type TOKEN_INTEGER , lexeme %s.\n", line_num, yytext);
}
{id} {
create_and_store_token(TOKEN_ID, yytext, line_num);
fprintf(yyout,"Line %d : found token of type TOKEN_ID, lexeme %s.\n", line_num, yytext);
}
"//" {line_num=skip_single_line_comment(line_num); fprintf(yyout,"The number of the line is:%d.\n", line_num);}
"/*" {line_num=skip_multiple_line_comment(line_num); fprintf(yyout,"The number of the line is:%d.\n", line_num);}
%%
static int
skip_single_line_comment(int num)
{
char c;
/* Read until we find \n or EOF */
while((c = input()) != '\n' && c != EOF)
;
/* Maybe you want to place back EOF? */
if(c == EOF)
unput(c);
return num=num+1;
}
static int
skip_multiple_line_comment(int num)
{
char c;
for(;;)
{
switch(input())
{
/* We expect ending the comment first before EOF */
case EOF:
fprintf(stderr, "Error unclosed comment, expect */\n");
exit(-1);
goto done;
break;
/* Is it the end of comment? */
case '*':
if((c = input()) == '/'){
num=num+1;
goto done;
}
unput(c);
break;
default:
/* skip this character */
break;
}
}
done:
/* exit entry */
return num ;
}
void main(int argc, char **argv){
yyin=fopen("C:\\temp\\test1.txt","r");
yyout=fopen("C:\\temp\\test1Soltion.txt","w");
yylex();}
the input file:
21
41.e-21
a_23_e4_5
8
1.1E+21
a1_c23_e4_56
The output:
Line 0 : found token of type TOKEN_INTEGER , lexeme 21.
Line 1 : found token of type TOKEN_INTEGER , lexeme 41.
.Line 1 : found token of type TOKEN_ID, lexeme e.
-Line 1 : found token of type TOKEN_INTEGER , lexeme 21.
Line 2 : found token of type TOKEN_ID, lexeme a_23_e4_5.
81.1E+Line 4 : found token of type TOKEN_INTEGER , lexeme 21.
Line 5 : found token of type TOKEN_ID, lexeme a1_c23_e4_56.

Answer 1

You have several problems in your code: (from top to bottom)
Sign is bad... you are saying that a sign is one of +, | or -. You have used three characters in between the square brackets [ and ] which makes them possible... you could use (\+|-) or [+-], but not what you have written. In order for the - to be accepted not as a range indicator, is to stick it to one of the square brackets that delimite the charset (better to the last one, so if you have to use the negation ^ character, you can do it without interference)
An exponent trailer to a floating point allows both e and E, so the actual regexp should be [eE].
a Floating point number can begin with 0. you can have something like -00013.26 and be valid...
your floating point number is forced to have digits at both sides of the dot ., so you'll not recognize anything like 3. or .26 as floating point numbers. You have written ([1-9]+(\.({DIGIT}+{Expo}{DIGIT}+))) which accepts a variable number of digits in the set [1-9] (you disallow 0 in front of a decimal point) but always greater than zero, followed with a dot, and followed with at least one digit after the dot... this makes 41.e-21 not to be recognized as a floating point. Even 40.25 will not be recognized as floating point (but as the tokens 4 (integer) followed by 0 integer, and a dot (which will be echoed to output by default) and then the integer 25)
you don't allow to put signs in front of number (this is common in compiler implementation, but not to read number sequences as you are trying) You have not included support for a sign in front of a number... this is the reason that 41.e-21 is parsed as Int(41), .(echoed), e(identifier),- (is not valid as sign here, because you don't allow signed integers), and 21 as an integer.
you don't accept integers of less than two digits: again, the use of + makes you to have one digit (different than 0) followed by at least one more digit... this makes the mesh you have on line four and five.
So the only thing you recognize correctly is the identifier, that by the way has to begin with a lower case alphabetic... you have not included support for uppercase beginning identifiers.