Skip declared tokens at some locations in javacc - lexical-analysis

Question

I declared a token <VOTE>. I want to use this token to match inside a block delimited by braces. If it comes outside of block, I want to skip it. Currently the token is matched the same regardless of whether it is inside a block or outside and this makes parsing the stuff outside the block a problem.
TOKEN: { < VOTE: " vote" > }
TOKEN: { < #STRING_LITERAL: ["a" - "z", "A" - "Z", "0" - "9", ".", "_", "-", "!","`", "*", "#", "#", "$", "%", ":", "[", "]", "\"", "^", ";", "\\", "<", ">", "/", ",", "?", "=", "&","(",")","+","|"] >}
TOKEN: { < DATA: < STRING_LITERAL > ( < STRING_LITERAL > ) * >
| < OPEN_BRACE: "{" >
| < CLOSE_BRACE: "}" >}
**Production:**
void test():{}
{
(<OPEN_BRACE>
<VOTE>{System.out.println("Voted");}
<CLOSE_BRACE>)
|
<DATA>{System.out.println("Skipped"+token.image);}
}
The above production works for the block
hi i am { vote }
But it throws an exception for following block
hi i vote am { vote }
because "vote" is outside the braces. Please show me how to skip "vote" outside braces.
I could mention <VOTE> in the production like this
void test():{}
{
(<OPEN_BRACE>
<VOTE>{System.out.println("VOted");}
<CLOSE_BRACE>)
|
(<DATA>|<VOTE>){System.out.println("Skipped"+token.image);}
}
But I don't want to do this because I have lot of tokens, so when I apply this fix in the full scale system it would be hard to maintain.

Answer 1

What you need to use are lexical states. This is covered in the FAQ and the documentation, the relevant parts of which you should read.
Basically what you want to do is something like this
<DEFAULT> TOKEN:{
< #STRING_LITERAL: ["a" - "z", "A" - "Z", "0" - "9", ".", "_", "-", "!","`", "*", "#", "#", "$", "%", ":", "[", "]", "\"", "^", ";", "\\", "<", ">", "/", ",", "?", "=", "&","(",")","+","|"] >
| < DATA: < STRING_LITERAL > ( < STRING_LITERAL > ) * > }
<DEFAULT, INBRACES> TOKEN : {
< OPEN_BRACE: "{" > : INBRACES
| < CLOSE_BRACE: "}" > : DEFAULT }
<INBRACES> TOKEN: {
< VOTE: "vote" > }
<DEFAULT, INBRACES> SKIP { " " | "\r" | "\n" }
Now between braces, "vote" will be a VOTE, but outside, it will be a DATA. Braces and spaces are treated the same regardless of where they are.