Javacc manual

This is generated into the generated parser directory as Token. In the above example, we have defined a variable of type Token and assigned the value of the regular expression to it. This example illustrates an important attribute of the SKIP specification. However, the following is not a legal input:. This is because the space character after xyz is in the SKIP category and therefore causes one token to end and another to begin.

This requires to be a separate token and hence does not match the grammar. The details of the matching algorithm are described in the JavaCC documentation. As a corollary, one must define as tokens anything within which characters such as white space characters must not be present. This section gives instructions on how to run the JJTree examples and the output you can expect to see.

This example is just the JavaCC grammar, with a little extra code in the parser's main method to call the dump method on the generated tree. It illustrates how the default behavior of JJTree will produce a tree of non-terminals. The only bit of JJTree-specific code is an action in the start production that dumps the constructed parse tree when the parse is complete. It uses JJTree simple mode. Running JavaCC in the normal way generates the remaining Java code.

Compile and run the Java program as usual. This example is the same grammar as Example1. It illustrates how unnecessary intermediate nodes can be suppressed, and how actions in the grammar can attach extra information to the nodes.

This is a modification of the first example to illustrate how the parse tree can be customized:. Look at Example. This example is a modification of Example2. This instructs JJTree to treat all undecorated non-terminals as if they were decorated as void.

The default JJTree behavior is to treat such non-terminals as if they were decorated with the name of the non-terminal. This is a modification of Example3. This instructs JJTree to insert a jjtAccept method into all nodes it generates, and to produce a visitor class.

The visitor is used to dump the tree. This example again can be run in the same manner as you ran Example2. One thing to take care in this case is that you must run jjtree on a clean directory that does not contain previously generated files. A quick and dirty way to do what most real parsers do, namely generate an abstract syntax tree, is to embed the tree classes in the JJ file, like so:. The toString method is convenient, though it might trick you into believing we made a code generator that targets LISP.

But the tree is really there. In a real compiler, you don't dump a main method into the parser. You'd make a nice function called parse perhaps, and call that from your own code. And you'd define the tree classes in their own files. Using JavaCC In practice, the scanning and parsing phases of a compiler are handled by code that is generated by a parser generator. One parser generator for Java is called JavaCC.

In this case, these methods are no-ops. Hence you can permanently leave these methods in your code and they automatically kick in when you use the debug options. To help the user in customizing error messages generated by the parser and lexer, the user is offered the facilities described in this section. To customize error reporting by the lexer, the lexer class must be subclassed and this method redefined in the subclass.

To help with creating your error reporting scheme, the following variables are available:. JJTree maintains some state in the parser object itself. It encapsulates all this state with an object that can be referred to via the jjtree field. The parser state implements an open stack where nodes are held until they can be added to their parent node.

Therefore, we need to change that in our JavaCC file. Because we will use our Java1. Part of the Java1. Do so by starting in the directory where the JavaCC file is found, then invoke the JavaCC executable, passing in the file:. The time has come to use the generated files to create a new NetBeans Lexer plugin.

In the Projects window, right-click the Libraries node, and choose Add Module Dependency, as shown below:. The first class you need to implement is org. TokenId represents one type of token. It has three properties:. The next class you need to implement is org. LanguageHierarchy provides a list of token types for our language and creates a new instance of our lexer.

The last class you need to implement is org. Lexer reads input text and returns tokens for it. In our case, the Lexer implementation needs to delegate to the lexer generated by JavaCC. Reader throws an IOException. However, though the LexerInput class logically corresponds to java. Reader , its read method does not throw a checked exception. Hence the BeginToken and the readChar methods below validate the returned character and throw the exception, if necesary.

After replacing the generated JavaCharStream with the code in the previous step, everything should compile and your module structure should be as follows:. In the next section, you register your NetBeans lexer so that the NetBeans Platform infrastructure can find it and load it into the application. In SJTokenId , define the following method, which returns an instance of org. Language :. The instance created statically above needs to be called from somewhere. The call is done from the layer.

When the module containing the above class is built, the generated-layer. If you are creating lexer support for your own application created on the NetBeans Platform, you explictly need to add the enablement module yourself, as explained below.

The code name base for this module is org. Now your NetBeans Platform application will be able to find the lexer support that you are creating in this tutorial. See the Appendix for the complete list of tokens. Copy them from the appendix into your module. Next, we need to map the categories to fonts and colors. This is done declaratively, in an XML file, where we list the categories and then declare the fonts and colors that should be applied.

In the main package of the module, that is, org. This file defines how to visualize the tokens produced by the lexer. The fontcolor tag properties are as follows:. All properties that are not defined explicitly are inherited from this default coloring. Default coloring is customizable in the Options window.