lab.clj.antlr

Created: 2014-05-19 14:33
Updated: 2017-12-19 11:30
License: epl-1.0

README.md

lab.antlr

This project is intended as a kind of research lab for exploring ANTLR 4 with Clojure. I do not expect to ever publish it as a product, so it does not have an alpha-beta-production status. But I do expect to add to it from time to time. What's included here is enough (I hope) machinery and documentation to get you started working with ANTLR grammars right away, without having to worry about the boring details of running the ANTLR tools, organizing your code, etc ). What's here seems to work, so if you want to use it, just clone it and hammer away.

The original motivation was to develop a formal grammar for Clojure, so some of the examples are specific to that language. But the tools and documentation here are language-agnostic; ANTLR 4 can be used to develop any grammar, and you can use the tools here to experiment with any grammar. Once I get a Clojure grammar worked up it will go in a different repo.

See the ANTLR Grammar Repository for many examples, including a rudimentary Clojure grammar.

WARNING ANTRL 4 is substantially different from earlier versions, and many of the examples you can find by searching the internet are based on earlier versions. Caveat lector.

See the docs in doc for more information. If you're going to do serious grammar development, get a copy of The Definitive ANTLR 4 Reference.

Organization

This project exploits leiningen's conventions:

  • Grammar files go in src/antlr
  • Generated java implementation code goes in src/java
  • Compiled parser classes go in target
  • Test data is in test/data
  • Clojure source (a few testing tools) is in src/clojure
  • Clojure test files are in test exclusive of test/data

Shell scripts in the project root directory:

Usage

To get started, clone the repo and then from the project root run:

1.  $ ./antlr4.sh hello.g4     ## generate java parser source files from src/grammar/Clojure.g4
2.  $ lein javac              ## compile parser source to target/classes

Caveat: It's "antlr", not "antrl". ;)

This generates the lexer and parser Java code from the ANTLR grammar (src/antlr/hello.g4), then compiles it. Use regen.sh as you work on your grammar; it combines clean, gen, compile steps.

The Clojurish Thing To Do is to write a leiningen plugin to run ANTLR, so we can do $ lein antlr hello, but I haven't gotten around to it yet.

Testing

To test the parser, you have two options. The ANTLR way:

$ ./grun.sh hello start "hello.txt" ## run ANTLR TestRig against parser and test file

grun.sh runs ANTLR's TestRig; see ANTLR 4 TestRig for further information.

The Clojure way(s):

$ lein test hello

Another Clojure way is to run the parser from a repl; see Testing ANTLR 4 Lexers/Parsers with Clojure for details.

Lexing

See ANTLR 4 Lexing.

Parsing

See ANTLR 4 Parsing.

Crawling the tree

See Crawling the tree

Handlers

The point of crawling the tree is to do something at each node. That is the responsibility of the node Handlers.

Client Testing

See Testing Parser Clients for information about testing application-specific client code.

Road map

  • implement ANTLR visitors
  • implement lein plugin
  • implement lein template

See also

Getting Started with ANTRL v4 by Terrence Parr, the maniac behind ANTLR (his words, not mine!)

Using Antlr from Clojure A blog post (March 2014) with a minimal example of how to parse Java files using ANTLR 4 from Clojure, without the lein infrastructure we use.

clj-antlr is a more ambitious effort to Clojurize ANTLR.

.

Cookies help us deliver our services. By using our services, you agree to our use of cookies Learn more