JuliaSyntax.jl: A new Julia compiler frontend in Julia

JuliaSyntax.jl is a new Julia language frontend designed for precise error reporting, speed and flexibility. In this talk we'll tour the JuliaSyntax parser implementation and tree data structures, highlighting benefits for users and tool builders. We'll discuss how to losslessly map Julia source text for character-precise error reporting and how a "parse stream" abstraction cleanly separates the parser from syntax tree creation while being 10x faster than Julia's reference parser.

JuliaSyntax aims to be a complete compiler frontend (parser, data structures and code lowering) designed for the growing needs of the julia community, as split broadly into users, tool authors and core developers.

For users we need interactivity and precision

• Speed: Bare parsing is 20x faster; parsing to a basic tree 10x faster, and parsing to Expr around 6x faster
• Robustness: The parse tree covers the full source text regardless of syntax errors so partially complete source text can be processed in use cases like editor tooling and REPL completions
• Precision: We map every character of the source text so highlighting of errors or other compiler diagnostics can be fully precise

For tool authors, JuliaSyntax aims to be accessible and flexible

• Lossless parsing accounts for all source text, including comments and whitespace so that tools can faithfully work with the source code.
• We support Julia source code versions different from the Julia version running JuliaSyntax itself so only one tooling deployment is needed per machine
• JuliaSyntax is hackable and accessible to the community, due to being written in Julia itself
• Layered tree data structures support various use cases from code formatting to semantic analysis.

For core developers, JuliaSyntax aims to provide familiarity and easy of integration:

• The code mirrors the structure of the flisp parser
• It depends only on Base
• The syntax tree data structures are hackable independently from the parser implementation.

For a detailed description of the package aims and current status, see the source repository documentation on github at https://github.com/JuliaLang/JuliaSyntax.jl#readme