Skip to content

Commit

Permalink
feat: add typescript bindings using wasm
Browse files Browse the repository at this point in the history
  • Loading branch information
Specy committed Sep 28, 2024
1 parent f71a5e4 commit 57f07a9
Show file tree
Hide file tree
Showing 20 changed files with 1,025 additions and 6 deletions.
40 changes: 38 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ jobs:
with:
command: test

publish:
publish-crates-io:
if: startsWith(github.ref, 'refs/tags/v')
needs: [ conformance, test ]

name: Publish
name: Publish to Crates.io
runs-on: "ubuntu-latest"
steps:
- name: Checkout
Expand All @@ -109,3 +109,39 @@ jobs:
run: cargo publish --token ${REGISTRY_TOKEN}
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}

publish-npm:
if: startsWith(github.ref, 'refs/tags/v')
needs: [ conformance, test ]

name: Publish to NPM
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: nightly
override: true

- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: '20'

- name: Cd in to the node bindings
run: cd bindings/typescript

- name: Install Dependencies
run: npm install

- name: Build
run: npm run build

- name: Publish
run: npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
21 changes: 20 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ authors = [

[dependencies]
clap = { version = "4.5", optional = true, features = ["derive"] }
colored = { version = "2.1" }
console_error_panic_hook = { version = "0.1.1", optional = true }
dirs = { version = "5.0", optional = true }
indexmap = { version = "2.3" }
itertools = { version = "0.13" }
Expand All @@ -21,16 +21,34 @@ prettytable-rs = { version = "0.10" }
ptree = { version = "0.5" }
regex = { version = "1.10" }
rustyline = { version = "14.0", optional = true }
serde_renamed = { package = "serde", version = "1.0", features = ["derive"], optional = true }
serde-wasm-bindgen = { version = "0.6.5", optional = true }
smallvec = { version = "1.13" }
smol_str = { version = "0.3" }
thiserror = { version = "1.0" }
wasm-bindgen = { version = "0.2.83", optional = true }

[target.'cfg(not(target_family = "wasm"))'.dependencies]
colored = { version = "2.1" }

[dev-dependencies]
criterion = { version = "0.5", features = ["html_reports"] }

[features]
default = ["repl"]
repl = ["clap", "dirs", "rustyline"]
serde = [
"indexmap/serde",
"serde_renamed",
"smol_str/serde",
"smallvec/serde",
]
wasm = [
"console_error_panic_hook",
"serde",
"serde-wasm-bindgen",
"wasm-bindgen",
]

[profile.release]
lto = "fat"
Expand All @@ -40,6 +58,7 @@ codegen-units = 1
result_large_err = "allow"

[lib]
crate-type = ["cdylib", "rlib"]
bench = false
doctest = false

Expand Down
24 changes: 24 additions & 0 deletions bindings/typescript/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*

node_modules
dist
dist-ssr
*.local

# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
104 changes: 104 additions & 0 deletions bindings/typescript/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
## Overview

`dotlr` is a library for creating and inspecting LR family of parsers in TypeScript. It provides an interface to parse grammars, generate parsing tables, and trace parsing of inputs. The library leverages WebAssembly (WASM) to ensure efficient parsing.

It is focused on providing educational resources for learning about parsing algorithms and compiler construction. The library is designed to be easy to use and understand, making it ideal for students, educators, and developers interested in language processing.

### Table of Contents
1. [Installation](#installation)
2. [Basic Usage](#basic-usage)
3. [Defining a Grammar](#defining-a-grammar)
4. [Creating LR(1) Parser of the Grammar](#creating-lr1-parser-of-the-grammar)
5. [Creating LALR(1) Parser of the Grammar](#creating-lalr1-parser-of-the-grammar)

## Installation

Before using the `dotlr` library, you need to install it. The following instructions assume you have a project with `npm` already set up.

```bash
npm install dotlr
```

### Importing the Library

To use the `dotlr` library, import it into your TypeScript files:

```ts
import { Grammar, LR1Parser, LALRParser } from 'dotlr';
```
this library uses `ts-results` under the hood to handle errors and results.
```ts
import { Ok, Err } from 'ts-results';
```
## Basic Usage

The core of the `dotlr` library revolves around defining a grammar and using it to create a parser. The following steps will guide you through this process.

## Defining a Grammar

A grammar is a set of rules that define how input strings can be parsed. You can create a grammar using `Grammar.parse()` method. Here's an example:

For more information on the syntax of the grammar, look [here](https://github.com/umut-sahin/dotlr?tab=readme-ov-file#usage)

```ts
const grammarStr = `
S -> A
A -> 'a' A
A -> 'b'
`;

const grammarResult = Grammar.parse(grammarStr);

if (grammarResult.ok) {
const grammar = grammarResult.val;
console.log("Grammar successfully parsed!");
console.log(grammar.getSymbols());
console.log(grammar.getProductions());
} else {
console.error("Failed to parse grammar:", grammarResult.val);
}
```

- **Grammar.parse()**: Parses a string representation of a grammar and returns a `Grammar` object.
- **grammar.getSymbols()**: Returns all symbols (non-terminal and terminal) used in the grammar.
- **grammar.getProductions()**: Retrieves the list of productions (rules) defined in the grammar.

## Creating LR(1) Parser of the Grammar

The `LR1Parser` class allows you to create an LR(1) parser for the grammar and use it to parse input.

```ts
const lr1ParserResult = LR1Parser.fromGrammar(grammar);

if (lr1ParserResult.ok) {
const lr1Parser = lr1ParserResult.val;

const input = "aab";
const parseResult = lr1Parser.parse(input);

if (parseResult.ok) {
const parseTree = parseResult.val;
console.log("Parse successful!");
console.log(parseTree);
} else {
console.error("Parse error:", parseResult.val);
}
} else {
console.error("Failed to create LR(1) parser:", lr1ParserResult.val);
}
```

- **LR1Parser.fromGrammar()**: Consumes the `Grammar` object and returns an `LR1Parser`, you cannot reuse the *Grammar* object, if you need it, you can clone it by using `grammar.clone()`.
- **parser.parse()**: method attempts to parse the given input string according to the LR(1) grammar. Returns a parse tree if successful.
- **parser.trace()** method can be used to trace the parsing process. It returns a trace and the resulting parse tree at each step, if parsing is successful.
- **parser.tokenize()** method can be used to tokenize the input string. It returns a list of tokens.
- **parser.getActionTable()** method returns the action table of the parser, which is used to determine the next action based on the current state and input token.
- **parser.getGotoTable()** method returns the goto table of the parser, which is used to determine the next state based on the current state and non-terminal symbol.
- **parser.getParseTables()** method returns the parsing tables of the parser, which include the action and goto tables.
- **parser.getAutomaton()** method returns the automaton of the parser, which represents the states and transitions of the LR(1) parser.
- **parser.getFirstTable()** method returns the first table of the parser, which contains the first sets of symbols.
- **parser.getFollowTable()** method returns the follow table of the parser, which contains the follow sets of symbols.

## Creating LALR(1) Parser of the Grammar

The `LALR1Parser` is similar to the `LR1Parser`, but it uses Look-Ahead LR parsing, the API is the same.
16 changes: 16 additions & 0 deletions bindings/typescript/build.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import {execSync} from 'child_process';
import fs from "fs/promises"


async function init() {
console.log("Starting build...")
execSync('tsc', {stdio: 'inherit'});
await fs.cp("./src/pkg", "./dist/pkg", {recursive: true});
await fs.unlink("./dist/pkg/package.json");
await fs.unlink("./dist/pkg/README.md");
await fs.unlink("./dist/pkg/.gitignore");
console.log("Build complete")

}

init()
45 changes: 45 additions & 0 deletions bindings/typescript/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

35 changes: 35 additions & 0 deletions bindings/typescript/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"type": "module",
"name": "dotlr",
"description": "An LR(1) parser generator and visualizer created for educational purposes.",
"keywords": [
"educational",
"lalr-parsing",
"lr-parsing",
"parser-generator"
],
"license": "MIT OR Apache-2.0",
"version": "0.1.18",
"main": "dist/index.js",
"exports": {
".": "./dist/index.js",
"./types": "./dist/types.js",
"./utils": "./dist/utils.js"
},
"module": "dist/index.js",
"typings": "dist/index.d.ts",
"repository": "https://github.com/umut-sahin/dotlr",
"scripts": {
"build": "npm i -g wasm-pack && npm run build-all",
"build-all": "npm run build-wasm:no-test && npm run build-lib",
"build-wasm": "cd .. && cargo test && wasm-pack build --out-dir bindings/typescript/src/pkg --out-name dotlr --features wasm --no-default-features",
"build-wasm:no-test": "cd .. && wasm-pack build --out-dir bindings/typescript/src/pkg --out-name dotlr --features wasm --no-default-features",
"build-lib": "node build.js"
},
"devDependencies": {
"typescript": "^5.5.0"
},
"dependencies": {
"ts-results": "^3.3.0"
}
}
Loading

0 comments on commit 57f07a9

Please sign in to comment.