Skip to content

Commit

Permalink
Remove line about Docker in README (#302)
Browse files Browse the repository at this point in the history
  • Loading branch information
richardpringle authored Oct 2, 2023
1 parent 77cbfb3 commit 87f5599
Showing 1 changed file with 46 additions and 36 deletions.
82 changes: 46 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,113 +40,123 @@ versions with no additional cost at all.

## Terminology

* `Revision` - A historical point-in-time state/version of the trie. This
represents the entire trie, including all `Key`/`Value`s at that point
in time, and all `Node`s.
* `View` - This is the interface to read from a `Revision` or a `Proposal`.
* `Node` - A node is a portion of a trie. A trie consists of nodes that are linked
- `Revision` - A historical point-in-time state/version of the trie. This
represents the entire trie, including all `Key`/`Value`s at that point
in time, and all `Node`s.
- `View` - This is the interface to read from a `Revision` or a `Proposal`.
- `Node` - A node is a portion of a trie. A trie consists of nodes that are linked
together. Nodes can point to other nodes and/or contain `Key`/`Value` pairs.
* `Hash` - In this context, this refers to the merkle hash for a specific node.
* `Root Hash` - The hash of the root node for a specific revision.
* `Key` - Represents an individual byte array used to index into a trie. A `Key`
- `Hash` - In this context, this refers to the merkle hash for a specific node.
- `Root Hash` - The hash of the root node for a specific revision.
- `Key` - Represents an individual byte array used to index into a trie. A `Key`
usually has a specific `Value`.
* `Value` - Represents a byte array for the value of a specific `Key`. Values can
- `Value` - Represents a byte array for the value of a specific `Key`. Values can
contain 0-N bytes. In particular, a zero-length `Value` is valid.
* `Key Proof` - A proof that a `Key` exists within a specific revision of a trie.
- `Key Proof` - A proof that a `Key` exists within a specific revision of a trie.
This includes the hash for the node containing the `Key` as well as all parents.
* `Range Proof` - A proof that consists of two `Key Proof`s, one for the start of
- `Range Proof` - A proof that consists of two `Key Proof`s, one for the start of
the range, and one for the end of the range, as well as a list of all `Key`/`Value`
pairs in between the two. A `Range Proof` can be validated independently of an
actual database by constructing a trie from the `Key`/`Value`s provided.
* `Change Proof` - A proof that consists of a set of all changes between two
- `Change Proof` - A proof that consists of a set of all changes between two
revisions.
* `Put` - An operation for a `Key`/`Value` pair. A put means "create if it doesn't
- `Put` - An operation for a `Key`/`Value` pair. A put means "create if it doesn't
exist, or update it if it does. A put operation is how you add a `Value` for a
specific `Key`.
* `Delete` - An operation indicating that a `Key` that should be removed from the trie.
* `Batch Operation` - An operation of either `Put` or `Delete`.
* `Batch` - An ordered set of `Batch Operation`s.
* `Proposal` - A proposal consists of a base `Root Hash` and a `Batch`, but is not
- `Delete` - An operation indicating that a `Key` that should be removed from the trie.
- `Batch Operation` - An operation of either `Put` or `Delete`.
- `Batch` - An ordered set of `Batch Operation`s.
- `Proposal` - A proposal consists of a base `Root Hash` and a `Batch`, but is not
yet committed to the trie. In Firewood's most recent API, a `Proposal` is required
to `Commit`.
* `Commit` - The operation of applying one or more `Proposal`s to the most recent
- `Commit` - The operation of applying one or more `Proposal`s to the most recent
`Revision`.

## Roadmap

**LEGEND**

- [ ] Not started
- [ ] :runner: In progress
- [x] Complete

### Green Milestone

This milestone will focus on additional code cleanup, including supporting
concurrent access to a specific revision, as well as cleaning up the basic
reader and writer interfaces to have consistent read/write semantics.

- [x] Concurrent readers of pinned revisions while allowing additional batches
to commit, to support parallel reads for the past consistent states. The revisions
are uniquely identified by root hashes.
to commit, to support parallel reads for the past consistent states. The revisions
are uniquely identified by root hashes.
- [x] Pin a reader to a specific revision, so that future commits or other
operations do not see any changes.
operations do not see any changes.
- [x] Be able to read-your-write in a batch that is not committed. Uncommitted
changes will not be shown to any other concurrent readers.
changes will not be shown to any other concurrent readers.
- [x] Add some metrics framework to support timings and volume for future milestones
To support this, a new method Db::metrics() returns an object that can be serialized
into prometheus metrics or json (it implements [serde::Serialize])
To support this, a new method Db::metrics() returns an object that can be serialized
into prometheus metrics or json (it implements [serde::Serialize])

### Seasoned milestone

This milestone will add support for proposals, including proposed future
branches, with a cache to make committing these branches efficiently.

- [x] Be able to support multiple proposed revisions against latest committed
version.
version.
- [x] Be able to propose a batch against the existing committed revision, or
propose a batch against any existing proposed revision.
propose a batch against any existing proposed revision.
- [x] Commit a batch that has been proposed will invalidate all other proposals
that are not children of the committed proposed batch.
that are not children of the committed proposed batch.
- [x] Be able to quickly commit a batch that has been proposed.
- [x] Remove RLP encoding

### Dried milestone

The focus of this milestone will be to support synchronization to other
instances to replicate the state. A synchronization library should also
be developed for this milestone.

- [ ] :runner: Add support for Ava Labs generic test tool via grpc client
- [ ] :runner: Pluggable encoding for nodes, for optional compatibility with MerkleDB
- [ ] Support replicating the full state with corresponding range proofs that
verify the correctness of the data.
verify the correctness of the data.
- [ ] Support replicating the delta state from the last sync point with
corresponding range proofs that verify the correctness of the data.
corresponding range proofs that verify the correctness of the data.
- [ ] Enforce limits on the size of the range proof as well as keys to make
synchronization easier for clients.
synchronization easier for clients.
- [ ] MerkleDB root hash in parity for seamless transition between MerkleDB
and Firewood.
and Firewood.
- [ ] Add metric reporting
- [ ] Migrate to a fully async interface, consider tokio\_uring, monoio, etc
- [ ] Migrate to a fully async interface, consider tokio_uring, monoio, etc
- [ ] Refactor `Shale` to be more idiomatic, consider rearchitecting it

## Build

Firewood currently is Linux-only, as it has a dependency on the asynchronous
I/O provided by the Linux kernel (see `libaio`). Unfortunately, Docker is not
able to successfully emulate the syscalls `libaio` relies on, so Linux or a
Linux VM must be used to run Firewood. We intend to migrate to io\_uring which
should allow for this emulation.
I/O provided by the Linux kernel (see `libaio`).

## Run

There are several examples, in the examples directory, that simulate real world
use-cases. Try running them via the command-line, via `cargo run --release
--example simple`.

## Release

See the [release documentation](./RELEASE.md) for detailed information on how to release Firewood.

## CLI

Firewood comes with a CLI tool called `fwdctl` that enables one to create and interact with a local instance of a Firewood database. For more information, see the [fwdctl README](fwdctl/README.md).

## Test

```
cargo test --release
```

## License

Firewood is licensed by the Ecosystem License. For more information, see the
[LICENSE file](./LICENSE.md).

0 comments on commit 87f5599

Please sign in to comment.