From 87f5599757bf6c596e9a8bb3a80fe2bdd53826ff Mon Sep 17 00:00:00 2001 From: Richard Pringle Date: Mon, 2 Oct 2023 13:14:06 -0400 Subject: [PATCH] Remove line about Docker in README (#302) --- README.md | 82 +++++++++++++++++++++++++++++++------------------------ 1 file changed, 46 insertions(+), 36 deletions(-) diff --git a/README.md b/README.md index f68868705..f3a884401 100644 --- a/README.md +++ b/README.md @@ -40,113 +40,123 @@ versions with no additional cost at all. ## Terminology -* `Revision` - A historical point-in-time state/version of the trie. This - represents the entire trie, including all `Key`/`Value`s at that point - in time, and all `Node`s. -* `View` - This is the interface to read from a `Revision` or a `Proposal`. -* `Node` - A node is a portion of a trie. A trie consists of nodes that are linked +- `Revision` - A historical point-in-time state/version of the trie. This + represents the entire trie, including all `Key`/`Value`s at that point + in time, and all `Node`s. +- `View` - This is the interface to read from a `Revision` or a `Proposal`. +- `Node` - A node is a portion of a trie. A trie consists of nodes that are linked together. Nodes can point to other nodes and/or contain `Key`/`Value` pairs. -* `Hash` - In this context, this refers to the merkle hash for a specific node. -* `Root Hash` - The hash of the root node for a specific revision. -* `Key` - Represents an individual byte array used to index into a trie. A `Key` +- `Hash` - In this context, this refers to the merkle hash for a specific node. +- `Root Hash` - The hash of the root node for a specific revision. +- `Key` - Represents an individual byte array used to index into a trie. A `Key` usually has a specific `Value`. -* `Value` - Represents a byte array for the value of a specific `Key`. Values can +- `Value` - Represents a byte array for the value of a specific `Key`. Values can contain 0-N bytes. In particular, a zero-length `Value` is valid. -* `Key Proof` - A proof that a `Key` exists within a specific revision of a trie. +- `Key Proof` - A proof that a `Key` exists within a specific revision of a trie. This includes the hash for the node containing the `Key` as well as all parents. -* `Range Proof` - A proof that consists of two `Key Proof`s, one for the start of +- `Range Proof` - A proof that consists of two `Key Proof`s, one for the start of the range, and one for the end of the range, as well as a list of all `Key`/`Value` pairs in between the two. A `Range Proof` can be validated independently of an actual database by constructing a trie from the `Key`/`Value`s provided. -* `Change Proof` - A proof that consists of a set of all changes between two +- `Change Proof` - A proof that consists of a set of all changes between two revisions. -* `Put` - An operation for a `Key`/`Value` pair. A put means "create if it doesn't +- `Put` - An operation for a `Key`/`Value` pair. A put means "create if it doesn't exist, or update it if it does. A put operation is how you add a `Value` for a specific `Key`. -* `Delete` - An operation indicating that a `Key` that should be removed from the trie. -* `Batch Operation` - An operation of either `Put` or `Delete`. -* `Batch` - An ordered set of `Batch Operation`s. -* `Proposal` - A proposal consists of a base `Root Hash` and a `Batch`, but is not +- `Delete` - An operation indicating that a `Key` that should be removed from the trie. +- `Batch Operation` - An operation of either `Put` or `Delete`. +- `Batch` - An ordered set of `Batch Operation`s. +- `Proposal` - A proposal consists of a base `Root Hash` and a `Batch`, but is not yet committed to the trie. In Firewood's most recent API, a `Proposal` is required to `Commit`. -* `Commit` - The operation of applying one or more `Proposal`s to the most recent +- `Commit` - The operation of applying one or more `Proposal`s to the most recent `Revision`. ## Roadmap **LEGEND** + - [ ] Not started - [ ] :runner: In progress - [x] Complete ### Green Milestone + This milestone will focus on additional code cleanup, including supporting concurrent access to a specific revision, as well as cleaning up the basic reader and writer interfaces to have consistent read/write semantics. + - [x] Concurrent readers of pinned revisions while allowing additional batches -to commit, to support parallel reads for the past consistent states. The revisions -are uniquely identified by root hashes. + to commit, to support parallel reads for the past consistent states. The revisions + are uniquely identified by root hashes. - [x] Pin a reader to a specific revision, so that future commits or other -operations do not see any changes. + operations do not see any changes. - [x] Be able to read-your-write in a batch that is not committed. Uncommitted -changes will not be shown to any other concurrent readers. + changes will not be shown to any other concurrent readers. - [x] Add some metrics framework to support timings and volume for future milestones -To support this, a new method Db::metrics() returns an object that can be serialized -into prometheus metrics or json (it implements [serde::Serialize]) + To support this, a new method Db::metrics() returns an object that can be serialized + into prometheus metrics or json (it implements [serde::Serialize]) ### Seasoned milestone + This milestone will add support for proposals, including proposed future branches, with a cache to make committing these branches efficiently. + - [x] Be able to support multiple proposed revisions against latest committed -version. + version. - [x] Be able to propose a batch against the existing committed revision, or -propose a batch against any existing proposed revision. + propose a batch against any existing proposed revision. - [x] Commit a batch that has been proposed will invalidate all other proposals -that are not children of the committed proposed batch. + that are not children of the committed proposed batch. - [x] Be able to quickly commit a batch that has been proposed. - [x] Remove RLP encoding ### Dried milestone + The focus of this milestone will be to support synchronization to other instances to replicate the state. A synchronization library should also be developed for this milestone. + - [ ] :runner: Add support for Ava Labs generic test tool via grpc client - [ ] :runner: Pluggable encoding for nodes, for optional compatibility with MerkleDB - [ ] Support replicating the full state with corresponding range proofs that -verify the correctness of the data. + verify the correctness of the data. - [ ] Support replicating the delta state from the last sync point with -corresponding range proofs that verify the correctness of the data. + corresponding range proofs that verify the correctness of the data. - [ ] Enforce limits on the size of the range proof as well as keys to make - synchronization easier for clients. + synchronization easier for clients. - [ ] MerkleDB root hash in parity for seamless transition between MerkleDB -and Firewood. + and Firewood. - [ ] Add metric reporting -- [ ] Migrate to a fully async interface, consider tokio\_uring, monoio, etc +- [ ] Migrate to a fully async interface, consider tokio_uring, monoio, etc - [ ] Refactor `Shale` to be more idiomatic, consider rearchitecting it ## Build + Firewood currently is Linux-only, as it has a dependency on the asynchronous -I/O provided by the Linux kernel (see `libaio`). Unfortunately, Docker is not -able to successfully emulate the syscalls `libaio` relies on, so Linux or a -Linux VM must be used to run Firewood. We intend to migrate to io\_uring which -should allow for this emulation. +I/O provided by the Linux kernel (see `libaio`). ## Run + There are several examples, in the examples directory, that simulate real world use-cases. Try running them via the command-line, via `cargo run --release --example simple`. ## Release + See the [release documentation](./RELEASE.md) for detailed information on how to release Firewood. ## CLI + Firewood comes with a CLI tool called `fwdctl` that enables one to create and interact with a local instance of a Firewood database. For more information, see the [fwdctl README](fwdctl/README.md). ## Test + ``` cargo test --release ``` ## License + Firewood is licensed by the Ecosystem License. For more information, see the [LICENSE file](./LICENSE.md).