Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add proposal #181

Merged
merged 1 commit into from
Aug 4, 2023
Merged

feat: add proposal #181

merged 1 commit into from
Aug 4, 2023

Conversation

xinifinity
Copy link
Contributor

This commits add the proposal feature which satisfies all the requirements from seasoned milestone for proposal, namely:

  • support multiple proposed revisions against latest committed version.
  • you can propose a batch against the existing committed revision, or propose a batch against any existing proposed revision.
  • commit a batch that has been proposed. Note that this invalidates all other proposals that are not children of the committed proposed batch.

On top, you can get a DbRev from the Proposal, which you can get the root hash or get (range) proof, etc.

Copy link
Collaborator

@rkuris rkuris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but I think this code could be a lot cleaner with a macro. If you don't like macros, then a helper function would work too.

Comment on lines +546 to +552
None => match prev_deltas.get(&e_pid) {
Some(p) => data.extend(&p[..e_off + 1]),
None => data.extend(
self.base_space
.get_slice(e_pid << PAGE_SIZE_NBIT, e_off as u64 + 1)?,
),
},
Copy link
Collaborator

@rkuris rkuris Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: nearly similar changes 3 times warrants a new function. Something like this (error handling needs work):

Suggested change
None => match prev_deltas.get(&e_pid) {
Some(p) => data.extend(&p[..e_off + 1]),
None => data.extend(
self.base_space
.get_slice(e_pid << PAGE_SIZE_NBIT, e_off as u64 + 1)?,
),
},
None => data.extend(self.from_prev(&e_pid, e_off)),

and

fn from_prev(&'a self, prev_deltas, page_id, offset) -> &'a [u8] {
    match prev_deltas.get(&e_pid) {
        Some(p) => &p[..offset + 1],
        None => self.base_space.get_slice(page_id << PAGE_SIZE_NBIT, offset as u64 + 1).unwrap(),
    }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are not exactly same logic for each case, maybe instead a macro would work?

fn db_proposal() {
let cfg = DbConfig::builder().wal(WalConfig::builder().max_revisions(10).build());

let db = Db::new("test_db_proposal", &cfg.clone().truncate(true).build())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a tempdir. We seem to do this all over! Maybe we should make a ticket to centralize fetching a test directory, as a method on Db like Db::test("name") that only appears with #[cfg(test)] or something?

firewood/tests/db.rs Show resolved Hide resolved
}];
let proposal_2 = proposal.clone().propose(batch_2).unwrap();
let rev = proposal_2.get_revision();
let val = rev.kv_get(b"k");
Copy link
Collaborator

@rkuris rkuris Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this repeated code could use a macro:

Suggested change
let val = rev.kv_get(b"k");
assert_val!(rev, "k", "v");

and

macro_rules! assert_val {
    ($rev: expr, $key:literal, $expected_val:literal) => {
        let actual = $rev.kv_get($key.as_bytes()).unwrap();
        assert_eq!(actual, $expected_val.as_bytes().to_vec());
    };
}

@xinifinity
Copy link
Contributor Author

Approving, but I think this code could be a lot cleaner with a macro. If you don't like macros, then a helper function would work too.

Thanks! I will address the comment in a follow up.

@xinifinity xinifinity merged commit e36f99b into main Aug 4, 2023
5 checks passed
@xinifinity xinifinity deleted the proposal branch August 4, 2023 17:12
@rkuris
Copy link
Collaborator

rkuris commented Aug 8, 2023 via email

let mut committed = self.committed.lock();
if *committed {
return Ok(());
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xinifinity @rkuris
🤔 , should we change the function signature here so that self is consumed? I think it would make more sense to have a DbError::Invalid_Proposal(Proposal<Store, SharedStore>) that one has to explicitly unwrap if you want to use a proposal that can't be committed.


if let ProposalBase::Proposal(p) = &self.parent {
p.commit()?;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might make the above comment difficult to implement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think there's an interesting race condition here. We could end up committing the parent, but not committing this proposal... that's unintuitive to me and does not behave like a transaction would.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After talking to @rkuris, it seems like the result would be the same, so it doesn't really matter which context the proposal-base commit was actually triggered by.

I think it might be a good idea to leave a comment here stating that though.

Comment on lines +1136 to +1137
m: Arc<RwLock<DbInner<S>>>,
r: Arc<Mutex<DbRevInner<T>>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use full names instead of letters, please?

Comment on lines +1225 to +1227
let parent_root_hash = p.rev.kv_root_hash().ok();
let parent_root_hash =
parent_root_hash.expect("parent_root_hash should not be none");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we call .ok() then .expect. We should just use expect AND add a TODO to handle the error properly

Comment on lines +1233 to +1235
let parent_root_hash = p.kv_root_hash().ok();
let parent_root_hash =
parent_root_hash.expect("parent_root_hash should not be none");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here.

Comment on lines +1245 to +1246
let kv_root_hash = self.rev.kv_root_hash().ok();
let kv_root_hash = kv_root_hash.expect("kv_root_hash should not be none");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants