-
Notifications
You must be signed in to change notification settings - Fork 435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement leveldb storage for dataposter #1736
Conversation
…re already in their own packages)
…from leveldb package
…re already in their own packages)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI it looks like tests are failing due to cmp, and I think the linter's complaining about "printf: (*testing.common).Errorf call needs 2 args but has 3 args (govet)" for storage_test.go line 182
func (s *Storage[Item]) GetLast(ctx context.Context) (*Item, error) { | ||
lastItemIdx, err := s.lastItemIdx(ctx) | ||
if err != nil { | ||
return nil, fmt.Errorf("getting last item index: %w", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should return nil, nil
if the last item index is missing (i.e. there are no items). We should also think about what happens when all items are pruned. In that case, we probably want to have pruning remove the last item index from the db.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Checking length, if it's 0 then returning nil, nil. So no need to remove last item idx from db when pruning (for all public methods behavior will be exactly the same).
} | ||
if err := b.Put(countKey, []byte(strconv.Itoa(cnt+1))); err != nil { | ||
return fmt.Errorf("updating length counter: %w", err) | ||
if bytes.Compare(key, lastItemIdx) > 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't love assuming the return value of lastItemIdx is empty if leveldb.ErrNotFound is returned. In practice I'm sure it'll be empty, but I don't think it's good Go practice to read the value returned in an error is present.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
@@ -71,6 +71,8 @@ func (s *Storage[Item]) Put(_ context.Context, index uint64, prevItem *Item, new | |||
if queueIdx > len(s.queue) { | |||
return fmt.Errorf("attempted to set out-of-bounds index %v in queue starting at %v of length %v", index, s.firstNonce, len(s.queue)) | |||
} | |||
// TODO: come up with a better way of comparing prev/new values, | |||
// we shouldn't use cmp.Diff outside tests. | |||
if diff := cmp.Diff(prevItem, s.queue[queueIdx]); diff != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI the original code did intentionally use pointer comparisons, because that worked with the batch poster's design. But I agree it isn't ideal. We should either RLP encode both and compare them here, or we could use reflect.DeepEqual.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
The merge-base changed after approval.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
No description provided.