Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New very fast unixfs implementation. #347

Closed
wants to merge 41 commits into from
Closed

New very fast unixfs implementation. #347

wants to merge 41 commits into from

Conversation

Jorropo
Copy link
Contributor

@Jorropo Jorropo commented Jun 12, 2023

Depends on ipfs/go-cid#159.

DagPB + Unixfs parsing with gogoproto is ~1500ns here it is ~250ns.
I'll do boxo/ipld/unixfs vs go-unixfsnode vs boxo/unixfs decoding in the future.

It can download single block raw leaves files over car.
I first meant it to be used for unixfs correctness checking, however 
later I've implemented it for range requests.

This make it fail at both, so let's keep correctness, I'll implement 
range requests later.
This fixes bug introduced in 4707f58.

This is was completely wrong (does not load the digest and incorrectly truncate the data incorrectly.)
@Jorropo Jorropo force-pushed the unixfs branch 2 times, most recently from 4b123c2 to 234debc Compare June 12, 2023 09:25
@codecov
Copy link

codecov bot commented Jun 12, 2023

Codecov Report

Merging #347 (5b77262) into main (56861d6) will decrease coverage by 1.29%.
Report is 80 commits behind head on main.
The diff coverage is 42.50%.

❗ Current head 5b77262 differs from pull request most recent head 2599f37. Consider uploading reports for the commit 2599f37 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #347      +/-   ##
==========================================
- Coverage   66.41%   65.13%   -1.29%     
==========================================
  Files         207      211       +4     
  Lines       24692    26194    +1502     
==========================================
+ Hits        16400    17062     +662     
- Misses       6866     7590     +724     
- Partials     1426     1542     +116     
Files Coverage Δ
unixfs/unixfs.go 54.16% <54.16%> (ø)
unixfs/json.go 0.00% <0.00%> (ø)
unixfs/feather/entry.go 53.35% <53.35%> (ø)
unixfs/pb.go 37.41% <37.41%> (ø)

... and 63 files with indirect coverage changes

@Jorropo Jorropo force-pushed the unixfs branch 3 times, most recently from b56b758 to c7b9772 Compare June 12, 2023 10:26
```
$ benchstat /mnt/ramdisk/{old,new}
name   old time/op    new time/op    delta
PB-12     730ns ± 5%     255ns ± 2%   -65.04%  (p=0.000 n=9+10)

name   old alloc/op   new alloc/op   delta
PB-12      224B ± 0%        0B       -100.00%  (p=0.000 n=10+10)

name   old allocs/op  new allocs/op  delta
PB-12      4.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)
```
@Jorropo Jorropo force-pushed the unixfs branch 2 times, most recently from 7f0f457 to 5b77262 Compare November 18, 2023 18:53
@Jorropo
Copy link
Contributor Author

Jorropo commented Mar 4, 2024

Will release this in a new repo

@Jorropo Jorropo closed this Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: 🎉 Done
Development

Successfully merging this pull request may close these issues.

1 participant