Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Register memory mapped files and raise when written to #16208

Merged
merged 1 commit into from
May 14, 2024
Merged

Conversation

ritchie46
Copy link
Member

It isn't perfect, but this will at least catch the most common case in python where people read memory mapped and write to the same file.

fixes #16119

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels May 14, 2024
Copy link

codecov bot commented May 14, 2024

Codecov Report

Attention: Patch coverage is 98.64865% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.03%. Comparing base (8066952) to head (43af84a).
Report is 4 commits behind head on main.

Files Patch % Lines
...olars-lazy/src/physical_plan/executors/scan/ipc.rs 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16208      +/-   ##
==========================================
+ Coverage   80.99%   81.03%   +0.04%     
==========================================
  Files        1392     1392              
  Lines      178924   178978      +54     
  Branches     2901     2907       +6     
==========================================
+ Hits       144911   145037     +126     
+ Misses      33508    33435      -73     
- Partials      505      506       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

codspeed-hq bot commented May 14, 2024

CodSpeed Performance Report

Merging #16208 will improve performances by 29.2%

Comparing mmap (43af84a) with main (5b77f01)

Summary

⚡ 1 improvements
✅ 34 untouched benchmarks

Benchmarks breakdown

Benchmark main mmap Change
test_filter2 2.8 ms 2.1 ms +29.2%

@ritchie46 ritchie46 merged commit 07d58ab into main May 14, 2024
27 checks passed
@ritchie46 ritchie46 deleted the mmap branch May 14, 2024 07:53
MarcoGorelli pushed a commit to MarcoGorelli/polars that referenced this pull request May 15, 2024
Wouittone pushed a commit to Wouittone/polars that referenced this pull request Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

write_ipc overwrites value in previously loaded dataframe
1 participant