Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support compression in VMware sparse disk #115

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

chessbyte
Copy link
Member

@chessbyte chessbyte commented Feb 27, 2020

Much of the comments and understanding in this PR derives from the VMware Virtual Disk Format 5.0 Documentation and this repo

@chessbyte chessbyte changed the title Support compression in VMware sparse disk [WIP] Support compression in VMware sparse disk Feb 27, 2020
@chessbyte chessbyte force-pushed the support_compression_in_vmware_sparse_disk branch from 1cd4b2d to c30a383 Compare February 27, 2020 18:28
@chessbyte chessbyte changed the title [WIP] Support compression in VMware sparse disk Support compression in VMware sparse disk Feb 27, 2020
@chessbyte chessbyte force-pushed the support_compression_in_vmware_sparse_disk branch 8 times, most recently from 5548095 to 82cc7f9 Compare February 28, 2020 13:52
# 'K' D' 'M' 'V' 0x02 0x00 0x00 0x00

# flags contains the following bits of information.
'L', 'flags', # bit 0: valid new line detection test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a way to do bit level here, so you can actually give each of these flags an individual name here, and then you don't need the method that maps to a hash.

Copy link
Member

@Fryguy Fryguy Mar 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried doing this, but all the ways to do it are kind of clunky, so probably ok to go with what's here...

So I thought you could do

'b', :flag1,
'b', :flag2,
'b', :flag3,
...

Unfortunately the way the b identifier works is that it consumes 1 bit, then moves the iterator forward the rest of the byte. So, the only way to read consecutive bits is to do something like...

'b8', :flags

which then gives something like {:flags => "01100100"}.


I think BinaryStruct could be enhanced with some sort of bit decoding, but I'm not sure what a nice interface would look like offhand. I'm guessing the caller would just want to deal with bits individually, but under the covers as we build the format string we'd have to understand consecutive b definitions and pack them into a single b<N>.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That being said, it might be easier to use the bit decoding and do something like this...

# Instead of
'L',  'flags'

# use this
'b16' 'flags'

# and then immediately after decoding do:

sparseHeader["flags"] = sparseHeader["flags"].chars.map {|c| c == "1"}

# Then perhps something like...

sparseHeader["valid_new_line_detection_test"] = sparseHeader["flags"][0]
sparseHeader["redundant_grain_table_used"]    = sparseHeader["flags"][1]
sparseHeader["zero_grain_gte_used"]           = sparseHeader["flags"][2]
sparseHeader["grains_compressed"]             = sparseHeader["flags"][16]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be b32 since we need to access bit 0 as well as bit 16 and bit 17?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes...I typoed it.

Additionally, add support for USE_REDUNDANT_GRAIN_TABLE flag
Additionally, enhance the comments and logic around SPARSE_EXTENT_HEADER parsing.
@chessbyte chessbyte force-pushed the support_compression_in_vmware_sparse_disk branch from 3d37db2 to f6e9f29 Compare February 25, 2021 15:35
@miq-bot miq-bot added the stale label Feb 27, 2023
@miq-bot
Copy link
Member

miq-bot commented Feb 27, 2023

This pull request has been automatically marked as stale because it has not been updated for at least 3 months.

If these changes are still valid, please remove the stale label, make any changes requested by reviewers (if any), and ensure that this issue is being looked at by the assigned/reviewer(s)

Thank you for all your contributions! More information about the ManageIQ triage process can be found in the triage process documentation.

@miq-bot miq-bot closed this May 29, 2023
@miq-bot
Copy link
Member

miq-bot commented May 29, 2023

This pull request has been automatically closed because it has not been updated for at least 3 months.

Feel free to reopen this pull request if these changes are still valid.

Thank you for all your contributions! More information about the ManageIQ triage process can be found in the triage process documentation.

@Fryguy Fryguy reopened this Jul 27, 2023
@Fryguy Fryguy added pinned and removed stale labels Jul 27, 2023
@miq-bot
Copy link
Member

miq-bot commented Jul 27, 2023

Checked commits chessbyte/manageiq-smartstate@8521141~...f6e9f29 with ruby 2.6.10, rubocop 1.28.2, haml-lint 0.35.0, and yamllint
1 file checked, 36 offenses detected

lib/disk/modules/VMWareSparseDisk.rb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants