Commits on Oct 10, 2023

1. refactor: simplify unpackOne, drop unused param to ExtractSingleSquash …

   unpackOne required the caller to provide it with a boolean 'isSquashfs'
   which then made each caller have to consider the layer type.
   
   Update unpackOne to take a Descriptor and let it do the
   determination of how to unpack the layer in a single place.
   
   The result of calling unpackLayer is either error, or the contents
   available at the provided path.  The caller does not have to check
   if the content is already present.
   
   Also here, drop the 'storage' parameter to ExtractSingleSquash
   that had become unused.
   
   Signed-off-by: Scott Moser <[email protected]>

   

   smoser committed Oct 10, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for f6c9a49
   • Browse repository at this point

   Copy the full SHA

   f6c9a49 View commit details

   Browse the repository at this point in the history

2. fix: Update ExtractSingleSquash, adding policy. …

   Overall, there are 3 "good things" done by this change:
   1. Fix bug in the current code which tries mounting with each option
      every time.  The problem with doing that is really that the kernel
      mount option didn't work very well.  It would fail with "already
      mounted", and then squashfuse would end up getting used to mount over
      the top.
   
   2. Fix race conditions in the current code.
   
      overlay.Unpack starts a thread pool and tries to unpack all layers
      at once.  That is fine if there are no duplicate layers.  But
         if there are duplicate layers used by a stacker.yaml file, then
      there were races on extraction.  The end result really was that things
      would get mounted more than once.
   
      Example stacker that shows this:
   
       l1:
         from:
           type: docker
           url: docker://busybox:latest
         run: |
           echo build layer 1
   
       l2:
         from:
           type: docker
           url: docker://busybox:latest
         run: |
           echo build layer 1
   
     There, the busybox layer would get extracted multiple times.
   
     The code here has a single lock on ExtractSingleSquash, it would
     be better to have lock being taken per extractDir.
   
   3. Allow the user to control the list of extractors.
   
      If they knew that they could not use kernel mounts (or could, but
      didn't want to) or wanted to use unsquashfs they can now do that.
   
      STACKER_SQUASHFS_EXTRACT_POLICY=kmount stacker build ..
   
      or
   
      STACKER_SQUASHFS_EXTRACT_POLICY="squashfuse kmount" stacker build ...
   
      This adds a SquashExtractor interface, with 3 implementers
      (KernelExtractor, SquashFuseExtractor, UnsquashfsExtractor).
   
      A ExtractPolicy is basically a list of Extractors to try.
      The first time ExtractPolicy is used it will try each of the Extractors
      in order.  It then stores the result in .Extractor and uses that
      subsequently.
   
   Signed-off-by: Scott Moser <[email protected]>

   

   smoser committed Oct 10, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 9795f43
   • Browse repository at this point

   Copy the full SHA

   9795f43 View commit details

   Browse the repository at this point in the history

3. fix: Improve and add some debug messages in overlay storage. …

   This seeks to improve some of the existing debug messages and add some
   additional debug in the overlay storage.
   
   Signed-off-by: Scott Moser <[email protected]>

   

   smoser committed Oct 10, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for fd41a4b
   • Browse repository at this point

   Copy the full SHA

   fd41a4b View commit details

   Browse the repository at this point in the history