Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

caching: Don't try to re-download Docker images which already exist #249

Open
ethanjli opened this issue Jun 14, 2024 · 0 comments
Open
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ethanjli
Copy link
Member

ethanjli commented Jun 14, 2024

Currently, [dev] plt stage and stage set-next by default attempt to download all container images - even those which are already available in the Docker daemon's container storage; this will lead to a failure+error if we have no internet connection, even if all images are already in the Docker daemon's container storage. The only way to avoid that situation is to add an optional --no-cache-img flag to [dev] plt stage/stage set-next/etc. It would be better if we didn't need to provide that flag in most cases, and we instead changed the default behavior of [dev] plt stage to skip attempting to download any container images which are already in the Docker daemon (by querying the Docker daemon); usually all container images would be in the Docker daemon, and we'd only get an error without internet connection if we actually needed the internet to download a container image we didn't have.

  • Even with this usability improvement, we would still need to provide --no-cache-img if we were trying to run [dev] plt stage (and other commands which need to ensure that container images are downloaded before the next boot) without without sudo and we weren't in the docker user group.
  • We might want to add an optional --recache-imgs flag to attempt to re-download all container images and not just those not yet in the Docker daemon. This is only useful for refreshing container images with a :latest tag or a similar tag, which is behavior we want to discourage anyways; and for file downloads the only way to force a re-download is to delete the cached image - so maybe we don't need to prioritize adding a --recache-imgs flag.
  • caching: Enable caching & loading of container images in Forklift's cache #245 may affect how we try to implement this issue. For example, if we store container images as tarballs in a local directory, then we don't need to query the Docker daemon to determine which container images are already in it, and instead we just check which container images are already in our cache. Then we would just need to ensure we have all necessary container images in our cache, and then we can load them into the Docker daemon as part of [dev] plt stage/stage set-next/etc. (though on the next run of stage apply we would want to load any missing images from the Forklift cache into the Docker daemon if necessary).
@ethanjli ethanjli added the enhancement New feature or request label Jun 14, 2024
@ethanjli ethanjli self-assigned this Jun 14, 2024
@ethanjli ethanjli changed the title Don't try to re-download Docker images which already exist ui: Don't try to re-download Docker images which already exist Jun 14, 2024
@ethanjli ethanjli changed the title ui: Don't try to re-download Docker images which already exist caching: Don't try to re-download Docker images which already exist Jun 14, 2024
@ethanjli ethanjli added this to the Backlog milestone Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant