chore: secure hermetic-build docker image

[diegomarquezp]

This PR improves the Hermetic Build Docker image in a few different ways:

• Use secure base images
• Use a secure repository for Python dependencies. This was done by pointing to the Airlock Python registries, which requires a couple of special configuration files and a build-time secret.
  • The build-time secret must be configured before building the image locally (instructions added in the development guide). Cloud Build automatically injects the secrets and no special setup is needed.
  • A few dependencies were submitted for approval and they were automatically added to the staging repository. Once they become part of the "trusted" repository, we will use the "safest" trusted repository.
• Use multiple stages where only the relevant binaries are copied into the final image
  • The library_generation python package and its dependencies are built in a separate stage
• Use Alpine-based images
  • Since this kind of distro is not compatible with glibc-based binaries, a special compatibility enablement step was added
  • Since the UNIX tools are from FreeBSD, the -d flag in the rm command is not supported. This is why we removed its usage in the scripts
• Reduction of image size to ~1 GB. Most space is used by:
  • OpenJDK: ~300MB
  • GLIBC compatibility: ~250MB
• Reduction of build time to 150 seconds (parallel stage builds via buildkit).

Pending items

• Discuss a way to pass the Application Default Credentials to the now (now failing - see job output) hermetic_library_generation workflow in sdk-platform-java (other repos don't need to build the image). We can add a secret in the repo or migrate this action to Cloud Build.
• Discuss a strategy for the integration tests now that we need a secret to passed it to the image building workflow. The current approach is to use Cloud Build instead of GitHub Actions since it has out-of-the box compatibility with APPLICATION_DEFAULT_CREDENTIAL secrets such as the one declared in the Dockerfile. In the meantime, I left a build workflow in .cloudbuild to replace the integration tests and removed the integration tests portion of the existing GitHub Action. Another possible approach is to have a repository secret and keep using the GitHub Action.
  • If approach is validated, create the Cloud Build trigger

[JoeWang1127]

In order to create a SEA, do we have to bundle source code into a single js file?

Can we combine the two steps into one?

[diegomarquezp]

Yes, bundling is necessary:

sdk-platform-java/.cloudbuild/library_generation/library_generation.Dockerfile

Lines 39 to 40 in fb98222

	# This is because SEA (see below) cannot
	# resolve external modules in a multi-file project.

From the docs: The single executable application feature currently only supports running a single embedded script using the CommonJS module system.

Can be combine the two steps?

Do you mean using a single RUN + &&?

[JoeWang1127]

I was referring whether we can build a SEA from multiple js source code, looks like it has to be bundled together first.

[JoeWang1127]

Why can't we use a python base image?

[diegomarquezp]

The standalone owlbot seems to need a few runtime libraries (.so) in the linux environment that are available in the node image by default (besides the node runtime). The python image doesn't have them.
If we want to have a python image as base, I can try to use a python base and install node or whichever libraries the executable needs.

[diegomarquezp]

From discussion with @blakeli0 and @JoeWang1127, let's use a vanilla Alpine based image.

[JoeWang1127]

Can we change the image name in this file?

[diegomarquezp]

Is this about the _IMAGE_ID of the image? We can change it, but how can we improve its name?

[JoeWang1127]

Is this about the _IMAGE_ID of the image?

Yes.

Can we create a image repo in AR?

[diegomarquezp]

What if we leave the changes related to the AR migration as a follow up? Should be a small PR.

[sonarcloud]

Quality Gate passed for 'gapic-generator-java-root'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarcloud]

Quality Gate passed for 'java_showcase_integration_tests'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[blakeli0]

We can add a secret in the repo or migrate this action to Cloud Build.

Agree that we should migrate them to Cloud Build, same as the integration tests. This gets us closer to not dependent on Github specific features.

[blakeli0]

How do we plan to update these base images? It might be fine to not update this one, but for the Java and Python one, we may want to update the Maven/JDK/Python version regularly.

[blakeli0]

What does Docker Buildkit caching mean? Since the build does not take too much time(~1.5 minute), I think it's OK to download the maven dependencies every time. What I'm trying to avoid is that we build an image with stale dependencies.

[diegomarquezp]

The Buildkit caching allows to reuse the specified folders in the specific step. This would not impact any CI pipeline as they don't preserve any kind of cache. The main purpose is to speed up local builds for development, and can be disabled via docker build --no-cache.

My perspective is that we probably won't work on modifying the java source code when working on the Docker image, so several image builds may benefit from caching the mvn install output.