Skip to content

Commit

Permalink
Use .bazelrc from Aspect and support --nolegacy_external_runfiles
Browse files Browse the repository at this point in the history
Adopt https://docs.aspect.build/guides/bazelrc/ which has tons of useful
options. There is just one option that disables `external/` repositories
in runfiles and highlights a flaw in the ruleset that is also addressed
properly in this commit.
  • Loading branch information
p0deje committed Jul 23, 2023
1 parent 3b0786b commit 7e39275
Show file tree
Hide file tree
Showing 4 changed files with 195 additions and 10 deletions.
193 changes: 190 additions & 3 deletions examples/gem/.bazelrc
Original file line number Diff line number Diff line change
@@ -1,14 +1,201 @@
#########################################################
# COPIED FROM https://docs.aspect.build/guides/bazelrc/ #
#########################################################

# Do not upload locally executed action results to the remote cache.
# This should be the default for local builds so local builds cannot poison the remote cache.
# It should be flipped to `--remote_upload_local_results` on CI
# by using `--bazelrc=.aspect/bazelrc/ci.bazelrc`.
# Docs: https://bazel.build/reference/command-line-reference#flag--remote_upload_local_results
build --noremote_upload_local_results

# Don't allow network access for build actions in the sandbox.
# Ensures that you don't accidentally make non-hermetic actions/tests which depend on remote
# services.
# Developers should tag targets with `tags=["requires-network"]` to opt-out of the enforcement.
# Docs: https://bazel.build/reference/command-line-reference#flag--sandbox_default_allow_network
build --sandbox_default_allow_network=false

# Warn if a test's timeout is significantly longer than the test's actual execution time.
# Bazel's default for test_timeout is medium (5 min), but most tests should instead be short (1 min).
# While a test's timeout should be set such that it is not flaky, a test that has a highly
# over-generous timeout can hide real problems that crop up unexpectedly.
# For instance, a test that normally executes in a minute or two should not have a timeout of
# ETERNAL or LONG as these are much, much too generous.
# Docs: https://bazel.build/docs/user-manual#test-verbose-timeout-warnings
test --test_verbose_timeout_warnings

# Allow the Bazel server to check directory sources for changes. Ensures that the Bazel server
# notices when a directory changes, if you have a directory listed in the srcs of some target.
# Recommended when using
# [copy_directory](https://github.com/aspect-build/bazel-lib/blob/main/docs/copy_directory.md) and
# [rules_js](https://github.com/aspect-build/rules_js) since npm package are source directories
# inputs to copy_directory actions.
# Docs: https://bazel.build/reference/command-line-reference#flag--host_jvm_args
startup --host_jvm_args=-DBAZEL_TRACK_SOURCE_DIRECTORIES=1

# Allow exclusive tests to run in the sandbox. Fixes a bug where Bazel doesn't enable sandboxing for
# tests with `tags=["exclusive"]`.
# Docs: https://bazel.build/reference/command-line-reference#flag--incompatible_exclusive_test_sandboxed
test --incompatible_exclusive_test_sandboxed

# Use a static value for `PATH` and does not inherit `LD_LIBRARY_PATH`. Doesn't let environment
# variables like `PATH` sneak into the build, which can cause massive cache misses when they change.
# Use `--action_env=ENV_VARIABLE` if you want to inherit specific environment variables from the
# client, but note that doing so can prevent cross-user caching if a shared cache is used.
# Docs: https://bazel.build/reference/command-line-reference#flag--incompatible_strict_action_env
build --incompatible_strict_action_env

# Propagate tags from a target declaration to the actions' execution requirements.
# Ensures that tags applied in your BUILD file, like `tags=["no-remote"]`
# get propagated to actions created by the rule.
# Without this option, you rely on rules authors to manually check the tags you passed
# and apply relevant ones to the actions they create.
# See https://github.com/bazelbuild/bazel/issues/8830 for details.
# Docs: https://bazel.build/reference/command-line-reference#flag--experimental_allow_tags_propagation
build --experimental_allow_tags_propagation
fetch --experimental_allow_tags_propagation
query --experimental_allow_tags_propagation

# Do not automatically create `__init__.py` files in the runfiles of Python targets. Fixes the wrong
# default that comes from Google's internal monorepo by using `__init__.py` to delimit a Python
# package. Precisely, when a `py_binary` or `py_test` target has `legacy_create_init` set to `auto (the
# default), it is treated as false if and only if this flag is set. See
# https://github.com/bazelbuild/bazel/issues/10076.
# Docs: https://bazel.build/reference/command-line-reference#flag--incompatible_default_to_explicit_init_py
build --incompatible_default_to_explicit_init_py

# Attempt to build & test every target whose prerequisites were successfully built.
# Docs: https://bazel.build/docs/user-manual#keep-going
build --keep_going

# Output test errors to stderr so users don't have to `cat` or open test failure log files when test
# fail. This makes the log noiser in exchange for reducing the time-to-feedback on test failures for
# users.
# Docs: https://bazel.build/docs/user-manual#test-output
test --test_output=errors

# Show the output files created by builds that requested more than one target. This helps users
# locate the build outputs in more cases
# Docs: https://bazel.build/docs/user-manual#show-result
build --show_result=20

# Bazel picks up host-OS-specific config lines from bazelrc files. For example, if the host OS is
# Linux and you run bazel build, Bazel picks up lines starting with build:linux. Supported OS
# identifiers are `linux`, `macos`, `windows`, `freebsd`, and `openbsd`. Enabling this flag is
# equivalent to using `--config=linux` on Linux, `--config=windows` on Windows, etc.
# Docs: https://bazel.build/reference/command-line-reference#flag--enable_platform_specific_config
common --enable_platform_specific_config

# Output a heap dump if an OOM is thrown during a Bazel invocation
# (including OOMs due to `--experimental_oom_more_eagerly_threshold`).
# The dump will be written to `<output_base>/<invocation_id>.heapdump.hprof`.
# You may need to configure CI to capture this artifact and upload for later use.
# Docs: https://bazel.build/reference/command-line-reference#flag--heap_dump_on_oom
common --heap_dump_on_oom

# Speed up all builds by not checking if output files have been modified. Lets you make changes to
# the output tree without triggering a build for local debugging. For example, you can modify
# [rules_js](https://github.com/aspect-build/rules_js) 3rd party npm packages in the output tree
# when local debugging.
# Docs: https://github.com/bazelbuild/bazel/blob/1af61b21df99edc2fc66939cdf14449c2661f873/src/main/java/com/google/devtools/build/lib/pkgcache/PackageOptions.java#L185
build --noexperimental_check_output_files
fetch --noexperimental_check_output_files
query --noexperimental_check_output_files

# Don't apply `--noremote_upload_local_results` and `--noremote_accept_cached` to the disk cache.
# If you have both `--noremote_upload_local_results` and `--disk_cache`, then this fixes a bug where
# Bazel doesn't write to the local disk cache as it treats as a remote cache.
# Docs: https://bazel.build/reference/command-line-reference#flag--incompatible_remote_results_ignore_disk
build --incompatible_remote_results_ignore_disk

# Directories used by sandboxed non-worker execution may be reused to avoid unnecessary setup costs.
# Save time on Sandbox creation and deletion when many of the same kind of action run during the
# build.
# No longer experimental in Bazel 6: https://github.com/bazelbuild/bazel/commit/c1a95501a5611878e5cc43a3cc531f2b9e47835b
# Docs: https://bazel.build/reference/command-line-reference#flag--reuse_sandbox_directories
build --experimental_reuse_sandbox_directories

# Do not build runfiles symlink forests for external repositories under
# `.runfiles/wsname/external/repo` (in addition to `.runfiles/repo`). This reduces runfiles &
# sandbox creation times & prevents accidentally depending on this feature which may flip to off by
# default in the future. Note, some rules may fail under this flag, please file issues with the rule
# author.
# Docs: https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles
build --nolegacy_external_runfiles

# Some actions are always IO-intensive but require little compute. It's wasteful to put the output
# in the remote cache, it just saturates the network and fills the cache storage causing earlier
# evictions. It's also not worth sending them for remote execution.
# For actions like PackageTar it's usually faster to just re-run the work locally every time.
# You'll have to look at an execution log to figure out what other action mnemonics you care about.
# In some cases you may need to patch rulesets to add a mnemonic to actions that don't have one.
# https://bazel.build/reference/command-line-reference#flag--modify_execution_info
build --modify_execution_info=PackageTar=+no-remote

# Stream stdout/stderr output from each test in real-time.
# Docs: https://bazel.build/docs/user-manual#test-output
test:debug --test_output=streamed

# Run one test at a time.
# Docs: https://bazel.build/reference/command-line-reference#flag--test_strategy
test:debug --test_strategy=exclusive

# Prevent long running tests from timing out.
# Docs: https://bazel.build/docs/user-manual#test-timeout
test:debug --test_timeout=9999

# Always run tests even if they have cached results.
# Docs: https://bazel.build/docs/user-manual#cache-test-results
test:debug --nocache_test_results

# Speed up all builds by not checking if external repository files have been modified.
# Docs: https://github.com/bazelbuild/bazel/blob/1af61b21df99edc2fc66939cdf14449c2661f873/src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryOptions.java#L244
build --noexperimental_check_external_repository_files
fetch --noexperimental_check_external_repository_files
query --noexperimental_check_external_repository_files

# Directories used by sandboxed non-worker execution may be reused to avoid unnecessary setup costs.
# Save time on Sandbox creation and deletion when many of the same kind of action run during the
# build.
# Docs: https://bazel.build/reference/command-line-reference#flag--reuse_sandbox_directories
build --reuse_sandbox_directories

# Avoid this flag being enabled by remote_download_minimal or remote_download_toplevel
# See https://meroton.com/blog/bazel-6-errors-build-without-the-bytes/
build --noexperimental_action_cache_store_output_metadata

########################
# RUBY RULESET OPTIONS #
########################

# Enable runfiles on all platforms. Runfiles are on by default on Linux and MacOS but off on
# Windows.
#
# If you are developing on Windows, you must either run bazel with administrator privileges or
# enable developer mode. If you do not you may hit this error on Windows:
#
# Bazel needs to create symlinks to build the runfiles tree.
# Creating symlinks on Windows requires one of the following:
# 1. Bazel is run with administrator privileges.
# 2. The system version is Windows 10 Creators Update (1703) or later
# and developer mode is enabled.
#
# Docs: https://bazel.build/reference/command-line-reference#flag--windows_enable_symlinks
# Docs: https://bazel.build/reference/command-line-reference#flag--enable_runfiles
startup --windows_enable_symlinks
build --enable_runfiles

# Allow to run Bazel without pre-installed JDK.
# Docs: https://bazel.build/reference/command-line-reference#flag--java_runtime_version
build --java_runtime_version=remotejdk_11
test --test_output=errors

# JRuby needs $HOME for File#expand_path to work: https://github.com/jruby/jruby/issues/5661.
# TruffleRuby needs $HOME too: https://github.com/oracle/truffleruby/issues/2784.
build --action_env=HOME
test --test_env=HOME

# JRuby 9.3 fails with "Errno::EACCES: Permission denied - NUL" on Windows:
# JRuby might fail with "Errno::EACCES: Permission denied - NUL" on Windows:
# https://github.com/jruby/jruby/issues/7182#issuecomment-1112953015.
build --action_env=JAVA_OPTS="-Djdk.io.File.enableADS=true"
test --test_env=JAVA_OPTS="-Djdk.io.File.enableADS=true"
Expand All @@ -17,4 +204,4 @@ test --test_env=JAVA_OPTS="-Djdk.io.File.enableADS=true"
# 1. Add breakpoint with `binding.break`.
# 2. Run tests: `bazel test --config debug spec:add`.
# 3. Connect to debugger: `bazel-gem/external/bundle/bin/rdbg -A`.
test:debug --cache_test_results=no --test_output=streamed --run_under="@bundle//:bin/rdbg --nonstop --open --command"
test:debug --run_under="@bundle//:bin/rdbg --nonstop --open --command"
2 changes: 0 additions & 2 deletions examples/gem/.rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,3 @@ AllCops:
NewCops: enable
SuggestExtensions: false
TargetRubyVersion: 2.6
Exclude:
- external/**/*
2 changes: 1 addition & 1 deletion examples/gem/ruby_version.bzl
Original file line number Diff line number Diff line change
@@ -1 +1 @@
RUBY_VERSION = "jruby-9.4.0.0"
RUBY_VERSION = "3.2.1"
8 changes: 4 additions & 4 deletions ruby/private/binary.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def generate_rb_binary_script(ctx, binary, bundler = False, args = []):
toolchain_bindir = toolchain.bindir

if binary:
binary_path = binary.path
binary_path = binary.short_path
else:
binary_path = ""

Expand Down Expand Up @@ -93,17 +93,17 @@ def rb_binary_impl(ctx):
tools.append(ruby_toolchain.ruby)

if ruby_toolchain.version.startswith("jruby"):
env["JAVA_HOME"] = java_toolchain.java_runtime.java_home
env["JAVA_HOME"] = java_toolchain.java_runtime.java_home_runfiles_path
tools.extend(java_toolchain.java_runtime.files.to_list())

for file in transitive_srcs:
if file.basename == "Gemfile":
env["BUNDLE_GEMFILE"] = file.path
env["BUNDLE_GEMFILE"] = file.short_path

for dep in ctx.attr.deps:
if dep.label.workspace_name == "bundle":
bundler = True
env["BUNDLE_PATH"] = dep.label.workspace_root
env["BUNDLE_PATH"] = "../" + dep.label.workspace_name

runfiles = ctx.runfiles(transitive_data + transitive_srcs + tools)
env.update(ctx.attr.env)
Expand Down

0 comments on commit 7e39275

Please sign in to comment.