Skip to content

Commit

Permalink
docs: add buck2 docs
Browse files Browse the repository at this point in the history
Signed-off-by: Austin Seipp <[email protected]>
  • Loading branch information
thoughtpolice committed Jun 25, 2024
1 parent 40dd167 commit 7c8f3e1
Showing 1 changed file with 207 additions and 0 deletions.
207 changes: 207 additions & 0 deletions docs/buck2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# Buck2 builds

> [!NOTE]
> This document is primarily of interest to developers. See also [Contributing]
> for more information on how to contribute in general.
There is experimental support for building `jj` with [Buck2], a hermetic and
reproducible build system, an alternative to Cargo.

> [!IMPORTANT]
> Buck2 support is a work in progress, and is not yet complete. It is not
> recommended for primary development use.
## Step 0: Installing Dotslash

We use [Dotslash] to manage Buck2 build tooling in a way that's consistent
across all developers and amenable to version control. This helps ensure every
developer gets consistent results (which is a big part of the selling point of
hermetic build tools.)

You can install Dotslash binaries by following the instructions at:

- <https://dotslash-cli.com/docs/installation/>

Or, if you have Rust installed, you can install Dotslash by running:

```sh
cargo install dotslash
```

Or, if you have Nix, you can install Dotslash with Nix by running:

```sh
nix profile install 'nixpkgs#dotslash'
```

## Building `jj` with Buck2

Assuming `dotslash` exists somewhere in your `$PATH`, you can now build `jj`
with the included `buck2` dotslash file that exists under `./tools/bin`:

```sh
./tools/bin/buck2 run //:jj -- version
```

Dotslash will transparently run the correct version of `buck2`, and `buck2` will
build everything needed to run `jj`.

## Buck2 crash course

The following is an extremely minimal crash course in Buck2 concepts and how to
use it.

### Targets

Buck2 is used to build **targets**, that exists in **packages**, which are part
of a **cell**. The most explicit syntax for referring to a target is the
following:

```text
cell//path/to/package:target-name
```

You normally use a so-called "fully qualified" name like the above as an
argument to `buck2 build`.

A cell is a short name that maps to a directory in the code repository. A
package is a subdirectory underneath the cell that contains the build rules for
the targets. A target is a buildable unit of code, like a binary or a library,
named in the `BUILD` file inside that package.

A fully-qualified reference to a target works anywhere in the source code tree,
so you can build or test any component no matter what directory you're in.

So, given a cell named `foobar//` located underneath `code/foobar`, and a
package `bar/baz` in that cell, leads to a file

```text
code/foobar/bar/baz/BUILD
```

Which contains the targets that can be built.

There are several shorthands for a target:

- NIH.

### Graphs: Target & Action

NIH.

### Package files

NIH.

### Mode files

In order to support concepts like debug and release builds, we use the concept
of "mode files" in Buck2. These are files that contain a list of command line
options to apply to a build to achieve the desired effect.

For example, to build in debug mode, you can simply include the contents of the
file `mode//debug` (using cell syntax) onto the command line. This can
conveniently be done with "at-file" syntax when invoking `buck2`:

```sh
buck2 build //:jj @mode//debug
buck2 build //:jj @mode//release
```

Where `@path/to/file` is the at-file syntax for including the contents of a file
on the command line. This syntax supports `cell//` references to Buck cells, as
well.

In short, `buck2 build @mode//file` will apply the contents of `file` to your
invocation. We keep a convenient set of these files maintained under the
`mode//` cell, located under [`./buck/mode`](../buck/mode).

#### At-file syntax

The `buck2` CLI supports a convenient modern feature called "at-file" syntax,
where the invocation `buck2 @path/to/file` is effectively equivalent to the
bash-ism `buck2 $(cat path/to/file)`, where each line of the file is a single
command line entry, in a consistent and portable way that doesn't have any limit
to the size of the underlying file.

For example, assuming the file `foo/bar` contained the contents

```text
--foo=1
--bar=false
```

Then `buck2 --test @foo/bar` and `buck2 --test --foo=1 --bar=false` are
equivalent.

### Buck Extension Language (BXL)

NIH.

## Examples

Some examples are included below.

### Run BoringSSL speed tests

```sh
buck2 run third-party//bssl:bssl @mode//release -- speed
```

### Build all third-party Rust dependencies

```sh
buck2 build third-party//rust
```

### Build all `http_archive` dependencies

```sh
buck2 build $(buck2 uquery "kind('http_archive', deps('//...'))" | grep third-party//)
```

## Development notes

Notes for `jj` developers using Buck2.

### Build mode reference

- `@mode//debug`
- `@mode//release`

### Cargo dependency management

Although Buck2 downloads and runs `rustc` on its own to build crate
dependencies, our `Cargo.toml` build files act as the source of truth for
dependency information in both Cargo and Buck2.

Updating the dependency graph for Cargo-based projects typically comes in one of
two forms:

- Updating a dependency version in the top-level workspace `Cargo.toml` file
- Adding a newly required dependency to `[dependencies]` in the `Cargo.toml`
file for a crate

After doing either of these actions, you can synchronize the Buck2 dependencies
with the Cargo dependencies with the following script:

```bash
./buck/third-party/rust/synchronize.py
```

This must be run from the root of the repository.

This will re-synchronize all `third-party//rust` crates with the versions in the
workspace Cargo file, and then also update the `BUILD` files in the source code
with any newly added build dependencies that were added or removed (not just
updated).

### `rust-analyzer` support

Coming soon.

<!-- References -->

[Contributing]: https://martinvonz.github.io/jj/latest/contributing/
[Buck2]: https://buck2.build/
[Dotslash]: https://dotslash-cli.com/

0 comments on commit 7c8f3e1

Please sign in to comment.