Skip to content

Commit

Permalink
Document output directories generated by fromager
Browse files Browse the repository at this point in the history
  • Loading branch information
rd4398 committed Sep 27, 2024
1 parent b815993 commit 97cdc83
Showing 1 changed file with 129 additions and 0 deletions.
129 changes: 129 additions & 0 deletions docs/files.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,132 @@ The following example shows `graph.json` file for the top-level dependency `whee
}
}
```

## Output Directories

During the wheel building process, fromager generates multiple output directories namely `sdists-repo`, `wheels-repo` and `work-dir`. These directories contain important information related to the wheel build.

### sdist-repo

This directory contains the source distributions for the package and its dependencies that we are building. The directory structure of `sdists-repo` is as follows

```
sdists-repo
├── builds
└── downloads
```

The `builds` and `downloads` sub-directories contain the source distributions that are built and downloaded respectively in `tar.gz` format. For example, the `sdists-repo` for `stevedore` package looks as follows:

```
sdists-repo
├── builds
│   ├── pbr-6.1.0.tar.gz
│   ├── setuptools-75.1.0.tar.gz
│   └── stevedore-5.3.0.tar.gz
└── downloads
├── pbr-6.1.0.tar.gz
├── setuptools-75.1.0.tar.gz
└── stevedore-5.3.0.tar.gz
```

We can see source distributions for `pbr` and `setuptools` since these are dependencies of `stevedore`.

### wheels-repo

This directory contains the wheels for a package and its dependencies that are built by fromager, used as prebuilt and the ones that are downloaded from indices. The directory structure of `wheels-repo` is as follows

```
wheels-repo
├── build
├── downloads
├── prebuilt
└── simple
```

* The `build` sub-directoy contains the configuration information related to the wheel build
* The `downloads` sub-directory contains the wheels in `.whl` format that were downloaded from an index
* The `prebuilt` sub-directory contains wheels that are being used as prebuilt
* The `simple` sub-directory consists of sub-directories that include wheels built by fromager in `whl` format along with an `index.html` file which contains the link to download the wheel. The `index.html` file present at the root level of this directory includes links to download wheels for the package and its dependencies

For example, the `wheels-repo` for `stevedore` package looks as follows:

```
wheels-repo
├── build
├── downloads
│   ├── pbr-6.1.0-0-py2.py3-none-any.whl
│   ├── setuptools-75.1.0-0-py3-none-any.whl
│   └── stevedore-5.3.0-0-py3-none-any.whl
├── prebuilt
└── simple
├── index.html
├── pbr
│   ├── index.html
│   └── pbr-6.1.0-0-py2.py3-none-any.whl
├── setuptools
│   ├── index.html
│   └── setuptools-75.1.0-0-py3-none-any.whl
└── stevedore
├── index.html
└── stevedore-5.3.0-0-py3-none-any.whl
```

### work-dir

This directory contains information that is required during the wheel build process. This information includes logs, constraints, requirements, graph for dependency resolution and the order in which the package and its dependencies will be built. The directory structure of `work-dir` is as follows:

```
work-dir
├── build-order.json
├── constraints.txt
├── graph.json
├── logs
├── sample-package-foo
   ├── build-backend-requirements.txt
   ├── build.log
   ├── build-meta.json
   ├── build-sdist-requirements.txt
   ├── build-system-requirements.txt
   └── requirements.txt
```

* The `build-order.json` file is an output file that contains the bottom-up order in which the dependencies need to be built for a specific wheel
* The `constraints.txt` is the input file used to control the resolution of python packages that are needed to build a wheel
* The `graph.json` is an output file that contains all the paths fromager can take to resolve a dependency during building the wheel
* The `logs` sub-directory contains detailed logs for fromager's `build-sequence` command including various settings and overrides for each individual package and its dependencies whose wheel was built by fromager
* The `work-dir` also includes sub-directories for the package and its dependencies. These sub-directories include various types of requirements files including `build-backend-requirements.txt`, `build-sdists-requirements.txt`, `build-system-requirements.txt` and the general `requirements.txt`. Files like `build.log` which store the logs generated by PyPI and `build-meta.json` that stores the metadata for the build are also located in `work-dir`. These sub-directories also include all the other relevant information for a particular package

For example, the `work-dir` for `stevedore` package after `bootstrap` command looks as follows:

```
.
├── build-order.json
├── constraints.txt
├── graph.json
├── logs
├── pbr-6.1.0
│   ├── build-backend-requirements.txt
│   ├── build.log
│   ├── build-meta.json
│   ├── build-sdist-requirements.txt
│   ├── build-system-requirements.txt
│   └── requirements.txt
├── setuptools-75.1.0
│   ├── build-backend-requirements.txt
│   ├── build.log
│   ├── build-meta.json
│   ├── build-sdist-requirements.txt
│   ├── build-system-requirements.txt
│   └── requirements.txt
└── stevedore-5.3.0
├── build-backend-requirements.txt
├── build.log
├── build-meta.json
├── build-sdist-requirements.txt
├── build-system-requirements.txt
└── requirements.txt
```

0 comments on commit 97cdc83

Please sign in to comment.