Doc: jobs option and package build setting #496
Conversation
docs/how-to.md
Outdated
| @@ -0,0 +1,15 @@ | |||
| # How To | |||
|
|
|||
| ## Use --jobs option, new cpu_cores_per_job and memory_per_job_gb settings | |||
There was a problem hiding this comment.
It's usually better to title a How To with a description of a goal a user might have, rather than the solution to the problem. In this case, if they didn't already know about any of these options, what might they be thinking about and looking for in the documentation?
|
|
||
| The `memory_per_job_gb` is a package setting that allows to scale parallel jobs by available virtual memory without swap. The default value is set to 1.0 which indicates that each parallel job requires 1 GB virtual memory | ||
| Example: `memory_per_job_gb = 0.5` indicates that each parallel job requires 512 MB virtual memory | ||
| This setting should always have value greater than or equal to 0.1 |
There was a problem hiding this comment.
When should someone use --jobs and when should they use these options?
There was a problem hiding this comment.
I have tried to add information about this but am not sure about it
docs/how-to.md
Outdated
| This setting should always have value greater than or equal to 0.1 | ||
|
|
||
| The above mentioned options are related to build systems and can be used as a per package setting when multiple cores and significant amount of memory is available on a build system. On the other hand, the `--jobs` option can be | ||
| used to spin up paralell jobs to improve speed of the wheel build process. |
There was a problem hiding this comment.
I think --jobs overrides the default calculations based on the other settings, doesn't it? So if someone passes --jobs 4 then at most 4 processes will run in parallel when building a given wheel, but if they don't use that option fromager computes a number of jobs and that value can be influenced based on the per-package settings.
And the jobs are all within the context of building a single wheel. We are not currently building multiple wheels in parallel, but some of the expensive builds do run the C++ compiler in parallel.
There was a problem hiding this comment.
Oh right. I have made changes and pushed latest content
docs/how-to.md
Outdated
|
|
||
| The above mentioned options are related to build systems and can be used as a per package setting when multiple cores and significant amount of memory is available on a build system. | ||
| On the other hand, the `--jobs` overrides the default calculations based on the other settings. For example, when someone passes `--jobs 4` then at most 4 processes will run in parallel when building a given wheel, but if they don't use this option, fromager computes a number of jobs and that value can be influenced based on the per-package settings. | ||
| Note that the jobs are all within the context of building a single wheel. |
There was a problem hiding this comment.
OK, this information is right, but I suggest moving it up to the top of the section (before what is now line 5) as an introduction.
docs/how-to.md
Outdated
| This setting should always have value greater than or equal to 0.1 | ||
|
|
||
| The above mentioned options are related to build systems and can be used as a per package setting when multiple cores and significant amount of memory is available on a build system. | ||
| On the other hand, the `--jobs` overrides the default calculations based on the other settings. For example, when someone passes `--jobs 4` then at most 4 processes will run in parallel when building a given wheel, but if they don't use this option, fromager computes a number of jobs and that value can be influenced based on the per-package settings. |
There was a problem hiding this comment.
| On the other hand, the `--jobs` overrides the default calculations based on the other settings. For example, when someone passes `--jobs 4` then at most 4 processes will run in parallel when building a given wheel, but if they don't use this option, fromager computes a number of jobs and that value can be influenced based on the per-package settings. | |
| On the other hand, the `--jobs` overrides the default calculations based on the other settings. For example, when you pass `--jobs 4` then at most 4 processes will run in parallel when building a given wheel. By default, fromager computes a number of jobs and that value can be influenced based on the per-package settings. |
Fixes: #398
This commit attempts to add documentation for the
--jobsoption and package build settings likenew cpu_cores_per_jobandmemory_per_job_gbThis commit also creates a separate section titled "How To" in the official fromager docs where the above content can be found.