Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize CPU/RAM requests #33

Open
chrisamiller opened this issue May 4, 2022 · 2 comments
Open

optimize CPU/RAM requests #33

chrisamiller opened this issue May 4, 2022 · 2 comments
Labels
performance optimizations of run time adn/or cost

Comments

@chrisamiller
Copy link
Member

Several related issues here:

  1. if you create a custom N1 VM, it can have up to 6.5 GB of memory per vCPU. So for example when we select 32GB for mutect step, this forces us to request 6 CPUs. A list of places where we've been bumped up to more cores is here:
    /storage1/fs1/mgriffit/Active/griffithlab/pipeline_test/gcp_wdl_test/saved_results/final_results_v1_fusions_ens95/workflow_artifacts/extra_cpu_requests.txt

  2. WGS runs require more resources than exomes in many cases, but our memory/CPU values are set so that the largest data sets will run. Either a) Provide a parameter that allows for specifying WGS or Exome at the top level or b) use the bam size directly to estimate mem usage in some of these steps.

@malachig
Copy link
Member

malachig commented Jul 8, 2022

To find these messages about CPU requirements being adjusted in the cromwell log:

journalctl -u cromwell | grep -i "GCE"

@malachig
Copy link
Member

malachig commented Jul 8, 2022

@malachig malachig added the performance optimizations of run time adn/or cost label Jul 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance optimizations of run time adn/or cost
Projects
None yet
Development

No branches or pull requests

2 participants