Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OxCGRT data merge, npi model computation docker deployment #523

Merged
merged 63 commits into from
Sep 11, 2020
Merged

OxCGRT data merge, npi model computation docker deployment #523

merged 63 commits into from
Sep 11, 2020

Commits on Aug 14, 2020

  1. 504 - Extending the NPI model data 

    * added a dummy featrues for every intervention which is turned on when a intervention is turned of for the first time and turned of when the original intervention is put in place again.
* extended the data for another month by using th last know countermeasures in each region and data from johns hopkins
    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    cdc665f View commit details
    Browse the repository at this point in the history

  2. fixed workflow yaml

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    f02699c View commit details
    Browse the repository at this point in the history

  3. fixed workflow yml

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    13cd069 View commit details
    Browse the repository at this point in the history

  4. maybe resolving failing dependency installation in github actions by … 

    …updating pip and setuptools
    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    1668cc4 View commit details
    Browse the repository at this point in the history

  5. fixed the extension of data, removed the cancled columns from interve… 

    …ntion json
    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    1985199 View commit details
    Browse the repository at this point in the history

  6. Introduce new intervention icons

    @marekpukaj @JanataPavel
    marekpukaj authored and JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    990b620 View commit details
    Browse the repository at this point in the history

  7. Fix linting

    @marekpukaj @JanataPavel
    marekpukaj authored and JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    ddb4265 View commit details
    Browse the repository at this point in the history

  8. Show NPI model chart only for model channel

    @marekpukaj @JanataPavel
    marekpukaj authored and JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    9424092 View commit details
    Browse the repository at this point in the history

  9. extrapolating data

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    14a4ec8 View commit details
    Browse the repository at this point in the history

  10. fixed data preprocessing - removing deaths from countermeasures

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    2750f49 View commit details
    Browse the repository at this point in the history

  11. short-term model improvements 

    * extrapolation
    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    5dc053b View commit details
    Browse the repository at this point in the history

  12. added extrapolation date

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    5d36d17 View commit details
    Browse the repository at this point in the history

  13. Merge OxCGRT countermeasure data

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    47eb823 View commit details
    Browse the repository at this point in the history

  14. poetry update

    @JanataPavel
    JanataPavel committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    722f9dd View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2020

  1. Merged NPI data from OxCGRT 

    Prepared docker container which can be run on GCP compute instances from github pipelines

defined conda environment to accelerate the computation of the npi model
    @JanataPavel
    JanataPavel committed Aug 25, 2020
    Configuration menu
    Copy the full SHA
    4a0281f View commit details
    Browse the repository at this point in the history

  2. fixed invalid github workflow yml

    @JanataPavel
    JanataPavel committed Aug 25, 2020
    Configuration menu
    Copy the full SHA
    3701fc4 View commit details
    Browse the repository at this point in the history

  3. add upload data step, removed decrypt secrets to workflow

    @JanataPavel
    JanataPavel committed Aug 25, 2020
    Configuration menu
    Copy the full SHA
    45a2907 View commit details
    Browse the repository at this point in the history

  4. fixed upload data step

    @JanataPavel
    JanataPavel committed Aug 25, 2020
    Configuration menu
    Copy the full SHA
    0fd5a67 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2020

  1. moved GCP setup in workflow, added extrapolation perriod to the model

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    73df2c7 View commit details
    Browse the repository at this point in the history

  2. fixed extrapolation period argument

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    531ade5 View commit details
    Browse the repository at this point in the history

  3. removed steps from compute npi workflow 

    * don't run previous steps in workflows, instead download the latest r_estimates.csv inside docker. The other steps are quick
Created a script which deletes the instance when docker exits. This script is copied to the GCP console, but I included it in the repo for consistency
    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    017ca56 View commit details
    Browse the repository at this point in the history

  4. fixed create-with-container command in workflow

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    1684243 View commit details
    Browse the repository at this point in the history

  5. added env file in compute-npi-model to hopefully fix the gcloud command

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    c6f3317 View commit details
    Browse the repository at this point in the history

  6. fixed workflow

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    4b38a58 View commit details
    Browse the repository at this point in the history

  7. fixed workflow

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    ee15976 View commit details
    Browse the repository at this point in the history

  8. set region for compute

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    cd810fc View commit details
    Browse the repository at this point in the history

  9. use different service account

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    08df6eb View commit details
    Browse the repository at this point in the history

  10. update gcloud in workflow

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    9ecf289 View commit details
    Browse the repository at this point in the history

  11. changed order of arguments

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    db405a4 View commit details
    Browse the repository at this point in the history

  12. screw instance templates, it just refuses to work - defining the inst… 

    …ance in the command
    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    c25b45c View commit details
    Browse the repository at this point in the history

  13. set gcp project in workflow

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    4f93c8e View commit details
    Browse the repository at this point in the history

  14. fixed typo

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    8ccafcb View commit details
    Browse the repository at this point in the history

  15. pass foretold channel env variable directly

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    0648d51 View commit details
    Browse the repository at this point in the history

  16. fixed parameter name

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    79e0cdc View commit details
    Browse the repository at this point in the history

  17. appending newline to env file

    @JanataPavel
    JanataPavel committed Aug 26, 2020
    Configuration menu
    Copy the full SHA
    ca0f679 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2020

  1. debugging workflow

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    d11506e View commit details
    Browse the repository at this point in the history

  2. redefine the machine

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    1c34615 View commit details
    Browse the repository at this point in the history

  3. the cpu has to also be specified

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    2bb5fa5 View commit details
    Browse the repository at this point in the history

  4. dropped the machine type added vm type instead

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    ff7079c View commit details
    Browse the repository at this point in the history

  5. added scopes to the vm instance, so that it can pull docker image

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    2f1c622 View commit details
    Browse the repository at this point in the history

  6. fixed syntax

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    818c3b8 View commit details
    Browse the repository at this point in the history

  7. building docker container, debugging startup script

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    77738d7 View commit details
    Browse the repository at this point in the history

  8. use url to pass the startup script to the instance

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    515edda View commit details
    Browse the repository at this point in the history

  9. extracting branch name, refactoring workflow

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    5a966f2 View commit details
    Browse the repository at this point in the history

  10. reformatted workflow yml

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    1cdb66c View commit details
    Browse the repository at this point in the history

  11. fixed workflow step

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    97e3dfd View commit details
    Browse the repository at this point in the history

  12. make sure the startup script won't block model

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    bf4e855 View commit details
    Browse the repository at this point in the history

  13. trying it without the startup script

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    70c5b49 View commit details
    Browse the repository at this point in the history

  14. another try to not block the npi model by startup script

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    084d9e2 View commit details
    Browse the repository at this point in the history

  15. more disk-space (conda image is large), debugging startup script

    @JanataPavel
    JanataPavel committed Aug 27, 2020
    Configuration menu
    Copy the full SHA
    0fa21d5 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2020

  1. fixed preprocessing of countermeasures, debugging startup script

    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    d74668a View commit details
    Browse the repository at this point in the history

  2. removed the startup script, killing the instance from the docker cont… 

    …ainer, reformatted code
    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    6ee9774 View commit details
    Browse the repository at this point in the history

  3. fixed linting after black update

    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    28520c7 View commit details
    Browse the repository at this point in the history

  4. Filter out subregions from OxCGRT data 

    * OxCGRT added data for subregions (e.g. US states) which broke the pipeline. Fix is to filter it out, but we might use them in the future
    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    925eab1 View commit details
    Browse the repository at this point in the history

  5. fixed key passing to the container

    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    e401898 View commit details
    Browse the repository at this point in the history

  6. Strip the quotes from the key - they are necessary when passing them … 

    …in the env file
    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    c39f794 View commit details
    Browse the repository at this point in the history

  7. fixed run model script

    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    2623574 View commit details
    Browse the repository at this point in the history

  8. fixed extrapolation date, changed channel, run on 40 countries

    @JanataPavel
    JanataPavel committed Aug 28, 2020
    Configuration menu
    Copy the full SHA
    ed121e2 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2020

  1. small fixes 

    * Triggering the npi-model computing workflow manually
* More tune interactions of the model (to hopefully shrink the confidence interval)
* Made sure that each NUTS sampling process created by pymc3 only uses one thread - The parallelization doesn't work and this greatly speeds up the computation
    @JanataPavel
    JanataPavel committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    3d88b1e View commit details
    Browse the repository at this point in the history

  2. fixed lining

    @JanataPavel
    JanataPavel committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    fec049b View commit details
    Browse the repository at this point in the history

  3. pre-PR clean-up

    @JanataPavel
    JanataPavel committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    7100824 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/master' into 518-npi-model-data-… 

    …merge
    @JanataPavel
    JanataPavel committed Sep 1, 2020
    Configuration menu
    Copy the full SHA
    89c422f View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2020

  1. lgtm based fixes

    @JanataPavel
    JanataPavel committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    8da8cd8 View commit details
    Browse the repository at this point in the history