Skip to content

Commit

Permalink
ingest workflow: Add trial_name input
Browse files Browse the repository at this point in the history
Allow trial runs to be uploaded to the S3 bucket with the additional
`/trial/<trial_name>` prefix and do not trigger rebuilds when doing a
trial run.

This was easily done because the `s3_dst` has been made a top level
config param in the previous commit.
  • Loading branch information
joverlee521 committed Sep 13, 2023
1 parent 0bf4f07 commit 99b830e
Showing 1 changed file with 37 additions and 0 deletions.
37 changes: 37 additions & 0 deletions .github/workflows/fetch-and-ingest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,40 @@ on:

# Manually triggered using GitHub's UI
workflow_dispatch:
inputs:
trial_name:
description: 'Short name for a trial run. WARNING: without this we will overwrite files in s3://nextstrain-data/files/workflows/rsv'

jobs:
set_config_overrides:
runs-on: ubuntu-latest
steps:
- id: s3_dst
run: |
S3_DST=s3://nextstrain-data/files/workflows/rsv
if [[ -n "$TRIAL_NAME" ]]; then
S3_DST+=/trial/"$TRIAL_NAME"
fi
echo "s3_dst=$S3_DST" >> "$GITHUB_OUTPUT"
env:
TRIAL_NAME: ${{ inputs.trial_name }}
- id: trigger_rebuild
run: |
TRIGGER_REBUILD=true
if [[ -n "$TRIAL_NAME" ]]; then
TRIGGER_REBUILD=false
fi
echo "trigger_rebuild=$TRIGGER_REBUILD" >> "$GITHUB_OUTPUT"
outputs:
s3_dst: ${{ steps.s3_dst.outputs.s3_dst }}
trigger_rebuild: ${{ steps.trigger_rebuild.outputs.trigger_rebuild }}

fetch-and-ingest:
needs: [set_config_overrides]
permissions:
id-token: write
uses: nextstrain/.github/.github/workflows/pathogen-repo-build.yaml@master
Expand All @@ -45,6 +76,12 @@ jobs:
--env AWS_ACCESS_KEY_ID \
--env AWS_SECRET_ACCESS_KEY \
--env PAT_GITHUB_DISPATCH="$GH_TOKEN_NEXTSTRAIN_BOT_WORKFLOW_DISPATCH" \
--env S3_DST \
--env TRIGGER_REBUILD \
ingest \
--configfiles config/config.yaml config/optional.yaml \
--config s3_dst="$S3_DST" trigger_rebuild="$TRIGGER_REBUILD" \
--printshellcmds
env: |
S3_DST: ${{ needs.set_config_overrides.outputs.s3_dst }}
TRIGGER_REBUILD: ${{ needs.set_config_overrides.outputs.trigger_rebuild }}

0 comments on commit 99b830e

Please sign in to comment.