Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.0.0 Release #18

Merged
merged 85 commits into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
94ca694
Updating nf-core .github/workflows and .pre-commit-config.yaml files
kylacochrane Mar 12, 2024
fad838e
Fixing errors associated with nf-core lint
kylacochrane Mar 13, 2024
0b29aeb
Adjusted test.config max_memory
kylacochrane Mar 13, 2024
bfdd0c8
Updated branch.yml to reference phac-nml
kylacochrane Mar 13, 2024
005ec05
Merge pull request #4 from phac-nml/nf-core-linting
kylacochrane Mar 13, 2024
c580e2f
Committing local changes before rebasing origin/dev
kylacochrane Mar 14, 2024
b9c476b
Fixed file formatting and linting
kylacochrane Mar 14, 2024
be04c68
Adding the KRACKEN mini_gut database
kylacochrane Mar 14, 2024
fbb000d
Changed db PATH to kraken branch to access minigut_kraken db
kylacochrane Mar 14, 2024
172cee3
Updates for functionality of pipeline
kylacochrane Mar 14, 2024
69a692d
fixed EC whitespace in kraken2/main.nf
kylacochrane Mar 14, 2024
f8418f5
Added when statement to kraken module
kylacochrane Mar 14, 2024
7828098
Set workflow variable meta.single_end and updated kraken module
kylacochrane Mar 14, 2024
b639792
Separated kraken module inputs so workflow can have separate channels
kylacochrane Mar 14, 2024
5bb1238
Separated output entries
kylacochrane Mar 14, 2024
d39bca1
Updated workflow kraken_db channel
kylacochrane Mar 14, 2024
a2b375f
Updated nextflow_schema.json
kylacochrane Mar 14, 2024
3a76046
Updated kraken module to compress output txt and tsv files
kylacochrane Mar 14, 2024
b299e57
Formatting changes
kylacochrane Mar 14, 2024
6d29fc9
Merge pull request #5 from phac-nml/kraken
kylacochrane Mar 15, 2024
59f6bb5
Removed IridaNextExample modules/tests/data and inclusion in workflow
kylacochrane Mar 15, 2024
1f6d2d9
Merge pull request #6 from phac-nml/remove_irida_modules
kylacochrane Mar 15, 2024
5c32ea0
Adding BRACKEN module and functionality (#7)
kylacochrane Mar 21, 2024
3261690
Adding the ADJUST_BRACKEN module and python script
kylacochrane Mar 21, 2024
a9028e5
Update README.md
kylacochrane Mar 21, 2024
081a614
Update README.md
kylacochrane Mar 21, 2024
46567b0
Update README.md
kylacochrane Mar 21, 2024
cab26de
Updated python script comments
kylacochrane Mar 21, 2024
b00a361
Merge pull request #8 from phac-nml/adjustbracken
kylacochrane Mar 21, 2024
ea672bc
Added KRONA functionality
kylacochrane Mar 21, 2024
dfdf7ef
Merge pull request #9 from phac-nml/krona
kylacochrane Mar 21, 2024
91307a2
Added IRIDANextPlugIn
kylacochrane Mar 21, 2024
b907bc4
Merge pull request #10 from phac-nml/iridanext_plugin
kylacochrane Mar 21, 2024
af55dc0
Added CI tests for main workflow; Updated corresponding config files
kylacochrane Mar 25, 2024
0f829be
Added CI tests to GitHub Actions: ci.yml
kylacochrane Mar 25, 2024
4a6de1a
Separated test database to expand testing options
kylacochrane Mar 25, 2024
4960dae
Grammar and regrex edits
kylacochrane Mar 26, 2024
d508102
Update README.md
kylacochrane Mar 26, 2024
a98f0c3
Update README.md
kylacochrane Mar 26, 2024
c896146
Update test to assess the nextflow_pipeline; make related changes to …
kylacochrane Mar 26, 2024
52e193f
Updated main.nf.test to improve IRIDA Next JSON data structure
kylacochrane Mar 26, 2024
319df69
Updated nextflow.config to uncompress IRIDANext JSON
kylacochrane Mar 27, 2024
1510dc1
Fixed EC errors
kylacochrane Mar 27, 2024
ad7bd8d
Updated workflow and tests for Azure functionality
kylacochrane Apr 2, 2024
dd6bd99
Update README.md
kylacochrane Apr 3, 2024
63bc6a0
Merge pull request #11 from phac-nml/CITests
kylacochrane Apr 3, 2024
67c55ff
Add top5 module for species identification and workflow functionality
kylacochrane Apr 3, 2024
1b5be4c
Added module and MERGE_CSV process for IRIDANext JSON formatting
kylacochrane Apr 8, 2024
63ab209
Updated the iridanext.config file to include TOP_5 results as metadat…
kylacochrane Apr 8, 2024
3549f0b
Added assert statements to main.nf.test to ensure test data metadata …
kylacochrane Apr 8, 2024
11612e1
EC error fix
kylacochrane Apr 8, 2024
2545fad
Changed the TOP_5 process to TOP_N, where N is a passed paramter that…
kylacochrane Apr 9, 2024
e422376
Changed parameter top_results to top_n
kylacochrane Apr 9, 2024
dd084f0
Updating bracken_top_n_linelist.py to encode all missing values to: None
kylacochrane Apr 10, 2024
c739693
Added assert statements to expand tests
kylacochrane Apr 10, 2024
dc25c28
Added nf-core CSVTK/CONCAT module and updated workflow to accomodate
kylacochrane Apr 10, 2024
b1fa9d2
Removed old MERGE_CSV local/module
kylacochrane Apr 10, 2024
1e69ed6
Updated iridanext.config to accomodate module changes for metadata
kylacochrane Apr 10, 2024
9266173
Fixed error in workflow/speciesabundance.nf
kylacochrane Apr 10, 2024
a5f1ebc
Updated nextflow_schema.json, modules.json, and modules.config files
kylacochrane Apr 10, 2024
5131bb7
Fixed error with nextflow_schema.json file
kylacochrane Apr 10, 2024
cf88aae
Updated function in bracken_top_n_linelist.py
kylacochrane Apr 10, 2024
d6636da
Added comments to bracken_top_n_linelist.py
kylacochrane Apr 10, 2024
48ed2a1
Merge pull request #12 from phac-nml/top5
kylacochrane Apr 11, 2024
fad7945
Updated module output nomenclature
kylacochrane Apr 11, 2024
e27dbd5
Update iridanext.config and main.nf.test
kylacochrane Apr 11, 2024
d009b7f
Updated module naming for consistency (kraken2krona)
kylacochrane Apr 11, 2024
6e75e48
Added kmer_len as a parameter to set the kmer distribution files used…
kylacochrane Apr 12, 2024
2c5431c
Added additional workflow parameter to allow Kraken/Bracken database …
kylacochrane Apr 12, 2024
eaaca57
Formatting changes
kylacochrane Apr 12, 2024
3c41163
Updated README.md
kylacochrane Apr 15, 2024
8e1d364
Changed parameter name from 'combined_db' to 'database' for clarity
kylacochrane Apr 15, 2024
66419b2
Updated output and usage documentation
kylacochrane Apr 15, 2024
ec9de94
Updated main.nf.test database parameter
kylacochrane Apr 15, 2024
3a8113e
Update pyhton script in ADJUST_BRACKEN to output the Adjusted-Kraken-…
kylacochrane Apr 16, 2024
b9ce5e9
Updated workflow [bracken2krona] to use Adjusted-Kraken-Style-Report …
kylacochrane Apr 16, 2024
9439b86
Updated documentation
kylacochrane Apr 16, 2024
394e8bd
Fixed main.nf.test
kylacochrane Apr 16, 2024
0c1a100
Added additional output tests
kylacochrane Apr 17, 2024
5071d42
Update the adjust bracken abundances step to report proportions inste…
kylacochrane Apr 17, 2024
be3cfd8
Merge pull request #13 from phac-nml/add_docs
kylacochrane Apr 17, 2024
8a99747
Updated versioning in CHANGELOG.md and nextflow.config
kylacochrane Apr 17, 2024
e3dbc5f
Added link to Galaxy-based pipeline for reference
kylacochrane Apr 17, 2024
5ec0d15
Merge pull request #17 from phac-nml/version
kylacochrane Apr 17, 2024
f162cfe
Updated kraken2 process to high
kylacochrane Apr 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `main` branch :x:
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
- "latest-everything"
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
Expand All @@ -50,6 +50,9 @@ jobs:
wget -qO- https://code.askimed.com/install/nf-test | bash
sudo mv nf-test /usr/local/bin/

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Run nf-test
run: |
nf-test test
Expand Down
65 changes: 13 additions & 52 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,72 +11,33 @@ on:
types: [published]

jobs:
EditorConfig:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

- uses: actions/setup-node@v3

- name: Install editorconfig-checker
run: npm install -g editorconfig-checker

- name: Run ECLint check
run: editorconfig-checker -exclude README.md $(find .* -type f | grep -v '.git\|.py\|.md\|json\|yml\|yaml\|html\|css\|work\|.nextflow\|build\|nf_core.egg-info\|log.txt\|Makefile')

Prettier:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions/setup-node@v3

- name: Install Prettier
run: npm install -g prettier

- name: Run Prettier --check
run: prettier --check ${GITHUB_WORKSPACE}

PythonBlack:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Check code lints with Black
uses: psf/black@stable

# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
- name: Set up Python 3.11
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
message: |
## Python linting (`black`) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

* Install [`black`](https://black.readthedocs.io/en/stable/): `pip install black`
* Fix formatting errors in your pipeline: `black .`

Once you push these changes the test should pass, and you can hide this comment :+1:
python-version: 3.11
cache: "pip"

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!
- name: Install pre-commit
run: pip install pre-commit

Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
- name: Run pre-commit
run: pre-commit run --all-files

nf-core:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1

- uses: actions/setup-python@v4
- uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
python-version: "3.11"
architecture: "x64"
Expand All @@ -99,7 +60,7 @@ jobs:

- name: Upload linting log file artifact
if: ${{ always() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4
with:
name: linting-logs
path: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/linting_comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download lint results
uses: dawidd6/action-download-artifact@v2
uses: dawidd6/action-download-artifact@f6b0bace624032e30a85a8fd9c1a7f8f611f5737 # v3
with:
workflow: linting.yml
workflow_conclusion: completed
Expand All @@ -21,7 +21,7 @@ jobs:
run: echo "pr_number=$(cat linting-logs/PR_number.txt)" >> $GITHUB_OUTPUT

- name: Post PR comment
uses: marocchino/sticky-pull-request-comment@v2
uses: marocchino/sticky-pull-request-comment@331f8f5b4215f0445d3c07b4967662a32a2d3e31 # v2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
number: ${{ steps.pr_number.outputs.pr_number }}
Expand Down
4 changes: 4 additions & 0 deletions .nf-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ lint:
- docs/images/nf-core-speciesabundance_logo_dark.png
- .github/workflows/awstest.yml
- .github/workflows/awsfulltest.yml
- lib/Utils.groovy
- lib/WorkflowMain.groovy
- lib/NfcoreTemplate.groovy
- lib/WorkflowSpeciesabundance.groovy
files_unchanged:
- assets/sendmail_template.txt
- assets/email_template.html
Expand Down
7 changes: 6 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
repos:
- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v2.7.1"
rev: "v3.1.0"
hooks:
- id: prettier
- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
rev: "2.7.3"
hooks:
- id: editorconfig-checker
alias: ec
12 changes: 9 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,18 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## 1.0.0dev - 2024/03/11

Development release of phac-nml/speciesabundance, created with the [nf-core](https://nf-co.re/) template.
## 2.0.0 - 2024/04/18

### `Added`

- The initial release of phac-nml/speciesabundance as a Nextflow pipeline following [nf-core](https://nf-co.re/) standards.
- This pipeline will estimate the relative abundance of sequence reads originating from different species in a sample from Illumina short-read data.

### `Changed`

- Migrated SpeciesAbundance to a Nextflow pipeline, with integration testing using [nf-test](https://www.nf-test.com/).
- Updated the SpeciesAbundance release version from 1 to 2 to reflect migration from the [Galaxy-based](https://github.com/Public-Health-Bioinformatics/irida-plugin-species-abundance) pipeline.

### `Fixed`

### `Dependencies`
Expand Down
Loading
Loading