Skip to content

Commit

Permalink
Merge branch 'fix-pydantic-dump' of github.com:mashehu/tools into fix…
Browse files Browse the repository at this point in the history
…-pydantic-dump

# Conflicts:
#	nf_core/utils.py
  • Loading branch information
mashehu committed Oct 22, 2024
2 parents bc37fb0 + fdef700 commit 930b170
Show file tree
Hide file tree
Showing 28 changed files with 316 additions and 311 deletions.
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.9
rev: v0.7.0
hooks:
- id: ruff # linter
args: [--fix, --exit-non-zero-on-fix] # sort imports and fix
Expand All @@ -19,7 +19,7 @@ repos:
alias: ec

- repo: https://github.com/pre-commit/mirrors-mypy
rev: "v1.11.2"
rev: "v1.12.0"
hooks:
- id: mypy
additional_dependencies:
Expand Down
33 changes: 19 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

- Keep pipeline name in version.yml file ([#3223](https://github.com/nf-core/tools/pull/3223))
- Fix Manifest DOI text ([#3224](https://github.com/nf-core/tools/pull/3224))
- Do not assume pipeline name is url ([#3225](https://github.com/nf-core/tools/pull/3225))

### Download

Expand All @@ -19,7 +20,11 @@

- Include .nf-core.yml in `nf-core pipelines bump-version` ([#3220](https://github.com/nf-core/tools/pull/3220))
- create: add shortcut to toggle all switches ([#3226](https://github.com/nf-core/tools/pull/3226))
- Fix pydantic dump ([#3227](https://github.com/nf-core/tools/pull/3227))
- Remove unrelated values when saving `.nf-core` file ([#3227](https://github.com/nf-core/tools/pull/3227))
- chore(deps): update pre-commit hook pre-commit/mirrors-mypy to v1.12.0 ([#3230](https://github.com/nf-core/tools/pull/3230))
- chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.7.0 ([#3229](https://github.com/nf-core/tools/pull/3229))
- Update python:3.12-slim Docker digest to 032c526 ([#3232](https://github.com/nf-core/tools/pull/3232))
- use correct `--profile` options for `nf-core subworkflows test` ([#3233](https://github.com/nf-core/tools/pull/3233))

## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11]

Expand Down Expand Up @@ -53,7 +58,7 @@

**Highlights**

- Pipeline commands are renamed from `nf-core <comand>` to `nf-core pipelines <command>` to follow the same command structure as modules and subworkflows commands.
- Pipeline commands are renamed from `nf-core <command>` to `nf-core pipelines <command>` to follow the same command structure as modules and subworkflows commands.
- More customisation for pipeline templates. The template has been divided into features which can be skipped, e.g. you can create a new pipeline without any traces of FastQC in it.
- A new Text User Interface app when running `nf-core pipelines create` to help us guide you through the process better (no worries, you can still use the cli if you give all values as parameters)
- We replaced nf-validation with nf-schema in the pipeline template
Expand Down Expand Up @@ -206,7 +211,7 @@

### Download

- Replace `--tower` with `--platform`. The former will remain for backwards compatability for now but will be removed in a future release. ([#2853](https://github.com/nf-core/tools/pull/2853))
- Replace `--tower` with `--platform`. The former will remain for backwards compatibility for now but will be removed in a future release. ([#2853](https://github.com/nf-core/tools/pull/2853))
- Better error message when GITHUB_TOKEN exists but is wrong/outdated
- New `--tag` argument to add custom tags during a pipeline download ([#2938](https://github.com/nf-core/tools/pull/2938))

Expand Down Expand Up @@ -515,7 +520,7 @@
- Refactored the CLI parameters related to container images. Although downloading other images than those of the Singularity/Apptainer container system is not supported for the time being, a generic name for the parameters seemed preferable. So the new parameter `--singularity-cache-index` introduced in [#2247](https://github.com/nf-core/tools/pull/2247) has been renamed to `--container-cache-index` prior to release ([#2336](https://github.com/nf-core/tools/pull/2336)).
- To address issue [#2311](https://github.com/nf-core/tools/issues/2311), a new parameter `--container-library` was created allowing to specify the container library (registry) from which container images in OCI format (Docker) should be pulled ([#2336](https://github.com/nf-core/tools/pull/2336)).
- Container detection in configs was improved. This allows for DSL2-like container definitions inside the container parameter value provided to process scopes [#2346](https://github.com/nf-core/tools/pull/2346).
- Add apptainer to the list of false positve container strings ([#2353](https://github.com/nf-core/tools/pull/2353)).
- Add apptainer to the list of false positive container strings ([#2353](https://github.com/nf-core/tools/pull/2353)).

#### Updated CLI parameters

Expand Down Expand Up @@ -549,7 +554,7 @@ _In addition, `-r` / `--revision` has been changed to a parameter that can be pr
- GitPod base image: Always self-update to the latest version of Nextflow. Add [pre-commit](https://pre-commit.com/) dependency.
- GitPod configs: Update Nextflow as an init task, init pre-commit in pipeline config.
- Refgenie: Create `nxf_home/nf-core/refgenie_genomes.config` path if it doesn't exist ([#2312](https://github.com/nf-core/tools/pull/2312))
- Add CI tests to test running a pipeline whe it's created from a template skipping different areas
- Add CI tests to test running a pipeline when it's created from a template skipping different areas

## [v2.8 - Ruthenium Monkey](https://github.com/nf-core/tools/releases/tag/2.8) - [2023-04-27]

Expand Down Expand Up @@ -585,7 +590,7 @@ _In addition, `-r` / `--revision` has been changed to a parameter that can be pr

- Add an `--empty-template` option to create a module without TODO statements or examples ([#2175](https://github.com/nf-core/tools/pull/2175) & [#2177](https://github.com/nf-core/tools/pull/2177))
- Removed the `nf-core modules mulled` command and all its code dependencies ([2199](https://github.com/nf-core/tools/pull/2199)).
- Take into accout the provided `--git_remote` URL when linting all modules ([2243](https://github.com/nf-core/tools/pull/2243)).
- Take into account the provided `--git_remote` URL when linting all modules ([2243](https://github.com/nf-core/tools/pull/2243)).

### Subworkflows

Expand Down Expand Up @@ -994,7 +999,7 @@ Please note that there are many excellent integrations for Prettier available, f
- `input:` / `output:` not being specified in module
- Allow for containers from other biocontainers resource as defined [here](https://github.com/nf-core/modules/blob/cde237e7cec07798e5754b72aeca44efe89fc6db/modules/cat/fastq/main.nf#L7-L8)
- Fixed traceback when using `stageAs` syntax as defined [here](https://github.com/nf-core/modules/blob/cde237e7cec07798e5754b72aeca44efe89fc6db/modules/cat/fastq/main.nf#L11)
- Added `nf-core schema docs` command to output pipline parameter documentation in Markdown format for inclusion in GitHub and other documentation systems ([#741](https://github.com/nf-core/tools/issues/741))
- Added `nf-core schema docs` command to output pipeline parameter documentation in Markdown format for inclusion in GitHub and other documentation systems ([#741](https://github.com/nf-core/tools/issues/741))
- Allow conditional process execution from the configuration file ([#1393](https://github.com/nf-core/tools/pull/1393))
- Add linting for when condition([#1397](https://github.com/nf-core/tools/pull/1397))
- Added modules ignored table to `nf-core modules bump-versions`. ([#1234](https://github.com/nf-core/tools/issues/1234))
Expand All @@ -1013,7 +1018,7 @@ Please note that there are many excellent integrations for Prettier available, f

- Update repo logos to utilize [GitHub's `#gh-light/dark-mode-only`](https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#specifying-the-theme-an-image-is-shown-to), to switch between logos optimized for light or dark themes. The old repo logos have to be removed (in `docs/images` and `assets/`).
- Deal with authentication with private repositories
- Bump minimun Nextflow version to 21.10.3
- Bump minimum Nextflow version to 21.10.3
- Convert pipeline template to updated Nextflow DSL2 syntax
- Solve circular import when importing `nf_core.modules.lint`
- Disable cache in `nf_core.utils.fetch_wf_config` while performing `test_wf_use_local_configs`.
Expand All @@ -1035,15 +1040,15 @@ Please note that there are many excellent integrations for Prettier available, f
- Defaults in `nextflow.config` must now match the variable _type_ specified in the schema
- If you want the parameter to not have a default value, use `null`
- Strings set to `false` or an empty string in `nextflow.config` will now fail linting
- Bump minimun Nextflow version to 21.10.3
- Changed `questionary` `ask()` to `unsafe_ask()` to not catch `KeyboardInterupts` ([#1237](https://github.com/nf-core/tools/issues/1237))
- Bump minimum Nextflow version to 21.10.3
- Changed `questionary` `ask()` to `unsafe_ask()` to not catch `KeyboardInterrupts` ([#1237](https://github.com/nf-core/tools/issues/1237))
- Fixed bug in `nf-core launch` due to revisions specified with `-r` not being added to nextflow command. ([#1246](https://github.com/nf-core/tools/issues/1246))
- Update regex in `readme` test of `nf-core lint` to agree with the pipeline template ([#1260](https://github.com/nf-core/tools/issues/1260))
- Update 'fix' message in `nf-core lint` to conform to the current command line options. ([#1259](https://github.com/nf-core/tools/issues/1259))
- Fixed bug in `nf-core list` when `NXF_HOME` is set
- Run CI test used to create and lint/run the pipeline template with minimum and latest edge release of NF ([#1304](https://github.com/nf-core/tools/issues/1304))
- New YAML issue templates for tools bug reports and feature requests, with a much richer interface ([#1165](https://github.com/nf-core/tools/pull/1165))
- Handle synax errors in Nextflow config nicely when running `nf-core schema build` ([#1267](https://github.com/nf-core/tools/pull/1267))
- Handle syntax errors in Nextflow config nicely when running `nf-core schema build` ([#1267](https://github.com/nf-core/tools/pull/1267))
- Erase temporary files and folders while performing Python tests (pytest)
- Remove base `Dockerfile` used for DSL1 pipeline container builds
- Run tests with Python 3.10
Expand Down Expand Up @@ -1149,7 +1154,7 @@ This marks the first Nextflow DSL2-centric release of `tools` which means that s
- Updated `nf-core modules install` and `modules.json` to work with new directory structure ([#1159](https://github.com/nf-core/tools/issues/1159))
- Updated `nf-core modules remove` to work with new directory structure [[#1159](https://github.com/nf-core/tools/issues/1159)]
- Restructured code and removed old table style in `nf-core modules list`
- Fixed bug causing `modules.json` creation to loop indefinitly
- Fixed bug causing `modules.json` creation to loop indefinitely
- Added `--all` flag to `nf-core modules install`
- Added `remote` and `local` subcommands to `nf-core modules list`
- Fix bug due to restructuring in modules template
Expand Down Expand Up @@ -1230,7 +1235,7 @@ This marks the first Nextflow DSL2-centric release of `tools` which means that s
## [v1.13.2 - Copper Crocodile CPR :crocodile: :face_with_head_bandage:](https://github.com/nf-core/tools/releases/tag/1.13.2) - [2021-03-23]

- Make module template pass the EC linter [[#953](https://github.com/nf-core/tools/pull/953)]
- Added better logging message if a user doesn't specificy the directory correctly with `nf-core modules` commands [[#942](https://github.com/nf-core/tools/pull/942)]
- Added better logging message if a user doesn't specify the directory correctly with `nf-core modules` commands [[#942](https://github.com/nf-core/tools/pull/942)]
- Fixed parameter validation bug caused by JSONObject [[#937](https://github.com/nf-core/tools/issues/937)]
- Fixed template creation error regarding file permissions [[#932](https://github.com/nf-core/tools/issues/932)]
- Split the `create-lint-wf` tests up into separate steps in GitHub Actions to make the CI results easier to read
Expand Down Expand Up @@ -1470,7 +1475,7 @@ making a pull-request. See [`.github/CONTRIBUTING.md`](.github/CONTRIBUTING.md)
### Linting

- Refactored PR branch tests to be a little clearer.
- Linting error docs explain how to add an additional branch protecton rule to the `branch.yml` GitHub Actions workflow.
- Linting error docs explain how to add an additional branch protection rule to the `branch.yml` GitHub Actions workflow.
- Adapted linting docs to the new PR branch tests.
- Failure for missing the readme bioconda badge is now a warn, in case this badge is not relevant
- Added test for template `{{ cookiecutter.var }}` placeholders
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ version: 2.4.1
doi: 10.1038/s41587-020-0439-x
date-released: 2022-05-16
url: https://github.com/nf-core/tools
prefered-citation:
preferred-citation:
type: article
authors:
- family-names: Ewels
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.12-slim@sha256:af4e85f1cac90dd3771e47292ea7c8a9830abfabbe4faa5c53f158854c2e819d
FROM python:3.12-slim@sha256:032c52613401895aa3d418a4c563d2d05f993bc3ecc065c8f4e2280978acd249
LABEL authors="[email protected],[email protected]" \
description="Docker image containing requirements for nf-core/tools"

Expand Down
2 changes: 1 addition & 1 deletion docs/api/_src/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ This API documentation is for the [`nf-core/tools`](https://github.com/nf-core/t
- [Module commands](./module_lint_tests/) (run by `nf-core modules lint`)
- [Subworkflow commands](./subworkflow_lint_tests/) (run by `nf-core subworkflows lint`)
- [nf-core/tools Python package API reference](./api/)
- [nf-core/tools pipeline commands API referece](./api/pipelines/)
- [nf-core/tools pipeline commands API reference](./api/pipelines/)
4 changes: 2 additions & 2 deletions nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1371,7 +1371,7 @@ def command_subworkflows_create(ctx, subworkflow, directory, author, force, migr
)
@click.option(
"--profile",
type=click.Choice(["none", "singularity"]),
type=click.Choice(["docker", "singularity", "conda"]),
default=None,
help="Run tests with a specific profile",
)
Expand Down Expand Up @@ -1590,7 +1590,7 @@ def command_subworkflows_remove(ctx, directory, subworkflow):
"limit_output",
is_flag=True,
default=False,
help="Limit ouput to only the difference in main.nf",
help="Limit output to only the difference in main.nf",
)
@click.option(
"-a",
Expand Down
2 changes: 1 addition & 1 deletion nf_core/components/update.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ def __init__(
self.branch = branch

def _parameter_checks(self):
"""Checks the compatibilty of the supplied parameters.
"""Checks the compatibility of the supplied parameters.
Raises:
UserWarning: if any checks fail.
Expand Down
2 changes: 1 addition & 1 deletion nf_core/modules/lint/main_nf.py
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ def check_process_section(self, lines, registry, fix_version, progress_bar):
url = None
line = raw_line.strip(" \n'\"}:")

# Catch preceeding "container "
# Catch preceding "container "
if line.startswith("container"):
line = line.replace("container", "").strip(" \n'\"}:")

Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/conf/base.config
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ process {
maxErrors = '-1'

// Process-specific resource requirements
// NOTE - Please try and re-use the labels below as much as possible.
// NOTE - Please try and reuse the labels below as much as possible.
// These labels are used and recognised by default in DSL2 files hosted on nf-core/modules.
// If possible, it would be nice to keep the same label naming convention when
// adding in your local modules too.
Expand Down
4 changes: 2 additions & 2 deletions nf_core/pipeline-template/docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ First, go to the [{{ name }} releases page](https://github.com/{{ name }}/releas

This version number will be logged in reports when you run the pipeline, so that you'll know what you used when you look back in the future. {% if multiqc %}For example, at the bottom of the MultiQC reports.{% endif %}

To further assist in reproducbility, you can use share and re-use [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.
To further assist in reproducibility, you can use share and reuse [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.

:::tip
If you wish to share such profile (such as upload as supplementary material for academic publications), make sure to NOT include cluster specific paths to files, nor institutional specific profiles.
Expand Down Expand Up @@ -146,7 +146,7 @@ The pipeline also dynamically loads configurations from [https://github.com/nf-c
Note that multiple profiles can be loaded, for example: `-profile test,docker` - the order of arguments is important!
They are loaded in sequence, so later profiles can overwrite earlier profiles.

If `-profile` is not specified, the pipeline will run locally and expect all software to be installed and available on the `PATH`. This is _not_ recommended, since it can lead to different results on different machines dependent on the computer enviroment.
If `-profile` is not specified, the pipeline will run locally and expect all software to be installed and available on the `PATH`. This is _not_ recommended, since it can lead to different results on different machines dependent on the computer environment.

{%- if test_config %}

Expand Down
6 changes: 3 additions & 3 deletions nf_core/pipeline-template/nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ validation {
defaultIgnoreParams = ["genomes"]
help {
enabled = true
command = "nextflow run $manifest.name -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
command = "nextflow run {{ name }} -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
fullParameter = "help_full"
showHiddenParameter = "show_hidden"
{% if is_nfcore -%}
Expand All @@ -283,15 +283,15 @@ validation {
\033[0;34m |\\ | |__ __ / ` / \\ |__) |__ \033[0;33m} {\033[0m
\033[0;34m | \\| | \\__, \\__/ | \\ |___ \033[0;32m\\`-._,-`-,\033[0m
\033[0;32m`._,._,\'\033[0m
\033[0;35m ${manifest.name} ${manifest.version}\033[0m
\033[0;35m {{ name }} ${manifest.version}\033[0m
-\033[2m----------------------------------------------------\033[0m-
"""
afterText = """${manifest.doi ? "\n* The pipeline\n" : ""}${manifest.doi.tokenize(",").collect { " https://doi.org/${it.trim().replace('https://doi.org/','')}"}.join("\n")}${manifest.doi ? "\n" : ""}
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
* Software dependencies
https://github.com/${manifest.name}/blob/master/CITATIONS.md
https://github.com/{{ name }}/blob/master/CITATIONS.md
"""{% endif %}
}{% if is_nfcore %}
summary {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ def toolBibliographyText() {
}

def methodsDescriptionText(mqc_methods_yaml) {
// Convert to a named map so can be used as with familar NXF ${workflow} variable syntax in the MultiQC YML file
// Convert to a named map so can be used as with familiar NXF ${workflow} variable syntax in the MultiQC YML file
def meta = [:]
meta.workflow = workflow.toMap()
meta["manifest_map"] = workflow.manifest.toMap()
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/create/basicdetails.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def compose(self) -> ComposeResult:
@on(Input.Submitted)
def show_exists_warn(self):
"""Check if the pipeline exists on every input change or submitted.
If the pipeline exists, show warning message saying that it will be overriden."""
If the pipeline exists, show warning message saying that it will be overridden."""
config = {}
for text_input in self.query("TextInput"):
this_input = text_input.query_one(Input)
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/create/finaldetails.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def on_button_pressed(self, event: Button.Pressed) -> None:
@on(Input.Submitted)
def show_exists_warn(self):
"""Check if the pipeline exists on every input change or submitted.
If the pipeline exists, show warning message saying that it will be overriden."""
If the pipeline exists, show warning message saying that it will be overridden."""
outdir = ""
for text_input in self.query("TextInput"):
this_input = text_input.query_one(Input)
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/create/githubrepo.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def compose(self) -> ComposeResult:
yield TextInput(
"repo_org",
"Organisation name",
"The name of the organisation where the GitHub repo will be cretaed",
"The name of the organisation where the GitHub repo will be created",
default=self.parent.TEMPLATE_CONFIG.org,
classes="column",
)
Expand Down
Loading

0 comments on commit 930b170

Please sign in to comment.