v2.3.6.0

What's Changed

enterics related

• update enterics visualization docker image by @jpramosbarroso in #516
• Make create_enterics_qc_viz whitespace tolerant by @dpark01 in #517
• update personal branch by @jpramosbarroso in #520
• create_enterics_qc_viz simplify and improve portability by @jpramosbarroso in #521
• update docker for try/except block with new error message by @jpramosbarroso in #522

assembly scaffolding & reference selection improvements

• scaffold_and_refine_multitaxa -- more thorough assembly outputs by @dpark01 in #524
• scaffolding and reference selection based on ANI by @dpark01 in #528
• Reference selection improvements and nextclade generalization by @dpark01 in #529
• scaffolding regression fixes plus docker updates by @dpark01 in #547
• scaffold multitaxa: drop empty assembly output rows by @dpark01 in #554
• report reference genome length in align_reads output; use this value for assemble_refbased. reference_genome_length by @tomkinsc in #538

demux_deplete and terra table data model updates

• demux_deplete: make depletion optional, load biosample metadata to tables by @dpark01 in #551
• demux_deplete small updates by @dpark01 in #552
• demux_deplete fixes/updates by @dpark01 in #555

miscellaneous fixes and edge cases

• allow additional options for picard in FastqToUBAM by @tomkinsc in #518
• WDL code/style cleanups by @dpark01 in #523
• expanded parameterization of align_and_count and additional output metrics by @tomkinsc in #525
• guard against shell pipefails in align_and_count in instances where empty inputs would result in div-by-zero conditions by @tomkinsc in #535
• classify_single: parameterize taxa to deplete by @dpark01 in #536
• bugfix fetch_sra_to_bam by @tomkinsc in #542
• Update polyphonia input parameters and defaults by @lakras in #545

docker container updates

• build and container updates by @dpark01 in #526
• update viral-core 2.3.0 -> 2.3.1 by @tomkinsc in #527
• increase default instance memory for kraken2 task 72 -> 90GB by @tomkinsc in #540
• nextclade updates by @dpark01 in #557
• bump upstream docker images by @dpark01 in #556

build & CI related

• add build environment spec to .readthedocs.yml config by @tomkinsc in #541
• documentation build fixes by @tomkinsc in #546
• bugfix dx deploy failures by @tomkinsc in #548

Full Changelog: v2.2.4.0...v2.3.6.0

v2.2.4.0

What's Changed

New workflows

• Enterics QC visualizations
  • WDL for creating Enterics Visualizations by @schaluva in #500
  • create generalized version of enterics visualization WDL by @schaluva in #501
  • Visualization HTML PR by @jpramosbarroso in #513
  • Fixing duplicate inputs bug by @jpramosbarroso in #514
  • integrating threshold file by @jpramosbarroso in #515
• Terra-on-GCP-specific features
  • add check_terra_env task by @tomkinsc in #497
  • add workflows to add to or create library and sample tables based on demultiplexing output by @tomkinsc in #507
  • mark check_terra_env task as volatile by @tomkinsc in #510
  • include user email among outputs of check_terra_env task by @tomkinsc in #508
• transmission reconstruction from @ispecht
  • add reconstructR workflow by @dpark01 in #504
• mass-scale metagenomic denovo assembly
  • add new workflow scaffold_and_refine_multitaxa by @dpark01 in #506
  • more scaffolding updates by @dpark01 in #511

Updates to existing workflows

• pass through sbt template if optional defaults_yaml is not provided and author_list string is empty by @tomkinsc in #492
• mark downsample_bams task as volatile (nondeterministic) by @tomkinsc in #494
• expose numberOfNegativeControls in illumina_demux by @tomkinsc in #498
• add masked_positions and masked_positions_file arguments to task polyphonia_detect_cross_contamination by @lakras in #499
• fix mem high watermark reporting by @tomkinsc in #502
• mass update of upstream docker tags by @dpark01 in #505
  • broadinstitute/viral-core: 2.1.33 -> 2.2.4
  • broadinstitute/viral-classify: 2.1.33.0 -> 2.2.3.0 (later bumped to 2.2.4.0 in #511)
  • broadinstitute/viral-baseimage: 0.1.20 -> 0.2.0
  • staphb/vadr: 1.5.1 -> 1.6.3
  • staphb/pangolin: 4.3.1-pdata-1.22 -> 4.3.1-pdata-1.23.1
  • nextstrain/nextclade: 2.12.0 -> 2.14.0

Build and CI

• enable merge_group builds in github actions CI by @dpark01 in #503
• allow docker image pinning to be skipped with #skip-global-version-pin annotation by @tomkinsc in #493
• dockstore updates by @dpark01 in #512
• clean up disk space in Github Actions VMs #505

New Contributors

• @jpramosbarroso made their first contribution in #513
• @ispecht 's code is utilized in a new workflow for #504

Full Changelog: v2.1.33.17...v2.2.4.0

v2.1.33.17

What's Changed

new features or functionality

• extend beast task and workflow to glob and output *.ops, *.rates, and *.root files by @tomkinsc in #478
• Subsampler and augur workflow by @dpark01 in #483

bug fixes and small changes

• syntax bugfixes by @dpark01 in #462
• tweaks to demux_only and demux_metag by @dpark01 in #464
• omit --dirs from the MultiQC invocation to only show sample names in report by @tomkinsc in #467
• hotfix for bug in output path by @dpark01 in #480
• sc2 nextstrain: reduce temp file sizes by @dpark01 in #484
• Fetch_SRA_to_BAM - tolerate no isolate defined by @dpark01 in #489
• Update genbank.WDL so author_list input is optional by @tomkinsc in #473

documentation updates

• Enhancing documentation for viral-pipelines by @golu099 in #461
• move transposed parameter_meta block outside command block scope to where it belongs by @tomkinsc in #477
• Update LICENSE copyright year by @tomkinsc in #476

parameterization and default VM shapes

• de novo assembly & scaffold mem default 31 -> 63 GB by @tomkinsc in #465
• parameterize disk size for illumina_demux task by @tomkinsc in #470
• max_records_in_ram param 2000000 -> 10000000 for Nova S2 by @tomkinsc in #468

upstream docker tags / version dependencies

• bump pangolin by @dpark01 in #482

build updates and fixes

• delete the .travis directory by @tomkinsc in #448
• dnanexus CI build fixes by @dpark01 in #485 #486 #487 #488
• GitHub actions update by @tomkinsc in #474

Full Changelog: v2.1.33.16...v2.1.33.17

v2.1.33.16

What's Changed

functionality changes and bug fixes

• Convert amplicon16S_analysis workflow (QIIME) from single-sample to multi-sample by @golu099 in #452
• samtools ampliconstats needs bed in certain order by @dpark01 in #454

parameterization and defaults

• parameterize and update defaults for VM shapes in nextstrain tasks by @dpark01 in #455
• parameterize more assembly knobs by @dpark01 in #457

update docker tags / version dependencies

• update to pangolin 4.2 / pdata 1.18 by @dpark01 in #451
• Update pangolin, nextclade, and vadr docker images by @dpark01 in #456

build updates and fixes

• documentation build fixes for ReadTheDocs by @tomkinsc in #450
• update comment about DX_API_TOKEN by @tomkinsc in #449

Full Changelog: v2.1.33.15...v2.1.33.16

v2.1.33.15

What's Changed

new and changed workflows

• Add workflow for QIIME2 classification of 16S reads by @golu099 in #439
• simplify augur_from_assemblies workflow by @dpark01 in #440
• denovo assembly SPAdes updates by @dpark01 in #441
• genbank prep and assemble_denovo updates by @dpark01 in #443
• add new workflow nextclade_single by @dpark01 in #444
• update classify_single to accept multiple bams by @dpark01 in #447

smaller docker, VM, and mechanical updates

• more disk for nextstrain; dockstore API fix by @dpark01 in #437
• Make Pipelines Azure compatible Closes #143 by @vsmalladi in #438
• set default GPU to K80->P4 in BEAST task; parameterize beagle_order by @tomkinsc in #442
• pangolin 4.1.3 / pdata 1.17 by @dpark01 in #445

New Contributors

• @vsmalladi made their first contribution in #438
• @golu099 made their first contribution in #439

Full Changelog: v2.1.33.14...v2.1.33.15

v2.1.33.14

What's Changed

Feature changes

• task alignment_metrics: more resilient to large amplicon designs by @dpark01 in #435

Docker image updates

• update nextclade and vadr docker images by @dpark01 in #433
• bump pangolin to 4.1.2-1.14 and nextclade to 2.5.0 by @dpark01 in #434

Full Changelog: v2.1.33.13...v2.1.33.14

v2.1.33.13

What's Changed

Feature changes

• CDC SC2 viral pipeline updates (parameters, reporting, QC) by @dpark01 in #431
• expose exclude/drop list input for nextstrain_build_subsample task by @tomkinsc in #428

Docker image updates

• nextclade 2.0 updates by @dpark01 in #430
• update pangolin docker to 4.1.1-pdata-1.11 by @dpark01 in #432

VM shape tuning

• parameterize machine_mem_gb for ancestral_traits task by @tomkinsc in #427
• in export_auspice_json parameterize task mem, raise default to 64GB; increase AUGUR_RECURSION_LIMIT from 10k to 15k by @tomkinsc in #429

Full Changelog: v2.1.33.12...v2.1.33.13

v2.1.33.12

What's Changed

• lofreq task portability, misc other edits by @dpark01 in #426

Full Changelog: v2.1.33.11...v2.1.33.12

v2.1.33.11

What's Changed

• incorporate assemble_refbased as refine/polish for denovo by @dpark01 in #422
• new metagenomic_denovo workflow by @dpark01 in #423
• consensus calling major_cutoff knobs and defaults by @dpark01 in #424
• update pangolin to pdata 1.9 by @dpark01 in #425

Full Changelog: v2.1.33.10...v2.1.33.11

v2.1.33.10

What's Changed

Minor functional updates

• CDC contractual updates to sarscov2_illumina_full (min coverage, genome name default prefix) - #419
• pangolin tasks: update_dbs_now invokes update data only (not pangolin software) - #420

Docker image updates only:

• pangolin docker: several routine docker image updates - #414 #416 #417 #421

Full Changelog: v2.1.33.9...v2.1.33.10