Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove kraken-biom dep #14

Merged
merged 21 commits into from
Mar 29, 2024
Merged

Remove kraken-biom dep #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d2f1cd2
Add gitignore
Ulthran Nov 9, 2023
f783174
Fix test workflow
Ulthran Nov 9, 2023
e054c05
Add conda_env_check
Ulthran Nov 9, 2023
384dd04
Fix empty test check
Ulthran Nov 9, 2023
fe98d07
Don't use pin with pip in env
Ulthran Nov 9, 2023
eafa840
Fix test path
Ulthran Nov 9, 2023
cd08820
Test sbx_test_action v1
Ulthran Nov 9, 2023
0f1375d
Strip newline
Ulthran Nov 9, 2023
fd5652a
Add summary script
Ulthran Feb 14, 2024
ab767f3
Refactor CI/CD workflows
Ulthran Feb 14, 2024
a90eb54
Merge with main
Ulthran Feb 14, 2024
b4502c6
Little refactoring in e2e test
Ulthran Feb 14, 2024
4570ef4
Actually write output file
Ulthran Feb 14, 2024
296bbd0
Resolve abs paths
Ulthran Feb 14, 2024
7cba513
Update empty input report line
Ulthran Feb 15, 2024
fb8f1aa
Update test checks
Ulthran Feb 15, 2024
6d5ed60
Actually fix test values
Ulthran Feb 15, 2024
e98b4a7
Find bacteria line
Ulthran Feb 16, 2024
a22930f
DEBUG
Ulthran Feb 16, 2024
343a433
Functioning summary script and unit tests
Ulthran Feb 23, 2024
f942fcd
Improve unit testing
Ulthran Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 0 additions & 24 deletions .github/workflows/linter.yml
View file
Open in desktop

This file was deleted.

15 changes: 15 additions & 0 deletions .github/workflows/pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Tests

on:
pull_request:
branches:
- main
push:
branches:
- main

jobs:
run-tests:
uses: ./.github/workflows/tests.yml
secrets: inherit

42 changes: 42 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Release

on:
release:
types: [published]

workflow_dispatch:

jobs:
run-tests:
uses: ./.github/workflows/tests.yml
secrets: inherit

build-and-push-to-dockerhub:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
needs: run-tests

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ctbushman/sbx_mapping

- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
54 changes: 46 additions & 8 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,64 @@ on:
branches: [ master, main ]
push:
branches: [ master, main ]
workflow_call:
workflow_dispatch:
schedule:
- cron: "0 13 * * 1"

jobs:
jobs:
lint:
name: Lint Code
runs-on: ubuntu-latest

steps:
- name: Checkout Code
uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: 3.12

- name: Install Dependencies
run: pip install black snakefmt

- name: Run Linter
run: |
black --check .
snakefmt --check *.smk

test-unit:
name: Run Extension Unit Tests
runs-on: ubuntu-latest

steps:
- name: Checkout Code
uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: 3.12

- name: Install Dependencies
run: pip install pytest

- name: Run Unit Tests
run: pytest scripts/

test-e2e:
name: Test Extension with Sunbeam
runs-on: ubuntu-latest
needs:
- test-unit
- lint

steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Test with Sunbeam
uses: sunbeam-labs/sbx_test_action@v0
with:
sunbeam-version: 'dev'
install-options: '-m'
extension-name: 'sunbeam-labs/sbx_kraken'
extension-branch-name: ${{ github.head_ref }}

uses: sunbeam-labs/sbx_test_action@v1

- name: Dump Logs
shell: bash
if: always()
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.tests/data/hosts/*.fasta.*
__pycache__/
100 changes: 100 additions & 0 deletions .tests/data/kraken2-outputs/AP159-taxa.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
87.97 14867039 14867039 U 0 unclassified
12.03 2033986 530 R 1 root
12.02 2031274 9769 R1 131567 cellular organisms
10.73 1813577 88192 D 2 Bacteria
5.52 933149 20754 D1 1783272 Terrabacteria group
3.33 562260 22054 P 1239 Firmicutes
2.54 429097 1049 C 186801 Clostridia
2.50 422352 25214 O 186802 Clostridiales
1.07 180394 10794 F 186803 Lachnospiraceae
0.44 73621 3052 G 830 Butyrivibrio
0.23 38775 38775 S 831 Butyrivibrio fibrisolvens
0.11 18745 18745 S 185008 Butyrivibrio hungatei
0.08 13049 0 S 43305 Butyrivibrio proteoclasticus
0.08 13049 13049 S1 515622 Butyrivibrio proteoclasticus B316
0.15 25976 2541 G 1506553 Lachnoclostridium
0.03 5155 5155 S 1871021 Lachnoclostridium phocaeense
0.02 4029 0 G1 2608895 unclassified Lachnoclostridium
0.02 4029 4029 S 1834196 Lachnoclostridium sp. YL32
0.02 3524 3157 S 29347 [Clostridium] scindens
0.00 367 367 S1 411468 [Clostridium] scindens ATCC 35704
0.02 3052 3052 S 208479 [Clostridium] bolteae
0.01 2512 0 S 89153 [Clostridium] hylemonae
0.01 2512 2512 S1 553973 [Clostridium] hylemonae DSM 15053
0.01 2047 0 S 84030 [Clostridium] saccharolyticum
0.01 2047 2047 S1 610130 [Clostridium] saccharolyticum WM1
0.01 2006 0 S 29370 [Clostridium] sphenoides
0.01 2006 2006 S1 1297793 [Clostridium] sphenoides JCM 1415
0.01 1110 0 S 66219 Lachnoclostridium phytofermentans
0.01 1110 1110 S1 357809 Lachnoclostridium phytofermentans ISDg
0.12 20338 4398 F1 186928 unclassified Lachnospiraceae
0.04 7159 7159 S 2109691 Lachnospiraceae bacterium GAM79
0.02 2671 2671 S 2594789 Lachnospiraceae bacterium KGMB03038
0.01 2444 2444 S 2109690 Lachnospiraceae bacterium Choco86
0.01 1519 1519 S 2093742 Lachnospiraceae bacterium KM106-2
0.01 1080 1080 S 712991 Lachnospiraceae bacterium oral taxon 500
0.01 1067 1067 S 1898203 Lachnospiraceae bacterium
0.09 15864 3548 G 572511 Blautia
0.04 6677 37 G1 2648079 unclassified Blautia
0.03 4743 4743 S 2479767 Blautia sp. SC05B48
0.01 1893 1893 S 1912897 Blautia sp. N6H1-15
0.00 4 4 S 1796616 Blautia sp. YL58
0.02 2937 2937 S 33035 Blautia producta
0.02 2701 0 S 33038 [Ruminococcus] gnavus
0.02 2701 2701 S1 411470 [Ruminococcus] gnavus ATCC 29149
0.00 1 1 S 1532 Blautia coccoides
0.07 12045 0 G 46205 Pseudobutyrivibrio
0.07 12045 12045 S 185007 Pseudobutyrivibrio xylanivorans
0.05 8952 243 G 841 Roseburia
0.03 4771 4771 S 301301 Roseburia hominis
0.02 3938 0 S 166486 Roseburia intestinalis
0.02 3938 3938 S1 536231 Roseburia intestinalis L1-82
0.02 3870 45 G 207244 Anaerostipes
0.01 1930 1930 S 1229621 Anaerostipes rhamnosivorans
0.01 1895 1895 S 649756 Anaerostipes hadrus
0.02 3304 0 G 2569097 Anaerobutyricum
0.02 3304 3304 S 39488 Anaerobutyricum hallii
0.01 1675 0 G 1164882 Lachnoanaerobaculum
0.01 1675 1675 S 617123 Lachnoanaerobaculum umeaense
0.01 1476 0 G 1843210 Anaerocolumna
0.01 1476 0 G1 2642604 unclassified Anaerocolumna
0.01 1476 1476 S 2696063 Anaerocolumna sp. CBA3638
0.01 963 0 G 1663717 Herbinix
0.01 963 963 S 1679721 Herbinix luporum
0.00 762 0 G 2039240 Anaerotignum
0.00 762 0 S 28446 Anaerotignum propionicum
0.00 762 762 S1 991789 Anaerotignum propionicum DSM 1682
0.00 754 370 G 698776 Cellulosilyticum
0.00 209 0 G1 2643091 unclassified Cellulosilyticum
0.00 209 209 S 2497860 Cellulosilyticum sp. WCF-2
0.00 175 0 S 29360 Cellulosilyticum lentocellum
0.00 175 175 S1 642492 Cellulosilyticum lentocellum DSM 5427
0.51 86485 1367 F 541000 Ruminococcaceae
0.31 52268 957 G 1263 Ruminococcus
0.15 25311 0 G1 2608920 unclassified Ruminococcus
0.15 25311 25311 S 2564099 Ruminococcus sp. JE7A12
0.10 16250 0 S 1264 Ruminococcus albus
0.10 16250 16250 S1 697329 Ruminococcus albus 7 = DSM 20455
0.03 5656 5656 S 1161942 Ruminococcus champanellensis
0.02 4094 4094 S 1160721 Ruminococcus bicirculans
0.09 15611 0 G 216851 Faecalibacterium
0.09 15611 15611 S 853 Faecalibacterium prausnitzii
0.03 5466 0 G 946234 Flavonifractor
0.03 5466 5466 S 292800 Flavonifractor plautii
0.02 3543 0 G 1905344 Ruthenibacterium
0.02 3543 3543 S 1550024 Ruthenibacterium lactatiformans
0.02 3136 6 F1 552397 unclassified Ruminococcaceae
0.01 1903 0 F2 2305133 unclassified Ruminococcaceae (miscellaneous)
0.01 1903 1903 S 1572656 Ruminococcaceae bacterium CPB6
0.01 1227 1227 S 29343 [Clostridium] cellulosi
0.02 2901 0 G 1738645 Caproiciproducens
0.02 2901 0 G1 2643836 unclassified Caproiciproducens
0.02 2901 2901 S 2507162 Caproiciproducens sp. NJN-50
0.01 2193 0 G 253238 Ethanoligenens
0.01 2193 2193 S 253239 Ethanoligenens harbinense
0.30 50190 663 F 31979 Clostridiaceae
0.23 39350 2303 G 1485 Clostridium
0.11 18483 18094 S 1491 Clostridium botulinum
0.00 144 144 S1 929506 Clostridium botulinum BKT015925
0.00 76 0 S1 36827 Clostridium botulinum B
0.00 65 65 S2 935198 Clostridium botulinum B str. Eklund 17B (NRP)
78 changes: 78 additions & 0 deletions .tests/data/kraken2-outputs/DNAfreewater1.20230906-taxa.tsv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
61.11 22 22 U 0 unclassified
38.89 14 0 R 1 root
38.89 14 0 R1 131567 cellular organisms
36.11 13 0 D 2 Bacteria
16.67 6 0 P 1224 Proteobacteria
11.11 4 0 C 28216 Betaproteobacteria
11.11 4 0 O 80840 Burkholderiales
11.11 4 0 F 119060 Burkholderiaceae
11.11 4 0 G 48736 Ralstonia
11.11 4 0 S 329 Ralstonia pickettii
11.11 4 4 S1 402626 Ralstonia pickettii 12J
5.56 2 0 P1 68525 delta/epsilon subdivisions
5.56 2 0 C 28221 Deltaproteobacteria
5.56 2 0 O 213115 Desulfovibrionales
5.56 2 0 F 194924 Desulfovibrionaceae
5.56 2 0 G 872 Desulfovibrio
5.56 2 2 S 901 Desulfovibrio piger
11.11 4 0 D1 1783270 FCB group
11.11 4 0 D2 68336 Bacteroidetes/Chlorobi group
11.11 4 1 P 976 Bacteroidetes
8.33 3 0 C 200643 Bacteroidia
8.33 3 0 O 171549 Bacteroidales
5.56 2 0 F 815 Bacteroidaceae
5.56 2 0 G 816 Bacteroides
5.56 2 2 S 817 Bacteroides fragilis
2.78 1 0 F 171552 Prevotellaceae
2.78 1 0 G 838 Prevotella
2.78 1 0 S 52227 Prevotella dentalis
2.78 1 1 S1 908937 Prevotella dentalis DSM 3688
8.33 3 0 D1 1783272 Terrabacteria group
5.56 2 0 P 201174 Actinobacteria
2.78 1 0 C 1760 Actinobacteria
2.78 1 0 O 85010 Pseudonocardiales
2.78 1 0 F 2070 Pseudonocardiaceae
2.78 1 0 G 1847 Pseudonocardia
2.78 1 0 G1 2619320 unclassified Pseudonocardia
2.78 1 1 S 1690815 Pseudonocardia sp. HH130630-07
2.78 1 0 C 84998 Coriobacteriia
2.78 1 0 O 84999 Coriobacteriales
2.78 1 0 F 84107 Coriobacteriaceae
2.78 1 0 F1 84113 unclassified Coriobacteriaceae
2.78 1 1 S 1531429 Coriobacteriaceae bacterium 68-1-3
2.78 1 0 P 1239 Firmicutes
2.78 1 0 C 909932 Negativicutes
2.78 1 0 O 909929 Selenomonadales
2.78 1 0 F 1843491 Selenomonadaceae
2.78 1 0 G 970 Selenomonas
2.78 1 1 G1 2637378 unclassified Selenomonas
2.78 1 0 D 2759 Eukaryota
2.78 1 0 D1 33154 Opisthokonta
2.78 1 0 K 33208 Metazoa
2.78 1 0 K1 6072 Eumetazoa
2.78 1 0 K2 33213 Bilateria
2.78 1 0 K3 33511 Deuterostomia
2.78 1 0 P 7711 Chordata
2.78 1 0 P1 89593 Craniata
2.78 1 0 P2 7742 Vertebrata
2.78 1 0 P3 7776 Gnathostomata
2.78 1 0 P4 117570 Teleostomi
2.78 1 0 P5 117571 Euteleostomi
2.78 1 0 P6 8287 Sarcopterygii
2.78 1 0 P7 1338369 Dipnotetrapodomorpha
2.78 1 0 P8 32523 Tetrapoda
2.78 1 0 P9 32524 Amniota
2.78 1 0 C 40674 Mammalia
2.78 1 0 C1 32525 Theria
2.78 1 0 C2 9347 Eutheria
2.78 1 0 C3 1437010 Boreoeutheria
2.78 1 0 C4 314146 Euarchontoglires
2.78 1 0 O 9443 Primates
2.78 1 0 O1 376913 Haplorrhini
2.78 1 0 O2 314293 Simiiformes
2.78 1 0 O3 9526 Catarrhini
2.78 1 0 O4 314295 Hominoidea
2.78 1 0 F 9604 Hominidae
2.78 1 0 F1 207598 Homininae
2.78 1 0 G 9605 Homo
2.78 1 1 S 9606 Homo sapiens
Loading
Loading