tests: import tests with details from snapd-testing-tools (canonical#…

…13846) * Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..94ac5ffe58 94ac5ffe58 Add details on tests 501578c719 add more checks in os.query to check is-core_xx e8929207ff fix os-query for ubuntu comparing with core 226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems b89ec98b23 use local variables in os.query tool dacfd81de9 fix is_core functions 1db5214d5f Improve the remote docs (canonical#36) 2e4a3153a2 1 more comment 3a0fc57e1e add explanation about why we check for ( Do | Doing ) 4cf8e635bf fix os.query test after merge b89b4f8647 fix artifacts name d30cee6da0 Merge remote-tracking branch 'upstream/main' 5ef5dcbe8f Tests use artifacts in spread tests (canonical#51) 555c43d2ab Support auto-refresh with Do instead of Doing 96c2b0c19c remove tests support for ubuntu 23.04 (EoL) 74082c0c34 Tests improve remote wait (canonical#49) 5121bfb659 remove support for opensuse leap 15.4 (canonical#48) 30df700d08 Add new systems support (canonical#47) 1f08938925 Support check amazon linux version (canonical#46) 43533bdd97 Change the exit value checking for test formats (canonical#45) 3c88244c04 Update check-test-format to support a dir and a list of files (canonical#44) 510d95f429 add extra check for error in auto-refresh detection function 3289d4031b Try open the log with latin-1 encoding when utf-8 is not working 9db785499f improved how the tools are waiting for system reboot 2a5c4414a3 fix shellcheck errors 5e7b63883d Fixes for osquery and tests pkgs (canonical#43) 4c9145e2ac support reboot waiting for auto-refresh 45768f5188 show changes in unknown status after refresh 8013c30c2a Remove support for ubuntu 22.10 b32b80bf54 Fix remote.rait-for test in bionic 5675c625e9 Enable fedora 38 55f4471957 Support for new oss f2e88b357c New tool used to query spread json reports cacd35ede0 utils/spread-shellcheck: explain disabled warnings (canonical#42) c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs b84eea92e2 spread-shellcheck: fix quotes in environment variables (canonical#41) ab1e51c29f New comparison in os-query for core systems (canonical#40) e5ae22a5d4 systemd units can be overwritten 63540b845a Fix error messages in remote pull and push 75e8a426a5 make sure the unit is removed in tests.systemd test 9089ff5c02 Update tests to use the new tests.systemd stop-unit 44ecd5e56a Move tests.systemd stop-units to stop-unit 01a2a83b4b Update tests.systemd to have stop units as systemd.sh 162e93bd35 update tests.systemd CLI options to be the same than retry command 14aa43a405 new feature to re-run failed spread tests (canonical#39) 604cb782db Fix shellcheck in systemd tool bfc71082c8 Update the tests.systemd to allow parameters waiting for service status 8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs d90935d2a4 A comment explaining about the default values for wait-for 3232c5dba7 Add support for ubuntu 23.04 a7164fba07 remove fedora 35 support, add fedora 37 support 89b9eb5301 Update systems supported 92bb6a0664 Include snap-sufix in the snaps.name tool git-subtree-dir: tests/lib/external/snapd-testing-tools git-subtree-split: 94ac5ffe58b7e52f19f142453a78bd897bace122 * Update tests/lib/external/snapd-testing-tools/tests/log-analyzer/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/os.query/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/not/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/os.paths/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/remote.setup/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/tests.pkgs/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/spread-manager/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/snaps.name/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/tests.systemd/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/spread-shellcheck/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Update tests/lib/external/snapd-testing-tools/tests/retry/task.yaml Co-authored-by: Miguel Pires <[email protected]> * Squashed 'tests/lib/external/snapd-testing-tools/' changes from 94ac5ffe58..fc10196efd fc10196efd Add suggestions to details git-subtree-dir: tests/lib/external/snapd-testing-tools git-subtree-split: fc10196efd33bd52c94323781de647ecf257f1d0 --------- Co-authored-by: Miguel Pires <[email protected]>
pedronis · Apr 22, 2024 · f2c7b3f · f2c7b3f
1 parent 6baa741
commit f2c7b3f
Show file tree

Hide file tree

Showing 31 changed files with 119 additions and 1 deletion.
diff --git a/tests/lib/external/snapd-testing-tools/tests/check-test-format/task.yaml b/tests/lib/external/snapd-testing-tools/tests/check-test-format/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the spread checker tool
 
+details: |
+    Verify that the check-test-format tool checks properly the desired
+    order and mandatory keys in spread tasks.
+
 backends: [google]
 
 systems: [ ubuntu-20.04-64 ]

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/task.yaml
@@ -1,5 +1,10 @@
 summary: integration tests for log-analyzer tool
 
+details: |
+    This test checks the log-analyzer properly retrieves the tests-to-reexecute
+    based on the spread log analysis. Also checks it properly retrieves the
+    tests by result status and all the executed tasks.
+
 backends: [google]
 
 # Github actions agents are just running ubuntu jammy

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-1/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-1/task.yaml
@@ -1,5 +1,7 @@
 summary: summary
 
+details: test
+
 prepare: |
     echo "prepare"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-2/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-2/task.yaml
@@ -1,5 +1,7 @@
 summary: summary
 
+details: test
+
 prepare: |
     echo "prepare"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-3/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-3/task.yaml
@@ -1,5 +1,7 @@
 summary: summary
 
+details: test
+
 prepare: |
     echo "prepare"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-4/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-4/task.yaml
@@ -1,5 +1,7 @@
 summary: summary
 
+details: test
+
 prepare: |
     echo "prepare"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-5/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-analyzer/tests/test-5/task.yaml
@@ -1,5 +1,7 @@
 summary: summary
 
+details: test
+
 prepare: |
     echo "prepare"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/log-parser/task.yaml b/tests/lib/external/snapd-testing-tools/tests/log-parser/task.yaml
@@ -1,5 +1,10 @@
 summary: test for the log parser tool
 
+details: |
+    This test checks the log-parser tool is able to read different spread
+    logs and produce a json file with the logs info. This json file contains
+    all the information from the log but structured in json format.
+
 backends: [google]
 
 # Github actions agents are just running ubuntu jammy

diff --git a/tests/lib/external/snapd-testing-tools/tests/not/task.yaml b/tests/lib/external/snapd-testing-tools/tests/not/task.yaml
@@ -1,5 +1,8 @@
 summary: test for the not tool
 
+details: |
+    Check the not tool works the same as the logical not (!).
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/os.paths/task.yaml b/tests/lib/external/snapd-testing-tools/tests/os.paths/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the os.paths tool
 
+details: |
+    Check the os.paths tool properly retrieves the snap-mount-dir,
+    media-dir and libexec-dir in the current system.
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/os.query/task.yaml b/tests/lib/external/snapd-testing-tools/tests/os.query/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the os.query tool
 
+details: |
+    Check the os.query tool properly detects the current system. Also
+    verify the version comparison works.
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/quiet/task.yaml b/tests/lib/external/snapd-testing-tools/tests/quiet/task.yaml
@@ -1,5 +1,8 @@
 summary: test for the quiet tool
 
+details: |
+    Check the quiet tool hides the command stdout and but shows the stderr
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.exec/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.exec/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the remote.exec tool
 
+datails: |
+    Check the remote.exec tool is able to run commands in the
+    remote instance through ssh.
+
 backends: [google]
 
 # Amazon linux is skipped because no sshpass available

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.pull/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.pull/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the remote.pull tool
 
+details: |
+    Check the remote.pull tool is able to retrieve files from the
+    remote instance through scp
+
 backends: [google]
 
 # Amazon linux is skipped because no sshpass available

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.push/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.push/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the remote.push tool
 
+details: |
+    Check the remote.push tool is able to send files from
+    localhost to the remote instance through scp
+
 backends: [google]
 
 # Amazon linux is skipped because no sshpass available

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.refresh/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.refresh/task.yaml
@@ -1,4 +1,9 @@
-summary: smoke test for the remote.wait-for tool
+summary: smoke test for the remote.refresh tool
+
+details: |
+    Check the remote.refresh tool properly handles refreshes in
+    the remote instance including the refresh for several snaps
+    which trigger reboots
 
 backends: [google-nested]
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.setup/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.setup/task.yaml
@@ -1,5 +1,10 @@
 summary: smoke test for the remote.setup tool
 
+details: |
+    Check the remote.setup tool properly sets up up the
+    information required to allow connecting to the remote
+    instance through ssh.
+
 backends: [google]
 
 restore: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/remote.wait-for/task.yaml b/tests/lib/external/snapd-testing-tools/tests/remote.wait-for/task.yaml
@@ -1,5 +1,10 @@
 summary: smoke test for the remote.wait-for tool
 
+details: |
+    Check the remote.wait-for tool waits properly for several
+    events in the remote instance. Supported events are:
+    device-initialized, reboot, refresh, ssh, no-ssh, etc
+
 backends: [google-nested]
 
 systems: [ubuntu-20.04-64]

diff --git a/tests/lib/external/snapd-testing-tools/tests/repack-kernel/task.yaml b/tests/lib/external/snapd-testing-tools/tests/repack-kernel/task.yaml
@@ -1,5 +1,10 @@
 summary: tests for the repack-kernel tool
 
+details: |
+    Check the repack-kernel tools properly handles extraction of the
+    kernel snap to a workspace directory, and then repacks it back
+    to a snap
+
 backends: [google]
 
 # ubuntu-core-initramfs seems to be only available on ubuntu-20.04

diff --git a/tests/lib/external/snapd-testing-tools/tests/retry/task.yaml b/tests/lib/external/snapd-testing-tools/tests/retry/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the retry tool
 
+details: |
+    Check the retry tool allows retrying a command. Verify that on
+    failure the exit code from the final attempt is returned
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/snaps.cleanup/task.yaml b/tests/lib/external/snapd-testing-tools/tests/snaps.cleanup/task.yaml
@@ -1,5 +1,10 @@
 summary: smoke test for the snaps.cleanup tool
 
+details: |
+    Check the snaps.cleanup tool handles properly the
+    removal of the installed snaps in the current system.
+    The test also verifies the fundamental snaps are not removed
+
 backends: [google]
 
 systems: [ubuntu-16.04-64, ubuntu-18.04-64, ubuntu-20.04-64, ubuntu-22.04-64]

diff --git a/tests/lib/external/snapd-testing-tools/tests/snaps.name/task.yaml b/tests/lib/external/snapd-testing-tools/tests/snaps.name/task.yaml
@@ -1,5 +1,10 @@
 summary: smoke test for the snaps.name tool
 
+details: |
+    Check the snaps.name tool properly retrieves the names of the
+    gadget and kernel snaps installed in the system. Also verifies
+    the snap-sufix command works properly.
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task1/task.yaml b/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task1/task.yaml
@@ -1,5 +1,7 @@
 summary: this is the summary
 
+details: test
+
 prepare: |
     echo "preparing"
 

diff --git a/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task3/task.yaml b/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task3/task.yaml
@@ -1,5 +1,7 @@
 summary: this is the summary
 
+details: test
+
 manual: true
 
 prepare: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task4/task.yaml b/tests/lib/external/snapd-testing-tools/tests/spread-manager/checks/task4/task.yaml
@@ -1,5 +1,7 @@
 summary: this is the summary
 
+details: test
+
 #manual: true
 
 prepare: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/spread-manager/task.yaml b/tests/lib/external/snapd-testing-tools/tests/spread-manager/task.yaml
@@ -1,5 +1,9 @@
 summary: smoke test for the spread-manager tool
 
+details: |
+    Check the spread-manager tool allows setting a list of tests as manual.
+    Also checks manual tag can be removed for a list of tests.
+
 backends: [google]
 
 systems: [ ubuntu-20.04-64 ]

diff --git a/tests/lib/external/snapd-testing-tools/tests/spread-shellcheck/task.yaml b/tests/lib/external/snapd-testing-tools/tests/spread-shellcheck/task.yaml
@@ -1,5 +1,11 @@
 summary: smoke test for the spread-shellcheck tool
 
+details: |
+    Check the spread-shellcheck properly runs the shellcheck
+    tool in all the sections of the spread tasks. Also check the
+    tool can run in a directory tree and skips checks when
+    it finds "shellcheck disable".
+
 backends: [google]
 
 systems: [ ubuntu-20.04-64 ]

diff --git a/tests/lib/external/snapd-testing-tools/tests/tests.backup/task.yaml b/tests/lib/external/snapd-testing-tools/tests/tests.backup/task.yaml
@@ -1,5 +1,9 @@
 summary: tests for the tests.backup tool
 
+details: |
+    Check the tests.backup tool properly backs up and
+    restores a specific directory.
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/tests.cleanup/task.yaml b/tests/lib/external/snapd-testing-tools/tests/tests.cleanup/task.yaml
@@ -1,5 +1,9 @@
 summary: tests for the tests.cleanup tool
 
+details: |
+    Check the tests.cleanup tool allows to defer, pop and
+    restore commands in tests.
+
 backends: [google]
 
 systems: [ubuntu-18.04-64]

diff --git a/tests/lib/external/snapd-testing-tools/tests/tests.pkgs/task.yaml b/tests/lib/external/snapd-testing-tools/tests/tests.pkgs/task.yaml
@@ -1,5 +1,10 @@
 summary: Smoke test for tests.pkgs tool
 
+details: |
+    Check the tests.pkgs tool allows managing native packages independently
+    of the system being used. Verify that packages can be installed, queried
+    and removed among other operations.
+
 backends: [google]
 
 execute: |

diff --git a/tests/lib/external/snapd-testing-tools/tests/tests.systemd/task.yaml b/tests/lib/external/snapd-testing-tools/tests/tests.systemd/task.yaml
@@ -1,5 +1,10 @@
 summary: smoke test for the tests.systemd tool
 
+details: |
+    Check the tests.systemd properly manages systemd services.
+    Verify the tool is able to create units, wait for status and
+    stop units.
+
 backends: [google]
 
 systems: [-ubuntu-14.04-64]