Skip to content

Commit

Permalink
Modified YAMLs matter 1.3 (project-chip#32116)
Browse files Browse the repository at this point in the history 
* chore(TC-DESC-2.1): modified step 1b comments as per verification step document

* chore(TL-LTIME-3.1): updated step 9 command and log as verification step

modifications as per verfication step document issue #122

* chore(TC-CNET-4.22): modifications for steps 1, 3, 5

modifications in logs for step 1, 3, 5 as verification step document
changes issue #125

* chore(TC-DRLK-2.1): modification for step 13a PICS

modification for step 13a (From DRLK.S.F0b && DRLK.S.A0015 to
DRLK.S.F0b && DRLK.S.A0016) as per test plan changes issue #128

* chore(TC-LCFG-2.1):modifications for PICS step 1a,3

In step 1a and 3 supportedLocales attribute PICS has to be updated
LCFG.S.A0001 instead LCFG.S.A0000 as per test plan issue #129

* chore(TC-APPLAUNCHER-3.x.1): modify commands

TC-APPLAUNCHER-3.7.1, TC-APPLAUNCHER-3.8.1 and TCAPPLAUNCHER-3.9.1
was updated as per verification step document (DUT as Client) issue #139

* Updates Test_TC_LWM_1_1.yaml

* Updates Test_TC_TCCM_1_1.yaml

* Updates Test_TC_LWM_1_1.yaml

* Updates  Test_TC_DISHM_1_1.yaml

* Updates Test_TC_DISHM_3_1.yaml

* Updates Test_TC_DISHM_3_3.yaml

* chore(TC-CONTENTLAUNCHER-10.4): update command and logs

command and logs has been updated as per verification step document
for all steps

* chore(TC-CONTENTLAUNCHER-10.6):update command

command has been updated as per verification step document

* Updates Test_TC_ICDM_3_4.yaml

* chore(TC-ALOGIN-12.2): update command in precondition

command has been updated in precondition as per verification
step document

* Updates Test_TC_CNET_4_9.yaml

* Updates Test_TC_CNET_4_10.yaml

* Updates Test_TC_CNET_4_11.yaml

* Adds NetworkIndex to response in CNET TCs (4.9, 4.10, 4.11)

* Updates Test_TC_TCCM_3_1.yaml

* Updates Test_TC_TCCM_3_3.yaml

* Updates Test_TC_LWM_3_1.yaml

* Updates Test_TC_LWM_3_3.yaml

* Updates Test_TC_OTCCM_1_1.yaml

* Updates Test_TC_OTCCM_2_1.yaml

* Updates Test_TC_S_3_1.yaml

* Restores TC-LWM-1.1

* Restores TC-TCCM-1.1

* Fix yaml formatting

* Fix restyle

* Fix restyle

* Fix restyle

* Removes (DEPONOFF) from pics ref

---------

Co-authored-by: raul-marquez-csa <[email protected]>
Co-authored-by: Raul Marquez <[email protected]>
  • Loading branch information
@gvargas-csa @raul-marquez-csa
3 people authored Feb 16, 2024
1 parent 5589ecb commit 1a34f77
Show file tree
Hide file tree
Showing 24 changed files with 218 additions and 113 deletions.
4 changes: 2 additions & 2 deletions src/app/tests/suites/certification/Test_TC_ALOGIN_12_2.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ tests:
Product maker needs to provide instructions for how to trigger the command on the DUT. For comparison, the DUT behavior for this test step can be simulated using chip-tool (when DUT is a commissioner) or tv-casting-app (when DUT is a commissionee):
The following command on the example tv-app will launch a content app on endpoint 4:
./chip-tool applicationlauncher launch-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tv-casting-app applicationlauncher launch-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tool applicationlauncher launch-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
./chip-tv-casting-app applicationlauncher launch-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
On TH (tv-app), Verify the launch app command is received successfully.
1666779708.066501][24994:24994] CHIP:DMG: InvokeRequestMessage =
[1666779708.066504][24994:24994] CHIP:DMG: {
Expand Down
5 changes: 2 additions & 3 deletions src/app/tests/suites/certification/Test_TC_APPLAUNCHER_3_7_1.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,9 @@ tests:
verification: |
Product maker needs to provide instructions for how to trigger the command on the DUT. For comparison, the DUT behavior for this test step can be simulated using chip-tool (when DUT is a commissioner) or tv-casting-app (when DUT is a commissionee):
./chip-tool applicationlauncher launch-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tv-casting-app applicationlauncher launch-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tool applicationlauncher launch-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
./chip-tv-casting-app applicationlauncher launch-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
On TH (tv-app), Verify the launch app command is received successfully.
Expand Down
5 changes: 3 additions & 2 deletions src/app/tests/suites/certification/Test_TC_APPLAUNCHER_3_8_1.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,12 @@ tests:
verification: |
Product maker needs to provide instructions for how to trigger the command on the DUT. For comparison, the DUT behavior for this test step can be simulated using chip-tool (when DUT is a commissioner) or tv-casting-app (when DUT is a commissionee):
./chip-tool applicationlauncher stop-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tool applicationlauncher stop-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
./chip-tv-casting-app applicationlauncher stop-app'{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tv-casting-app applicationlauncher stop-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
On TH (tv-app), Verify the stop app command is received successfully.
74562683.652082][12550:12550] CHIP:DMG: InvokeRequestMessage =
[1674562683.652085][12550:12550] CHIP:DMG: {
[1674562683.652088][12550:12550] CHIP:DMG: suppressResponse = false,
Expand Down
5 changes: 2 additions & 3 deletions src/app/tests/suites/certification/Test_TC_APPLAUNCHER_3_9_1.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,11 @@ tests:
verification: |
Product maker needs to provide instructions for how to trigger the command on the DUT. For comparison, the DUT behavior for this test step can be simulated using chip-tool (when DUT is a commissioner) or tv-casting-app (when DUT is a commissionee):
./chip-tool applicationlauncher hide-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tool applicationlauncher hide-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
./chip-tv-casting-app applicationlauncher hide-app '{"catalogVendorID": 123, "applicationID": "exampleid"}' 1 1
./chip-tv-casting-app applicationlauncher hide-app 1 1 --Application '{"catalogVendorID": 123, "applicationID": "exampleid"}'
On TH (tv-app), Verify the hide app command is received successfully.
[1674562762.318820][12550:12550] CHIP:DMG: InvokeRequestMessage =
[1674562762.318828][12550:12550] CHIP:DMG: {
[1674562762.318834][12550:12550] CHIP:DMG: suppressResponse = false,
Expand Down
14 changes: 14 additions & 0 deletions src/app/tests/suites/certification/Test_TC_CNET_4_10.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,11 @@ tests:
"Step 4: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to PIXIT.CNET.THREAD_1ST_OPERATIONALDATASET and Breadcrumb
field set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -137,6 +142,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label: "Step 5: TH reads Networks attribute from the DUT"
PICS: CNET.S.A0001
Expand Down Expand Up @@ -264,6 +271,11 @@ tests:
"Step 14: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to extended PAN ID of
PIXIT.CNET.THREAD_1ST_OPERATIONALDATASET and Breadcrumb field set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -276,6 +288,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label: "Step 15: TH sends the CommissioningComplete command to the DUT"
PICS: CNET.S.C04.Rsp
Expand Down
14 changes: 14 additions & 0 deletions src/app/tests/suites/certification/Test_TC_CNET_4_11.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,11 @@ tests:
"Step 4: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID and Breadcrumb field
set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -137,6 +142,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label:
"Step 5: TH sends AddOrUpdateWiFiNetwork command to the DUT with SSID
Expand Down Expand Up @@ -306,6 +313,11 @@ tests:
"Step 15: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID and Breadcrumb field
set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -318,6 +330,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label:
"Step 16: TH sends AddOrUpdateWiFiNetwork command to the DUT with SSID
Expand Down
131 changes: 104 additions & 27 deletions src/app/tests/suites/certification/Test_TC_CNET_4_22.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,37 @@ tests:
"Step 1: Verify each element in the ThreadScanResults list will have
the following fields:"
verification: |
Via the TH (all-clusters-app), verify:
-the PanId value is in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
./chip-tool networkcommissioning scan-networks 54 0 --Breadcrumb 1
Via the TH (chip-tool), verify:
-the NetworkingStatus value as 0 that mentions success.
-the PanId value is in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
[1658297246.000815][7679:7684] CHIP:DMG: Received Command Response Data, Endpoint=0 Cluster=0x0000_0031 Command=0x0000_0001
[1658297246.001029][7679:7684] CHIP:TOO: Endpoint: 0 Cluster: 0x0000_0031 Command 0x0000_0001
[1658297246.001190][7679:7684] CHIP:TOO: ScanNetworksResponse: {
[1658297246.001270][7679:7684] CHIP:TOO: networkingStatus: 0
[1658297246.001357][7679:7684] CHIP:TOO: threadScanResults: 1 entries
[1658297246.001485][7679:7684] CHIP:TOO: [1]: {
[1658297246.001548][7679:7684] CHIP:TOO: PanId: 64206
[1658297246.001607][7679:7684] CHIP:TOO: ExtendedPanId: 3861484836749312
[1658297246.001665][7679:7684] CHIP:TOO: NetworkName: GRL
[1658297246.001721][7679:7684] CHIP:TOO: Channel: 11
[1658297246.001777][7679:7684] CHIP:TOO: Version: 4
[1658297246.001841][7679:7684] CHIP:TOO: ExtendedAddress: 166E0A0000000001
[1658297246.001900][7679:7684] CHIP:TOO: Rssi: -81
[1658297246.001955][7679:7684] CHIP:TOO: Lqi: 48
[1658297246.002010][7679:7684] CHIP:TOO: }
[1658297246.002070][7679:7684] CHIP:TOO: }
[1658297246.002181][7679:7684] CHIP:DMG: ICR moving to [AwaitingDe]
[1658297246.002298][7679:7684] CHIP:EM: Sending Standalone Ack for MessageCounter:124278362 on exchange 24525i
cluster: "LogCommands"
command: "UserPrompt"
PICS: PICS_USER_PROMPT && CNET.S.C00.Rsp && CNET.S.C01.Tx
Expand Down Expand Up @@ -144,15 +166,47 @@ tests:
"Step 3: Verify each element in the ThreadScanResults list will have
the following fields:"
verification: |
Via the TH (all-clusters-app), verify:
-the PanId value is in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
./chip-tool networkcommissioning scan-networks 54 0 --Breadcrumb 2
Via the TH (chip-tool), verify:
-the NetworkingStatus value as 0 that mentions success.
-the PanId value is in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
[1658297283.141075][7695:7700] CHIP:DMG: Received Command Response Data, Endpoint=0 Cluster=0x0000_0031 Command=0x0000_0001
[1658297283.141183][7695:7700] CHIP:TOO: Endpoint: 0 Cluster: 0x0000_0031 Command 0x0000_0001
[1658297283.141356][7695:7700] CHIP:TOO: ScanNetworksResponse: {
[1658297283.141438][7695:7700] CHIP:TOO: networkingStatus: 0
[1658297283.141534][7695:7700] CHIP:TOO: threadScanResults: 2 entries
[1658297283.141663][7695:7700] CHIP:TOO: [1]: {
[1658297283.141729][7695:7700] CHIP:TOO: PanId: 64206
[1658297283.141788][7695:7700] CHIP:TOO: ExtendedPanId: 3861484836749312
[1658297283.141847][7695:7700] CHIP:TOO: NetworkName: GRL
[1658297283.141904][7695:7700] CHIP:TOO: Channel: 11
[1658297283.141961][7695:7700] CHIP:TOO: Version: 4
[1658297283.142023][7695:7700] CHIP:TOO: ExtendedAddress: 166E0A0000000001
[1658297283.142079][7695:7700] CHIP:TOO: Rssi: -80
[1658297283.142134][7695:7700] CHIP:TOO: Lqi: 52
[1658297283.142188][7695:7700] CHIP:TOO: }
[1658297283.142271][7695:7700] CHIP:TOO: [2]: {
[1658297283.142331][7695:7700] CHIP:TOO: PanId: 64206
[1658297283.142386][7695:7700] CHIP:TOO: ExtendedPanId: 3861484836749312
[1658297283.142443][7695:7700] CHIP:TOO: NetworkName: GRL
[1658297283.142500][7695:7700] CHIP:TOO: Channel: 11
[1658297283.142551][7695:7700] CHIP:TOO: Version: 4
[1658297283.142609][7695:7700] CHIP:TOO: ExtendedAddress: 166E0A0000000001
[1658297283.142665][7695:7700] CHIP:TOO: Rssi: -83
[1658297283.142718][7695:7700] CHIP:TOO: Lqi: 44
[1658297283.142771][7695:7700] CHIP:TOO: }
[1658297283.142828][7695:7700] CHIP:TOO: }
[1658297283.142958][7695:7700] CHIP:DMG: ICR moving to [AwaitingDe]
[1658297283.143074][7695:7700] CHIP:EM: Sending Standalone Ack for MessageCounter:115607423 on exchange 55136i
cluster: "LogCommands"
command: "UserPrompt"
PICS: PICS_USER_PROMPT && CNET.S.C00.Rsp && CNET.S.C01.Tx
Expand Down Expand Up @@ -201,15 +255,38 @@ tests:
"Step 5: Verify each element in the ThreadScanResults list will have
the following fields:"
verification: |
Via the TH (all-clusters-app), verify:
-the PanId value is in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
./chip-tool networkcommissioning scan-networks 54 0 --Breadcrumb 3
Via the TH (chip-tool), verify:
-the NetworkingStatus value should be as 0 that mentions success.
-the PanId value should be with in the range of 0 to 65534.
-the ExtendedPanId value is type of uint64.
-the NetworkName value is type of string with a size of 1 to 16 bytes.
-the Channel is type of uint16 with range 0 to 65535.
-the version value is type of uint8(0 to 255).
-the ExtendedAddress is a hwaddr with a size of 8 bytes
-the RSSI is of type int8 with a range of -120 to 0.
-the LQI is type of uint8(0 to 255).
[1658297312.201709][7708:7713] CHIP:DMG: },
[1658297312.201898][7708:7713] CHIP:DMG: Received Command Response Data, Endpoint=0 Cluster=0x0000_0031 Command=0x0000_0001
[1658297312.202021][7708:7713] CHIP:TOO: Endpoint: 0 Cluster: 0x0000_0031 Command 0x0000_0001
[1658297312.202196][7708:7713] CHIP:TOO: ScanNetworksResponse: {
[1658297312.202276][7708:7713] CHIP:TOO: networkingStatus: 0
[1658297312.202359][7708:7713] CHIP:TOO: threadScanResults: 1 entries
[1658297312.202481][7708:7713] CHIP:TOO: [1]: {
[1658297312.202544][7708:7713] CHIP:TOO: PanId: 64206
[1658297312.202604][7708:7713] CHIP:TOO: ExtendedPanId: 3861484836749312
[1658297312.202663][7708:7713] CHIP:TOO: NetworkName: GRL
[1658297312.202718][7708:7713] CHIP:TOO: Channel: 11
[1658297312.202775][7708:7713] CHIP:TOO: Version: 4
[1658297312.202838][7708:7713] CHIP:TOO: ExtendedAddress: 166E0A0000000001
[1658297312.202897][7708:7713] CHIP:TOO: Rssi: -78
[1658297312.202952][7708:7713] CHIP:TOO: Lqi: 60
[1658297312.203007][7708:7713] CHIP:TOO: }
[1658297312.203066][7708:7713] CHIP:TOO: }
[1658297312.203176][7708:7713] CHIP:DMG: ICR moving to [AwaitingDe]
[1658297312.203284][7708:7713] CHIP:EM: Sending Standalone Ack for MessageCounter:217378941 on exchange 37389i
cluster: "LogCommands"
command: "UserPrompt"
PICS: PICS_USER_PROMPT && CNET.S.C00.Rsp && CNET.S.C01.Tx
Expand Down
14 changes: 14 additions & 0 deletions src/app/tests/suites/certification/Test_TC_CNET_4_9.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,11 @@ tests:
"Step 4: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID and Breadcrumb field
set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -131,6 +136,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label: "Step 5: TH reads Networks attribute from the DUT"
PICS: CNET.S.A0001
Expand Down Expand Up @@ -266,6 +273,11 @@ tests:
"Step 14: TH sends RemoveNetwork Command to the DUT with NetworkID
field set to PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID and Breadcrumb field
set to 1"
verification: |
Verify that DUT sends NetworkConfigResponse to command with the following fields:
NetworkingStatus is success
NetworkIndex is 'Userwifi_netidx'
PICS: CNET.S.C04.Rsp && CNET.S.C05.Tx
command: "RemoveNetwork"
arguments:
Expand All @@ -278,6 +290,8 @@ tests:
values:
- name: "NetworkingStatus"
value: 0
- name: "NetworkIndex"
value: 0

- label: "Step 15: TH sends the CommissioningComplete command to the DUT"
PICS: CNET.S.C04.Rsp
Expand Down
4 changes: 2 additions & 2 deletions src/app/tests/suites/certification/Test_TC_CONTENTLAUNCHER_10_6.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ tests:
verification: |
Product maker needs to provide instructions for how to trigger the command on the DUT. For comparison, the DUT behavior for this test step can be simulated using chip-tool (when DUT is a commissioner) or tv-casting-app (when DUT is a commissionee):
./chip-tool contentlauncher launch-url 'https://www.foo.com' '{"providerName": "provider"}' 1 1
./chip-tv-casting-app contentlauncher launch-url 'https://www.foo.com' '{"providerName": "provider"}' 1 1
./chip-tool contentlauncher launch-url 'https://www.foo.com' 1 1 --DisplayString '{"providerName": "provider"}'
./chip-tv-casting-app contentlauncher launch-url 'https://www.foo.com' 1 1 --DisplayString '{"providerName": "provider"}'
On TH (tv-app), Verify the launch url command is received successfully.
[1658531502040] [21924:325733] CHIP: [DMG] InvokeRequestMessage =
Expand Down
Loading

0 comments on commit 1a34f77

Please sign in to comment.