Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed tests from type: to testtype: / some readme updates #2

Merged
merged 1 commit into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 24 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,28 @@
# testsuite
# Test Suite

testsuite for homie client libraries
Test suite for Homie client libraries.

this is under early development
This project is in early development.

## Purpose

This test suite is intended for coders who write homie compatible software to quickly get a solid set of tests to run against their implementations.

The main goal being consistent behaviour across clients.
This test suite is designed for developers writing Homie-compatible software, providing a comprehensive set of tests to validate their implementations. The primary goal is to ensure consistent behavior across different clients.

## Format

The testsuite provided multiple YAML testset files containing test definitions.
The data format of these files is described in the following chapters.
The test suite provides multiple YAML test set files, each containing test definitions. The format and structure of these files are explained in the following sections.

### Testset
### Test Set

A YAML file containing a set of tests to be run. A testset contains the following fields:
A YAML file that contains a group of tests to be executed. A test set includes the following fields:

- **`description`** (required): A string providing a brief explanation of the testset and its contained tests.
- **`tests`** (required): A list of `test` definitions (see next chapter)
- **`description`** (required): A brief explanation of the test set and the tests it contains.
- **`tests`** (required): A list of `test` definitions (see the next section).

#### Example:

```yaml
description: validating boolean values
description: Validating boolean values

tests:
- ...test definition ...
Expand All @@ -35,29 +32,29 @@ tests:

### Test

Defines a specific test to be executed. A test consist of the following fields:
A test defines a specific scenario to be executed and validated. A test consists of the following fields:

- **`description`** (required): A string providing a brief explanation of the test case or the scenario it represents.
- **`testtype`** (required): A string indicating the type of the property value or data being tested. Valid values are:
- propertyformat
- propertyvalueinteger
- propertyvalue
- homieid
- **`description`** (required): A brief explanation of the test case or the scenario it represents.
- **`testtype`** (required): A string that indicates the type of the property value or data being tested. Valid values include:
- `propertyformat`
- `propertyvalueinteger`
- `propertyvalue`
- `homieid`
- ...
- **`definition`**(optional depending on `type`): The test definition (this can for example be a property, node or device descriptions).
- **`input_data`**(optional depending on `type`): The input data provided for the test, which may be represented as a string or other formats, depending on the `type`.
- **`output_data`**(optional depending on `type`): The expected result after processing the `input_data`. This field is typically represented in the correct data type (e.g., `integer` for an integer test).
- **`valid`**(required): A boolean indicating whether the defined test is supposed to pass (true) or fail (false)
- **`definition`** (optional, depending on `testtype`): The test definition, which could include properties, nodes, or device descriptions.
- **`input_data`** (optional, depending on `testtype`): The input data provided for the test, which may be represented as a string or other formats, depending on the `testtype`.
- **`output_data`** (optional, depending on `testtype`): The expected result after processing the `input_data`, typically represented in the appropriate data type (e.g., `integer` for an integer test).
- **`valid`** (required): A boolean indicating whether the test is expected to pass (`true`) or fail (`false`).

#### Example
#### Example:

Here’s an example of how the data looks in practice:
Here’s an example of how the test data looks in practice:

```yaml
description: Normal integer value without format works
testtype: propertyvalueinteger
definition:
datatype: integer
datatype: integer
input_data: "12"
output_data: 12
valid: true
Expand Down
24 changes: 12 additions & 12 deletions homie5/formats/boolean.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,73 +2,73 @@ description: validating boolean formats

tests:
- description: proper format is accepted
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "off,on"
valid: true
- description: reverse format is accepted
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "on,off"
valid: true
- description: format is optional
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
valid: true
- description: empty strings are not allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "off,"
valid: false
- description: empty strings are not allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: ",on"
valid: false
- description: space is allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "off, "
valid: true
- description: space is allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: " ,on"
valid: true
- description: single entry is not allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "on"
valid: false
- description: more than 2 entries are not allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "on,off,between"
valid: false
- description: duplicate entries are not allowed
type: propertydescription
testtype: propertydescription
definition:
# this is not in the spec but makes sense?
datatype: boolean
format: "on,on"
valid: false
- description: anything allowed, as long a comma separated
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "@**&#%&@&, 5454 "
valid: true
- description: utf-8 characters are allowed
type: propertydescription
testtype: propertydescription
definition:
datatype: boolean
format: "👍,👎"
Expand Down
30 changes: 15 additions & 15 deletions homie5/values/boolean.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,96 +2,96 @@ description: validating boolean values

tests:
- description: Passes valid boolean values
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "true"
valid: true
- description: Passes valid boolean values
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "false"
valid: true
- description: an empty string is not valid
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "\x00"
valid: false
- description: uppercase true is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "TRUE"
valid: false
- description: uppercase false is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "FALSE"
valid: false
- description: capitalcase is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "True"
valid: false
- description: capitalcase is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "False"
valid: false
- description: shorthand is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: t
valid: false
- description: shorthand is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: f
valid: false
- description: numeric string is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "1"
valid: false
- description: numeric string is not allowed
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
input_data: "0"
valid: false

# validating values with format
- description: with format booleans are accepted
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
format: "on,off"
input_data: "true"
valid: true
- description: with format booleans are accepted
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
format: "on,off"
input_data: "false"
valid: true
- description: with format labels are not accepted
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
format: "on,off"
input_data: "on"
valid: false
- description: with format labels are not accepted
type: propertyvalue
testtype: propertyvalue
definition:
datatype: boolean
format: "on,off"
Expand Down
Loading