Skip to content

Commit

Permalink
Merge pull request #1601 from telefonicaid/task/add_metdadata_measures
Browse files Browse the repository at this point in the history 
add metadata in attributes and static_attributes test cases and process metadata jexl expressions
  • Loading branch information
@fgalan
fgalan authored Apr 18, 2024
2 parents 73dbc7e + 20196ff commit 9006856
Show file tree
Hide file tree
Showing 4 changed files with 292 additions and 8 deletions.
1 change: 1 addition & 0 deletions CHANGES_NEXT_RELEASE
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
- Add: process JEXL expressions in metadata attributes (#1601)
- Fix: TimeInstant mapped from attribute overrides default behaviours (#1557)
- Fix: reduce information showed handling errors to just config flags (#1594)
- Upgrade pymongo dep from 4.3.3 to 4.6.3
Expand Down
57 changes: 49 additions & 8 deletions doc/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,43 @@ e.g.:
}
```

Metadata could also has `expression` like attributes in order to expand it:

e.g.:

```json
{
"entity_type": "Lamp",
"resource": "/iot/d",
"protocol": "PDI-IoTA-UltraLight",
..etc
"commands": [
{"name": "on","type": "command"},
{"name": "off","type": "command"}
],
"attributes": [
{"object_id": "s", "name": "state", "type":"Text"},
{"object_id": "l", "name": "luminosity", "type":"Integer",
"metadata":{
"unitCode":{"type": "Text", "value" :"CAL"}
}
}
],
"static_attributes": [
{"name": "category", "type":"Text", "value": ["actuator","sensor"]},
{"name": "controlledProperty", "type": "Text", "value": ["light"],
"metadata":{
"includes":{"type": "Text",
"value" :["state", "luminosity"],
"expression": "level / 100"
},
"alias":{"type": "Text", "value" :"lamp"}
}
},
]
}
```

### NGSI-LD data and metadata considerations

When provisioning devices for an NGSI-LD Context Broker, `type` values should typically correspond to one of the
Expand Down Expand Up @@ -525,8 +562,8 @@ expression. In all cases the following data is available to all expressions:
- `subservice`: device subservice
- `staticAttributes`: static attributes defined in the device or config group

Additionally, for attribute expressions (`expression`, `entity_name`) and `entityNameExp` measures are avaiable in the
**context** used to evaluate them.
Additionally, for attribute expressions (`expression`, `entity_name`), `entityNameExp` and metadata expressions
(`expression`) measures are available in the **context** used to evaluate them.

### Examples of JEXL expressions

Expand Down Expand Up @@ -974,9 +1011,9 @@ Will now generate the following NGSI v2 payload:

Timestamp processing done by IOTA is as follows:

* An attribute `TimeInstant` is added to updated entities
* In the case of NGSI-v2, a `TimeInstant` metadata is added in each updated attribute. With NGSI-LD, the Standard
`observedAt` property-of-a-property is used instead.
- An attribute `TimeInstant` is added to updated entities
- In the case of NGSI-v2, a `TimeInstant` metadata is added in each updated attribute. With NGSI-LD, the Standard
`observedAt` property-of-a-property is used instead.

Depending on the `timestamp` configuration and if the measure contains a value named `TimeInstant` with a correct value,
the IoTA behaviour is described in the following table:
Expand All @@ -999,9 +1036,13 @@ The `timestamp` conf value used is:

Some additional considerations to take into account:

* If there is an attribute which maps a measure to `TimeInstant` attribute (after [expression evaluation](#expression-language-support) if any is defined), then that value will be used as `TimeInstant,
overwriting the above rules specified in "Behaviour" column. Note that an expression in the could be used in that mapping.
* If the resulting `TimeInstant` not follows [ISO_8601](https://en.wikipedia.org/wiki/ISO_8601) (either from a direct measure of after a mapping, as described in the previous bullet) then it is refused (so a failover to server timestamp will take place).
- If there is an attribute which maps a measure to `TimeInstant` attribute (after
[expression evaluation](#expression-language-support) if any is defined), then that value will be used as
`TimeInstant, overwriting the above rules specified in "Behaviour" column. Note that an expression in the could be
used in that mapping.
- If the resulting `TimeInstant` not follows [ISO_8601](https://en.wikipedia.org/wiki/ISO_8601) (either from a direct
measure of after a mapping, as described in the previous bullet) then it is refused (so a failover to server
timestamp will take place).

## Overriding global Context Broker host

Expand Down
33 changes: 33 additions & 0 deletions lib/services/ngsi/entities-NGSI-v2.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,7 @@ function sendUpdateValueNgsi2(entityName, measures, typeInformation, token, call

currentAttr.hitted = hitted;
currentAttr.value = valueExpression;

//store de New Attributte in entity data structure
if (hitted === true) {
if (entities[attrEntityName] === undefined) {
Expand All @@ -464,6 +465,38 @@ function sendUpdateValueNgsi2(entityName, measures, typeInformation, token, call

//RE-Populate de JEXLcontext (except for null or NaN we preffer undefined)
jexlctxt[currentAttr.name] = valueExpression;

// Expand metadata value expression
if (currentAttr.metadata) {
for (var metaKey in currentAttr.metadata) {
if (currentAttr.metadata[metaKey].expression && metaKey !== constants.TIMESTAMP_ATTRIBUTE) {
let newAttrMeta = {};
if (currentAttr.metadata[metaKey].type) {
newAttrMeta['type'] = currentAttr.metadata[metaKey].type;
}
let metaValueExpression;
try {
metaValueExpression = jexlParser.applyExpression(
currentAttr.metadata[metaKey].expression,
jexlctxt,
typeInformation
);
//we fallback to null if anything unexpecte happend
if (
metaValueExpression === null ||
metaValueExpression === undefined ||
Number.isNaN(metaValueExpression)
) {
metaValueExpression = null;
}
} catch (e) {
metaValueExpression = null;
}
newAttrMeta['value'] = metaValueExpression;
currentAttr.metadata[metaKey] = newAttrMeta;
}
}
}
}

//now we can compute explicit (Bool or Array) with the complete JexlContext
Expand Down
209 changes: 209 additions & 0 deletions test/functional/testCases.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,74 @@ const testCases = [
}
]
},
{
describeName: '0021 Simple group with active attributes with metadata',
provision: {
url: 'http://localhost:' + config.iota.server.port + '/iot/services',
method: 'POST',
json: {
services: [
{
resource: '/iot/json',
apikey: globalEnv.apikey,
entity_type: globalEnv.entity_type,
commands: [],
lazy: [],
attributes: [
{
object_id: 'a',
name: 'attr_a',
type: 'Boolean',
metadata: {
accuracy: {
value: 0.8,
type: 'Float'
}
}
}
],
static_attributes: []
}
]
},
headers: {
'fiware-service': globalEnv.service,
'fiware-servicepath': globalEnv.servicePath
}
},
should: [
{
shouldName:
'A - WHEN sending defined object_ids (measures) through http IT should send measures to Context Broker preserving value types, name mappings and metadatas',
type: 'single',
measure: {
url: 'http://localhost:' + config.http.port + '/iot/json',
method: 'POST',
qs: {
i: globalEnv.deviceId,
k: globalEnv.apikey
},
json: {
a: false
}
},
expectation: {
id: globalEnv.entity_name,
type: globalEnv.entity_type,
attr_a: {
value: false,
type: 'Boolean',
metadata: {
accuracy: {
value: 0.8,
type: 'Float'
}
}
}
}
}
]
},
// 0100 - JEXL TESTS
{
describeName: '0100 - Simple group with active attribute + JEXL expression boolean (!)',
Expand Down Expand Up @@ -1477,6 +1545,75 @@ const testCases = [
}
]
},
{
describeName: '0191 - Simple group with JEXL expression in metadata',
provision: {
url: 'http://localhost:' + config.iota.server.port + '/iot/services',
method: 'POST',
json: {
services: [
{
resource: '/iot/json',
apikey: globalEnv.apikey,
entity_type: globalEnv.entity_type,
commands: [],
lazy: [],
attributes: [
{
object_id: 'a',
name: 'attr_a',
type: 'Boolean',
expression: 'a?threshold[90|tostring].max:true',
metadata: {
unit: {
type: 'Text',
expression: '"hola" + "adios"'
}
}
}
]
}
]
},
headers: {
'fiware-service': globalEnv.service,
'fiware-servicepath': globalEnv.servicePath
}
},
should: [
{
shouldName:
'A - WHEN sending a measure through http IT should send to Context Broker expanded metadata value ',
type: 'single',
isRegex: true,
measure: {
url: 'http://localhost:' + config.http.port + '/iot/json',
method: 'POST',
qs: {
i: globalEnv.deviceId,
k: globalEnv.apikey
},
json: {
a: false
}
},
expectation: {
id: globalEnv.entity_name,
type: globalEnv.entity_type,
attr_a: {
value: true,
type: 'Boolean',
metadata: {
unit: {
type: 'Text',
value: 'holaadios'
}
}
}
}
}
]
},
// 0200 - COMMANDS TESTS
{
describeName: '0200 - Simple group with commands',
Expand Down Expand Up @@ -1705,6 +1842,78 @@ const testCases = [
}
]
},
{
describeName: '0320 - Simple group with active attributes + metadata in Static attributes ',
provision: {
url: 'http://localhost:' + config.iota.server.port + '/iot/services',
method: 'POST',
json: {
services: [
{
resource: '/iot/json',
apikey: globalEnv.apikey,
entity_type: globalEnv.entity_type,
commands: [],
lazy: [],
attributes: [],
static_attributes: [
{
name: 'static_a',
type: 'Number',
value: 4,
metadata: {
accuracy: {
value: 0.8,
type: 'Float'
}
}
}
]
}
]
},
headers: {
'fiware-service': globalEnv.service,
'fiware-servicepath': globalEnv.servicePath
}
},
should: [
{
shouldName:
'A - WHEN sending measures through http IT should store metatada static attributes into Context Broker',
type: 'single',
measure: {
url: 'http://localhost:' + config.http.port + '/iot/json',
method: 'POST',
qs: {
i: globalEnv.deviceId,
k: globalEnv.apikey
},
json: {
a: 10
}
},
expectation: {
id: globalEnv.entity_name,
type: globalEnv.entity_type,
a: {
value: 10,
type: 'Text'
},
static_a: {
value: 4,
type: 'Number',
metadata: {
accuracy: {
value: 0.8,
type: 'Float'
}
}
}
}
}
]
},
// 0400 - TIMESTAMP TESTS
{
describeName: '0400 - Simple group with active attribute + timestamp:false',
Expand Down

0 comments on commit 9006856

Please sign in to comment.