Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document payload normalization #939

Merged
merged 7 commits into from
Aug 29, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions doc/content/integrations/payload-formatters/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,29 +9,29 @@ This section explains how to set up payload formatters for a specific end device

<!--more-->

In addition to the written instructions linked below, a video with instructions for creating Javascript and device repository payload formatters is available on [The Things Network youtube channel](https://youtu.be/4tii7MiD-yM).
In addition to the written instructions linked below, a video with instructions for creating JavaScript and device repository payload formatters is available on [The Things Network YouTube channel](https://youtu.be/4tii7MiD-yM).

<details><summary>Show video</summary>
{{< youtube "4tii7MiD-yM" >}}
</details>

## Types of Payload Formatters

{{% tts %}} supports three types of payload formatters: Javascript, CayenneLPP, and Repository payload formatters.
{{% tts %}} supports three types of payload formatters: JavaScript, CayenneLPP, and Repository payload formatters.

### Javascript
### JavaScript

{{% tts %}} allows you to write your own custom payload formatters in Javascript. To find out how to write a custom Javascript payload formatter, see the [Javascript Payload Formatters]({{< ref "/integrations/payload-formatters/javascript" >}}) section.
{{% tts %}} allows you to write your own custom payload formatters in JavaScript. To find out how to write a custom JavaScript payload formatter, see the [JavaScript Payload Formatters]({{< ref "/integrations/payload-formatters/javascript" >}}) section.

### CayeneLPP

{{% tts %}} can automatically decode CayenneLPP formatted payloads, no custom code neccessary. To find out how to use CayenneLPP payload formatters, see the [CayenneLPP]({{< relref "cayenne" >}}) section.

### Repository

Device manufacturers may submit payload formatters designed to work with their devices. These are publicly available in {{% tts %}} [Device Repository](https://github.com/TheThingsNetwork/lorawan-devices/tree/master).
Device manufacturers may submit payload formatters designed to work with their devices. These are publicly available in {{% tts %}} [Device Repository](https://github.com/TheThingsNetwork/lorawan-devices).

Device repository payload formatters work right out of the box - no customization needed!
Device repository payload formatters work right out of the box no customization needed!

## Application and Device Specific Payload Formatters

Expand Down
4 changes: 2 additions & 2 deletions doc/content/integrations/payload-formatters/device-repo.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ title: "Device Repository"
description: ""
---

Device manufacturers may submit payload formatters designed to work with their devices. The source code for these is publicly available in the [lorawan-devices Github Repository](https://github.com/TheThingsNetwork/lorawan-devices/tree/master) and the devices are on display in The Things Industries [Device Repository](https://www.thethingsnetwork.org/device-repository/).
Device manufacturers may submit payload formatters designed to work with their devices. The data and code are publicly available in the [Github Repository](https://github.com/TheThingsNetwork/lorawan-devices). You can find all devices in the Device Repository on [this page](https://www.thethingsnetwork.org/device-repository/).

If you're a device manufacturer and you'd like to add your device to the open source device repo, find out how [here](https://github.com/TheThingsNetwork/lorawan-devices/blob/master/README.md). Adding your device makes it easy for users to use it with {{% tts %}}, and is a good way to publicly document information and code related to your device.
If you're a device manufacturer and you would like to add your device to the repository, find out how [here](https://github.com/TheThingsNetwork/lorawan-devices/blob/master/README.md). Adding your device makes it easy for users to use it with {{% tts %}}, and is a good way to publicly document information and code related to your device.
22 changes: 11 additions & 11 deletions doc/content/integrations/payload-formatters/javascript/_index.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
---
title: "Javascript"
title: "JavaScript"
description: ""
alias: "/integrations/payload-formatters/javascript"
---

Javascript payload formatters allow you to write your own functions to encode or decode messages. Javascript functions are executed using an [JavaScript ECMAScript 5.1](https://www.ecma-international.org/ecma-262/5.1/) engine.
JavaScript payload formatters allow you to write functions to encode or decode messages. JavaScript functions are executed using an [JavaScript ECMAScript 5.1](https://www.ecma-international.org/ecma-262/5.1/) runtime.

<!--more-->

Tips:

- The payload formatters should be simple and lightweight.
- Use arithmetic operations and bit shifts to convert binary data to fields.
- Avoid using non-trivial logic or polyfills.

{{< note >}} Payload formatters use ECMAScript 5 (2009), which has some distinct differences compared to newer, commonly used ECMAScript revisions. See [here](https://www.javatpoint.com/es5-vs-es6) for a quick comparison. Notably, `let`, `const`, and arrow functions are not supported by ES5. {{</ note >}}

{{< note >}} For security, the runtime does not support modules, `require` syntax, or any input/output other than defined below. {{</ note >}}
{{< note >}}
Payload formatters use ECMAScript 5.1 (2009), which has some distinct differences compared to newer, commonly used ECMAScript revisions. See [here](https://www.javatpoint.com/es5-vs-es6) for a quick comparison.

{{< warning >}} The maximum size of a user-defined Javascript payload formatter is 40KB (40960 characters), unless the source of the payload formatter is [Device Repository]({{< ref "/integrations/payload-formatters/device-repo" >}}). {{</ warning>}}
For security, the runtime does not support modules (`require` syntax) or any network or file system access.
{{</ note >}}

There are three different types of {{% tts %}} JavaScript payload formatters:
{{< warning >}} The maximum size of a JavaScript payload formatter is 40KB (40,960 bytes). Payload formatters loaded from the [Device Repository]({{< ref "/integrations/payload-formatters/device-repo" >}}) can be larger. {{</ warning>}}

- [Uplink Decoder]({{< ref "/integrations/payload-formatters/javascript/uplink-decoder" >}})
- [Downlink Encoder]({{< ref "/integrations/payload-formatters/javascript/downlink-encoder" >}})
- [Downlink Decoder]({{< ref "/integrations/payload-formatters/javascript/downlink-decoder" >}})
Learn more about payload formatters and examples:

Read the documentation below to further learn about these formatters and find associated examples.
- [Uplink]({{< relref "uplink" >}})
- [Downlink]({{< relref "downlink" >}})

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
title: "Downlink"
description: ""
weight: 2
aliases:
- "/integrations/payload-formatters/javascript/downlink-encoder"
- "/integrations/payload-formatters/javascript/downlink-decoder"
---

The `encodeDownlink()` function is called when a downlink message with decoded payload is scheduled to be sent to the end device. The `encodeDownlink()` function encodes the JSON object of the downlink message to binary payload for transmission to the end device.

The `decodeDownlink()` function does the opposite of `encodeDownlink()`: it decodes the binary payload back to decoded payload.

<!--more-->

The functions take an input object and return an output object:

```js
function encodeDownlink(input) {
// input has the following structure:
// {
// field: "value"
// }
return {
bytes: [1, 2, 3], // FRMPayload (byte array)
fPort: 1,
warnings: ["warning 1", "warning 2"], // optional
errors: ["error 1", "error 2"] // optional (if set, the encoding failed)
}
}

function decodeDownlink(input) {
// input has the following structure:
// {
// bytes: [1, 2, 3], // FRMPayload (byte array)
// fPort: 1
// }
return {
data: {
field: "value"
},
warnings: ["warning 1", "warning 2"], // optional
errors: ["error 1", "error 2"] // optional (if set, the decoding failed)
}
}
```

Encoded and decoded downlink payload are incorporated in the [`as.down.data.receive`]({{< ref "/reference/api/events#event:as.down.data.receive" >}}) event, where the `data` object contains:

```json
{
"f_port": 1,
"frm_payload": "AQID",
"decoded_payload": {
"field": "value"
}
}
```

The encoded payload is in `frm_payload` (Base64 encoded), while the decoded payload is in `decoded_payload`.

If an error is present in `errors`, the payload is invalid and the message will be dropped. Any warnings in `warnings` are informative.

{{< note >}} You can test your downlink encoder and decoder as well as schedule downlinks from {{% tts %}} Console. {{</ note >}}

## Downlink Example: The Things Node

Here is an example of an encoder function that uses the `color` field to send a byte on a specific FPort:

```js
var colors = ["red", "green", "blue"];

function encodeDownlink(input) {
return {
bytes: [colors.indexOf(input.data.color)],
fPort: 4,
};
}

function decodeDownlink(input) {
switch (input.fPort) {
case 4:
return {
data: {
color: colors[input.bytes[0]]
}
}
default:
return {
errors: ["unknown FPort"]
}
}
}
```

If the downlink with the following JSON payload is sent by the application:

```json
{
"color": "green"
}
```

The output of the `encodeDownlink()` function will have the following structure:

```json
{
"bytes": [1],
"fPort": 4
}
```

{{< figure src="downlink-encoder.png" alt="Testing a downlink encoder" >}}

The `data` object of the `as.down.data.receive` event will therefore contain:

```json
{
"f_port": 4,
"frm_payload": "AQ==",
"decoded_payload": {
"color": "green"
}
}
```
Loading