Docs

Integration templates

To make integrating with Blockbax as easy as possible, we provide integration templates that can integrate third-party platforms and devices with a few clicks.

Using integration templates

Prerequisites

  • Blockbax account
  • A device for which an integration template is available

Steps to configure

  1. Open the Blockbax web app and go to ‘Settings’ -> ‘Inbound connectors’.
  2. Press the ‘+’ icon or ‘Create inbound connector’ button.
  3. Select the template you wish to use; available templates can be found here.
  4. (Optional) If you wish to change the payload converter and/or protocol click on Show advanced settings.
  5. (Optional) In the advanced settings, you can click on convert it to a custom conversion to change the payload converter.
  6. (Optional) Select Automatically create subjects to create subjects based on the ingestion ID.
  7. Press ‘Save changes’.
  8. Depending on the protocol you have selected, you can use the url or topic in the inbound connector settings to configure the third-party platform or device to forward payloads to.

Advanced tips

At times, even when using a template, it may not entirely address the final steps required to integrate your data seamlessly into the Blockbax platform. In such cases, a bit of additional configuration remains necessary. These advanced tips can assist you in successfully accomplishing this task.

Uncovering ingestion IDs

Sometimes, it’s challenging to determine the nature of the messages being sent and how you need to set up a subject type with the correct metric external IDs. Depending on the template you’ve chosen to utilize, there are two methods for resolving this problem.

Use the inbound connector execution logs

Take the following steps:

  1. Make sure at least one payload has been sent to the inbound connector.

  2. Open the inbound connector you created in the previous step.

  3. In the ‘Executions’ section expand the latest execution.

    As an example we use The Things Network integration template with the Oyster 2 of Digital Matter, but you should see something similar in the execution logs below.

    Example execution logs from an Oyster device from Digital Matter

    In Blockbax ingestion IDs are used to map data to your subjects and metrics. By default ingestion IDs are derived from the subjects’ external IDs and metrics’ external IDs (e.g. subjectExternalId$metricExternalId) but you can also override these with custom ones. The inbound connector script configured combines the device ID from The Things Stack and the property name returned by the payload formatter(s) into an ingestion ID (e.g. example-device$type). This allows for easy setup if you make sure your subject external IDs and metric external IDs match the ones you configure in Blockbax.

Log the payload

Some templates cannot fully parse the incoming payload because the format is not standardized or is susceptible to customizable fields. To fix these issues you will need to know how the payload is presented when it is received and how to parse it. The quickest way to check what the payload looks like is by logging it inside the inbound connector just by adding one line. If you have selected a template you can click on Show advanced settings and then click on convert it to a custom conversion and then add the following line:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

/**
 * @param {JsonPayload} payload
 * @param {Context} context
 */
function convertPayload(payload, context) {
    // this line right here
    context.logInfo(JSON.stringify(payload));

    // rest of code
}

Now every time a message is received the payload is logged and you will be able to see what is in the data.

Creating or updating integration templates

If you are want to provide a quick and easy way for your devices or your backend to integrate with Blockbax, you can create a template using this guide.

Prerequisites

  • Github account
  • node
  • npm

Steps

  1. Go to our integration templates repository.
  2. Fork and clone the integration templates repository.
  3. Add or update your integration template.
  4. Configure the test suite to test the integration template’s payload conversion script.
  5. In your forked repository open a merge request selecting the main branch of the integration templates repository as its target branch, we will check out your request and notify you when further changes are needed.

Before your templates are available within Blockbax our CI pipeline will validate that the config.yml, index.yml and test.yml files are correct and test the payload conversion script with tests defined in the tests.yml file. If all checks succeed, it will then attempt to publish the version of your template to Blockbax.

Add a new template
  1. If you do not yet have a group in the groups/<group-id>/ directory add a new directory with your group id, a group id can only contain letters, numbers and the ‘-’ character and cannot be longer than 20 characters.
  2. Configure the group index.
  3. To add a template add a groups/<group-id>/<template-id>/ directory with your template id, a template id can only contain letters, numbers and the ‘-’ character and cannot be longer than 20 characters.
  4. Configure the template.
  5. Add a test suite.
Update a template

To update a template you will need to do the following:

  1. Make the desired changes to the payload conversion script located at groups/<group-id>/template-id>/script.js and/or change the protocol and/or payloadFormat fields in the config.yml file.
  2. Add a test case or update the current test cases to illustrate what this change does. The test suite is located at groups/<group-id>/template-id>/tests.yml
  3. Bump the version number of the template inside the config.yml. This config file is located at groups/<group-id>/template-id>/config.yml.

Only if changes in the payload conversion script are detected or if the protocol and/or payloadFormat fields in the config.yml file are changed a new version will be accepted and a new version number is required. This new version number must be exactly one higher than the latest version.

Configure the group

A group index is located at groups/<group-id>/index.yml and should look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Group display name (max 50 characters)
name: Your company
# Group description (max 150 characters)
description: Example for your company or group
# Group contact information and socials (optional)
contact:
    email: your@company.com
    website: https://www.your-company.com
    linkedin: https://www.linkedin.com/company/company-x/
    facebook: https://www.facebook.com/company-x
    twitter: company-x
    instagram: company-x
    github: company-x
Configure the template

A template config is located at: groups/<group-id>/template-id>/config.yml and should look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Version used to track changes
version: 1
# Template display name (max 50 characters)
name: Your template
# Template description to provide additional information (max 150 characters)
description: Your template description
# Template protocol: HTTP, MQTT, or CoAP.
protocol: HTTP
# Template protocol format: STRING, BYTES, JSON, or CBOR.
payloadFormat: JSON
# A list of unique versions that are deprecated
deprecatedVersions: []

This example defines a payload conversion script called script.js which is located at groups/<group-id>/template-id>/script.js. When updating this template’s configuration or payload conversion script you will also need to bump this version number.

Add the payload conversion script

Inside the script.js file you will need to define the payload conversion to ingest the data. For example, this is the basis for a JSON payload conversion script:

1
2
3
4
5

/**
 * @param {JsonPayload} payload
 * @param {Context} context
 */
function convertPayload(payload, context) {}

More information about writing the conversion script can be found in the inbound connector documentation.

Configure the test suite

A test suite is located at: groups/<group-id>/template-id>/tests.yml and should look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Test name
default:
    # Description what is being tested
    description: describe what is being tested here
    # Reference to the payload inside the payloads directory
    payload: ./payloads/your-payload-example
    # Expected measurements from the payload conversion script
    expectedMeasurements:
        - ...
    # Expected logs from the payload conversion script
    expectedLogs:
        - ...
    # Expected error thrown in the payload conversion script
    expectedErrors:
        - ...

This example mentions an example payload which should be located at groups/<group-id>/template-id>/payloads/your-payload-example. In the test suite you are able to define multiple tests, but are required to specify one default test. For each test you can define expected measurements, expected logs and expected errors when the payload conversion script is run using the defined payload.

The payload file is read as a utf-8 string and depending on the configured payloadFormat of the template it is parsed/decoded to the expected payload format. CBOR & JSON are parsed to JavaScript objects using their respective parsers. For the STRING type the raw string contents are used and for the BYTES type the contents of the payload are expected to be in hexadecimal characters.

Example payloads for each payload format can be found in the examples/ directory.

Expected measurements can be defined like this:

1
2
3
4

expectedMeasurements:
    - ingestionId: test-subject-external-id$test-metric-external-id
      value: 2
      date: 2022-08-16T08:57:09.000+00:00

Expected logs can be defined like this:

1
2
3

expectedLogs:
    - level: INFO
      message: this is a info log message

Logs levels can have be INFO, WARN or ERROR.

Expected Errors can be defined like this:

1
2

expectedErrors:
    - message: this is an error message