Docs
Azure Event Hub
Measurements can be streamed from an external Azure Event Hub broker into Blockbax by using an inbound connector. Measurements and events can also be streamed from Blockbax to an external Azure Event Hub broker by using an outbound connector.
Inbound connectors
Inbound connectors are custom integrations to ingest measurements into the Blockbax Platform. They allow you to extract information from a custom payload and to ingest measurements in the format that Blockbax expects. They also allow you to log information during this conversion. A payload conversion script is defined as a plain JavaScript function. A dedicated page about payload conversion describes how to develop such a function.
Note that we do not process payloads with a size over 200 KiB. When this happens an error log will be output.
Configuration options
The following configuration options are available:
| Field | Description |
|---|---|
| State | Active or Inactive |
| Name | A descriptive name to recognize and find your inbound connector |
| Auto-create subjects | Automatically creates a subjects based on the incoming payload (see below) |
| Namespace | The prefix of the Azure Event Hub Namespace, e.g. my-namespace |
| Topic | The topic that the inbound connector will consume from |
| Consumer group ID | The consumer group ID that the client will use |
| Initial position in stream | The place in the stream to start (only has effect when the broker does not recognize the consumer group id) |
| Shared access key name | The name of the access key that the client will connect with |
| Shared access key | The access key that the client will connect with |
Auto-create subjects
Measurements are mapped to subjects and metrics by so-called ingestion IDs. By default, an ingestion ID is derived from the subjects’ external ID and the metrics’ external ID (e.g. MyCar$Location). You can also override this default with a custom ingestion ID. If a measurement with an unknown ingestion ID is encountered, Blockbax tries to find a single metric with an external ID equal to the part after the dollar sign. If this is successful, a subject will be created with an external ID equal to the part before the dollar sign. For example, for the ingestion ID MyCar$Location a subject with external ID MyCar will be created if the metric with external ID Location can be linked to exactly one subject type.
Outbound connectors
Outbound connectors give you the opportunity to stream your data out of the Blockbax platform and into your own infrastructure. You are able to configure two outbound connectors, one for events and the other for measurements. when these are set up, data is forwarded whenever an event is triggered or measurement is received. Note: you can either have your data produced to an Azure Event Hub or to a Kafka cluster, not both.
To produce data to your Azure Event Hub, the connector will need the following settings:
| Field | Description |
|---|---|
| Namespace | The prefix of your Azure Event Hub namespace |
| Topic | The topic you would like to have your data streamed to |
| Shared access key name | The name of your shared access key that Blockbax will connect with |
| Shared access key | Your shared access key that Blockbax will connect with |
Connection guide
Here you can find the information to help you get started quickly with this integration.
Stream measurements from Event Hub to Blockbax
Several steps in both Azure and Blockbax are needed to get the data from Event Hub to Blockbax:
- Create Shared Access Signature (SAS) with the ‘Listen’ access for the specific event hub (topic) you want to connect. Copy the name and (primary) key, you need it in the following step.
- Create an inbound connector for Azure Event Hub in Blockbax. When using the Avro format an optional schema registry url has to be provided. More information can be found here
- Validate if an info log appears that it is able to connect successfully. This can take up to 30 seconds.
Stream measurements from Blockbax to Event Hub
To stream data from Blockbax to Azure, the same procedure on Azure Event Hub has to be done, however now give the Shared Access Policy the ‘Send’ policy instead of the ‘Listen’. After this streaming the data out of Blockbax can be setup by creating an Outbound connector similarly to the inbound connector earlier.