Azure Event Hub

Azure Event Hub is a fully managed platform for streaming data on Azure. In this guide we will show you how you can set up integrations to let your data (such as measurements and events) flow in and out of Event Hub.

Goal

The goal is to set up an inbound and outbound integration between Azure and Blockbax.

Prerequisites

• Blockbax project (with administrator access)
• An Azure Event Hub

Steps to stream data from Azure to Blockbax

Several steps in both Azure and Blockbax are needed to get the data from Event Hub to Blockbax.

1. Go to your Event Hub overview and select the namespace that you want te receive data from
2. Select the ‘Event Hubs’ in the ‘Entities’ pane
3. Click on the Event Hub/Topic that you want to receive data from
4. Click on the ‘Connect’
5. Press ‘Add’ and create a Shared Access policy with the ‘Listen’ access

   

6. Once created click on it and copy the primary key

Once created we can create the inbound connector in Blockbax.

1. Go to settings in the project
2. Navigate to the tab ‘Inbound connectors’ and click on the green ‘+’ in the top right
3. In the setting ‘Protocol’ select the option ‘Azure Event Hub consume
4. Choose a suitable name for the inbound connector
5. Fill in the namespace with the namespace of your Azure Event Hub Namespace
6. Fill in the topic with the Azure Event Hub Instance
7. Choose a consumer group ID (note that if this is not unique the connector will only get a subset of all the data)
8. Fill in your credentials where the ‘Shared access key name’ is the name of your shared access key, and the ‘Shared access key’ is the primary key

   

9. Write a payload conversion script to convert the incoming payload to the correct Blockbax format. When using the Avro format an optional schema registry url has to be provided. More information can be found here
10. Click the ‘Create connector’ and validate if the ‘Successfully (re)connected inbound connector, will start processing measurements.’ log shows up. This can take up to 30 seconds. If you encounter any issues feel free to contact us

Steps to stream data from Blockbax to Azure

To stream data from Blockbax to Azure, the same procedure on Azure Event Hub has to be done, however now give the Shared Access Policy the ‘Send’ policy instead of the ‘Listen’

Once created we can create the outbound connector in Blockbax.

1. Go to settings in the project
2. Navigate to the tab ‘Outbound connectors’ and click on either of the ‘Create connector’ buttons, depending on if you want to stream out events or measurements
3. In the setting ‘Protocol’ select the option ‘Azure Event Hub produce’
4. Fill in the namespace and the topic (or Azure Event Hub Instance) that the data needs to be produced to
5. Fill in the authentication options, by using the name of the shared access key and using the shared access (primary) key

   

6. Press ‘Create connector’ and validate if data is coming into you Azure Event Hub Instance. If not, feel free to contact us