Event Hubs Management

Azure Event Hubs is a highly scalable data streaming platform and event ingestion service, capable of receiving and processing millions of events per second. Event Hubs can process and store events, data, or telemetry produced by distributed software and devices. Data sent to an event hub can be transformed and stored using any real-time analytics provider or batching/storage adapters. With the ability to provide publish-subscribe capabilities with low latency and at massive scale, Event Hubs serves as the "on ramp" for Big Data.

Challenges in managing Azure Event Hubs

Azure event hubs are designed for scale, it can process millions and millions of message on both directions – inbound and outbound. Some of the real world use cases include getting telemetry data from cars, games, application so on, IoT scenario where millions of devices push data to the cloud, gaming scenarios where you push user activities at scale to the cloud etc. There are tons of use cases why we need a technology like Azure Event Hubs.

Consider a scenario, you want to send messages to a mission critical solution for a long period, say 2 days, you must keep your machine running till the said duration and still, there might be chances of network outage or unforeseen problems which can break the sequence and you must start all of it again.

Serverless360 Capabilities

It would be of great value to associate, manage and monitor Azure Logic Apps in Serverless360 Composite Applications. Serverless360 comes with following capabilities to manage Azure Function Apps associated using the Service Principal.

What you can do with Serverless360

1. CRUD Operations - From Serverless360 UI, you can create/delete, enable/disable, edit properties, view shared access policy and copy the Event Hub URL. You can initiate bulk operations - enable/disable multiple Event Hubs in a single click.

2. Import Event Hubs - Let's consider a scenario, ACME corporation has created and configured the settings for an Event Hub in their test environment and wants to replicate the same in the development environment, in this scenario, the only option is to recreate the Event Hub with the same settings either in the Azure portal or in Service Bus Explorer. Obviously, this is a time-consuming process, Serverless360 provides a simple solution by allowing the users to import the Event Hub between namespace to make process easy.

3. Create and delete consumer groups - Event Hubs has two primary models for event consumption - direct receivers and higher-level abstractions, such as EventProcessorHost. Direct receivers are responsible for their own coordination of access to partitions within a consumer group. A consumer group is a view (state, position, or offset) into a partitioned event hub. Consumers groups let consuming applications have a separate view of the entire Event Hub. There is always a default consumer group. You can create up to 20 consumer groups in an Event Hub.

4. Monitoring - When an Event Hub is included in a Composite Application, you can enable a monitor to trigger alerts to the configured notification channel(s) by comparing the current value against the expected value. You can read extensive documentation on Monitoring here.

Activities Configuration

You can send events immediately from the list of event hubs or you can opt to create an Activity Configuration to schedule it for future. We have a common “Activities” tab in both – Event Hubs and Messaging Namespaces – from here you can create Activity Configurations and initiate them, see the in-progress activities and their status.

Serverless360 has the option to send events from the listed event hubs by simply entering the desired values and hit the Send button.

Activities allow you to push sample messages into desired Event Hubs or Queues or Topics. It can be used mainly for testing purposes. Ex: Sending 5000 sample messages into Event Hub with think time, batches, multiple threads etc. You can have multiple activities configured and executed them either instantaneously or via a schedule.

As every task is processed in the cloud – it will queue the task to a certain queue from where it will be picked up and processed. However, the processing is fast enough and you will be able to see the in-progress activity in “In Progress” tab. Sometimes you may not find an activity in “In Progress” tab if the number of events is too small. As the process is quick – the process might be completed momentarily.