Azure Event Grid is an awesome new service in Azure, which connects applications together through event messaging. This way of working decouples application components, which enables more scalability, maintainability, and extensibility. The image below illustrates this.
Orders are written in and Azure Cosmos DB database. Event Grid reacts to each new order and publishes that as an event and the Process Orders application can process the order further.
In this example, the Cosmos DB and the application that writes data into the Cosmos DB can be scaled independently from the Process Orders application. You can also add more event consumers for this event and maybe write new orders to a Data Lake, for analysis and predictive analytics. This solution is easy to maintain and to extend.
The concept of decoupling is not new, we’ve been doing this for years now with mechanisms like queues. How is this different? With queues, the Process Orders application in the example would poll the queue until he found a new message, which is expensive from a performance point of view. With Event Grid, events are sent real-time, as soon as they happen, without the need for polling, which is more performant.
Also, services like Cosmos DB or Azure Storage, weren’t capable of firing notifications before, you needed to poll them for changes (to see if a new row is inserted). The introduction of Event Grid is driving all the services in Azure to emit notifications, so that eventually, you can integrate every Azure service with anything else, in an event-driven way.
Let’s discuss some of the concepts in the Azure Event Grid.
Events are things that happen, packaged in small messages that describe the thing that happened with contexts like a timestamp and a source.
Topics are the way event publishers categorize events that they publish. These categories, or topics, can then be used by event subscribers to subscribe to the events that they are interested in. This is a similar mechanism as Topics in the Azure Service Bus. Topics in the Event Grid also contain information about the endpoint where the publishers send events to. And the contain information about the schema of the events (what they look like), which subscribers can use to consume events.
Event publishers are the sources of events. These are applications or resources that generate events, like Azure Blob Storage or Azure Resource Manager. You can also write custom event publishers. Eventually, all the services in Azure can serve as event publishers for the Event Grid, making it very easy to integrate systems in Azure. However, at the moment, the following event publishers are available or are coming soon:
Event Hubs, Azure Automation, Azure Active Directory, API Management, Azure Blob Storage, Azure Resource Manager, Custom publishers, Logic Apps, IoT Hub, Azure Service Bus, Azure Data Lake Store and Azure Cosmos DB.
Event Handlers are the applications that consume the events from Event Grid. These can be things like an HTTP webhook where events need to be written to, a Logic App or Azure Function that gets triggered when an event is raised or a queue where a new queue message should be written, based on an event.
Event Handlers indicate in which topics they are interested in by creating subscriptions. Additionally, Event Handlers can filter events by event type, or subject pattern.
Retrying failures is something that happens out-of-the-box. Depending on the type of event handler that is subscribed to a certain topic, Event Grid expects a certain code (like HTTP 200 OK) to know that the event has been delivered successfully. If the delivery failed, Event Grid will automatically retry with an incremental back off period. It first retries and then 30 seconds later and then 1 minute later and then 5 minutes later.
Azure Event Grid is serverless, meaning that you don’t have to maintain an instance of it, it will just run. You don’t even have to scale it, it does that automatically. You also only pay for what you use, you don’t pay a monthly fee.
Working with events in Azure isn’t new. There are services that do similar things that have been around for years, like Azure Logic Apps and Azure Service Bus Queues and Topics. So how is Azure Event Grid different?
Event Gird is all about real-time communication through events. It passes one event to one or more subscribers that can process the event.
Azure Service Bus Queues and Topics work in a similar way. An application generates an event in the form of a queue message and posts it on the queue. Depending if you are using queues or topics, one or multiple applications at the same time, get the queue message from the queue and process it. The difference here is that the ‘subscribers’ need to poll the queue and actively get the message from the queue and process it. This is not real-time communication.
Logic Apps decouples processes in a similar fashion. Applications generate an event (something happens) and a Logic App can be triggered by that, even in real-time for some trigger types. The Logic App than goes on to work through a process of API calls and responses. This goes much further than passing just one event to another service, it is process automation through orchestration.
The difference between Logic Apps and Event Grid is that Event Grid can provide the trigger for a Logic App and the Logic App is the subscribing application that automates a complete process. Two very different things.
Now that you have a basic understanding of the concepts in Azure Event Grid, let’s make it work in a small tutorial. This way, you can put the concepts in the real world and see what they actually mean.
In this tutorial, we are going to use the Azure Portal and the Azure Cloud Shell in the Azure Portal. We’ll create an Azure Function that will receive custom events that we will send to the Event Grid.
We will first create the Azure Function that will receive the custom event that we’ll send through the event grid. It will have a WebHook that gets triggered by the Event Grid, every time there is a new event. Let’s create the Azure Function:
log.Info(“C# HTTP trigger function processed a request.”);
// Get request body
string json = await req.Content.ReadAsStringAsync();
//output request body
log.Info(json);
return req.CreateResponse(HttpStatusCode.OK, json);
Alright. Now, we have an Azure Function that gets triggered whenever the Webhook URL is called.
In the next step, you’ll need the URL that we can use to trigger the Azure Function. You can find this when you click the </> Get function URL link in the Azure Function blade.
Now to create the Azure Event Grid Topic that we are going to use to send events to and subscribe to our Azure Function. We’ll do all of this from the Azure Cloud Shell, which is the Azure CLI in the cloud. You can access it in the Azur Portal, from the top-right menu:
Once initialized, you can start using it, just like you would use the Azure CLI from your computer.
First, we’ll create a custom topic that we will use to post events to. Use the following line of code to create the topic. Fill in a topic_name and the name of the resource group that you’ve created in step 1.
az eventgrid topic create –name topic_name -l westus2 -g resource_group_we_used_before
This results in something like this:
Next, we will subscribe to the topic using the webhook URL from the Azure Function that we’ve created in step 1. We do this by using the following line of code in the Azure Cloud Shell. You fill in the name for this subscription, the URL for the Azure Function, the resource group that we’ve used before and the name of the topic that we’ve just created.
az eventgrid topic event-subscription create –name name_for_the_subscription \
–endpoint URL_from_Azure_Function \
-g name_of_the_resource_group \
–topic-name name_of_the_topic
That’s it! Now we can start sending events.
Now that we have an Event Grid Topic listening for events and an Azure Function subscribed to events, we can start testing them. Let’s send an event to the Topic and see if the Azure Function gets triggered.
In the Azure Cloud Shell, we will first populate variables with the values of the Topic endpoint URL and the Topic key to use when we post events. We need this to know where we can post events to and to authenticate ourselves against the Topic. We do that by using these two lines of code, where you need to fill in the name of the topic and the resource group that we’ve used before.
endpoint=$(az eventgrid topic show –name name_of_the_topic -g name_of_the_resource_group –query “endpoint” –output tsv)
key=$(az eventgrid topic key list –name name_of_the_topic -g name_of_the_resource_group –query “key1” –output tsv)
Next, we create the event message. The event consists out of JSON. We are going to use the .json file that you can find here: https://raw.githubusercontent.com/Azure/azure-docs-json-samples/master/event-grid/customevent.json. We’ll populate a variable with this JSON content using this line of code:
body=$(eval echo “‘$(curl https://raw.githubusercontent.com/Azure/azure-docs-json-samples/master/event-grid/customevent.json)'”)
Now, we are ready to post the event. Using the following line of code, we’ll post the event and hopefully trigger the Azure Function:
curl -X POST -H “aeg-sas-key: $key” -d “$body” $endpoint
Now, go to the Azure Function and check the log (below the code editor). This should show the event payload as it is received and should look like this:
That’s it! You can send custom events to the Event Grid and they trigger the Azure Function that can process them. Using this, we can add more and different subscribers than the Azure Function. They can all listen to the events that we are sending and can all scale individually.
[adinserter block=”33″]
It’s still early for Azure Event Grid, but it is a very promising service. The list of event publishers is growing and will eventually include all Azure services. This opens up many possibilities and decouples solutions per default.
The service does overlap with built-in triggers in Azure Functions and Logic Apps, so we’ll have to see how this overlap evolves and how Event Grid can make its use case unique.
I’m going to keep an eye on Azure Event Grid, and I hope that you will too. Let me know what you think in the comments.
If you’re using Azure, it may be worthwhile for you to check out Stackify Retrace, the full lifecycle application management tool that will help you continually improve your .NET application.
Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.
If you would like to be a guest contributor to the Stackify blog please reach out to [email protected]
