In the case of Microservices Architecture, one of the common scenarios is to create a Messaging Service between two Microservices. The common candidates of this Azure Messaging service are Azure Event Grid and Azure Service Bus. In this article, we will look into the features of both these services and recommend the use of these services as a messaging service between two microservices.
Push vs Pull
Event Grid works on a Push mechanism, which means that the Event Grid pushes the event to its Subscribers, whereas in Service Bus, the subscribers pull the messages from the Bus.
As the Event Grid pushes the messages to the Subscriber, the rate with which the events are pushed to the Subscriber may be more than the rate at which the Subscriber can process the events. This may cause the Subscriber to Throttle or become unresponsive.
In order to validate the above, we created a Topic in the Event Grid and added a Logic App as a Subscriber to test the above. The Logic App invoked an Azure Function inside its workflow. We sent 200 messages to the Event Grid Topic separated in 50 seconds with JMeter. After some initial messages, the Azure Function started failing. Please refer to the screenshot below. Although we can improve this by using configuration, the problem would not be solved unless a mechanism like Service Bus is in between the Event Grid which will allow the Logic App to work at its own pace.
On the other hand, the Service Bus works on a Pull mechanism. The Subscriber picks the messages at their own pace from the service bus and hence do not cause exceptions even if thousands of events are fired at Service Bus. The service bus keeps adding messages to the Subscriber queues for the Subscriber to process.
In the above Architecture, we introduced a a Service Bus between the Event Grid and its Subscriber. Also in order to have a better granular control over each process we broke the Logic App into individual functions separated by Service Bus. Then, we again fired the same 200 messages to the Event Grid using the JMeter in 50 seconds. All the messages were processed successfully in 7-8 minutes and there were no failures. Even when 1000 messages were fired from JMeter in 50 seconds to the Event Grid, all the messages were processed successfully eventually in 13-14 minutes with no failures.
This shows that having Service Bus between the Event Grid and Subscriber makes the architecture more resilient and fault tolerant.
Loss of Messages
Service Bus allows the messages to be deleted from the Service Bus subscription only after the Subscriber explicitly deletes it from the subscription queue under the peek and lock mode. In the case of Event Grid, the message is deleted from the Event Grid Topic immediately after it is delivered to the Subscriber. In our experiments, we noticed that for Event Grid if there were exceptions while processing an event in the subscriber and if the Subscriber did not handle the exceptions properly by sending the message to a dead letter queue or some other source then the messages get lost.
In the case of Service Bus, if there are exceptions in the Subscriber while processing the message then the message gets moved to the Dead letter queue automatically where it can be retried again. In the case of Service Bus, the subscriber did not have to explicitly send the message to dead letter queue, the messages were sent to Dead-Letter queue automatically in case of exceptions.
Order of Delivery
Event Grid does not guarantee the order of Events delivery whereas with Service Bus we can get the ordered messages using the Message sessions.
Transaction and Duplicate Detection
EventGrid does not support Transaction and Duplicate Detection whereas Service Bus does.
Viewer
While the application is in production or during development, it is important that we can see what messages have been published in the Azure Messaging Service.
Microsoft has not released any default product for viewing messages published in Event Grid. However, one can add external applications like EventGridViewer
which can be added as a subscriber to the events. Please note that this EventGridViewer does not store the events, therefore in case one wants to look into the past events, another subscriber must be added that stores the data in some datastore and some mechanism should be provided for filtering the events and viewing the stored events. Also, this subscriber may also get throttled in case the messages are posted to it at a rate higher to
On the other hand, Microsoft has released a tool called Service Bus Explorer that can be used for seeing what all messages were published. Please refer to the Service Bus Explorer version available in Azure Portal.
Message Retention and Lost Policy
Both EventGrid and ServiceBus are resilient in trying to deliver the message to their Subscribers.
The EventGrid will drop the event if either of the limits of the retry policy is reached.
·Maximum number of attempts - The value must be an integer between 1 and 30. The de ault value is 30.
·Event time-to-live (TTL) - The value must be an integer between 1 and 1440. The default value is 1440 minutes:
EventGrid drops the event to the Dead-letter queue if configured, otherwise, the event is permanently lost. Also, f the dead letter container is not online for 4 hours then the event will be lost.
In the case of Azure Service Bus, the messages are pushed to the subscriber queues. While creating a Subscriber one can configure how long the message will remain in the Subscription queue before being removed. One can even set a setting that may even allow a message to remain in the subscription queue for a very long time.
Dead-letter queue
Both Service Bus and Event Grid support the Dead-letter queues. The only difference is that in Service Bus Dead-letter queue is available by default whereas in Event Grid it needs to be configured using a Storage account.
Integration with Azure Resources
Azure Resources like Storage Accounts, Azure Resources, Azure Subscriptions etc have default integration with Event Grid.
So when to use what?
In the case of Microservice architecture where microservices send valuable messages to each other, it is better to use Service Bus than Event Grid due to the following reasons:
1.Service Bus allows the subscribers to consume the messages at their own rate as opposed to Event Grid which pushes the events to the subscribers causing exceptions if the rate of event push is more than the rate at which the Subscriber processes the message. This Service Bus feature reduces the number of errors in the system and makes it more resilient and easy to maintain.
2.Service Bus allows the messages to be read in Peek and Lock mode which protects the messages from getting lost in case of exceptions whereas in the case of Event Grid there are chances of events being lost.
3.Service Bus allows the messages to be read in an orderly fashion whereas in Event Grid the messages can come in any order.
4.Service Bus allows for Transaction and Duplication detection which is not available in Event Grid.
5. There are better viewers like Service Bus Explorer available for Service Bus whereas there are no such out of the box tools available right now for Event Grid. This m kes it easier to maintain, observe and investigate any issues that might come in future for the Service Bus.
EventGrids should be used in scenarios where there is an event integration involved with Azure Resources like Azure Storage, Subscriptions etc as Azure provides natural integration with Event Grids. Even in those scenarios, it is better to put a Service bus between the Event Grid and the Subscriber to make the application more resilient.
Event Grids can also be useful in scenarios where the Microservice needs to inform the subscribers immediately when something happens. In those cases as well if the events are important for the consumer microservice then a Service Bus should be put in between the Event Grid and Service Bus.
