In cloud architecture (or microservices), applications are divided into smaller independent blocks that can be easily developed, deployed and maintained. Imagine you have a cloud architecture that has many services and many requests per second, you must ensure that no one request is lost and your web service is always ready to receive new requests instead of locked by are handling previous requests as well as ensuring that services communicate with each other smoothly and effectively.
So how do you do? The answer is the Message Broker !
What is a message broker?
A message broker (also known as an integration broker or interface engine ) is an intermediary module that transports messages from sender to receiver. It is an architentural pattern for checking, forwarding and navigating messages ; mediate between applications, minimize communication between them, and to maximize the efficiency of splitting up smaller blocks. The main task of a Message broker is to receive messages from applications and perform a certain action. Let’s look at the sequence diagram below:
As you can see, during Initialize , Service1 and Sevice2 init, then load the proxy and register to Broker. From there, Broker will forward the messages to the proxy that has been registered before. This pattern will have the following benefits:
- Service1 and Service2 do not need to know each other. It only needs to send a message to the proxy, from which the proxy will forward the message to Broker. From there Broker will forward messages to Service1 and Service2 that they have previously registered to receive messages.
- Service1 and Service2 communicate via Broker, so even though there is a language difference, they can communicate successfully.
- With this design pattern, we can set up asynchronous mechanism. For Service1 , it doesn’t need to care when the message reaches Service2 or when Service2 is finished, it only needs to send the message to the Message Broker . Service2 will retrieve the message whenever it wants. This feature can be leveraged to build log storage and processing systems.
Currently, there are many message broker software that can be mentioned such as: Amazon Web Services (AWS) Simple Queue Service (SQS) , Apache Kafka , Apache ActiveMQ . But the most popular of these names is RabbitMQ !
What is RabbitMQ?
RabbitMQ is an open-source Message broker , originally used for the Advanced Message Queuing Protocol (AMQP) , which was later developed to support Streaming Text Oriented Messaging Protocol (STOMP) , Message Queuing Telemetry Transport (MQTT) , and other other protocols. However, in this article, I will not go into the above protocols. RabbitMQ is written in Erlang , an uncommon language but quite suitable for the message Broker ‘s work.
RabbitMQ and in messaging generally use the following techniques:
- Producing means simply sending . The messaging application is called Producer .
- Queue is a post box located in RabbitMQ . Messages move through RabbitMQ and your application, but they can only be stored in the queue . Queue is limited to memory and disk of the host. In essence, it is a message cache with large data. Many producers can send messages to a queue and many consumers can receive data from a queue :
- Consuming has the same meaning as receiving . Consumer is an application that is primarily waiting to receive messages :
Note that the producer , consumer and broker do not need to depend on the same host. In fact, there are very few such applications. An application can be both a producer and a consumer .
Hello World!
In this article, we will use the Bunny Ruby Client as an example of a producer sending messages and a consumer receiving messages and printing them to the screen. In the picture below, P is the producer and C is the consumer , the middle box is a queue :
First, we install Bunny:
1 2 | gem <span class="token function">install</span> bunny --version <span class="token string">">= 2.13.0"</span> |
Sending
We will write message producer in sender.rb and message consumer in receiver.rb . The producer will connect to RabbitMQ , send a message and exit .
1 2 3 4 5 6 7 | <span class="token comment">#!/usr/bin/env ruby</span> <span class="token keyword">require</span> <span class="token string">"bunny"</span> connection <span class="token operator">=</span> <span class="token constant">Bunny</span> <span class="token punctuation">.</span> <span class="token keyword">new</span> <span class="token punctuation">(</span> hostname <span class="token punctuation">:</span> <span class="token string">"rabbit.local"</span> <span class="token punctuation">)</span> connection <span class="token punctuation">.</span> start |
connection will take a socket connection , handle the version of the protocol, authenticate, and so on. In this example, we will connect to the local , if we want to connect to another host , just use the option :hostname and specify its domain name or IP address. Next, we create a channel , a queue to send the message :
1 2 3 4 5 6 7 | channel <span class="token operator">=</span> connection <span class="token punctuation">.</span> create_channel <span class="token comment"># tạo một channel</span> queue <span class="token operator">=</span> channel <span class="token punctuation">.</span> <span class="token function">queue</span> <span class="token punctuation">(</span> <span class="token string">"hello"</span> <span class="token punctuation">)</span> <span class="token comment"># tạo queue tên là "hello"</span> channel <span class="token punctuation">.</span> default_exchange <span class="token punctuation">.</span> <span class="token function">publish</span> <span class="token punctuation">(</span> <span class="token string">"Hello World!"</span> <span class="token punctuation">,</span> routing_key <span class="token punctuation">:</span> queue <span class="token punctuation">.</span> name <span class="token punctuation">)</span> puts <span class="token string">" [x] Sent "Hello World!""</span> |
The queue will only be created if it does not already exist, the message is a byte array so you can pass anything you want. Finally, we close connecion :
1 2 | connection <span class="token punctuation">.</span> close |
Receiving
Now, consumers will listen for messages from RabbitMQ . Unlike producers , consumers will run to listen to messages and print them to the screen.
Similar to producers , we open the connection and channel , declare the queue , note that the name of the queue must be the same as the queue in sender.rb :
1 2 3 4 5 6 7 8 9 10 11 | <span class="token comment">#!/usr/bin/env ruby</span> <span class="token keyword">require</span> <span class="token string">"bunny"</span> connection <span class="token operator">=</span> <span class="token constant">Bunny</span> <span class="token punctuation">.</span> <span class="token keyword">new</span> <span class="token punctuation">(</span> hostname <span class="token punctuation">:</span> <span class="token string">"rabbit.local"</span> <span class="token punctuation">)</span> connection <span class="token punctuation">.</span> start channel <span class="token operator">=</span> connection <span class="token punctuation">.</span> create_channel <span class="token comment"># tạo một channel</span> queue <span class="token operator">=</span> channel <span class="token punctuation">.</span> <span class="token function">queue</span> <span class="token punctuation">(</span> <span class="token string">"hello"</span> <span class="token punctuation">)</span> <span class="token comment"># tạo queue với tên giống với sender.rb</span> |
Here, we also declare the queue to make sure that the queue existed before the consumer message .
Bunny::Queue#subscribe will deliver the message from the queue and it is also a callback that is executed when RabbitMQ pushes the message to the consumer asynchronously:
1 2 3 4 5 6 7 8 9 10 11 | <span class="token keyword">begin</span> puts <span class="token string">" [*] Waiting for messages. To exit press CTRL+C"</span> queue <span class="token punctuation">.</span> <span class="token function">subscribe</span> <span class="token punctuation">(</span> block <span class="token punctuation">:</span> <span class="token keyword">true</span> <span class="token punctuation">)</span> <span class="token keyword">do</span> <span class="token operator">|</span> _delivery_info <span class="token punctuation">,</span> _properties <span class="token punctuation">,</span> body <span class="token operator">|</span> puts <span class="token string">" [x] Received <span class="token interpolation"><span class="token delimiter tag">#{</span> body <span class="token delimiter tag">}</span></span> "</span> <span class="token keyword">end</span> <span class="token keyword">rescue</span> <span class="token constant">Interrupt</span> <span class="token operator">=</span> <span class="token operator">></span> _ connection <span class="token punctuation">.</span> close <span class="token function">exit</span> <span class="token punctuation">(</span> <span class="token number">0</span> <span class="token punctuation">)</span> <span class="token keyword">end</span> |
Here, to prevent receiver.rb from ending immediately, Bunny::Queue#subscribe uses a block to wait for the thread to execute.
Now, run consumer :
1 2 | ruby receiver <span class="token punctuation">.</span> rb |
Then run sender :
1 2 | ruby sender <span class="token punctuation">.</span> rb |
The consumer will print the message it receives from the producer via RabbitMQ . Consumer will keep running to wait for the message .
Try running the producer on another terminal to get a better idea!