High availability MQTT Cluster
High availability MQTT Cluster is very important for any IoT deployments in production environment. Failures can happen due to Software failures, Disk errors, network connectivity, Hardware component failures, etc., To ensure that your services are up and running always to your customers without a single point of failure, the services need to be functionally and physically redundant. By default MQTT protocol a publish and subscribe model does not support high availability. But the resilient and fault tolerant MQTT high availability set up can be done with simple steps with the right MQTT Broker. With the latest release, Bevywise MQTT Broker comes up with the high availability server which avoids failure for MQTT communication. This article helps you to set up a high availability MQTT Cluster with the high availability server of Bevywise MQTT Broker.
The distributed cluster set up
The created cluster set up helps to ensure that the broker is consistently functional without any failures. It benefit yourself by enabling failover if any of the MQTT Broker you’re using is inaccessible. Consider you’re having numerous devices connected to a single MQTT Broker / Server. There is a high chance of any malfunctions to occur and the MQTT Broker may eventually fail, losing the data. This is where the clustered set up extends its help. This distributed architecture allows you to have two or more MQTT Broker / Servers. So that if one of theBroker / cluster node stops working, the other cluster node / MQTT Broker will take the responsibility and perform the task without giving room for any interruptions to take place on the client side. Thus cluster nodes are considered to be the MQTT Brokers.
For ambitious MQTT deployments with many millions of clients, any number of broker nodes can be deployed & the data can be distributed maintaining stability and availability. Even if network splits occur or any kind of connectivity problem between nodes arises, the cluster as a whole is still available, as long as at least a single node is still healthy, and heals itself in error scenarios. This is because, whatever may be the count of MQTT servers, everything will be in active state, and the data will be distributed between the nodes. So, the healthy one will take the role of publishing the data to the subscribed client.
HA Architecture
Let’s take the above architecture as an example. This illustrates that a High availability server is installed in each machine that runs the MQTT Broker. A Load Balancer is configured to balance the loads between the brokers and the High Availability server in one is primary while those in others are just back-ups.
Role of a Load Balancer
Load Balancer is also a part of high availability clustered set up. The role of this load balancer is to balance the incoming connections to your broker infrastructure. It is a decision maker whose role is to decide which MQTT Broker / server / cluster node in the cluster should take the role of failed MQTT server. That decision could be based on the weightage or the performing ability of the MQTT Broker. Install any load balancer of your choice and set up port forwarding.
Setting up high Availability
For setting up High availability you need at least two MQTT Brokers. The number of brokers may increase based on the count of clients.
You can get your MQTT Broker from our download page.
You have to install MQTT Broker in each virtual machines. Do the following steps to enable the high availability set up.
Let’s first take a peek at the load balancer set up.
Enabling High Availability with Nginx Load Balancer
Set up Nginx as a load balancer in one of the machines. Following are the steps to install and configure Nginx.
- Update and install Nginx using the following command.
$ sudo apt-get update
$ sudo apt-get install nginx
- Then configure the Nginx proxy using the following command
sudo vim /etc/nginx/ nginx.conf
#Nginx Configuration
stream {
upstream bw_brokers {
server <IP of MQTTRoute Broker1>:1883;
server <IP of MQTTRoute Broker2>:1883;
}
server {
listen 1883;
proxy_pass bw_brokers;
}
upstream bw_HA {
server <IP of MQTTRoute Broker1>:8082 fail_timeout=1s max_fails=1;
server <IP of MQTTRoute Broker2>:8082 backup;
}
server {
listen 8082;
proxy_pass bw_HA;
}
}
Ensure that you’ve provided the IP of the MQTT Brokers in the above configuration.
Setting up MQTT Broker for High Availability
- Go to Bevywise/MQTTRoute/conf/broker.conf file
- Provide the load balancer’s IP you got after setting up the load balancer.
[HA Server]
HASERVER_ENABLED = YES
HASERVER_IP = <IP address of Load Balancer>
HASERVER_PORT = 8082
Make sure you’ve provided the IP of load balancer in all the MQTT Brokers available in the clustered set up.
To configure the HA Server :
For Linux users,
- Unzip the HAserver.zip file which is present inside the ./Bevywise/MQTTRoute folder.
- Then open ./HAServer/config.json file which is present inside the unzipped folder.
- Configure the HA Server port. By default, HA Server runs at 8082.
To start the HA Server,
- Open the terminal and go to ./HAServer/bin
- Run HA Server using the command ‘sh startServer.sh’
Windows users can execute the exe file.
Database Configuration
Despite the count, all the MQTT Brokers should possess the ability to deliver data to a singe database. That database should run on a separate virtual machine. You can refer to the Digital Ocean document for setting up the same. Alternatively you can use Amazon RDS or Azure Data Store which by default provides the redundancy of the Data storage and much more features like time based snapshots for recovery. To know the detailed steps of database configuration, follow this link.
Now the complete High Availability set up is ready.
Validate the High Availability MQTT Cluster
MQTT client should have the necessary code to reconnect whether there arises a disconnection issu. Refer this blog for building MQTT client.
- Connect a MQTT Client that has the ability to reconnect and connect to the public IP of your load balancer machine.
- You can see the device on MQTT Route UI and also you can see the MQTT messages.
- Stop the broker which has the HA server currently active using the command “sh stopbroker.sh” once the complete set up is running.
- You can see your edge client being disconnected, reconnected again and start continue with its messaging. And it can be found seamlessly on the MQTT Route UI.
- The standby one has thus become active and started serving the requests.
Download MQTT Broker now for FREE to get started with your High availability MQTT Cluster.
Reach support for any queries regarding any queries or suggestions.