MQTT Broker Google Pubsub Integration

MQTT Broker Google Pubsub Integration

by | Jan 14, 2019 | MQTT Broker

Google Pubsub is one of the leading Queue engine used to collect data from the publisher or MQTT Application for better data analytics. Integration of Bevywise MQTT Broker with Google Pubsub helps you push data from your IoT & IIoT edge devices via MQTT Broker to google pubsub.

Set up Google Compute for Pubsub

Google Pubsub is accessible from your Google cloud services account.  You can follow the Google Pubsub document to create Google Pubsub with a publish topic.

 

Create Ubuntu Instance

We will be running the Bevywise MQTT Broker on the Ubuntu instance which was created in Google Cloud. Refer to the Google help document for creating the Ubuntu instance. in Google cloud.

 

Installation Google Pub/Sub dependencies

Once you created an Ubuntu instance, login to that instance and install the below dependencies.

  • Open terminal and install below gloud cloud packages
  • sudo pip install google-cloud-core
  • sudo pip install google-api-core
  • sudo pip install –upgrade google-cloud-pubsub
  • Go to “/usr/local/lib/python2.7/dist-packages/PyInstaller/hooks/” folder.

cd /usr/local/lib/python2.7/dist-packages/PyInstaller/hooks/

  • In that foldder, open “hook-google.cloud.py” file by

sudo vim hook-google.cloud.py

  • In that file, remove all the default lines and add the below code

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata(‘google-cloud-core’)
datas += copy_metadata(‘google-api-core’)
datas += copy_metadata(‘google-cloud-pubsub’)

  • save and exit file

MQTT Broker Google Pubsub Integration

  • Install Bevywise MQTTBroker in the created Ubuntu instance.
  • After installation, go to Bevywise/MQTT Broker/extension folder.
  • In that folder, open “custom_store.py” and replace the below code

import os
import sys
import json

from google.cloud import pubsub_v1
sys.path.append(os.getcwd()+’/lib’)
sys.path.append(‘/usr/local/lib/python2.7/dist-packages’)
project_Id = ‘project_name’
broker_Id = ‘testing’
def handle_Received_Payload(data):
global pubsub_v1
global project_Id
global broker_Id
publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path(project_Id, ‘broker’)
data = json.dumps(data)
data = data.encode(‘utf-8’)
publisher.publish(topic_path, data=data, device=broker_Id)

  • In this code, “project id” represent the project id which you while creating google pubsub
  • Save and exit the file
  • Go to Bevywise/MQTT Broker/conf folder
  • Open “data_store.conf” and change the following parameters

    CUSTOMSTORAGE = ENABLED

  • Save and exit the file.
  • If you want to run the MQTTBroker with authentication, then follow the steps
  • open Bevywise/MQTTBroker/conf folder
  • In that open “broker.conf” file and change

    AUTHENTICATION_ENABLED = YES

  • Save and exit the file.
  • Google Pubsub use grpc authentication, so we must export the grpc ssl certificate path in the “/etc/environment” to set the default path for grpc ssl root certificate.
  • Go to /etc folder and open “environment” file by

sudo vim environment

  • In that file, paste the below command and save it.

export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=”usr/local/lib/python2.7/dist-packages/grpc/_cython/_credentials/roots.pem”

  • Next restarted the instance.
  • Once you restarted the instance, go to “Bevywise/MQTT Broker/bin” folder
  • Next start the Bevywise MQTTBroker by

sudo sh runbroker.sh

Now Bevywise MQTTBroker collects the data from your edge device and send to the Google pub/sub via Custom store. From Google Pub/Sub, you can hook and send the data to your data analytics tool.

Get started your data analytics by downloading our MQTTbroker.

download now

 

MQTT Broker – Manage devices better

MQTT Broker – Manage devices better

by | Oct 16, 2018 | MQTT Broker, MQTT security

Today we are happy to announce an enhanced version of the MQTT Broker.The newly updated version will help you identify MQTT connection error, Manage devices better & also provides Dynamic MQTT Authentication.

MQTT Broker manage devices better

You will be able to initiate a command to any particular device from the User Interface of the MQTT Broker. This has been previously supported via REST API built over the broker. The REST APIs were used by customers to initiate a command from their application.

When you do a real time project, IP Address of the edge device helps you get more information & exact location of the device. The new update shows your the client IP Address on the UI. Going forward we will be adding alerts based on the changes in the client IPs to inform suspicious activity.

MQTT Connection Error Monitoring

There is no ideal system in the world. You may face some MQTT connection error while connecting your devices. But it is very important to know the issues and mitigate them. All Edge devices will not behave perfectly as per the Protocol. Any Edge device can be disconnected from the MQTT Broker for various reasons. In a similar way, any packet that is not as per the standard can be dropped as well. We have added support to highlight the actual cause for MQTT connection error. It helps you identify and clear the alarm easily. The following scenarios will show you a warning on the User Interface :

  1. Reuse of the Client identifier.
  2. Usage of wrong MQTT credentials.
  3. Inconsistency in connectivity & Timeout issues.
  4. Usage of invalid Certificates for SSL / TLS connections.
  5. Not supported characters in Client Identifier.
  6. Missing of required Client Details.
  7. Atypical disconnection.
  8. Error in the MQTT Packet formation.
  9. Server not able to handle new request.

Dynamic MQTT Authentication

You can now configure your MQTT Broker parameters from the User Interface. When the authentication is enabled, you will be able to add new MQTT Username and password and also delete authentication keys dynamically. This helps you in take actions swiftly when you find some wrong usage of authentication keys.

Further You will be able to enable / disable TLS and authentication from the User Interface. Also enable Custom Store option from the UI. But these setting changes need a restart of the broker from the terminal. We working on making these changes dynamic as possible.

The new version of the MQTT Broker can be hosted on AWS Securely. We would always recommend you to use Monit to run the MQTT Broker as a service and control over the web also.

Download the MQTT Broker now.

download now

We would be happy to answer your question and hear your feedback. Feel free to write your queries to support.

Localize MQTT Broker to your language

Localize MQTT Broker to your language

by | Aug 28, 2018 | MQTT Broker

We have been working hard to ensure that you can be able to localize MQTT Broker into your language and work with ease.  MQTT Broker, the enterprise MQTT broker has been enhanced with a few more most useful set of features to play with.

Dynamic Update of User Interface

You will be able to monitor the connected devices from the UI without refreshing it. All the events happening in the back end of the broker is sent to the Broker for dynamic updates. The update happens at the Dashboard level as well as the individual devices level. The connection/ disconnection status, Events published by device / commands received by the device, new device additions and subscriptions are all updated.

Localize MQTT Broker

We have added support for working with the MQTT Broker on your own language. We by default have added Chinese and working on a few more languages. You can add your language text to the Bevywise/MQTTRoute/web/build/js/localization.js file and translate the broker to your language.

Do keep us posted with the updated file, so that we can share it with your own language community.

Build Custom UI

The complete User Interface is presented to you using the BrokerUItab.html. You can design your own User Interface and replace this file. You can also update the brokerscript.js file present below <web/build/js> folder to make it work in tandem with the Broker UI. The function  function WebSocketPush() towards the end of the brokerscript.js is the method that pushes the data to the client for all the dynamic updates. You can add your code here to have the dynamic updates as well in the custom UI. You can use REST APIs to build device management for the devices.

Try the Online MQTT Broker

We have set up an live MQTT Dashboard for trying our MQTT Broker. You can connect your devices to mqttroute.com. We have setup the broker in the non ssl mode for the ease of connecting. The MQTT Broker you can download for trial will have both SSL and Non SSL support.

Broker Address – mqttroute.com
Broker Port – 1883

You can run the downloaded version on your private cloud or behind the firewall for security and more.

download now
Feel free to write to us for support.

MQTT Broker Tableau Integration for IoT Data Analysis

MQTT Broker Tableau Integration for IoT Data Analysis

by | Aug 7, 2018 | MQTT Broker

MQTT Broker is a perfect middleware and fastest broker to be used for collecting data from your IoT edge devices and store and analyse it the way you needed.  Tableau is a cool BI tool that will help you analyse the data in all dimensions.  MQTT Broker Tableau integration will help you store all your data in the format you needed into the Tableau for a perfect data analysis. This blog helps you visualize data of MQTT Broker with Tableau by connecting it to MySQL or MongoDB.

MQTT Broker Tableau Integration using MySQL

Integrating MQTT Broker with MySQL :

For integrating MQTT Broker with MySQL, some dependencies should be installed. To achieve this, follow the below step.

  • Run the file named install_mysql_connector.sh  

All the required dependencies required for the integration process will be downloaded.

Changes to be done in MQTT Broker: 

  1. Open the Conf folder(Bevywise/MQTTRoute/conf)
  2. Change the DB Server to MySQL
  3. Give the MySQL Password in “MySQL Password =”
  4. Save the File
  5. Check that the MySQL is running on 3306 Port.
  6. Start the MQTT Broker using sh runbroker.sh inside the bin/ folder.
  7. Next you need to connect the client and verify the  creation of “bevywise” DB in MySQL.

To Verify the DB is Created or not in MySQL:

  1. Login to MySQL by the command “mysql -u<username> -p<password>”
  2. Next type “show databases;” command to view the created DB.
  3. In Output, you can see “bevywise” db is created.

Connect Tableau via MySQL ODBC Connector :

Tableau will use MySQL ODBC connector driver to talk to the MySQL. You might already have the required driver installed on your computer. If the driver is not available in your computer, Tableau displays a message in the connection dialog box with a link to the Driver Download page where you can find driver links and installation instructions. Click the “Data Source” drop down box and Select “MySQL”. Next choose OS & Bit Version and Install the driver.

Make the connection and set up the Tableau:

1.Start Tableau and under Connect, select MySQL. Then do the following:

  • Enter the name of the server that hosts the database.(Give “localhost” if MySQL is running in the same machine)
  • Enter the user name and password.
  • Select Sign In.

2. On the data source page, do the following:

  • From the Database drop-down list, select a database or use the text box to search for a database by name.
  • Under Table, select a table or use the text box to search for a table by name.
  • Drag the table to the canvas, and then select the sheet tab to start your analysis.

MQTT Broker Tableau Integration via MongoDB

Configuring MQTT Broker:

  1. Please download and install MongoDB Plugin for Bevywise MQTT Broker.
  2. Follo the steps in Readme.md file to configure and setup the MongoDB Plugin.
  3. Next Restart the MQTT broker and Connect the device.

To Verify the DB is Created or not in MongoDB:

  1. Login to Mongo DB by the command “mongo –port 27017”
  2. Next type “show dbs” command to view the Created DB.
  3. In Output, you can see “bevywise” db is created

Connecting Tableau via MySQL ODBC Connector :

Tableau use MongoDB Bi connector driver to talk to the MongoDB. You might already have the required driver installed on your computer. If the driver is not installed on your computer, Tableau displays a message in the connection dialog box with a link to the Driver Download page where you can find driver links and installation instructions. Click the “Data Source” drop down box and Select “MongoDB Bi Connector”. Next choose OS & Bit Version and Download the driver.

Creating Authentication for Mongo DB:

Login to the MongoDB shell by the command “mongo –port 27017“

Next use the “bevywise” database by the command ” use bevywise
Type the below query to create a username and password for the DB

db.createUser(

{

  user: “<Username>”,

  pwd: “<Password>”,

roles: [ { role: “userAdminAnyDatabase”, db: “bevywise” } ]

  }

)

Re-start and login to the MongoDB by the command

mongo –port 27017 -u “myUserAdmin” -p “abc123” –authenticationDatabase “bevywise”

For more reference refer the Enable Auth link

Installation and Setup of MongoDB Bi connector in windows:

  1. To install the Mongo DB Bi connector, Double click the downloaded .msi Connector file and Install it.
  2. Next you need to check the installation of connector inside the MongoDB folder which is inside the program files.
  3. Follow the link to setup the MongoDB BI Connector.
  4. After Installation, don’t exit from the terminal.

Make the connection and set up the Tableau:

1.Start Tableau and under Connect, select MongoDB BI Connector. For a complete list of data connections, select More under To a Server. Then do the following:

  • Enter the name of the server as “localhost”.
  • Enter your user name with associated database name, and password.
  • Select Sign In.
  • If Tableau can’t make the connection, verify that your credentials are correct. If you still can’t connect, your computer is having trouble locating the server. Contact your network administrator or database administrator.

2. On the data source page, do the following:

  • (Optional) Select the default data source name at the top of the page, and then enter a unique data source name for use in Tableau. For example, use a data source naming convention that helps other users of the data source figure out which data source to connect to.
  • Select the sheet tab to start your analysis.

Download the MQTT Broker now to get started with the MQTT Broker Tableau integration.

download now
Feel free to contact support for any questions or help.

Docker MQTT Broker for easy service manageability

Docker MQTT Broker for easy service manageability

by | Jul 9, 2018 | MQTT Broker

Docker is one of the leading container applications that is used most widely for running a well managed deployment environment.  Docker MQTT Broker will help you have more control over your IoT production environment. Most than 60% of the servers are using either Ubuntu / Debian for their hosting.

 You can use the following instructions to run the MQTT Broker on Docker on any of cloud providers like Amazon EC2 or Google Cloud or Microsoft cloud or on premise behind the firewall.

Requirements for Docker MQTT Broker

1) Ubuntu 18.04 

2) Docker 20.10.2

3) Mysql 5.7

Setting up the MySQL

We recommend MySQL to be run on a physical machine than a container for the data persistence purpose. Follow the below instructions to set up the MySQL. You have to install the MySQL Server first.

 $ sudo apt-get update

 $ sudo apt-get install -y mysql-server-5.7

Modify the MySQL configuration. By default the mysql runs at 127.0.0.1 and allows only the local application to connect to it. You have to change it as 0.0.0.0 for application running on containers and other machines to access the MySQL.  

$ gedit /etc/mysql/my.cnf

    bind-address = 0.0.0.0

Restart the MySQL for the changes to take effect.

$ sudo service mysql restart

Providing data access. The root user of MySQL has access only form the local machine to read and write data into the databases. We need to allow the root user to access data from the container. Connect to the MySQL console from the localhost and then execute the following queries.

$ mysql -u root -p
Enter password: <provide your password here>

mysql> GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘root’ WITH GRANT OPTION;

mysql> FLUSH PRIVILEGES;

Your MySQL server is all set to connect from any container that runs on your Ubuntu server.

Creating Docker MQTT Broker

Docker needs to be installed in your ubuntu to create the docker image of the MQTT Broker and also run the dockerized MQTT Broker. To install the Docker, run the following command.

 $ apt-get install docker.io

Download the LInux version of the  MQTT Broker and Extract it from your /home/ubuntu folder. If you are extracting the archive from some other folder, make sure you use the right reference folder

[email protected]<yourmachinename>$ unzip Bevywise_MQTT_Route_Linux.zip 

We will be creating a docker image in the name of mqttroute. Create a directory for creating the docker file.

$ mkdir /home/ubuntu/mqttroute

Copy files of the MQTT Broker to the mqttroute folder.

$ cp -rf /home/ubuntu/Bevywise/MQTTRoute/* /home/ubuntu/mqttroute/

Modify the configuration of the MQTT Broker for MySQL connections 

$ cd /home/ubuntu/mqttroute/

$ gedit conf/data_store.conf

DB_SERVER = MYSQL

DBHOST=<mysql host ip>

MYSQL_USER= <your mysql user>

MYSQL_PASSWORD = <mysql db password>

Create the mount folder where the docker images will be mounted.

$sudo mkdir /bin/mqtt

Create the docker file

$ gedit Dockerfile

FROM ubuntu

COPY . /bin/mqtt

CMD [“sh”, “/bin/mqtt/bin/runbroker.sh”]

Build the docker image

 # docker build -t mqttroute .

Verify & Run Docker MQTT Broker

 # docker images

[email protected] # docker images

REPOSITORY     TAG     IMAGE ID     CREATED     VIRTUAL SIZE

mqtt         latest         45f7ee9a1ec7       7 seconds ago      81.15 MB

ubuntu    latest          7feff7652c69       4 weeks ago         81.15 MB

Start using the docker command. The following command will run the MQTT Broker as a daemon process. If you are planning to run MQTT Broker in a secure mode, you can refer to our previous blog on ssl certificate creation and placement of the files inside the MQTT Broker.

$ docker run – –restart=always -d -p 8080:8080 -p 1883:1883 -p 10433:10433  mqttroute

Download now to set up your own Docker MQTT Broker

download now

Feel free to contact support for any questions or feedback. 

 

Must Read Other Related Post