Understanding MQTT Protocol Packet Format

Understanding MQTT Protocol Packet Format

by | May 12, 2021 | MQTT

MQTT protocol communicates by exchanging a series of predefined MQTT control packets. The MQTT packets consist of up to three parts in the following order as a 2-byte fixed header that appears in all MQTT packets, Variable Headers, and Payload that appear in some MQTT packets. This document describes how to format an MQTT Message or Packet.

MQTT client to MQTT protocol

MQTT Protocol Packet Types

Types of MQTT Protocol Control Packets are:

  • CONNECT – Fixed Header / Variable Header / Payload – MQTT Client requests a connection to a Broker.
  • CONNACK – Fixed Header – Acknowledge connection request.
  • PUBLISH – Fixed Header / Variable Header – Publish message.
  • SUBACK – Fixed Header / Variable Header / Payload – Subscribe acknowledgement.
  • UNSUBSCRIBE – Fixed Header / Variable Header / Payload – Unsubscribe from a topic.
  • UNSUBACK – Fixed Header / Variable Header / Payload – Unsubscribe acknowledgement.
  • DISCONNECT – Fixed Header – Disconnect notification.
  • PUBREL – Fixed Header / Variable Header – Publish release(QoS 2 publish received).
  • PUBACK – Fixed Header / Variable Header – Publish acknowledgement.
  • PUBREC – Fixed Header / Variable Header – Publish received(QoS 2 publish received).
  • PUBCOMP – Fixed Header / Variable Header – Publish complete.
  • SUBSCRIBE – Fixed Header / Variable Header / Payload – Subscribes to a topic.
  • PINGREQ – Fixed Header – PING request.

MQTT Packet Sizes

The fixed header field consists of the control field, and the variable header contains the packet length field. The minimum size of a packet length field is 1 byte, which is for messages less than 127 bytes.

The maximum packet size is 256 MB. The minimum packet size is only 2 bytes with a single control field and a single packet length field. Smaller packets less than 127 bytes have a 1-byte packet length field. The Packets greater than 127 and less than 16383 use 2 bytes, and so on. 7-bits that are used with the 8th bit is the continuation bit.

MQTT Packet size

Control Field

The 8-bit control field is the principal byte of the 2-byte fixed header. It is divided into two 4 bit fields and contains all protocol commands and responses. The first four most important bits are the command or message type field, while the remaining 4 bits are used as control flags.

control field structure

Control Flags

While there are 16 possible flags, a limited is used. The Publish message makes the most use of these flags. The duplicate flag is used when a message is republished with QoS or 1 or 2. QoS flags are used when publishing to indicate a QoS level of -0,1,2. The Retain Message flag is also used when publishing a message.

Remaining Length

The Remaining Length is the number of bytes left in the current packet, including variable header and payload data. The Remaining length does exclude the bytes used to encode the Remaining Length.

The Remaining Length is encoded using a variable-length encoding scheme that uses one byte for values up to 127. The least seven important bits of each byte encodes the data, and the most important bit is used to indicate there are following bytes in the representation. Thus, each byte encodes 128 values and one “continuation bit”. The greatest number of bytes in the remaining length field is four.

Size of the remaining length field

Variable Header

Some types of MQTT control packets have a variable header component. Variable-length header fields are not always present in MQTT messages. Some MQTT message types or commands require this field to carry additional control information. It remains between the Fixed Header and the Payload. The Variable Header will vary depending on the packet type.

Example: MQTT Connect Message Structure

As an example, let’s take a look at the details of the CONNECT message packet in the Bevywise MQTTRoute / MQTTBroker.

After establishing a connection between the MQTT client and the broker, the first packet must be a CONNECT packet. The CONNECT packet only needs to be sent once over the network connection. The second CONNECT packet sent by the MQTT client is ignored and disconnected.

Fixed Header

Fixed header

Variable Header

The variable header must have four parts. They are the protocol name bytes, protocol level, connect flags, and keepalive.

Protocol name bytes

MQTTRoute will disable false protocol name.

Protocol name bytes

Protocol level

The protocol level for MQTTRoute is 4, other values lead to disconnection.

Protocol level

Connect flags

This field indicates the presence or absence of data in the payload.

Connect flags

Keep Alive

The MQTT client should send control packets that do not exceed the value of the defined Keep Alive. The Keep Alive value depends on the duration of the control packet of one transmission by the client.

Keep alive

MQTT Protocol Payload

The payload contains the MQTT client credentials (username, password, etc.). The client ID must be unique for each client. The broker responds with a unique client ID for each client. If you send a client ID in an empty field, the client will be rejected. The fields must be filled in the following order: Client ID, Will Topic, Will message, username, password.

The above fields must be in string format. Any packet that does not send a CONNECT packet after a reasonable amount of time will be rejected by the broker. After successful validation, the broker will respond in 2 format

  • Check if the client already exists. If so, the broker will respond with the disconnection.
  • The MQTT Broker will send a CONNACK packet with zero value.

After successful CONNACK, the broker will do a keepalive monitoring regularly.

To know more about the MQTT messages or packets structure, do visit the MQTT Developers page.

To know about the format of the MQTT-SN packets, visit the MQTT-SN Developers page.

Download the forever free version of the MQTTRoute now to validate all the features.

Download MQTT Broker

You can upgrade to the premium version of MQTTRoute for the full-fledged functionalities after your free trial period.

Effective use of MQTT Last WILL Messages

Effective use of MQTT Last WILL Messages

by | Mar 14, 2017 | MQTT

Every IoT network is a collaborative working of edge devices. All devices should work in unison to make the environment conducive for the humans. This article helps you make the best use of the MQTT Last WILL Messages.

Last Will is a way of peer to peer monitoring between MQTT clients that operate in a more understanding way. Before we go into the real time use cases, it is better to understand the spec better.

What is the MQTT Last WILL ?

Last Will is a message stored in the MQTT broker specific to particular clients. The Last Will message is a normal MQTT Message that has a will topic, will message , MQTT QoS and retain flag. These messages will be sent to the clients who have subscribed to those WILL topics by the broker when the particular client disconnects ungracefully.

The Last WILL Message should be sent to the broker when the clients connect to the broker via the Connect packet.

A more detailed explanation of how you have to send the connect packet is available here.

When it will be published ?

The WILL Messages will be published by the broker to all the interested subscribers when the particular client gets disconnected from the broker ungracefully. When the client disconnects using a proper disconnect message, the WILL will not be published. The client needs to notify the interesting parties on going down by publishing proper message by itself.

Putting it to use

All mission critical devices needs a back up. Let us take an example of a Database server in an online application. It is one of the core component that needs to run always for the complete service to be useful.

These applications will have masters , slaves and standby servers which needs to be deployed when any of the servers fails. The master server should add a WILL on Connection. The slave will be subscribing to the particular WILL. The WILL should be configured specific to each cluster of the master slave. In this scenario, the broker should be configured to publish the WILL for the graceful as well as ungraceful disconnection of the Master. When the master disconnects, the slave receives the master unavailable message and the necessary scripts can be triggered for the slave to become the master. The standby servers will  become slave and now will subscribe to the WILL of the new master.

This is one scenario of automated take over. Such scenarios exist in all industries like Medical, all kind of Manufacturing industries. We can also use the WILL Message triggers for monitoring the devices and their uptime from a central server.

You can avoid a lot of unwanted monitoring messages to each devices by making the right use of MQTT Last WILL Messages.

MQTT QoS level – What to use & When

MQTT QoS level – What to use & When

by | Feb 22, 2017 | MQTT

What is Quality of Service (QoS)?

MQTT protocol for IoT defines three Quality of Service (QoS) levels for message delivery. The MQTT QoS level is used when a client sends a subscribe request to a particular topic. When a client subscribes to multiple topics, it can use different QoS level for different topics based on the need.

By considering the message delivery, when the client publish messages to the broker, it starts sending a message with QoS levels. The MQTT broker, a publish subscribe model will pass on the messages to subscribing client with the QoS level for which the subscribing client designates.

The amount of data we are going to receive is going to be very large. It is very important we reduce the noise and just give more importance to the signals.

What does each QoS level means ?

QoS 0 — At most once and no guarantee of delivery

QoS 1 — Delivered at least once. It has the possibility of getting the messages multiple times.

QoS 2 — Delivered Exactly once.

QoS 0 – At Most Once

The entry-level or first level is QoS level 0. Also called “Fire and Forget“. This level ensures that the message is sent to the recipient only once or never. There won’t be any guarantee of message delivery that is the receiver may or may not receive the message. Broker won’t provide any acknowledgement for which it delivers the message.

QoS 1 – At Least Once

The next level in QoS is 1. This level ensures message must be sent to the recipient atlease once. There is a possibility of the message delivered multiple times to the receiver until the acknowledgement is sent by the broker to the receiver. The message will be stored by the publisher and it waits for receiver to send PUBACK packet. The packet identifier is used by the publisher to pair the PUBLISH packet to the suitable PUBACK packet. Within a certain time, if the publisher doesn’t receive PUBACK packet, then it resends the PUBACK packet.

QoS 2 – Exactly Once

The third level and the final level of QoS is 2 which is the referred as the highest level of service. This level ensures that the recipient receives each message only once. This is the safest quality of service level as atleast two response flows between publisher and receiver provides message guarantee. The delivery of message will be organized by the sender and the receiver using packet identifier of native PUBLISH message. The publisher sends the QoS 2 PUBLISH packet to the receiver. Then the recipient processes the publish messages suitably and acknowledges PUBLISH packet by sending the PUBREC packet to the publisher. If the receiver fails to send an acknowledgement, then the publisher will continue sending the PUBLISH packet with the duplicate flag (dup flag) till it receives PUBREC packet.

Once the publisher receives PUBREC packet it stores it and replies with the PUBREL packet. Now the recipient will respond with PUBCOMP packet after processing the messages. To stop recipient from processing the messages again and again, the reference to packet identifier of initial PUBLISH message will be stored by receiver.

The sender will have the confirmation for delivery once the entire QoS 2 flow gets completed.

Using it in Real time

Take an example of a Fuel level sensor in an manufacturing or an automobile.

The sensors sends data to an indicator which can show the current level of the fuel. The indicator showing the current level just need the live data. It does not need any of the past data and even if it misses to receive a few instances, it does not matter. You can use the QoS 0 (At most Once ) for the Fuel level Indicator.

It can also send data to a cut off valve during filling and fuel low warning when the level recedes the lower optimum level. The fuel cut off or the lower level optimal should not miss the data at any cost. We should use the QoS with the perfect delivery for the warning lights to glow. This mandates the usage of QoS 2 (Exactly Once). Even though the past data is not required in this cases, the exact delivery is a must.

If you build a system that records all the data for future analysis. For example, if we want to study the fuel levels with respect to time, we need all the data to be available for the big data analyser engine. For logging of data and storage, either you should build a engine alongside the broker or you can connect an agent with a QoS 2 (Only Once) and record all the data remotely.

The difference in QoS between clients

The quality of service attributing the messages published is between two clients connected to the server or the broker, not an end to end. Thus, the QoS level of the messages received by the client (subscriber) is determined as per the QoS level of messages published and the QoS level of subscribed topics. In short, the final QoS of received messages depends on the QoS of publishing & subscribing clients.

When connected to the broker, the subscribing client will instruct the broker with QoS attributed message to publish messages with a QoS attribute it needed.

The following table provides the schema which the client uses in different scenarios.

QoS of Messages Published QoS of Messages Subscribed Final QoS of Messages Received
2 0 0
2 1 1
2 2 2
1 0 0
1 1 1
1 2 1
0 0 0
0 1 0
0 2 0

Thus, the Qos of received messages is similar to the QoS of published & subscribed messages.

Don’t Ignore everything as Noise

The determination of whether the data is a signal or the noise help decide the QoS level to be used. When we do data publishing and collection in mission critical industry like Healthcare, Nuclear, Defence, etc., every data is important and it mandates the use of QoS 2 for every message. QoS 1 can be used instead of QoS 2, if your client can handle duplicate message.

A more detailed document on developing MQTT clients will help you create your own mqtt clients to connect with any MQTT broker.

Leave a comment or write to [email protected] for any questions or suggestions.

To Learn more about the protocol,

5 reasons "Why we choose MQTT"

5 reasons "Why we choose MQTT"

by | Aug 8, 2016 | MQTT

IoT network finds application in day-to-day essentials from Home appliances, Public safety, Farming and Hospitals to high precision jobs including Smart cities, Automobiles, Industrial Automations and Satellite tracking.  After analysing the available protocols, we decided to go with the MQTT protocol (MQ Telemetry Transport) for our communication system.

Bevywise Networks was started with the aim to create IoT devices, sensors and softwares as mobile applications. Professionals and regular persons can use this around the world to automate and simplify their social, business and personal needs.

So We wanted our devices to communicate in a secure and reliable way.  Hence We believe “Quality & Reliability is no more a luxury, It should be something built inside”.

Here are the 5 things about MQTT protocol which made us to choose it for our IoT implementation.

1. Security

Even though MQTT messaging uses an unsecured TCP, we can be able to encrypt data with TLS/SSL Internet security to make it robust, when implementing for the mission critical business. So that We can have partial and complete encryption based on the resourcefulness of the system and security mandate.

2. Central Broker

We may be getting billions of devices on the Internet over the next 5 to 10 years. MQTT Broker which can act as a server can effectively reduce the number of packets that fall into the internet and also the amount of processing the individual memory needed for the clients. We should be able to build a grid of highly interoperable brokers across different vendors.

3. MQTT protocol – QoS

MQTT defines three MQTT QoS levels which can cater to based on the importance of each messages and the repetitiveness of the messages in the environment.

At Most Once – Client configured with QoS level 0 will publish messages only once. This is just a confirmation message and the receiver client  will not  store or respond to it.

At least Once – Client configured with QoS level 1 will publish messages more than once. This message will be stored by the sender until a confirmation message is received from the other Client.

Exactly Once – Client configured with QoS level 2 ensures publishing the message exactly once. This is the most secure level of publishing messages.

4. Last WILL & Retained Message

Last WILL helps in knowing whether the particular client is available or not. It is not worth waiting for something that won’t happen. The listeners can be put on the power saver mode with interval based wake up to check the publisher availability.

Retained messages will help subscribers receive messages that were published some time before.

These messages highly decouple both the publisher and the subscriber to work independent of each device.

5. Flexible Subscription pattern

A Particular client can subscribe to all the topics published based on a pattern. For example, a smaller monitor for kitchen can listen to all topics on kitchen by subscribing to

Topic Name: home/kitchen/+’+’ represents various sensors in kitchen (eg. ‘+’ can be temperature sensor, motion detect sensor, etc.)

In a similar fashion, a client can subscribe to all the temperature level of the house.

Topic Name: home/+/room_temp’+’ represents various rooms with temperature sensor (eg. ‘+’ can be Bedroom, Hall, Kitchen etc.)

Not only these, there are many features of MQTT which benefits us. We have released the beta version of the IoT simulator for the MQTT Protocol. Please download and give it a try and visit our documentation to know more.