Developing MQTT Clients – Guide

Introduction

MQTT Client development guide will help you to develop clients as per the Oasis Standard of MQTT Protocol . The developed clients can be connected to the MQTTRoute , a message broker which follows the OASIS standard MQTT version 5.

Little on MQTT

Before we jump into MQTT clients, let us know little about MQTT basics. MQTT is a publish/subscribe & lightweight messaging protocol responsible for establishing a client broker connection & communication. This feature made MQTT suitable for Machine 2 Machine communication and the Internet of Things, where a small code footprint is required, and network bandwidth is precious. MQTT’s support for persistent sessions , enable devices connect over reliable network. The protocol runs over TCP/IP. The publish/subscribe message pattern provides one or too many message distribution and decoupling of MQTT clients . Other necessary features include quality of service levels, Last Will & Testament and Retained messages , MQTT over websocket .

Getting Started

Machine-to-Machine, Machine-to-User, Machine-to-Mobile devices communication cannot happen without a powerful central server. MQTT edge devices / IoT devices and applications communicate real time via the MQTT Broker which acts as the central server. You can use any cross platform / open source MQTT Broker or download the Enterprise MQTT Broker for FREE which can run on Windows and Linux now and get started.

Some of the MQTT Clients that MQTTRoute support to test MQTT Broker are Eclipse Paho , Eclipse Mosquitto Client , Bevywise MQTT Clients, MQTT X , and MQTT Explorer , etc from MQTT client library available in an immense range of programming languages or using mosquitto_pub and mosquitto_sub command line MQTT clients. You can connect it using a local IP address or hostname of your MQTT Broker.

How the MQTT Client should be

The principal use of MQTT clients is to establish a connection with the MQTT Broker to subscribe to topics for publishing and receiving messages. Here are some of the specific characteristics that every MQTT Client should possess.

• MQTT Client should be flexible for configurations that must support client authentication, facilitate configure certificates, various parameters while connecting, publishing & subscribing in the MQTT process, etc.
• Experiencing user interface in a free-flowing manner and improving user interaction handily on a full range of features.
• Should be available for different operating systems.

Know more about this by diving into the content below. We’ve classified this into three different categories namely MQTT Data Representation, MQTT Control Packet Format, and MQTT Control Packets.

1. MQTT Data Representation

MQTT is a binary-based protocol. The control elements are not text strings whereas binary bytes. All the MQTT commands have a related acknowledgement.

1.1 Bits

It explains bits. These Bits in a byte are flagged 7 to 0. Bit number 0 is the least significant bit and Bit number 7 is the most significant bit.

1.2 Two Byte Integer

MQTT defines the two Byte Integer data values. i.e, The higher-order byte (MSB-Most Significant Bit) is followed by the lower-order byte (LSB-Least Significant Bit).

The higher-order byte (MSB-Most Significant Bit) is followed by the next higher-order byte (MSB-Most Significant Bit), followed by the next higher-order byte (MSB-Most Significant Byte), followed by the lower-order byte (LSB-Least Significant Bit).

1.3 Four Byte Integer

MQTT describes the 32-bit unsigned integers in big-endian order are the two Byte Integer data values. i.e, The higher-order byte (MSB-Most Significant Bit) is followed by the next higher-order byte (MSB-Most Significant Bit), followed by the next higher-order byte (MSB-Most Significant Byte), followed by the lower-order byte (LSB-Least Significant Bit).

1.4 UTF-8 Encoded String structure

UTF-8 is the encryption method for Unicode. It can translate any Unicode character to a unique binary string that can be matched and can translate the binary string back to Unicode characters. The UTF-8 encoded string maximum size is 65,535 bytes and also it ranges from 0 to 65,535 bytes. The text fields in the MQTT control packets described later are encrypted as UTF-8 strings. In specific, the character data should not include encryption of code points between U+D800 and U+DFFF, and the null character encryption of code points is U+0000. If the client or server receives an MQTT control packet containing incorrectly generated UTF-8, then it is considered a false packet. This guide gives the clear view of the UTF-8 encoded string structure.

• U+0001..U+001F control characters
• U+007F..U+009F control characters
• Code points defined in the Unicode specification are non-characters (for example U+0FFFF)

1.5 Variable Byte Integer size

It uses a single byte value up to 127. Using Encoding Scheme the variable byte integer encoded is clearly explained in this MQTT guide. In LSB, the 7 bits of each byte encoded the data and the MSB is used to specify whether the bytes in the representation are as follows. Thus, each byte encodes 128 values and it is a “continuation bit”. In the Variable Byte Integer field, the maximum number of bytes is four. Use the minimum number of bytes required to denote the coded value.

2.MQTT Control Packet Format

2.1 Packet Structure

The MQTT protocol works by exchanging consecutive MQTT control packets in a defined way. This section describes the format of these packets. An MQTT control packet structure consists of three parts, which are shown in the below table.

MQTT control packet structure

2.1.1 Fixed Header

The below table consists of the fixed header which could be contained in each MQTT control packet.

Fixed Header format

2.1.2 MQTT Control Packet Type

The bits [7-4] of byte 1 in the fixed header contain the control packet types. In MQTT control packet types are represented as a 4-bit Unsigned value, these values are shown in the below table.

MQTT control packet types

2.1.3 Flags

The bits [3-0] of byte 1 is the fixed header that contains specific flags for every MQTT control packet as shown below. If a flag bit is marked as “Reserved” it should be reserved for future use and set the listed value. If invalid flags are received, then it is considered a false packet.

2.1.4 Remaining Length

The position starts at byte 2. The remaining length is a variable byte integer clearly explained here, which refers to the number of bytes remaining in the current control packet, including data in the variable header and payload. No bytes are used to encode the remaining length. The total number of bytes in an MQTT control packet is the packet size, which is equal to the length of the fixed header and the remaining length.

2.2 Variable Header

Some MQTT control packets have a variable header component. This is between the Fixed Header and the Payload. The content of the variable header will vary depending on the type of packet. The packet identifier field of the variable header is common to many packet types.

2.2.1 Packet Identifier

Two-byte integer packet identifier fields are included in the variable header component of many MQTT control packets. These MQTT control packets are PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK. MQTT control packets that contain a packet identifier shown in the below table.

A publish packet should not have a packet identifier if its QOS value is set to 0. Each time a client sends a new SUBSCRIBE, UNSUBSCRIBE or PUBLISH MQTT control packet (QoS>0), it must assign the currently unused zero packet identifier. Each time a server sends a new PUBLISH MQTT control packet with (QoS>0), must assign it a currently unused zero packet identifier.

The packet identifier is available for reuse after the sender activates the corresponding approval packet defined as follows. In the case of QoS1 PUBLISH, it is the corresponding PUBACK; for QoS2 releases, it is either PUBCOMP or PUBREC with a reason code of 128 or higher. This is the SUBACK or UNSUBACK related to SUBSCRIBE and UNSUBSCRIBE. A PUBACK, PUBREL or PUBCOMP packet must have the same packet identifier as the original PUBLISH packet. A SUBACK and UNSUBACK must have a packet identifier used in the corresponding SUBSCRIBE and UNSUBSCRIBE.

The packet identifiers used with the PUBLISH, SUBSCRIBE, and UNSUBSCRIBE packets generate separate and integrated identifiers for the client and server distinctly in one session. At any time, a packet identifier can not be used by more than one command. The client and the server assign packet identifiers to each other independently. As a result, client-server pairs can participate in simultaneous messaging using the same packet identifiers.

2.2.2 Properties

The property set is the last field in the variable header of the MQTT control packets such as CONNECT, CONNACK, PUBLISH, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK, DISCONNECT, and AUTH. In the CONNECT packet, there is a Will properties field with the payload, which is considered as an optional set of properties. The set of properties is made up of a property length, followed by properties.

2.2.2 Properties

The property length variable is encoded as a byte integer. The property length does not contain the bytes used to encrypt itself but also the length of the attribute. If there are no properties it should be denoted by adding the property length of zero.

2.2.2.1 Property

Property has an identifier that defines its use and data type followed by a value. The identifier is encoded as a variable byte. A false packet is when a control packet has an invalid identifier for its packet type or does not have the value of a specific data type. If obtained, use a CONNACK or DISCONNECT packet with the Reason code (0X81) described as an invalid packet. There is no importance in the order of properties with different identifiers.

2.3 Payload

The payload is the final part of the packet in some of the MQTT control packets. In the PUBLISH packet, the payload is the application message. MQTT control packets that require a payload.

2.4 Reason Codes

A Reason Code is a one-byte unsigned value. Reason code indicates the result of an operation. Reason codes less than 0x80 specifies the successful completion of an operation. The normal reason code for success is 0. Reason codes greater than 0x80 specifies failure. Some of the MQTT Control Packets have a single Reason Code as part of the Variable Header such as CONNACK, PUBACK, PUBREC, PUBREL, PUBCOMP, DISCONNECT, and AUTH. The two MQTT control packets contain a list of one or more Reason Codes in the Payload, which are SUBACK and UNSUBACK.

3. MQTT Control Packets

There are 15 MQTT Control packets in MQTTv5

1. CONNECT – Connection establishment between an MQTT Client and Broker

2. CONNACK – The connection request acknowledgement

3. PUBLISH – Publish message

4. PUBACK – Publish acknowledgement

5. PUBREC – Publish received(QoS 2 publish received)

6. PUBREL – Publish release(QoS 2 publish received)

7. PUBCOMP – Publish complete

8. SUBSCRIBE – Subscribe to topics

9. SUBACK – Subscribe acknowledgement

10. UNSUBSCRIBE – Unsubscribe from a topic

11. UNSUBACK – Unsubscribe acknowledgement

12. PINGREQ – PING request

13. PINGRESP – PING response

14. DISCONNECT – Disconnection between the MQTT Client and Broker

15. AUTH – Authentication exchange

3.1 CONNECT

After a client has established a network connection to a server, the first packet sent from the client to the server must be a connection packet. A client can send the CONNECT packet only once via a network connection.

3.1.1 CONNECT Fixed Header

Remaining Length – This is the length of the variable header and the length of the payload. It is encoded as a variable byte Integer.

3.1.2 CONNECT Variable Header

The variable header for the connect packet contains the following fields: Protocol Name, Protocol Level, Connect Flags, Keep Alive, and Properties.

3.1.2.1 Protocol Name

A server that supports multiple protocols uses the protocol name to determine whether the data is MQTT. The Protocol Name must be the UTF-8 Encoded String that represents “MQTT”.

3.1.2.2 Protocol Version

A server that supports multiple versions of the MQTT protocol uses the protocol version to determine which version is used by the MQTT client.

3.1.2.3 Connect Flags

Connect Flags specifies the presence or absence of the fields in the Payload.

3.1.2.4 Clean Start

A clean start is in bit 1 of the Connect Flags byte. If a connection packet with clean start is set to 1, the client and server must reject any existing session and start a new session.

3.1.2.5 Will Flag

Will Flag is in bit 2 of the Connect Flags. If the Will Flag is set to 1, it indicates that a Will message should be stored on the server and core correlated with the session. The Will message contains the Will properties, Will topic, and Will payload fields in the CONNECT Payload. The will message is published in some situations, that is shown below,

• Servers detected I / O error or network failure.
• The client failed to communicate while Keep alive.
• The client closes the network connection without sending a disconnected packet first with a Reason Code 0x00 (default disconnection).
• The server closes the network connection without receiving a disconnected packet with Reason Code 0x00 (default disconnection).

3.1.2.6 Will QoS

Will QoS is in bits 4 and 3 of the Connect Flags. When the will message is published, then the two bits that specify the QoS level are used. If the Will Flag is set to 1, then the value of the Will QoS maybe 0 (0x00), 1 (0x01), or 2 (0x02).

3.1.2.7 Will Retain

Will Retain is in bit 5 of the Connect Flags. If the Will Flag is set to 1 and the Will Retain is set to 1, the server will publish the Will message as a retained message.

3.1.2.8 User Name Flag

User Name Flag is in bit 7 of the Connect Flags. A Username must be in the payload when the User Name Flag is set to 1.

3.1.2.9 Password Flag

Password Flag is in bit 6 of the Connect Flags. The password must be in the payload when the Password Flag is set to 1.

3.1.2.10 Keep Alive

The MQTT client must send the control packets within the defined Keep Alive value. The Keep Alive value depends on the client’s control packet transaction duration

3.1.2.11 CONNECT Properties:

Property Length – The length of the properties in the MQTT CONNECT control packet.

Session Expiry Interval – The indicator of Session Expiry Interval is 17(0X11) Byte. If the Session Expiry Interval is used more than once then it is considered to be a protocol error. If it is set to 0 then the session ends, when the network connection is closed or it is set to 0XFFFFFFFF then the session not expire.

Receive Maximum – The indicator of Receive Maximum is 33(0X21) Byte. It applies only if the network connection is open. If the Receive Maximum is 0 then it takes the defaults as 65,535.

Maximum Packet Size – The indicator of Maximum Packet Size is 39(0X27) Byte. There is no limit in the packet size when the maximum packet size is absent.

Topic Alias Maximum – The indicator of Topic Alias Maximum is 34(0X22) Byte. If the Topic Alias Maximum is used more than once then it is considered to be a protocol error.

Request Response Information – The indicator of Request Response Information is 25(0X29) Byte. The value 0 indicates that the Request Response Information is not present.

Request Problem Information – The indicator of Request Problem Information is 23(0X17) Byte. If the value is 1 then the packet is not present in the packet.

User property – The indicator of User property is 38(0X26) Byte. This property is allowed to used more than once in the same name also.

Authentication Method – The indicator of the Authentication Method is 21(0X15) Byte. If it is used more than once then it is considered to be a protocol error. The extended authentication does not perform when the authentication method value is 0.

Authentication Data – The indicator of the Authentication Data is 22(0X16) Byte. When the authentication data is used while there is no authentication method then it is to be considered as a protocol error. It does not use more than once in the packet. The CONNECT properties are plainly explained in this guide.

Variable Header Example

3.1.3 CONNECT Payload

The payload of the CONNECT packet contains one or more length – prefixed fields, the existence of which is determined by the flags in the variable header. If these fields are present, the client identifies Will Properties, Will Topic, Will Payload, Username, and Password should appear

Client Identifier (Client ID) – Client identifier must be a UTF-8 Encoded String. Each client that connects to the server has a unique Client ID. This MQTT session between the client and server should be used in MQTT clients to identify the position they hold.

Will Properties – The Will Properties are the next field in the payload, if it is set to 1.The Will properties fields defines the application message attributes that should be sent with a will message and the properties that define when the will message should be published. A Will Property has the property length and properties.

Property length – It defines the length of the properties in the Will Properties field.

Will Delay Interval – The indicator of Will Delay Interval is 34(0X22) Byte. If the Will Delay Interval is used more than once then it is considered to be a protocol error. If the Will Delay interval is not present, then there is no delay.

Payload Format Indicator – The indicator of the payload format is 1(0X01) Byte. 0(0X00) Byte is equivalent to not sending the payload format indicator that indicates the message is unspecified bytes. If it is used more than once then it is to be considered as a protocol error.

Message Expiry Interval – The Indicator of Message Expiry Interval is 2(0X02) Byte. If it is used more than once in the packet considered to be a protocol error. If it is present the value of four bytes will be in the life time of the Will Message, and the output expires when the server releases the Will Message. If it is not present when the server releases the optional message, then the message expiration interval will not be sent.

Content Type – The indicator of the Content Type is 3(0X03) Byte. The content type is included more than once in the packet considered to be a protocol error. The sending and receiving application is the value of the content type.

Response Topic – The indicator of the Response Topic is 8(0X08) Byte. If the response topic is not absent then it identifies the Will message as the request.

Correlation Data – The indicator of the Correlation Data is the 9(0X09) Byte. Correlation data is used by the sender of the request message to identify which request was received when the response message was sent. The requester client does not require any Correlation data when it is absent.

User Property – The indicator of User property is 38(0X26) Byte. This property is allowed to use more than once in the same name also.

Will Topic – The Will Topic is the next field in the payload, if the Will flag is set to 1. It must be a UTF-8 Encoded String

Will Payload – The Will Payload is the next field in the payload, if the Will flag is set to 1. The Will Payload defines the application message payload to be published for the Will Topic.

Username – The Username is the next field in the payload, if the Username flag is set to 1. This can be used for authentication by the server.

Password – The Password is the next field in the payload, if the password flag is set to 1. It can be used to carry any credentials.

3.2 CONNACK

CONNACK Packet is the packet sent by the server in response to a CONNECT packet received from the client. The Server must send a CONNACK with a 0X00 (Success) Reason Code before sending any other packet other than AUTH. The server should not send more than one connection acknowledgement over a network connection.

3.2.2 CONNACK Variable Header

CONNECT acknowledge flags, CONNECT reason code and properties are the field in the Variable Header of the CONNACK Packet.

CONNECT acknowledge flags – Set this field in byte 1. Bits 7 to 1 are booked and set to zero.

Session Present – The position of the session present is bit 0 of the CONNECT acknowledge flags. The session current flag informs the client whether the server is using server status from the previous connection of the clientID. This allows the client and server to have a consistent view of the session states. If the server accepts 1 with a clean start, the server most set the Reason Code 0X00(success) in the CONNACK packet and additionally set the session state 0 or 1 in the CONNACK packet.

CONNECT Reason Code – In the Variable Header, Byte 2 is the CONNECT Reason Code. If a server sends a CONNECT packet with a Reason Code of 128 or higher. it must close the network connection. The Server sending the CONNACK packet with one of the below CONNECT Reason Code.

3.2.3 CONNACK Properties

Some of the CONNACK properties are similar to the CONNECT properties which are property Length, Session Expiry Interval, Receive Maximum, Maximum Packet Size, Topic Alias Maximum, User property, Authentication method, and Authentication Data. For detailed information Refer to section- 3.1.2.11

Maximum QoS – The indicator of the Maximum QoS is 36(0X24) Byte adding the error. if the Maximum QoS is not available, the client uses the Maximum QoS of 2. In PUBLISH packet, if a server does not support QoS 1 or QoS 2 it must send the Maximum QoS in the CONNACK packet indicates the highest QoS it can support. A Server that does not support QoS 1 or QoS 2 publish packets still accepts subscription packets containing the requested QoS of 0, 1, or 2.

Retain Available – The Indicator of the Retain Available is 37(0X25) Byte. If a server receives a connection packet containing a Will Message set with a will flag 1, the server must reflect the connection request if it does not support retained messages. It must send CONNACK with reason code 0X9A (Retain not available) then close the network connection.

Assigned Client Identifier – The indicator of the Assigned Client identifier is 18(0X12) Byte. if the Client connects using a zero-length Client identifier the server must respond with a connection and a designated client identifier. The assigned client identifier must be a new client identifier that is not currently used in any other session on the server.

Reason String – The indicator of the Reason String is 31 (0X1F) Byte. It is a UTF-8 encoded string. It is a human-readable string designed for detectable and should not be discriminated against by the client. The server uses this value to provide additional information to the client. the server should not send this asset if the client increases the size of the CONNACK packet beyond the specified maximum packet size.

Wildcard Subscription Available – The indicator of the Wildcard Subscription is 40(0X28) Byte. if the value is 0, then the wildcard subscriptions are not supported. If the value is 1, then it is supported. The wildcard subscription is included more than once in the packet and the value other than 0 or 1 is considered to be a protocol error.

Subscription Identifiers Available – The indicator of the Subscription identifier is 41(0X19) Byte. If the value is 0 then the subscription identifiers are not supported. If the value is 1 then it is supported the subscription identifiers are included more than once and the value other than 0 or 1 is considered to be a protocol.

Shared Subscription Available – The indicator of the Shared Subscription Available is 42(0X2A) Byte. If the value is 0 then the subscription identifiers are not supported. If the value is 1 then it is supported the subscription identifiers are included more than once and the value other than 0 or 1 is considered to be a protocol.

Server Keep-Alive – The indicator of the Server Keep-Alive is 19(0X13) Byte. If the Server sends a Keep-Alive in the CONNACK packet, this value should be used instead of the Client Keep-Alive value sent on CONNECT Packet. If the server does not send the keep-alive, the Server uses the keep-alive value set by the Client on the CONNECT packet. If the Server Keep-Alive is included in the packet more than once then it is considered to be a protocol error.

Response Information – The indicator of the response information is 26(0X1A) Byte. It is a UTF-8 encoded string. It is mainly used for creating a response topic. If it is included more than once then it is to be considered as a protocol error.

Server Reference – The indicator of the Server Reference is 28(0X1C) Byte. It is a UTF-8 encoded string. If it is included more than once in the packet then it is to be considered as a protocol error. The Server uses the server reference in a connection or disconnection in the packet with Reason code 0X9C (use another server) or reason code 0X9D (server moved).

3.2.4 CONNACK Payload

The Control packet CONNACK has no payload.

3.3 PUBLISH

A PUBLISH packet is sent from the client to the server or vice versa to carry an Application Message.

DUP – The position of the Duplicate flag is byte 1, bit 3. If the DUP flag is set to 0 indicates that this is the first time, the Client or Server has attempted to send the PUBLISH packet. If the DUP flag is set to 1, that indicates it may be a redistribution of the previous attempt to send the packet.

QoS – The position of the QoS is byte 1, bits 2-1. This field indicates the delivery assurance of an application message.

RETAIN – The position of the RETAIN Flag is byte 1, bit 0. If the RETAIN Flag is set to 1, in the publish packet sent by a client to a server, the server should replace any existing message for the topic and save the application message. If the RETAIN Flag is 0 in the PUBLISH packet sent by a client to the server, the server does not store the message as a retained and removes or replaces any existing retained message.

Remaining Length – It defines the variable header and payload length.

3.3.2 PUBLISH Variable Header

Topic Name, Packet Identifier, and Properties are the fields in the variable header of PUBLISH packet.

Topic Name – Payload data identifies the topic name of the published information channel. The Topic Name in the PUBLISH packet variable header should be the first field. This should be the UTF-8 encoded string. The Topic Name in the PUBLISH packet should not contain wildcard characters. The Topic Name in the PUBLISH packet sent by the server to the subscribing client must match the subscriber’s topic filter

Packet Identifier – In PUBLISH packets the packet identifier field is present only where the QoS level 1 or 2.

PUBLISH Properties – The PUBLISH packet Properties are similar to the CONNECT and CONNACK. The Properties are Property Length, Payload Format Indicator, Message Expiry interval, Topic Alias, Response Topic, Correlation Data, User Property, Subscription identifier, and Content -Type. For detailed information Refer to sections –3.1.2.11, 3.2.3.

3.3.3 PUBLISH Payload

There is an application message that is published on the payload. The content and format of the data depending on the application. The difference between the Length of the variable header and the remaining length field in the fixed header is the length of the payload. It is valid to have a published packet with a zero-length payload.

3.3.4 PUBLISH Actions

3.4 PUBACK – PUBLISH Acknowledgement

The response of the PUBLISH packet with QoS 1 is the PUBACK packet.

3.4.1 PUBACK Fixed Header

3.4.2 PUBACK Variable Header

Packet Identifier from the PUBLISH packet, PUBACK Reason Code, property length, and the properties are the fields in the Variable Header of the PUBACK packet

3.4.2.1 PUBACK Reason code

The PUBACK Reason code in the Byte 3. There is no reason code and the value of (0X00) (Success) is used when the remaining length is 2.

3.4.2.2 PUBACK Properties

Property Length – PUBACK packet variable header properties length is encoded as a variable byte integer. if the remaining length is less than 4, the property has no length and a value 0 is used.

User Property – The User Property is used to provide more additional information to the client. For more information Refer to section – 3.1.3

Reason string – It is used to provide all the information to the Client about why the network connection is closed. For more information about reason String Refer to section – 3.2.3

3.4.3 PUBACK payload

The PUBACK control packet has no payload.

3.4.4 PUBACK Actions

In the QoS 1 delivery protocol, the sender should handle the PUBLISH packet to be considered as “Un acknowledge” until the corresponding PUBACK packet is received from the receiver. In the QoS 1 delivery protocol, the client accepts the right of the application message and responds with a PUBACK packet containing the packet identifier from the incoming PUBLISH packet. After it sends a PUBACK packet the receiver should consider any incoming PUBLISH packet containing the same packet identifier as a utility message regardless of the structure of its DUP flag.

3.5 PUBREC – PUBLISH Received

The PUBLISH packet with QoS 2 is the response of the PUBREC packet. In QoS 2 protocol exchange PUBREC is the second packet.

3.5.1 PUBREC Fixed Header

3.5.2 PUBREC Variable Header

3.5.2.1 PUBREC Reason Code

3.5.2.2 PUBREC Properties

Property Length – PUBREC packet variable header properties length is encoded as a variable byte integer. if the remaining length is less than 4, the property has no length and a value 0 is used.

User Property – User Property is used to provide more additional information to the client. For more information Refer to section – 3.1.3

Reason string – It is used to provide all the information to the Client about why the network connection is closed. For more information about reason String Refer to section- 3.2.3

3.5.3 PUBREC Payload

The control packet of the PUBREC packet has no payload.

3.5.4 PUBREC Actions

3.6 PUBREL – PUBLISH Release

The response of the PUBREC packet is the PUBREL packet. The QoS 2 protocol exchange’s third packet is the PUBLISH Release packet.

Bits 3,2,1 and 0 of the Fixed Header are assigned in the PUBREL packet, which must be set to 0,0,1 and 0 respectively. Remaining Length – length of the variable header.

3.6.2 PUBREL Variable Header

The Packet identifier from the PUBREC packet, PUBREL Reason Code, and properties are the fields in the Variable Header of the PUBREL packet.

3.7 PUBCOMP – PUBLISH Complete

The response of the PUBREL packet is the PUBCOMP packet. The QoS 2 protocol exchange’s fourth and final packet is the PUBLISH Complete packet.

3.7.1 PUBCOMP Fixed Header

3.7.2 PUBCOMP Variable Header

3.7.2.1 PUBCOMP Reason Code

The PUBCOMP Reason code in the Byte 3. There is no reason code and the value of (0X00) (Success) is used when the remaining length is 2. The Client or Server must be sending the PUBCOMP packets with one of the values of the PUBCOMP Reason Code.

3.7.2.2 PUBCOMP Properties

Property Length – PUBCOMP packet variable header properties length is encoded as a variable byte integer. if the remaining length is less than 4, the property has no length and a value 0 is used.

User Property – The User Property is used to provide more additional information to the client. For more information Refer to section – 3.1.3

Reason string – It is used to provide all the information to the Client about why the network connection is closed. For more information about reason String Refer to section – 3.2.3

3.8 SUBSCRIBE

The SUBSCRIBE packet is sent to the server from the client to generate one or more subscriptions. Each subscription stores the client’s interest in one subscription or more subscription topics. The server sends the PUBLISH packets to the client to pass the application messages with maximum QoS to topics that are compatible with these subscriptions.

Bits 3,2,1 and 0 of the Fixed Header are assigned in the SUBSCRIBE packet, which must be set to 0,0,1 and 0 respectively. Remaining Length – length of the variable header and the length of the payload.

3.8.2 SUBSCRIBE Variable Header

The Packet identifier and properties are the fields in the Variable Header of the SUBSCRIBE packet.

3.8.2.1 SUBSCRIBE Properties

Property Length – The SUBSCRIBE packet variable header property length is encoded as a variable byte integer.

Subscription Identifier – The indicator of the Subscription identifier is 11 (0X0B) Byte. if the subscription identifier has the value as 0 and it is included in the packet more than once then it is to be considered as a protocol error. It can have a value from 1 to 268,435,455.

User Property – User Property is used to provide more additional information to the client. For more information Refer to section – 3.1.3

3.8.3 SUBSCRIBE Payload

The SUBSCRIBE packet payload contains a list of topic filters that identify which topics the client wants to subscribe to. Byte of subscription options following the topic filter. The Topic Filter is a UTF-8 encoded string. The SUBSCRIBE payload must contain at least one topic filter and a couple of subscription options.

3.8.3.1 Subscription Options

The Maximum QoS field is represented by bits 0 and 1 of subscription options. It is to be considered as a protocol error if the value of the maximum QoS field is 3. No Local Option is represented by bit 2 of subscription options. If No Local Bit is set to 1, considered a protocol error. Retain as published Option is represented by bit 3 of subscription options. If the RETAIN flag is set to 0, then the Application Messages are forwarded using these subscription options. If the RETAIN is set to 1, then the Retained Messages are sent when the subscription option is initiated. Retain Handling Option is represented by bit 4 and 5 of subscription options. When the subscription is established then the retained messages are sent. The value of Retain Handling option is 3 then it is a protocol error. The values and the description of the subscription options are,

0 = Send Retained Messages during Subscription.

1 = send Retained Messages to Subscription only if Subscription is not present.

2 = Do not send Retained Messages during Subscription.

Subscription Options payload Format :

RAP – Retain as Published.
NL – No Local.
Payload Byte Format :

3.8.4 SUBSCRIBE Actions

The server must respond with the SUBACK packet when it receives the SUBSCRIBE packet from the client. The SUBSCRIBE and SUBACK packets' packet identifier are the same.

3.9 SUBACK – SUBSCRIBE Acknowledgement

The Server sent the SUBACK packet to the client for the confirmation of receipt and processing of the SUBSCRIBE packet sent by the client. The list of Reason Codes contained by a SUBACK packet specifies the maximum QoS level requested by the SUBSCRIBE.

3.9.1 SUBACK Fixed Header

3.9.2 SUBACK Variable Header

The Packet Identifier from the SUBSCRIBE packet and properties are the fields in the packet of the SUBACK Variable Header.

3.10 UNSUBSCRIBE

To unsubscribe from some topics, the client sends the UNSUBSCRIBE request to the server.

3.10.1 UNSUBSCRIBE Fixed Header

3.10.2 UNSUBSCRIBE Variable Header

The Packet Identifier and the properties are the fields in the Variable Header of the UNSUBSCRIBE control packet.

3.10.2.1 UNSUBSCRIBE Properties

Property Length – The UNSUBSCRIBE packet variable header property length is encoded as a variable byte integer.

User Property – User Property is used to provide more additional information to the client. For more information Refer to section – 3.1.3

3.10.3 UNSUBSCRIBE Payload

The UNSUBSCRIBE packet payload contains a list of topic filters that identify which topics the client wants to subscribe to. Byte of subscription options following the topic filter. The Topic Filter is a UTF-8 encoded string. The UNSUBSCRIBE control packet payload must contain one topic filter. If the UNSUBSCRIBE control packet has no payload then it is a protocol error.

3.10.4 UNSUBSCRIBE Actions

The Topic Filters provided in the UNSUBSCRIBE packet should be characterized by the current topic filter that the client has on the server. If any filter matches correctly, its subscription must be removed otherwise no additional processing will occur.

• It should stop adding new messages that are compatible with topic filters in order to deliver to the client.
• It should complete any QoS 1 or QoS 2 messages that are compatible with the topic filters and it has delivered to the client.
• It may continue to deliver any existing messages to the client.

3.11 UNSUBACK – UNSUBSCRIBE ACKNOWLEDGEMENT

The Server sent the UNSUBACK packet to the client for the confirmation of receipt and processing of the UNSUBSCRIBE packet sent by the client.

3.11.1 UNSUBACK Fixed Header

3.11.2 UNSUBACK Variable Header

The packet identifier from the UNSUBSCRIBE control packet and properties are the fields in the variable header of the UNSUBACK packet.

3.11.2.1 UNSUBACK Properties

Property Length – The UNSUBACK packet variable header property length is encoded as a variable byte integer.

Reason string – It is used to provide all the information to the Client about why the network connection is closed. For more information about reason String Refer to section – 3.2.3

User Property – User Property is used to provide more additional information to the client. For more information Refer to section- 3.1.3

3.11.3 UNSUBACK Payload

The UNSUBACK control packet payload carries a list of Reason Codes. The order of the topic filters in the UNSUBSCRIBE packet matches the order of the reason codes in the UNSUBACK packet. The UNSUBACK control packet payload must contain one of the reason codes for each topic filter.

3.12 PINGREQ – PING Request

The PINGREQ control packet is sent to the server from the client. The uses of PINGREQ packet, indicate to the server that the client is alive while no other MQTT control packets are sent from the client to the server. Request the server to respond to make sure it is alive. The Network exercise identifies that the network connection is alive. The PINGREQ control packet is used in the processing of the keep alive. For more information Refer to section 3.1.2.10

3.12.1 PINGREQ Fixed Header

3.13 PINGRESP – PING Response

The PINGRESP control packet is sent to the server from the client that specifies the server as active. It is the response of a PINGREQ control packet. The PINGRESP control packet is used in the processing of the keep alive. For more information Refer to section 3.1.2.10

3.13.1 PINGRESP Fixed Header

3.13.2 PINGRESP Variable Header

There is no variable header in the PINGRESP control packet.

3.13.3 PINGRESP Payload

There is no payload in the PINGRESP control packet.

3.13.4 PINGRESP Actions

The client takes no action in obtaining this PINGRESP control packet.

3.14 DISCONNECT

The final MQTT control packet is the DISCONNECT packet which is sent from the client to the server. It specifies the reason, why the network connection is closed. The MQTT clients sent this packet before closing the network connection. Before sending the DISCONNECT packet with reason code 0X00 (normal disconnection) the network connection is closed, then the connection has a will message that is published. For more details refer to 3.1.2.5

3.14.1 DISCONNECT Fixed Header

The client or server should check that the reserved bits are set to 0. The reserved bits value is non zero it sends a disconnect packet with reason code 0X81. For more details refer to section 3.13.

Remaining Length – length of the variable header.

3.14.2 DISCONNECT Variable Header

3.14.2.1 DISCONNECT Reason Code

The DISCONNECT reason code is byte 1 in the variable header. The value 0X00 is used when the remaining length value is less than 1. A byte unsigned reason code field values are shown below.

4. Operational Behavior

4.1 Session State

To enable QoS 1 and QoS 2 protocols the client and server must connect the state with the client identifier, referred to as the session-level. The server stores subscriptions as part of the session state. The session can continue in the order of network connections. This will last until the latest network connection and session expiration interval.
The session state on the client is as follows:

• QoS 1 and QoS 2 messages have been sent to the server, but they are not fully acknowledged (partially acknowledged).
• QoS 2 messages were received from the server but they are not fully acknowledged (partially acknowledged)

The session state on the server is as follows:

• The existence of the state session even if the remaining session state is empty.
• Sub client subscriptions including any subscription identifiers.
• QoS 1 and QoS 2 messages have been sent to the client but they are not fully acknowledged.
• Exchange pending QoS 1 and QoS 2 messages for a client and optionally QoS 0 messages pending transfer to the client.
• QoS 2 messages received from a client. However, the Will message and Will delay interval are not fully acknowledged.
• if the session is not currently connected, then the session state will be discarded

4.1.1 Storing Session State

If the network connection is open, then the client and server session status should not be discarded. If the network connection is closed and the session expires, the server must discard the session status.

4.2 Network Connections

The MQTT protocol requires basic traffic to deliver a stream of ordered, lossless, flow of bytes from client to server and server to client.

4.3 QoS and Protocol Flows

QoS refers to the amount of assurance for an application message delivery. The delivery protocol is only concerned with the delivery of an application message from one sender to another.

4.3.1 QoS 0: At most once

The message was delivered to the corresponding client but no reply was sent by the receiver. The message will arrive at the recipient once or not.

4.3.2 QoS 1: At least Once

The message was retrieved several times by the sender until an acknowledgement is obtained.

4.3.3 QoS 2: Exactly Once

The sender and receiver engage to ensure only one copy of the message is received.

4.4 Message Receipt

When a server takes ownership of an incoming application message it must be added to the session for the client with compatible subscriptions.

4.5 Message Ordering

• When the client returns any published packets, the original PUBLISH packets must be returned in the same order in which they were sent (this can be applied to QoS 1 and QoS 2 messages).
• The Client must send the PUBACK packet in the order in which they are received the PUBLISH packet by the sender (QoS 1 messages).
• The Client must send the PUBREC packets in the order in which the related packets are received by the sender (QoS 2 messages).
• The Client must send PUBREL packets in the order in which the relevant PUBREC packets are received (QoS 2 messages).

4.6 Topic Names and Topic Filters

4.6.1 Topic Wildcards

The Topic level separator is used to separate the topic name. If so, it divides the title by several “topic levels”. The subscription topic filter may contain special wildcard characters that allow a client to subscribe to multiple topics simultaneously. Wildcard characters can be used in topic filters, but not within a topic name.

4.6.1.1 Topic level Separator

The forward slash (‘/’ U+002F) is used to separate each level within a topic tree and to provide a hierarchical structure for the topic name. The use of the topic level separator is significant when one of the two wildcard characters in the topic filter is represented by the subscribing client. Topic level separators can appear anywhere in the topic filter or topic name.

4.6.1.2 Multilevel Wildcard

The number sign (‘#’ U+0023) is a wildcard character that applies to any position in a topic. The multilevel wildcard represents both parent and child levels. The multilevel wildcard character must be specified on its own or follow the topic level separator. In both cases, it must be the last character specified in the topic filter.

4.6.1.3 Single level Wildcard

The single-level wildcard can be used at any level in the header filter. where it is used, it should occupy the entire size of the filter. It can be used on multiple levels in the topic filter and can be used in conjunction with a multilevel wildcard.

4.6.2 Topic beginning with $

Topics name beginning with $ character but not beginning with # or + character. The server should prevent clients from exchanging messages with other clients by using the topic names which start with a leading $ character.

4.6.3 Topic Semantic and Usage

• Each Topic Names and Topic Filters must contain at least one long character.
• Topic Names and Topic filters can include space characters and which are case-sensitive.
• Topic Names and Topic Filters should not include Null characters (Unicode U+0000) and should not encrypt more than 65,535 bytes.
• A leading or backward ‘/’ creates a unique Topic Name or Topic Filter.
• A Topic Name or Topic Filter with only the character ‘/’ is valid

When making a subscription, which is compatible with the server, do not make any changes or alterations to the default or unauthorized characters in the Topic Name or Topic Filters

4.7 Subscriptions

MQTT delivers two types of Subscriptions.

• Non-Shared Subscriptions
• Shared Subscriptions

4.7.1 Non-Shared Subscriptions

In an Non-Shared Subscription, each client subscribed to a particular topic receives a copy of all messages that are published to the same subscribed topic. A session must have only one Non-Shared Subscriptions with the same topic filter, So the topic filter can be used as a key to recognize the subscription in that session. If the multiple clients have their own non-shared subscription to the same topic then each client gets the published own copy of the application message on that subscribed topic. This means that unshared subscriptions cannot be used to load balance application messages between multiple consuming clients in which case each subscriber will be provided with each message.

4.7.2 Shared Subscriptions

In a shared subscription, each message is delivered only to the subscribed topics of the client. A shared subscription is associated with multiple subscribing sessions of MQTT. Like the Unshared subscription which is a Non-shared subscription, it also has a Topic Filter and Subscription Options. However, an issue that is compatible with its Topic Filter will only be sent to one of its subscription sessions. Shared subscriptions are useful where multiple consuming clients share the processing of publications in parallel.

The format of the special style of Topic Filter to identify the shared subscription is :

$Share/{ShareName}/{filter}

Where,

$Share – The sequence of characters representing the shared subscription Topic Filter.

{Share Name} – Character string without “/”, “+” or “#”.

{filter} – The rest of the string contains the same syntax and semantics of a topic filter in the Non-Shared Subscription.

4.8 Flow Control

The optimization of the transmission rate of data across MQTT clients is done by flow control. By using receive maximum value the clients and servers control and receives the number of unacknowledged PUBLISH packets. The Receive Maximum establishes a maximum of one sending allotment which is used to control the number of PUBLISH QoS > 0 packets that can be sent without receiving PUBACK (for QoS 1) or PUBCOMP (for QoS 2). The client or server sets the initial send quota value which does not exceed the Receive Maximum. When sending the PUBLISH packets each time the send quota value is decreased. If the send quota value comes to zero, then the client or server cannot send the PUBLISH packet with QoS>0. Further, the client or server continue the process then actually the send quota value is incremented in two ways,

• Each time the retrieval of PUBACK or PUBCOMP packets nevertheless carried an error code.
• Each time the retrieval of PUBREC packet with a reason code of 0X80 or greater.

The send quota is already equal to the initial send quota value so we didn’t increase the value. The attempt to increment the value of send quota in the above case may cause re transmitting of a PUBREL packet after setting up a new network connection.

4.9 Request/Response

The request message to a particular topic is published by an MQTT client. The application message with a response topic is considered a request message. Another MQTT client (recipient) has subscribed to the topic filter, which matches the topic name of the corresponding MQTT client (sender) used when the request message is published. As a result, it received the request message. There may or may not be multiple respondents subscribed to this topic name. the respondent will take appropriate action based on the request message and then issue a reply message to the topic name in the response topic property carried over to the request message. In regular use, the requester receives the response message from the subscribed response topic. If some other client subscribed to the same response topic, then the client processed and received the response message.

4.10 Server Redirection

A server can petition the client to redirect another server by sending the MQTT control packets CONNACK or DISCONNECT with Reason code 0X9C or 0X9D. By using these Reason codes the server used by the client location is indicated using the Server Reference property. The Server reference is UTF-8 Encoded String. The Reason code 0X9C indicates that the client provisionally redirects to another server. The Reason code 0X9D indicates that the client eternally redirects to another server.

4.11 Enhanced Authentication

The MQTT CONNECT packet with username and password field supports the basic authentication of the network connection. Enhanced Authentication enlarges this basic authentication with challenge/response style authentication. The authentication is an asset between the MQTT clients. If the server receives the authentication method that does not support the method, then it may send a CONNACK control packet with the Reason code 0X8C or 0X87 and close the network connection.

4.11.1 Re-Authentication

If the client has provided the authentication method in the CONNECT control packet, it can re-authenticate at any time after receiving the CONNACK control packet. This is done by sending the AUTH packet containing the Reason code 0X19 (re-authentication). The client authenticates the value must be set to the same value as the method of authentication originally used to authenticate the network connection. The server acknowledges with the AUTH packet containing the Reason code of 0X00 (success – re-authentication complete) for the re-authentication request sent by the client or the server acknowledges with the AUTH packet containing the Reason code of 0X18 (continue authentication more data is needed). This flow continues until the re-authentication is complete or it fails. If the re-authentication fails, then the MQTT clients must send the DISCONNECT control packet containing the relevant Reason code and close the network connection.

4.12 Handling Errors

4.12.1 Malformed Packet and Protocol Errors

Some of the error cases are the size of the implementation of MQTT clients, capabilities that support, and consequences of constantly processing a misleading packet. If the sender complies with this identification, it will not send Malformed packets or cause protocol errors. However, if a client sends MQTT control packets before receiving CONNACK, it may cause a protocol error which may have led to a misconception about server capabilities.

Malformed Packets and protocol Errors Reason code are,

• 0x81 Malformed Packet
• 0x82 Protocol Error
• 0x93 Exceeding the Receive Maximum
• 0x95 Packet too large
• 0x9A Not Retained
• 0x9B QoS is not supported
• 0x9E Shared Subscriptions are not supported
• 0xA1 Subscription Identifiers are not supported
• 0xA2 Wildcard Subscriptions are not supported

If a server detects a Malformed Packet or Protocol Error and specifies a Reason code in the specification, it must close the network connection.

4.12.2 Other Errors

Errors other than malformed packet and protocol errors cannot be expected by the sender because the recipient may be barred from communicating with the sender. The receiving client or server may encounter an unstable error such as a lack of memory that prevents it from successfully executing an individual MQTT control packet. Acknowledgement packets containing a reason code of 0X80 or greater are PUBACK, PUBREL, PUBCOMP, SUBACK, UNSUBACK that specifies that the received packet detecting the error using packet identifier.

5. Using WebSocket as a network transport

The following conditions are applied, when the MQTT is transported over a WebSocket connection:

• MQTT control packets must be transmitted in websocket binary data frames. If any other type of data frame is received, the recipient must close the network connection.
• The single websocket data frame may contain multiple or partial MQTT control packets. The receiver should not assume that the MQTT control packets are aligned within the websocket frame boundaries.
• The client must add “mqtt” to the list of WebSocket sub-protocols it provides
• WebSocket sub protocol name must be selected and withdrawn by the server “mqtt”.