WO2022031162A1

WO2022031162A1 - A system and method for mqtt client based high availability

Info

Publication number: WO2022031162A1
Application number: PCT/MY2020/050183
Authority: WO
Inventors: Binti Khalid PUTRI SHAHNIM; Bin Yaacob AZMI; Bin Abu Bakar AHMAD ZAKI
Original assignee: Mimos Berhad
Priority date: 2020-08-03
Filing date: 2020-11-30
Publication date: 2022-02-10

Abstract

The present invention discloses a method (100) for MQTT client based high availability of data transfer within a scenario; comprising at least one MQTT client connected to a server, and is associated with a topic for the scenario; at least one MQTT broker connected to the server, is associated with the topic for the scenario. The method further discloses establishing a connection between the at least one MQTT client to a cluster of MQTT brokers, the cluster of MQTT brokers include a plurality of brokers; directing the cluster of MQTT brokers by the MQTT client via an update module to manage the MQTT broker; monitoring the transmission by the MQTT client via a data sender module to manage the data transmission; and defining a set the rules for carrying out load balancing by the MQTT client via an application defined load balance engine. A system is also disclosed herein.

Description

A SYSTEM AND METHOD FOR MQTT CLIENT BASED HIGH AVAILABILITY

FIELD OF INVENTION

The present invention generally relates to management of data transmission. More particularly, the present invention relates to a system and methods for Message Queuing Telemetry Transport (MQTT) client based high availability of data transfer within a scenario.

BACKGROUND OF THE INVENTION

In recent world of networking technology, secured transmission of data is the prominent requirement. Furthermore, several advancements have been done to enhance the feasibility and efficiency of the data transmission. Contrary to that, there have been many circumstances where the security has been challenged in the open scenarios. In such cases to avoid infringers there are several governing who associates within two parties as a third party to secure the data transmission.

Furthermore, adhering to the above technique, there arise issues related to load bulk of the data being transferred such as duplicate files, redundancy challenged, and so forth. However, a secured technique (Message Queuing Telemetry Transport or MQTT) functioning as a third party or a virtual assurance, has been introduced to enhance the efficiency and security of data transmission. The MQTT includes a client and broker that receives all messages from the clients and then routes the messages to the appropriate destination clients. However, there are limitations with the operations and load balancing in the conventional methods and techniques to function the MQTT.

A prior art, US 105, 74619 by Harpas et al, aims to solve or address the current shortcomings of data transmission. However, Harpas does not teach or disclose an effective way to dynamically change load balancing behaviour at run-time, hence may still experience redundancy if deployed.

Accordingly, in light of foregoing discussion there is a need to overcome the above- mentioned limitations and drawbacks associated with the conventional methods and techniques to function the MQTT. SUMMARY OF INVENTION

In light of the limitations of the existing conventional systems as discussed above, it is evident that there arises a need of an efficient and secured data transmission technique. Furthermore, the present invention seeks to provide a system and method for MQTT client based high availability of data transfer within a scenario.

In one aspect, embodiments of the present disclosure provide a method for MQTT client based high availability of data transfer within a scenario. The method includes at least one MQTT client connected to a server, wherein the at least one MQTT client is associated with a topic for the scenario; at least one MQTT broker connected to the server, wherein the at least one MQTT client is associated with the topic for the scenario; establishing a connection between the at least one MQTT client to a cluster of MQTT brokers, wherein the cluster of MQTT brokers include a plurality of brokers; directing the cluster of MQTT brokers by the at least MQTT client via an update module to manage the MQTT broker; monitoring the transmission of data by the at least MQTT client via a data sender module to manage the data transmission; and defining a set the rules for carrying out load balancing by the at least MQTT client via an application defined load balance engine.

In an embodiment, establishing the connection between the at least one MQTT client to a cluster of MQTT brokers further includes retrieving address of a first broker by a first client 1 in the scenario under the given topic, wherein the first broker is available in the cluster of brokers; assigning the retrieved first broker as an anchor broker; connecting to the anchor broker and determining number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; updating the broker list in the first client; and sending data to the plurality of brokers. Furthermore, sending data to the plurality of brokers further includes generating a new data sequence number for each of the plurality of brokers; appending an origin identity and a payload to a message; sending the message in a data buffer, wherein sending the message includes checking the availability of brokers in the list of the plurality of brokers; generating a new data sequence on detection of availability of the brokers; releasing the data buffer on detection of unavailability of the brokers. In an embodiment, establishing connection between the at least one MQTT client to a cluster of MQTT brokers, in another platform of the scenario, further includes retrieving address of a first broker as the broker anchor by a first client 2 in the scenario under the given topic; connecting to broker anchor and determining the number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; retrieving address for the plurality of brokers present in the cluster; updating the broker list in the first client; and retrieving data from the broker. Furthermore, retrieving data from the broker further includes retrieving the new message buffer; extracting information including sequence number and origin ID of the data, and a combination thereof; comparing with predefined sequence number and origin ID available in the server; defining the data including deleting the duplicate files on detection of similar files while comparison and retrieving the final payload; and saving the new sequence number and origin ID on detection of dissimilar files while comparison and comparing again. Moreover, updating the broker list in the first client further includes retrieving a new list of brokers from broker anchor; comparing the new list of brokers against existing ones; updating the broker list and addresses on detection of dissimilar files while comparison; updating the last update status and time on detection of similar files while comparison; and checking status update whether active.

In one embodiment, defining a set the rules for carrying out load balancing further includes retrieving rules from a user application associated with the scenario; collecting metric performance data for the rules/set of rules; validating the rules by a rule injector module; providing feedback to the user application to set new rules on detection of failure of rules; injecting rules to the application defined load balance engine on detection of successful completion of rules. Furthermore, injecting rules to the application defined load balance engine further includes retrieving rules from the application defined module; checking availability for a new rule or set of rules; using round robin to select broker on detection of no new rules introduced, using the rule to do load balancing between the brokers on detection of new set of rules; and measuring the performance of the rules/set of rules.

In second aspect, embodiments of the present disclosure provide a system for MQTT client based high availability of data transfer within a scenario. The system includes a server communicably coupled to at the first client an application defined load balance engine operable to determine the data transfer load between the brokers and the clients; a data sender module operable to transfer data; an updater module operable to update new set of rules in the scenario; a rule injector operable to inject set of rules in the application defined load balance engine; an application defined module to execute the set of rules; and at the second client a parser operable to parse the data being retrieved from the anchor broker; a de-duplicator to detect a duplication of the data being retrieved from the anchor broker; and a payload operable to distribute the data load at the second client.

BRIEF DESCRIPTION OF DRAWINGS

The drawing/s mentioned herein disclose exemplary embodiments of the claimed invention. Detailed description and preparation of well-known compounds/substances/elements are omitted to not unnecessarily obscure the embodiments herein. Other objects, features, and advantages of the present invention will be apparent from the following description when read with reference to the accompanying drawing.

FIG. 1 illustrates an illustration of a flow chart of a method (100) for MQTT client based high availability of data transfer within a scenario, in accordance with an embodiment of the present disclosure.

FIG. 2 illustrates an illustration of a block diagram of a system (200) for MQTT client based high availability of data transfer within a scenario, in accordance with an embodiment of the present disclosure.

DETAILED DESCRPTION

This section is intended to provide explanation and description of various possible embodiments of the present invention. The embodiments used herein, and the various features and advantageous details thereof are explained more fully with reference to nonlimiting embodiments illustrated in the accompanying drawing/s and detailed in the following description. The examples used herein are intended only to facilitate understanding of ways in which the embodiments may be practiced and to enable the person skilled in the art to practice the embodiments used herein. Also, the examples/embodiments described herein should not be construed as limiting the scope of the embodiments herein. In one aspect, embodiments of the present disclosure provide a method for MQTT client based high availability of data transfer within a scenario. The method includes at least one MQTT client connected to a server, wherein the at least one MQTT client is associated with a topic for the scenario; at least one MQTT broker connected to the server, wherein the at least one MQTT client is associated with the topic for the scenario; characterized in that establishing a connection between the at least one MQTT client to a cluster of MQTT brokers, wherein the cluster of MQTT brokers include a plurality of brokers; directing the cluster of MQTT brokers by the at least MQTT client via an update module to manage the MQTT broker; monitoring the transmission of data by the at least MQTT client via a data sender module to manage the data transmission; and defining a set the rules for carrying out load balancing by the at least MQTT client via an application defined load balance engine.

In an embodiment, establishing the connection between the at least one MQTT client to a cluster of MQTT brokers further includes retrieving address of a first broker by a first client 1 in the scenario under the given topic, wherein the first broker is available in the cluster of brokers; assigning the retrieved first broker as an anchor broker; connecting to the anchor broker and determining number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; updating the broker list in the first client; and sending data to the plurality of brokers. Furthermore, sending data to the plurality of brokers further includes generating a new data sequence number for each of the plurality of brokers; appending an origin ID and a payload to a message; sending the message in a data buffer, wherein sending the message includes checking the availability of brokers in the list of the plurality of brokers; generating a new data sequence on detection of availability of the brokers; releasing the data buffer on detection of unavailability of the brokers.

In an embodiment, establishing connection between the at least one MQTT client to a cluster of MQTT brokers, in another platform of the scenario, further includes retrieving address of a first broker as the broker anchor by a first client 2 in the scenario under the given topic; connecting to broker anchor and determining the number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; retrieving address for the plurality of brokers present in the cluster; updating the broker list in the first client; and retrieving data from the broker. Furthermore, retrieving data from the broker further includes retrieving the new message buffer; extracting information including sequence number and origin ID of the data, and a combination thereof; comparing with predefined sequence number and origin ID available in the server; defining the data including deleting the duplicate files on detection of similar files while comparison and retrieving the final payload; and saving the new sequence number and origin ID on detection of dissimilar files while comparison and comparing again. Moreover, updating the broker list in the first client further includes retrieving a new list of brokers from broker anchor; comparing the new list of brokers against existing ones; updating the broker list and addresses on detection of dissimilar files while comparison; updating the last update status and time on detection of similar files while comparison; and checking status update whether active.

In second aspect, embodiments of the present disclosure provide a system for MQTT client based high availability of data transfer within a scenario. The system includes a server communicably coupled to at the first client an application defined load balance engine operable to determine the data transfer load between the brokers and the clients; a data sender module operable to transfer data; an updater module operable to update new set of rules in the scenario; a rule injector operable to inject set of rules in the application defined load balance engine; an application defined module to execute the set of rules; and at the second client a parser operable to parse the data being retrieved from the anchor broker; a de-duplicator to detect a duplication of the data being retrieved from the anchor broker; and a payload operable to distribute the data load at the second client. Throughout the present invention, the term “MQTT” refers to MQ Telemetry Transport or Message Queuing Telemetry Transport is an open OASIS and lightweight, publish-subscribe network protocol that transports messages between devices. The protocol runs over TCP/IP, however, any network protocol that provides ordered, lossless, bi-directional connections can support MQTT. Furthermore, the MQTT is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited.

The MQTT protocol defines two types of network entities such as a message broker and a number of clients. The MQTT broker is a server that receives messages from the clients and then routes the messages to the appropriate destination clients. Furthermore, the MQTT client is an arbitrary device that runs an MQTT library and connects to an MQTT broker over a network. Particularly, the MQTT is completely organised by one independent body but however in order to enhance security of the transmission third parties are also involved.

Throughout the present disclosure, the term “topic” refers to a particular situation, a unique command, a distinct protocol, and/or a specific contract between the client and the broker. For example, the topic may include association with a particular business field such as the client and broker belongs to fabric technology and the data meant to be transmitted between them may include information about a particular seasonal showcasing of a new set of garments.

At step (102), at least one MQTT client connected to a server, wherein the at least one MQTT client is associated with a topic for the scenario.

At step (104), at least one MQTT broker connected to the server, wherein the at least one MQTT broker is associated with the topic for the scenario.

At step (106), a connection between the at least one MQTT client to a cluster of MQTT brokers is established, wherein the cluster of MQTT brokers include a plurality of brokers.

At step (108), the cluster of MQTT brokers is directed by the at least MQTT client via an update module to manage the MQTT brokers. At step (110), transmission of data by the at least MQTT client is monitored via a data sender module to manage the data transmission.

At step (112), a set the rules is defined for carrying out load balancing by the at least MQTT client via an application defined load balance engine.

Furthermore, the term “scenario” used herein refers to a network framework of multiple devices connecting with each other through defined protocols. In a preferred embodiment, the present disclosure relates to transmission of data under defined rule for load balancing of the sharing between the client and the broker. Furthermore, there is provided a cluster of brokers in a desired scenario also referred as MQTT brokers and a plurality of MQTT clients. At initiation, a connection is being performed between the at least one MQTT client to the cluster of MQTT brokers.

Throughout the present disclosure, the term “address” may include IP address of devices, network arena, and so forth. Throughout the present disclosure, the term “set of rules” may include a defined algorithm or a standard protocol to be satisfied before performing or establishing any activity.

In an embodiment, establishing the connection between the at least one MQTT client to a cluster of MQTT brokers further includes retrieving address of a first broker by a first client in the scenario under the given topic, wherein the first broker is available in the cluster of brokers; assigning the retrieved first broker as an anchor broker; connecting to the anchor broker and determining number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; updating the list of broker in the first client; and sending data to the plurality of brokers.

Furthermore, the connection between the at least one MQTT client to a cluster of MQTT brokers is established by achieving parameters related to the scenario under the given topic. Such parameters include obtaining retrieving address of the first broker by the first client in the scenario under the given topic. The terms “first broker” and “first client” as used herein refers to clients and brokers initiated in the scenario having true recognitions related to the given topic. In a case where the available clients and brokers are not related to the given topic are discarded or detained. Furthermore, the anchor broker is assigned and the availability of other brokers is also determined to achieve the list of brokers in the scenario for the given topic. Moreover, once the list of brokers is achieved the data is sent to the plurality of brokers.

According to an embodiment, sending data to the plurality of brokers further includes generating a new data sequence number for each of the plurality of brokers; appending an origin identity and a payload to a message; sending the message in a data buffer, wherein sending the message includes checking the availability of brokers in the list of the plurality of brokers; generating a new data sequence on detection of availability of the brokers; and releasing the data buffer on detection of unavailability of the brokers.

Furthermore, the data being sent to the plurality of brokers through generation of new data sequence number for each of the plurality of brokers by conjoining origin identity and a payload to a message. The origin identity and payload are associated with the given topic in the scenario.

In one embodiment, establishing connection between the at least one MQTT client to a cluster of MQTT brokers, in another platform of the scenario, further includes retrieving address of a first broker as the broker anchor by a first client in the scenario under the given topic; connecting to broker anchor and determining the number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; and updating the broker list in the first client; and retrieving data from the broker.

In an embodiment, retrieving data from the broker further includes retrieving the new message buffer; extracting information including sequence number and origin identity of the data, and a combination thereof; comparing with predefined sequence number and origin identity available in the server; defining the data including deleting the duplicate files on detection of similar files while comparison and retrieving the final payload; and saving the new sequence number and origin identity on detection of dissimilar files while comparison and comparing again.

As mentioned herein above the similar and dissimilar files may relate to the predefined sequence number and origin identity available in the server when tracked and compared. Furthermore, the comparison is carried out on the basis of information associated with the given topic in the scenario. According to one embodiment, updating the broker list in the first client further includes retrieving a new list of brokers from broker anchor; comparing the new list of brokers against existing ones; updating the broker list and addresses on detection of dissimilar files while comparison; updating the last update status and time on detection of similar files while comparison; and checking status update whether active.

In one embodiment, defining a set the rules for carrying out load balancing further includes retrieving rules from a user application associated with the scenario; collecting metric performance data for the rules/set of rules; validating the rules by a rule injector module; providing feedback to the user application to set new rules on detection of failure of rules; injecting rules to the application defined load balance engine on detection of successful completion of rules.

In accordance to an embodiment, injecting rules to the application defined load balance engine further includes retrieving rules from the application defined module; checking availability for a new rule or set of rules; using round robin to select broker on detection of no new rules introduced, using the rule to do load balancing between the brokers on detection of new set of rules; and measuring the performance of the rules/set of rules.

Furthermore, the new rule or set of rules may include the data type, data size, parameters associated with the given topic and the like. As used herein above, the round-robin (rr) is one of the algorithms employed by process and network schedulers is generally used, time slices (also known as time quanta) are assigned to each process in equal portions and in circular order, handling all processes without priority (also known as cyclic executive). The roundrobin scheduling is simple, easy to implement, and starvation-free. The round-robin scheduling can be applied to other scheduling problems, such as data packet scheduling in computer networks.

FIG. 2 illustrates an illustration of a block diagram of a system (200) for MQTT client based high availability of data transfer within a scenario, in accordance with an embodiment of the present disclosure. The system (200) includes a server (202) communicably coupled to, at the first client end, an application defined load balance engine (204) operable to determine data transfer load between the brokers and the clients; a data sender module (206) operable to transfer data; an updater module (208) operable to update new set of rules in the scenario; a rule injector (210) operable to inject set of rules in the application defined load balance engine (204); an application defined module (212) to execute the set of rules; and at the second client a parser (214) operable to parse the data being retrieved from the anchor broker; a de-duplicator (216) to detect a duplication of the data being retrieved from the anchor broker; and a payload (218) operable to distribute the data load at the second client. As will be readily apparent to a person skilled in the art, the present invention may easily be produced in other specific forms without departing from its essential composition and properties. The present embodiments should be construed as merely illustrative and non- restrictive and the scope of the present invention being indicated by the claims rather than the foregoing description, and all changes which come within therefore intended to be embraced therein.

Claims

1. A method (100) for Message Queuing Telemetry Transport, MQTT client based high availability of data transfer within a scenario, the method (100) comprising: at least one MQTT client connected to a server, wherein the at least one MQTT client is associated with a topic for the scenario; and at least one MQTT broker connected to the server, wherein the at least one MQTT broker is associated with the topic for the scenario; characterized in that, establishing a connection between the at least one MQTT client to a cluster of MQTT brokers, wherein the cluster of MQTT brokers include a plurality of brokers (106); directing the cluster of MQTT brokers by the at least MQTT client via an update module to manage the MQTT brokers (108); monitoring transmission of data by the at least MQTT client via a data sender module to manage the data transmission (110); and defining a set of rules for carrying out load balancing by the at least MQTT client via an application defined load balance engine (112).

2. The method (100) as claimed in claim 1, wherein establishing the connection between the at least one MQTT client to a cluster of MQTT brokers further includes: retrieving address of a first broker by a first client in the scenario under a given topic, wherein the first broker is available in the cluster of brokers; assigning the retrieved first broker as an anchor broker; connecting to the anchor broker and determining number of brokers existing in the cluster of brokers associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; updating list of the broker in the first client; and sending data to the plurality of brokers.

3. The method (100) as claimed in claim 2, wherein sending data to the plurality of brokers further includes: generating a new data sequence number for each of the plurality of brokers; appending an origin identity and a payload to a message; and sending the message in a data buffer, wherein sending the message includes: checking availability of brokers in the list of the plurality of brokers; generating a new data sequence on detection of availability of the brokers; and releasing the data buffer on detection of unavailability of the brokers.

4. The method (100) as claimed in claim 1, wherein the method (100) further comprises establishing connection between the at least one MQTT client to a cluster of MQTT brokers, in another platform of the scenario, includes: retrieving address of a first broker as the broker anchor by a first client in the scenario under the given topic; connecting to broker anchor and determining the number of brokers existing in the cluster associated with the anchor broker; retrieving address for the plurality of brokers present in the cluster; updating the list of the broker in the first client; and retrieving data from the broker.

5. The method (100) as claimed in claim 4, wherein retrieving data from the broker further includes: retrieving the new message buffer; extracting information including sequence number and origin identification (ID) of the data, and a combination thereof; comparing with predefined sequence number and origin ID available in the server; defining the data includes deleting the duplicate files on detection of similar files while comparison and retrieving the final payload; and saving the new sequence number and origin ID on detection of dissimilar files while comparison and comparing again.

6. The method (100) as claimed in claim 4, wherein updating the list of the broker in the first client further includes: retrieving a new list of brokers from the broker anchor; comparing the new list of brokers against existing ones; updating the list of broker and addresses on detection of dissimilar files during comparison; updating the last update status and time on detection of similar files during comparison; and checking status update whether the broker is active.

7. The method (100) as claimed in claim 1, wherein defining a set of rules for carrying out load balancing further includes: retrieving rules from a user application associated with the scenario; collecting metric performance data for the rules or the set of rules; validating the rules by a rule injector module; providing feedback to the user application to set new rules on detection of failure of rules; and injecting rules to the application defined load balance engine on detection of successful completion of rules.

8. The method (100) as claimed in claim 7, wherein injecting rules to the application defined load balance engine further includes: retrieving rules from an application defined module; checking availability for a new rule or set of rules; using round robin to select broker on detection of no new rules introduced; using the rule to do load balancing between the brokers on detection of new set of rules; and measuring the performance of the rules or the set of rules.

9. A system (200) for Message Queue Telemetry Transfer (MQTT) client based high availability of data transfer within a scenario, the system (200) comprising: a server (202) communicably coupled to a first client and a second client, characterised in that, at the first client: an application defined load balance engine (204) operable to determine data transfer load between brokers and clients; a data sender module (206) operable to transfer data; an updater module (208) operable to update new set of rules in the scenario; a rule injector (210) operable to inject set of rules in the application defined load balance engine (204); an application defined module (212) to execute the set of rules; and at the second client: a parser (212) operable to parse the data being retrieved from an anchor broker; a de-duplicator (214) to detect a duplication of the data being retrieved from the anchor broker; and a payload (216) operable to distribute the data load at the second client.