CN117560415A - Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) - Google Patents
Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) Download PDFInfo
- Publication number
- CN117560415A CN117560415A CN202410038662.1A CN202410038662A CN117560415A CN 117560415 A CN117560415 A CN 117560415A CN 202410038662 A CN202410038662 A CN 202410038662A CN 117560415 A CN117560415 A CN 117560415A
- Authority
- CN
- China
- Prior art keywords
- msgid
- response message
- micro
- asynchronous
- mqtt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000001360 synchronised effect Effects 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000006854 communication Effects 0.000 title claims abstract description 39
- 238000004891 communication Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 213
- 230000008569 process Effects 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims description 17
- 230000002618 waking effect Effects 0.000 claims description 6
- 230000007175 bidirectional communication Effects 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/26—Special purpose or proprietary protocols or architectures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides an intelligent community micro-service architecture (MQTT) asynchronous and synchronous communication method and system, which relate to the technical field of the Internet of things and comprise the following steps: a plurality of micro service nodes, MQTT broaders and equipment ends; the method comprises the steps that a plurality of micro-service nodes and MQTT broaders are connected with a plurality of equipment ends in a two-way communication mode based on an MQTT protocol; the plurality of micro service nodes are configured to issue asynchronous request messages or synchronous request messages and process returned response messages; the MQTT Broker is configured to forward an asynchronous request message, a synchronous request message or a response message, and the equipment side is configured to process the asynchronous request message or the synchronous request message and return the response message. The invention realizes the MQTT asynchronous and synchronous communication function under the micro-service multi-node scene, and solves the problems of repeated consumption of asynchronous messages and loss of synchronous messages in the existing scheme.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to an MQTT asynchronous and synchronous communication method and system of an intelligent community micro-service architecture.
Background
Currently, internet of things equipment used in smart communities is accessed to a service system through an MQTT (message queue telemetry transport) protocol. The traditional MQTT communication is an asynchronous message model based on a subscription/release mode, but synchronous communication based on a request/response mode is needed to be carried out with equipment in certain scenes, the state of a request is needed to be saved in a node instance which initiates the request, and then the received response message is correctly matched with the request and then subsequent message processing is carried out.
For the case that the service system is a micro-service architecture, there may be multiple node instances for the backend service and dynamic increase and decrease may be performed. If the back-end service nodes subscribe to the response message by adopting a common subscription mode, the problem of repeated consumption of the message exists, and if the shared subscription mode is adopted, the response message of a certain synchronous request is possibly distributed to other nodes by the load balancing rule of the MQTT Broker (message proxy server), so that the message is lost.
At present, the existing MQTT synchronous-asynchronous scheme does not reserve an asynchronous mode, certain IoT Middleware (IoT-Middleware internet of things) needs special customization, certain mqTT standard protocols need to be changed, certain mqTT standard protocols need to be realized by means of certain third-party components, certain mqTT standard protocols are not suitable for micro-service scenes, and the use requirements cannot be met.
Disclosure of Invention
In view of this, the embodiment of the application provides a method and a system for asynchronous and synchronous communication of an intelligent community micro-service architecture (MQTT) so as to realize the functions of the asynchronous and synchronous communication of the MQTT in a micro-service multi-node scene and solve the problems of repeated consumption of asynchronous messages and loss of synchronous messages in the existing scheme.
The embodiment of the application provides the following technical scheme: an intelligent community micro-service architecture MQTT asynchronous and synchronous communication method comprises the following steps:
establishing bidirectional communication connection between a plurality of micro-service nodes and an MQTT Broker and between the MQTT Broker and a plurality of equipment ends based on an MQTT protocol; the method comprises the steps that a plurality of micro service nodes are deployed by adopting a distributed cluster in a micro service scene;
each micro-service node shares and subscribes to the response topics of all equipment ends, each micro-service node commonly subscribes to the synchronous response forwarding topics, each micro-service node creates an MsgId storage library, and each equipment end subscribes to the request topics containing the unique ID of the equipment;
responding to an asynchronous request message issued by a micro-service node, forwarding the asynchronous request message to a corresponding equipment end through an MQTT Broker, receiving a response message returned after the equipment end processes the asynchronous request message, distributing the response message to any micro-service node according to a load balancing rule through the MQTT Broker, and processing the asynchronous response message after judging that the response message is an asynchronous response message through the micro-service node;
and responding to a synchronous request message issued by a micro-service node, generating a globally unique MsgId for the synchronous request message, forwarding the synchronous request message and the generated globally unique MsgId to a corresponding equipment end through an MQTT Broker, receiving a response message returned after the equipment end processes the synchronous request message and the corresponding MsgId, distributing the response message and the corresponding MsgId to any micro-service node according to a load balancing rule through the MQTT Broker, judging whether the response message belongs to an instance domain of the node through the MsgId corresponding to the response message after judging that the response message is the synchronous response message through the micro-service node, and if so, processing the synchronous response message.
According to one embodiment of the present application, further comprising: after each micro-service node creates an MsgId storage library, starting an MsgId timeout checking thread, detecting whether each MsgId in the MsgId storage library is overtime or not at regular time, if any MsgId is overtime, waking up a thread waiting for the response of the MsgId to perform request timeout processing, and removing the MsgId from the MsgId storage library; the MsgId storage library comprises an MsgId, a generation time stamp of the MsgId and a timeout threshold.
According to one embodiment of the present application, the method for periodically detecting whether each msid in the MsgId repository is overtime, if any MsgId is overtime, waking up a thread waiting for the MsgId to respond to perform overtime processing includes:
the MsgId timeout checking thread traverses the MsgId storage library at regular time, and judges whether the current MsgId request is overtime or not according to the generation timestamp of each MsgId and the timeout threshold; before the timeout, the timeout threshold may be dynamically adjusted according to the current service system or MQTT Broker load.
According to one embodiment of the present application, processing the asynchronous response message after the micro service node determines that the response message is an asynchronous response message includes:
the micro service node determines whether the response message is an asynchronous response message by judging whether the asynchronous response message has an MsgId or not or whether the asynchronous response message has a MsgId with a set special format or not; if the asynchronous response message has no MsgId or has no MsgId with a set special format, the response message is an asynchronous response message.
According to one embodiment of the present application, in response to a synchronization request message issued by a micro service node, generating a globally unique MsgId for the synchronization request message, further includes:
storing the globally unique MsgId, a generation timestamp of the MsgId and a timeout threshold together into a local MsgId storage library, blocking a request thread of the synchronous request message, and waiting for a response message to return;
if the returned response message belongs to the instance domain of the current node, the request thread waiting for the MsgId response message is awakened to process the message.
According to an embodiment of the present application, after the micro service node determines that the response message is a synchronous response message, determining, by using an MsgId corresponding to the response message, whether the response message belongs to an instance domain of the node, further includes:
if the response message does not belong to the example domain of the node, the response message is issued to an MQTT Broker through a synchronous response forwarding topic, and the response message is broadcast to all micro-service nodes which are commonly subscribed to the synchronous response forwarding topic through the MQTT Broker;
and each micro service node judges whether the response message belongs to the instance domain of the current node or not through the MsgId corresponding to the response message, if so, wakes up a request thread waiting for the MsgId response message to process the message, and if not, discards the response message.
According to one embodiment of the present application, each micro service node determines, according to the MsgId corresponding to the response message, whether the response message belongs to an instance domain of the current node, including:
and each micro service node searches in the MsgId storage library of the node, and if the MsgId corresponding to the response message exists in the MsgId storage library of the node, the response message is judged to belong to the instance domain of the current node.
The application also provides an intelligent community micro-service architecture MQTT asynchronous and synchronous communication system, comprising: service system service node, MQTT Broker and equipment end; the service system service node comprises a plurality of micro service nodes which are distributed in a distributed cluster under a micro service scene, and the micro service nodes and the MQTT Broker, and the MQTT Broker and the plurality of equipment ends are connected in a two-way communication mode based on an MQTT protocol;
the plurality of micro service nodes are configured to issue asynchronous request messages or synchronous request messages and process returned response messages; the MQTT Broker is configured to forward the asynchronous request message, the synchronous request message or the response message, and the equipment end is configured to process the asynchronous request message or the synchronous request message and return the response message; wherein,
when one micro-service node issues an asynchronous request message, the asynchronous request message is forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the asynchronous request message is received, the response message is distributed to any micro-service node according to a load balancing rule through the MQTT Broker, and the asynchronous response message is processed after the micro-service node judges that the response message is an asynchronous response message;
when a micro-service node issues a synchronization request message, a global unique MsgId is generated for the synchronization request message, the synchronization request message and the generated global unique MsgId are forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the synchronization request message and the corresponding MsgId are received, the response message and the corresponding MsgId are distributed to any micro-service node according to a load balancing rule through the MQTT Broker, after the micro-service node judges that the response message is a synchronization response message, whether the response message belongs to an instance domain of the node is judged through the MsgId corresponding to the response message, if yes, the synchronization response message is processed.
According to one embodiment of the application, each micro-service node shares a response topic subscribed to all equipment ends, each micro-service node commonly subscribes to a synchronous response forwarding topic, each micro-service node creates an MsgId storage library, and each equipment end subscribes to a request topic containing an equipment unique ID.
According to one embodiment of the application, the MsgId repository includes an MsgId, a timestamp generated by the MsgId, and a timeout threshold, and the timestamp is used for starting an MsgId timeout checking thread, and detecting whether each MsgId in the MsgId repository is timeout or not according to the timestamp generated by each MsgId and the timeout threshold.
Compared with the prior art, the beneficial effects that above-mentioned at least one technical scheme that this description embodiment adopted can reach include at least:
1. according to the embodiment of the application, the MQTT asynchronous and synchronous communication in the micro-service scene is realized by only using the standard MQTT protocol, other protocols such as HTTP and the like are not introduced, and the light weight and simplicity are realized;
2. the embodiment of the application only needs the MQTT Broker, namely the essential basic component for MQTT communication, and does not depend on any other third party service or component, such as Redis/Kafka/RabbitMQ/ZooKeeper and the like, and the system complexity is low, the development difficulty is low, and the operation and maintenance cost is low;
3. the embodiment of the application has strong universality, and the used MQTT Broker can be any program software or an Internet traffic control (IoT) cloud platform supporting the MQTT v5.0 protocol, does not need special customization and can be replaced at will;
4. according to the embodiment of the application, the independent overtime check thread is used for processing the overtime of the synchronous request, the overtime time of the synchronous request can be dynamically changed, and the single synchronous request can be subjected to operations such as continuous time or immediate overtime, so that the system resource scheduling is more convenient;
5. in the embodiment of the application, each micro-service node is a stateless node, and each service node does not need to know any information of other nodes, does not need to additionally synchronize the states and data of other nodes, and is beneficial to dynamic capacity expansion or reduction of the nodes in a micro-service scene.
In summary, the embodiment of the application has the characteristics of light weight, low cost, strong universality, high flexibility, support of multithread concurrency, suitability for micro-service scenes and the like, realizes the functions of MQTT asynchronous and synchronous communication in the micro-service multi-node scenes, and solves the problems of repeated consumption of asynchronous messages and loss of synchronous messages.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an architecture of an MQTT asynchronous and synchronous communication system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the initial stage flow of the MQTT asynchronous and synchronous communication method according to the embodiment of the present invention;
FIG. 3 is a schematic diagram of an asynchronous request flow for an MQTT asynchronous and synchronous communication method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a synchronous request flow of the MQTT asynchronous and synchronous communication method according to an embodiment of the present invention.
Detailed Description
Embodiments of the present application are described in detail below with reference to the accompanying drawings.
Other advantages and effects of the present application will become apparent to those skilled in the art from the present disclosure, when the following description of the embodiments is taken in conjunction with the accompanying drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. The present application may be embodied or carried out in other specific embodiments, and the details of the present application may be modified or changed from various points of view and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The embodiment of the invention provides an MQTT asynchronous and synchronous communication method of an intelligent community micro-service architecture, which comprises the following steps:
establishing bidirectional communication connection between a plurality of micro-service nodes and an MQTT Broker and between the MQTT Broker and a plurality of equipment ends based on an MQTT protocol; the method comprises the steps that a plurality of micro service nodes are deployed by adopting a distributed cluster in a micro service scene;
each micro-service node shares and subscribes to the response topics of all equipment ends, each micro-service node commonly subscribes to the synchronous response forwarding topics, each micro-service node creates an MsgId storage library, and each equipment end subscribes to the request topics containing the unique ID of the equipment;
responding to an asynchronous request message issued by a micro-service node, forwarding the asynchronous request message to a corresponding equipment end through an MQTT Broker, receiving a response message returned after the equipment end processes the asynchronous request message, distributing the response message to any micro-service node according to a load balancing rule through the MQTT Broker, and processing the asynchronous response message after judging that the response message is an asynchronous response message through the micro-service node;
and responding to a synchronous request message issued by one micro-service node, generating a globally unique MsgId (Message Identification message identifier) for the synchronous request message, forwarding the synchronous request message and the generated globally unique MsgId to a corresponding equipment end through an MQTT Broker, receiving a response message and a corresponding MsgId returned by the equipment end after the equipment end processes the synchronous request message, distributing the response message and the corresponding MsgId to any micro-service node according to a load balancing rule through the MQTT Broker, judging that the response message is a synchronous response message through the micro-service node, judging whether the response message belongs to an instance domain of the node through the MsgId corresponding to the response message, and if so, processing the synchronous response message.
Aiming at the problems of repeated consumption of asynchronous messages and loss of synchronous messages in the existing scheme, the embodiment of the invention ensures a native asynchronous communication mode by utilizing a shared subscription mode provided by MQTT v5.0, and the MQTT Broker cluster carries out routing distribution of response messages based on a preset load balancing rule so as to avoid the problem of repeated consumption of the messages; the unique ID with the special prefix is utilized to match the synchronous request and the response message one by one, and the synchronous response message which is distributed to the node but not belonging to the domain of the example is subjected to secondary broadcast forwarding, so that the synchronous response message is ensured to reach the example of the node to which the synchronous response message belongs, and the problem of loss of synchronous response is avoided; and, using a separate timeout checking thread to handle the synchronization request timeout, a timeout period dynamic update function may be provided.
In the embodiment, only the standard MQTT protocol is used for realizing MQTT asynchronous and synchronous communication in a micro-service scene, other protocols such as HTTP are not introduced, development, use and maintenance costs are low, and light and compact are realized. In the embodiment, only the MQTT Broker is needed as a necessary basic component for MQTT communication, and the MQTT Broker is not dependent on any other third party service or component, such as Redis/Kafka/RabbitMQ/ZooKeeper and the like, so that the complexity of the system is low, the development difficulty is low, and the operation and maintenance cost is low; in addition, the universality is strong, the used MQTT Broker can be any program software supporting the MQTT v5.0 protocol or a third party platform such as an internet traffic (IoT) cloud platform, and the like, special customization is not needed, and the MQTT Broker can be replaced at will; in this embodiment, each micro-service node is a stateless node, and operations such as maintaining the state and data of the node and synchronizing the life cycle of the cache are not needed, so that dynamic capacity expansion or reduction of the node in the micro-service scene is facilitated. Based on the above, the embodiment only stores the globally unique msgId generated by each node in the affiliated node instance, and does not need to be synchronized to other nodes or third-party caches, so that space waste, bandwidth waste and synchronization are avoided.
Therefore, the embodiment of the invention has the characteristics of light weight, low cost, strong universality, high flexibility, support of multi-thread concurrency, suitability for micro-service scenes and the like, simultaneously supports the functions of MQTT asynchronous and synchronous communication of load balancing in the micro-service multi-node scenes, and solves the problems of repeated consumption of asynchronous messages and loss of synchronous messages.
In specific implementation, as shown in fig. 2, the MQTT asynchronous and synchronous communication method of the present embodiment mainly includes the following stages:
1. an initialization stage:
(1) Each Service instance node (the micro Service node described above) shares a subscription device response Topic (Topic) "$share/g/resp/#" where "$share/g/" is a shared Topic grouping prefix and "resp/#" means subscribing to all topics beginning with "resp/", i.e., subscribing to response messages of all devices.
The MQTT Broker distributes the device response message to a certain Service instance sharing the topic according to the load balancing rule, so that the problem of repeated consumption caused by the fact that asynchronous responses are broadcast to all Service instances is avoided.
The load balancing rules of the MQTT Broker include the following common rules: polling, weighted polling, random, weighted random, fastest response, hash algorithm, etc.
(2) Each Service instance commonly subscribes to a synchronous response forwarding topic "sync/resp";
the MQTT Broker broadcasts a message published to the topic to each Service instance subscribed to the topic;
(3) Each Service instance creates an MsgId storage library for storing the MsgId, the generation time stamp of the MsgId, the overtime threshold value and other related information;
(4) Each Service instance starts an msid timeout checking thread, periodically detects whether each msid in the repository is timeout, if so, wakes up the thread waiting for the Id response to request timeout processing, and removes the msid from the msid repository.
Specifically, the MsgId timeout checking thread traverses the MsgId repository at regular time, judges whether the current MsgId request is overtime according to the timestamp generated by each MsgId and the timeout threshold, and the timeout threshold of each MsgId can be dynamically adjusted according to the current service system or MQTT Broker load before timeout, so as to avoid the problem of invalid response message caused by long-time blocking or excessively fast timeout of a large number of requests, thereby causing the problem of wasting system resources or low request success rate, and can also perform operations such as waiting for a certain request for a duration or immediately overtime.
(5) Each Device subscribes to a request topic containing a Device unique ID (e.g., device1 subscribes to topic "req/dev 1"), and when a Device, such as Device1, receives the request message, the response message is processed to be published to the Broker through topic "resp/dev 1".
2. Asynchronous request flow is shown in fig. 3.
The scheme reserves the asynchronous communication mode of the MQTT.
When a certain Service instance node issues an asynchronous request message to an MQTT Broker, the MQTT Broker forwards the request to corresponding equipment, the equipment processes the request and issues corresponding response messages, the MQTT Broker distributes the response messages to any one Service instance node according to a load balancing rule, and the Service instance node judges that the response is an asynchronous response message, and then directly carries out subsequent processing on the messages.
In one embodiment, the micro service node determines whether the response message is an asynchronous response message by determining whether the asynchronous response message has an MsgId or whether the asynchronous response message has a set MsgId of a special format; if the asynchronous response message has no MsgId or has no MsgId with a set special format, the response message is an asynchronous response message.
In the embodiment, the shared subscription mode provided by the MQTT v5.0 is utilized to ensure the original asynchronous communication mode, and the MQTT Broker cluster carries out the route distribution of the response message based on the preset load balancing rule, so that the problem of repeated consumption of the message is avoided.
The synchronization request flow is shown in fig. 4. (1) When a certain Service instance initiates a synchronous request, a globally unique MsgId is generated through a distributed ID generation algorithm and put into a request message load, then the request is issued, meanwhile, the MsgId, a sending time stamp of the MsgId and a timeout threshold are stored in a local MsgId storage library, then the request thread is blocked, and a response message is waited to return. The MsgId repository adopts thread safety design and coding, so that the MsgId repository supports multithreading concurrency, and a single Service instance node can initiate a plurality of synchronous requests and wait for return.
Among the above-mentioned distributed ID generation algorithms, common algorithms include: database self-increment, redis increment, number segment pattern, UUID algorithm, swflag algorithm, uid-generator algorithm, etc.
In the embodiment, the independent timeout checking thread is used for processing the timeout of the synchronous request, the timeout time of the synchronous request can be dynamically changed, and operations such as continuous time or immediate timeout can be performed on a single synchronous request, so that the system resource scheduling is more convenient.
(2) The device side processes the request message forwarded by the MQTT Broker, and puts the MsgId carried by the request into a response message load for returning.
(3) The response message may be forwarded to any Service instance node through the load balancing algorithm of the MQTT Broker; the Service instance node receiving the message judges that the response is synchronous, and then judges whether the MsgId in the response belongs to the instance domain of the node: if yes, waking up a request thread waiting for the MsgId response message to process the message, and ending the request; if not, the message is published to the MQTT Broker by synchronous response forwarding Topic (sync/resp).
(4) If the response message does not belong to the instance domain of the node, the MQTT Broker broadcasts the forwarded response message to all Service instance nodes which are commonly subscribed to synchronous response forwarding Topic; each instance judges after receiving the message: judging whether the response message belongs to an instance domain of the current node or not through the MsgId corresponding to the response message, if so, waking up a request thread waiting for the MsgId response message to process the message, and ending the request; if not, the message is discarded and the request ends.
Specifically, each micro service node searches in the MsgId repository of the node, and if the MsgId corresponding to the response message exists in the MsgId repository of the node, the response message is judged to belong to the instance domain of the current node.
In the embodiment, the unique ID with the special prefix is utilized to match the synchronous request and the response message one by one, and the synchronous response message which is distributed to the node but not belonging to the domain of the example is subjected to secondary broadcast forwarding, so that the synchronous response message is ensured to reach the example of the node to which the synchronous response message belongs, and the problem of loss of synchronous response is avoided. In addition, the embodiment adopts the mode of MQTT Broker rebroadcast release, each node does not need to know any information of other nodes, and the implementation is more concise without data synchronization problem.
(5) If a Service instance has not received a response message of a certain MsgId due to various reasons such as offline equipment, network failure, equipment failure, etc., the MsgId timeout checking thread of the instance wakes up the thread waiting for the Id response to perform request timeout processing after a preset or specified time according to the generation time of the MsgId in the MsgId repository, and removes the MsgId from the MsgId repository.
As shown in fig. 1, the present application further provides an MQTT asynchronous and synchronous communication system of a smart community micro-service architecture, including: service system service node, MQTT Broker and equipment end; the service system service node comprises a plurality of micro service nodes which are distributed in a distributed cluster under a micro service scene, and the micro service nodes and the MQTT Broker, and the MQTT Broker and the plurality of equipment ends are connected in a two-way communication mode based on an MQTT protocol;
the plurality of micro service nodes are configured to issue asynchronous request messages or synchronous request messages and process returned response messages; the MQTT Broker is configured to forward the asynchronous request message, the synchronous request message or the response message, and the equipment end is configured to process the asynchronous request message or the synchronous request message and return the response message; wherein,
when one micro-service node issues an asynchronous request message, the asynchronous request message is forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the asynchronous request message is received, the response message is distributed to any micro-service node according to a load balancing rule through the MQTT Broker, and the asynchronous response message is processed after the micro-service node judges that the response message is an asynchronous response message;
when a micro-service node issues a synchronization request message, a global unique MsgId is generated for the synchronization request message, the synchronization request message and the generated global unique MsgId are forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the synchronization request message and the corresponding MsgId are received, the response message and the corresponding MsgId are distributed to any micro-service node according to a load balancing rule through the MQTT Broker, after the micro-service node judges that the response message is a synchronization response message, whether the response message belongs to an instance domain of the node is judged through the MsgId corresponding to the response message, if yes, the synchronization response message is processed.
In the system architecture of the embodiment, service is a specific back-end Service node, distributed cluster deployment under a micro-Service scene is adopted, the number of running instance nodes can be dynamically increased or decreased according to real-time load, and each instance node is connected with an MQTT Broker cluster or a single MQTT Broker program through an MQTT protocol; each device is connected to an MQTT Broker cluster or a single MQTT Broker program via the MQTT protocol.
According to one embodiment of the application, each micro-service node shares a response topic subscribed to all equipment ends, each micro-service node commonly subscribes to a synchronous response forwarding topic, each micro-service node creates an MsgId storage library, and each equipment end subscribes to a request topic containing an equipment unique ID.
According to one embodiment of the application, the MsgId repository includes an MsgId, a timestamp generated by the MsgId, and a timeout threshold, and the timestamp is used for starting an MsgId timeout checking thread, and detecting whether each MsgId in the MsgId repository is timeout or not according to the timestamp generated by each MsgId and the timeout threshold.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions easily conceivable by those skilled in the art within the technical scope of the present application should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. An intelligent community micro-service architecture (MQTT) asynchronous and synchronous communication method is characterized by comprising the following steps:
establishing bidirectional communication connection between a plurality of micro-service nodes and an MQTT Broker and between the MQTT Broker and a plurality of equipment ends based on an MQTT protocol; the method comprises the steps that a plurality of micro service nodes are deployed by adopting a distributed cluster in a micro service scene;
each micro-service node shares and subscribes to the response topics of all equipment ends, each micro-service node commonly subscribes to the synchronous response forwarding topics, each micro-service node creates an MsgId storage library, and each equipment end subscribes to the request topics containing the unique ID of the equipment;
responding to an asynchronous request message issued by a micro-service node, forwarding the asynchronous request message to a corresponding equipment end through an MQTT Broker, receiving a response message returned after the equipment end processes the asynchronous request message, distributing the response message to any micro-service node according to a load balancing rule through the MQTT Broker, and processing the asynchronous response message after judging that the response message is an asynchronous response message through the micro-service node;
and responding to a synchronous request message issued by a micro-service node, generating a globally unique MsgId for the synchronous request message, forwarding the synchronous request message and the generated globally unique MsgId to a corresponding equipment end through an MQTT Broker, receiving a response message returned after the equipment end processes the synchronous request message and the corresponding MsgId, distributing the response message and the corresponding MsgId to any micro-service node according to a load balancing rule through the MQTT Broker, judging whether the response message belongs to an instance domain of the node through the MsgId corresponding to the response message after judging that the response message is the synchronous response message through the micro-service node, and if so, processing the synchronous response message.
2. The intelligent community micro service architecture MQTT asynchronous and synchronous communication method of claim 1, further comprising:
after each micro-service node creates an MsgId storage library, starting an MsgId timeout checking thread, detecting whether each MsgId in the MsgId storage library is overtime or not at regular time, if any MsgId is overtime, waking up a thread waiting for the response of the MsgId to perform request timeout processing, and removing the MsgId from the MsgId storage library; the MsgId storage library comprises an MsgId, a generation time stamp of the MsgId and a timeout threshold.
3. The method of claim 2, wherein the steps of periodically detecting whether each MsgId in the MsgId repository is timeout, and if any MsgId is timeout, waking up a thread waiting for the MsgId to respond to the timeout, include:
the MsgId timeout checking thread traverses the MsgId storage library at regular time, and judges whether the current MsgId request is overtime or not according to the generation timestamp of each MsgId and the timeout threshold; before the timeout, the timeout threshold may be dynamically adjusted according to the current service system or MQTT Broker load.
4. The method for asynchronous and synchronous communication of the intelligent community micro service architecture MQTT according to claim 1, wherein the processing the asynchronous response message after determining, by the micro service node, that the response message is an asynchronous response message comprises:
the micro service node determines whether the response message is an asynchronous response message by judging whether the asynchronous response message has an MsgId or not or whether the asynchronous response message has a MsgId with a set special format or not; if the asynchronous response message has no MsgId or has no MsgId with a set special format, the response message is an asynchronous response message.
5. The method of intelligent community micro-service architecture MQTT asynchronous and synchronous communication according to claim 2, wherein generating a globally unique MsgId for a synchronization request message issued by one micro-service node in response to the synchronization request message, further comprises:
storing the globally unique MsgId, a generation timestamp of the MsgId and a timeout threshold together into a local MsgId storage library, blocking a request thread of the synchronous request message, and waiting for a response message to return;
if the returned response message belongs to the instance domain of the current node, the request thread waiting for the MsgId response message is awakened to process the message.
6. The method for asynchronous and synchronous communication of the intelligent community micro service architecture MQTT of claim 5, wherein after the micro service node determines that the response message is a synchronous response message, determining whether the response message belongs to the instance domain of the node according to the MsgId corresponding to the response message, further comprising:
if the response message does not belong to the example domain of the node, the response message is issued to an MQTT Broker through a synchronous response forwarding topic, and the response message is broadcast to all micro-service nodes which are commonly subscribed to the synchronous response forwarding topic through the MQTT Broker;
and each micro service node judges whether the response message belongs to the instance domain of the current node or not through the MsgId corresponding to the response message, if so, wakes up a request thread waiting for the MsgId response message to process the message, and if not, discards the response message.
7. The method for asynchronous and synchronous communication of the intelligent community micro service architecture MQTT of claim 6, wherein each micro service node determines whether the response message belongs to the instance domain of the current node according to the MsgId corresponding to the response message, comprising:
and each micro service node searches in the MsgId storage library of the node, and if the MsgId corresponding to the response message exists in the MsgId storage library of the node, the response message is judged to belong to the instance domain of the current node.
8. An intelligent community micro-service architecture MQTT asynchronous and synchronous communication system, comprising: service system service node, MQTT Broker and equipment end; the service system service node comprises a plurality of micro service nodes which are distributed in a distributed cluster under a micro service scene, and the micro service nodes and the MQTT Broker, and the MQTT Broker and the plurality of equipment ends are connected in a two-way communication mode based on an MQTT protocol;
the plurality of micro service nodes are configured to issue asynchronous request messages or synchronous request messages and process returned response messages; the MQTT Broker is configured to forward the asynchronous request message, the synchronous request message or the response message, and the equipment end is configured to process the asynchronous request message or the synchronous request message and return the response message; wherein,
when one micro-service node issues an asynchronous request message, the asynchronous request message is forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the asynchronous request message is received, the response message is distributed to any micro-service node according to a load balancing rule through the MQTT Broker, and the asynchronous response message is processed after the micro-service node judges that the response message is an asynchronous response message;
when a micro-service node issues a synchronization request message, a global unique MsgId is generated for the synchronization request message, the synchronization request message and the generated global unique MsgId are forwarded to a corresponding device end through an MQTT Broker, a response message returned after the device end processes the synchronization request message and the corresponding MsgId are received, the response message and the corresponding MsgId are distributed to any micro-service node according to a load balancing rule through the MQTT Broker, after the micro-service node judges that the response message is a synchronization response message, whether the response message belongs to an instance domain of the node is judged through the MsgId corresponding to the response message, if yes, the synchronization response message is processed.
9. The intelligent community micro service architecture MQTT asynchronous and synchronous communication system of claim 8, wherein each micro service node shares a response topic subscribed to all device ends, each micro service node subscribes to a synchronous response forwarding topic in common, each micro service node creates an MsgId repository, and each device end subscribes to a request topic containing a device unique ID.
10. The system of claim 9, wherein the MsgId repository includes an MsgId, a timestamp of the MsgId generation, and a timeout threshold for starting an MsgId timeout checking thread, and detecting whether each MsgId in the MsgId repository is timed out according to the timestamp of each MsgId generation and the timeout threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410038662.1A CN117560415B (en) | 2024-01-11 | 2024-01-11 | Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410038662.1A CN117560415B (en) | 2024-01-11 | 2024-01-11 | Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117560415A true CN117560415A (en) | 2024-02-13 |
| CN117560415B CN117560415B (en) | 2024-03-22 |
Family
ID=89813224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410038662.1A Active CN117560415B (en) | 2024-01-11 | 2024-01-11 | Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117560415B (en) |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110336736A (en) * | 2019-05-27 | 2019-10-15 | 四川长虹电器股份有限公司 | The shared method subscribed to is realized based on MQTT server cluster |
| US20200067789A1 (en) * | 2016-06-24 | 2020-02-27 | QiO Technologies Ltd. | Systems and methods for distributed systemic anticipatory industrial asset intelligence |
| CN111212085A (en) * | 2020-01-16 | 2020-05-29 | 厦门网宿有限公司 | Internet of things platform synchronous calling method, Internet of things system and network equipment |
| US10701167B1 (en) * | 2016-09-01 | 2020-06-30 | Amazon Technologies, Inc. | Adaptive quorum for a message broker service |
| CN111580995A (en) * | 2020-05-12 | 2020-08-25 | 南京甄视智能科技有限公司 | Synchronous communication method and system of distributed cloud platform and Internet of things intelligent terminal based on MQTT asynchronous communication scene |
| CN111711663A (en) * | 2020-05-26 | 2020-09-25 | 北京金山云网络技术有限公司 | Processing method and device for publishing and subscribing services and electronic equipment |
| CN112817791A (en) * | 2020-12-31 | 2021-05-18 | 西安合智宇信息科技有限公司 | Mobile terminal monitoring method for mining state of working face cluster |
| CN112929431A (en) * | 2021-02-01 | 2021-06-08 | 深圳市科漫达智能管理科技有限公司 | Message synchronous and asynchronous response conversion method and device based on micro-service framework |
| CN114116791A (en) * | 2021-09-30 | 2022-03-01 | 武汉武钢绿色城市技术发展有限公司 | Data synchronization method and system based on MQTT protocol message |
| CN114338781A (en) * | 2021-12-29 | 2022-04-12 | 神州数码系统集成服务有限公司 | Artificial intelligence micro-service control method, system, equipment, storage medium and application |
| CN114338806A (en) * | 2022-02-28 | 2022-04-12 | 湖南云畅网络科技有限公司 | Synchronous message processing method and system |
| CN114422371A (en) * | 2022-01-20 | 2022-04-29 | 重庆邮电大学 | Elastic micro-service system based on distributed and container virtualization and implementation method |
| CN114978683A (en) * | 2022-05-20 | 2022-08-30 | 深圳市艾迪思特信息技术有限公司 | Reverse proxy system based on MQTT protocol |
| KR20230074923A (en) * | 2021-11-22 | 2023-05-31 | 전남대학교산학협력단 | System, method and program for managing mobility based on mqtt |
| CN116860466A (en) * | 2023-06-12 | 2023-10-10 | 深圳市网力软件有限公司 | Asynchronous communication to synchronous communication method based on MQTT |
-
2024
- 2024-01-11 CN CN202410038662.1A patent/CN117560415B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200067789A1 (en) * | 2016-06-24 | 2020-02-27 | QiO Technologies Ltd. | Systems and methods for distributed systemic anticipatory industrial asset intelligence |
| US10701167B1 (en) * | 2016-09-01 | 2020-06-30 | Amazon Technologies, Inc. | Adaptive quorum for a message broker service |
| CN110336736A (en) * | 2019-05-27 | 2019-10-15 | 四川长虹电器股份有限公司 | The shared method subscribed to is realized based on MQTT server cluster |
| CN111212085A (en) * | 2020-01-16 | 2020-05-29 | 厦门网宿有限公司 | Internet of things platform synchronous calling method, Internet of things system and network equipment |
| CN111580995A (en) * | 2020-05-12 | 2020-08-25 | 南京甄视智能科技有限公司 | Synchronous communication method and system of distributed cloud platform and Internet of things intelligent terminal based on MQTT asynchronous communication scene |
| CN111711663A (en) * | 2020-05-26 | 2020-09-25 | 北京金山云网络技术有限公司 | Processing method and device for publishing and subscribing services and electronic equipment |
| CN112817791A (en) * | 2020-12-31 | 2021-05-18 | 西安合智宇信息科技有限公司 | Mobile terminal monitoring method for mining state of working face cluster |
| CN112929431A (en) * | 2021-02-01 | 2021-06-08 | 深圳市科漫达智能管理科技有限公司 | Message synchronous and asynchronous response conversion method and device based on micro-service framework |
| CN114116791A (en) * | 2021-09-30 | 2022-03-01 | 武汉武钢绿色城市技术发展有限公司 | Data synchronization method and system based on MQTT protocol message |
| KR20230074923A (en) * | 2021-11-22 | 2023-05-31 | 전남대학교산학협력단 | System, method and program for managing mobility based on mqtt |
| CN114338781A (en) * | 2021-12-29 | 2022-04-12 | 神州数码系统集成服务有限公司 | Artificial intelligence micro-service control method, system, equipment, storage medium and application |
| CN114422371A (en) * | 2022-01-20 | 2022-04-29 | 重庆邮电大学 | Elastic micro-service system based on distributed and container virtualization and implementation method |
| CN114338806A (en) * | 2022-02-28 | 2022-04-12 | 湖南云畅网络科技有限公司 | Synchronous message processing method and system |
| CN114978683A (en) * | 2022-05-20 | 2022-08-30 | 深圳市艾迪思特信息技术有限公司 | Reverse proxy system based on MQTT protocol |
| CN116860466A (en) * | 2023-06-12 | 2023-10-10 | 深圳市网力软件有限公司 | Asynchronous communication to synchronous communication method based on MQTT |
Non-Patent Citations (4)
| Title |
|---|
| EMQX TEAM: "共享订阅介绍与示例|mqtt5.0特性详解", HTTPS://WWW.EMQX.COM/ZH/BLOG/INTRODUCTION-TO-MQTT5-PROTOCOL-SHARED-SUBSCRIPTION, 10 November 2023 (2023-11-10) * |
| SU PYAE WIN: "shedding light on technological treasures within iot application layer protocol", IEEE, 19 July 2023 (2023-07-19) * |
| 潘学殿: "面向铁路物联网平台的MQTT协议优化、实现与验证", 中国优秀硕士学位论文全文数据库信息科技辑, 15 July 2023 (2023-07-15) * |
| 齐兵: "基于MQTT协议扩展的IoT设备完整性监控", 计算机系统应用, 15 November 2022 (2022-11-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117560415B (en) | 2024-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3490224B1 (en) | Data synchronization method and system | |
| CN111565229B (en) | Communication system distributed method based on Redis | |
| CN108768826A (en) | Based on the message route method under MQTT and Kafka high concurrent scenes | |
| CN105338086A (en) | Distributed message forwarding method | |
| WO2011026430A1 (en) | Method and system for data synchronization in content delivery network | |
| CN111212085B (en) | Method for synchronously calling Internet of things platform, internet of things system and network equipment | |
| CN102224715A (en) | Optimized polling in low resource devices | |
| CN102118263A (en) | Method and system for distribution of configuration information | |
| CN111615066A (en) | Distributed micro-service registration and calling method based on broadcast | |
| CN112583895B (en) | TCP communication method, system and device | |
| CN105141676A (en) | Session persistence sharing method and session persistence sharing system under multiple servers | |
| CN113811013B (en) | Cluster capacity dynamic adjustment method, device, storage medium and communication system | |
| CN114328618A (en) | Cache data synchronization method, device, equipment and storage medium | |
| CN115102986B (en) | Internet of things data distribution and storage method and system in edge environment | |
| CN112087390A (en) | Network routing method and device | |
| CN114025002A (en) | MQTT information transmission method, system and communication equipment | |
| CN113572864A (en) | Data processing method, network element equipment and readable storage medium | |
| CN117560415B (en) | Asynchronous and synchronous communication method and system for smart community micro-service architecture (MQTT) | |
| CN103685344A (en) | Synergetic method and system for multiple P2P (point-to-point) cache peers | |
| CN112202833A (en) | CDN system, request processing method and scheduling server | |
| KR101066328B1 (en) | Method for balancing load in mobile environment and mobile device implementing the same | |
| CN114422591B (en) | Point-to-point communication method, data communication system, computer device, and storage medium | |
| CN111083182B (en) | Distributed Internet of things equipment management method and device | |
| CN115022392B (en) | IOT-oriented distributed publishing and subscribing service method and system | |
| CN116996579B (en) | Method and system for improving downloading speed of fragmentation request based on multiplexing 302 address |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |