CN116248746A - MQTT-based data processing method, device and computer-readable storage medium - Google Patents

MQTT-based data processing method, device and computer-readable storage medium Download PDF

Info

Publication number
CN116248746A
CN116248746A CN202310071603.XA CN202310071603A CN116248746A CN 116248746 A CN116248746 A CN 116248746A CN 202310071603 A CN202310071603 A CN 202310071603A CN 116248746 A CN116248746 A CN 116248746A
Authority
CN
China
Prior art keywords
link information
data
connection request
mqtt
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310071603.XA
Other languages
Chinese (zh)
Inventor
鞠飞
邓邱伟
李莉
翟建光
张洪朋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Haier Intelligent Home Appliance Technology Co Ltd
Original Assignee
Qingdao Haier Intelligent Home Appliance Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Haier Intelligent Home Appliance Technology Co Ltd filed Critical Qingdao Haier Intelligent Home Appliance Technology Co Ltd
Priority to CN202310071603.XA priority Critical patent/CN116248746A/en
Publication of CN116248746A publication Critical patent/CN116248746A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H04L67/143—Termination or inactivation of sessions, e.g. event-controlled end of session
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
    • H04L43/103—Active monitoring, e.g. heartbeat, ping or trace-route with adaptive polling, i.e. dynamically adapting the polling rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/0823—Network architectures or network communication protocols for network security for authentication of entities using certificates
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
    • H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H04L67/143—Termination or inactivation of sessions, e.g. event-controlled end of session
    • H04L67/145—Termination or inactivation of sessions, e.g. event-controlled end of session avoiding end of session, e.g. keep-alive, heartbeats, resumption message or wake-up for inactive or interrupted session
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3226—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using a predetermined code, e.g. password, passphrase or PIN
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3263—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements
    • H04L9/3268—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements using certificate validation, registration, distribution or revocation, e.g. certificate revocation list [CRL]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Cardiology (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Quality & Reliability (AREA)
  • Computer And Data Communications (AREA)

Abstract

The application relates to the technical field of data processing, and discloses a data processing method, device and computer-readable storage medium based on MQTT. The data processing method based on the MQTT comprises the following steps: storing link information corresponding to the equipment for the accessed equipment connection request; setting a monitoring period of the link information according to the service type corresponding to the equipment connection request; and processing the link information according to the monitoring period. By adopting the data processing method provided by the disclosure, the link information can be monitored to determine whether the link information needs to be subjected to keep-alive processing continuously. The link information which does not need to be subjected to continuous keep-alive processing can be processed in time, so that the resource occupation is reduced, the data processing amount is reduced, and the response speed and the processing efficiency of the MQTT background server side are improved.

Description

MQTT-based data processing method, device and computer-readable storage medium
Technical Field
The present application relates to the field of data processing technologies, for example, to a data processing method, apparatus and computer readable storage medium based on MQTT.
Background
The internet of things is used as an ecological support of intelligent home, more and more devices adopt MQTT (Message Queuing Telemetry Transport, message queue telemetry transmission protocol), and as a bridge for communication between the devices and business services, an MQTT background server is facing ten millions or even billions of link pressures, and the background server is also facing various software and hardware resource problems.
In the related art, by setting an authentication method based on the MQTT access internet of things, specifically by increasing the key types and the number, authentication control on the accessed equipment is realized, so that the number of the accessed equipment is reduced.
In carrying out the above embodiment, there are the following problems:
in devices accessing the internet of things, it is found from analysis of access data links that most clients are not actively linked and do not require real-time data interaction. These clients occupy a lot of resources, so that the response speed and processing efficiency of the MQTT background server are still greatly affected.
It should be noted that the information disclosed in the foregoing background section is only for enhancing understanding of the background of the present application and thus may include information that does not form the prior art that is already known to those of ordinary skill in the art.
Disclosure of Invention
The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview, and is intended to neither identify key/critical elements nor delineate the scope of such embodiments, but is intended as a prelude to the more detailed description that follows.
The data processing method, the device and the computer-readable storage medium based on the MQTT improve the response speed and the processing efficiency of the MQTT background server.
In some embodiments, the MQTT-based data processing method comprises: storing link information corresponding to the equipment for the accessed equipment connection request; setting a monitoring period of the link information according to the service type corresponding to the equipment connection request; and processing the link information according to the monitoring period.
Optionally, the step of processing the link information according to the monitoring period includes: acquiring characteristic data of the link information in the monitoring period; and processing the link information according to the characteristic data.
Optionally, the step of processing the link information according to the feature data includes: based on the characteristic data being lower than a set threshold, carrying out offline processing on the link information; and cleaning the link data associated with the link information in the database. Wherein the feature data includes: traffic data and/or liveness.
Optionally, the step of storing the link information corresponding to the device includes: saving the link information to a link maintenance data table; and updating interaction data for the link information into the link maintenance data table after the equipment connection request is accessed.
Optionally, before the step of storing the link information corresponding to the device in the accessed device connection request, the processing method further includes: responding to the equipment connection request, and acquiring a user key and subscription rights included in the connection request; in the authentication service, comparing the user key and the subscription authority with data information in the high-availability cache service; based on the data information of the high-availability cache service including the user key and the subscription authority, passing authentication and carrying out equipment connection; comparing the user key and the subscription authority with data in a database based on the data information of the high-availability cache service without the user key and the subscription authority; and carrying out equipment connection based on the authentication passing if the database comprises the user key and the subscription authority.
Optionally, before the step of obtaining the user key and the subscription authority included in the connection request, the processing method further includes: performing certificate verification on the equipment connection request at a server; and after the certificate passes verification, performing password authentication.
Optionally, the processing method further includes: setting the life cycle of the link information according to the service type; acquiring business interaction data of the link information in a preset time before the life cycle is finished when the life cycle is finished; based on the business interaction data meeting the offline condition, performing offline processing on the link information; acquiring the interaction time of the last service interaction data in the preset time based on the fact that the service interaction data does not meet the offline condition; and updating the life cycle according to the interaction time.
In some embodiments, there is provided an MQTT-based data processing apparatus, comprising: the storage module is configured to store link information corresponding to the equipment for the accessed equipment connection request; the monitoring module is configured to set the monitoring period of the link information according to the service type corresponding to the equipment connection request; and the processing module is configured to process the link information according to the monitoring period.
In some embodiments, there is provided an MQTT-based data processing apparatus comprising a processor and a memory storing program instructions, the processor being configured, when executing the program instructions, to perform a processing method as previously described.
In some embodiments, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein the program when run performs the aforementioned processing method.
The data processing method, device and computer readable storage medium based on the MQTT provided by the embodiment of the disclosure can realize the following technical effects:
according to the MQTT-based data processing method, for an accessed device connection request, firstly, link information corresponding to the device connection request is acquired and stored in a database. And setting a corresponding monitoring period according to the service type corresponding to the equipment connection request. And monitoring the link information according to the monitoring period to determine whether the link information needs to be kept alive or removed, and processing the link information according to the monitored data information. By adopting the data processing method provided by the disclosure, the link information can be monitored to determine whether the link information needs to be subjected to keep-alive processing continuously. The link information which does not need to be subjected to continuous keep-alive processing can be processed in time, so that the resource occupation is reduced, the data processing amount is reduced, and the response speed and the processing efficiency of the MQTT background server side are improved.
The foregoing general description and the following description are exemplary and explanatory only and are not restrictive of the application.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which like reference numerals refer to similar elements, and in which:
FIG. 1 is a system diagram of an MQTT based data processing apparatus provided in one embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an authentication and keep-alive flow for an MQTT based data processing device provided in an embodiment of the present disclosure;
FIG. 3 is a flow diagram of an MQTT based data processing method provided in an embodiment of the present disclosure;
FIG. 4 is a flow diagram of a data processing method based on MQTT provided in an embodiment of the present disclosure;
FIG. 5 is a flow diagram of a data processing method based on MQTT provided in an embodiment of the present disclosure;
FIG. 6 is a system block diagram of an MQTT based data processing device provided in accordance with one embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an MQTT-based data processing apparatus provided in one embodiment of the present disclosure.
Reference numerals:
110 equipment side; 120 data processing means; 121 server side; 122 an authentication service module; 123 a safety monitoring module; 124 high availability cache module; 125 data flow module.
210 device side; 220MQTT authentication module; 222 service end; 224 an authentication service module; 230MQTT keep-alive module; 232 a security monitoring module.
600: a data processing device; 602: a storage module; 604: a monitoring module; 606: and a processing module.
700: a data processing device; 702: a processor; 704: a memory; 706: a communication interface; 708: a bus.
Detailed Description
So that the manner in which the features and techniques of the disclosed embodiments can be understood in more detail, a more particular description of the embodiments of the disclosure, briefly summarized below, may be had by reference to the appended drawings, which are not intended to be limiting of the embodiments of the disclosure. In the following description of the technology, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the disclosed embodiments. However, one or more embodiments may still be practiced without these details. In other instances, well-known structures and devices may be shown simplified in order to simplify the drawing.
In some embodiments, as shown in connection with fig. 1, an MQTT-based data processing apparatus 120 is provided, comprising a server 121, an authentication service module 122, a security monitoring module 123, a high availability cache module 124, and a data streaming module 125. The server 121 performs data interaction with the device side 110. The server 121 receives the connection request sent by the device side 110, the authentication service module 122 performs certificate verification and authentication with the high availability cache module 124, and if the connection request does not pass, the device side 110 is fed back. The certificate verification and authentication pass, device connection is performed, and service separation and request separation are performed on service data through the data flow module 125. And maintaining the business data information to a database, and setting a monitoring period for the link information corresponding to each connection request. The security monitoring module 123 performs monitoring analysis on the service data information of the link information according to the monitoring period. And processing the link information according to the analysis result.
In some embodiments, as shown in connection with fig. 2, a process flow diagram is provided for authenticating a connection request of a device and maintaining a keep-alive for accessed link information based on MQTT. The device side 210 sends a connection request to the server side 222, and the MQTT authentication module 220 performs certificate authentication and authentication on the connection request, and accesses the device after the certificate authentication passes and the authentication also passes. If not, rejecting the device access. The MQTT keep-alive module 230 monitors link information corresponding to the accessed device. The security monitoring module 232 periodically analyzes the service interaction data information of the link information to determine whether the corresponding link information needs to be deleted. And deleting the link information and the corresponding data when the deletion is judged to be needed. Therefore, the resource utilization rate of the MQTT background server is improved, and the response rate and the processing efficiency are improved.
In some embodiments, an MQTT-based data processing apparatus is provided that includes a processor and a memory storing program instructions, the processor being configured, upon execution of the program instructions, to perform an MQTT-based data processing method. The processor comprises a server, an authentication service module, a security monitoring module, a high-availability cache module and a data flow module.
In some embodiments, as shown in connection with FIG. 3, the MQTT based data processing method includes:
s302, the processor stores the link information corresponding to the equipment for the accessed equipment connection request.
S304, the processor sets a monitoring period of the link information according to the service type corresponding to the equipment connection request.
S306, the processor processes the link information according to the monitoring period.
According to the MQTT-based data processing method, for an accessed device connection request, firstly, link information corresponding to the device connection request is acquired and stored in a database. And setting a corresponding monitoring period according to the service type corresponding to the equipment connection request. And monitoring the link information according to the monitoring period to determine whether the link information needs to be kept alive or removed, and processing the link information according to the monitored information. By adopting the data processing method provided by the disclosure, the link information can be monitored to determine whether the link information needs to be subjected to keep-alive processing continuously. The link information which does not need to be continuously kept alive can be processed in time, so that the resource occupation is reduced, the data processing amount is reduced, and the response speed and the processing efficiency of the MQTT background service are improved.
The keep-alive process is to keep the device side and the server of the MQTT in a connected state.
Optionally, the step of processing the link information according to the monitoring period includes: acquiring characteristic data of link information in a monitoring period; and processing the link information according to the characteristic data.
In this embodiment, when the monitoring period has arrived, feature data corresponding to the link information in the period is acquired, the state of the link information is analyzed according to the feature data, and the link information is processed according to the analysis structure. By analyzing the data of the monitoring period, the active condition of the link information can be accurately known, and the maximum utilization rate of the MQTT link pool can be improved.
Optionally, the step of processing the link information according to the feature data includes: based on the characteristic data being lower than a set threshold value, carrying out offline processing on the link information; and cleaning the link data associated with the link information in the database. Wherein the feature data includes: traffic data and/or liveness.
In this embodiment, the characteristic data includes flow data. And monitoring the flow data of the link information in the monitoring period. And deleting the link information with the flow data lower than the set threshold, namely, carrying out offline processing on the link information which is not frequently used, and deleting the link data associated with the link information, thereby improving the maximum utilization rate of the MQTT link pool.
In this embodiment, the characteristic data includes liveness. And (3) monitoring the activity of the link information in the period, performing offline processing on the link information in an inactive state for a long time, and deleting the link data associated with the link information to improve the resource utilization rate. The liveness refers to the statistic value of the access frequency and the access duration of the device side based on the MQTT in each monitoring period.
Optionally, the setting threshold is specifically set according to the service type, so that adjustment of the connection state of the link information is realized, the link pressure of the MQTT background server is reduced, and the response rate is improved.
Optionally, according to the service type corresponding to the equipment connection request, the monitoring period of the link information is set so as to accord with the interactive data characteristics of the service type, and the processing efficiency is improved. In a specific operation process, the monitoring period may be specifically set according to a specific service, which is not described herein.
Optionally, due to different subscription data authorities corresponding to different devices, a shorter monitoring period is set for devices with higher subscription authorities, so that the screening frequency of the link information is improved, the occupancy rate of the link information to hardware resources is reduced, and the processing efficiency of the MQTT back-end service is improved.
Optionally, the step of storing the link information corresponding to the device includes: saving the link information to a link maintenance data table; and updating the interactive data for the link information into the link maintenance data table after the device connection request is accessed.
In this embodiment, a link maintenance data table is created. For each accessed device, storing the link information corresponding to the connection request of each accessed device into a link maintenance data table; and updating the interactive data and the interactive time which are generated aiming at the link information into a link maintenance data table. By creating the link maintenance data table, the link information of all the admitted devices and the link data related to the link information are stored in the link maintenance data table. Meanwhile, corresponding monitoring periods are set for the service types corresponding to each connection request, and the corresponding monitoring periods are correspondingly written into the link maintenance data table. Therefore, the link maintenance data table is periodically scanned according to the monitoring period, and the link information can be timely processed, so that the response rate and the processing efficiency of the MQTT background server side are improved.
In some embodiments, as shown in connection with FIG. 4, the MQTT based data processing method includes:
s402, responding to a device connection request, and acquiring a user key and subscription rights included in the connection request by a processor;
s404, the processor compares the user key and the subscription authority with the data information in the high-availability cache service in the authentication service;
s406, the processor performs equipment connection based on that the data information of the high-availability cache service comprises a user key and subscription authority, and authentication is passed;
s408, the processor compares the user key and the subscription authority with data in the database based on the data information of the high-availability cache service without the user key and the subscription authority;
and S410, the processor performs equipment connection based on the fact that the database comprises the user key and the subscription authority, and authentication is passed.
S412, the processor stores the link information corresponding to the equipment for the accessed equipment connection request;
s414, the processor sets a monitoring period of the link information according to the service type corresponding to the equipment connection request;
s416, the processor processes the link information according to the monitoring period.
In this embodiment, before the device is accessed, the device connection request is authenticated, so as to improve the security and legality of the MQTT connection request. And acquiring a user key and subscription rights included in the connection request. In the authentication service, the user key and subscription rights are compared with data information in the high availability cache service. When the data information of the high available cache service comprises a user key and subscription authority, the authentication is passed, and the equipment connection is carried out. When the data information of the high-availability cache service does not comprise the user key and the subscription authority, the user key and the subscription authority are compared with the data in the database. When the database comprises the user key and the subscription authority, the authentication is passed, and the equipment connection is carried out.
When verifying the user key and the subscription authority, the user use data is timed or set to trigger the node to update into the cache by introducing a high-availability cache mechanism. By comparing the user key and subscription authority with the data information in the high-availability cache service, compared with the mode of directly comparing with the database, the data amount in the high-availability cache is smaller, and therefore the data reading and writing speed and the connection response speed can be improved.
When the data information of the high available cache service does not include the user key and the subscription authority, the step of comparing the user key and the subscription authority with the data in the database specifically includes: the user key is firstly compared and verified. And after the user key is successfully verified, checking the authority of the subscription subject, after all the authentications pass, returning to agree with the connection request, and after the equipment receives the response, performing the next operation.
Wherein, the authority of subscribing the theme comprises: subscription, publishing, subscribing and publishing, three different rights.
In some embodiments, as shown in connection with FIG. 5, the MQTT based data processing method includes:
s502, responding to the equipment connection request by the processor, and performing certificate verification on the equipment connection request at the server;
s504, the processor performs password authentication after the certificate passes verification.
S506, the processor acquires a user key and subscription rights included in the connection request;
s508, the processor compares the user key and the subscription authority with the data information in the high-availability cache service in the authentication service;
s510, the processor performs equipment connection based on that the data information of the high-availability cache service comprises a user key and subscription authority, and authentication is passed;
s512, the processor compares the user key and the subscription authority with the data in the database based on the data information of the high-availability cache service without the user key and the subscription authority;
and S514, the processor performs equipment connection based on the authentication passing if the database comprises the user key and the subscription authority.
S516, the processor stores the link information corresponding to the equipment for the accessed equipment connection request;
s518, the processor sets a monitoring period of the link information according to the service type corresponding to the equipment connection request;
s520, the processor processes the link information according to the monitoring period.
In this embodiment, the device connection request is certificate-verified and password-verified prior to authentication. The link request is verified in multiple dimensions by the certificate, the user name and the password and the subscription theme authority, so that legal user access is ensured, invalid requests are removed, invalid software and hardware resource waste is reduced, and the response efficiency of the MQTT server is improved.
Optionally, three layers of rights are used to authenticate and manage the connection request. In the first step, TLS (Transport Layer Security transport layer security protocol)/SSL (Secure Socket Layer secure sockets layer) certificates are used for mutual authentication. The server generates a TLS/SSL certificate by using the public key, the domain name, the information of the publisher and the like, provides the certificate and the private key to the MQTT client, and performs authentication when a secure connection request is initiated. In the second step, the password is concatenated using MD5 (MD 5 Message-Digest Algorithm MD5 Message digest algorithm). Based on the certification of the certificate, the server side can generate MD5 connection passwords for each network access device aiming at the unique identity characteristic attribute of the network access device, and after the device passes the certification and then requests to be sent to the server side during connection, the device carries out password security certification. And thirdly, checking subscription rights. After the connection security authentication passes, the server side can also perform authority verification of the currently applied subscription topic on each type of equipment, including but not limited to comprehensive scanning detection of topic paths, publishing, subscribing, publishing and subscribing authority states, client ID information and other information.
After the three layers of verification pass, the equipment side can establish connection in the service end of the MQTT, and finally monitors the required data change of the MQTT information on line.
Optionally, after completing the device connection, the processing method further includes: and carrying out distribution processing on the service data according to the request path of the equipment connection request.
In the embodiment, data flow is carried out on the data request by means of the MQTT distributed architecture, the existing business service is isolated in code and deployment, and access routes are carried out to the corresponding business server through the internal interfaces to carry out independent processing and return, so that distribution pressure on hardware resources of the server and timely return of business data are reduced, and user experience of a hardware end is improved.
After the equipment is successfully connected, the subscription and the release of the information operate the MQTT data under the current connection. The device side can issue a service execution request to the business background through the message of the MQTT, and the request is routed to the corresponding service through the registry for processing. The background service can publish the message to the service end through the message publication, and after the device subscribes to the message, the operation is executed at the device side.
Optionally, the processing method further comprises: setting the life cycle of the link information according to the service type; acquiring business interaction data of link information in a preset time before the life cycle is finished when the life cycle is finished; based on the business interaction data meeting the offline condition, performing offline processing on the link information; acquiring the interaction time of the last service interaction data in a preset time based on the fact that the service interaction data does not meet the offline condition; the lifecycle is updated according to the interaction time.
In this embodiment, the lifecycle is set according to the service type. And when the life cycle is finished, acquiring interaction data in a preset time before the end. If the interactive data exist, the link information is indicated to have service flow, namely the offline condition is not satisfied. And updating a new life cycle according to the last time of business interaction in the preset time so as to keep the current link information alive. If no interactive data exists, the service interaction is not stored in the preset time, namely the offline condition is met, the link information is processed offline, and the link data related to the link information is deleted. Thus, the maximum utilization rate of the MQTT link pool can be improved.
Optionally, the preset time may be set reasonably according to the service type and the life cycle, which is not described herein.
For example, a longer lifecycle is set for the service type of the device instruction; and setting a shorter life cycle aiming at the service type of the system upgrade.
Optionally, different lifecycles are set for the linking information of the required MQTT for different functional needs. For a one-time request, the life cycle is long enough to complete one data interaction, and the monitoring cycle is set to be longer than the life cycle. In this way. And the client actively disconnects the functional requirement after completing one data interaction operation. And because the monitoring period is larger than the life period, the resource utilization rate can be improved by setting the monitoring service to assist in scanning the unopened invalid connection. For the service requiring long-term connection maintenance, when the life cycle is finished, if the data interaction service exists in the preset time before the end, the life cycle is updated by taking the last data interaction time point as the starting point, so as to maintain the current active connection access.
Optionally, by adding a monitoring service of the state of the link information in the MQTT distributed architecture, a single link life cycle is set for each link information, and the data traffic of the link information is monitored. And in the life cycle, screening and filtering the liveness and flow data of the link information according to the monitoring cycle at regular time, downloading the link information in an inactive state for a long time, deleting the link data corresponding to the link information, and ensuring the maximum utilization rate of the MQTT link pool. The monitoring service for the link information is executed in a silent state, and the user and the equipment side have no perception. The device side only perceives when there is a data request. If further data interaction is required, the linking information can be reconnected by resending the connection request.
In some embodiments, as shown in fig. 6, there is provided an MQTT-based data processing apparatus 600, comprising: a storage module 602, configured to store link information corresponding to an accessed device for a connection request of the device; a monitoring module 604, configured to set a monitoring period of the link information according to a service type corresponding to the device connection request; and a processing module 606 configured to process the link information according to the monitoring period.
By adopting the data processing apparatus 600 provided by the present disclosure, the link information can be monitored to determine whether the link information requires continuous keep-alive processing. The link information which does not need to be continuously kept alive can be processed in time, so that the resource occupation is reduced, the data processing amount is reduced, and the response speed and the processing efficiency of the MQTT background service are improved.
The technical effects of employing the processing method of the present disclosure will be described below taking an example of device firmware upgrade based on MQTT notification. In the firmware upgrading scene of the intelligent equipment, an equipment upgrading request initiated by an App terminal user firstly carries out message notification based on the MQTT on the equipment through a server terminal, and after the equipment receives the MQTT message, upgrading operation is carried out according to the content contained in the message.
For the upgrade scenario, in the case of adopting the implementation manner in the related technology, the device needs to keep the online state for a long time to receive the upgrade message, so that the MQTT link resource, the server-side hardware and the network resource are occupied for a long time, but before the upgrade message is received next time, no data exchange exists, which causes a great amount of resource waste.
By adopting the processing device provided by the disclosure, the monitoring module of the MQTT can periodically detect the data exchange condition of the link information according to the monitoring period in the life cycle of the link information, and actively disconnect the link of the scene of equipment upgrading. When the device is upgraded next time, before the device only needs to issue an MQTT upgrading message notice through the App terminal, an instruction is issued to the MQTT link of the device activation device through the Wifi module protocol, and finally the purposes of controlling the number of the MQTT links of the device, guaranteeing efficient data access of effective links and improving user experience of the intelligent household device are achieved.
The embodiment of the disclosure provides a data processing device based on MQTT, the structure of which is shown in FIG. 7, comprising:
a processor (processor) 702 and a memory (memory) 704, and may also include a communication interface (Communication Interface) 706 and a bus 708. Wherein the processor 702, the communication interface 706, and the memory 704 may communicate with each other via a bus 708. The communication interface 706 may be used for information transfer. The processor 702 may invoke logic instructions in the memory 704 to perform the MQTT-based data processing methods of the above-described embodiments.
The memory 704 is used as a computer readable storage medium for storing a software program, a computer executable program, and program instructions/modules corresponding to the methods in the embodiments of the present disclosure. The processor 702 executes the program instructions/modules stored in the memory 704 to perform the functional applications and data processing, i.e., to implement the MQTT-based data processing method in the method embodiments described above. Therefore, the above embodiments have all the advantages and will not be described herein.
Memory 704 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created according to the use of the terminal device, etc. In addition, memory 704 may include high-speed random access memory, and may also include non-volatile memory.
Embodiments of the present disclosure provide a computer-readable storage medium storing computer-executable instructions configured to perform the MQTT-based data processing method described above.
The computer readable storage medium may be a transitory computer readable storage medium or a non-transitory computer readable storage medium.
The above description and the drawings illustrate embodiments of the disclosure sufficiently to enable those skilled in the art to practice them. Other embodiments may involve structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in, or substituted for, those of others. The scope of the embodiments of the present disclosure encompasses the full ambit of the claims, as well as all available equivalents of the claims. In the context of this application, although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without changing the meaning of the description, so long as all occurrences of the "first element" are renamed consistently and all occurrences of the "second element" are renamed consistently. The first element and the second element are both elements, but may not be the same element. Moreover, the terminology used in the present application is for the purpose of describing embodiments only and is not intended to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a," "an," and "the" (the) are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and/or" as used in this application is meant to encompass any and all possible combinations of one or more of the associated listed. Furthermore, the terms "comprises," "comprising," and/or "includes," when used in this application, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Without further limitation, an element defined by the phrase "comprising one …" does not exclude the presence of other like elements in a process, method or apparatus comprising such elements. In this context, each embodiment may be described with emphasis on the differences from the other embodiments, and the same similar parts between the various embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method sections disclosed in the embodiments, the description of the method sections may be referred to for relevance.
Those of skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. The skilled artisan may use different methods for each particular application to achieve the described functionality, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the embodiments disclosed herein, the disclosed methods, articles of manufacture (including but not limited to devices, apparatuses, etc.) may be practiced in other ways. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the units may be merely a logical function division, and there may be additional divisions in the case of actual implementations, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form. The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to implement the present embodiment. In addition, each functional unit in the embodiments of the present disclosure may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. In the description corresponding to the flowcharts and block diagrams in the figures, operations or steps corresponding to different blocks may also occur in different orders than that disclosed in the description, and in some cases no particular order exists between different operations or steps. For example, two consecutive operations or steps may actually be performed substantially in parallel, they may also be performed in reverse order in some cases, which may be dependent on the functions involved. Each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Claims (10)

1. A data processing method based on MQTT, the processing method comprising:
storing the link information corresponding to the equipment for the accessed equipment connection request;
setting a monitoring period of the link information according to the service type corresponding to the equipment connection request;
and processing the link information according to the monitoring period.
2. The processing method according to claim 1, wherein the step of processing the link information according to the monitoring period includes:
acquiring characteristic data of the link information in the monitoring period;
and processing the link information according to the characteristic data.
3. The processing method according to claim 2, wherein the step of processing the link information according to the feature data includes:
based on the characteristic data being lower than a set threshold, carrying out offline processing on the link information; and is combined with
Clearing link data associated with the link information in a database;
wherein the feature data includes: traffic data and/or liveness.
4. A processing method according to any one of claims 1 to 3, wherein the step of storing the link information corresponding to the device includes:
saving the link information to a link maintenance data table; and
after the device connection request is accessed, the interactive data aiming at the link information is updated into the link maintenance data table.
5. A processing method according to any one of claims 1 to 3, characterized in that before the step of storing the link information corresponding to the device in the accessed device connection request, the processing method further comprises:
responding to the equipment connection request, and acquiring a user key and subscription rights included in the connection request;
in the authentication service, comparing the user key and the subscription authority with data information in the high-availability cache service;
based on the data information of the high-availability cache service including the user key and the subscription authority, passing authentication and carrying out equipment connection;
comparing the user key and the subscription authority with data in a database based on the data information of the high-availability cache service without the user key and the subscription authority;
and carrying out equipment connection based on the authentication passing if the database comprises the user key and the subscription authority.
6. The method of processing according to claim 5, wherein prior to the step of obtaining the user key and subscription rights included in the connection request, the method of processing further comprises:
performing certificate verification on the equipment connection request at a server; and
after the certificate passes verification, password authentication is performed.
7. A process according to any one of claims 1 to 3, further comprising:
setting the life cycle of the link information according to the service type;
acquiring business interaction data of the link information in a preset time before the life cycle is finished when the life cycle is finished;
based on the business interaction data meeting the offline condition, performing offline processing on the link information;
acquiring the interaction time of the last service interaction data in the preset time based on the fact that the service interaction data does not meet the offline condition;
and updating the life cycle according to the interaction time.
8. A MQTT-based data processing apparatus, comprising:
the storage module is configured to store the link information corresponding to the equipment for the accessed equipment connection request;
the monitoring module is configured to set the monitoring period of the link information according to the service type corresponding to the equipment connection request;
and the processing module is configured to process the link information according to the monitoring period.
9. A MQTT-based data processing apparatus comprising a processor and a memory storing program instructions, wherein the processor is configured, when executing the program instructions, to perform the processing method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program when run performs the processing method of any one of claims 1 to 7.
CN202310071603.XA 2023-01-12 2023-01-12 MQTT-based data processing method, device and computer-readable storage medium Pending CN116248746A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310071603.XA CN116248746A (en) 2023-01-12 2023-01-12 MQTT-based data processing method, device and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310071603.XA CN116248746A (en) 2023-01-12 2023-01-12 MQTT-based data processing method, device and computer-readable storage medium

Publications (1)

Publication Number Publication Date
CN116248746A true CN116248746A (en) 2023-06-09

Family

ID=86623523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310071603.XA Pending CN116248746A (en) 2023-01-12 2023-01-12 MQTT-based data processing method, device and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN116248746A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116668493A (en) * 2023-07-25 2023-08-29 森智明信息科技南京有限公司 Message processing system of mqtt Internet of things equipment under distributed scene

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116668493A (en) * 2023-07-25 2023-08-29 森智明信息科技南京有限公司 Message processing system of mqtt Internet of things equipment under distributed scene
CN116668493B (en) * 2023-07-25 2023-10-24 森智明信息科技南京有限公司 Message processing system of mqtt Internet of things equipment under distributed scene

Similar Documents

Publication Publication Date Title
CN108616596B (en) Block chain self-adaptive consensus method based on dynamic authorization and network environment perception
CN108833521B (en) Message pushing method, device, system, computer equipment and storage medium
JP6446032B2 (en) Synchronizing credential hashes across directory services
JP2023541599A (en) Service communication methods, systems, devices and electronic equipment
CN106815099B (en) Authentication system and method
US9654480B2 (en) Systems and methods for profiling client devices
CN109417552B (en) Method and industrial computing device for performing secure communications
US20110161663A1 (en) Intelligent caching for ocsp service optimization
CN110933084B (en) Cross-domain shared login state method, device, terminal and storage medium
CN112035215A (en) Node autonomous method, system and device of node cluster and electronic equipment
CN103001965A (en) Method for updating server certificates and servers
WO2021051581A1 (en) Server cluster file synchronization method and apparatus, electronic device, and storage medium
US10581923B2 (en) System and method for configuration of a connected device connection
CN104702969A (en) Method and system for performing passive transcoding of video and audio data and transcoding server
JP6614280B1 (en) Communication apparatus and communication method
US20200287974A1 (en) System and method for switching between publish/subscribe services
CN110730081A (en) Block chain network-based certificate revocation method, related equipment and medium
CN116248746A (en) MQTT-based data processing method, device and computer-readable storage medium
CN115189897A (en) Access processing method and device for zero trust network, electronic equipment and storage medium
US10791119B1 (en) Methods for temporal password injection and devices thereof
CN111988262B (en) Authentication method, authentication device, server and storage medium
CN114726872A (en) Data synchronization method and device, computer equipment and storage medium
CN106027555B (en) A kind of method and system improving content distributing network safety using SDN technology
CN111427703A (en) Industrial data real-time display method and system
CN107872421B (en) Node authentication method and system and related equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination