CN117240903B - Internet of things offline message dynamic management configuration system - Google Patents
Internet of things offline message dynamic management configuration system Download PDFInfo
- Publication number
- CN117240903B CN117240903B CN202311343102.9A CN202311343102A CN117240903B CN 117240903 B CN117240903 B CN 117240903B CN 202311343102 A CN202311343102 A CN 202311343102A CN 117240903 B CN117240903 B CN 117240903B
- Authority
- CN
- China
- Prior art keywords
- message
- node
- data
- internet
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 claims abstract description 48
- 230000005856 abnormality Effects 0.000 claims abstract description 44
- 230000002159 abnormal effect Effects 0.000 claims abstract description 32
- 238000011084 recovery Methods 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims description 35
- 238000012549 training Methods 0.000 claims description 32
- 230000005540 biological transmission Effects 0.000 claims description 29
- 238000007726 management method Methods 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000001514 detection method Methods 0.000 claims description 15
- 238000002955 isolation Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 4
- 230000004048 modification Effects 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000013500 data storage Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000004148 unit process Methods 0.000 claims description 3
- 238000012384 transportation and delivery Methods 0.000 claims description 2
- 230000006855 networking Effects 0.000 claims 2
- 238000005192 partition Methods 0.000 claims 1
- 230000001105 regulatory effect Effects 0.000 abstract 1
- 238000009825 accumulation Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000010365 information processing Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a system for dynamically managing and configuring an offline message of an Internet of things, which comprises a message generating and issuing module, a message processing and transmitting module, an abnormality detecting and recovering module and a configuration management module, wherein the message generating and issuing module is responsible for generating and issuing a message to a message queue, the message processing and transmitting module comprises a message priority scheduling unit and a message deduplication unit, the message priority scheduling unit is used for ensuring that a message with high priority can be processed as soon as possible, the message deduplication unit is used for processing repeated messages received by equipment in an offline period, the abnormality detecting and recovering module is used for detecting abnormal conditions and taking corresponding recovery measures, and the configuration management module is used for managing and regulating the dynamically managing and configuring the offline message.
Description
Technical Field
The invention relates to the field of message communication and equipment management in the Internet of things, in particular to an offline message dynamic management configuration system of the Internet of things.
Background
Along with the continuous development of networks, the requirements of people on intellectualization are higher and higher, the technology of the internet of things is also gradually rising, and a message queue is a middleware technology for storing and transmitting messages, and can temporarily store undelivered messages during the offline period of equipment and retransmit the messages to the equipment after the equipment is reconnected, so that the message queue can realize the caching and priority scheduling of the messages and ensure the timely transmission of important messages. During the offline period of the device, a large number of undelivered messages may be generated, and the offline message processing technology involves temporarily storing the undelivered messages in a local cache or a cloud storage, and after the device is online, transmitting the undelivered messages to the device one by one for processing, so as to ensure the reliable transmission of the messages and avoid the loss of the messages. Devices in an internet of things system typically need to be dynamically configured, and remote configuration techniques allow an administrator or other system to remotely update the device configuration over a network without having to physically connect directly to the device. The equipment management platform is a centralized system for managing the equipment of the Internet of things, and can monitor the state and configuration information of the equipment, perform fault diagnosis and take corresponding recovery measures. The device management platform provides support for dynamic configuration and exception recovery. The data synchronization technique is used to ensure that data collected by the device during offline can be synchronized to the cloud or other systems after the device is online, thereby avoiding the problem of data loss or inconsistency. In the internet of things system, equipment needs to carry out identity authentication, so that only legal equipment can receive and process sensitive information, and the security of the system is protected.
According to the system, the information generation and release module, the information processing and transmission module, the abnormality detection and recovery module, the configuration management module and other modules are introduced, so that the efficiency and stability of information dynamic management in an offline state of the Internet of things are improved, the information is generated and released to an information queue, the information has different priorities, the information needs to be scheduled and processed according to the priorities, an MFH algorithm is provided for sequencing the priorities of the information in the information queue according to the importance and the emergency degree of the information, the dynamic update of the priorities is supported, the priority order of the information can be adjusted in real time, a waiting time matrix is provided on the basis of the original algorithm, the insertion and deletion work can be efficiently carried out on the dynamic information, the overhead of the queue maintenance in a processing scene of the dynamic information is small, the minimum (or maximum) value can be found in the constant time complexity, the whole queue does not need to be traversed, the searching efficiency is improved, the situation of information accumulation is reduced, the important information delay processing caused by low priority information accumulation is avoided, and the important information processing can be ensured to be obtained as soon as possible. The method comprises the steps of processing repeated messages received by equipment in an offline period, providing an improved machine learning algorithm for isolating forest GIF to process possible abnormal conditions in the process of message transmission and equipment processing, rapidly detecting abnormal data isolated from other data points by analyzing equipment state data or message data, providing a hyper-rectangular block on the basis of the original isolated forest algorithm, and filtering and hanging the detected abnormal messages by utilizing the depth of nodes in a tree to ensure that the abnormal messages do not enter a message queue or a cache any more, thereby identifying possible abnormal conditions. Once the abnormality is found, the recovery module is combined with the aid of the GIF algorithm to assist in abnormality detection to quickly take corresponding recovery measures such as reconnection of a network and restarting equipment, configuration information of the equipment and the system is finally stored, verification is carried out when the equipment and the system configuration is updated, record of change history of the configuration is supported, and the equipment and the system can be ensured to operate according to the preset configuration. The system for dynamically managing and configuring the offline information of the Internet of things improves the reliability and stability of the system of the Internet of things, optimizes the communication efficiency of equipment, promotes the application of the technology of the Internet of things in various fields, explores more optimization methods and solutions related to the system for dynamically managing and configuring the offline information of the Internet of things, promotes the progress and innovation of academic research, and lays a solid foundation for the construction and development of future intelligent society.
Disclosure of Invention
Aiming at the problems, the invention aims to provide an Internet of things offline message dynamic management configuration system.
The aim of the invention is realized by the following technical scheme:
in order to achieve the above-mentioned problems in the background art, the following method is provided, and the specific process is described as follows: the system comprises a message generation and release module, a message processing and transmission module, an abnormality detection and recovery module and a configuration management module, wherein the message generation and release module is responsible for generating and releasing messages to a message queue and reasonably classifying and releasing the messages, the message processing and transmission module comprises a message priority scheduling unit and a message deduplication unit, the message priority scheduling unit provides an MFH algorithm to prioritize the messages in the message queue according to the importance degree and the emergency degree of the messages, so that the important messages can be processed as soon as possible, the message deduplication unit processes repeated messages received by equipment in an offline period, the abnormality detection and recovery module provides an improved isolated forest GIF algorithm to process abnormal situations possibly occurring in the process of message transmission and equipment processing, and adopts corresponding recovery measures, and the configuration management module is used for managing and adjusting the offline message dynamic management configuration.
Further, the message generation and publication module is configured to generate and publish messages to the message queue for subsequent offline message processing and post-device-on-line message delivery, as well as other device and background system subscription and reception, based on device sensor data, user requests, or other trigger conditions.
Furthermore, the message priority scheduling unit provides the MFH algorithm to order according to the priority of the messages and dynamically schedule the processing sequence of the messages, so that important messages can be processed and transmitted as soon as possible, the message processing efficiency is improved, the messages are prevented from being accumulated in the queue, and the important messages are prevented from being delayed to be processed, so that the performance, the reliability and the response capability of the system are improved.
Further, the specific procedure of the proposed MFH algorithm is as follows: the Internet of things is provided withThe same Internet of things equipment is marked as +.>Wherein->Is 1 st Internet of things equipment +.>Is 2 nd Internet of things equipment +.>Is->The station internet of things device, a user sends a request by sending a signal to the device to obtain a desired output, and the turn-around time of the signal is defined as:wherein->For turn-around time of signal,/->The time required for transmitting a signal to the internet of things device and receiving the output result, i.e. the transmission time,/->Time spent for the internet of things device to complete signal execution, i.e. processing time, +.>For time->If the signal needs to wait before executing, update to: />Wherein->Is the total number of signals->In the internet of things environment, assuming that the arrival time and the service time between signals are exponentially distributed, the probability that the internet of things device is occupied is expressed as: />Wherein->Probability of being busy for Internet of things equipment, +.>For average arrival rate>For average service rate->For the number of the internet of things devices, the average message number of the message queues is defined as:
wherein the method comprises the steps ofThe probability of 0 signals in the internet of things is:
latency in message queuesUse->The turn-around time of the signal is calculated as:the transmission time and processing of signals is typically fixed, minimizing the turnaround time by minimizing the average latency of signals in the information queue, so the minimization problem is expressed as: />And satisfies the following constraints: />、/>And +.>、/>Assume two signals (+)>And->) The waiting internet of things equipment has the priority of +.>And->And signal->At->Previously performed, i.e.)>，/>Wherein->Ordering of representative signals, using the formula->Indicating signal->Execution time of (1), wherein->Indicating signal->Size of->Representation->The bit architecture is used to determine the bit pattern,representing one million instructions per second, a data structure of a latency matrix is proposed, in which the elements are defined as follows: 1) The diagonal element is set to 1; 2) The waiting time of each signal is calculated and placed in the upper half of the diagonal of the matrix by the order of signal arrival, assuming that the ordering of the signals is +.>Then->，/>，And +.>Wherein->Indicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Execution time of->Indicating signal->Execution time of->Indicating signal->Is expressed as a matrix:
3) The inverse value of the filling waiting time of the lower half of the diagonal line of the matrix is
Worst case execution with reduced latency per signalAfter the signal arrives at the Internet of things system, the signal is assigned with priority by using an MFH algorithm, a priority queue is realized by using a fibonacci pile, and each node is->Contains a pointer to its parent node +.>Pointer child node +.>Wherein->The nodes in the heap are represented as such,representation->Is +_a parent node of->Representation->Pointer child node, of->Is in +.>Are connected together in sub-lists, each sub-node in the sub-list +.>All have left +.>And right->Two pointer nodes, wherein->Representing any other child node in the child list, < +.>Representation->When the fibonacci pile is empty, a new signal based on the priority value is inserted into the fibonacci pile through the node, so that the priority of the information is found through an MFH algorithm, scheduling operation is carried out according to the priority, and the next message with higher priority is acquired by a message processing and transmission module to be processed, so that important messages can be processed more quickly.
Further, the message deduplication unit is used for maintaining a record table or database of a message queue and a message cache, detecting whether the unique identification exists in the message queue or the cache to judge whether the identification is a duplicate message or not by generating the unique identification of the message, and directly filtering out the duplicate messages to optimize data storage and transmission, thereby ensuring the accuracy and idempotent of message processing, optimizing the message queue to reduce unnecessary message transmission, avoiding the duplicate processing and resource waste caused by receiving the duplicate message in an offline period, and improving the performance and response speed of the system.
Further, the abnormality detection and recovery module continuously detects the state, sensor data, communication state and the like of the equipment of the Internet of things, detects and analyzes the state data of the equipment by the improved GIF algorithm provided by the invention, judges whether an abnormality occurs and identifies the type and reason of the abnormality, sends an abnormality notification to a relevant system administrator or application program to remind the system administrator or application program of paying attention to the abnormality, and finally adopts corresponding recovery measures to process the abnormality for subsequent analysis and optimization.
Further, the specific steps of the proposed improved GIF algorithm are as follows:
generatingPartitioning the state data by a random tree, wherein +.>For the number of random trees and calculating the number of nodes required for each tree to isolate each training vector, the vector with the smallest average path length is detected as abnormal data, a random isolation tree is created, assuming +.>Personal state training data->Wherein n represents the number of training data, +.>Represents training data 1->Represents training data 2->Indicate->Training data, and->Wherein->Representation->Set of dimension real numbers->Indicate->The 1 st component of the training data, +.>Indicate->The 2 nd component of the training data,indicate->Training data->The vectors are re-used->Representing a matrix of all training data acquired, wherein +.>Representation->Dimension data set, ->Represents 1 st->Vector of dimensions>Represents 2 +.>Vector of dimensions>Indicate->Personal->The vector is then divided into two subsets, randomly selected +.>One component of (2) is denoted +.>And at->Uniformly selecting a split value>Wherein->First->Personal trainingTraining data->A component dividing the data set into two parts, left branch corresponding set +.>Right branch corresponding setA random tree is created by applying this process to each branch in turn until the branch contains a unique data point, the boundary of the hyper-rectangular block being defined as: />Wherein->Indicate->Super rectangular block margin of individual node, +.>Indicate->1-dimensional lower bound of individual nodes, +.>Indicate->1-dimensional upper bound of individual nodes, +.>Indicate->Personal node +.>Upper bound of dimension (I)>Indicate->Personal node +.>Dimension upper bound, assume->To include->A subset of data present in the smallest block of the individual nodes, wherein +.>For the data subset, a random variable is selected from an exponential distribution>，/>The splitting rate of (2) is>Calculated, wherein->Is->Is also +.>Is>Representing +.>Upper bound of dimension (I)>Representing data childrenConcentrated->Dimension lower bound, assume node ++>Is greater than the splitting time of its parent node plus +.>Then at node->Creating a new node in combination with the previously randomly selected component->Splitting value of uniform selection +.>If data point->At the option->The base value of the component is greater than the upper bound +.>Then from->Uniformly selecting the split value->If the value is smaller than the lower bound +.>Then from->Uniformly selecting the split value->According to the split value->Data points->Is created as a node +.>Left and right branches of (1), assume node +.>Is smaller than the splitting time of its parent node plus +.>The calling node, now recursive downwards along the random tree +.>To calculate the isolation point +.>Is not equal to the desired path length of (a)Defining an abnormality score +_>Wherein->Is->Is (are) desirable to be (are)>Is->The size is +.>Is formulated as +.>Wherein->Is->The number of subharmonics, therefore->When x is abnormal, the score isWhen->Tend to->The abnormality score tends to 0 when +.>Relative to->When smaller, the corresponding abnormality score tends to 1, defining an abnormality threshold +.>When->When (I)>Is detected as abnormal when->When (I)>Detected as normal data, and therefore by proposingAccording to the GIF algorithm of the system, the possible abnormal situation is detected according to the state data of the equipment, and measures such as restarting, automatic reconnection and standby equipment switching are adopted to recover normal operation, so that the stability and reliability of the system are improved, and the equipment can be ensured to recover normal work under the abnormal situation.
Further, the configuration management module is used for managing configuration parameters in the internet of things system, storing configuration information of the equipment and the system, and is responsible for synchronizing the configuration parameters to each equipment and each system to keep consistency, verifying validity and correctness when the equipment and the system are updated, supporting record of change history of configuration, guaranteeing safety of configuration data, preventing unauthorized configuration modification, and ensuring that the equipment and the system can operate according to preset configuration.
The invention has the beneficial effects that: the invention provides a dynamic management configuration system of an offline message of the Internet of things, which improves the efficiency and stability of dynamic management of the message in the offline state of the Internet of things by introducing a plurality of modules such as a message generation and release module, a message processing and transmission module, an abnormality detection and recovery module, a configuration management module and the like, generates and releases the message to a message queue, and in the dynamic management configuration system of the offline message of the Internet of things, the message has different priorities and needs to be scheduled and processed according to the priorities, and provides an MFH algorithm to prioritize the message in the message queue according to the importance and the emergency degree of the message, so as to support the dynamic update of the priorities and adjust the priority order of the message in real time. The method comprises the steps of processing repeated messages received by equipment in an offline period, providing an improved machine learning algorithm for isolating forest GIF to process possible abnormal conditions in the process of message transmission and equipment processing, rapidly detecting abnormal data isolated from other data points by analyzing equipment state data or message data, providing a hyper-rectangular block on the basis of the original isolated forest algorithm, and filtering and hanging the detected abnormal messages by utilizing the depth of nodes in a tree to ensure that the abnormal messages do not enter a message queue or a cache any more, thereby identifying possible abnormal conditions. Once the abnormality is found, the recovery module is combined with the aid of the GIF algorithm to assist in abnormality detection to quickly take corresponding recovery measures such as reconnection of a network and restarting equipment, configuration information of the equipment and the system is finally stored, verification is carried out when the equipment and the system configuration is updated, record of change history of the configuration is supported, and the equipment and the system can be ensured to operate according to the preset configuration. Meanwhile, in the internet of things, the device may enter an offline state due to network interruption, signal interference and the like. The offline message dynamic management configuration can ensure that the unreceived message can be transmitted in time after the equipment is on line again, so that the message loss is avoided.
Drawings
The invention will be further described with reference to the accompanying drawings, in which embodiments do not constitute any limitation on the invention, and other drawings can be obtained by one of ordinary skill in the art without undue effort from the following drawings.
Fig. 1 is a schematic diagram of the structure of the present invention.
Detailed Description
The invention will be further described with reference to the following examples.
The system comprises a message generation and release module, a message processing and transmission module, an abnormality detection and recovery module and a configuration management module, wherein the message generation and release module is responsible for generating and releasing messages to a message queue and reasonably classifying and releasing the messages, the message processing and transmission module comprises a message priority scheduling unit and a message deduplication unit, the message priority scheduling unit provides an MFH algorithm to prioritize the messages in the message queue according to the importance degree and the emergency degree of the messages, so that the important messages can be processed as soon as possible, the message deduplication unit processes repeated messages received by equipment in an offline period, the abnormality detection and recovery module provides an improved isolated forest GIF algorithm to process possible abnormal situations in the process of message transmission and equipment processing, and adopts corresponding recovery measures, and the configuration management module is used for managing and adjusting the offline message dynamic management configuration.
Preferably, the message generation and publication module is configured to generate and publish messages to the message queue for subsequent offline message processing and post-device-on-line messaging, as well as other device and background system subscriptions and receptions, based on device sensor data, user requests, or other trigger conditions.
Preferably, the message priority scheduling unit provides the MFH algorithm to sort according to the priority of the messages and dynamically schedule the processing sequence of the messages, so as to ensure that important messages can be processed as soon as possible, avoid the important messages from being delayed to be processed due to the accumulation of low-priority messages, and improve the performance, reliability and response capability of the system.
Specifically, the specific procedure of the proposed MFH algorithm is as follows: the Internet of things is provided withThe same Internet of things equipment is marked as +.>Wherein->Is 1 st Internet of things equipment +.>Is 2 nd Internet of things equipment +.>Is->The station internet of things device, a user sends a request by sending a signal to the device to obtain a desired output, and the turn-around time of the signal is defined as:wherein->For turn-around time of signal,/->The time required for transmitting a signal to the internet of things device and receiving the output result, i.e. the transmission time,/->Time spent for the internet of things device to complete signal execution, i.e. processing time, +.>For time->If the signal needs to wait before executing, update to: />Wherein->Is the total number of signals->In the internet of things environment, assuming that the arrival time and the service time between signals are exponentially distributed, the probability that the internet of things device is occupied is expressed as: />Wherein/>Probability of being busy for Internet of things equipment, +.>For average arrival rate>For average service rate->For the number of the internet of things devices, the average message number of the message queues is defined as:
wherein the method comprises the steps ofThe probability of 0 signals in the internet of things is:
latency in message queuesUse->The turn-around time of the signal is calculated as:since the transmission time and processing of signals is typically fixed, the turnaround time can be minimized by minimizing the average latency of the signals in the information queue, so the minimization problem is expressed as: />And the following constraints need to be satisfied: />、/>And +.>、/>，/>Ensuring that the average number of signals present in the system must have an upper limit +.>，/>Defining upper and lower bounds of signal arrival rate, whereinLower bound representing signal arrival rate, +.>Indicating the upper bound of the signal arrival rate, otherwise the system of the internet of things will become highly fluctuating and unstable, the upper and lower bound of the signal arrival rate depending on the system, +.>、/>Is tightly constrained by 1, otherwise it would result in an indeterminate turnaround time, each signal remaining in the information queue after it arrives at the system, its priority must be calculated for the subsequent execution of each signal while minimizing its latency, assuming two signals (>And->) The waiting internet of things equipment has the priority of +.>And->And signal->At->Previously performed, i.e.)>，/>Wherein->Representing the ordering of the signals, the processing time is a surface to be considered in determining the signal priority, since in almost all cases it depends to a large extent on the size of the signal, expressed by +.>Indicating signal->Execution time of (1), wherein->Indicating signal->Size of->Representation->Bit architecture (I/O)>Representing one million instructions per second, a data structure of a latency matrix is proposed, in which the elements are defined as follows: 1) The diagonal element is set to 1; 2) The waiting time of each signal is calculated and placed in the upper half of the diagonal of the matrix by the order of signal arrival, assuming that the ordering of the signals is +.>Then，/>，/>And +.>WhereinIndicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Latency of->Indicating signal->Execution time of->Indicating signal->Execution time of->Indicating signal->Is expressed as a matrix:
3) The lower half of the matrix diagonal represents the reverse ordering of the signals, so the inverse of the fill latency is
Worst case execution with reduced latency per signalAfter the signal arrives at the Internet of things system, the signal is assigned with priority by using an MFH algorithm, a priority queue is realized by using a fibonacci pile, and each node is->Contains a pointer to its parent node +.>Pointer child node +.>Wherein->The nodes in the heap are represented as such,representation->Is +_a parent node of->Representation->Pointer child node, of->Is in +.>Are connected together in sub-lists, each sub-node in the sub-list +.>All have left +.>And right->Two pointer nodes, wherein->Representing any other child node in the child list, < +.>Representation->When fibonacci heap is empty, new signals based on priority values pass through the nodeThe message processing and transmitting module is inserted into a fibonacci stack, so that the priority of the information is found through an MFH algorithm, and scheduling operation is carried out according to the priority, so that the message processing and transmitting module can acquire and process the next message with higher priority, and the important message can be processed more quickly.
Preferably, the message deduplication unit is used for maintaining a record table or database of a message queue and a message cache, detecting whether the unique identification of the message exists in the message queue or the cache to judge whether the message is a duplicate message or not, and directly filtering out the duplicate messages to optimize data storage and transmission, thereby ensuring the accuracy and idempotent of message processing, optimizing the message queue to reduce unnecessary message transmission, avoiding the duplicate processing and resource waste caused by receiving the duplicate message in an offline period, and improving the performance and response speed of the system.
Preferably, the abnormality detection and recovery module continuously detects the state, sensor data, communication state and the like of the equipment of the Internet of things, detects and analyzes the state data of the equipment by the improved GIF algorithm provided by the invention, judges whether an abnormality occurs and identifies the type and reason of the abnormality, sends an abnormality notification to a relevant system administrator or application program to remind the system administrator or application program of paying attention to the abnormality, and finally adopts corresponding recovery measures to process the abnormality and use the abnormality for subsequent analysis and optimization.
Specifically, the specific steps of the proposed improved GIF algorithm are as follows:
generatingPartitioning the state data by a random tree, wherein +.>Is the number of random trees and calculates eachThe number of nodes required for each training vector to be isolated by the individual tree, the vector with the smallest average path length is detected as outlier data, since the outlier data is more concentrated than the outlier data, more nodes are needed to isolate, a random isolation tree is created, assuming +.>Personal state training data->Wherein n represents the number of training data, +.>Represents training data 1->Represents training data 2->Indicate->Training data, and->Wherein->Representation->Set of dimension real numbers->Indicate->The 1 st component of the training data, +.>Indicate->Number of trainingAccording to component 2>Indicate->Training data->The vectors are re-used->Representing a matrix of all training data acquired, wherein +.>Representation->Dimension data set, ->Represents 1 st->Vector of dimensions>Represents 2 +.>Vector of dimensions>Indicate->Personal->The vector is then divided into two subsets, randomly selected +.>One component of (2) is denoted +.>And atUniformly selecting a split value>Wherein->First->Training data->A component dividing the data set into two parts, left branch corresponding set +.>Right branch corresponding set +.>Creating a random tree by applying the process to each branch in turn until the branch contains a unique data point, the process can be applied to the entire state training set multiple times in order to create a random tree, the boundaries of the hyper-rectangular block being defined as: />Wherein->Indicate->Super rectangular block margin of individual node, +.>Indicate->1-dimensional lower bound of individual nodes, +.>Indicate->1-dimensional upper bound of individual nodes, +.>Indicate->Personal node +.>Upper bound of dimension (I)>Indicate->Personal node +.>Dimension upper bound, assume->To include->A subset of data present in the smallest block of the individual nodes, wherein +.>For the data subset, a random variable is selected from an exponential distribution>，/>The splitting rate of (2) is>Calculated, wherein->Is->Is also the splitting rate ofIs>Representing +.>Upper bound of dimension (I)>Representing +.>Dimension lower bound, assume node ++>Is greater than the splitting time of its parent node plus +.>Then at node->Creating a new node in combination with the previously randomly selected component->Splitting value of uniform selection +.>If data point->At the option->The base value of the component is greater than the upper bound +.>Then from->Uniformly selecting the split value->If the value is smaller than the lower bound +.>Then fromUniformly selecting the split value->According to the split value->Data points->Is created as a node +.>Left and right branches of (1), assume node +.>Is smaller than the splitting time of its parent node plus +.>The calling node, now recursive downwards along the random tree +.>To calculate the isolation point +.>Is>Defining an abnormality score +_>Wherein->Is->Is (are) desirable to be (are)>Is->The size is +.>Is formulated as +.>Wherein->Is->The number of subharmonics (can be approximated as +.>Wherein->) Therefore, the root of Reinforcement>When x is abnormal, the score is +.>When->Tend to->When, that is, when x is not an equal correlation point, the anomaly score tends to 0,when->Relative to->When smaller, i.e. when x is an outlier, the corresponding anomaly score tends to be 1, defining an anomaly threshold +.>When->When (I)>Is detected as abnormal whenWhen (I)>Detected as normal data, the closer the abnormality score is to 1,/->The more likely an abnormality, the closer the abnormality score is to 0, +.>The more likely is normal data, so that the possibly occurring abnormal situation is detected according to the state data of the equipment through the proposed GIF algorithm, measures such as restarting, automatic reconnection and switching of standby equipment are adopted to restore normal operation, the stability and the reliability of the system are improved, and the equipment can be ensured to normally restore work under the abnormal situation.
Preferably, the configuration management module is used for managing configuration parameters in the internet of things system, storing configuration information of the equipment and the system, synchronizing the configuration parameters to each equipment and each system to keep consistency, verifying legality and correctness when the equipment and the system are updated, supporting record of change history of configuration, guaranteeing safety of configuration data, preventing unauthorized configuration modification, ensuring that the equipment and the system can operate according to preset configuration, and reducing operation and maintenance cost while improving adaptability, performance and safety of the system.
According to the system, the information generation and release module, the information processing and transmission module, the abnormality detection and recovery module, the configuration management module and other modules are introduced, so that the efficiency and stability of information dynamic management in an offline state of the Internet of things are improved, the information is generated and released to an information queue, the information has different priorities, the information needs to be scheduled and processed according to the priorities, an MFH algorithm is provided for sequencing the priorities of the information in the information queue according to the importance and the emergency degree of the information, the dynamic update of the priorities is supported, the priority order of the information can be adjusted in real time, a waiting time matrix is provided on the basis of the original algorithm, the insertion and deletion work can be efficiently carried out on the dynamic information, the overhead of the queue maintenance in a processing scene of the dynamic information is small, the minimum (or maximum) value can be found in the constant time complexity, the whole queue does not need to be traversed, the searching efficiency is improved, the situation of information accumulation is reduced, the important information delay processing caused by low priority information accumulation is avoided, and the important information processing can be ensured to be obtained as soon as possible. The method comprises the steps of processing repeated messages received by equipment in an offline period, providing an improved machine learning algorithm for isolating forest GIF to process possible abnormal conditions in the process of message transmission and equipment processing, rapidly detecting abnormal data isolated from other data points by analyzing equipment state data or message data, providing a hyper-rectangular block on the basis of the original isolated forest algorithm, and filtering and hanging the detected abnormal messages by utilizing the depth of nodes in a tree to ensure that the abnormal messages do not enter a message queue or a cache any more, thereby identifying possible abnormal conditions. Once the abnormality is found, the recovery module is combined with the aid of the GIF algorithm to assist in abnormality detection to quickly take corresponding recovery measures such as reconnection of a network and restarting equipment, configuration information of the equipment and the system is finally stored, verification is carried out when the equipment and the system configuration is updated, record of change history of the configuration is supported, and the equipment and the system can be ensured to operate according to the preset configuration. The system for dynamically managing and configuring the offline information of the Internet of things improves the reliability and stability of the system of the Internet of things, optimizes the communication efficiency of equipment, promotes the application of the technology of the Internet of things in various fields, explores more optimization methods and solutions related to the system for dynamically managing and configuring the offline information of the Internet of things, promotes the progress and innovation of academic research, and lays a solid foundation for the construction and development of future intelligent society.
Finally, it should be noted that the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the scope of the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications can be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (4)
1. The system is characterized by comprising a message generation and release module, a message processing and transmission module, an abnormality detection and recovery module and a configuration management module; the message generating and publishing module is responsible for generating and publishing messages to the message queue and reasonably classifying and publishing the messages, the message processing and transmitting module comprises a message priority scheduling unit and a message deduplication unit, the message priority scheduling unit provides an MFH algorithm to order the priorities of the messages in the message queue according to the importance and the emergency degree of the messages, so that the important messages can be processed as soon as possible, the message deduplication unit processes repeated messages received by the equipment in an offline period, the abnormality detection and recovery module provides an improved isolated forest GIF algorithm to process abnormal situations possibly occurring in the message transmission and equipment processing process, corresponding recovery measures are adopted, and the configuration management module is used for managing and adjusting the offline message dynamic management configuration; the message priority scheduling unit puts forward the processing sequence of the MFH algorithm for sequencing and dynamically scheduling the messages according to the priority of the messages; the specific procedure of the proposed MFH algorithm is as follows: let n identical thing networking devices in thing networking, record as S 1 ，S 2 ，…，S n Wherein S is 1 Is the 1 st Internet of things equipment, S 2 Is the 2 nd Internet of things equipment, S n For the nth internet of things device, a user sends a request by sending a signal to the device to obtain a desired output, and the turn-around time of the signal is defined as: TT (T) i )＝T T (T i )+T P (T i ) Wherein TT is the turnaround time of the signal, T T For the time required for transmitting the signal to the internet of things equipment and receiving the output result, i.e. the transmission time, T P Time spent for the Internet of things equipment to complete signal execution, namely processing time, T i If the total number of signals at time i is the number of signals to be waited before execution, the total number of signals is updated as follows: TT (T) i )＝T T (T i )+T P (T i )+T W (T i ) Wherein T is W (T i ) For the total number of signals T i In the internet of things environment, assuming that the arrival time and the service time between signals are exponentially distributed, the probability that the internet of things device is occupied is expressed as:wherein P is Busy For the probability that the internet of things equipment is busy, alpha is the average arrival rate, beta is the average service rate, n is the number of the internet of things equipment, and the average message number of the message queues is defined as:
wherein P is 0 The probability of 0 signals in the internet of things is:
latency T in message queue W By usingThe turn-around time of the signal is calculated as:the transmission time and processing of the signals is fixed, minimizing the turnaround time by minimizing the average latency of the signals in the information queue, so the minimization problem is expressed as: />And satisfies the following constraints: q is 1 to or less M ≤m、α l ≤α≤α u P 0 、P Busy < 1, assume two signals T i And T j Waiting for the Internet of things equipment, wherein the priorities are x and y respectively, and the signal T i At T j Previously performed, i.e. x > y, T i →T j Wherein→the ordering of representative signals, using the formula Representing signal T i In which size (T) i ) Representing signal T i N represents an n-bit architecture, MIPS represents one million instructions per second, and a data structure of a latency matrix is proposed, in which the elements are defined as follows: 1) The diagonal element is set to 1; 2) Performed in the order of arrival of the signals, the latency of each signal is calculated and placed in the upper half of the diagonal of the matrix, assuming that the ordering of the signals is T 1 →T 2 →T 3 →T 4 T is then W (T 1 )＝0，T W (T 2 )＝T P (T 1 )，T W (T 3 )＝T P (T 1 )+T P (T 2 ) T is as follows W (T 4 )＝T P (T 1 )+T P (T 2 )+T P (T 3 ) Wherein T is W (T 1 ) Representing signal T 1 Latency of T W (T 2 ) Representing signal T 2 Latency of T W (T 3 ) Representing signal T 3 Latency of T W (T 4 ) Representing signal T 4 Latency of T P (T 1 ) Representing signal T 1 Execution time of T P (T 2 ) Representing signal T 2 Execution time of T P (T 3 ) Representing signal T 3 Is expressed as a matrix:
3) The inverse value of the filling waiting time of the lower half of the diagonal line of the matrix is
After the signal arrives at the Internet of things system, the MFH algorithm is used for distributing priority to the signal, the priority queue is realized by using a fibonacci pile, each node x comprises a pointer P [ x ] pointing to a father node and a pointer child node [ x ] pointing to any child node of the node x, wherein x represents the node in the pile, P [ x ] represents the father node of x, and [ x ] represents the pointer child node of x, the child elements of x are connected together in a child list of x through a circular double linked list, each child node y in the child list is provided with a left [ y ] and a right [ y ] pointer node, wherein y represents any other child node in the child list, and [ y ] represents the pointer node of y, when the fibonacci pile is empty, a new signal based on a priority value is inserted into the fibonacci pile through the node, so that the priority of information is found through the node, and scheduling operation is performed according to the priority, so that the message processing and the transmission of the message are more important to ensure that the message processing of the next priority module is more quickly processed;
the abnormality detection and recovery module continuously detects the state, sensor data, communication state and the like of the Internet of things equipment, detects and analyzes the equipment state data through the improved GIF algorithm, judges whether an abnormality occurs and identifies the type and reason of the abnormality, then sends an abnormality notification to a relevant system administrator or application program to remind the system administrator or application program of noticing the abnormality, and finally adopts corresponding recovery measures to process the abnormality and use for subsequent analysis and optimization, and the specific steps of the improved GIF algorithm are as follows:
generating t, t > 0 random trees to partition the state data, wherein t is the number of the random trees, calculating the node number required by each tree to isolate each training vector, detecting the vector with the minimum average path length as abnormal data, creating a random isolation tree, and supposing that n state training data (x 1 ，x 2 ，...，x n ) Where n represents the number of training data, x 1 Represent training data 1 st, x 2 Represent training data 2, x n Represents the nth training data, and x i ＝[x i，1 ，x i，2 ，...，x i，d ] T ∈R d Wherein R is d Representing d-dimensional real number set, x i，1 Represents the 1 st component, x, of the ith training data i，2 Represents the 2 nd component, x, of the ith training data i，d Represents the ith training data and the (d) th vector, and then uses X= [ X ] 1 ，x 2 ，...，x n ]∈R n×d Representing a matrix of all training data acquired, where R n×d Representing an n x d-dimensional dataset, x 1 Represents the 1 st d-dimensional vector, x 2 Represents the 2 nd d-dimensional vector, x n Representing the nth d-dimensional vector, then creating a random node and dividing the state data set into two subsets, randomly selecting R d One component of (2) is denoted as q and is represented in [ min ] i＝1，2，...，n x i，q ；max i＝1，2，...，n x i，q ]Uniformly selecting a split value p, wherein x i，q The (th) component of the (th) training data, the data set is divided into two parts, and the left branch corresponds to the (x) set i ，x i，q P is less than or equal to, the right branch corresponds to the set { x } i ，x i，q > p, a random tree is created by applying the process to each branch in turn until the branch contains a unique data point, the boundary of the hyper-rectangular block being defined as: b (B) r ＝(l r1 ，u r1 ]×...×(l rd ，u rd ]Wherein B is r Hyper-rectangular block margin representing the r node, l r1 Representing the 1-dimensional lower bound of the r-th node, u r1 1-dimensional upper bound representing the r-th node, l rd Represents the d-dimensional upper bound of the r-th node, u rd The d-dimensional upper bound representing the r-th node, assuming X b ＝{x i |x i ∈B r Is the subset of data present in the smallest block containing the nth node, where X b For the subset of data, a random variable e, is selected from an exponential distribution, the split rate of which is represented by the formulaCalculated, where σ is the splitting rate of e, also B r Is>J-dimensional upper bound representing a subset of data, +.>A j-dimensional lower bound representing a subset of data, if the splitting time of a node r is greater than the splitting time of its parent node plus e, a new node is created on the node r, combining the previously randomly selected component q with the uniformly selected splitting value p, if the value of the data point x is greater than the upper bound u of the current block on the basis of selecting the q component rd Then from [ x ] r，q ；max i＝1，n，...，n x i，q ]Uniformly selecting the fraction value p, if the fraction value is smaller than the lower limit l of the current block rd Then from [ min ] i＝1，n，...，n x i，q ；x r，q ]Uniformly selecting a split value p, creating a random tree as left and right branches of a node r according to the features of the split value p and a data point x, and assuming that the splitting time of the node r is less than the splitting time of a parent node plus e, then along the randomThe tree recursively calls the left and right branches of node r down, calculates the desired path length h (x) of the isolation point x using the average of the path lengths required for each generated random tree isolation point, defining an anomaly score +_>Wherein E [ h (x)]Is the expectation of h (x), c (n) is the data set with h (x) of size n, expressed by +.>Wherein H (n) is the nth harmonic number, so when E [ H (x) ]]When c (n), the anomaly score of x is s (x) =0.5, when h (x) tends to + -infinity, the anomaly score tends to 0, when h (x) is smaller relative to c (n), the corresponding anomaly score tends to 1, defining an anomaly threshold s 0 ∈[0，1]When s (x) > s 0 When x is detected as abnormal, when s (x). Ltoreq.s 0 And when x is detected as normal data, the possibly occurring abnormal situation is detected according to the state data of the equipment through the proposed GIF algorithm, and equipment restarting, automatic reconnection and standby equipment switching measures are adopted to recover normal operation, so that the stability and reliability of the system are improved, and the equipment can be ensured to recover normal work under the abnormal situation.
2. The system of claim 1, wherein the message generation and distribution module is configured to generate and distribute messages to the message queue for subsequent offline message processing and post-device-on-line message delivery, as well as other device and background system subscription and reception, based on device sensor data, user requests, or other trigger conditions.
3. The system according to claim 1, wherein the message deduplication unit is configured to maintain a record table or database of a message queue and a message cache, and determine whether to duplicate messages by generating unique identifiers of messages, detecting whether the identifiers are already present in the message queue or cache, and directly filtering out the duplicate messages to optimize data storage and transmission.
4. The system according to claim 1, wherein the configuration management module is configured to manage configuration parameters in the system of the internet of things, store configuration information of the devices and the system, synchronize the configuration parameters to each device and the system to maintain consistency, perform validity and correctness verification when updating configuration of the devices and the system, support change history of the recorded configuration, ensure security of configuration data, prevent unauthorized configuration modification, and ensure that the devices and the system can operate according to a predetermined configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311343102.9A CN117240903B (en) | 2023-10-17 | 2023-10-17 | Internet of things offline message dynamic management configuration system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311343102.9A CN117240903B (en) | 2023-10-17 | 2023-10-17 | Internet of things offline message dynamic management configuration system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117240903A CN117240903A (en) | 2023-12-15 |
| CN117240903B true CN117240903B (en) | 2024-02-23 |
Family
ID=89082668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311343102.9A Active CN117240903B (en) | 2023-10-17 | 2023-10-17 | Internet of things offline message dynamic management configuration system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117240903B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104468819A (en) * | 2014-12-24 | 2015-03-25 | 福建江夏学院 | Message pushing system and method for internet of things |
| CN110636013A (en) * | 2019-09-30 | 2019-12-31 | 佛山科学技术学院 | Dynamic scheduling method and device for message queue |
| CN112822245A (en) * | 2020-12-31 | 2021-05-18 | 无锡恺易物联网科技发展有限公司 | Agricultural Internet of things big data access system and method |
| CN115344839A (en) * | 2022-06-30 | 2022-11-15 | 西安电子科技大学 | Identity authentication method and device based on user keystroke dynamics |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11237897B2 (en) * | 2019-07-25 | 2022-02-01 | International Business Machines Corporation | Detecting and responding to an anomaly in an event log |
| US11531676B2 (en) * | 2020-07-29 | 2022-12-20 | Intuit Inc. | Method and system for anomaly detection based on statistical closed-form isolation forest analysis |
-
2023
- 2023-10-17 CN CN202311343102.9A patent/CN117240903B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104468819A (en) * | 2014-12-24 | 2015-03-25 | 福建江夏学院 | Message pushing system and method for internet of things |
| CN110636013A (en) * | 2019-09-30 | 2019-12-31 | 佛山科学技术学院 | Dynamic scheduling method and device for message queue |
| CN112822245A (en) * | 2020-12-31 | 2021-05-18 | 无锡恺易物联网科技发展有限公司 | Agricultural Internet of things big data access system and method |
| CN115344839A (en) * | 2022-06-30 | 2022-11-15 | 西安电子科技大学 | Identity authentication method and device based on user keystroke dynamics |
Non-Patent Citations (1)
| Title |
|---|
| 乔治 ; 周川 ; 纪现才 ; 曹亚男 ; 郭莉 ; .基于地域特征和异构社交关系的事件推荐算法研究.中文信息学报.2016,(05),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117240903A (en) | 2023-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107968840B (en) | Real-time processing method and system for monitoring alarm data of large-scale power equipment | |
| US4630264A (en) | Efficient contention-resolution protocol for local multiaccess networks | |
| CN108696374B (en) | Method and device for updating client configuration | |
| CN104092719B (en) | Document transmission method, device and distributed cluster file system | |
| CN107451853B (en) | Method, device and system for real-time red packet distribution and storage medium | |
| CN113596150B (en) | Message pushing method, device, computer equipment and storage medium | |
| CN113422842B (en) | Distributed power utilization information data acquisition system considering network load | |
| CN113722127A (en) | Efficient lightweight easy-to-use distributed network message middleware | |
| CN111966289A (en) | Partition optimization method and system based on Kafka cluster | |
| Farazi et al. | Average age of information in multi-source self-preemptive status update systems with packet delivery errors | |
| US20160065660A1 (en) | Computer system, computer, and load balancing method | |
| WO2016074714A1 (en) | Method for providing m2m data | |
| JP7348293B2 (en) | Data processing methods and equipment | |
| CN110727507B (en) | Message processing method and device, computer equipment and storage medium | |
| CN107104956B (en) | A kind of remote sensing distribution method and system based on user's priority | |
| WO2017215415A1 (en) | Resource control method and apparatus, and iptv server | |
| JP2014531072A (en) | Distributing events to many devices | |
| EP3998754B1 (en) | Data distribution method, storage server and subscription server | |
| CN117240903B (en) | Internet of things offline message dynamic management configuration system | |
| CN111352746B (en) | Message flow limiting method and storage medium | |
| CN113301558B (en) | Message transmission method, device, system and storage medium | |
| CN117614797A (en) | Data acquisition system and method | |
| CN111756800A (en) | Method and system for processing burst flow | |
| CN110750362A (en) | Method and apparatus for analyzing biological information, and storage medium | |
| CN113965538B (en) | Equipment state message processing method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |