CN113704057A - Distributed data management system based on block chain - Google Patents
Distributed data management system based on block chain Download PDFInfo
- Publication number
- CN113704057A CN113704057A CN202110881890.1A CN202110881890A CN113704057A CN 113704057 A CN113704057 A CN 113704057A CN 202110881890 A CN202110881890 A CN 202110881890A CN 113704057 A CN113704057 A CN 113704057A
- Authority
- CN
- China
- Prior art keywords
- data
- equipment
- central server
- time
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a distributed data management system based on a block chain, which comprises a device continuous operation state real-time monitoring module, an operation device self identification information registration module, a central server data management platform, a redundant data timing cleaning module and a corresponding data real-time feedback module, wherein the device continuous operation state real-time monitoring module is used for monitoring the device state of trying to access a central server, the operation device self identification information registration module is used for uploading the self identification before the operation device is accessed to the central server, the central server data management platform is used for realizing the real-time data interaction among operation devices, the redundant data timing cleaning module is used for cleaning the redundant data in the central server at regular time, the corresponding data real-time feedback module is used for monitoring the data information uploaded in the operation state, the abnormal data are fed back, the equipment is managed, and the data management efficiency is improved.
Description
Technical Field
The invention relates to the field of distributed management, in particular to a distributed data management system based on a block chain.
Background
As distributed systems grow and resources are scattered from ever centralized domains, the issues of managing hardware, software, users, updates, configuration, and security have diversified. And in addition there is a synchronization problem associated with internal independent system changes. Differences in operating systems, hardware, or protocols make this more complex.
The distributed management system needs to provide the following functions: managing user and workstation configuration, managing software distribution, update and permissions, managing hardware monitoring, maintenance and cataloging functions, and in addition, the management system should provide a means of collecting network data and reporting these data to the administrator.
A distributed network management system may take advantage of the network platform to spread management functions across a network rather than centralizing them in a single data center. The administrator can still run the management system from one location, while the management authority distributed over the network collects the information and responds to the management system. Information that was unavailable due to time and transmission limitations is now available to the administrator. Fault-tolerant management and preventive maintenance are also greatly simplified. Warnings may also be raised of impending problems.
At present, the equipment inserts management platform only need send the connection request, the management platform can not filter the equipment of non-operation state, data management is handled after receiving the data that equipment sent, but equipment carries out data transmission and often can appear repeating data, can lead to the management platform to manage to repeating data can't arrange in order or directly not manage, can lead to repeated invalid data more and more, this application aims at managing equipment, avoid appearing non-operation state equipment and inserts the management platform, the data that the similarity is high that the management platform received are screened and cleared up, improve data management efficiency.
Disclosure of Invention
The invention aims to provide a distributed data management system and a distributed data management method based on a block chain, which aim to solve the problems in the prior art.
In order to achieve the purpose, the invention provides the following technical scheme:
a distributed data management system based on a block chain comprises a device continuous operation state real-time monitoring module, an operation device self identification information registration module, a central server data management platform, a redundant data timing cleaning module and a corresponding data real-time feedback module; the system comprises a device continuous operation state real-time monitoring module, an operation device self identification information registration module and a central server data management platform, wherein the device continuous operation state real-time monitoring module, the operation device self identification information registration module and the central server data management platform are sequentially connected through an intranet;
the device continuous operation state real-time monitoring module is used for monitoring the state of a device trying to access a central server, the operation device self identification information registration module is used for uploading a self identification before the operation device is accessed into the central server, the central server data management platform is used for realizing real-time data interaction between operation devices, the redundant data timing cleaning module is used for cleaning redundant data inside the central server at regular time, and the corresponding data real-time feedback module is used for monitoring data information uploaded in the operation state and feeding back abnormal data.
By adopting the technical scheme: the device continuous operation state real-time monitoring module comprises a device operation rate analysis submodule in preset time and a non-operation state device disconnection submodule, the device operation rate analysis submodule in the preset time is used for analyzing the operation rate of a device trying to access a central server in the preset time, the state of the device in the preset time is determined according to the operation rates of different devices, when the operation rate of the device is lower than a set threshold value, the device is marked, all marked devices are extracted, the extracted device is sent to the non-operation state device disconnection submodule, the non-operation state device disconnection submodule is used for enabling the central server to refute the trying connection request sent by the device extracted by the device operation rate analysis submodule in the preset time, and the connection request of the device with the operation rate lower than the set threshold value is not accepted any more.
By adopting the technical scheme: the equipment operation rate analysis submodule sets the time for monitoring the operation states of different equipment to be ti-tj within the preset time, and sets the operation time of the different equipment to be T within the preset time1、T2、T3、…、Tn-1、TnSetting the downtime of different equipment in the above time as t1、t2、t3、…、tn-1、tnWherein, the time for setting each device to be started but not to transmit data is T0, the operation rate of the device is K, and the operation rate is determined according to the publicFormula (II):
and calculating the equipment operation rates of different attempts to access the central server, responding to the connection attempt request sent by the equipment by the central server when the equipment operation rate is greater than or equal to 50%, rejecting the connection attempt request sent by the equipment by the central server when the equipment operation rate is less than 50%, and not receiving the connection request of the equipment with the operation rate lower than a set threshold value any more.
By adopting the technical scheme: the operating equipment self identification information registration module comprises an operating equipment identification uploading submodule and an operating equipment access request submodule, wherein the operating equipment identification uploading submodule counts the IP address and the ID of equipment trying to access the operating equipment of the central server and sends the counted information to the central server data management platform for authorized login, and the operating equipment access request submodule is used for sending an access request to the central server data management platform again.
By adopting the technical scheme: the central server data management platform comprises an operating equipment access response submodule and an inter-equipment data information real-time interaction submodule, wherein the operating equipment access response submodule is used for accessing after responding to an access request of operating equipment, and the inter-equipment data information real-time interaction submodule is used for enabling different equipment accessed into the central server data management platform to send data information to the central server for data processing and calling data information in the central server at any time so as to realize data interaction.
By adopting the technical scheme: the timing cleaning module for the redundant data comprises an operation equipment uploading data volume counting and sorting submodule and a key mark operation equipment internal data similarity analyzing submodule, wherein the operation equipment uploading data volume counting and sorting submodule is used for counting the data volume uploaded by the operation equipment, sorting different statistical equipment data volumes and marking the operation equipment with the first five data volumes, the key mark operation equipment internal data similarity analyzing submodule is used for analyzing the data similarity sent by the operation equipment with the first five data volumes, when two data segments with the similarity being more than or equal to 93% exist in the operation equipment, one complete similar part of the two data segments is selected and deleted, and when two data segments with the similarity being less than 93% and more than or equal to 65% exist in the operation equipment, the two data segments are marked, and sending the marked data segments to a corresponding data real-time feedback module, and if all the data segments exist in the operating equipment and the similarity of all the data segments is less than 65%, not processing the operating equipment.
By adopting the technical scheme: the corresponding data real-time feedback module comprises a data repeated transmission marking submodule in the directional time segment and an abnormal data manual feedback channel, the data repeated transmission marking submodule in the directional time segment is used for detecting that the data repeated transmission marking submodule in different time segments are detected, different operating equipment sends repeated data to the central server data management platform, the repeated data are marked, the marked repeated data are sent to the abnormal data manual feedback channel, and the abnormal data manual feedback channel is used for receiving the marked repeated data and manually processing the data segments marked by the data similarity analysis submodule in the key mark operating equipment.
A distributed data management method based on a block chain comprises the following steps:
s1: monitoring the state of the equipment trying to access the central server by using an equipment continuous running state real-time monitoring module;
s2: uploading the self identification before the running equipment is accessed to the central server by utilizing a self identification information registration module of the running equipment;
s3: real-time data interaction among the operating devices is realized by utilizing a central server data management platform;
s4: utilizing a redundant and miscellaneous data timing cleaning module to perform timing cleaning on redundant and miscellaneous data in a central server;
s5: and monitoring the data information uploaded in the running state by using a corresponding data real-time feedback module, and feeding back abnormal data.
By adopting the technical scheme: the data management method further comprises the following steps:
s1-1: analyzing the operation rate of equipment trying to access a central server within preset time by using an equipment operation rate analysis submodule within the preset time, determining the state of the equipment within the preset time according to the operation rates of different equipment, marking the equipment when the operation rate of the equipment is lower than a set threshold, extracting all marked equipment, sending the extracted equipment to a non-operation state equipment disconnection submodule, enabling the central server to refute a connection trying request sent by the equipment extracted by the equipment operation rate analysis submodule within the preset time, and not receiving the connection request of the equipment of which the operation rate is lower than the set threshold;
s2-1: the operating equipment identification uploading submodule is used for counting the IP address and the ID of the equipment trying to access the operating equipment of the central server, the counted information is sent to the central server data management platform for authorized login, and the operating equipment access request submodule sends an access request to the central server data management platform again;
s3-1: the operating equipment access response submodule is used for responding an access request of the operating equipment and then accessing, and the data information real-time interaction submodule among the equipment enables different equipment accessed to the data management platform of the central server to send data information to the central server for data processing and call the data information in the central server at any time, so that data interaction is realized;
s4-1: the data volume uploaded by the operating equipment is counted by using an operating equipment uploading data volume counting and sorting submodule, different counted equipment data volumes are sorted, the operating equipment with the first five data volume sorted is marked, a data similarity analyzing submodule inside the operating equipment with the important mark analyzes the data similarity sent by the operating equipment with the first five data volume sorted, when two data segments with the similarity of more than or equal to 93 percent exist inside the operating equipment, the completely similar part of the two data segments is selected to be deleted, when two data segments with the similarity of less than 93 percent and more than or equal to 65 percent exist inside the operating equipment, the two data segments are marked, the marked data segments are sent to a corresponding data real-time feedback module, when all the data segments with the similarity of less than 65 percent exist inside the operating equipment, the operating equipment is not processed;
s5-1: and detecting that the data repeated transmission marking submodule in the directional time segment is in different time segments, sending repeated data to a central server data management platform by different running equipment, marking the repeated data, sending the marked repeated data to an abnormal data manual feedback channel, and receiving the marked repeated data by the abnormal data manual feedback channel and manually processing the data segments marked by the key marking running equipment internal data similarity analysis submodule.
By adopting the technical scheme: in step S4-1, the statistical sorting submodule for the data size uploaded by the operating device is used to count the data size uploaded by the operating device, sort the different device data sizes that are counted, mark the operating device that is the first five times of the data size sorting, and the analysis submodule for the data similarity inside the marked operating device analyzes the data similarity sent inside the operating device that is the first five times of the data size sorting, which further includes the following steps:
the analysis submodule for setting the similarity of the data in the key mark operation equipment analyzes the data in the operation equipment A, segments the data in the operation equipment A, and sets the segmented data to be L1、L2、L3、…、Ln-1、LnAnd carrying out pairwise similarity comparison on the divided fragment data. Setting the similarity of different fragment data as C1、C2、C3、…、Cn-1、CnIt is set that there is a similar error rate of 2% in each piece of data, and the actual similarity of each piece of data is set to R (C)n) According to the formula:
R(Cn)＝Cn*(1-2％)
calculating the actual similarity of each fragment data in the current device A when R (C)n) More than or equal to 93 percent, extracting two fragment data of similarity contrast, carrying out similarity contrast again for three times, and setting weightThe new contrast has a similarity of C01、C02、C03Setting the similarity mean of the current two segment data for re-comparison as
When is coming into contact with
If the two data segments are completely similar, the two data segments are marked and sent to the corresponding data real-time feedback module;
when R (C) is more than or equal to 65 ≦ Rn) If the data rate is less than 93%, marking two data segments monitored in the equipment A, and sending the marked data segments to a corresponding data real-time feedback module;
when R (C)n) If the ratio is less than 65%, the operation equipment A is not processed.
Compared with the prior art, the invention has the beneficial effects that: the method aims to manage the equipment, avoid the equipment in a non-running state from accessing the management platform, screen and clear the data with high similarity received by the management platform and improve the data management efficiency;
the device continuous operation state real-time monitoring module is used for monitoring the state of a device trying to access a central server, the identification information registration module of the operation device is used for uploading the identification of the operation device before the operation device is accessed into the central server, the data management platform of the central server is used for realizing real-time data interaction between operation devices, the redundant data timing cleaning module is used for cleaning redundant data inside the central server at regular time, and the corresponding data real-time feedback module is used for monitoring the data information uploaded by the operation state and feeding back abnormal data.
Drawings
In order that the present invention may be more readily and clearly understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings.
FIG. 1 is a block chain-based distributed data management system;
FIG. 2 is a schematic diagram illustrating steps of a block chain-based distributed data management method according to the present invention;
FIG. 3 is a diagram illustrating specific steps of a block chain-based distributed data management method according to the present invention;
fig. 4 is a schematic diagram of an implementation method of a block chain-based distributed data management method according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1 to 4, in the embodiment of the present invention, a distributed data management system and method based on a block chain are provided, where the system includes a device continuous operation state real-time monitoring module, an operation device self-identification information registration module, a central server data management platform, a redundant data timing cleaning module, and a corresponding data real-time feedback module; the system comprises a device continuous operation state real-time monitoring module, an operation device self identification information registration module and a central server data management platform, wherein the device continuous operation state real-time monitoring module, the operation device self identification information registration module and the central server data management platform are sequentially connected through an intranet;
the device continuous operation state real-time monitoring module is used for monitoring the state of a device trying to access a central server, the operation device self identification information registration module is used for uploading a self identification before the operation device is accessed into the central server, the central server data management platform is used for realizing real-time data interaction between operation devices, the redundant data timing cleaning module is used for cleaning redundant data inside the central server at regular time, and the corresponding data real-time feedback module is used for monitoring data information uploaded in the operation state and feeding back abnormal data.
By adopting the technical scheme: the device continuous operation state real-time monitoring module comprises a device operation rate analysis submodule in preset time and a non-operation state device disconnection submodule, the device operation rate analysis submodule in the preset time is used for analyzing the operation rate of a device trying to access a central server in the preset time, the state of the device in the preset time is determined according to the operation rates of different devices, when the operation rate of the device is lower than a set threshold value, the device is marked, all marked devices are extracted, the extracted device is sent to the non-operation state device disconnection submodule, the non-operation state device disconnection submodule is used for enabling the central server to refute the trying connection request sent by the device extracted by the device operation rate analysis submodule in the preset time, and the connection request of the device with the operation rate lower than the set threshold value is not accepted any more.
By adopting the technical scheme: the equipment operation rate analysis submodule sets the time for monitoring the operation states of different equipment to be ti-tj within the preset time, and sets the operation time of the different equipment to be T within the preset time1、T2、T3、…、Tn-1、TnSetting the downtime of different equipment in the above time as t1、t2、t3、…、tn-1、tnWherein, the time for setting each device to be started but not to transmit data is T0, the device operation rate is K, and according to the formula:
and calculating the equipment operation rates of different attempts to access the central server, responding to the connection attempt request sent by the equipment by the central server when the equipment operation rate is greater than or equal to 50%, rejecting the connection attempt request sent by the equipment by the central server when the equipment operation rate is less than 50%, and not receiving the connection request of the equipment with the operation rate lower than a set threshold value any more.
By adopting the technical scheme: the operating equipment self identification information registration module comprises an operating equipment identification uploading submodule and an operating equipment access request submodule, wherein the operating equipment identification uploading submodule counts the IP address and the ID of equipment trying to access the operating equipment of the central server and sends the counted information to the central server data management platform for authorized login, and the operating equipment access request submodule is used for sending an access request to the central server data management platform again.
By adopting the technical scheme: the central server data management platform comprises an operating equipment access response submodule and an inter-equipment data information real-time interaction submodule, wherein the operating equipment access response submodule is used for accessing after responding to an access request of operating equipment, and the inter-equipment data information real-time interaction submodule is used for enabling different equipment accessed into the central server data management platform to send data information to the central server for data processing and calling data information in the central server at any time so as to realize data interaction.
By adopting the technical scheme: the timing cleaning module for the redundant data comprises an operation equipment uploading data volume counting and sorting submodule and a key mark operation equipment internal data similarity analyzing submodule, wherein the operation equipment uploading data volume counting and sorting submodule is used for counting the data volume uploaded by the operation equipment, sorting different statistical equipment data volumes and marking the operation equipment with the first five data volumes, the key mark operation equipment internal data similarity analyzing submodule is used for analyzing the data similarity sent by the operation equipment with the first five data volumes, when two data segments with the similarity being more than or equal to 93% exist in the operation equipment, one complete similar part of the two data segments is selected and deleted, and when two data segments with the similarity being less than 93% and more than or equal to 65% exist in the operation equipment, the two data segments are marked, and sending the marked data segments to a corresponding data real-time feedback module, and if all the data segments exist in the operating equipment and the similarity of all the data segments is less than 65%, not processing the operating equipment.
By adopting the technical scheme: the corresponding data real-time feedback module comprises a data repeated transmission marking submodule in the directional time segment and an abnormal data manual feedback channel, the data repeated transmission marking submodule in the directional time segment is used for detecting that the data repeated transmission marking submodule in different time segments are detected, different operating equipment sends repeated data to the central server data management platform, the repeated data are marked, the marked repeated data are sent to the abnormal data manual feedback channel, and the abnormal data manual feedback channel is used for receiving the marked repeated data and manually processing the data segments marked by the data similarity analysis submodule in the key mark operating equipment.
A distributed data management method based on a block chain comprises the following steps:
s1: monitoring the state of the equipment trying to access the central server by using an equipment continuous running state real-time monitoring module;
s2: uploading the self identification before the running equipment is accessed to the central server by utilizing a self identification information registration module of the running equipment;
s3: real-time data interaction among the operating devices is realized by utilizing a central server data management platform;
s4: utilizing a redundant and miscellaneous data timing cleaning module to perform timing cleaning on redundant and miscellaneous data in a central server;
s5: and monitoring the data information uploaded in the running state by using a corresponding data real-time feedback module, and feeding back abnormal data.
By adopting the technical scheme: the data management method further comprises the following steps:
s1-1: analyzing the operation rate of equipment trying to access a central server within preset time by using an equipment operation rate analysis submodule within the preset time, determining the state of the equipment within the preset time according to the operation rates of different equipment, marking the equipment when the operation rate of the equipment is lower than a set threshold, extracting all marked equipment, sending the extracted equipment to a non-operation state equipment disconnection submodule, enabling the central server to refute a connection trying request sent by the equipment extracted by the equipment operation rate analysis submodule within the preset time, and not receiving the connection request of the equipment of which the operation rate is lower than the set threshold;
s2-1: the operating equipment identification uploading submodule is used for counting the IP address and the ID of the equipment trying to access the operating equipment of the central server, the counted information is sent to the central server data management platform for authorized login, and the operating equipment access request submodule sends an access request to the central server data management platform again;
s3-1: the operating equipment access response submodule is used for responding an access request of the operating equipment and then accessing, and the data information real-time interaction submodule among the equipment enables different equipment accessed to the data management platform of the central server to send data information to the central server for data processing and call the data information in the central server at any time, so that data interaction is realized;
s4-1: the data volume uploaded by the operating equipment is counted by using an operating equipment uploading data volume counting and sorting submodule, different counted equipment data volumes are sorted, the operating equipment with the first five data volume sorted is marked, a data similarity analyzing submodule inside the operating equipment with the important mark analyzes the data similarity sent by the operating equipment with the first five data volume sorted, when two data segments with the similarity of more than or equal to 93 percent exist inside the operating equipment, the completely similar part of the two data segments is selected to be deleted, when two data segments with the similarity of less than 93 percent and more than or equal to 65 percent exist inside the operating equipment, the two data segments are marked, the marked data segments are sent to a corresponding data real-time feedback module, when all the data segments with the similarity of less than 65 percent exist inside the operating equipment, the operating equipment is not processed;
s5-1: and detecting that the data repeated transmission marking submodule in the directional time segment is in different time segments, sending repeated data to a central server data management platform by different running equipment, marking the repeated data, sending the marked repeated data to an abnormal data manual feedback channel, and receiving the marked repeated data by the abnormal data manual feedback channel and manually processing the data segments marked by the key marking running equipment internal data similarity analysis submodule.
By adopting the technical scheme: in step S4-1, the statistical sorting submodule for the data size uploaded by the operating device is used to count the data size uploaded by the operating device, sort the different device data sizes that are counted, mark the operating device that is the first five times of the data size sorting, and the analysis submodule for the data similarity inside the marked operating device analyzes the data similarity sent inside the operating device that is the first five times of the data size sorting, which further includes the following steps:
the analysis submodule for setting the similarity of the data in the key mark operation equipment analyzes the data in the operation equipment A, segments the data in the operation equipment A, and sets the segmented data to be L1、L2、L3、…、Ln-1、LnAnd carrying out pairwise similarity comparison on the divided fragment data. Setting the similarity of different fragment data as C1、C2、C3、…、Cn-1、CnIt is set that there is a similar error rate of 2% in each piece of data, and the actual similarity of each piece of data is set to R (C)n) According to the formula:
R(Cn)＝Cn*(1-2％)
calculating the actual similarity of each fragment data in the current device A when R (C)n) Not less than 93%, extracting two fragment data of similarity comparison, performing similarity comparison again for three times, and setting the similarity of re-comparison as C01、C02、C03Setting the similarity mean of the current two segment data for re-comparison as
When is coming into contact with
If the two data segments are completely similar, the two data segments are marked and sent to the corresponding data real-time feedback module;
when R (C) is more than or equal to 65 ≦ Rn)＜93Percent, marking two data segments monitored in the equipment A, and sending the marked data segments to a corresponding data real-time feedback module;
when R (C)n) If the ratio is less than 65%, the operation equipment A is not processed.
Example 1: and under the limiting conditions, the equipment operation rate analysis submodule in the preset time sets the time for monitoring the operation states of different equipment to be ti-tj, sets the operation time of the equipment B in the preset time to be 124h, and sets the downtime of the equipment B in the preset time to be 78h, wherein the time for starting each piece of equipment but not transmitting data is set to be 12h, the equipment operation rate is set to be K, and according to the formula:
the operation rate of the equipment B which tries to access the central server differently is calculated to be 63.5%, the operation rate of the equipment B is calculated to be 63.5% and is more than 50%, and the central server responds to the connection trying request sent by the equipment B.
Example 2: and under the limiting conditions, the equipment operation rate analysis submodule in the preset time sets the time for monitoring the operation states of different equipment to be ti-tj, sets the operation time of the equipment B in the preset time to be 92h, and sets the downtime of the equipment B in the preset time to be 216h, wherein the time for starting each piece of equipment but not transmitting data is set to be 41h, the equipment operation rate is set to be K, and according to the formula:
the operation rate of the equipment B which tries to access the central server differently is 38.1 percent, the operation rate of the equipment B is 63.5 percent and is less than 50 percent, and the central server rejects the connection trying request sent by the equipment B and does not accept the connection trying request of the equipment B any more.
Example 3: limiting conditions, setting a data similarity analysis submodule inside the key mark operation equipment to analyze the data inside the operation equipment A, and analyzing the data inside the operation equipment AAnd (3) carrying out fragment division on the data, setting the divided fragment data as L1, L2 and L3, and carrying out pairwise similarity comparison on the divided fragment data. The degrees of similarity of the different fragment data were set to 98%, 78%, 41%, the similar error rate in each fragment data was set to 2%, and the actual degree of similarity of each fragment data was set to R (C)n) According to the formula:
R1(Cn)＝98％*(1-2％)≈96％
R2(Cn)＝78％*(1-2％)≈76％
R3(Cn)＝41％*(1-2％)≈40％
the actual similarity of each fragment data in the current device A is calculated when R1 (C)n) Not less than 93%, extracting two fragment data L1, L2 of similarity contrast, repeating similarity contrast three times, setting the similarity of re-contrast at 95%, 94%, 96%, and setting the mean of the similarity of re-contrast of current two fragment data at 95%, 94%, 96%
When 95% is more than or equal to 93%, the completely similar parts of the two data segments L1 and L2 monitored in the device A are alternatively deleted;
wherein, R2 (C) is more than or equal to 65 percentn) < 93%, marking two data segments L1 and L3 monitored in the device A, and sending the marked data segments to the corresponding data real-time feedback module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (2)
1. A distributed data management system based on a block chain is characterized in that: the system comprises a device continuous running state real-time monitoring module, a running device self identification information registration module, a central server data management platform, a redundant data timing cleaning module and a corresponding data real-time feedback module; the system comprises a device continuous operation state real-time monitoring module, an operation device self identification information registration module and a central server data management platform, wherein the device continuous operation state real-time monitoring module, the operation device self identification information registration module and the central server data management platform are sequentially connected through an intranet;
the device continuous operation state real-time monitoring module is used for monitoring the state of a device trying to access a central server, the operation device self identification information registration module is used for uploading a self identification before the operation device is accessed to the central server, the central server data management platform is used for realizing real-time data interaction between operation devices, the redundant data timing cleaning module is used for cleaning redundant data in the central server at regular time, and the corresponding data real-time feedback module is used for monitoring data information uploaded in the operation state and feeding back abnormal data;
the management method of the distributed data management system comprises the following steps:
s1: monitoring the state of the equipment trying to access the central server by using an equipment continuous running state real-time monitoring module;
s2: uploading the self identification before the running equipment is accessed to the central server by utilizing a self identification information registration module of the running equipment;
s3: real-time data interaction among the operating devices is realized by utilizing a central server data management platform;
s4: utilizing a redundant and miscellaneous data timing cleaning module to perform timing cleaning on redundant and miscellaneous data in a central server;
s5: monitoring data information uploaded in the running state by using a corresponding data real-time feedback module, and feeding back abnormal data;
the data management method further comprises the following steps:
s1-1: analyzing the operation rate of equipment trying to access a central server within preset time by using an equipment operation rate analysis submodule within the preset time, determining the state of the equipment within the preset time according to the operation rates of different equipment, marking the equipment when the operation rate of the equipment is lower than a set threshold, extracting all marked equipment, sending the extracted equipment to a non-operation state equipment disconnection submodule, enabling the central server to refute a connection trying request sent by the equipment extracted by the equipment operation rate analysis submodule within the preset time, and not receiving the connection request of the equipment of which the operation rate is lower than the set threshold;
s2-1: the operating equipment identification uploading submodule is used for counting the IP address and the ID of the equipment trying to access the operating equipment of the central server, the counted information is sent to the central server data management platform for authorized login, and the operating equipment access request submodule sends an access request to the central server data management platform again;
s3-1: the operating equipment access response submodule is used for responding an access request of the operating equipment and then accessing, and the data information real-time interaction submodule among the equipment enables different equipment accessed to the data management platform of the central server to send data information to the central server for data processing and call the data information in the central server at any time, so that data interaction is realized;
s4-1: the data volume uploaded by the operating equipment is counted by using an operating equipment uploading data volume counting and sorting submodule, different counted equipment data volumes are sorted, the operating equipment with the first five data volume sorted is marked, a data similarity analyzing submodule inside the operating equipment with the important mark analyzes the data similarity sent by the operating equipment with the first five data volume sorted, when two data segments with the similarity of more than or equal to 93 percent exist inside the operating equipment, the completely similar part of the two data segments is selected to be deleted, when two data segments with the similarity of less than 93 percent and more than or equal to 65 percent exist inside the operating equipment, the two data segments are marked, the marked data segments are sent to a corresponding data real-time feedback module, when all the data segments with the similarity of less than 65 percent exist inside the operating equipment, the operating equipment is not processed;
s5-1: and detecting that the data repeated transmission marking submodule in the directional time segment is in different time segments, sending repeated data to a central server data management platform by different running equipment, marking the repeated data, sending the marked repeated data to an abnormal data manual feedback channel, and receiving the marked repeated data by the abnormal data manual feedback channel and manually processing the data segments marked by the key marking running equipment internal data similarity analysis submodule.
2. The block chain-based distributed data management system of claim 1, wherein: in step S4-1, the statistical sorting submodule for the data size uploaded by the operating device is used to count the data size uploaded by the operating device, sort the different device data sizes that are counted, mark the operating device that is the first five times of the data size sorting, and the analysis submodule for the data similarity inside the marked operating device analyzes the data similarity sent inside the operating device that is the first five times of the data size sorting, which further includes the following steps:
the analysis submodule for setting the similarity of the data in the key mark operation equipment analyzes the data in the operation equipment A, segments the data in the operation equipment A, and sets the segmented data to be L1, L2, L3, … and Ln-1、LnCarrying out pairwise similarity comparison on the divided fragment data;
setting the similarity of different fragment data as C1、C2、C3、…、Cn-1、CnIt is set that there is a similar error rate of 2% in each piece of data, and the actual similarity of each piece of data is set to R (C)n) According to the formula:
R(Cn)＝Cn*(1-2％)
calculating the actual similarity of each fragment data in the current device A when R (C)n) More than or equal to 93 percent, and extracting two of similarity contrastsSegment data, three-time similarity comparison is performed again, and the similarity of the re-comparison is set as C01、C02、C03Setting the similarity mean of the current two segment data for re-comparison as
When is coming into contact with
If the two data segments are completely similar, the two data segments are marked and sent to the corresponding data real-time feedback module;
when R (C) is more than or equal to 65 ≦ Rn) If the data rate is less than 93%, marking two data segments monitored in the equipment A, and sending the marked data segments to a corresponding data real-time feedback module;
when R (C)n) If the ratio is less than 65%, the operation equipment A is not processed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110881890.1A CN113704057A (en) | 2020-06-02 | 2020-06-02 | Distributed data management system based on block chain |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110881890.1A CN113704057A (en) | 2020-06-02 | 2020-06-02 | Distributed data management system based on block chain |
| CN202010490239.7A CN111651326B (en) | 2020-06-02 | 2020-06-02 | Block chain-based distributed data management system and method |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010490239.7A Division CN111651326B (en) | 2020-06-02 | 2020-06-02 | Block chain-based distributed data management system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113704057A true CN113704057A (en) | 2021-11-26 |
Family
ID=72343322
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110881890.1A Pending CN113704057A (en) | 2020-06-02 | 2020-06-02 | Distributed data management system based on block chain |
| CN202010490239.7A Active CN111651326B (en) | 2020-06-02 | 2020-06-02 | Block chain-based distributed data management system and method |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010490239.7A Active CN111651326B (en) | 2020-06-02 | 2020-06-02 | Block chain-based distributed data management system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN113704057A (en) |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1309211C (en) * | 2003-05-13 | 2007-04-04 | 中兴通讯股份有限公司 | Distributed central management method for special shaped network equipment in distributing network environment |
| CN101882141A (en) * | 2009-05-08 | 2010-11-10 | 北京众志和达信息技术有限公司 | Method and system for implementing repeated data deletion |
| KR101334767B1 (en) * | 2012-04-12 | 2013-11-29 | 미래산업 주식회사 | Handling System for Semiconductor device |
| CN105635207A (en) * | 2014-10-31 | 2016-06-01 | 小米科技有限责任公司 | Operating state detection method and device |
| CN105718604A (en) * | 2016-04-19 | 2016-06-29 | 北京金控数据技术股份有限公司 | Method and device for automatically screening equipment |
| CN108228083A (en) * | 2016-12-21 | 2018-06-29 | 伊姆西Ip控股有限责任公司 | For the method and apparatus of data deduplication |
| WO2018187410A1 (en) * | 2017-04-07 | 2018-10-11 | Walmart Apollo, Llc | Systems and methods for data backup and authentication using blockchain |
| US20190258971A1 (en) * | 2018-02-20 | 2019-08-22 | Tyson York Winarski | Blockchain project management with process feedback control |
| US20190317810A1 (en) * | 2018-04-13 | 2019-10-17 | Tri Nguyen | Network Infrastructure performing Distributed Computational Processing and Providing Secure and Encrypted Connection to the Internet |
| CN109669845A (en) * | 2018-12-04 | 2019-04-23 | 郑州云海信息技术有限公司 | A kind of cloud operating system equipment alarm notification method and system |
| CN109889609B (en) * | 2019-03-30 | 2021-09-21 | 苏州燕灏物联科技有限责任公司 | Networked comprehensive intelligent monitoring system and function implementation method thereof |
| CN111024996A (en) * | 2019-11-28 | 2020-04-17 | 深圳市中和智通智能科技有限公司 | Intelligent electric meter for identity authentication based on block chain technology |
| CN111061790A (en) * | 2019-12-13 | 2020-04-24 | 江苏智谋科技有限公司 | Information acquisition system and method for customer data management |
| CN111078794A (en) * | 2019-12-18 | 2020-04-28 | 杭州桐硕教育科技有限公司 | Big data storage system |
-
2020
- 2020-06-02 CN CN202110881890.1A patent/CN113704057A/en active Pending
- 2020-06-02 CN CN202010490239.7A patent/CN111651326B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111651326A (en) | 2020-09-11 |
| CN111651326B (en) | 2021-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1764951B1 (en) | Statistical trace-based method, apparatus, node and system for real-time traffic classification | |
| KR101375813B1 (en) | Active security sensing device and method for intrusion detection and audit of digital substation | |
| CN104487989B (en) | Autonomous network sentry | |
| CN102271068A (en) | Method for detecting DOS/DDOS (denial of service/distributed denial of service) attack | |
| CN105051696A (en) | An improved streaming method and system for processing network metadata | |
| CN109150859B (en) | Botnet detection method based on network traffic flow direction similarity | |
| CN1175621C (en) | Method of detecting and monitoring malicious user host machine attack | |
| CN104883363A (en) | Method and device for analyzing abnormal access behaviors | |
| CN110929896A (en) | Security analysis method and device for system equipment | |
| CN112688822A (en) | Edge computing fault or security threat monitoring system and method based on multi-point cooperation | |
| CN108092970A (en) | A kind of wireless network maintaining method and its equipment, storage medium, terminal | |
| CN111651326B (en) | Block chain-based distributed data management system and method | |
| Jay et al. | Unsupervised learning based intrusion detection for goose messages in digital substation | |
| CN103490964B (en) | A kind of method and device for realizing that flexible configuration terminal accesses quantity | |
| CN112953935A (en) | Multiple server request connection system based on communication equipment | |
| CN103957128A (en) | Method and system for monitoring data flow direction in cloud computing environment | |
| TW201931119A (en) | System and method for monitoring cloud network convergence | |
| CN109309679A (en) | A kind of Network scan detection method and detection system based on TCP flow state | |
| CN106550345A (en) | Wifi signal intelligent scanning systems with self-teaching and optimization ability | |
| CN111242509A (en) | Service management system for intelligent community and service management method thereof | |
| CN111016720A (en) | Attack identification method based on K nearest neighbor algorithm and charging device | |
| CN111092893A (en) | Network security protection method based on XDR ticket data | |
| CN113411299B (en) | Terminal equipment virtual agent system based on big data | |
| CN114666166B (en) | Industrial internet trusted data acquisition and storage method and system | |
| CN113824733B (en) | Computer network management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |