CN113822570A - Enterprise production data storage method and system based on big data analysis - Google Patents
Enterprise production data storage method and system based on big data analysis Download PDFInfo
- Publication number
- CN113822570A CN113822570A CN202111102554.9A CN202111102554A CN113822570A CN 113822570 A CN113822570 A CN 113822570A CN 202111102554 A CN202111102554 A CN 202111102554A CN 113822570 A CN113822570 A CN 113822570A
- Authority
- CN
- China
- Prior art keywords
- node
- abnormal
- data structure
- category
- graph data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Abstract
The invention relates to an enterprise production data storage method and system based on big data analysis, and belongs to the technical field of data storage. The method comprises the following steps: performing sliding window operation on the abnormal vector sequence of each device to obtain the correlation characteristics of any two devices under each sliding window, obtaining the correlation strength between corresponding abnormal states of each device in any two devices, and establishing an initial graph data structure to obtain an intermediate graph data structure; for any first node in the first node set, acquiring a second node set which is connected with the first node in the initial graph data structure and is in the intermediate graph data structure, clustering the second node set, acquiring the category association degree and the maximum category association degree of the first node and each category, and if the maximum category association degree is greater than a preset threshold value, adding the first node to the intermediate graph data structure to obtain a target graph data structure; the method provided by the invention improves the speed and reliability of data storage.
Description
Technical Field
The invention relates to the field of data storage, in particular to an enterprise production data storage method and system based on big data analysis.
Background
In an enterprise, due to the complexity of a production line, a plurality of devices participating in production are provided, and various data can be generated correspondingly. For example, in a chemical industry, the participating production facilities may include various chemical reaction chambers, storage facilities, high pressure and high temperature devices, etc.; in the production process, various state data of each device, such as the reaction temperature of the chemical reaction chamber, the reaction rate, the consumption rate of reactants and the like, need to be recorded in real time, and state data, such as the internal pressure, the temperature, the mechanical vibration amplitude and the like, of the storage device; the various status data of each device may reflect the real-time status of the device; however, since the production process of chemicals is often accompanied by various safety risks, such as explosion, gas leakage, dust pollution, etc. of high temperature and high pressure or chemical reaction equipment, in order to analyze the safety risks in the production process, obtain failure characteristics of the equipment or trace the origin of danger, big data analysis or statistics on the equipment data is required for the safety analysis and safety production of enterprises.
But because the generated data is excessive, a large amount of storage space is occupied; the data contains a large amount of useless information, so the useless information needs to be deleted, and the conventional storage method is to delete the outdated old data, but the characteristic information of some devices, such as fault information, is lost; this is not conducive to big data analysis or statistics of the plant data.
Disclosure of Invention
In order to solve the above technical problems, the present invention aims to provide an enterprise production data storage method and system based on big data analysis, and the adopted technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a method and a system for storing enterprise production data based on big data analysis, including the following steps:
acquiring historical state data of all equipment in a preset time period;
obtaining an abnormal vector sequence of each device according to the historical state data of all devices in the preset time period;
performing sliding window operation on the abnormal vector sequence of each device according to a preset sliding window length and a preset sliding window step length to obtain the correlation characteristics of any two devices under each sliding window, and acquiring the correlation strength between corresponding abnormal states of each device in any two devices according to the correlation characteristics of any two devices under each sliding window;
establishing an initial graph data structure according to the association strength between the abnormal state of each device and the corresponding abnormal state of each device in any two devices, taking the abnormal state of each device as a node, and taking the association strength between the two abnormal states as an edge weight;
deleting nodes of the initial graph data structure, the edge weights of which do not meet preset requirements, to obtain an intermediate graph data structure and a first node set deleted from the initial graph data structure;
for any first node in the first node set, acquiring a second node set which is connected with the first node in the initial graph data structure and is in the intermediate graph data structure, clustering the second node set to obtain a plurality of classes, and acquiring the class association degree of the first node and each class; obtaining the maximum category association degree of the category association degrees of the first node and each category, and if the maximum category association degree is greater than a preset threshold, adding the first node to the intermediate graph data structure to obtain a target graph data structure;
and screening the abnormal vectors of each device according to the target graph data structure, and storing the abnormal vectors obtained by screening.
The invention also provides an enterprise production data storage system based on big data analysis, which comprises a storage and a processor, so as to realize the enterprise production data storage method based on big data analysis and acquisition of historical state data of all equipment in a preset time period.
Preferably, the historical state data is a historical state time sequence of each operating state of the equipment; the obtaining of the abnormal vector sequence of each device according to the historical state data of all devices in the preset time period includes:
and for any sampling moment, calculating the abnormal degree of each element in the historical state time sequence of the equipment at the sampling moment to obtain the abnormal vector of the equipment at the sampling moment, and further obtaining the abnormal vector sequence of the equipment based on all the sampling moments.
Preferably, for any sampling time, calculating the degree of abnormality of each element in the time series of the historical state of the device at the sampling time includes:
calculating the abnormal degree by adopting a safety threshold value according to the following formula:
wherein, E is the abnormal degree of any element in any historical state time sequence of the equipment at the sampling time, E is an element in the historical state time sequence of the equipment at the sampling time, EmaxUpper bound of safety threshold, eminIs the lower bound of the safety threshold value,
is a preset value.
Preferably, the sliding window operation is performed on the abnormal vector sequence of each device according to a preset sliding window length and a preset sliding window step length, so as to obtain the associated features of any two devices under each sliding window, including:
and for any one sliding window, acquiring the abnormal vector sequences of any two devices under the window, and performing typical correlation analysis on the two abnormal vector sequences to obtain correlation characteristics so as to obtain the correlation characteristics of any two devices under each sliding window.
Preferably, obtaining the association strength between the corresponding abnormal states of each device in the two devices according to the association characteristics of the two devices under each sliding window includes:
acquiring an abnormal vector set of each device in any two devices according to the association characteristics of any two devices under each sliding window, and clustering the abnormal vector sets to obtain a clustering result set, wherein each category in the clustering result set represents each abnormal state;
and acquiring the association strength between any one category in the clustering result set corresponding to one of the any two devices and any one category in the clustering result set corresponding to the other device according to the clustering result set corresponding to each of the any two devices, so as to obtain the association strength between the corresponding abnormal states of each device in any two devices.
Preferably, the obtaining the association strength between any one category in the clustering result set corresponding to one of the two arbitrary devices and any one category in the clustering result set corresponding to the other device to obtain the association strength between the corresponding abnormal states of the devices in the two arbitrary devices includes:
calculating the correlation strength between the corresponding abnormal states of each of any two devices according to the following formula:
wherein q is the correlation strength between the abnormal states of any two devices, t (i) is the number of groups in which any abnormal vector in any one of the categories in the clustering result set corresponding to one of any two devices and any abnormal vector in any one of the categories in the clustering result set corresponding to the other device are simultaneously in the ith window, N is the total number of windows, ρ is the total number of windowsiThe associated features of the two devices selected for the ith sliding window.
Preferably, for any first node in the first node set, obtaining a second node set that is connected to the first node in the initial graph data structure and is in the intermediate graph data structure, and performing clustering processing on the second node set to obtain a plurality of categories, and obtaining the category association degree between the first node and each category, the method includes:
calculating the category association degree of the first node and each category according to the following formula:
v is any one first node in the first node set, s is any one category after the second node set is subjected to clustering processing, L (v, s) is the category association degree of the first node v and the category s, Q is the total number of nodes in the category s, v is the total number of nodes in the category spAnd vqRespectively, any two nodes in the class s, w1(v, v)q) Is the first node v and the node v in the class sqThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node v and all nodes in the class s in the initial graph data structure, w2(vp，vq) As node v within class spAnd node vqEdge weights, ∑, in the intermediate graph data structurep，q∈sw2(vp，vq) Is the sum of the edge weights of any two nodes in the class s in the intermediate graph data structure.
Preferably, adding the first node to the intermediate graph data structure to obtain a target graph data structure, includes:
adding the first node into the intermediate graph data structure, connecting with any one node in the second node set, and calculating the edge weight of any one node in the first node and the second node set, including:
calculating an edge weight value of any one of the first node and the second node set according to the following formula:
wherein u is the category corresponding to the maximum category association degree in the category association degrees of the first node x and each category, and x is added to the categoryFirst node, x, of the inter-graph data structuretIs any one node in the category u, w3(x, x)t) For the first node x and any node x in the class utY is the total number of nodes within the class u, w4(x, x)t) For the first node x and any node x in the class utThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node x and all nodes in the class u in the initial graph data structure,
the number of edges between the nodes in the category u;
the degree of association between the first node x and the class u.
Preferably, after obtaining the target graph data structure, the method further includes:
acquiring a third node set which is not added to the intermediate graph data structure in the first node set;
for any one third node in the third node set, acquiring a fourth node set which is connected with the third node in the initial graph data structure and is in the intermediate graph data structure, clustering the fourth node set to obtain a plurality of classes, and acquiring the class association degree of the third node and each class; acquiring a target category corresponding to the maximum category association degree in the category association degrees of the third node and each category, wherein for any node pair in the target category, the node pair comprises two fourth nodes connected with edges, and if one abnormal vector is in the same window in the third node and the two fourth nodes of the node pair respectively, the window is acquired, and at least one window is finally acquired;
acquiring a plurality of abnormal vector sequences representing the third node from each acquired window, acquiring the correlation characteristics of any two abnormal vector sequences, acquiring the mean value of the correlation characteristics larger than a preset threshold value, and acquiring the edge weight of the third node and the node pair;
according to the edge weights of the third node and the node pairs, the edge weights of the third node and all the node pairs including a certain fourth node are obtained, and the sum of the edge weights is obtained and is the edge weight of the third node and the certain fourth node;
if the edge weight of the third node and the certain fourth node meets the preset requirement and the maximum class association degree in the class association degrees of the third node and each class is greater than a preset threshold value, adding the third node into the intermediate graph data structure to obtain a final graph data structure;
the screening the abnormal vectors of the devices according to the target graph data structure and storing the abnormal vectors obtained by screening, comprising the following steps:
and screening the abnormal vectors of each device according to the final graph data structure, and storing the abnormal vectors obtained by screening.
The enterprise production data storage method based on big data analysis provided by the invention has the technical effects that: compared with the traditional mode of directly deleting outdated old data or the mode of only compressing and storing abnormal vector sequences of single equipment, the enterprise production data storage method based on big data analysis provided by the invention avoids the loss of some important information caused by not considering the mutual influence of abnormal states among different equipment, and accelerates the speed and the accuracy of data storage; and selecting the historical state data of all the devices in the preset time period to obtain the abnormal vector sequence of each device, and the abnormal vector sequence of each device is subjected to sliding window operation to obtain the associated characteristics of any two devices under each sliding window, and obtains the correlation strength between the corresponding abnormal states of each of any two devices, establishing an initial graph data structure by taking the abnormal state of each device as a node and the correlation strength between the two abnormal states as an edge weight, and the first node set deleted in the initial graph data structure obtains an intermediate graph data structure, and a second node set which is connected with any one first node in the first node set in the initial graph data structure and is in the intermediate data structure graph is obtained, clustering the second node set, and acquiring the category association degree of the first node and each category; the method comprises the steps of obtaining the maximum category correlation degree of a first node and the category correlation degrees of all categories, if the maximum category correlation degree is larger than a preset threshold value, adding the first node into an intermediate graph data structure to obtain a target graph data structure, deleting useless device data, screening the state data of devices according to the target graph data structure while reducing data storage quantity, considering the correlation existing among abnormal vector sequences of different devices, wherein the state data with the correlation is also important information, avoiding the loss of device data with mutual influence, and improving the speed and reliability of data storage.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of an enterprise production data storage method based on big data analysis according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments, and all other embodiments obtained by those skilled in the art based on the embodiments of the present invention belong to the protection scope of the embodiments of the present invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment provides an enterprise production data storage method based on big data analysis, which is described in detail as follows:
as shown in fig. 1, the enterprise production data storage method based on big data analysis includes the following steps:
and step S001, acquiring historical state data of all equipment in a preset time period.
In this embodiment, the historical state data of each operating state of all the devices in six months of the enterprise needs to be acquired by various sensors on each device, for example, temperature data may be acquired by a temperature sensor or pressure data may be acquired by a pressure sensor, and the acquired historical state data of all the devices includes all the operating states of each device.
As another embodiment, different time periods may be set for the obtained historical status data of all the devices of the enterprise according to different requirements, for example, historical status data of each operating status of all the devices in a year of the enterprise may be obtained, but the obtained historical status data of all the devices includes all the operating statuses of each device.
And step S002, obtaining abnormal vector sequences of all the equipment according to the historical state data of all the equipment in the preset time period.
In this embodiment, for any sampling time, the abnormal degree of each element in the historical state time sequence of the device at the sampling time is calculated, so as to obtain an abnormal vector of the device at the sampling time, and further obtain an abnormal vector sequence of the device based on all sampling times.
As a specific implementation manner, first, all the states existing in the device a are obtained, and for any one operating state, the state data corresponding to the state is obtained by sampling according to a sampling period, so that, based on sampling times corresponding to a plurality of sampling periods, the state corresponds to a historical state time series, and the historical state time series includes a plurality of state data obtained by sampling according to the sequence of the sampling times in the state.
In this embodiment, the state data of each state of the device is obtained by sampling according to the same sampling period and sampling time. The method comprises the steps of removing noise of all historical state time sequences of the equipment by adopting Gaussian filtering, obtaining elements (elements are state data) on all the historical state time sequences of the equipment at any sampling moment, and calculating the abnormal degree of each element on all the historical state time sequences of the equipment at the sampling moment, wherein the abnormal degrees form a multi-dimensional vector and are called abnormal vectors of the equipment A at the sampling moment. Since one sampling time corresponds to one abnormal vector, the abnormal vector at each sampling time can be obtained for the historical state time series of the equipment A obtained based on a plurality of sampling times, and the abnormal vector at each sampling time forms the abnormal vector series of the equipment A.
In this embodiment, the abnormal degree of each element in all the historical state time sequences of the device a at the sampling time is obtained by adopting a safety threshold judgment mode, where the safety threshold has an upper limit and a lower limit, where the upper limit is an upper limit and the lower limit is a lower limit, a range of the safety threshold is set according to actual needs, and the safety threshold has an upper limit and a lower limit; if the value of the element is within the range of the safety threshold, judging that the abnormal degree of the element is zero and the element is not an abnormal element; if the value of the element is out of the range of the safety threshold, namely exceeds the upper and lower limits of the safety threshold, judging that the abnormal degree of the element is not zero and the element is an abnormal element; the degree of anomaly of an element is therefore calculated according to the following formula:
wherein E is the abnormal degree of any element in any historical state time sequence of the sampling moment device A, E is an element in the historical state time sequence of the sampling moment device A, and E ismaxUpper bound of safety threshold, eminIs the lower bound of the safety threshold value,
is a preset threshold value, which is a corresponding state value of the device in a normal state, in this embodiment
The value of (b) is set when the device is shipped from the factory.
In this embodiment, the abnormal vector sequences of all devices in an enterprise may be sequentially obtained by the above method; it should be noted that the data object stored in the present embodiment is an abnormal vector sequence of each device, and the setting of the safety threshold is to be set according to the actual situation.
And S003, performing sliding window operation on the abnormal vector sequence of each device according to the preset sliding window length and the sliding window step length to obtain the correlation characteristics of any two devices under each sliding window, and obtaining the correlation strength between the corresponding abnormal states of each device in any two devices according to the correlation characteristics of any two devices under each sliding window.
In this embodiment, the abnormal vector sequence of each device needs to be subjected to sliding window operation through a set sliding window, where the sliding window is a one-dimensional vector with a length of 24 hours; as other embodiments, the sliding window may be set to have different lengths according to different requirements; furthermore, the sliding window step size is set by actual requirements, for example, the sliding window step size is one tenth of the window length.
For any sliding window, acquiring abnormal vector sequences of any two devices under the window, and performing typical correlation analysis on the two abnormal vector sequences to obtain correlation characteristics so as to obtain the correlation characteristics of any two devices under each sliding window; the method specifically comprises the following steps: for any sliding window, obtaining abnormal vector sequences of the device B and the device C under the window, then performing typical correlation analysis on the abnormal vector sequences of the device B and the device C to obtain a correlation coefficient between the abnormal vector sequences of the device B and the device C, namely a correlation characteristic ρ, and further obtaining correlation characteristics of the device B and the device C under each sliding window, namely:
ρ＝{ρ1，ρ2，...，ρi，...，ρN}
wherein the correlation characteristic between the abnormal vector sequences of the rho device B and the device C is rho1For the associated features of device B and device C under the first sliding window, ρ2Is the associated characteristic, rho, of the device B and the device C under the second sliding windowiIs the associated characteristic of the device B and the device C under the ith sliding window, rhoNAnd (3) the correlation characteristics of the equipment B and the equipment C under the nth sliding window, wherein N is the total number of the windows, and the correlation characteristics rho between the abnormal vector sequences of the equipment B and the equipment C under each sliding window need to be eliminated to be 0, namely elements in rho are not 0.
It should be noted that the larger the absolute value of the obtained associated feature ρ is, the stronger the correlation between the abnormal vector sequences of the device B and the device C is, and the occurrence of abnormal states of the device B and the device C is related; the smaller the absolute value of the correlation characteristic ρ, the less correlation between the anomaly vector sequences of the device B and the device C.
Acquiring an abnormal vector set of each device in any two devices according to the association characteristics of any two devices under each sliding window, clustering the abnormal vector sets to obtain a clustering result set, acquiring the association strength of any one category in the clustering result set corresponding to one device in any two devices and any one category in the clustering result set corresponding to the other device according to the clustering result set corresponding to each device in any two devices, and acquiring the association strength between the corresponding abnormal states of each device in any two devices; the method specifically comprises the following steps: and respectively acquiring abnormal vector sets of the equipment B and the equipment C according to the obtained associated features of the equipment B and the equipment C under the sliding window each time, wherein the abnormal vector sets are formed by the eliminated abnormal vectors with zero associated features of the equipment B and the equipment C under the sliding window each time.
Wherein the set of exception vectors for device B is:
Sb＝{Sb，1，Sb，2，…，Sb，i，…，Sb，N}
the set of exception vectors for device C is:
Sc＝{Sc，1，Sc，2，...，Sc，i，...，Sc，N}
wherein S isbIs an abnormal vector set of device B, Sb，1Set of abnormal vectors of the device B in the window being a first sliding window, Sb，2Set of abnormal vectors of the device B in the window, S, being a second sliding windowb，iSet of abnormal vectors for device B in window of ith sliding window, Sb，NThe abnormal vector set of the device B in the window of the Nth sliding window is N, wherein N is the total number of the windows; scIs an abnormal vector set, S, of the device Cc，1Set of abnormal vectors of the in-window device C being a first sliding window, Sc，2Set of abnormal vectors of the in-window device C being a second sliding window, Sc，iSet of abnormal vectors for the in-window device C of the ith sliding window, Sc，NThe abnormal vector set of the device C in the window of the Nth sliding window is N, and N is the total number of the windows.
Then clustering the abnormal vector sets of the equipment B and the equipment C by using a mean shift clustering algorithm to respectively obtain two clustering result sets Cb＝{cb，1，cb，2，...，cb，m,.. } and cc＝{cc，1，cc，2，...，cc，n,.. }; wherein, cbIs a pair of SbClustering result set after clustering of medium abnormal vectors, cb，1Is a pair of SbThe first clustering result after the middle abnormal vector is clustered, cb，2Is a pair of SbSecond clustering result after clustering of medium abnormal vector, cb，mIs a pair of SbThe m-th clustering result after the middle abnormal vector is clustered; wherein, ccIs a pair of ScClustering result set after clustering of medium abnormal vectors, cc，1Is a pair of ScThe first clustering result after the middle abnormal vector is clustered, cc，2Is a pair of ScSecond clustering result after clustering of medium abnormal vector, cc，nIs a pair of ScThe nth clustering result after the medium abnormal vector is clustered; it should be noted that each clustering result in the clustering result set is a category, and each category represents each abnormal state.
Then, according to the clustering result sets corresponding to the equipment B and the equipment C, the mth category C in the clustering result set corresponding to the equipment B is obtainedb，mAnd the nth category C in the corresponding clustering result set in the device Cc，nThe corresponding abnormal state C of each equipment in the equipment B and the equipment C is obtainedb，mAnd cc，nThe strength of the association between.
The corresponding abnormal state C of each of the equipment B and C is calculated according to the following formulab，mAnd cc，nStrength of association between:
wherein q is the corresponding abnormal state C of each of the equipment B and the equipment Cb，mAnd cc，nThe correlation strength between the two sets, T (i), is the clustering result set c corresponding to the device BbThe mth category c in (1)b，mAny abnormal vector in the cluster result set C corresponding to the device CcThe group number of any abnormal vector in the nth category in the ith window at the same time refers to the clustering result set cbThe mth category c in (1)b，mAny one of the abnormal vectors and the clustering result set ccIn the nth category of (1) any one of the abnormal vectors is simultaneously in a window to form a group, N is the total number of the windows, and rhoiThe associated features of device B and device C under the ith sliding window.
By the method, the association strength between any one category in the clustering result set corresponding to one device of any two devices and any one category in the clustering result set corresponding to the other device can be obtained, and the association strength between the corresponding abnormal states of the devices of any two devices can be obtained.
Step S004, establishing an initial graph data structure according to the association strength between the abnormal state of each device and the corresponding abnormal state of each device in any two devices, with the abnormal state of each device as a node, and with the association strength between two abnormal states as an edge weight.
Through the clustering process, each category in the clustering result set can be obtained, and each category is each abnormal state of the equipment, so that the abnormal state of each equipment can be obtained.
In this embodiment, an initial graph data structure is established according to the association strength between the abnormal state of each device and the corresponding abnormal state of each device in any two devices, with the abnormal state of each device as a node and the association strength between two abnormal states as an edge weight.
Step S005 is to delete the nodes whose edge weights do not meet the preset requirements in the initial graph data structure, so as to obtain an intermediate graph data structure and a first node set deleted from the initial graph data structure.
In this embodiment, the node whose edge weight is smaller than the threshold in the initial graph data structure is set to zero, and the node whose degree is zero is deleted, where the zero means that a certain node is not connected to any other node; obtaining an intermediate graph data structure according to the nodes left after deletion and the connection relation among the nodes; and resulting in nodes that are deleted from the initial graph data structure, the nodes comprising the first set of nodes.
In this embodiment, the intermediate graph data structure represents an association relationship between abnormal states of different devices, and as long as an abnormal state is included in the intermediate graph data structure, a corresponding abnormal vector must have other abnormal states associated therewith, so that the abnormal vectors cannot be deleted, but do not represent the abnormal state not included in the graph and other states not associated therewith, which is equivalent to removing some nodes in the initial graph data structure, and forming the intermediate graph data structure by nodes left after removal; the nodes that are rejected may be some noisy data, or may be rejected because the associated features are not sufficiently explored in the above steps, resulting in too small edge weights.
Step S006, for any first node in the first node set, obtaining a second node set which is connected with the first node in the initial graph data structure and is in the intermediate graph data structure, clustering the second node set to obtain a plurality of categories, and obtaining the category association degree of the first node and each category; and acquiring the maximum category association degree of the category association degrees of the first node and each category, and if the maximum category association degree is greater than a preset threshold, adding the first node to the intermediate graph data structure to obtain a target graph data structure.
Each node in the first set of nodes is referred to as a first node. In this embodiment, according to any one first node in the first node set deleted in the initial graph data structure, each node connected to the first node in the initial graph data structure and located in the intermediate graph data structure is obtained, the obtained nodes may be referred to as second nodes to form a second node set, the second node set is clustered to obtain a plurality of classes, and the class association degree between the first node and each class is obtained; as a specific embodiment, the following is specifically mentioned: according to the first node set S deleted from the initial graph data structureDObtaining a second node set S which is connected with the first node v in the initial graph data structure and is positioned in the intermediate graph data structureG2For the second node set SG2Performing spectral clustering to obtain a plurality of categories, wherein the number of the categories is determined by a spectral clustering operation result, the category association degree of the first node v and each category is obtained, and the category association degree of the first node v and each category is calculated according to the following formula:
wherein v is a first set of nodes SDAny one of themA first node, S, being a second set of nodes SG2L (v, s) is the category association degree of the first node v and the category s, Q is the total number of nodes in the category s, v is the total number of the nodes in the category spAnd vqRespectively, any two nodes in the class s, w1(v, v)q) Is the first node v and the node v in the class sqThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node v and all nodes in the class s in the initial graph data structure, w2(vp，vq) As node v within class spAnd node vqEdge weights, ∑, in the intermediate graph data structurep，q∈sw2(vp，vq) Is the sum of the edge weights of any two nodes in the class s in the intermediate graph data structure.
Note that w2 (v) isp，vq) Or may be a node v within the class spAnd node vqEdge weights in the initial graph data structure, the ∑ sp，q∈sw2(vp，vq) Or the sum of the edge weights of any two nodes in the category s in the initial graph data structure, and the obtained result is the same as the result obtained in the intermediate graph data structure.
It should be noted that the larger the value of L (v, s), the closer the first node v is connected to the class s, and the stronger the correlation between the abnormal state represented by the node v and the abnormal state of each node in the class s.
Therefore, the degree of association between any one first node in the first node set and each category can be obtained through the above method. To this end, any first node in the first node set corresponds to one or more categories and the degree of association of the one or more categories.
Then, the maximum category association degree of the category association degrees of the first node in the first node set and each category is obtained, and if the maximum category association degree is larger than a preset threshold value, the abnormal state represented by the first node and the maximum category association degree are representedIf the abnormal state of the node in the category corresponding to the large category association degree has strong correlation, adding the first node into the intermediate graph data structure to obtain a target graph data structure, specifically: when the first node is collected SDThe category association degree of any one first node x with the category u is maximum and is greater than a preset threshold, and the first node x is added to the intermediate graph data structure, wherein the specific adding method comprises the following steps:
adding a first node x to the intermediate graph data structure with a second set of nodes SG2Are connected. The first node x and the second node set SG2The process of calculating the edge weight of any node in the system comprises the following steps:
calculating a first node x and a second node set S according to the following formulaG2The edge weight of any one node:
wherein u is the category corresponding to the maximum category association degree of the first node x and the category association degrees of the categories, x is the first node added to the intermediate graph data structure, and x is the second node added to the intermediate graph data structuretIs any one node in the category u, w3(x, x)t) For the first node x and any node x in the class utY is the total number of nodes within the class u, w4(x, x)t) For the first node x and any node x in the class utThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node x and all nodes in the class u in the initial graph data structure,
the number of edges between nodes in the class u,
is thatThe category association degree of the first node x and the category u; wherein
The mean value of the sum of the edge weights of the first node x and all nodes in the category u in the initial graph data structure;
is the mean value of the sum of the edge weights, x, in the intermediate graph data structure between any two nodes in the class utAnd xwRespectively, are any two nodes within the class u, sigmat，w∈uw5(xt，xw) The edge weight values of any two nodes in the class u in the intermediate graph data structure;
for the first node x and the node x in the initial graph data structuretThe ratio of the edge weight of (a) to the sum of the edge weights of the first node x and all nodes in the class u;
is the average of the weights of all edges in the first node x and the class u.
It is noted that
Or the average value of the sum of the edge weights in the initial graph data structure between any two nodes in the class u, and the sigmat，w∈uw5(xt，xw) Or the edge weight of any two nodes in the category u in the initial graph data structure; the results obtained are the same as those obtained in the intermediate graph data structure.
Processing other first nodes according to the process, adding the first nodes into the intermediate graph data structure if the adding condition is met, and specifically adding according to the processAnd (4) adding. Finally, the first node set S can be obtained through the above processDAnd sequentially adding the first nodes meeting the conditions into the intermediate graph data structure to obtain a target graph data structure.
And S007, screening abnormal vectors of each device according to the target graph data structure, and storing the screened abnormal vectors.
In this embodiment, after the target graph data structure is obtained, in consideration that some deeper relationships may be ignored, through the above processing procedure, each node in the first node set that is not added to the intermediate graph data structure is obtained, the obtained nodes are referred to as third nodes, and all the third nodes form a third node set. The reason why each third node in the third node set cannot be added to the intermediate graph data structure at this time is two: the third node is a noise node, and the noise node means that the third node has a smaller degree of association with each category after clustering processing is performed on the second node set due to the influence of noise of data or due to contingency; on the other hand, the third node and each category after the clustering processing of the second node set have a deeper or difficultly mined association relationship, so that each category after the clustering processing of the third node and the second node set has a smaller association degree.
For any one third node in the third node set, acquiring a fourth node set which is connected with the third node in the initial graph data structure and is in the intermediate graph data structure, clustering the fourth node set to obtain a plurality of classes, and acquiring the class association degree of the third node and each class; and acquiring a target category corresponding to the maximum category association degree in the category association degrees of the third node and each category, wherein for any node pair in the target category, the node pair comprises two fourth nodes connected with edges, and if an abnormal vector in each of the third node and the two fourth nodes in the node pair is in the same window, namely the abnormal vector in the third node, the abnormal vector in one of the fourth nodes in the node pair and the abnormal vector in the other fourth node in the node pair are in the same window, acquiring the window, and finally acquiring at least one window.
As a specific embodiment, the following is specifically mentioned: need to acquire a third node set SVIs referred to as the first set of nodes SDCan no longer be added to the nodes remaining in the intermediate graph data structure, after which a fourth set S of nodes in the intermediate graph data structure, which is connected to the third node z in the initial graph data structure, is obtainedWAnd for the fourth node set SWCarrying out mean shift clustering algorithm processing to obtain a plurality of categories, and obtaining the category association degree of the third node z and each category; the calculation process of the category association degree is the same as that of the category association degree; obtaining a target class k corresponding to the maximum class association degree in the class association degrees of the third node z and each class, and for any node pair in the target class k, the node pair comprises two fourth nodes z connected with edges1And z2If the third node z and the node pair z1And z2And if one abnormal vector is in the same window, obtaining the window, and finally obtaining at least one window.
Wherein, each window obtained corresponds to three abnormal vector sequences which respectively represent three nodes z and z1、z2A change in the abnormal state of the corresponding device.
Acquiring a plurality of abnormal vector sequences representing the third node from each acquired window, acquiring the correlation characteristics of any two abnormal vector sequences, acquiring the mean value of the correlation characteristics larger than a preset threshold value, and acquiring the edge weight of the third node and the node pair; the method specifically comprises the following steps: acquiring a plurality of abnormal vector sequences of the third node z corresponding to each window from each acquired window, and acquiring the correlation characteristics of any two abnormal vector sequences in the plurality of abnormal vector sequences, so that a plurality of correlation characteristics can be acquired, and the calculation mode of the correlation characteristics is the same as that of the correlation characteristics; fromObtaining the associated features larger than a preset threshold value from the associated features, calculating the mean value d of the associated features larger than the preset threshold value, wherein the larger the value of the mean value d is, the node pair z is1And z2The represented abnormal state has a strong autocorrelation with the abnormal state represented by the third node z, so the third node z and the node pair z are connected1And z2The edge weight of (d) is set to d.
In this embodiment, the following settings are set:
k＝{z1，z2，...，z10，...，z18，...，zj，zj+1，...}
wherein z is1Is the 1 st node in the object class k, z2Is the 2 nd node in the object class k, z10Is the 10 th node in the object class k, z18Is the 18 th node in the object class k, zjIs the jth node, z, in the target class kj+1Is the j +1 th node in the target class k.
There are many combinations of any two nodes in the object class k. Such as: node z2Can be divided by node z2Other nodes may be combined, for example: node z2And node z10Form a node pair, node z2Can be compared with node zjForm a node pair or node z2Can be compared with node z18Forming a node peer, wherein each node pair comprises a node z2The node pair of (2). Therefore, in the target category k, any node may form a node pair with a plurality of nodes in the target category k.
Therefore, for a certain fourth node, taking the node z as an example, the fourth node z in the target class k is obtained2All node pairs consisting of a plurality of nodes in the target class k acquire a third node z and the obtained edge weight of each node pair, and the sum of the edge weights is the third node z and the contained node z2The edge weights of all node pairs of (i.e. third node z and fourth node z)2The edge weight of (2). Similarly, the third node z and other fourth nodes in the target class k are obtainedThe edge weight of the node.
Finally, if the edge weight of the third node and a certain fourth node meets the preset requirement and the maximum class association degree in the class association degrees of the third node and each class is greater than the preset threshold value, adding the third node into the intermediate graph data structure to obtain a final graph data structure; the method specifically comprises the following steps: and when the edge weight values of the third node z and the fourth node z meet the preset requirement, and the category association degree of the third node z and the target category k in each category is maximum and is greater than a preset threshold value, adding the third node z into the intermediate graph data structure.
Referring to the above adding process of adding nodes into the intermediate graph data structure, it should be noted that each parameter involved in the calculation process needs to be adjusted to a corresponding parameter used in the calculation.
Therefore, the third nodes in the third node set which meet the conditions can be added to the intermediate graph data structure through the steps in sequence, and the final graph data structure is obtained.
In this embodiment, the screening abnormal vectors of each device according to the target graph data structure diagram, and storing the abnormal vectors obtained by the screening, includes:
as a specific embodiment, when an abnormal vector sequence of a device is obtained, if an abnormal vector in the abnormal vector sequence is included in the final graph data structure, the abnormal vector of the device is not deleted, and if the abnormal vector in the abnormal vector sequence is not included in the final graph data structure, the abnormal vector of the device is deleted. Finally, the filtered abnormal vector is stored, for example, in a local storage device or in a cloud server.
The technical effects of the enterprise production data storage method based on big data analysis provided by the embodiment include: firstly, compared with the traditional mode of directly deleting outdated old data or the mode of only compressing and storing abnormal vector sequences of a single device, the enterprise production data storage method based on big data analysis provided by the embodiment avoids the loss of some important information caused by the fact that the mutual influence of abnormal states among different devices is not considered, and accelerates the speed and the accuracy of data storage; selecting historical state data of all equipment in a preset time period, obtaining an abnormal vector sequence of each equipment, and the abnormal vector sequence of each device is subjected to sliding window operation to obtain the associated characteristics of any two devices under each sliding window, and obtains the correlation strength between the corresponding abnormal states of each of any two devices, establishing an initial graph data structure by taking the abnormal state of each device as a node and the correlation strength between the two abnormal states as an edge weight, and the first node set deleted in the initial graph data structure obtains an intermediate graph data structure diagram, and a second node set which is connected with any one first node in the first node set in the initial graph data structure and is positioned in the intermediate graph data structure diagram is obtained, clustering the second node set, and acquiring the category association degree of the first node and each category; the method comprises the steps of obtaining the maximum category correlation degree of a first node and the category correlation degrees of all categories, if the maximum category correlation degree is larger than a preset threshold value, adding the first node into an intermediate data structure chart to obtain a target chart data structure, deleting useless device data, screening the state data of the devices according to the target chart data structure while reducing the data storage amount, considering the correlation existing among abnormal vector sequences of different devices, wherein the state data with the correlation is also important information, avoiding the loss of the device data with mutual influence, and improving the speed and reliability of data storage.
The enterprise production data storage system based on big data analysis of the embodiment comprises a memory and a processor, wherein the processor executes a computer program stored in the memory to realize the method for acquiring and acquiring historical state data of all equipment in a preset time period as described in the embodiment of the enterprise production data storage method based on big data analysis.
It should be noted that the order of the above-mentioned embodiments of the present invention is merely for description and does not represent the merits of the embodiments, and in some cases, actions or steps recited in the claims may be executed in an order different from the order of the embodiments and still achieve desirable results.
The embodiments in the present specification are all described in a progressive manner, and the same and similar parts among the various embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments.
Claims (10)
1. An enterprise production data storage method based on big data analysis is characterized by comprising the following steps:
acquiring historical state data of all equipment in a preset time period;
obtaining an abnormal vector sequence of each device according to the historical state data of all devices in the preset time period;
performing sliding window operation on the abnormal vector sequence of each device according to a preset sliding window length and a preset sliding window step length to obtain the correlation characteristics of any two devices under each sliding window, and acquiring the correlation strength between corresponding abnormal states of each device in any two devices according to the correlation characteristics of any two devices under each sliding window;
establishing an initial graph data structure according to the association strength between the abnormal state of each device and the corresponding abnormal state of each device in any two devices, taking the abnormal state of each device as a node, and taking the association strength between the two abnormal states as an edge weight;
deleting nodes of the initial graph data structure, the edge weights of which do not meet preset requirements, to obtain an intermediate graph data structure and a first node set deleted from the initial graph data structure;
for any first node in the first node set, acquiring a second node set which is connected with the first node in the initial graph data structure and is in the intermediate graph data structure, clustering the second node set to obtain a plurality of classes, and acquiring the class association degree of the first node and each class; obtaining the maximum category association degree of the category association degrees of the first node and each category, and if the maximum category association degree is greater than a preset threshold, adding the first node to the intermediate graph data structure to obtain a target graph data structure;
and screening the abnormal vectors of each device according to the target graph data structure diagram, and storing the screened abnormal vectors.
2. The enterprise production data storage method based on big data analysis as claimed in claim 1, wherein the historical state data is a historical state time series of each operation state of the equipment; the obtaining of the abnormal vector sequence of each device according to the historical state data of all devices in the preset time period includes:
and for any sampling moment, calculating the abnormal degree of each element in the historical state time sequence of the equipment at the sampling moment to obtain the abnormal vector of the equipment at the sampling moment, and further obtaining the abnormal vector sequence of the equipment based on all the sampling moments.
3. The method for storing enterprise production data based on big data analysis as claimed in claim 2, wherein for any sampling time, calculating the degree of abnormality of each element in the time sequence of the historical state of the equipment at the sampling time comprises:
calculating the abnormal degree by adopting a safety threshold value according to the following formula:
wherein, E is the abnormal degree of any element in any historical state time sequence of the equipment at the sampling time, E is an element in the historical state time sequence of the equipment at the sampling time, EmaxUpper bound of safety threshold, eminIs the lower bound of the safety threshold value,
is a preset value.
4. The enterprise production data storage method based on big data analysis according to claim 1, wherein the sliding window operation is performed on the abnormal vector sequence of each device according to a preset sliding window length and a sliding window step length to obtain the correlation characteristics of any two devices under each sliding window, and the method comprises:
and for any one sliding window, acquiring the abnormal vector sequences of any two devices under the window, and performing typical correlation analysis on the two abnormal vector sequences to obtain correlation characteristics so as to obtain the correlation characteristics of any two devices under each sliding window.
5. The enterprise production data storage method based on big data analysis according to claim 1, wherein the obtaining of the correlation strength between the abnormal states of any two devices according to the correlation characteristics of any two devices under each sliding window comprises:
acquiring an abnormal vector set of each device in any two devices according to the association characteristics of any two devices under each sliding window, and clustering the abnormal vector sets to obtain a clustering result set, wherein each category in the clustering result set represents each abnormal state;
and acquiring the association strength between any one category in the clustering result set corresponding to one of the any two devices and any one category in the clustering result set corresponding to the other device according to the clustering result set corresponding to each of the any two devices, so as to obtain the association strength between the corresponding abnormal states of each device in any two devices.
6. The method as claimed in claim 5, wherein the obtaining of the association strength between any one category in the clustering result set corresponding to one of the two arbitrary devices and any one category in the clustering result set corresponding to the other device to obtain the association strength between the abnormal conditions occurring in each of the two arbitrary devices includes:
calculating the correlation strength between the corresponding abnormal states of each of any two devices according to the following formula:
wherein q is the correlation strength between the abnormal states of any two devices, t (i) is the number of groups in which any abnormal vector in any one of the categories in the clustering result set corresponding to one of any two devices and any abnormal vector in any one of the categories in the clustering result set corresponding to the other device are simultaneously in the ith window, N is the total number of windows, ρ is the total number of windowsiThe associated features of the two devices selected for the ith sliding window.
7. The big data analysis-based enterprise production data storage method of claim 1, wherein for any first node in the first node set, obtaining a second node set that is connected to the first node in the initial graph data structure and is in the intermediate graph data structure, clustering the second node set to obtain a plurality of categories, and obtaining the category association degree between the first node and each category, comprises:
calculating the category association degree of the first node and each category according to the following formula:
v is any one first node in the first node set, s is any one category after the second node set is subjected to clustering processing, L (v, s) is the category association degree of the first node v and the category s, Q is the total number of nodes in the category s, v is the total number of nodes in the category spAnd vqRespectively, any two nodes in the class s, w1(v, v)q) Is the first node v and the node v in the class sqThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node v and all nodes in the class s in the initial graph data structure, w2(vp,vq) As node v within class spAnd node vqEdge weights, ∑, in the intermediate graph data structurep,q∈sw2(vp,vq) Is the sum of the edge weights of any two nodes in the class s in the intermediate graph data structure.
8. The big-data-analysis-based enterprise production data storage method of claim 1, wherein the adding the first node to the intermediate graph data structure to obtain a target graph data structure comprises:
adding the first node into the intermediate graph data structure, connecting with any one node in the second node set, and calculating the edge weight of any one node in the first node and the second node set, including:
calculating an edge weight value of any one of the first node and the second node set according to the following formula:
wherein u is the category corresponding to the maximum category association degree of the first node x and the category association degrees of the categories, x is the first node added to the intermediate graph data structure, and x is the second node added to the intermediate graph data structuretIs a categoryu any one node, w3(x, x)t) For the first node x and any node x in the class utY is the total number of nodes within the class u, w4(x, x)t) For the first node x and any node x in the class utThe edge weights in the initial graph data structure,
is the sum of the edge weights of the first node x and all nodes in the class u in the initial graph data structure,
the number of edges between the nodes in the category u is shown;
the degree of association between the first node x and the class u.
9. The big-data-analysis-based enterprise production data storage method of claim 1, wherein after obtaining the target graph data structure, the method further comprises:
acquiring a third node set which is not added to the intermediate graph data structure in the first node set;
for any one third node in the third node set, acquiring a fourth node set which is connected with the third node in the initial graph data structure and is in the intermediate graph data structure, clustering the fourth node set to obtain a plurality of classes, and acquiring the class association degree of the third node and each class; acquiring a target category corresponding to the maximum category association degree in the category association degrees of the third node and each category, wherein for any node pair in the target category, the node pair comprises two fourth nodes connected with edges, and if an abnormal vector is respectively in the same window in the third node and the two fourth nodes in the node pair, the window is acquired, and at least one window is finally acquired;
acquiring a plurality of abnormal vector sequences representing the third node from each acquired window, acquiring the correlation characteristics of any two abnormal vector sequences, acquiring the mean value of the correlation characteristics larger than a preset threshold value, and acquiring the edge weight of the third node and the node pair;
according to the edge weights of the third node and the node pairs, the edge weights of the third node and all the node pairs including a certain fourth node are obtained, and the sum of the edge weights is obtained and is the edge weight of the third node and the certain fourth node;
if the edge weight of the third node and the certain fourth node meets the preset requirement and the maximum class association degree in the class association degrees of the third node and each class is greater than a preset threshold value, adding the third node into the intermediate graph data structure to obtain a final graph data structure;
the screening the abnormal vectors of the devices according to the target graph data structure and storing the abnormal vectors obtained by screening, comprising the following steps:
and screening the abnormal vectors of each device according to the final graph data structure, and storing the abnormal vectors obtained by screening.
10. An enterprise production data storage system based on big data analysis, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111102554.9A CN113822570B (en) | 2021-09-20 | 2021-09-20 | Enterprise production data storage method and system based on big data analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111102554.9A CN113822570B (en) | 2021-09-20 | 2021-09-20 | Enterprise production data storage method and system based on big data analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822570A true CN113822570A (en) | 2021-12-21 |
| CN113822570B CN113822570B (en) | 2023-09-26 |
Family
ID=78922689
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111102554.9A Active CN113822570B (en) | 2021-09-20 | 2021-09-20 | Enterprise production data storage method and system based on big data analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822570B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114647386A (en) * | 2022-04-02 | 2022-06-21 | 南京大昌智能科技有限公司 | Big data distributed storage method based on artificial intelligence |
| CN115292392A (en) * | 2022-10-10 | 2022-11-04 | 南通海隼信息科技有限公司 | Data management method for intelligent warehousing |
| CN117235649A (en) * | 2023-11-09 | 2023-12-15 | 广东正德工业科技股份有限公司 | Industrial equipment state intelligent monitoring system and method based on big data |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110043626A1 (en) * | 2009-08-18 | 2011-02-24 | Wesley Kenneth Cobb | Intra-trajectory anomaly detection using adaptive voting experts in a video surveillance system |
| CN106909664A (en) * | 2017-02-28 | 2017-06-30 | 国网福建省电力有限公司 | A kind of power equipment data stream failure recognition methods |
| CN107977301A (en) * | 2017-11-21 | 2018-05-01 | 东软集团股份有限公司 | Detection method, device, storage medium and the electronic equipment of unit exception |
| CN108829794A (en) * | 2018-06-04 | 2018-11-16 | 北京交通大学 | Alert analysis method based on interval graph |
| EP3499396A1 (en) * | 2017-12-12 | 2019-06-19 | Institute for Imformation Industry | Abnormal behavior detection model building apparatus and abnormal behavior detection model building method thereof |
| CN110391936A (en) * | 2019-07-25 | 2019-10-29 | 长沙学院 | A kind of novel clustering algorithm based on timing alarm |
| CA3098860A1 (en) * | 2018-04-23 | 2019-10-31 | Huawei Technologies Co., Ltd. | Alarm log compression method, apparatus, and system, and storage medium |
| US20210097081A1 (en) * | 2019-09-30 | 2021-04-01 | Sap Se | Throughput optimization in distributed database systems using hypergraph partitioning |
| CN112732798A (en) * | 2021-02-18 | 2021-04-30 | 北京工商大学 | Sequence data association rule mining method based on fragment clustering |
| US20210209938A1 (en) * | 2020-09-25 | 2021-07-08 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus, system, and computer-readable medium for traffic pattern prediction |
| CN113408829A (en) * | 2021-08-19 | 2021-09-17 | 南通倍佳机械科技有限公司 | Hazardous area equipment data compression method and system based on big data analysis |
-
2021
- 2021-09-20 CN CN202111102554.9A patent/CN113822570B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110043626A1 (en) * | 2009-08-18 | 2011-02-24 | Wesley Kenneth Cobb | Intra-trajectory anomaly detection using adaptive voting experts in a video surveillance system |
| CN106909664A (en) * | 2017-02-28 | 2017-06-30 | 国网福建省电力有限公司 | A kind of power equipment data stream failure recognition methods |
| CN107977301A (en) * | 2017-11-21 | 2018-05-01 | 东软集团股份有限公司 | Detection method, device, storage medium and the electronic equipment of unit exception |
| EP3499396A1 (en) * | 2017-12-12 | 2019-06-19 | Institute for Imformation Industry | Abnormal behavior detection model building apparatus and abnormal behavior detection model building method thereof |
| CA3098860A1 (en) * | 2018-04-23 | 2019-10-31 | Huawei Technologies Co., Ltd. | Alarm log compression method, apparatus, and system, and storage medium |
| CN108829794A (en) * | 2018-06-04 | 2018-11-16 | 北京交通大学 | Alert analysis method based on interval graph |
| CN110391936A (en) * | 2019-07-25 | 2019-10-29 | 长沙学院 | A kind of novel clustering algorithm based on timing alarm |
| US20210097081A1 (en) * | 2019-09-30 | 2021-04-01 | Sap Se | Throughput optimization in distributed database systems using hypergraph partitioning |
| US20210209938A1 (en) * | 2020-09-25 | 2021-07-08 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method, apparatus, system, and computer-readable medium for traffic pattern prediction |
| CN112732798A (en) * | 2021-02-18 | 2021-04-30 | 北京工商大学 | Sequence data association rule mining method based on fragment clustering |
| CN113408829A (en) * | 2021-08-19 | 2021-09-17 | 南通倍佳机械科技有限公司 | Hazardous area equipment data compression method and system based on big data analysis |
Non-Patent Citations (4)
| Title |
|---|
| JINBO LI.ETC: "Clustering-based anomaly detection in multivariate time series data", APPLIED SOFT COMPUTING JOURNAL * |
| SIMON MECKEL.ETC: "Combined compression of multiple correlated data streams for online-diagnosis systems", MICROPROCESSORS AND MICROSYSTEMS * |
| 何明亮;陈泽茂;左进;: "基于多窗口机制的聚类异常检测算法", 信息网络安全 * |
| 吴东;郭春;申国伟;: "一种基于多因素的告警关联方法", 计算机与现代化 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114647386A (en) * | 2022-04-02 | 2022-06-21 | 南京大昌智能科技有限公司 | Big data distributed storage method based on artificial intelligence |
| CN114647386B (en) * | 2022-04-02 | 2023-12-26 | 水发科技信息(山东)有限公司 | Big data distributed storage method based on artificial intelligence |
| CN115292392A (en) * | 2022-10-10 | 2022-11-04 | 南通海隼信息科技有限公司 | Data management method for intelligent warehousing |
| CN117235649A (en) * | 2023-11-09 | 2023-12-15 | 广东正德工业科技股份有限公司 | Industrial equipment state intelligent monitoring system and method based on big data |
| CN117235649B (en) * | 2023-11-09 | 2024-02-13 | 广东正德工业科技股份有限公司 | Industrial equipment state intelligent monitoring system and method based on big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822570B (en) | 2023-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113822570A (en) | Enterprise production data storage method and system based on big data analysis | |
| CN108762228B (en) | Distributed PCA-based multi-working-condition fault monitoring method | |
| CA2634328C (en) | Method and system for trend detection and analysis | |
| CN112987675B (en) | Method, device, computer equipment and medium for anomaly detection | |
| CN111008363B (en) | Multivariable causal-driven complex electromechanical system service safety situation evaluation method | |
| CN109740687B (en) | Fermentation process fault monitoring method based on DLAE | |
| CN112668105B (en) | Helicopter transmission shaft abnormity judgment method based on SAE and Mahalanobis distance | |
| Jones et al. | Phase I control charts for times between events | |
| CN111324110A (en) | Fermentation process fault monitoring method based on multiple shrinkage automatic encoders | |
| CN112414694A (en) | Equipment multistage abnormal state identification method and device based on multivariate state estimation technology | |
| CN112000081B (en) | Fault monitoring method and system based on multi-block information extraction and Mahalanobis distance | |
| CN113568774A (en) | Real-time anomaly detection method for multi-dimensional time sequence data by using unsupervised deep neural network | |
| CN117076258A (en) | Remote monitoring method and system based on Internet cloud | |
| Sterritt | Discovering rules for fault management | |
| CN112418460A (en) | Fault diagnosis method and fault diagnosis device for engineering vehicle | |
| CN111740998A (en) | Network intrusion detection method based on stacked self-encoder | |
| CN116641941A (en) | Hydraulic system early fault dynamic detection method based on typical variable analysis | |
| CN116307787A (en) | Fingerprint-based server operation and maintenance fault identification method and positioning method | |
| CN115700553A (en) | Anomaly detection method and related device | |
| CN115001781A (en) | Terminal network state safety monitoring method | |
| CN113724211A (en) | Fault automatic identification method and system based on state induction | |
| CN112378651A (en) | Data-driven-based equipment dynamic reliability assessment method | |
| CN117828285B (en) | Production data management system and method | |
| Erguo et al. | An input-training neural network based nonlinear principal component analysis approach for fault diagnosis | |
| CN117349770B (en) | Structural health monitoring multi-strain sensor data anomaly detection and repair method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230830 Address after: 1711, 17th Floor, Building 10, Yard 1, Jinfang Road, Chaoyang District, Beijing, 100012 Applicant after: Beijing Hanbo Network Technology Co.,Ltd. Address before: 450000 No. b2008-1, 20th floor, Xicheng science and technology building, No. 41, Jinsuo Road, high tech Industrial Development Zone, Zhengzhou City, Henan Province Applicant before: Henan Fitch Network Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |