CN116244293A - Method, system, device and medium for eliminating abnormal value of equipment - Google Patents
Method, system, device and medium for eliminating abnormal value of equipment Download PDFInfo
- Publication number
- CN116244293A CN116244293A CN202310194526.7A CN202310194526A CN116244293A CN 116244293 A CN116244293 A CN 116244293A CN 202310194526 A CN202310194526 A CN 202310194526A CN 116244293 A CN116244293 A CN 116244293A
- Authority
- CN
- China
- Prior art keywords
- data
- operation data
- historical operation
- characteristic
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Abstract
The application discloses a method, a system, a device and a medium for eliminating abnormal values of equipment, which are applied to the field of equipment operation state monitoring, wherein the method, the system, the device and the medium are used for obtaining historical operation data of equipment with a plurality of measuring points, performing sliding windowing on the historical operation data, calculating characteristic values of the measuring points in each sliding window to obtain corresponding characteristic frequency numbers, determining data to be deleted according to the characteristic frequency numbers, and repeating the operation until no abnormal data exist in a state matrix. By the method, unsupervised anomaly detection is carried out, manual or small amount of manual participation is not needed, a fixed threshold value is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
Description
Technical Field
The application relates to the field of equipment operation state monitoring, in particular to a method, a system, a device and a medium for eliminating abnormal values of equipment.
Background
Along with the development of big data and intelligent sensing technology, the data reconstruction method based on data driving is widely used in equipment operation state monitoring, but the data reconstruction method is seriously dependent on a state matrix, the construction quality of the state matrix directly influences the estimation precision of the data reconstruction method, abnormal parameters are removed in the construction process of the equipment state matrix, and only equipment operation parameters in a normal operation state are reserved.
At present, a method mainly adopted is manual elimination or fixed threshold value setting, but the manual elimination requires engineers to analyze equipment operation parameters one by one, and eliminates abnormal values according to self experience, and the process of the method is quite time-consuming and labor-consuming; by setting threshold limits for each characteristic data in the state matrix and rejecting the data which does not meet the threshold limits, the method is easy to cause false recognition, and especially for long-period data, the difficulty in selecting a proper threshold value is high, so that the conditions that excessive normal values are rejected and abnormal values are reserved are caused.
In view of the above problems, a solution to the above technical problems is sought for by those skilled in the art.
Disclosure of Invention
The method, the system, the device and the medium are used for obtaining historical operation data of equipment with a plurality of measuring points, performing sliding windowing on the historical operation data, calculating characteristic values of the measuring points in each sliding window to obtain corresponding characteristic frequency numbers, determining data to be deleted according to the characteristic frequency numbers, and repeating operation until no abnormal data exist in a state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
In order to solve the above technical problems, the present application provides a method for rejecting abnormal values of a device, including:
acquiring historical operation data of equipment of a plurality of measuring points;
sliding windowing is carried out on the historical operation data, and characteristic values of all measuring points in all sliding windows are calculated;
acquiring the feature frequency corresponding to each feature value;
and determining historical operation data to be removed according to the feature frequency.
Preferably, determining the historical operating data to be rejected according to the feature frequency comprises:
calculating a feature frequency score according to the feature frequency;
sequentially sequencing the characteristic frequency scores according to the sequence from small to large, and marking window intervals of the first beta characteristic frequency scores as eliminating intervals;
and deleting the historical operation data in the rejection interval.
Preferably, after acquiring the historical operation data of the device under the plurality of measuring points, sliding windowing is performed on the historical operation data, and before calculating the characteristic value of each measuring point in each window, the method further comprises:
and carrying out normalization processing on the historical operation data of each measuring point.
Preferably, the acquiring the historical operation data of the equipment with the plurality of measuring points further comprises:
quantitatively describing historical operation data of each measuring point through a correlation analysis algorithm;
aligning the historical operation data of each measuring point on a time coordinate and forming an original state matrix of the equipment;
the historical operation data comprise real measurement point data, test data and model calculation data of the equipment, wherein the abscissa in an original state matrix of the equipment is a time coordinate, and the ordinate is a corresponding selected measurement point.
Preferably, sliding windowing is performed on the historical operation data, and characteristic values of each measuring point in each sliding window are calculated, including:
acquiring the historical operation data corresponding to the measuring points in each sliding window;
determining corresponding characteristic data according to the historical operation data in each sliding window;
acquiring confidence parameters corresponding to the feature data in advance;
determining a confidence interval corresponding to each feature data according to the initial feature value and the confidence parameter corresponding to each feature data;
determining a coincidence interval between confidence intervals in the sliding windows according to the characteristic data and the corresponding confidence intervals in the sliding windows;
and setting the initial characteristic values in the sliding windows corresponding to the overlapping intervals to be the same, wherein the initial characteristic values are the characteristic values of the corresponding measuring points.
Preferably, calculating the feature frequency score from the feature frequency comprises:
calculating a characteristic frequency score according to a preset formula;
wherein, the preset formula is:
wherein R is a feature frequency score, n is the number of selected features, M i Is the characteristic value of a certain characteristic, f (M i ) Is the characteristic value M i Frequency of occurrence.
Preferably, the determining the historical operation data to be rejected according to the feature frequency further comprises:
judging whether abnormal data exists in the original state matrix of the equipment;
if so, returning to the step of performing sliding windowing on the historical operation data and calculating the characteristic value of each measuring point in each sliding window.
In order to solve the above technical problem, the present application further provides an equipment outlier rejection system, including:
the acquisition module is used for acquiring historical operation data of equipment of a plurality of measuring points;
the processing module is used for carrying out sliding windowing processing on the historical operation data and calculating characteristic values of all measuring points in all sliding windows;
the computing module is used for acquiring the feature frequency corresponding to each feature value;
and the determining module is used for determining historical operation data to be removed according to the characteristic frequency.
In order to solve the technical problem, the application also provides an equipment outlier removing device, which comprises a memory, a control unit and a control unit, wherein the memory is used for storing a computer program;
and the processor is used for realizing the steps of the device outlier rejection method when executing the computer program.
In order to solve the above technical problem, the present application further provides a computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements the steps of the device outlier rejection method as described above.
According to the method for eliminating the abnormal value of the equipment, the historical operation data of the equipment with the plurality of measuring points are obtained, sliding windowing processing is carried out on the historical operation data, the characteristic value of each measuring point in each sliding window is calculated to obtain the corresponding characteristic frequency, the data to be deleted is determined according to the characteristic frequency, and the operation is repeated until no abnormal data exist in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
The equipment outlier eliminating system, the equipment outlier eliminating device and the storage medium have the same beneficial effects as the equipment outlier eliminating method.
Drawings
For a clearer description of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for eliminating abnormal values of a device provided in the present application;
FIG. 2 is a block diagram of an outlier rejection system provided herein;
fig. 3 is a block diagram of an abnormal value removing apparatus for an apparatus according to another embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments herein without making any inventive effort are intended to fall within the scope of the present application.
The core of the application is to provide a method, a system, a device and a medium for eliminating abnormal values of equipment, which are used for obtaining historical operation data of the equipment with a plurality of measuring points, performing sliding windowing processing on the historical operation data, calculating characteristic values of the measuring points in each sliding window to obtain corresponding characteristic frequency numbers, determining the data to be deleted according to the characteristic frequency numbers, and repeating the operation until no abnormal data exists in a state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
In order to provide a better understanding of the present application, those skilled in the art will now make further details of the present application with reference to the drawings and detailed description.
Fig. 1 is a flowchart of a method for eliminating abnormal values of a device provided in the present application, as shown in fig. 1, where the method includes:
s10: acquiring historical operation data of equipment of a plurality of measuring points;
it should be noted that, for different devices, a plurality of test points are set, the embodiment of the application does not specifically define the type of the device to be tested and the scene to which the device is applied, the embodiment of the application does not specifically define the specific number of the test points and the type of the test points, the embodiment of the application does not specifically define the type and the number of the historical operation data, the historical operation data and the device have strong correlation, and the application does not specifically define the processing mode of the historical operation data.
S11: sliding windowing is carried out on the historical operation data, and characteristic values of all measuring points in all sliding windows are calculated;
it should be noted that, according to the method provided by the embodiment of the application, the historical operation data is subjected to sliding windowing, the abnormal points in the sliding window are not directly calculated, the characteristic values of the measuring points in the sliding window are calculated, the abnormal measuring points are removed according to the characteristic values, the dividing number of the sliding window is not specifically limited, the width of the sliding window and the step length of the sliding window are not specifically limited, the mode of determining the abnormal measuring points according to the characteristic values is not specifically limited, the types and the number of the characteristic values are not specifically limited, the characteristic values of the measuring point data in the sliding window can be determined according to the characteristics of the actual operation data of the device, and the characteristic values include, but are not limited to, data mean values, variances, energy values, effective values, peaks, peak values, kurtosis values and kurtosis degrees.
S12: acquiring the feature frequency corresponding to each feature value;
s13: and determining historical operation data to be removed according to the feature frequency.
It should be noted that, the frequency of occurrence of each feature value, that is, the feature frequency, is determined according to the feature frequency, and the corresponding historical operation data to be removed is determined according to the feature frequency, in the embodiment of the present application, the mode of removing the top historical operation data to be removed according to the feature frequency is not limited, and the mode of calculating and determining the removal interval through a preset formula may be performed, so that the embodiment of the present application only provides a preferred embodiment, and is not limited to the mode only described above.
Therefore, according to the method provided by the embodiment of the application, the historical operation data of the equipment with the plurality of measuring points are obtained, the historical operation data are subjected to sliding windowing, the characteristic values of the measuring points in each sliding window are calculated to obtain the corresponding characteristic frequency, the data to be deleted are determined according to the characteristic frequency, and the operation is repeated until no abnormal data exist in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
Based on the foregoing embodiments, the present application provides a preferred embodiment, where determining, according to the feature frequency, historical operating data that needs to be removed includes:
calculating a feature frequency score according to the feature frequency;
sequentially sequencing the characteristic frequency scores according to the sequence from small to large, and marking window intervals of the first beta characteristic frequency scores as eliminating intervals;
and deleting the historical operation data in the rejection interval.
It should be noted that, when determining the historical operation data to be removed according to the feature frequency, calculating the score of the feature frequency according to the feature frequency, the embodiment of the application does not specifically limit the calculation mode of the feature frequency score, and can calculate the feature frequency score according to a preset formula. Specifically, the preset formula may assign a corresponding weight score to obtain a corresponding feature frequency score, or divide the feature frequency score into corresponding frequency intervals according to the feature frequency specific data, map different frequency intervals into corresponding feature frequency scores to obtain corresponding feature frequency scores, and determine a corresponding score according to the occurrence frequency of the feature values, which is not limited herein, and may be calculated according to practical situations.
Regarding the processing of the feature frequency scores, the frequency probabilities of occurrence of the feature values in the sliding window interval are different, the corresponding feature frequency scores are also different, the jump pull distance of the feature frequency scores is larger in the statistical process, for example, the feature frequency scores corresponding to the 5 feature values are respectively 10, 5, 10 and 9, and the corresponding score 5 is larger than other score pull distances, so that the feature frequency scores are required to be removed as abnormal parameters. The existing rejecting means is used for rejecting through self experience manually, characteristic frequency scores are ordered in a certain sequence for realizing automatic rejecting, and characteristic values corresponding to the front beta or rear beta ordered characteristic frequency scores are selected for rejecting.
It can be understood that the window sections of the first β are marked as the reject sections, or the window sections of the second β are marked as the reject sections, in which order the window sections are ordered, preferably from small to large. The method comprises the steps of sorting the scores from small to large, setting a marking parameter beta, marking window intervals of the former beta characteristic frequency scores as a reject interval, improving reject efficiency, improving accuracy of the determined reject interval, and deleting historical operation data in the reject interval, wherein the embodiment of the application does not specifically limit the value of beta, if the historical operation data of the former beta window intervals are rejected, abnormal data still exist, modifying the value of beta in a manual mode and re-rejecting the value, the embodiment of the application is not limited to the mode only comprising the above mode, and the embodiment of the application can be selected according to actual conditions.
Therefore, according to the method provided by the embodiment of the application, by acquiring the historical operation data of the equipment with the plurality of measuring points, performing sliding windowing on the historical operation data, calculating the characteristic values of the measuring points in each sliding window, calculating the characteristic frequency score, deleting the original data of the window intervals of the prior beta characteristic frequency scores according to the sequence from small to large, and repeating the operation until no abnormal data exists in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
On the basis of the foregoing embodiment, the present application provides a preferred embodiment, after acquiring the historical operation data of the device under the plurality of measurement points, performing sliding windowing on the historical operation data, and before calculating the feature values of each measurement point in each window, further including:
and carrying out normalization processing on the historical operation data of each measuring point.
Before performing sliding windowing on the historical operation data and calculating the characteristic values of each measuring point in each window, performing normalization on the historical operation data of each measuring point, and performing sliding windowing on the historical operation data after normalization.
Therefore, according to the method provided by the embodiment of the application, by acquiring the historical operation data of the equipment with the plurality of measuring points, carrying out the sliding windowing processing after carrying out the normalization processing on the historical operation data, calculating the characteristic values of the measuring points in each sliding window, calculating the characteristic frequency score, deleting the original data of the window intervals of the former beta characteristic frequency scores according to the sequence from small to large, and repeating the operation until no abnormal data exists in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
On the basis of the above embodiment, the present application provides a preferred embodiment, and further includes, after acquiring the historical operation data of the device at the plurality of measurement points:
quantitatively describing historical operation data of each measuring point through a correlation analysis algorithm;
aligning the historical operation data of each measuring point on a time coordinate and forming an original state matrix of the equipment;
the historical operation data comprise real measurement point data, test data and model calculation data of the equipment, wherein the abscissa in an original state matrix of the equipment is a time coordinate, and the ordinate is a corresponding selected measurement point.
After the historical operation data of the device is obtained, the historical operation data is processed by correlation to form an original state matrix of the device, wherein the method for forming the original state matrix of the device can be used for quantitatively describing the historical operation data of each measuring point by a correlation analysis algorithm, the historical operation data of each measuring point is aligned on a time coordinate and forms the original state matrix of the device, the historical operation data can be used for forming the actual measurement point data of the device, the test data, the model calculation data and the like, the actual measurement point data can be used for forming the actual current value, the voltage value, the motor rotation speed value and the like measured by using corresponding sensors on the device as measuring points, and the test data is simulation data obtained by manually pressurizing and the like in the actual operation environment of the simulation device. The model calculation data may be data such as flow difference values or pressure difference values obtained by using a certain model corresponding to a plurality of pipeline inlets and a plurality of pipeline outlets, and in this embodiment, only the data corresponding to the equipment are collectively referred to, and specific data may be set according to actual situations. The abscissa in the original state matrix of the equipment is a time coordinate, and the ordinate is a corresponding selected measuring point, and the embodiment of the application is not limited to a mode only comprising the original state matrix of the forming equipment, and can be changed according to actual conditions.
It can be seen that, according to the method provided by the embodiment of the application, by acquiring the historical operation data of the device at the plurality of measuring points and forming the device original state matrix, normalizing the historical operation data in the device original state matrix, then performing sliding windowing on the device original state matrix, calculating the characteristic values of the measuring points in each sliding window, calculating the characteristic frequency score, deleting the original data of the window intervals of the front beta characteristic frequency scores according to the sequence from small to large, and repeating the operation until no abnormal data exists in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
Based on the foregoing embodiments, the present application provides a preferred embodiment, performing sliding windowing on historical operation data, and calculating a characteristic value of each measurement point in each sliding window, where the method includes:
acquiring historical operation data corresponding to the measuring points in each sliding window;
determining corresponding characteristic data according to historical operation data in each sliding window;
acquiring confidence parameters corresponding to the feature data in advance;
determining a confidence interval corresponding to each feature data according to the initial feature value and the confidence parameter corresponding to each feature data;
determining a coincidence interval between the confidence intervals in each sliding window according to the characteristic data in each sliding window and the corresponding confidence interval;
and setting the initial characteristic values in the sliding windows corresponding to the overlapping sections to be the same, wherein the initial characteristic values are characteristic values of the corresponding measuring points.
It will be appreciated that the processing within each sliding window takes one sliding window as an example, and there is measurement point data within the sliding window, which determines its characteristic data according to the historical operation data corresponding to the measurement point data, where what characteristic data is corresponding is determined by the historical operation data. It should be noted that, 100 pieces of historical operation data are assumed in a sliding window, the characteristic data corresponds to a data processing mode of the historical operation data, such as mean processing, variance processing, and the like, the mean value obtained by the corresponding mean processing is taken as one characteristic data, and the variance obtained by the variance processing is taken as one characteristic data. The feature data includes, but is not limited to, a data mean value, a variance, an energy value, an effective value, a peak-to-peak value, a kurtosis and the like, that is, feature data to be performed is determined according to current historical operation data, and in combination with the embodiment, 7 feature data including a data mean value, a variance, an energy value, an effective value, a peak-to-peak value and a kurtosis can be determined according to current historical operation data, and the corresponding feature data processing process and the existing processing mode can be the same or different, and are not limited herein.
Correspondingly, the confidence parameters of each feature data are obtained, and as the initial feature value corresponding to the feature data is the initial result value obtained after different data processing modes, for example, the data mean is one feature data, the variance is one feature data, the peak value is one feature data, the initial feature value obtained under each feature data is different, and the set confidence parameters can be the same or different, as an embodiment, one feature corresponds to one confidence parameter, that is, the confidence parameters of each feature data are different. And determining a corresponding confidence interval according to the initial characteristic value and the confidence parameter corresponding to each characteristic data.
Taking feature data of a feature mean value as an example, setting a confidence parameter, and determining a confidence interval of the feature value according to the confidence parameter;
wherein the confidence interval is MC= [ M-a, M+a ], wherein MC represents the confidence interval, M represents the characteristic value, and a represents the confidence parameter.
Because each sliding window is in a sliding state, the data in each sliding window are the same or different, and the initial characteristic value difference under the characteristic data corresponding to the historical operation data corresponding to the adjacent sliding window is smaller, the adjacent sliding window can determine the coincidence interval of the confidence interval according to the respective characteristic data and the corresponding confidence interval. The corresponding initial characteristic value can be determined to be updated to be the same according to the overlapping interval, and the determining process can be used for preprocessing the characteristic value according to the fact that the initial characteristic value in the later sliding window to which the overlapping interval belongs is the same as the initial characteristic value in the former sliding window to which the overlapping interval belongs.
Taking two sliding windows as an example, each sliding window has 100 historical operation data, each 100 characteristic data determined by the historical operation data has 7 characteristic data of data mean value, variance, energy value, effective value, peak-to-peak value and kurtosis, taking the characteristic data of the data mean value as an example, calculating corresponding initial characteristic values (mean value results) of 5 and 6 according to the respective 100 historical operation data in the first sliding window and the second sliding window, and confidence parameters of the corresponding first sliding window and the second sliding window are 1, wherein the first confidence interval of the corresponding first sliding window is [4,6], the second confidence interval of the second sliding window is [5,7], the coincidence interval of the two confidence intervals is [5,6], and determining the characteristic value of the measuring point under the characteristic data (data mean) according to the initial characteristic value of 5 and the coincidence interval in the first sliding window is 5.
In the embodiment of the present application, the range size of the communication section is not specifically limited, and the value size of the communication parameter is not specifically limited.
Therefore, in the embodiment of the application, by setting the confidence intervals, if the confidence intervals of the characteristic values of the characteristics are overlapped, the characteristic values are judged to be the same, so that the recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
On the basis of the foregoing embodiments, the present application provides a preferred embodiment, wherein calculating the feature frequency score according to the feature frequency includes:
calculating a characteristic frequency score according to a preset formula;
wherein, the preset formula is:
wherein R is a feature frequency score, n is the number of selected features, M i Is the characteristic value of a certain characteristic, f (M i ) Is the characteristic value M i Frequency of occurrence.
Therefore, in the manner provided by the embodiment of the application, the feature frequency score is calculated according to the feature frequency score, the number of features, the feature value of a certain feature and the frequency of occurrence of the feature value, and the original data of window intervals of the previous beta feature frequency scores are deleted in the order from small to large, and the operation is repeated until no abnormal data exist in the state matrix. By the method, unsupervised anomaly detection is carried out, manual work or only a small amount of manual participation is not needed, labor force is greatly liberated, and recognition and rejection efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
Based on the foregoing embodiment, the present application provides a preferred embodiment, and further includes, after determining, according to the feature frequency, historical operating data that needs to be removed:
judging whether abnormal data exists in the original state matrix of the equipment; if the abnormal value exists, the step of sliding windowing is carried out on the historical operation data, the characteristic value of each measuring point in each sliding window is calculated, and if the abnormal value does not exist, the abnormal value is totally removed.
Therefore, according to the method provided by the embodiment of the application, the abnormal data is repeatedly judged and removed, the unsupervised abnormal detection is carried out, no manual work or only a small amount of manual participation is needed, the labor force is greatly liberated, and the recognition and removal efficiency is greatly improved; the method can comprehensively consider the distribution condition of the equipment operation data, and better solves the problems that excessive normal values are removed and abnormal values are reserved due to the fixed removal threshold value.
Based on the angle of the functional module, the application further provides a corresponding embodiment of the system for eliminating abnormal values of equipment, as shown in fig. 2, fig. 2 is a structural diagram of the system for eliminating abnormal values of equipment, provided by the application, and the system includes:
an acquisition module 10, configured to acquire historical operation data of devices at a plurality of measurement points;
the processing module 11 is used for performing sliding windowing processing on the historical operation data and calculating characteristic values of all measuring points in all sliding windows;
the calculating module 12 is configured to obtain a feature frequency corresponding to each feature value;
and the determining module 13 is used for determining historical operation data to be removed according to the feature frequency.
Since the embodiments of the system portion and the embodiments of the method portion correspond to each other, the embodiments of the system portion refer to the description of the embodiments of the method portion, which is not repeated herein.
The abnormal value eliminating system of the equipment provided by the embodiment corresponds to the abnormal value eliminating method of the equipment, so that the abnormal value eliminating system of the equipment has the same beneficial effects as the method.
Fig. 3 is a block diagram of an apparatus outlier removing apparatus according to another embodiment of the present application, and as shown in fig. 3, the apparatus includes: a memory 20 for storing a computer program;
a processor 21 for implementing the steps of the device outlier rejection method as mentioned in the above embodiments when executing a computer program.
The device outlier removing apparatus provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.
Processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in hardware in at least one of a digital signal processor (Digital Signal Processor, DSP), a Field programmable gate array (Field-Programmable Gate Array, FPGA), a programmable logic array (Programmable Logic Array, PLA). The processor 21 may also comprise a main processor, which is a processor for processing data in an awake state, also called central processor (Central Processing Unit, CPU), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with an image processor (Graphics Processing Unit, GPU) for taking care of rendering and rendering of the content that the display screen is required to display. In some embodiments, the processor 21 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, where the computer program, after being loaded and executed by the processor 21, can implement the relevant steps of the device outlier rejection method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may further include an operating system 202, data 203, and the like, where the storage manner may be transient storage or permanent storage. The operating system 202 may include Windows, unix, linux, among others. The data 203 may include, but is not limited to, data contained in a device outlier rejection method, and the like.
In some embodiments, the device outlier removing apparatus may further include a display 22, an input/output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
It will be appreciated by those skilled in the art that the configuration shown in fig. 3 does not constitute a limitation of the device outlier rejection means and may include more or less components than those illustrated.
The device outlier removing apparatus provided in the embodiment of the present application includes a memory and a processor, where the processor can implement the following method when executing a program stored in the memory: an equipment outlier rejection method.
Finally, the present application also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps as described in the method embodiments above.
It will be appreciated that the methods of the above embodiments, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored on a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution contributing to the prior art, or in a software product stored in a storage medium, performing all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The method, the system, the device and the medium for eliminating the abnormal value of the equipment provided by the application are described in detail. In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the claims of the present application.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
Claims (10)
1. A method for rejecting an outlier of a device, the method comprising:
acquiring historical operation data of equipment of a plurality of measuring points;
performing sliding windowing on the historical operation data, and calculating characteristic values of the measuring points in each sliding window;
acquiring the characteristic frequency corresponding to each characteristic value;
and determining the historical operation data to be removed according to the characteristic frequency.
2. The apparatus outlier rejection method according to claim 1, wherein the determining the historical operating data to be rejected according to the feature frequency comprises:
calculating a feature frequency score according to the feature frequency;
sequentially sequencing the characteristic frequency scores according to the sequence from small to large, and marking window intervals of the first beta characteristic frequency scores as eliminating intervals;
and deleting the historical operation data in the rejection interval.
3. The method for eliminating abnormal values of equipment according to claim 2, wherein after the historical operation data of the equipment under the plurality of measuring points is obtained, before performing sliding windowing processing on the historical operation data and calculating the characteristic value of each measuring point in each window, the method further comprises:
and carrying out normalization processing on the historical operation data of each measuring point.
4. The method for eliminating abnormal values of equipment according to claim 3, wherein the step of acquiring the historical operating data of the equipment with a plurality of measuring points further comprises the steps of:
quantitatively describing the historical operation data of each measuring point through a correlation analysis algorithm;
aligning the historical operation data of each measuring point on a time coordinate and forming an original state matrix of the equipment;
the historical operation data comprise real measurement point data, test data and model calculation data of the equipment, wherein the abscissa in an original state matrix of the equipment is the time coordinate, and the ordinate is the measurement point which is correspondingly selected.
5. The method for eliminating abnormal values of equipment according to claim 4, wherein the sliding windowing process is performed on the historical operation data, and the characteristic value of each measuring point in each sliding window is calculated, and the method comprises the steps of:
acquiring the historical operation data corresponding to the measuring points in each sliding window;
determining corresponding characteristic data according to the historical operation data in each sliding window;
acquiring confidence parameters corresponding to the feature data in advance;
determining a confidence interval corresponding to each feature data according to the initial feature value and the confidence parameter corresponding to each feature data;
determining a coincidence interval between confidence intervals in the sliding windows according to the characteristic data and the corresponding confidence intervals in the sliding windows;
and setting the initial characteristic values in the sliding windows corresponding to the overlapping intervals to be the same, wherein the initial characteristic values are the characteristic values of the corresponding measuring points.
6. The apparatus outlier rejection method according to claim 5, wherein the calculating a feature frequency score from the feature frequency comprises:
calculating the characteristic frequency score according to a preset formula;
wherein, the preset formula is:
wherein R is the feature frequency score, n is the number of selected features, M i For the feature value of a certain feature, f (M i ) Is the characteristic value M i Frequency of occurrence.
7. The method for eliminating abnormal values of equipment according to any one of claims 1 to 6, wherein after determining the historical operating data to be eliminated according to the characteristic frequency, further comprises:
judging whether abnormal data exists in the original state matrix of the equipment;
and if the historical operation data exist, returning to the step of performing sliding windowing on the historical operation data and calculating the characteristic value of each measuring point in each sliding window.
8. A device outlier rejection system, the system comprising:
the acquisition module is used for acquiring historical operation data of equipment of a plurality of measuring points;
the processing module is used for carrying out sliding windowing on the historical operation data and calculating the characteristic value of each measuring point in each sliding window;
the computing module is used for acquiring the characteristic frequency corresponding to each characteristic value;
and the determining module is used for determining the historical operation data to be removed according to the characteristic frequency.
9. An abnormal value removing apparatus for a device, comprising a memory for storing a computer program;
a processor for implementing the steps of the device outlier rejection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, which when executed by a processor, implements the steps of the device outlier rejection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310194526.7A CN116244293A (en) | 2023-02-27 | 2023-02-27 | Method, system, device and medium for eliminating abnormal value of equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310194526.7A CN116244293A (en) | 2023-02-27 | 2023-02-27 | Method, system, device and medium for eliminating abnormal value of equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116244293A true CN116244293A (en) | 2023-06-09 |
Family
ID=86632810
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310194526.7A Pending CN116244293A (en) | 2023-02-27 | 2023-02-27 | Method, system, device and medium for eliminating abnormal value of equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116244293A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116596703A (en) * | 2023-07-17 | 2023-08-15 | 吉林省骅涛科技有限公司 | Electricity saver and intelligent control method thereof |
-
2023
- 2023-02-27 CN CN202310194526.7A patent/CN116244293A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116596703A (en) * | 2023-07-17 | 2023-08-15 | 吉林省骅涛科技有限公司 | Electricity saver and intelligent control method thereof |
| CN116596703B (en) * | 2023-07-17 | 2023-09-19 | 吉林省骅涛科技有限公司 | Electricity saver and intelligent control method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106980858B (en) | Language text detection and positioning system and language text detection and positioning method using same | |
| CN112735094B (en) | Geological disaster prediction method and device based on machine learning and electronic equipment | |
| CN110928237B (en) | Vibration signal-based numerical control machining center flutter online identification method | |
| CN112732700B (en) | Steel rolling production data slicing method, system, medium and electronic terminal | |
| CN116244293A (en) | Method, system, device and medium for eliminating abnormal value of equipment | |
| CN110969600A (en) | Product defect detection method and device, electronic equipment and storage medium | |
| CN116413604A (en) | Battery parameter monitoring method, system, device and storage medium | |
| CN114219936A (en) | Object detection method, electronic device, storage medium, and computer program product | |
| CN109102486B (en) | Surface defect detection method and device based on machine learning | |
| CN116137061B (en) | Training method and device for quantity statistical model, electronic equipment and storage medium | |
| CN111368837A (en) | Image quality evaluation method and device, electronic equipment and storage medium | |
| CN115482239A (en) | Image positioning method, system, device and medium | |
| CN106326097B (en) | Method and device for testing page perception performance | |
| CN114358091A (en) | Pile damage identification method, equipment and medium based on convolutional neural network | |
| CN116302848B (en) | Detection method and device for bias of evaluation value, electronic equipment and medium | |
| CN111797737A (en) | Remote sensing target detection method and device | |
| CN113435464B (en) | Abnormal data detection method and device, electronic equipment and computer storage medium | |
| CN114037865B (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN117571321B (en) | Bearing fault detection method, device, equipment and storage medium | |
| CN112817821B (en) | Data processing method, device, equipment and storage medium | |
| CN109298999B (en) | Core software testing method and device based on data distribution characteristics | |
| CN116468076A (en) | Driving behavior analysis method and device, electronic equipment and storage medium | |
| Chen et al. | A cellular automatic method for the edge detection of images | |
| Choi et al. | Estimation of the number of spikes using a generalized spike population model and application to RNA-seq data | |
| CN117521019A (en) | Method and system for detecting hidden danger of server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |