CN114648089A - Method, device and equipment for detecting abnormal data of inclinometer and storage medium - Google Patents
Method, device and equipment for detecting abnormal data of inclinometer and storage medium Download PDFInfo
- Publication number
- CN114648089A CN114648089A CN202011500114.4A CN202011500114A CN114648089A CN 114648089 A CN114648089 A CN 114648089A CN 202011500114 A CN202011500114 A CN 202011500114A CN 114648089 A CN114648089 A CN 114648089A
- Authority
- CN
- China
- Prior art keywords
- data
- abnormal
- encoder
- inclinometer
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C25/00—Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Manufacturing & Machinery (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
Provided are an inclinometer abnormal data detection method, device, equipment and storage medium, wherein the method comprises the following steps: acquiring data to be detected, which is acquired by an inclinometer; inputting the data to be detected into a trained convolution self-encoder, and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained by training based on original sample data of the inclinometer, and all the original sample data are normal data; when the data discrimination value is larger than an abnormal threshold value, determining that the data to be detected corresponding to the data discrimination value is abnormal data, wherein the abnormal threshold value is obtained according to a trained convolution encoder, the text obtains a convolution self-encoder through original sample data training, and judges the abnormal threshold value of the acquired data, so that the dependency relationship between the acquired data to be detected can be obtained, and the accuracy of abnormal data detection is improved.
Description
Technical Field
The invention relates to the technical field of measurement, in particular to a method, a device, equipment and a storage medium for detecting abnormal data of an inclinometer.
Background
The inclinometer is an instrument for measuring the apex angle and azimuth angle of engineering structures such as drill holes, foundation pits, foundation foundations, walls, dam slopes and the like. Some related research organizations have subsequently developed intelligent inclinometers such as resistance strain gauges, accelerometer gauges, and electronic gauges. Various inclinometers are widely applied to the fields of water conservancy and hydropower, mineral metallurgy, traffic and urban geotechnical engineering, and play an important role in ensuring the design, construction and use safety of geotechnical engineering.
In actual geotechnical measurement, because a measuring head of an inclinometer is often collided or impacted in the process of penetrating into a soil layer, the measuring head is easy to age or fail, and abnormal data can be generated in the measuring process. The traditional methods for detecting abnormal data are mostly divided into two types: the first is to artificially establish corresponding recognition rules according to the characteristics of abnormal data and normal data, the detection efficiency is low under the condition of large collected data volume, the second is to automatically learn the characteristics of the abnormal data and the normal data by adopting a machine learning or deep learning method, and then adopt a classification strategy for the data to be detected, although the latter solves the problem that the artificial modeling of the abnormal data and the normal data is difficult compared with the former, the occupation ratio of the abnormal data in the total data volume is very small in the actual measurement, the problem that the training is insufficient due to the adoption of the classification strategy for detection is often caused, the final accuracy is low, and therefore, the problem that how to improve the accuracy of the detection of the abnormal data of the inclinometer becomes urgent need to be solved.
Disclosure of Invention
In view of the above problems in the prior art, an object of the present invention is to provide a method, an apparatus, a device and a storage medium for detecting abnormal data of an inclinometer, which can improve the accuracy of detecting abnormal data of the inclinometer.
In order to solve the technical problems, the specific technical scheme is as follows:
in a first aspect, provided herein is a method of inclinometer anomaly data detection, the method comprising the steps of:
acquiring data to be detected acquired by an inclinometer;
inputting the data to be detected into a trained convolution self-encoder, and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained based on the original sample data training of the inclinometer;
and when the data discrimination value is larger than an abnormal threshold value, determining that the data to be detected corresponding to the data discrimination value is abnormal data, wherein the abnormal threshold value is obtained according to the trained convolutional encoder.
In a second aspect, there is also provided herein an inclinometer abnormal data detection apparatus, the apparatus comprising:
the data acquisition module to be detected is used for acquiring data to be detected acquired by the inclinometer;
the data discrimination value calculation module is used for inputting the data to be detected into a trained convolution self-encoder and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained based on the original sample data training of the inclinometer;
and the abnormal data determining module is used for determining the data to be detected corresponding to the data discrimination value as abnormal data when the data discrimination value is larger than an abnormal threshold value, wherein the abnormal threshold value is obtained according to the trained convolutional encoder.
In a third aspect, there is also provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps as described above when executing the computer program.
In a fourth aspect, there is also provided herein a computer readable storage medium storing an executable computer program which, when executed by a processor, performs the method steps as described above.
According to the method, the device, the equipment and the storage medium for detecting the abnormal data of the inclinometer, the dependence relationship between the acquired to-be-detected data can be acquired through the convolution self-encoder obtained by training the original sample data (all the original sample data are normal data) and the abnormal threshold for judging the abnormity of the acquired data, so that the accuracy of detecting the abnormal data is improved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating steps of a method for inclinometer anomaly data detection according to an embodiment herein;
FIG. 2 is a block diagram of a convolutional auto-encoder in an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating the data processing flow of the convolutional auto-encoder in an embodiment herein;
FIG. 4 is a schematic diagram illustrating a convolutional auto-encoder training process in an embodiment herein;
FIG. 5 is a schematic diagram illustrating an anomaly threshold acquisition step in an embodiment herein;
FIG. 6 is a schematic diagram illustrating the super parameter determination step in an embodiment herein;
FIG. 7 is a schematic diagram illustrating an abnormal inclinometer data detection device according to an embodiment of the present disclosure;
fig. 8 shows a schematic structural diagram of a computer device in the embodiment herein.
Description of the symbols of the drawings:
100. a module for acquiring data to be tested;
200. a data discrimination value calculation module;
300. an abnormal data determination module;
802. a computer device;
804. a processor;
806. a memory;
808. a drive mechanism;
810. an input/output module;
812. an input device;
814. an output device;
816. a presentation device;
818. a graphical user interface;
820. a network interface;
822. a communication link;
824. a communication bus.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments herein without making any creative effort, shall fall within the scope of protection.
It should be noted that the terms "first," "second," and the like in the description and claims herein and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments herein described are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or device.
In the use process of the inclinometer, some abnormal data can be generated due to aging or failure of a measuring head, in the prior art, a method of manual identification and machine learning is usually adopted during abnormal data detection, the low efficiency of the manual identification is gradually eliminated, the machine learning mainly automatically learns the characteristics of the abnormal data and the normal data, and then a classification strategy is adopted for the data to be detected, but the proportion of the abnormal data in the total data volume is usually very small in actual measurement, and the problem of insufficient training is often caused by adopting the classification strategy for detection, so that the final accuracy is low.
In order to solve the above problems, an embodiment of the present specification provides a method for detecting abnormal data of an inclinometer, where normal data, that is, correct data, is used as training data, a feature extraction and reconstruction is performed on the data through a self-encoder, and a convolutional neural network structure is combined to establish a convolutional self-encoder for the normal data, so as to implement detection on the abnormal data.
Specifically, as shown in fig. 1, a schematic step diagram of an inclinometer abnormal data detection method provided for the embodiment of the present specification, the present specification provides the operation steps of the method as described in the embodiment or the flowchart, but more or less operation steps may be included based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual system or apparatus product executes, it can execute sequentially or in parallel according to the method shown in the embodiment or the figures. Specifically, as shown in fig. 1, the method may include:
s101: acquiring data to be detected acquired by an inclinometer;
s102: inputting the data to be detected into a trained convolution self-encoder, and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained by training based on original sample data of the inclinometer, and the original sample data only contains normal data;
s103: and when the data discrimination value is larger than an abnormal threshold value, determining that the data to be detected corresponding to the data discrimination value is abnormal data, wherein the abnormal threshold value is obtained according to the trained convolution encoder.
In the embodiment of the specification, the trained convolutional encoders are used for detecting data to be detected, and abnormal data are determined according to the change relationship between the data to be detected after the data to be detected is reconstructed by the convolutional encoders and the data to be detected.
As shown in fig. 2, which is a schematic diagram of a framework structure of a convolutional auto-encoder in an embodiment of the present specification, a convolutional neural network auto-encoder (referred to as a convolutional auto-encoder herein) includes an encoder and a decoder that are symmetrically designed, the encoder includes a plurality of convolutional layers and a plurality of pooling (pooling) layers, each of the convolutional layers is connected to one of the pooling layers, and the encoder is configured to obtain a feature vector of sample data; the decoder comprises a plurality of anti-pooling layers and a plurality of anti-convolution layers, wherein each anti-pooling layer is connected with one anti-convolution layer, the decoder is used for reconstructing sample data according to the feature vector, namely the number of the convolution layers, the pooling layers, the anti-pooling layers and the anti-convolution layers is the same, as shown in fig. 3, for the data processing flow based on the convolution self-encoder in the embodiment of the specification, the plurality of convolution layers are arranged to realize the stacking of the convolution layers and further realize the extraction of the complex features of the data, the pooling layers can reduce the dimensionality of the data and reduce the computational complexity, the anti-pooling layers can restore the features of the data according to the restored features of the data to obtain the reconstructed data, and the number of the convolution layers, the pooling layers, the anti-pooling layers and the anti-convolution layers can be determined according to the error range of the reconstructed data and the data, this specification is not intended to be limiting.
In actual work, the training data can be measured data of different depths of an inclinometer drilling at the same position in a certain time period, and in order to ensure enough training data, the training data can also comprise measured data of different depths of the same inclinometer drilling at adjacent positions.
It should be noted that, because the measurement data of inclinometers of different models have a large difference, in order to obtain enough training data, the measurement data of the inclinometer of the same model in the same area can be selected as the training data for training. In addition, when abnormal data is detected, the measured data of the adjacent well logs in the same area or the same area by the same or the same type of inclinometer is required, so that the accuracy of abnormal data detection can be improved, and the false detection of the abnormal data is avoided.
In the well logging, the inclinometer is fixed in the well logging, different measuring heads are arranged at different depths of the inclinometer, each measuring head measures data of a corresponding position, the measured data of the inclinometer can be deep displacement of soil in the horizontal direction and the vertical direction of the well drilling, for example, two groups of deep data in the horizontal XY direction and one group of deep data in the vertical Z direction, different combined data types are selected according to different detection requirements, and the specification does not limit the data types.
As shown in fig. 4, in the training process of the convolutional self-encoder, on the basis of obtaining training data, i.e. original sample data, the convolutional self-encoder may also be obtained by training through the following steps:
s1021: constructing a convolution self-encoder;
s1022: acquiring original sample data acquired by an inclinometer, and inputting the sample data into the convolution self-encoder to obtain reconstructed sample data, wherein all the original sample data are normal data;
s1023: calculating to obtain a loss function according to the reconstructed sample data and the original sample data;
s1024: repeating the training step of the convolution self-encoder until the loss function is converged to obtain a trained convolution self-encoder;
s1025: and calculating to obtain an abnormal threshold value of the data acquired by the inclinometer according to the converged loss function.
The initial convolution self-encoder is constructed through the network structure of the provided convolution self-encoder, original sample data is input into the initial convolution self-encoder, loss function calculation is carried out based on reconstructed original sample data and original sample data, repeated training is carried out for many times, namely the reconstructed original data is used as an input value, the loss function which is gradually converged is obtained through repeated training in a circulating mode until the loss function is not converged any more, the reliable and stable convolution self-encoder which is trained and completed can be obtained, the network structure and the structure parameters are saved at the same time, and when abnormal data detection is carried out, detection on other data to be detected can be achieved through restoration of the saved network structure and the structure parameters.
On the basis of obtaining the trained convolutional self-encoder, an abnormal threshold needs to be obtained, and abnormal data can be judged according to comparison between reconstructed data to be detected and the abnormal threshold, wherein the abnormal threshold can be a function of a loss function, so that the situation that smaller measurement is caused by abnormal judgment based on the loss function only can be avoided, judgment of the abnormal data is affected, for example, normal data can also be judged as abnormal data, and generalization of abnormal data detected by the convolutional self-encoder can be improved through calculation of the abnormal threshold.
On the basis of the above embodiment, as shown in fig. 5, the calculating of the anomaly threshold may further include the following steps:
s2501: setting a super parameter;
s2502: determining a function of the anomaly threshold according to the converged loss function and the super-parameter.
The super parameter may be an empirical parameter, which is manually set according to different detection objects, detection requirements, and inclinometer models, or a plurality of super parameters may be manually set, and the most suitable super parameter is obtained through detection and verification.
In a specific embodiment, it can be set that in a certain logging, the inclinometer is provided with S measuring heads with different depths, each measuring head collects the same soil deep displacement data at the time T, each inclinometer collects two sets of deep displacement data in the X direction and the Y direction at each time, and the X direction is the same as the Y directiont sShowing X-direction deep displacement data, Y, collected by the s-th probe at the t-th timet sAnd the Y-direction deep displacement data acquired by the s-th probe at the t-th time is shown.
During training, X-direction measurement data { X at T moments acquired by S measuring heads with different depths1 1，X2 1，X3 1…XT 1}，{X1 2，X2 2，X3 2…XT 2}，{X1 2，X2 2，X3 2…XT 2}……{X1 S，X2 S，X3 S…XT SAnd Y-direction measurement data Y1 1，Y2 1，Y3 1…YT 1}，{Y1 2，Y2 2，Y3 2…YT 2}，{Y1 2，Y2 2，Y3 2…YT 2}……{Y1 S，Y2 S，Y3 S…YT SInputting the data into an encoder in a manner as shown in FIG. 3, encoding the data through a plurality of convolutional layers and pooling layers to obtain a feature vector Z, and then inputting Z into a decoder, decoding the data through a plurality of anti-convolutional layers and anti-pooling layers to obtain reconstructed X-direction measurement data { M }1 1，M2 1，M3 1…MT 1}，{M1 2，M2 2，M3 2…MT 2}，{M1 2，M2 2，M3 2…MT 2}……{M1 S，M2 S，M3 S…MT SAnd reconstructed Y-direction measurement data { N }1 1，N2 1，N3 1…NT 1}，{N1 2，N2 2，N3 2…NT 2}，{N1 2，N2 2，N3 2…NT 2}……{N1 S，N2 S，N3 S…NT SAnd then, calculating a loss function according to the original sample data and the reconstructed sample data, where in an embodiment of this specification, the loss function formula is:
repeating the above operations to carry out multiple rounds of training until the loss function loss is converged, namely the loss function loss is kept unchanged and is equivalent to being close to the minimum value, simultaneously saving the network structure and related structure parameters to obtain the trained convolutional self-encoder, and calculating an abnormal threshold value which is used as the threshold value for judging whether the data is abnormal or not in the detection of the data to be detectedIn the present embodiment, the calculation formula is threshold ═ β × l, where β is a superparameter and l is the value of the loss function loss that converges last. The anomaly threshold can be finely adjusted according to the detection result of the data to be detected by setting the super parameter β, so that the generalization of the trained convolutional self-encoder model is ensured, and in some other embodiments, the anomaly threshold may also be a loss function loss and other functional relationships of the super parameter β, which are not limited in this specification.
On the basis of the abnormal threshold value determination formula, in order to further provide the accuracy of abnormal data detection, the super parameters can be determined, a plurality of super parameters can be set through user experience, and each super parameter is verified through known data (including normal data and abnormal data) to obtain verification results of different super parameters, so that better super parameters can be determined, and the accuracy of abnormal data detection is improved.
Alternatively, as shown in fig. 6, the step schematic diagram for determining the super parameter may include the following steps:
s2503: setting a plurality of groups of super parameters, and determining a plurality of groups of abnormal threshold values according to the plurality of groups of super parameters;
s2504: acquiring verification data acquired by an inclinometer, wherein the verification data comprises known normal data and known abnormal data;
s2505: obtaining a verification data discrimination value through the trained convolution self-encoder according to the verification data;
s2506: determining a plurality of groups of verification results according to a plurality of groups of abnormal threshold values and the verification data discrimination values;
s2507: determining a plurality of groups of abnormal detection performance indexes according to a plurality of groups of verification results and the verification data;
s2508: and determining a final abnormal threshold according to the super parameter corresponding to the maximum abnormal performance detection index in the multiple groups of abnormal performance detection indexes.
The final converged loss function can be determined by the trained convolutional auto-encoder, and then multiple sets of super parameters can be determined by combining the set super parameters, where the multiple sets of super parameters can be set in an equal interval manner, such as 0.1, 0.2, 0.3, 0.4, 0.5, and the specific setting manner and number are not limited in this specification. The plurality of sets of abnormal thresholds represent different criteria for determining abnormal data, for example, the abnormal data may be determined as normal data or abnormal data at different abnormal thresholds, and the normal data may be determined as normal data or abnormal data at different abnormal thresholds. Therefore, the abnormal threshold with the best judgment effect can be selected through the known verification data of the normal data and the abnormal data, namely, the super parameter with the best judgment effect can be determined.
Further, the verification data may obtain different verification results through different anomaly thresholds, and the anomaly detection performance index may be determined according to the verification results, where the anomaly detection performance index may represent detection performance of the super parameter, and it may be understood that the anomaly detection performance index may represent detection accuracy of different super parameters. In the embodiment of the present specification, the abnormality detection performance index may be expressed by the following formula:
wherein F1 score is an abnormal detection performance index, TP is the number of data that are actually abnormal data and are detected as abnormal data, FP is the number of data that are actually abnormal data and are detected as normal data, FP is the number of data that are actually normal data and are detected as normal data, FN is the number of data that are actually normal data and are detected as abnormal data, and n is a weight parameter, and is set according to the weights of the detection accuracy and the detection coverage, in the embodiment of the present specification, n is 2, that is, the abnormal detection performance index in formula (1) may be:
the F1 score is an index for evaluating the comprehensive abnormal detection effect of the convolutional self-encoder model, the larger the value of the index is, the better the effect of the model is represented, therefore, by selecting the super parameter corresponding to the maximum value in a plurality of groups of abnormal detection performance indexes, the final abnormal threshold value can be determined, in a further embodiment, on the basis of determining the best super parameter in a plurality of groups of super parameters, a plurality of super parameters can be set according to smaller intervals by taking the best super parameter as the center, the verification of verification data is carried out, the better super parameter is further determined, the super parameter with the maximum F1 score is determined more closely by reducing the range of the super parameter for a plurality of times, and the accuracy and the reliability of abnormal data detection can be improved.
After the trained convolution self-encoder and the reliable abnormal threshold value are obtained, abnormal data of the data to be detected can be detected, a data discrimination value is obtained by inputting the data to be detected into the trained convolution self-encoder, and the normality or the abnormality of the data is determined by comparing the data discrimination value with the abnormal threshold value, wherein when the data discrimination value is larger than the abnormal threshold value, the detected data is represented as abnormal data, and when the data discrimination value is not larger than the abnormal threshold value, the detected data is represented as normal data.
The data discrimination value obtaining process comprises the following steps:
inputting the data to be detected into the trained convolution self-encoder to obtain reconstructed data to be detected;
and calculating to obtain a data discrimination value according to the reconstructed data to be detected and the data to be detected.
In a specific embodiment, the same type of inclinometer in another adjacent well can be provided with S measuring heads with different depths, each measuring head collects the same soil deep displacement data at the time T, and each inclinometer at each time collects two groups of deep displacement data, namely X ' direction and Y direction, X ' direction 't sRepresents X-direction deep displacement data, Y ', acquired by an s-th probe at time t't sAnd the Y-direction deep displacement data acquired by the s-th probe at the t-th time is shown. Will be S is notX-direction measurement data { X 'of T moments to be detected collected by same-depth measuring head'1 1，X’2 1，X’3 1…X’T 1}，{X’1 2，X’2 2，X’3 2…X’T 2}，{X’1 2，X’2 2，X’3 2…X’T 2}……{X’1 S，X’2 S，X’3 S…X’T SAnd Y-direction measurement data { Y'1 1，Y’2 1，Y’3 1…Y’T 1}，{Y’1 2，Y’2 2，Y’3 2…Y’T 2}，{Y’1 2，Y’2 2，Y’3 2…Y’T 2}……{Y’1 S，Y’2 S，Y’3 S…Y’T SInputting the data into the trained encoder to encode the data through a plurality of convolutional layers and pooling layers to obtain a feature vector Z 'in the same manner as shown in FIG. 3, inputting Z' into the trained decoder of the offline training part to decode the data through a plurality of deconvolution layers and pooling layers to obtain reconstructed X 'direction measurement data { M'1 1，M’2 1，M’3 1…M’T 1}，{M’1 2，M’2 2，M’3 2…M’T 2}，{M’1 2，M’2 2，M’3 2…M’T 2}……{M’1 S，M’2 S，M’3 S…M’T SAnd reconstructed Y 'direction measurement data { N'1 1，N’2 1，N’3 1…N’T 1}，{N’1 2，N’2 2，N’3 2…N’T 2}，{N’1 2，N’2 2，N’3 2…N’T 2}……{N’1 S，N’2 S，N’3 S…N’T SAnd calculating a data discrimination value according to the data to be measured and the reconstructed data to be measured, wherein the data discrimination value formula is as follows:
if the value of the data discrimination value dis is larger than the abnormal threshold value threshold obtained after model training is completed, the data acquired by the measuring head s of the inclinometer at the time t is abnormal data.
The method for detecting the abnormal data of the inclinometer provided by the embodiment of the specification can be used for obtaining the self-encoder and verifying the obtained abnormal threshold value only by training normal data, so that the accuracy of detecting the abnormal data of the data to be detected can be improved, and meanwhile, a convolutional neural network structure is introduced into the self-encoder, so that not only can the time dependency between data collected by the same measuring head of the inclinometer be captured, but also the space dependency between data collected by different measuring heads of the inclinometer can be captured, and therefore the abnormal data can be detected more accurately and effectively.
On the basis of the above-mentioned inclinometer abnormal data detection, an embodiment of the present specification further provides an inclinometer abnormal data detection apparatus, as shown in fig. 7, the apparatus includes:
the data acquisition module 100 to be measured is used for acquiring data to be measured acquired by an inclinometer;
a data discrimination value calculation module 200, configured to input the data to be measured to a trained convolutional auto-encoder, and calculate to obtain a data discrimination value, where the trained convolutional auto-encoder is obtained by training based on original sample data of the inclinometer;
an abnormal data determining module 300, configured to determine, when the data discrimination value is greater than an abnormal threshold, that the data to be detected corresponding to the data discrimination value is abnormal data, where the abnormal threshold is obtained according to a trained convolutional encoder.
On the basis of the foregoing embodiment, the apparatus may further include a convolutional auto-encoder training module, where the convolutional auto-encoder training module is configured to obtain a trained convolutional auto-encoder according to the original sample data, and the convolutional auto-encoder training module may include:
the convolutional self-encoder building unit is used for building a convolutional self-encoder;
the training unit is used for acquiring original sample data acquired by an inclinometer and inputting the sample data into the convolution self-encoder to obtain reconstructed sample data, wherein all the original sample data are normal data;
the loss function calculation unit is used for calculating to obtain a loss function according to the reconstructed sample data and the original sample data;
a repeating unit, configured to repeat the training step of the convolutional auto-encoder until the loss function converges, to obtain a trained convolutional auto-encoder;
and the abnormal threshold value calculating unit is used for calculating the abnormal threshold value of the data acquired by the inclinometer according to the converged loss function.
The device adapts to the provided method for detecting the abnormal data of the inclinometer, the accuracy of detecting the abnormal data of the data to be detected can be improved, and meanwhile, a convolutional neural network structure is introduced into the self-encoder, so that not only can the time dependency between the data collected by the same measuring head of the inclinometer be captured, but also the space dependency between the data collected by different measuring heads of the inclinometer can be captured, and the abnormal data can be detected more accurately and effectively.
As shown in fig. 8, for a computer device provided for embodiments herein, the computer device 802 may include one or more processors 804, such as one or more Central Processing Units (CPUs), each of which may implement one or more hardware threads. The computer device 802 may also include any memory 806 for storing any kind of information, such as code, settings, data, etc. For example, and without limitation, memory 806 may include any one or more of the following in combination: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent fixed or removable components of computer device 802. In one case, when the processor 804 executes the associated instructions, which are stored in any memory or combination of memories, the computer device 802 can perform any of the operations of the associated instructions. The computer device 802 also includes one or more drive mechanisms 808, such as a hard disk drive mechanism, an optical disk drive mechanism, etc., for interacting with any memory.
Corresponding to the methods in fig. 1-6, the embodiments herein also provide a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, performs the steps of the above-described method.
Embodiments herein also provide computer readable instructions, wherein when executed by a processor, a program thereof causes the processor to perform the method as shown in fig. 1-6.
It should be understood that, in various embodiments herein, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments herein.
It should also be understood that, in the embodiments herein, the term "and/or" is only one kind of association relation describing an associated object, meaning that three kinds of relations may exist. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided herein, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments herein.
In addition, functional units in the embodiments herein may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions in the present disclosure may substantially or partially contribute to the prior art, or all or part of the technical solutions may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the methods described in the embodiments herein. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The principles and embodiments of this document are explained herein using specific examples, which are presented only to aid in understanding the methods and their core concepts; meanwhile, for the general technical personnel in the field, according to the idea of this document, there may be changes in the concrete implementation and the application scope, in summary, this description should not be understood as the limitation of this document.
Claims (10)
1. An inclinometer abnormal data detection method, characterized by comprising:
acquiring data to be detected acquired by an inclinometer;
inputting the data to be detected into a trained convolution self-encoder, and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained by training based on original sample data of the inclinometer, and all the original sample data are normal data;
and when the data discrimination value is larger than an abnormal threshold value, determining that the data to be detected corresponding to the data discrimination value is abnormal data, wherein the abnormal threshold value is obtained according to the trained convolutional encoder.
2. The method of claim 1, wherein the convolutional self-encoder comprises a symmetrically designed encoder and decoder;
the encoder comprises a plurality of convolutional layers and a plurality of pooling layers, wherein each convolutional layer is connected with one pooling layer, and the encoder is used for obtaining a feature vector of sample data;
the decoder comprises a plurality of anti-pooling layers and a plurality of anti-convolution layers, each of the anti-pooling layers being connected to one of the anti-convolution layers, the decoder being configured to reconstruct sample data from the feature vectors.
3. The method of claim 1, wherein the trained convolutional auto-encoder is trained based on original sample data of the inclinometer, further comprising:
constructing a convolution self-encoder;
acquiring original sample data acquired by an inclinometer, and inputting the sample data into the convolution self-encoder to obtain reconstructed sample data;
calculating to obtain a loss function according to the reconstructed sample data and the original sample data;
repeating the training step of the convolution self-encoder until the loss function is converged to obtain a trained convolution self-encoder;
and calculating to obtain an abnormal threshold value of the data acquired by the inclinometer according to the converged loss function.
4. The method of claim 3, wherein calculating an anomaly threshold for inclinometer acquired data according to the converged loss function further comprises:
setting a super parameter;
determining a function of the anomaly threshold according to the converged loss function and the super-parameter.
5. The method of claim 4, wherein said calculating an anomaly threshold for inclinometer collected data according to the converged loss function comprises:
setting a plurality of groups of super parameters, and determining a plurality of groups of abnormal threshold values according to the plurality of groups of super parameters;
acquiring verification data acquired by an inclinometer, wherein the verification data comprises known normal data and known abnormal data;
obtaining a verification data discrimination value through the trained convolution self-encoder according to the verification data;
determining a plurality of groups of verification results according to a plurality of groups of abnormal threshold values and the verification data discrimination values;
determining a plurality of groups of abnormal detection performance indexes according to a plurality of groups of verification results and the verification data;
and determining a final abnormal threshold according to the super parameter corresponding to the maximum abnormal performance detection index in the multiple groups of abnormal performance detection indexes.
6. The method of claim 5, wherein the anomaly detection performance indicator is calculated by the formula:
where F1 score is the index of abnormality detection performance, TP is the number of data that are actually abnormal data and detected as abnormal data, FP is the number of data that are actually abnormal data but detected as normal data, FP is the number of data that are actually normal data and detected as normal data, FN is the number of data that are actually normal data but detected as abnormal data, and n is the weight parameter.
7. The method of claim 1, wherein the inputting the data to be tested into the trained convolutional auto-encoder, and the calculating the data discrimination value further comprises:
inputting the data to be detected into the trained convolution self-encoder to obtain reconstructed data to be detected;
and calculating to obtain a data discrimination value according to the reconstructed data to be detected and the data to be detected.
8. An inclinometer abnormal data detection device, characterized in that the device comprises:
the data acquisition module to be detected is used for acquiring data to be detected acquired by the inclinometer;
the data discrimination value calculation module is used for inputting the data to be detected into a trained convolution self-encoder and calculating to obtain a data discrimination value, wherein the trained convolution self-encoder is obtained based on the original sample data training of the inclinometer;
and the abnormal data determining module is used for determining the data to be detected corresponding to the data discrimination value as abnormal data when the data discrimination value is greater than an abnormal threshold value, wherein the abnormal threshold value is obtained according to the trained convolutional encoder.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method steps of any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores an executable computer program, which when executed by a processor implements the method steps of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011500114.4A CN114648089A (en) | 2020-12-17 | 2020-12-17 | Method, device and equipment for detecting abnormal data of inclinometer and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011500114.4A CN114648089A (en) | 2020-12-17 | 2020-12-17 | Method, device and equipment for detecting abnormal data of inclinometer and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114648089A true CN114648089A (en) | 2022-06-21 |
Family
ID=81990037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011500114.4A Pending CN114648089A (en) | 2020-12-17 | 2020-12-17 | Method, device and equipment for detecting abnormal data of inclinometer and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114648089A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115082803A (en) * | 2022-08-19 | 2022-09-20 | 广州大学 | Cultivated land abandoned land monitoring method and device based on vegetation season change and storage medium |
-
2020
- 2020-12-17 CN CN202011500114.4A patent/CN114648089A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115082803A (en) * | 2022-08-19 | 2022-09-20 | 广州大学 | Cultivated land abandoned land monitoring method and device based on vegetation season change and storage medium |
| CN115082803B (en) * | 2022-08-19 | 2023-02-03 | 广州大学 | Cultivated land abandoned land monitoring method and device based on vegetation season change and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8510242B2 (en) | Artificial neural network models for determining relative permeability of hydrocarbon reservoirs | |
| CN104035431B (en) | The acquisition methods of kernel functional parameter and system for non-linear process monitoring | |
| CN106407649A (en) | Onset time automatic picking method of microseismic signal on the basis of time-recursive neural network | |
| CN112036042B (en) | Power equipment abnormality detection method and system based on variational modal decomposition | |
| CN108039987A (en) | Critical infrastructures fragility assessment method based on multi-layer-coupled relational network | |
| CN112465124B (en) | Twin depth space-time neural network model acquisition/fault diagnosis method and device | |
| CN112149750A (en) | Water supply network pipe burst identification data driving method | |
| CN108805195A (en) | A kind of motor group method for diagnosing faults based on two-value deep-neural-network | |
| CN114648089A (en) | Method, device and equipment for detecting abnormal data of inclinometer and storage medium | |
| CN115015683A (en) | Cable production performance test method, device, equipment and storage medium | |
| CN111460701A (en) | Fault diagnosis model training method and device | |
| Cao et al. | Seismic damage identification of high arch dams based on an unsupervised deep learning approach | |
| CN114330486A (en) | Power system bad data identification method based on improved Wasserstein GAN | |
| CN115184054B (en) | Mechanical equipment semi-supervised fault detection and analysis method, device, terminal and medium | |
| CN115659249B (en) | Abnormality detection method for intelligent station measurement control system | |
| CN116541771A (en) | Unbalanced sample bearing fault diagnosis method based on multi-scale feature fusion | |
| CN111967364B (en) | Composite fault diagnosis method, device, electronic equipment and storage medium | |
| CN114662977A (en) | Method and system for detecting abnormity of motion state of offshore drilling platform and electronic equipment | |
| CN113239075A (en) | Construction data self-checking method and system | |
| CN115878391A (en) | Method and device for detecting disk abnormality | |
| CN116429406B (en) | Construction method and device of fault diagnosis model of large-scale mechanical equipment | |
| CN116957361B (en) | Ship task system health state detection method based on virtual-real combination | |
| CN106246164A (en) | Coal bed gas well level monitoring system based on Fibre Optical Sensor | |
| CN117705178A (en) | Wind power bolt information detection method and device, electronic equipment and storage medium | |
| CN109474445B (en) | Distributed system root fault positioning method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |