CN115905916A - Method for extracting effective information of arch dam deformation monitoring - Google Patents
Method for extracting effective information of arch dam deformation monitoring Download PDFInfo
- Publication number
- CN115905916A CN115905916A CN202211408842.1A CN202211408842A CN115905916A CN 115905916 A CN115905916 A CN 115905916A CN 202211408842 A CN202211408842 A CN 202211408842A CN 115905916 A CN115905916 A CN 115905916A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring data
- deformation
- monitoring
- continuity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000002159 abnormal effect Effects 0.000 claims abstract description 38
- 238000010586 diagram Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 8
- 238000005457 optimization Methods 0.000 claims abstract description 7
- 238000012216 screening Methods 0.000 claims abstract description 4
- 230000000694 effects Effects 0.000 claims description 12
- 230000008030 elimination Effects 0.000 claims description 2
- 238000003379 elimination reaction Methods 0.000 claims description 2
- 238000010009 beating Methods 0.000 claims 1
- 238000000605 extraction Methods 0.000 abstract description 6
- 239000000284 extract Substances 0.000 abstract 1
- 230000009191 jumping Effects 0.000 abstract 1
- 230000007547 defect Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Landscapes
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention discloses an arch dam deformation monitoring effective information extraction method, which comprises the following steps: (1) Performing Gaussian blur and binarization processing on a deformation monitoring data scatter diagram to preliminarily obtain a continuous point set of data; (2) Screening the effective continuous point set by using an optimization method, and identifying an optimal main trend line of the deformation monitoring data; (3) The extraction capability of the convolutional neural network on the detail characteristics of the continuity data is applied, and the identification of the local continuity data is perfected; (4) And acquiring deformation data of a complete sequence from the overall trend line of the monitoring data, judging and eliminating abnormal data values based on the jumping characteristics of the original data, and interpolating missing values based on the statistical characteristics of the original data. The method extracts the main trend line by simulating an artificial vision mechanism, further effectively judges and interpolates the abnormal data values, can be suitable for monitoring data containing various complex characteristics, and has higher data processing efficiency and accuracy.
Description
The technical field is as follows:
the invention relates to the field of dam safety monitoring, in particular to an effective information extraction method for arch dam deformation monitoring.
Technical background:
the method has important significance for evaluating the running state of the dam body and ensuring long-term safety of the structure by analyzing the monitoring data of the deformation of the arch dam. The running conditions of the arch dam are complex, the arch dam is interfered by a plurality of factors such as external environment, load influence, instrument faults and the like, and the collected deformation monitoring information has the problems of incomplete, error, repetition, redundancy, error and the like. Therefore, before analyzing the deformation characteristics of the dam body, it is usually necessary to process abnormal values and missing values in the deformation monitoring information to extract effective information therein.
The existing abnormal value processing method mainly comprises a probability identification method, a criterion evaluation method, a wavelet denoising method and the like based on a data amplitude variation threshold. The methods have satisfactory effects on identifying and processing abnormal values of data with obvious jump, but when the abnormal values are more and the data structure is more complex, all the abnormal values in the data are difficult to be effectively identified, and the accuracy of identifying the abnormal values still needs to be ensured by a manual identification method. The manual identification method firstly needs to identify a main trend line of the deformation monitoring data, then the data deviating from the main trend line is judged as an abnormal value, and the data near the main trend line is effective data. The premise of effective application of the manual identification method is as follows: under the influence of factors such as water pressure, temperature change and the like, the arch dam deformation monitoring data has an obvious continuous change trend, and the change trend can be reflected by a main trend line of the deformation monitoring data; correspondingly, the effective deformation monitoring information should be distributed near the main trend line of the data, and abnormal values and effective information in the monitoring data can be distinguished by identifying the main trend line of the data. Because the manual identification method needs a large amount of manual judgment work, the defects of time and labor consumption, low efficiency, strong subjectivity, difficulty in quantification, difficulty in real-time data processing and the like exist.
Aiming at the defects that the existing processing method for abnormal values of deformation monitoring data of the arch dam cannot realize high-efficiency processing under a complex data structure, and the like, the method for extracting the effective information of deformation monitoring of the arch dam is provided. The automatic identification of the main trend line of the arch dam deformation monitoring data is realized by simulating a human vision mechanism, and an abnormal value processing and missing value interpolation method is provided on the basis, so that the extraction of effective deformation monitoring information is realized.
The invention content is as follows:
the invention mainly aims to solve the defects of the prior art and provide an effective information extraction method for monitoring deformation of an arch dam.
The technical scheme is as follows: the invention relates to an arch dam deformation monitoring effective information extraction method, which comprises the following steps:
(1) Continuous point set identification of deformation monitoring data
Drawing a scatter diagram of the deformation monitoring data, and carrying out Gaussian blur and binarization processing on the scatter diagram; and gradually adjusting the vertical coordinate range and continuity indexes of the data scatter diagram, and carrying out primary identification and elimination on abnormal value points of the monitoring data to obtain a plurality of continuous point sets. In order to obtain a better continuous data identification effect, the scattered point pattern, the Gaussian blur radius and the image binarization threshold value parameters need to be optimized.
(2) Deformation monitoring data main trend line identification
And according to the attribute parameters of the continuous point sets, screening effective sets of the continuous point sets obtained in the last step by using an optimization method, connecting to form a main trend line with the longest effective length, the widest time coverage range and the least sudden jump in the whole range, and further judging the isolated abnormal value points in the whole range.
(3) Local continuity data identification of deformation monitoring data
Aiming at the problem of misjudgment of local abnormal values and continuity data in the monitoring data, the detail characteristics of continuity data distribution in the scatter diagram are extracted through the convolutional neural network, the deformed local continuity monitoring data in the scatter diagram are effectively identified, and a more complete main trend line of the continuity monitoring data is obtained.
(4) Abnormal value and missing value processing of deformation monitoring data
Obtaining deformation data of a complete sequence from a main trend line of continuous deformation monitoring data, and distinguishing and eliminating abnormal values in the original monitoring data by comparing the deformation data with the difference of the jitter characteristics of the original monitoring data; and interpolating the deleted missing data based on the statistical characteristics of the original data so as to extract effective information of the monitoring data. After the abnormal values are eliminated, two indexes of the discrete degree and the number ratio of the missing data need to be checked, and the missing monitoring data can be subjected to effective data interpolation under the condition that certain conditions are met.
Compared with the prior art, the invention has the beneficial effects that: the method identifies the main trend line by simulating an artificial vision mechanism, and effectively identifies and interpolates the abnormal data value by combining an image identification and optimization algorithm, so that the method can be applied to monitoring data containing various complex characteristics compared with the conventional data processing method; compared with a manual method, the method simplifies the data identification process and greatly improves the efficiency and accuracy of data processing.
Description of the drawings:
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram showing the effect of Gaussian blur and binarization processing on a scatter diagram;
FIG. 3 is a schematic diagram of the preliminary identification of outlier points of a scatter plot;
FIG. 4 is a flow chart of the preliminary identification of outlier points in a scatter plot;
FIG. 5 is a diagram of several continuous point set recognition effects;
FIG. 6 is a diagram of a continuum of point set attribute parameters;
FIG. 7 is a flow chart of searching a valid set of consecutive points;
FIG. 8 is a flow chart of identifying local continuity data;
FIG. 9 is a schematic diagram of a neural network training dataset for a local scatter plot;
FIG. 10 is an effect diagram of filling main trend lines with local continuity data;
FIG. 11 is a flow chart of outlier and missing value processing through global trend lines;
fig. 12 is a diagram showing effects of processing an abnormal value and a missing value.
The specific implementation mode is as follows:
the following detailed description of the embodiments of the present invention will be described in conjunction with the accompanying drawings, and the scope of the invention is not limited to the description of the embodiments.
The method for extracting the effective information of monitoring the deformation of the arch dam, disclosed by the invention, has the implementation process shown in figure 1, and comprises the following steps of:
(1) Continuous point set identification of deformation monitoring data
Firstly, identifying a continuous point set for deformation monitoring data by using an image processing technology, comprising the following processes:
1. drawing a deformation monitoring data scatter diagram in a certain scatter sample type;
2. and performing Gaussian blur processing on the scatter diagram, and calculating weighted average of a certain point in the image and the pixel values around the point according to the weight of Gaussian distribution to obtain degraded pixel values at different points.
3. And performing image binarization processing on the Gaussian blur scatter diagram, setting a proper threshold value, and setting a pixel gray value with the gray value smaller than the threshold value as 0 and a pixel gray value larger than or equal to the threshold value as 255.
When image processing is performed, in order to obtain a better continuous data identification effect, optimization of each parameter is required. Through comparison of various schemes, the Gaussian blur radius is set to be 2, the image binarization threshold value is set to be 150, the scatter pattern is an 'x' -shaped scatter of 9 pixels, the operation schematic diagram of the above steps is shown in FIG. 2, and FIG. 2 (a), FIG. 2 (b) and FIG. 2 (c) are respectively the effect diagrams of the original scatter diagram, the Gaussian blur and the binarization processing.
Next, simulating a human vision mechanism, and identifying and eliminating abnormal value points of the data scatter diagram, wherein the link comprises two basic operations: adjusting the vertical coordinate range of the data scatter diagram to ensure that continuous points and abnormal value points in the data points show enough discrimination; and adjusting the continuity index, judging the abnormal value point, and judging that the two points are continuous when the distance between the two adjacent points is less than the continuity index, otherwise, the two points are discontinuous. The process of identifying abnormal values is schematically illustrated in fig. 3, and fig. 3 (a) and 3 (b) respectively perform the continuous point set identification and the adjustment of the ordinate range for the first time and the second time. As shown in fig. 4, the flow of this step is as follows:
1. obtaining the ordinate range (y) of the data scatter diagram 0min ，y 0max )；
2. Calculating the continuity index of the deformation monitoring data: Δ δ = α (y) 0max -y 0min )；
3. Solving a set of consecutive points from the deformation monitoring data according to the gradient change at the data points:
4. removing discontinuous data, and repeating the steps 1 to 3 on the retained continuous point set data;
5. when the continuity index is no longer changed, the abnormal value point identification process is ended.
Wherein: y is 0max 、y 0min Respectively the maximum value and the minimum value of the vertical coordinate range of the scatter diagram, alpha is a coefficient, and delta y m And Δ x m And the difference value of the ordinate and the abscissa of two adjacent data points is obtained.
As shown in fig. 5, after the above process, the deformation monitoring data scatter diagram is divided into several continuous data segments, and most abnormal value points are effectively identified and hidden.
(2) Deformation monitoring data main trend line identification
And screening effective continuous point sets according to the attributes of the continuous point sets, wherein the main trend lines with the longest effective length, the widest time coverage range and the least sudden jump can be formed by connecting the effective continuous point sets in the whole range, and the effective continuous point sets are connected in sequence to obtain the main data trend lines so as to judge the isolated abnormal value points in the whole range.
Setting the ith continuous point set Bi and the attribute parameters thereof as shown in fig. 6, and searching the continuous point set forming the main trend line of the deformation monitoring data by using an optimization algorithm as shown in fig. 7, the process is as follows:
1. sequencing each continuous point set according to the coordinate size of the initial point, and listing all possible combination modes of each continuous point set;
2. randomly generating a combination mode of a continuous point set, and calculating a trend line evaluation function of the combination mode:
wherein: a is 1 And a 2 Is a gain factor; b is the loss coefficient;
as a set of consecutive points B j The initial end abscissa value of (1);
As a set of consecutive points B j The horizontal coordinate value of the termination end of (2);
As a set of consecutive points B j The initial end longitudinal coordinate value of (a);
As a set of consecutive points B j Longitudinal coordinate value of the termination end;
As a set of consecutive points B j+1 The longitudinal coordinate value of the starting end. />
3. Updating the combination mode of the continuous point set through an optimization algorithm, solving a corresponding trend line evaluation function, and obtaining an optimal combination mode after multiple iterations;
4. and sequentially connecting the continuous point sets in the optimal combination mode to identify a main trend line of the data.
(3) Local continuity data identification of deformation monitoring data
Due to the complexity of monitoring the morphological characteristics of the data in the time and space dimensions, the problem of misjudgment of local continuous data and abnormal value points can be caused by carrying out data continuity judgment by using a uniform continuity index. In order to extract all effective monitoring data, the detail features of the continuity data distribution in the scatter diagram are extracted through the convolutional neural network, and the deformed local continuity monitoring data in the scatter diagram is identified, as shown in fig. 8, the method includes the following processes:
1. drawing a scatter diagram of deformation monitoring data of a plurality of measuring points, dividing the scatter diagram into square pictures with the resolution of 96 multiplied by 96, and constructing a data set of a local scatter diagram, as shown in fig. 9;
2. dividing a local scatter diagram into continuity and non-continuity, manually marking, and dividing a local scatter diagram data set into a training set and a test set according to the ratio of 8: 2;
3. constructing a convolutional neural network, inputting a picture data set into the neural network for training and testing, extracting classification features through the convolutional neural network, classifying by using a Softmax classifier, adjusting network parameters through back propagation by using a small batch gradient descent method, and updating and optimizing the weight and parameters of the convolutional neural network;
4. and inputting the monitoring data scatter diagram with the partially continuous points not completely identified into the trained convolutional neural network, and identifying all the continuous data.
5. And solving trend lines of all local continuity data, and filling the trend lines into the main trend lines through a translation operation.
The effects of identifying local continuity data and filling up the main trend lines are shown in fig. 10, where fig. 10 (a) is the set of identified valid continuous points and fig. 10 (b) is the identified overall trend line.
(4) Abnormal value and missing value processing of deformation monitoring data
After obtaining the overall trend line of the deformation monitoring data, it is necessary to further process the abnormal value and the missing value, as shown in fig. 11, the process is as follows:
1. calculating the jitter characteristic value of the ith data relative to the original data:
2. calculating the jitter mean value and standard deviation of n original deformation monitoring data:
3. calculating the jitter characteristic value of the original deformation monitoring data relative to the data on the main trend line:
4. the criterion for judging the abnormality of the deformation monitoring data is as follows:
typically k =2 or 3.
5. Calculating the discrete degree and the proportion of the missing data
If c is satisfied v If the data is more than 36 percent and eta is less than 20 percent, the missing data can be effectively interpolated.
6. And performing missing value interpolation processing on the data with the abnormal values removed by combining an expectation maximization method (EM algorithm) and a data amplification method (DA algorithm).
Wherein: y is i Monitoring data for the ith deformation; y is i-1 The (i-1) th deformation monitoring data; y is i+1 For the (i + 1) th strain monitorMeasuring data; t is t i Is y i Corresponding monitoring time; t is t i-1 Is y i-1 Corresponding monitoring time; t is t i+1 Is y i+1 Corresponding monitoring time. y' 1 ，...，y′ j ，...，y′ m (m < n) is a continuous deformation monitoring data sequence consisting of data on the main trend line, and the monitoring time sequence corresponding to the data sequence measured value is t' 1 ，...，t′ j ，...，t′ m (m＜n)，n m The number of missing data.
Fig. 12 shows the effect of removing abnormal values and interpolating missing data, where fig. 12 (a) shows the monitoring data and the overall trend line, and fig. 12 (b) shows the effect of removing abnormal values and interpolating missing values.
Claims (3)
1. A method for extracting effective information of monitoring deformation of an arch dam is characterized by comprising the following steps:
(1) And (3) identifying continuous point sets of deformation monitoring data: drawing a scatter diagram of the deformation monitoring data, and carrying out Gaussian blur and binarization processing on the scatter diagram; and gradually adjusting the range of the ordinate and continuity indexes of the data scatter diagram, and carrying out primary identification and elimination on abnormal value points of the monitoring data to obtain a plurality of continuous point sets.
(2) And (3) identifying main trend lines of the deformation monitoring data: and according to the attribute parameters of the continuous point sets, screening a plurality of continuous point sets obtained in the last step by using an optimization method, and connecting to form a main trend line with the longest effective length, the widest time coverage range and the least sudden jump in the whole range.
(3) And (3) identifying local continuity data of the deformation monitoring data: aiming at the problem of misjudgment of local abnormal values and continuity data in monitoring data, detail features of continuity data distribution in a scatter diagram are extracted through a convolutional neural network, deformed local continuity monitoring data in the scatter diagram are effectively identified, and a more complete main trend line of the continuity monitoring data is obtained.
(4) Processing abnormal values and missing values of deformation monitoring data: obtaining deformation data of a complete sequence from a main trend line of continuous deformation monitoring data, and distinguishing and eliminating abnormal values in the original monitoring data by comparing the deformation data with the difference of the beating characteristics of the original monitoring data; and interpolating the deleted missing data based on the statistical characteristics of the original data so as to extract effective information in the monitoring data.
2. The method for extracting effective information on monitoring of arch dam deformation as claimed in claim 1, wherein in step (1), in order to obtain better continuous data identification effect, the scatter point pattern, the gaussian blur radius and the image binarization threshold parameter need to be optimized.
3. The method for extracting effective information on monitoring of arch dam deformation according to claim 1, wherein in the step (4), after the abnormal value is removed, two indexes of discrete degree and quantity ratio of the missing data need to be checked, and the missing monitoring data can be subjected to effective data interpolation under the condition that a certain condition is met.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2022106376338 | 2022-06-07 | ||
| CN202210637633 | 2022-06-07 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115905916A true CN115905916A (en) | 2023-04-04 |
Family
ID=86488992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211408842.1A Pending CN115905916A (en) | 2022-06-07 | 2022-11-10 | Method for extracting effective information of arch dam deformation monitoring |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115905916A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595327A (en) * | 2023-07-19 | 2023-08-15 | 水利部交通运输部国家能源局南京水利科学研究院 | Sluice deformation monitoring data preprocessing system and method |
-
2022
- 2022-11-10 CN CN202211408842.1A patent/CN115905916A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595327A (en) * | 2023-07-19 | 2023-08-15 | 水利部交通运输部国家能源局南京水利科学研究院 | Sluice deformation monitoring data preprocessing system and method |
| CN116595327B (en) * | 2023-07-19 | 2023-09-29 | 水利部交通运输部国家能源局南京水利科学研究院 | Sluice deformation monitoring data preprocessing system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108804731B (en) | Time series trend feature extraction method based on important point dual evaluation factors | |
| CN113554629A (en) | Strip steel red rust defect detection method based on artificial intelligence | |
| CN104268505A (en) | Automatic cloth defect point detection and recognition device and method based on machine vision | |
| CN111709931B (en) | Automatic acquisition method and system for strain clamp defect detection and identification report | |
| CN114820625B (en) | Automobile top block defect detection method | |
| CN109726649B (en) | Remote sensing image cloud detection method and system and electronic equipment | |
| CN113989257A (en) | Electric power comprehensive pipe gallery settlement crack identification method based on artificial intelligence technology | |
| CN109859199B (en) | Method for detecting quality of freshwater seedless pearls through SD-OCT image | |
| CN115905916A (en) | Method for extracting effective information of arch dam deformation monitoring | |
| CN117152119A (en) | Profile flaw visual detection method based on image processing | |
| CN111753794A (en) | Fruit quality classification method and device, electronic equipment and readable storage medium | |
| CN114897855A (en) | Method for judging defect type based on X-ray picture gray value distribution | |
| CN111402236A (en) | Hot-rolled strip steel surface defect grading method based on image gray value | |
| CN115908142A (en) | Contact net tiny part damage testing method based on visual recognition | |
| CN113421223B (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN111724376A (en) | Paper defect detection method based on texture feature analysis | |
| CN117455911B (en) | Production test method and system for network cable crystal head | |
| CN112861443B (en) | Advanced learning fault diagnosis method integrated with priori knowledge | |
| CN117409254A (en) | Gastrodia elata objective quality classification evaluation method based on ResNet34 residual neural network | |
| CN117522862A (en) | Image processing method and processing system based on CT image pneumonia recognition | |
| CN112784922A (en) | Extraction and classification method of intelligent cloud medical images | |
| CN116934729A (en) | Bridge crack repair trace and secondary crack classification detection method based on image preprocessing and YOLOv7 | |
| CN113034403B (en) | Working method for denoising medical image data through Gaussian filtering | |
| CN113160166B (en) | Medical image data mining working method through convolutional neural network model | |
| CN116309493A (en) | Method and system for detecting defects of textile products |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |