CN115905916A - Method for extracting effective information of arch dam deformation monitoring - Google Patents

Abstract

The invention discloses an arch dam deformation monitoring effective information extraction method, which comprises the following steps: (1) Performing Gaussian blur and binarization processing on a deformation monitoring data scatter diagram to preliminarily obtain a continuous point set of data; (2) Screening the effective continuous point set by using an optimization method, and identifying an optimal main trend line of the deformation monitoring data; (3) The extraction capability of the convolutional neural network on the detail characteristics of the continuity data is applied, and the identification of the local continuity data is perfected; (4) And acquiring deformation data of a complete sequence from the overall trend line of the monitoring data, judging and eliminating abnormal data values based on the jumping characteristics of the original data, and interpolating missing values based on the statistical characteristics of the original data. The method extracts the main trend line by simulating an artificial vision mechanism, further effectively judges and interpolates the abnormal data values, can be suitable for monitoring data containing various complex characteristics, and has higher data processing efficiency and accuracy.

Description

A method for extracting effective information from arch dam deformation monitoring

Technical field:

The invention relates to the field of dam safety monitoring, and in particular to a method for extracting effective information from arch dam deformation monitoring.

Technical background:

Analyzing the deformation monitoring data of arch dams is of great significance for evaluating the operating performance of the dam body and ensuring the long-term safety of the structure. The operating conditions of arch dams are complex. Affected by the external environment and loads, as well as interference from many factors such as instrument failures, the collected deformation monitoring information often has problems such as incompleteness, errors, duplications, redundancy, and mistakes. Therefore, before analyzing the deformation characteristics of the dam body, it is usually necessary to process the abnormal values and missing values in the deformation monitoring information and extract the effective information.

The existing outlier processing methods mainly include probability identification method based on data amplitude threshold, criterion judgment method, wavelet denoising and other methods. These methods are satisfactory for the identification and processing of data outliers with obvious jumps, but when there are many outliers and the data structure is complex, it is difficult to effectively identify all outliers in the data, and manual identification methods are still needed to ensure the accuracy of outlier identification. The manual identification method first needs to identify the main trend line of the deformation monitoring data, and then identify the data that deviates from the main trend line as outliers, and the data near the main trend line is valid data. The premise for the effective application of the manual identification method is that under the influence of factors such as water pressure and temperature change, the deformation monitoring data of the arch dam has an obvious continuous change trend, which can be reflected by the main trend line of the deformation monitoring data; accordingly, the effective information of deformation monitoring should be distributed near the main trend line of the data, and the outliers and effective information in the monitoring data can be distinguished by identifying the main trend line of the data. Since the manual identification method requires a lot of manual judgment work, it has the disadvantages of being time-consuming and labor-intensive, low efficiency, strong subjectivity, difficult to quantify, and difficult to process data in real time.

In view of the shortcomings of the current arch dam deformation monitoring data outlier processing method that cannot achieve efficient processing under complex data structures, a method for extracting effective information from arch dam deformation monitoring is proposed. By simulating the human visual mechanism, the main trend line of the arch dam deformation monitoring data is automatically identified. On this basis, an outlier processing and missing value interpolation method is proposed, thereby realizing the extraction of effective deformation monitoring information.

Summary of the invention:

The main purpose of the present invention is to solve the deficiencies of the prior art and provide a method for extracting effective information from arch dam deformation monitoring, which can realize efficient extraction of effective information from arch dam deformation monitoring data by simulating the human visual mechanism to identify the main trend line of monitoring data and process abnormal values.

Technical solution: The effective information extraction method for arch dam deformation monitoring of the present invention comprises the following steps:

(1) Identification of continuous point sets of deformation monitoring data

Draw a scatter plot of deformation monitoring data, perform Gaussian blur and binarization on the scatter plot; gradually adjust the vertical coordinate range and continuity index of the data scatter plot, preliminarily identify and eliminate the abnormal value points of the monitoring data, and obtain several sets of continuous points. In order to obtain better continuous data recognition effect, it is necessary to optimize the scatter pattern, Gaussian blur radius, and image binarization threshold parameters.

(2) Identification of main trend line of deformation monitoring data

According to the attribute parameters of the continuous point set, the optimization method is used to screen the effective set of several continuous point sets obtained in the previous step, and connect them to form the main trend line with the longest effective length, the widest time coverage and the least jumps in the overall range, so as to identify the isolated outlier points in the overall range.

(3) Identification of local continuity data of deformation monitoring data

In order to solve the misjudgment problem of local outliers and continuity data in monitoring data, a convolutional neural network is used to extract the detailed features of the distribution of continuity data in the scatter plot, and the deformed local continuity monitoring data in the scatter plot is effectively identified to obtain a more complete main trend line of the continuity monitoring data.

(4) Processing of abnormal values and missing values in deformation monitoring data

The complete sequence of deformation data is obtained from the main trend line of the continuous deformation monitoring data. By comparing the difference between the original monitoring data and the jumping characteristics, the outliers in the original monitoring data are identified and eliminated; based on the statistical characteristics of the original data, the missing data after elimination are interpolated to extract the effective information of the monitoring data. After the outliers are eliminated, the two indicators of the discrete degree and the proportion of the missing data need to be tested. Only when certain conditions are met can the missing monitoring data be effectively interpolated.

Compared with the prior art, the present invention has the following beneficial effects: the method of the present invention identifies the main trend line by imitating the artificial visual mechanism, and effectively identifies and interpolates data outliers by combining image recognition and optimization algorithms. Compared with conventional data processing methods, it can be applied to monitoring data containing various complex features; compared with manual methods, the method simplifies the data recognition process and greatly improves the efficiency and accuracy of data processing.

Description of the drawings:

Fig. 1 is a flow chart of the method of the present invention;

FIG2 is a diagram showing the effect of Gaussian blurring and binarization of a scatter plot;

FIG3 is a schematic diagram of preliminary identification of outlier points in a scatter plot;

FIG4 is a flow chart for preliminary identification of outlier points in a scatter plot;

FIG5 is a diagram showing the recognition effect of several continuous point sets;

FIG6 is a schematic diagram of attribute parameters of a continuous point set;

FIG7 is a flow chart of searching for a valid continuous point set;

FIG8 is a flow chart of identifying local continuity data;

FIG9 is a schematic diagram of a neural network training data set of a local scatter plot;

Figure 10 is a diagram showing the effect of filling the main trend line with local continuity data;

FIG11 is a flowchart for outlier and missing value processing through the overall trend line;

FIG12 is a diagram showing the effect of processing outliers and missing values.

Specific implementation method:

The specific implementation of the patent of the present invention is described in detail below in conjunction with the accompanying drawings. The protection scope of the patent of the present invention is not limited to the description of this implementation.

The effective information extraction method for arch dam deformation monitoring of the present invention, the implementation process of which is shown in FIG1, comprises the following steps:

(1) Identification of continuous point sets of deformation monitoring data

The first step is to use image processing technology to identify continuous point sets of deformation monitoring data, including the following process:

1. Draw a scatter plot of deformation monitoring data in a certain scatter style;

2. Perform Gaussian blur processing on the scatter plot, and take the weighted average of the pixel values at a certain point in the image and its surroundings according to the weight of the Gaussian distribution to obtain the degraded pixel values at different points.

3. Perform image binarization on the Gaussian blurred scatter plot, set an appropriate threshold, set the grayscale value of pixels with a grayscale less than the threshold to 0, and set the grayscale value of pixels greater than or equal to the threshold to 255.

When performing image processing, in order to obtain better continuous data recognition results, it is necessary to optimize the parameters. After comparing multiple solutions, the Gaussian blur radius is set to 2, the image binarization threshold is set to 150, and the scatter pattern is a 9-pixel "×" scatter. The schematic diagram of the above steps is shown in Figure 2. Figure 2(a), Figure 2(b) and Figure 2(c) are the original scatter plot, Gaussian blur and binarization processing effect diagrams respectively.

The next step is to imitate the human visual mechanism to identify and remove outliers in the data scatter plot. This step includes two basic operations: adjusting the vertical coordinate range of the data scatter plot so that the continuous points and outliers in the data points show sufficient distinction; adjusting the continuity index to distinguish the outliers. When the distance between two adjacent points is less than the continuity index, the two points are judged to be continuous, otherwise they are discontinuous. The schematic diagram of the outlier identification process is shown in Figure 3. Figure 3 (a) and Figure 3 (b) are the initial and second continuous point set identification and vertical coordinate range adjustment, respectively. As shown in Figure 4, the process of this step is as follows:

1. Get the vertical coordinate range of the data scatter plot (y _0min , y _0max );

2. Calculate the continuity index of deformation monitoring data: Δδ = α (y _0max -y _0min );

3. Solve the continuous point set from the deformation monitoring data according to the gradient change at the data point:

4. Eliminate discontinuous data and repeat steps 1 to 3 for the retained continuous point set data;

5. When the continuity index no longer changes, the outlier point identification process ends.

Among them: y _0max and y _0min are the maximum and minimum values of the vertical coordinate range of the scatter plot respectively, α is the coefficient, Δy _m and Δx _m are the differences between the vertical and horizontal coordinates of two adjacent data points.

As shown in Figure 5, after the above process, the deformation monitoring data scatter plot is divided into several continuous data segments, and most of the outlier points are effectively identified and hidden.

(2) Identification of main trend line of deformation monitoring data

For the several sets of continuous points obtained in the previous step, based on the properties of the continuous point sets, with the goal of being able to connect within the overall range to form a main trend line with the longest effective length, the widest time coverage and the least jumps, the effective continuous point sets are screened, and they are connected in sequence to obtain the main trend line of the data, thereby identifying isolated outlier points within the overall range.

Assume that the i-th continuous point set Bi and its attribute parameters are shown in Figure 6. The optimization algorithm is used to search for the continuous point set constituting the main trend line of the deformation monitoring data, as shown in Figure 7. The process is as follows:

1. Sort each set of continuous points in order according to the coordinate size of the starting point, and list all possible combinations of each set of continuous points;

2. Randomly generate a combination of continuous point sets and calculate its trend line evaluation function:

为连续点集合B_j的起始端横坐标值；

为连续点集合B_j的终止端横坐标值；

为连续点集合B_j的起始端纵坐标值；

为连续点集合B_j的终止端纵坐标值；

为连续点集合B_j+1的起始端纵坐标值。Where: _a1 and _a2 are gain coefficients; b is the loss coefficient;

is the horizontal coordinate value of the starting end of the continuous point set B _j ;

is the horizontal coordinate value of the terminal end of the continuous point set B _j ;

is the ordinate value of the starting point of the continuous point set B _j ;

is the ordinate value of the terminal end of the continuous point set B _j ;

is the ordinate value of the starting point of the continuous point set B _j+1 .

3. Update the combination of continuous point sets through optimization algorithms, solve the corresponding trend line evaluation function, and obtain the optimal combination after multiple iterations;

4. Connect the continuous point sets under the optimal combination method in sequence to identify the main trend line of the data.

(3) Identification of local continuity data of deformation monitoring data

Due to the complexity of the morphological characteristics of monitoring data in time and space dimensions, using a unified continuity index to judge data continuity may lead to misjudgment of local continuous data and outliers. In order to extract all valid monitoring data, the detailed features of the continuous data distribution in the scatter plot are extracted through a convolutional neural network, and the deformed local continuity monitoring data in the scatter plot is identified, as shown in Figure 8, including the following process:

1. Draw a scatter plot of deformation monitoring data of multiple measuring points, divide it into square pictures with a resolution of 96×96, and construct a data set of local scatter plots, as shown in Figure 9;

2. The local scatter plots are divided into two categories: continuous and discontinuous, and manually labeled. The local scatter plot data set is divided into a training set and a test set according to a ratio of 8:2;

3. Construct a convolutional neural network, input the image data set into the neural network for training and testing, extract classification features through the convolutional neural network, use the Softmax classifier for classification, use the small batch gradient descent method, adjust the network parameters through back propagation, and update and optimize the weights and parameters of the convolutional neural network;

4. Input the scatter plot of monitoring data where local continuous points are not fully identified into the trained convolutional neural network to identify all continuous data.

5. Find the trend line for all local continuity data and fill it into the main trend line through translation operation.

The effect of identifying local continuous data and filling the main trend line is shown in Figure 10. Figure 10(a) is the identified valid continuous point set, and Figure 10(b) is the identified overall trend line.

(4) Processing of abnormal values and missing values in deformation monitoring data

After obtaining the overall trend line of the deformation monitoring data, it is necessary to further process the outliers and missing values, as shown in Figure 11. The process is as follows:

1. Calculate the jitter characteristic value of the i-th data relative to the original data:

2. Calculate the mean and standard deviation of the runout of n original deformation monitoring data:

3. Calculate the jump characteristic value of the original deformation monitoring data relative to the data on the main trend line:

4. The criteria for determining abnormal deformation monitoring data are:

Generally k=2 or 3.

若满足c_v＞36％、η＜20％时，可对缺失数据进行有效插补。5. Calculate the degree of dispersion and proportion of missing data

If c _v > 36% and η < 20%, the missing data can be effectively interpolated.

6. Combine the expectation maximization method (EM algorithm) and the data augmentation method (DA algorithm) to interpolate missing values in the data after removing outliers.

Where: _yi is the i-th deformation monitoring data; yi _-1 is the i-1-th deformation monitoring data; yi ₊₁ is the i+1-th deformation monitoring data; _ti is the monitoring time corresponding to _yi ; ti _-1 is the monitoring time corresponding to yi _-1 ; ti ₊₁ is the monitoring time corresponding to yi ₊₁ . y′ ₁ , ..., y′ _j , ..., y′ _m (m＜n) is the continuous deformation monitoring data sequence composed of the data on the main trend line. The monitoring time series corresponding to the measured values of this data sequence is t′ ₁ , ..., t′ _j , ..., t′ _m (m＜n), and n _m is the number of missing data.

The effect of removing outliers and interpolating missing data is shown in Figure 12. Figure 12(a) is the monitoring data and the overall trend line, and Figure 12(b) is the effect diagram after removing outliers and interpolating missing values.

Claims (3)

1. A method for extracting effective information from arch dam deformation monitoring, characterized by comprising the following steps: (1) Identification of continuous point sets of deformation monitoring data: Draw a scatter plot of the deformation monitoring data, perform Gaussian blur and binarization on the scatter plot; gradually adjust the vertical coordinate range and continuity index of the data scatter plot, preliminarily identify and eliminate abnormal value points of the monitoring data, and obtain several continuous point sets. (2) Identification of the main trend line of deformation monitoring data: Based on the attribute parameters of the continuous point set, the optimization method is used to screen the effective set of several continuous point sets obtained in the previous step, and connect them to form the main trend line with the longest effective length, the widest time coverage and the least sudden jumps in the overall range. (3) Identification of local continuity data of deformation monitoring data: To address the problem of misjudgment of local outliers and continuity data in monitoring data, a convolutional neural network is used to extract detailed features of the distribution of continuity data in the scatter plot, and the deformation local continuity monitoring data in the scatter plot is effectively identified to obtain a more complete main trend line of the continuity monitoring data. (4) Processing of outliers and missing values in deformation monitoring data: A complete sequence of deformation data is obtained from the main trend line of the continuous deformation monitoring data. By comparing the difference between the data and the original monitoring data, the outliers in the original monitoring data are identified and eliminated. Based on the statistical characteristics of the original data, the missing data after elimination are interpolated to extract effective information from the monitoring data. 2. The effective information extraction method for arch dam deformation monitoring according to claim 1 is characterized in that, in step (1), in order to obtain a better continuity data recognition effect, it is necessary to optimize the scatter point pattern, Gaussian blur radius, and image binarization threshold parameters. 3. The method for extracting effective information from arch dam deformation monitoring according to claim 1 is characterized in that, in step (4), after the outliers are eliminated, the two indicators of the discrete degree and the quantity proportion of the missing data need to be tested, and the missing monitoring data can be effectively interpolated only when certain conditions are met.

Images