CN108346144B - Automatic bridge crack monitoring and identifying method based on computer vision - Google Patents

Abstract

The invention discloses a bridge crack automatic monitoring and identifying method based on computer vision. Aiming at the automatic monitoring and identification problems of the bridge cracks, the invention realizes the automatic processing of the whole process of model training, crack identification and result display of the real steel box girder crack image containing complex background interference information. The method is convenient and accurate, and improves the efficiency of bridge crack detection and the accuracy and stability of the detection result.

Description

Automatic bridge crack monitoring and identifying method based on computer vision

Technical Field

The invention relates to the field of civil engineering monitoring, in particular to a bridge crack automatic monitoring and identifying method based on computer vision.

Background

With the rapid development of national economic construction, more and more large-scale infrastructure construction plays an ever-more important role, in particular to a large-scale steel box girder sea-crossing bridge. Due to the fact that the large steel box girder sea-crossing bridge bears complex vehicle load for a long time, fatigue damage accumulation in different degrees is often caused at the welding seam of the steel box girder due to the existence of initial defects, and then fatigue cracks are formed. The fatigue crack can expand along the welding seam direction or towards the components such as the top plate and the diaphragm plate under the coupling action of disaster factors such as long-term effect, fatigue effect and mutation effect of the load, so that the resistance of the bridge structure is attenuated, and a disaster accident can be caused under an extreme condition. Therefore, the bridge management department invests a large amount of manpower, material resources and financial resources to manually inspect the interior of the steel box girder every year. At present, the cracks of the steel box girder are mainly detected by visual inspection personnel or by means of professional equipment, and the cracks are positioned and marked. Such detection is inefficient and inaccurate, takes a long period of time to detect, and relies heavily on the subjective awareness of the inspector.

With the wide application of image processing methods in civil engineering, some crack identification methods based on traditional image processing algorithms such as threshold segmentation and morphological calculation exist at present. However, these methods are often not really effective in the interior of actual bridges. This is because the internal environment of the steel box girder is very complicated, and in the photographed image, such as the structural member boundary, the state of the complicated structural surface (such as anticorrosive paint, magnetic powder, local corrosion, etc.), the uneven lighting condition, etc., all bring great difficulty to the recognition of the fatigue crack in the steel box girder. The biggest influence is that, often the inspection personnel will draw a marking line along the crack trend with the marker pen after discovering the crack to record the section position and the preliminary size measurement result that this crack was located around the crack. In the conventional image processing process, the manual marks and the handwritten handwriting bring huge interference to the identification of real cracks. The fatigue crack has relatively small scale, and the crack width is only 10^-1mm level, which is more easily treated as noise in conventional image processing. In addition, some recognition methods also require the provision of camera internal and external parameters for image capture (such as object distance, image distance, capture angle, etc.), or require additional specialized measurement equipment. Overall, conventional fracture identification methods require excessive manual intervention and are costly.

Disclosure of Invention

Based on the defects, the invention provides the automatic bridge crack monitoring and identifying method based on computer vision, which can be used for offline identification and evaluation of crack images and real-time crack monitoring.

The technology adopted by the invention is as follows: a bridge crack automatic monitoring and identification method based on computer vision comprises the following steps:

step one, training set preparation: an original input image is cut into a sub-unit set of 64 multiplied by 3, samples with a certain proportion are randomly extracted from the sub-unit set, and the number of the samples can be determined according to needs; simultaneously observing the image characteristics of the subunits, and respectively labeling, wherein the number 1 represents a crack unit, the number 2 represents a handwriting unit, the number 3 represents a background unit, after the completion, a newly added subunit set is fused into an original training set, each subunit corresponds to a corresponding label, in order to consider the influence of unbalanced three types of subunit sample numbers, the number of the three types of subunits at the moment is displayed, the subunit number with the minimum number is taken as a reference, the same number of samples are randomly extracted from the other two types of subunit samples, then, each subunit sample is rotated counterclockwise by 90 degrees, 180 degrees and 270 degrees to generate three new samples, the data expansion is completed, each newly expanded subunit sample has the same label as that before the rotation, and the training set is manufactured;

step two, training a crack recognizer based on a depth network model: establishing a deep convolution neural network fusing multi-level features and completing initialization, wherein the size and the function of each layer are shown in table 1, a 64 × 64 × 3 subunit in a training set is used as input, a corresponding label is used as output, parameters in the network are trained, a loss function in the training process is a softmax loss function, an optimization algorithm is a random gradient descent algorithm of driving variables, initial values of learning rate, momentum parameters and weight parameters are used, and the obtained deep network is a crack recognizer;

TABLE 1 size and function of layers in deep networks

Step three, identifying the crack unit image: dividing the image into 64 multiplied by 3 subunits, inputting each subunit into a crack identifier, outputting corresponding label values on an output layer, namely, the subunit with the label value of 1 is a crack unit, the subunit with the label value of 2 is a writing unit, the subunit with the label value of 3 is a background unit, and respectively displaying the identification results of each type;

step four, post-processing output: and (4) carrying out image segmentation on each crack subunit by adopting an optimal entropy threshold method, outputting a binarization crack pixel point identification result, and obtaining the length and width information of the crack according to the binarization crack pixel point.

The invention also has the following technical characteristics:

1. in the second step, the loss function in the training process is softmax loss function, and its formula is as follows:

in the formula, L is a loss function, m is the number of samples, and C is the number of classifications; 1{ y⁽ⁱ⁾J is an index function when the y is⁽ⁱ⁾1 when each sample is classified into the jth class, and 0 if not;

b_jfor the weights and offsets to be updated, x⁽ⁱ⁾For input, λ is a weight parameter.

2. In the second step, the optimization algorithm is a stochastic gradient descent algorithm of the driving amount, and the formula is as follows:

v in the formula_WFor the weight update rate, α_WLearning the rate, η, for weight_WIn order to weight the momentum parameter,

partial differentiation of the weight for the loss function; v is_bTo bias the update rate, α_bTo bias the learning rate, η_bIn order to bias the momentum parameter of the magnetic resonance,

is the partial differential of the loss function versus the bias.

3. And step four, performing image segmentation on each crack subunit by adopting an optimal entropy threshold method, wherein the formula is as follows:

in the formula, p_iRepresenting the proportion of the ith gray level, n_iRepresenting the number of ith gray scale, n representing the total number of pixels, P_iExpressing cumulative probability of ith gradation, H_P(t) represents the foreground entropy, H_B(T) represents background entropy, H (T) represents image total entropy, and T represents a gray division value when the image total entropy takes a maximum value.

4. And step four, after the threshold segmentation is completed, inputting the pixel resolution at the user interaction interface to obtain the information of the real length and width of the crack.

Aiming at the automatic monitoring and identification problems of the bridge cracks, the invention realizes the automatic processing of the whole process of model training, crack identification and result display of the real steel box girder crack image containing complex background interference information. The method is convenient and accurate, and improves the efficiency of bridge crack detection and the accuracy and stability of the detection result. The whole crack identification process is automated, and the manual participation degree in the crack identification process is obviously reduced. The method can meet the real-time data processing requirement of online monitoring and early warning of the crack, namely, the training set is not updated, the acquired image is directly identified, and the result output delay can be as low as a second level. The method improves the automation, the intellectualization, the accuracy and the robustness of bridge crack identification, and provides a solution for the automatic monitoring and identification of civil engineering bridge cracks.

Drawings

FIG. 1 is a flow chart of automatic monitoring and identification of bridge cracks based on computer vision and deep learning

FIG. 2 is a deep convolutional neural network graph incorporating multilevel features;

FIG. 3 is a comparison graph of the recognition results of a long crack unit;

FIG. 4 is a comparison graph of the recognition results of a plurality of crack units;

FIG. 5 is a comparison graph of crack magnified image recognition results;

FIG. 6 is a diagram of a binarization identification result of a long crack;

FIG. 7 is a diagram of a result of binarization identification of a plurality of cracks;

fig. 8 is a graph of the binarization identification result of the crack enlarged image.

Detailed Description

The invention is further illustrated by way of example in the accompanying drawings of the specification:

example 1:

as shown in fig. 1, a method for automatically monitoring and identifying a bridge crack based on computer vision is realized based on an MATLAB environment:

firstly, training set preparation: an original input image is cut into a sub-unit set of 64 multiplied by 3, samples with a certain proportion are randomly extracted from the sub-unit set, and the number of the samples can be determined according to needs; and simultaneously observing the image characteristics of the subunits, and respectively labeling, wherein the number 1 represents a crack unit, the number 2 represents a handwriting unit, the number 3 represents a background unit, after the completion, a newly added subunit set is fused into an original training set, each subunit corresponds to a corresponding label, in order to consider the influence of unbalanced three types of subunit sample numbers, the number of the three types of subunits at the moment is displayed, the same number of samples are randomly extracted from the other two types of subunit samples by taking the subunit number with the minimum number as a reference, then, each subunit sample is rotated by 90 degrees, 180 degrees and 270 degrees counterclockwise to generate three new samples, data expansion is completed, and each newly expanded subunit sample has the same label as that before the rotation. Thus, the training set is finished.

And secondly, training a crack recognizer. And (3) building a deep convolutional neural network which is fused with multilevel features and is shown in fig. 2, and completing initialization, wherein the size and the function of each layer are shown in table 1. And training parameters in the network by taking the subunits of 64 multiplied by 3 in the training set as input and the corresponding labels as output. The loss function in the training process is softmaxloss function (shown in formula 1), and the optimization algorithm is a stochastic gradient descent algorithm with momentum (SGDM, shown in formula 2). And using initial values of the learning rate, the momentum parameter and the weight parameter to obtain a depth network which is the crack identifier.

In the formula, L is a loss function, m is the number of samples, and C is the number of classifications; 1{ y⁽ⁱ⁾J is an index function when the y is⁽ⁱ⁾1 when each sample is classified into the jth class, and 0 if not;

b_jfor the weights and offsets to be updated, x⁽ⁱ⁾As input, λ is a weight parameter;

v in the formula_WFor the weight update rate, α_WLearning the rate, η, for weight_WIn order to weight the momentum parameter,

partial differentiation of the weight for the loss function; v is_bTo bias the update rate, α_bTo bias the learning rate, η_bIn order to bias the momentum parameter of the magnetic resonance,

partial differentiation of the bias for a loss function;

TABLE 1 size and function of layers in deep networks

Layer classification	Height	Width of	Depth of field	Operation of	Height	Width of	Depth of field	Number of	Step pitch
										L0	64	64	3	Convolutional layer 1-1	10	10	3	16	2
L1	28	28	16	Go into layer 1-1	-	-	-	-	-
										L2	28	28	16	Active layer 1-1	-	-	-	-	-
L3	28	28	16	Pooling layer 1-1	2	2	-	-	2
										L4	14	14	16	Convolutional layers 1-2	5	5	16	25	1
L5	10	10	25	Go into layers 1-2	-	-	-	-	-
										L6	10	10	25	Active layer 1-2	-	-	-	-	-
L7	10	10	25	Pooling layers 1-2	2	2	-	-	2
										L8	5	5	25	Full connection layer 1	5	5	25	3	1
L9	14	14	16	Convolutional layer 2-1	7	7	16	25	1
										L10	8	8	25	Go into layer 2-1	-	-	-	-	-
L11	8	8	25	Active layer 2-1	-	-	-	-	-
										L12	8	8	25	Pooling layer 2-1	2	2	-	-	2
L13	4	4	25	Convolutional layer 2-2	4	4	25	36	1
										L14	1	1	36	Go into layer 2-2	-	-	-	-	-
L15	1	1	36	Active layer 2-2	-	-	-	-	-
										L16	1	1	36	Full connection layer 2	1	1	36	3	1
L17	4	4	25	Full connection 3-1	4	4	25	36	1
										L18	1	1	36	Active layer 3-1	-	-	-	-	-
L19	1	1	36	Missing layer	-	-	-	-	-
										L20	1	1	36	Full connection 3-2	1	1	36	3	1
L21	1	1	36	Full connection layer 4	1	1	36	3	1
										L22	1	1	3	Fusion layer	-	-	-	-	-
L23	1	1	3	A classification layer	-	-	-	-	-
										L24	1	1	1	Error layer	-	-	-	-	-

Thirdly, identifying the crack unit image: the image is divided into 64 × 64 × 3 subunits, each subunit is input into the crack identifier, the output layer is a corresponding label value, that is, the subunit with the label value of 1 is a crack unit, the subunit with the label value of 2 is a writing unit, the subunit with the label value of 3 is a background unit, and the recognition results of each type are respectively displayed, as shown in fig. 3-5.

Fourthly, post-processing output: and (3) carrying out image segmentation on each crack subunit by adopting an optimal entropy threshold method, outputting a binarization crack pixel point identification result as shown in a formula 3, and obtaining the length and width information of the crack according to the binarization crack pixel point as shown in figures 6-8. After the threshold segmentation is completed, the information of the real length and width of the crack is obtained by inputting the pixel resolution (mm/pixel).

In the formula, p_iRepresenting the proportion of the ith gray level, n_iRepresenting the number of ith gray scale, n representing the total number of pixels, P_iExpressing cumulative probability of ith gradation, H_P(t) represents the foreground entropy, H_B(T) represents background entropy, H (T) represents image total entropy, and T represents a gray division value when the image total entropy takes a maximum value.

The method is implemented in an MATLAB environment, can be directly suitable for crack images shot by a consumer-grade common camera, does not need special shooting or detection equipment, is high in identification precision, high in speed and low in cost, can be used for offline identification and evaluation, can also be used for real-time monitoring, and improves automation, intelligence, accuracy and robustness of steel box girder fatigue crack identification.

Claims (5)

1. A bridge crack automatic monitoring and identification method based on computer vision is characterized by comprising the following steps:

step one, training set preparation: cutting an original input image into a sub-unit set of 64 multiplied by 3, randomly extracting a certain proportion of samples from the sub-unit set, and determining the number of the samples according to the requirement; simultaneously observing the image characteristics of the subunits, respectively labeling, wherein the number 1 represents a crack unit, the number 2 represents a handwriting unit, the number 3 represents a background unit, after the completion, a newly added subunit set is fused into an original training set, each subunit corresponds to a corresponding label, the number of the three subunits at the time is displayed, the same number of samples are randomly extracted from the rest two types of subunit samples by taking the subunit with the minimum number as a reference, and then, the samples with the minimum number and the same number of samples are randomly extracted from the rest two types of subunit samples and are rotated by 90 degrees, 180 degrees and 270 degrees counterclockwise to generate three new samples, so that data expansion is completed, each newly expanded subunit sample has the same label as that before the rotation, and the training set is completely manufactured;

step two, training a crack recognizer based on a depth network model: establishing a deep convolution neural network fusing multi-level features and completing initialization, wherein the size and the function of each layer are shown in table 1, a 64 × 64 × 3 subunit in a training set is used as input, a corresponding label is used as output, parameters in the network are trained, a loss function in the training process is a softmax loss function, an optimization algorithm is a random gradient descent algorithm of driving variables, initial values of learning rate, momentum parameters and weight parameters are used, and the obtained deep network is a crack recognizer;

TABLE 1 size and function of layers in deep networks

Step three, identifying the crack unit image: dividing the image into 64 multiplied by 3 subunits, inputting each subunit into a crack identifier, outputting corresponding label values on an output layer, namely, the subunit with the label value of 1 is a crack unit, the subunit with the label value of 2 is a writing unit, the subunit with the label value of 3 is a background unit, and respectively displaying the identification results of each type;

step four, post-processing output: and (4) carrying out image segmentation on each crack subunit by adopting an optimal entropy threshold method, outputting a binarization crack pixel point identification result, and obtaining the length and width information of the crack according to the binarization crack pixel point.

2. The method for automatically monitoring and identifying the bridge crack based on the computer vision of claim 1, wherein the method comprises the following steps: step two, the loss function in the training process is a softmaxloss function, and the formula is as follows:

in the formula, L is a loss function, m is the number of samples, and C is the number of classifications; 1{ y⁽ⁱ⁾J is an index function when the y is⁽ⁱ⁾1 when each sample is classified into the jth class, and 0 if not;

b_jfor the weights and offsets to be updated, x⁽ⁱ⁾For input, λ is a weight parameter.

3. The method for automatically monitoring and identifying the bridge crack based on the computer vision of claim 1, wherein the method comprises the following steps: step two, the optimization algorithm is a random gradient descent algorithm of driving quantity, and the formula is as follows:

v in the formula_WFor the weight update rate, α_WLearning the rate, η, for weight_WIn order to weight the momentum parameter,

partial differentiation of the weight for the loss function; v is_bTo bias the update rate, α_bTo bias the learning rate, η_bIn order to bias the momentum parameter of the magnetic resonance,

is the partial differential of the loss function versus the bias.

4. The method for automatically monitoring and identifying the bridge crack based on the computer vision of claim 1, wherein the method comprises the following steps: step four, carrying out image segmentation on each crack subunit by adopting an optimal entropy threshold method, wherein the formula is as follows:

in the formula, p_iRepresenting the proportion of the ith gray level, n_iRepresenting the number of ith gray scale, n representing the total number of pixels, P_iExpressing cumulative probability of ith gradation, H_P(t) represents the foreground entropy, H_B(T) represents background entropy, H (T) represents image total entropy, and T represents a gray division value when the image total entropy takes a maximum value.

5. The method for automatically monitoring and identifying the bridge crack based on the computer vision of claim 1, wherein the method comprises the following steps: and step four, after threshold segmentation is completed, obtaining the information of the real length and width of the crack by inputting the pixel resolution.

Images