CN113436162B - A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots - Google Patents

A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots Download PDF

Info

Publication number
CN113436162B
CN113436162B CN202110697224.2A CN202110697224A CN113436162B CN 113436162 B CN113436162 B CN 113436162B CN 202110697224 A CN202110697224 A CN 202110697224A CN 113436162 B CN113436162 B CN 113436162B
Authority
CN
China
Prior art keywords
image
hydraulic oil
oil pipeline
sub
grayscale
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110697224.2A
Other languages
Chinese (zh)
Other versions
CN113436162A (en
Inventor
周天
肖志伟
吕冰冰
杨睿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Guotian Electronic Technology Co ltd
Original Assignee
Hunan Guotian Electronic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Guotian Electronic Technology Co ltd filed Critical Hunan Guotian Electronic Technology Co ltd
Priority to CN202110697224.2A priority Critical patent/CN113436162B/en
Publication of CN113436162A publication Critical patent/CN113436162A/en
Application granted granted Critical
Publication of CN113436162B publication Critical patent/CN113436162B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G06T7/41—Analysis of texture based on statistical description of texture
    • G06T7/45—Analysis of texture based on statistical description of texture using co-occurrence matrix computation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Molecular Biology (AREA)
  • Image Analysis (AREA)

Abstract

本发明涉及一种缺陷识别的技术领域，公开了一种水下机器人液压油管道表面焊缝缺陷识别方法，包括：获取液压油管道图像，对液压油管道图像进行图像灰度化和灰度拉伸的预处理，得到液压油管道灰度图像；利用图像增强策略对液压油管道灰度图像进行图像增强处理；利用图像分割网络对增强后的图像进行分割，得到若干子图像；利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征；将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。本发明还提供了一种水下机器人液压油管道表面焊缝缺陷识别装置。本发明实现了液压油管道的缺陷识别。

Figure 202110697224

The present invention relates to the technical field of defect identification, and discloses a method for identifying weld seam defects on the surface of hydraulic oil pipelines of underwater robots, including: acquiring images of hydraulic oil pipelines, performing image grayscale and grayscale drawing on the images of hydraulic oil pipelines The gray image of the hydraulic oil pipeline is obtained by the preprocessing of the extension; the gray image of the hydraulic oil pipeline is image enhanced by using the image enhancement strategy; the enhanced image is segmented by the image segmentation network to obtain several sub-images; The feature parameter extraction algorithm of the co-occurrence matrix extracts the feature parameters of the sub-image to obtain the image features of the sub-image; takes the image features of the sub-image as the input of the convolutional neural network, and uses the convolutional neural network to identify the weld seam on the surface of the hydraulic oil pipeline The location of the defect point. The invention also provides an underwater robot hydraulic oil pipeline surface weld seam defect identification device. The invention realizes the defect recognition of the hydraulic oil pipeline.

Figure 202110697224

Description

一种水下机器人液压油管道表面焊缝缺陷识别方法及装置A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots

技术领域technical field

本发明涉及缺陷识别的技术领域，尤其涉及一种水下机器人液压油管道表面焊缝缺陷识别方法及装置。The invention relates to the technical field of defect identification, in particular to a method and device for identifying defects of weld seams on the surface of hydraulic oil pipelines of underwater robots.

背景技术Background technique

为保证机器人本体下潜后安全操作运行，有必要开展液压油管道表面焊缝缺陷识别的检测。常用的检测方法包括目视法和压力测试法。目视法检测效率低，受检测人员主观因素响大，容易存在漏检或错检；压力测试法试验条件准备周期长、耗时费力、检测成本大。液压油管道表面焊缝如果存在不稳定因素，则会造成水下机器人本体停运、沉落等恶性事故。液压管路的性能关系到水下机器人下水后的航行和控制。In order to ensure the safe operation of the robot body after diving, it is necessary to carry out the detection of weld defect identification on the surface of the hydraulic oil pipeline. Commonly used detection methods include visual methods and pressure testing methods. The detection efficiency of the visual method is low, and the subjective factors of the inspected personnel are greatly affected, which is prone to missed or wrong detections; the test condition preparation period of the pressure test method is long, time-consuming and laborious, and the detection cost is high. If there are unstable factors in the weld seam on the surface of the hydraulic oil pipeline, it will cause vicious accidents such as outage and sinking of the underwater robot body. The performance of the hydraulic pipeline is related to the navigation and control of the underwater robot after launching.

鉴于此，如何自动提取焊缝表面缺陷表征的特征参数，精准识别液压油管道表面焊缝缺陷，成为本领域技术人员亟待解决的问题。In view of this, how to automatically extract the characteristic parameters of the surface defect representation of the weld and accurately identify the weld defect on the surface of the hydraulic oil pipeline has become an urgent problem to be solved by those skilled in the art.

发明内容Contents of the invention

本发明提供一种水下机器人液压油管道表面焊缝缺陷识别方法，利用图像增强策略对采集到的液压油管道图像进行图像增强处理，并利用图像分割网络对增强后的图像进行分割，得到若干子图像；利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，并利用卷积神经网络对所提取的特征参数进行处理，识别并确认缺陷点的位置。The invention provides a method for identifying weld defects on the surface of hydraulic oil pipelines of underwater robots, which uses an image enhancement strategy to perform image enhancement processing on the collected images of hydraulic oil pipelines, and uses an image segmentation network to segment the enhanced images to obtain several Sub-image: use the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the feature parameters of the sub-image, and use the convolutional neural network to process the extracted feature parameters to identify and confirm the position of the defect point.

为实现上述目的，本发明提供的一种水下机器人液压油管道表面焊缝缺陷识别方法，包括：In order to achieve the above object, the present invention provides a method for identifying weld defects on the surface of hydraulic oil pipelines of underwater robots, including:

获取液压油管道图像，对液压油管道图像进行图像灰度化和灰度拉伸的预处理，得到液压油管道灰度图像；Obtain the hydraulic oil pipeline image, perform image grayscale and grayscale stretching preprocessing on the hydraulic oil pipeline image, and obtain the hydraulic oil pipeline grayscale image;

利用图像增强策略对液压油管道灰度图像进行图像增强处理；Using the image enhancement strategy to carry out image enhancement processing on the grayscale image of the hydraulic oil pipeline;

利用图像分割网络对增强后的图像进行分割，得到若干子图像；Use the image segmentation network to segment the enhanced image to obtain several sub-images;

利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征；Using the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the feature parameters of the sub-image to obtain the image features of the sub-image;

将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。The image features of the sub-image are used as the input of the convolutional neural network, and the position of the weld defect point on the surface of the hydraulic oil pipeline is identified by the convolutional neural network.

可选地，所述对液压油管道图像进行图像灰度化和灰度拉伸的预处理，包括：Optionally, the preprocessing of performing image grayscale and grayscale stretching on the hydraulic oil pipeline image includes:

对液压油管道图像中每一个像素的三个分量求最大值，并将该最大值设置为该像素点的灰度值，得到液压油管道图像的灰度图，所述灰度化处理的公式为：Calculate the maximum value of the three components of each pixel in the hydraulic oil pipeline image, and set the maximum value as the grayscale value of the pixel point to obtain the grayscale image of the hydraulic oil pipeline image. The formula for the grayscale processing for:

G(i,j)＝max{R(i,j),G(i,j),B(i,j)}G(i,j)=max{R(i,j),G(i,j),B(i,j)}

其中：in:

(i,j)为液压油管道图像中的一个像素点；(i, j) is a pixel in the hydraulic oil pipeline image;

R(i,j),G(i,j),B(i,j)分别为像素点(i,j)在R、G、B三个颜色通道中的值；R(i,j), G(i,j), B(i,j) are the values of the pixel point (i,j) in the three color channels of R, G, and B respectively;

G(i,j)为像素点(i,j)的灰度值；G(i,j) is the gray value of the pixel point (i,j);

对于所述液压油管道图像的灰度图，利用分段线性变换的方式对图像灰度进行拉伸，所述灰度拉伸的公式为：For the grayscale image of the hydraulic oil pipeline image, the image grayscale is stretched by means of piecewise linear transformation, and the formula for the grayscale stretching is:

Figure BDA0003128335950000011
Figure BDA0003128335950000011

其中：in:

f(x,y)为灰度图；f(x,y) is a grayscale image;

MAXf(x,y),MINf(x,y)分别为灰度图的最大灰度值和最小灰度值。MAX f(x,y) and MIN f(x,y) are the maximum gray value and minimum gray value of the grayscale image respectively.

可选地，所述图像增强策略流程为：Optionally, the image enhancement strategy process is:

1)构建高斯滤波核函数矩阵，将高斯滤波核函数矩阵与液压油管道灰度图像进行卷积运算，得到高斯滤波后的液压油管道灰度图像；在本发明一个具体实施例中，所构建的高斯滤波核函数矩阵为：1) Construct the Gaussian filter kernel function matrix, carry out the convolution operation with the Gaussian filter kernel function matrix and the hydraulic oil pipeline grayscale image, obtain the hydraulic oil pipeline grayscale image after the Gaussian filter; In a specific embodiment of the present invention, the constructed The Gaussian filter kernel function matrix of is:

Figure BDA0003128335950000026
Figure BDA0003128335950000026

2)对液压油管道灰度图像进行直方图均衡化处理，其步骤为：2) Perform histogram equalization processing on the grayscale image of the hydraulic oil pipeline, the steps are:

统计液压油管道灰度图像各个灰度级对应的像素数目，得到图像的直方：Count the number of pixels corresponding to each gray level of the gray scale image of the hydraulic oil pipeline to obtain the histogram of the image:

Figure BDA0003128335950000021
Figure BDA0003128335950000021

其中：in:

k＝0，1,…，L-1，表示图像的灰度级；k=0, 1,..., L-1, representing the gray level of the image;

nk表示灰度级为k的像素数；n k represents the number of pixels with a gray level of k;

n表示液压油管道灰度图像的像素总数；n represents the total number of pixels of the hydraulic oil pipeline grayscale image;

计算液压油管道灰度图像的累积直方图：Compute the cumulative histogram of a grayscale image of a hydraulic oil pipeline:

Figure BDA0003128335950000022
k＝0，1，…，L-1
Figure BDA0003128335950000022
k=0,1,...,L-1

对累积直方图进行映射变换处理，将原始累积直方图映射到灰度范围[L0，Lk]：Perform mapping transformation processing on the cumulative histogram, and map the original cumulative histogram to the gray range [L 0 , L k ]:

S＝L0+(Lk-L0)c(k)S＝L 0 +(L k -L 0 )c(k)

统计映射变换后S中各灰度级的像素个数，得到新的图像直方图；Count the number of pixels of each gray level in S after the mapping transformation, and obtain a new image histogram;

3)采用自适应的水下图像颜色矫正算法对直方图均衡化的液压油管道灰度图颜色矫正，所述自适应的水下图像颜色矫正的公式为：3) Using an adaptive underwater image color correction algorithm to correct the color of the histogram equalized hydraulic oil pipeline grayscale image, the formula for the adaptive underwater image color correction is:

Figure BDA0003128335950000023
Figure BDA0003128335950000023

其中：in:

I(R,G,B)表示液压油管道灰度图像在R,G,B三个颜色通道的和；I(R,G,B) represents the sum of the three color channels of R, G, and B in the grayscale image of the hydraulic oil pipeline;

μ表示图像颜色通道的闵可夫斯基距离均值；μ represents the mean value of the Minkowski distance of the image color channel;

α表示液压油管道灰度图像在R,G,B三个颜色通道的最大值；α represents the maximum value of the grayscale image of the hydraulic oil pipeline in the three color channels of R, G, and B;

β为修正参数，其值越接近0，修正后的图像亮度越高，将其设置为0.2；β is a correction parameter, the closer its value is to 0, the higher the brightness of the corrected image, and it is set to 0.2;

4)利用基于图像梯度的水下图像亮度增强算法对颜色矫正后的液压油管道灰度图像进行图像亮度增强处理，所述图像亮度增强的公式为：4) Using an image gradient-based underwater image brightness enhancement algorithm to perform image brightness enhancement processing on the color-corrected hydraulic oil pipeline grayscale image, the formula for image brightness enhancement is:

Figure BDA0003128335950000024
Figure BDA0003128335950000024

Figure BDA0003128335950000025
Figure BDA0003128335950000025

Figure BDA0003128335950000027
Figure BDA0003128335950000027

其中：in:

[Tmin,Tmax]表示图像亮度增强的范围；[T min , T max ] indicates the range of image brightness enhancement;

s(x,y)表示图像像素(x,y)的亮度值；s(x,y) represents the brightness value of the image pixel (x,y);

E(x,y)表示增强后图像像素(x,y)的亮度值；E(x, y) represents the brightness value of the enhanced image pixel (x, y);

i表示梯度方向；i represents the gradient direction;

Figure BDA0003128335950000028
表示在不同梯度方向上的偏导数；
Figure BDA0003128335950000028
Represents the partial derivatives in different gradient directions;

qi表示在不同梯度方向上的目标梯度；q i represents the target gradient in different gradient directions;

wi(x,y)表示在不同梯度方向上梯度残差的影响权重，a表示梯度残差的灵敏度，其值范围在[-0.8,2]之间，在本发明一个具体实施例中，本发明将其取值为0.6；w i (x, y) represents the influence weight of the gradient residual in different gradient directions, a represents the sensitivity of the gradient residual, and its value range is between [-0.8, 2]. In a specific embodiment of the present invention, The present invention takes its value as 0.6;

g(x,y)为亮度约束函数，将增强后的亮度限制在目标范围内。g(x,y) is a brightness constraint function, which limits the enhanced brightness to the target range.

可选地，所述利用图像分割网络对增强后的图像进行分割，包括：Optionally, said utilizing an image segmentation network to segment the enhanced image includes:

所述图像分割网络为MASK R-CNN，该神经网络采用FPN金字塔结构，使用Resnet-101作为卷积网络，输出不同大小的子图像；Described image segmentation network is MASK R-CNN, and this neural network adopts FPN pyramid structure, uses Resnet-101 as convolutional network, the sub-image of output different sizes;

所述图像分割网络的目标函数为：The objective function of the image segmentation network is:

Figure BDA0003128335950000031
Figure BDA0003128335950000031

其中：in:

t′为预测到的图像分割边界；t' is the predicted image segmentation boundary;

t为图像分割边界的二值化结果；t is the binarization result of the image segmentation boundary;

M为输入图像；M is the input image;

R为分割图像；R is the segmented image;

D(t)表示分割边框的距离变换，即不同分割图像之间的距离图；D(t) represents the distance transformation of the segmentation border, that is, the distance map between different segmentation images;

将增强后的图像输入到图像分割网络中，根据预测所得到的分割边界t′进行图像分割，得到若干子图像。The enhanced image is input into the image segmentation network, and the image is segmented according to the predicted segmentation boundary t′ to obtain several sub-images.

可选地，所述利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，包括：Optionally, performing feature parameter extraction processing on sub-images using a feature parameter extraction algorithm based on a gray level co-occurrence matrix, including:

1)对于子图像f(x,y)，其大小为M×N，存在k个灰度级，其中任意两个像素i和像素j之间的距离为

Figure BDA0003128335950000032
且两个像素连线同坐标轴形成的夹角为θ，则对于图像像素i以及图像像素j，其灰度共生矩阵值为Pij(d,θ)，子图像的灰度共生矩阵为：1) For a sub-image f(x,y), its size is M×N, there are k gray levels, and the distance between any two pixels i and pixel j is
Figure BDA0003128335950000032
And the angle formed by the connection line of two pixels with the coordinate axis is θ, then for image pixel i and image pixel j, the gray level co-occurrence matrix value is P ij (d, θ), and the gray level co-occurrence matrix of the sub-image is:

Figure BDA0003128335950000033
Figure BDA0003128335950000033

2)提取灰度共生矩阵中的角二阶矩特征：2) Extract the second-order moment feature of the angle in the gray level co-occurrence matrix:

Figure BDA0003128335950000034
Figure BDA0003128335950000034

所述角二阶矩特征表示共生矩阵中纹理基元排列组合中各个元素的平方和；当ASM值较大时，图像呈现的纹理较粗糙，且能量大；相反则图像纹理细腻，能量小；The second-order moment feature of the angle represents the sum of squares of each element in the arrangement and combination of texture primitives in the co-occurrence matrix; when the ASM value is large, the texture presented by the image is rough and the energy is large; on the contrary, the image texture is delicate and the energy is small;

3)提取灰度共生矩阵中的熵特征：3) Extract the entropy feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000035
Figure BDA0003128335950000035

所述熵特征用于表示二维灰度图像表达的信息量，以及表征图像中脉络纹理的复杂度；当Ent趋近0时，表示灰度图像中几乎不存在纹理信息；如Ent较大，呈现的图像脉络走向更复杂；The entropy feature is used to represent the amount of information expressed by a two-dimensional grayscale image, and to characterize the complexity of vein texture in the image; when Ent approaches 0, it means that there is almost no texture information in the grayscale image; if Ent is larger, The image context presented is more complex;

4)提取灰度共生矩阵中的对比度特征：4) Extract the contrast feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000036
Figure BDA0003128335950000036

所述对比度特征表示灰度图像纹理脉络的清晰度，以及沟纹的深浅程度；对比度大时，图像呈现较清晰，沟纹较深；反之则表明图像较模糊；The contrast feature represents the clarity of the texture of the grayscale image and the depth of the groove; when the contrast is large, the image appears clearer and the groove is deeper; otherwise, it indicates that the image is blurred;

5)提取灰度共生矩阵中的聚类阴影特征：5) Extract the cluster shadow feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000041
Figure BDA0003128335950000041

Figure BDA0003128335950000042
Figure BDA0003128335950000042

Figure BDA0003128335950000043
Figure BDA0003128335950000043

对于每一幅子图像，将从灰度共生矩阵中所提取的特征作为子图像的特征。For each sub-image, the features extracted from the gray level co-occurrence matrix are used as the features of the sub-image.

可选地，所述利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置，包括：Optionally, the identifying the position of the weld defect point on the surface of the hydraulic oil pipeline by using a convolutional neural network includes:

将子图像的图像特征作为卷积神经网络的输入；所述卷积神经网络模型，由输入层、卷积层、池化层、全连接层和激活函数层组成；卷积层用于特征提取，公式为：The image feature of the sub-image is used as the input of the convolutional neural network; the convolutional neural network model is composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an activation function layer; the convolutional layer is used for feature extraction , the formula is:

Figure BDA0003128335950000044
Figure BDA0003128335950000044

其中：in:

Figure BDA0003128335950000045
代表第n层的第i个特征图；
Figure BDA0003128335950000045
Represents the i-th feature map of the n-th layer;

f()表示激活函数，在本发明一个具体实施例中，所采用的激活函数为ReLU激活函数；f() represents an activation function, and in a specific embodiment of the present invention, the activation function used is a ReLU activation function;

M代表输入子图像集合；M represents the set of input sub-images;

Figure BDA0003128335950000046
表示第n-1层的j个特征；
Figure BDA0003128335950000046
Indicates the j features of the n-1th layer;

Figure BDA0003128335950000047
表示第n层第i个特征图与n-1层第j个特征图连接之间的卷积核；
Figure BDA0003128335950000047
Represents the convolution kernel between the i-th feature map of the n-th layer and the j-th feature map of the n-1 layer;

“*”代表卷积运算；"*" stands for convolution operation;

Figure BDA0003128335950000048
代表第n层的第i个特征的偏置；
Figure BDA0003128335950000048
Represents the bias of the i-th feature of the n-th layer;

池化层将输入的特征图进行特征压缩，简化网络复杂度；全连接层将获取的特征映射到原样本标记空间并将输出值送入分类器。The pooling layer compresses the input feature map to simplify the network complexity; the fully connected layer maps the acquired features to the original sample label space and sends the output value to the classifier.

在传播过程中，输入信息从输入层经隐含单元层逐层处理，并传向输出层，每一层神经元的输出只影响下一层神经元的输入；如果输出层不能得到期望的输出，则进入反向传播，反向传播通过误差反传算法，对网络的权值与阈值进行学习修正；In the propagation process, the input information is processed layer by layer from the input layer through the hidden unit layer, and then transmitted to the output layer. The output of neurons in each layer only affects the input of neurons in the next layer; if the output layer cannot get the desired output , it enters backpropagation, and backpropagation uses the error backpropagation algorithm to learn and correct the weights and thresholds of the network;

当网络的均方差小于给定值时，进入NMS算法运算，则确定缺陷点位置，即完成液压油管道表面缺陷识别。When the mean square error of the network is less than a given value, enter the NMS algorithm operation, then determine the position of the defect point, that is, complete the identification of the surface defect of the hydraulic oil pipeline.

此外，为实现上述目的，本发明还提供一种水下机器人液压油管道表面焊缝缺陷识别装置，所述装置包括：In addition, in order to achieve the above object, the present invention also provides a device for identifying weld defects on the surface of hydraulic oil pipelines of underwater robots, said device comprising:

图像获取装置，用于获取液压油管道图像；An image acquisition device, configured to acquire images of hydraulic oil pipelines;

数据处理器，用于对液压油管道图像进行图像灰度化和灰度拉伸的预处理，利用图像增强策略对液压油管道灰度图像进行图像增强处理；利用图像分割网络对增强后的图像进行分割，得到若干子图像；The data processor is used to perform image grayscale and grayscale stretching preprocessing on the hydraulic oil pipeline image, and use the image enhancement strategy to perform image enhancement processing on the hydraulic oil pipeline grayscale image; use the image segmentation network to process the enhanced image Carry out segmentation to obtain several sub-images;

缺陷识别装置，用于利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征；将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。The defect identification device is used to extract the characteristic parameters of the sub-image by using the characteristic parameter extraction algorithm based on the gray level co-occurrence matrix to obtain the image characteristics of the sub-image; the image characteristics of the sub-image are used as the input of the convolutional neural network, and the convolution A neural network identifies the position of weld defect points on the surface of hydraulic oil pipelines.

此外，为实现上述目的，本发明还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有缺陷识别程序指令，所述缺陷识别程序指令可被一个或者多个处理器执行，以实现如上所述的水下机器人液压油管道表面焊缝缺陷识别的实现方法的步骤。In addition, in order to achieve the above object, the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores defect identification program instructions, and the defect identification program instructions can be executed by one or more processors, In order to realize the steps of the implementation method of the above-mentioned underwater robot hydraulic oil pipeline surface weld defect identification.

相对于现有技术，本发明提出一种水下机器人液压油管道表面焊缝缺陷识别方法，该技术具有以下优势：Compared with the prior art, the present invention proposes a method for identifying weld defects on the surface of hydraulic oil pipelines of underwater robots, which has the following advantages:

首先，本发明提出一种图像增强策略，针对水下图像的红色色调将会减弱，以蓝绿色调为主的问题，因此本发明采用自适应的水下图像颜色矫正算法对液压油管道灰度图像进行颜色矫正，所述自适应的水下图像颜色矫正的公式为：First of all, the present invention proposes an image enhancement strategy, aiming at the problem that the red tone of the underwater image will be weakened, and the blue-green tone is the main problem. Therefore, the present invention uses an adaptive underwater image color correction algorithm to correct the grayscale of the hydraulic oil pipeline The image is color-corrected, and the formula of the adaptive underwater image color correction is:

Figure BDA0003128335950000051
Figure BDA0003128335950000051

其中：I(R,G,B)表示液压油管道灰度图像在R,G,B三个颜色通道的和；μ表示图像颜色通道的闵可夫斯基距离均值，由于红色通道与其他颜色通道之间的距离较大，因此对于水下图像μ＞1，

Figure BDA0003128335950000052
α表示液压油管道灰度图像在R,G,B三个颜色通道的最大值；β为修正参数，其值越接近0，修正后的图像亮度越高，将其设置为0.2，相较于传统算法，本发明所述算法从图像在R,G,B三个颜色通道的和进行颜色矫正，对于红色通道与其他颜色通道之间的距离较大的情况，将会对其他颜色通道的颜色值进行减少，并利用修正参数对图像进行整体的色彩亮度增强。同时本发明利用基于图像梯度的水下图像亮度增强算法对颜色矫正后的液压油管道灰度图像进行图像亮度增强处理，所述图像亮度增强的公式为：Among them: I(R,G,B) represents the sum of the grayscale image of the hydraulic oil pipeline in the three color channels of R, G, and B; μ represents the mean value of the Minkowski distance of the image color channel, because the red channel and other color channels The distance between is large, so for the underwater image μ>1,
Figure BDA0003128335950000052
α represents the maximum value of the grayscale image of the hydraulic oil pipeline in the three color channels of R, G, and B; β is the correction parameter, the closer its value is to 0, the higher the brightness of the corrected image, and it is set to 0.2, compared with Traditional algorithm, the algorithm described in the present invention carries out color correction from the sum of three color channels of R, G, and B in the image, and for the situation that the distance between the red channel and other color channels is larger, the color of other color channels will be corrected. The value is reduced, and the correction parameter is used to enhance the overall color brightness of the image. At the same time, the present invention utilizes an underwater image brightness enhancement algorithm based on image gradients to perform image brightness enhancement processing on the color-corrected hydraulic oil pipeline grayscale image, and the formula for image brightness enhancement is:

Figure BDA0003128335950000053
Figure BDA0003128335950000053

Figure BDA0003128335950000054
Figure BDA0003128335950000054

Figure BDA0003128335950000059
Figure BDA0003128335950000059

其中：[Tmin,Tmax]表示图像亮度增强的范围；s(x,y)表示图像像素(x,y)的亮度值；E(x,y)表示增强后图像像素(x,y)的亮度值；i表示梯度方向；

Figure BDA00031283359500000510
表示在不同梯度方向上的偏导数；qi表示在不同梯度方向上的目标梯度；wi(x,y)表示在不同梯度方向上梯度残差的影响权重，a表示梯度残差的灵敏度，其值范围在[-0.8,2]之间；g(x,y)为亮度约束函数，将增强后的亮度限制在目标范围内。Among them: [T min , T max ] represents the range of image brightness enhancement; s(x, y) represents the brightness value of image pixel (x, y); E(x, y) represents the enhanced image pixel (x, y) The brightness value of ; i represents the gradient direction;
Figure BDA00031283359500000510
Represents the partial derivative in different gradient directions; q i represents the target gradient in different gradient directions; w i (x, y) represents the influence weight of the gradient residual in different gradient directions, a represents the sensitivity of the gradient residual, Its value range is between [-0.8, 2]; g(x, y) is a brightness constraint function, which limits the enhanced brightness to the target range.

同时，本发明利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，对于子图像f(x,y)，其大小为M×N，存在k个灰度级，其中任意两个像素i和像素j之间的距离为

Figure BDA0003128335950000055
且两个像素连线同坐标轴形成的夹角为θ，则对于图像像素i以及图像像素j，其灰度共生矩阵值为Pij(d,θ)，子图像的灰度共生矩阵为：At the same time, the present invention uses the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the feature parameters of the sub-image. For the sub-image f(x, y), its size is M×N, and there are k gray levels, among which any The distance between two pixels i and pixel j is
Figure BDA0003128335950000055
And the angle formed by the connection line of two pixels with the coordinate axis is θ, then for image pixel i and image pixel j, the gray level co-occurrence matrix value is P ij (d, θ), and the gray level co-occurrence matrix of the sub-image is:

Figure BDA0003128335950000056
Figure BDA0003128335950000056

提取灰度共生矩阵中的角二阶矩特征：Extract the angular second moment features in the gray level co-occurrence matrix:

Figure BDA0003128335950000057
Figure BDA0003128335950000057

所述角二阶矩特征表示共生矩阵中纹理基元排列组合中各个元素的平方和；当ASM值较大时，图像呈现的纹理较粗糙，且能量大；相反则图像纹理细腻，能量小；提取灰度共生矩阵中的熵特征：The second-order moment feature of the angle represents the sum of squares of each element in the arrangement and combination of texture primitives in the co-occurrence matrix; when the ASM value is large, the texture presented by the image is rough and the energy is large; on the contrary, the image texture is delicate and the energy is small; Extract the entropy features in the gray level co-occurrence matrix:

Figure BDA0003128335950000058
Figure BDA0003128335950000058

所述熵特征用于表示二维灰度图像表达的信息量，以及表征图像中脉络纹理的复杂度；当Ent趋近0时，表示灰度图像中几乎不存在纹理信息；如Ent较大，呈现的图像脉络走向更复杂；提取灰度共生矩阵中的对比度特征：The entropy feature is used to represent the amount of information expressed by a two-dimensional grayscale image, and to characterize the complexity of vein texture in the image; when Ent approaches 0, it means that there is almost no texture information in the grayscale image; if Ent is larger, The image context presented is more complex; the contrast features in the gray-level co-occurrence matrix are extracted:

Figure BDA0003128335950000061
Figure BDA0003128335950000061

所述对比度特征表示灰度图像纹理脉络的清晰度，以及沟纹的深浅程度；对比度大时，图像呈现较清晰，沟纹较深；反之则表明图像较模糊；提取灰度共生矩阵中的聚类阴影特征：The contrast feature represents the clarity of the texture veins of the grayscale image and the depth of the grooves; when the contrast is large, the image appears clearer and the grooves are deeper; otherwise, it indicates that the image is blurred; Class shadow features:

Figure BDA0003128335950000062
Figure BDA0003128335950000062

Figure BDA0003128335950000063
Figure BDA0003128335950000063

Figure BDA0003128335950000064
Figure BDA0003128335950000064

对于每一幅子图像，将从灰度共生矩阵中所提取的特征作为子图像的特征；根据所提取到的表示图像灰度分布均匀性及纹理粗细程度的特征，利用卷积神经网络进行液压油管道表面焊缝的缺陷识别。For each sub-image, the features extracted from the gray-level co-occurrence matrix are used as the features of the sub-image; according to the extracted features that represent the uniformity of the image gray distribution and the thickness of the texture, the convolutional neural network is used to perform hydraulic Defect identification of oil pipeline surface welds.

附图说明Description of drawings

图1为本发明一实施例提供的一种水下机器人液压油管道表面焊缝缺陷识别方法的流程示意图；Fig. 1 is a schematic flow chart of a method for identifying weld defects on the surface of a hydraulic oil pipeline of an underwater robot provided by an embodiment of the present invention;

图2为本发明一实施例提供的一种水下机器人液压油管道表面焊缝缺陷识别装置的结构示意图；Fig. 2 is a structural schematic diagram of an underwater robot hydraulic oil pipeline surface weld defect identification device provided by an embodiment of the present invention;

图3为本发明一实施例提供的水下机器人液压油管道表面焊缝缺陷识别装置的装置示意图；Fig. 3 is a device schematic diagram of an underwater robot hydraulic oil pipeline surface weld defect identification device provided by an embodiment of the present invention;

本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。The realization of the purpose of the present invention, functional characteristics and advantages will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

具体实施方式detailed description

应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

利用图像增强策略对采集到的液压油管道图像进行图像增强处理，并利用图像分割网络对增强后的图像进行分割，得到若干子图像；利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，并利用卷积神经网络对所提取的特征参数进行处理，识别并确认缺陷点的位置。参照图1所示，为本发明一实施例提供的水下机器人液压油管道表面焊缝缺陷识别方法示意图。Using the image enhancement strategy to carry out image enhancement processing on the collected hydraulic oil pipeline image, and use the image segmentation network to segment the enhanced image to obtain several sub-images; use the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the sub-images Feature parameter extraction processing, and use the convolutional neural network to process the extracted feature parameters to identify and confirm the position of the defect point. Referring to FIG. 1 , it is a schematic diagram of a method for identifying weld defects on the surface of a hydraulic oil pipeline of an underwater robot provided by an embodiment of the present invention.

在本实施例中，水下机器人液压油管道表面焊缝缺陷识别方法包括：In this embodiment, the method for identifying weld defects on the surface of hydraulic oil pipelines of underwater robots includes:

S1、获取液压油管道图像，对液压油管道图像进行图像灰度化和灰度拉伸的预处理，得到液压油管道灰度图像。S1. Obtain an image of the hydraulic oil pipeline, and perform image grayscale and grayscale stretching preprocessing on the image of the hydraulic oil pipeline to obtain a grayscale image of the hydraulic oil pipeline.

首先，本发明利用水下机器人的视觉检测装置获取液压油管道图像，并对液压油管道图像进行图像灰度化和灰度拉伸的预处理，所述图像灰度化和灰度拉伸的流程为：First, the present invention utilizes the visual detection device of the underwater robot to acquire the image of the hydraulic oil pipeline, and performs image grayscale and grayscale stretching preprocessing on the image of the hydraulic oil pipeline. The process is:

对液压油管道图像中每一个像素的三个分量求最大值，并将该最大值设置为该像素点的灰度值，得到液压油管道图像的灰度图，所述灰度化处理的公式为：Calculate the maximum value of the three components of each pixel in the hydraulic oil pipeline image, and set the maximum value as the grayscale value of the pixel point to obtain the grayscale image of the hydraulic oil pipeline image. The formula for the grayscale processing for:

G(i,j)＝max{R(i,j),G(i,j),B(i,j)}G(i,j)=max{R(i,j),G(i,j),B(i,j)}

其中：in:

(i,j)为液压油管道图像中的一个像素点；(i, j) is a pixel in the hydraulic oil pipeline image;

R(i,j),G(i,j),B(i,j)分别为像素点(i,j)在R、G、B三个颜色通道中的值；R(i,j), G(i,j), B(i,j) are the values of the pixel point (i,j) in the three color channels of R, G, and B respectively;

G(i,j)为像素点(i,j)的灰度值；G(i,j) is the gray value of the pixel point (i,j);

对于所述液压油管道图像的灰度图，利用分段线性变换的方式对图像灰度进行拉伸，所述灰度拉伸的公式为：For the grayscale image of the hydraulic oil pipeline image, the image grayscale is stretched by means of piecewise linear transformation, and the formula for the grayscale stretching is:

Figure BDA0003128335950000071
Figure BDA0003128335950000071

其中：in:

f(x,y)为灰度图；f(x,y) is a grayscale image;

MAXf(x,y),MINf(x,y)分别为灰度图的最大灰度值和最小灰度值。MAX f(x,y) and MIN f(x,y) are the maximum gray value and minimum gray value of the grayscale image respectively.

S2、利用图像增强策略对液压油管道灰度图像进行图像增强处理。S2. Using an image enhancement strategy to perform image enhancement processing on the grayscale image of the hydraulic oil pipeline.

进一步地，本发明利用图像增强策略对液压油管道灰度图像进行图像增强处理，所述图像增强策略流程为：Further, the present invention uses an image enhancement strategy to perform image enhancement processing on the grayscale image of the hydraulic oil pipeline, and the flow of the image enhancement strategy is as follows:

1)构建高斯滤波核函数矩阵，将高斯滤波核函数矩阵与液压油管道灰度图像进行卷积运算，得到高斯滤波后的液压油管道灰度图像；在本发明一个具体实施例中，所构建的高斯滤波核函数矩阵为：1) Construct the Gaussian filter kernel function matrix, carry out the convolution operation with the Gaussian filter kernel function matrix and the hydraulic oil pipeline grayscale image, obtain the hydraulic oil pipeline grayscale image after the Gaussian filter; In a specific embodiment of the present invention, the constructed The Gaussian filter kernel function matrix of is:

Figure BDA0003128335950000077
Figure BDA0003128335950000077

2)对液压油管道灰度图像进行直方图均衡化处理，其步骤为：2) Perform histogram equalization processing on the grayscale image of the hydraulic oil pipeline, the steps are:

统计液压油管道灰度图像各个灰度级对应的像素数目，得到图像的直方：Count the number of pixels corresponding to each gray level of the gray scale image of the hydraulic oil pipeline to obtain the histogram of the image:

Figure BDA0003128335950000072
Figure BDA0003128335950000072

其中：in:

k＝0，1，…,L-1，表示图像的灰度级；k=0, 1, ..., L-1, representing the gray level of the image;

nk表示灰度级为k的像素数；n k represents the number of pixels with a gray level of k;

n表示液压油管道灰度图像的像素总数；n represents the total number of pixels of the hydraulic oil pipeline grayscale image;

计算液压油管道灰度图像的累积直方图：Compute the cumulative histogram of a grayscale image of a hydraulic oil pipeline:

Figure BDA0003128335950000073
Figure BDA0003128335950000073

对累积直方图进行映射变换处理，将原始累积直方图映射到灰度范围[L0，Lk]：Perform mapping transformation processing on the cumulative histogram, and map the original cumulative histogram to the gray range [L 0 , L k ]:

S＝L0+(Lk-L0)c(k)S＝L 0 +(L k -L 0 )c(k)

统计映射变换后S中各灰度级的像素个数，得到新的图像直方图；Count the number of pixels of each gray level in S after the mapping transformation, and obtain a new image histogram;

3)采用自适应的水下图像颜色矫正算法对直方图均衡化的液压油管道灰度图像进行颜色矫正，所述自适应的水下图像颜色矫正的公式为：3) Using an adaptive underwater image color correction algorithm to correct the color of the histogram-equalized hydraulic oil pipeline grayscale image, the formula for the adaptive underwater image color correction is:

Figure BDA0003128335950000074
Figure BDA0003128335950000074

其中：in:

I(R,G，B)表示液压油管道灰度图像在R,G,B三个颜色通道的和；I(R, G, B) represents the sum of the three color channels of R, G, and B in the grayscale image of the hydraulic oil pipeline;

μ表示图像颜色通道的闵可夫斯基距离均值；μ represents the mean value of the Minkowski distance of the image color channel;

α表示液压油管道灰度图像在R,G,B三个颜色通道的最大值；α represents the maximum value of the grayscale image of the hydraulic oil pipeline in the three color channels of R, G, and B;

β为修正参数，其值越接近0，修正后的图像亮度越高，将其设置为0.2；β is a correction parameter, the closer its value is to 0, the higher the brightness of the corrected image, and it is set to 0.2;

4)利用基于图像梯度的水下图像亮度增强算法对颜色矫正后的液压油管道灰度图像进行图像亮度增强处理，所述图像亮度增强的公式为：4) Using an image gradient-based underwater image brightness enhancement algorithm to perform image brightness enhancement processing on the color-corrected hydraulic oil pipeline grayscale image, the formula for image brightness enhancement is:

Figure BDA0003128335950000075
Figure BDA0003128335950000075

Figure BDA0003128335950000076
Figure BDA0003128335950000076

Figure BDA0003128335950000078
Figure BDA0003128335950000078

其中：in:

[Tmin，Tmax]表示图像亮度增强的范围；[T min , T max ] represents the range of image brightness enhancement;

s(x,y)表示图像像素(x，y)的亮度值；s(x, y) represents the brightness value of the image pixel (x, y);

E(x,y)表示增强后图像像素(x,y)的亮度值；E(x, y) represents the brightness value of the enhanced image pixel (x, y);

i表示梯度方向；i represents the gradient direction;

Figure BDA0003128335950000086
表示在不同梯度方向上的偏导数；
Figure BDA0003128335950000086
Represents the partial derivatives in different gradient directions;

qi表示在不同梯度方向上的目标梯度；q i represents the target gradient in different gradient directions;

wi(x,y)表示在不同梯度方向上梯度残差的影响权重，a表示梯度残差的灵敏度，其值范围在[-0.8,2]之间，在本发明一个具体实施例中，本发明将其取值为0.6；w i (x, y) represents the influence weight of the gradient residual in different gradient directions, a represents the sensitivity of the gradient residual, and its value range is between [-0.8, 2]. In a specific embodiment of the present invention, The present invention takes its value as 0.6;

g(x,y)为亮度约束函数，将增强后的亮度限制在目标范围内。g(x,y) is a brightness constraint function, which limits the enhanced brightness to the target range.

S3、利用图像分割网络对增强后的图像进行分割，得到若干子图像。S3. Using the image segmentation network to segment the enhanced image to obtain several sub-images.

进一步地，本发明将增强后的图像输入到图像分割网络中，利用图像分割网络对增强后的图像进行分割，所述图像分割网络为MASK R-CNN，该神经网络采用FPN金字塔结构，使用Resnet-101作为卷积网络，输出不同大小的子图像；Further, the present invention inputs the enhanced image into the image segmentation network, utilizes the image segmentation network to segment the enhanced image, the image segmentation network is MASK R-CNN, the neural network adopts FPN pyramid structure, and uses Resnet -101 is used as a convolutional network to output sub-images of different sizes;

所述图像分割网络的目标函数为：The objective function of the image segmentation network is:

Figure BDA0003128335950000081
Figure BDA0003128335950000081

其中：in:

t′为预测到的图像分割边界；t' is the predicted image segmentation boundary;

t为图像分割边界的二值化结果；t is the binarization result of the image segmentation boundary;

M为输入图像；M is the input image;

R为分割图像；R is the segmented image;

D(t)表示分割边框的距离变换，即不同分割图像之间的距离图；D(t) represents the distance transformation of the segmentation border, that is, the distance map between different segmentation images;

将增强后的图像输入到图像分割网络中，根据预测所得到的分割边界t′进行图像分割，得到若干子图像。The enhanced image is input into the image segmentation network, and the image is segmented according to the predicted segmentation boundary t′ to obtain several sub-images.

S4、利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征。S4. Using a gray level co-occurrence matrix-based feature parameter extraction algorithm to perform feature parameter extraction processing on the sub-images to obtain image features of the sub-images.

进一步地，本发明利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，所述特征参数提取流程为：Further, the present invention uses a feature parameter extraction algorithm based on a gray level co-occurrence matrix to perform feature parameter extraction processing on sub-images, and the feature parameter extraction process is as follows:

1)对于子图像f(x,y)，其大小为M×N，存在k个灰度级，其中任意两个像素i和像素j之间的距离为

Figure BDA0003128335950000082
且两个像素连线同坐标轴形成的夹角为θ，则对于图像像素i以及图像像素j，其灰度共生矩阵值为Pij(d,θ)，子图像的灰度共生矩阵为：1) For a sub-image f(x,y), its size is M×N, there are k gray levels, and the distance between any two pixels i and pixel j is
Figure BDA0003128335950000082
And the angle formed by the connection line of two pixels with the coordinate axis is θ, then for image pixel i and image pixel j, the gray level co-occurrence matrix value is P ij (d, θ), and the gray level co-occurrence matrix of the sub-image is:

Figure BDA0003128335950000083
Figure BDA0003128335950000083

2)提取灰度共生矩阵中的角二阶矩特征：2) Extract the second-order moment feature of the angle in the gray level co-occurrence matrix:

Figure BDA0003128335950000084
Figure BDA0003128335950000084

所述角二阶矩特征表示共生矩阵中纹理基元排列组合中各个元素的平方和；当ASM值较大时，图像呈现的纹理较粗糙，且能量大；相反则图像纹理细腻，能量小；The second-order moment feature of the angle represents the sum of squares of each element in the arrangement and combination of texture primitives in the co-occurrence matrix; when the ASM value is large, the texture presented by the image is rough and the energy is large; on the contrary, the image texture is delicate and the energy is small;

3)提取灰度共生矩阵中的熵特征：3) Extract the entropy feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000085
Figure BDA0003128335950000085

所述熵特征用于表示二维灰度图像表达的信息量，以及表征图像中脉络纹理的复杂度；当Ent趋近0时，表示灰度图像中几乎不存在纹理信息；如Ent较大，呈现的图像脉络走向更复杂；The entropy feature is used to represent the amount of information expressed by a two-dimensional grayscale image, and to characterize the complexity of vein texture in the image; when Ent approaches 0, it means that there is almost no texture information in the grayscale image; if Ent is larger, The image context presented is more complex;

4)提取灰度共生矩阵中的对比度特征：4) Extract the contrast feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000091
Figure BDA0003128335950000091

所述对比度特征表示灰度图像纹理脉络的清晰度，以及沟纹的深浅程度；对比度大时，图像呈现较清晰，沟纹较深；反之则表明图像较模糊；The contrast feature represents the clarity of the texture of the grayscale image and the depth of the groove; when the contrast is large, the image appears clearer and the groove is deeper; otherwise, it indicates that the image is blurred;

5)提取灰度共生矩阵中的聚类阴影特征：5) Extract the cluster shadow feature in the gray level co-occurrence matrix:

Figure BDA0003128335950000092
Figure BDA0003128335950000092

Figure BDA0003128335950000093
Figure BDA0003128335950000093

Figure BDA0003128335950000094
Figure BDA0003128335950000094

对于每一幅子图像，将从灰度共生矩阵中所提取的特征作为子图像的特征。For each sub-image, the features extracted from the gray level co-occurrence matrix are used as the features of the sub-image.

S5、将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。S5. Using the image feature of the sub-image as the input of the convolutional neural network, using the convolutional neural network to identify the position of the weld defect point on the surface of the hydraulic oil pipeline.

进一步地，本发明将子图像的图像特征作为卷积神经网络的输入；所述卷积神经网络模型，由输入层、卷积层、池化层、全连接层和激活函数层组成；卷积层用于特征提取，公式为：Further, the present invention uses the image features of the sub-image as the input of the convolutional neural network; the convolutional neural network model is composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an activation function layer; Layer is used for feature extraction, the formula is:

Figure BDA0003128335950000095
Figure BDA0003128335950000095

其中：in:

Figure BDA0003128335950000096
代表第n层的第i个特征图；
Figure BDA0003128335950000096
Represents the i-th feature map of the n-th layer;

f()表示激活函数，在本发明一个具体实施例中，所采用的激活函数为ReLU激活函数；f() represents an activation function, and in a specific embodiment of the present invention, the activation function used is a ReLU activation function;

M代表输入子图像集合；M represents the set of input sub-images;

Figure BDA0003128335950000097
表示第n-1层的j个特征；
Figure BDA0003128335950000097
Indicates the j features of the n-1th layer;

Figure BDA0003128335950000098
表示第n层第i个特征图与n-1层第j个特征图连接之间的卷积核；
Figure BDA0003128335950000098
Represents the convolution kernel between the i-th feature map of the n-th layer and the j-th feature map of the n-1 layer;

“*”代表卷积运算；"*" stands for convolution operation;

Figure BDA0003128335950000099
代表第n层的第i个特征的偏置；
Figure BDA0003128335950000099
Represents the bias of the i-th feature of the n-th layer;

池化层将输入的特征图进行特征压缩，简化网络复杂度；全连接层将获取的特征映射到原样本标记空间并将输出值送入分类器。The pooling layer compresses the input feature map to simplify the network complexity; the fully connected layer maps the acquired features to the original sample label space and sends the output value to the classifier.

在传播过程中，输入信息从输入层经隐含单元层逐层处理，并传向输出层，每一层神经元的输出只影响下一层神经元的输入；如果输出层不能得到期望的输出，则进入反向传播，反向传播通过误差反传算法，对网络的权值与阈值进行学习修正；In the propagation process, the input information is processed layer by layer from the input layer through the hidden unit layer, and then transmitted to the output layer. The output of neurons in each layer only affects the input of neurons in the next layer; if the output layer cannot get the desired output , it enters backpropagation, and backpropagation uses the error backpropagation algorithm to learn and correct the weights and thresholds of the network;

当网络的均方差小于给定值时，进入NMS算法运算，则确定缺陷点位置，即完成液压油管道表面缺陷识别。When the mean square error of the network is less than a given value, enter the NMS algorithm operation, then determine the position of the defect point, that is, complete the identification of the surface defect of the hydraulic oil pipeline.

下面通过一个算法实验来说明本发明的具体实施方式，并对发明的处理方法进行测试。本发明算法的硬件测试环境为：Inter(R)Core(TM)i7-6700K CPU，软件为Matlab2018b；对比方法为基于LSTM的液压油管道表面焊缝缺陷识别方法以及基于随机森林的液压油管道表面焊缝缺陷识别方法。The specific implementation of the present invention will be described below through an algorithm experiment, and the processing method of the invention will be tested. The hardware testing environment of the algorithm of the present invention is: Inter(R)Core(TM)i7-6700K CPU, software is Matlab2018b; The comparison method is the hydraulic oil pipeline surface weld defect recognition method based on LSTM and the hydraulic oil pipeline surface based on random forest Weld defect identification method.

在本发明所述算法实验中，数据集为10G的液压油管道图像。本实验通过将液压油管道图像数据输入到算法模型中，将缺陷识别的准确率作为算法可行性的评价指标，其中缺陷识别的准确率越高，则说明算法的有效性、可行性越高。In the algorithm experiment of the present invention, the data set is a hydraulic oil pipeline image of 10G. In this experiment, the image data of the hydraulic oil pipeline is input into the algorithm model, and the accuracy of defect recognition is used as an evaluation index for the feasibility of the algorithm. The higher the accuracy of defect recognition, the higher the effectiveness and feasibility of the algorithm.

根据实验结果，基于LSTM的液压油管道表面焊缝缺陷识别方法的缺陷识别准确率为81.31％，基于随机森林的液压油管道表面焊缝缺陷识别方法的缺陷识别准确率为83.22％，本发明所述方法的缺陷识别准确率为86.95％，相较于对比算法，本发明所提出的水下机器人液压油管道表面焊缝缺陷识别方法能够实现更高的缺陷识别准确率。According to the experimental results, the defect identification accuracy rate of the LSTM-based hydraulic oil pipeline surface weld defect identification method is 81.31%, and the defect identification accuracy rate of the hydraulic oil pipeline surface weld defect identification method based on random forest is 83.22%. The defect identification accuracy rate of the above method is 86.95%. Compared with the comparison algorithm, the method for identifying the surface weld seam defects of the hydraulic oil pipeline of the underwater robot proposed by the present invention can achieve a higher defect identification accuracy rate.

发明还提供一种水下机器人液压油管道表面焊缝缺陷识别装置。参照图2所示，为本发明一实施例提供的水下机器人液压油管道表面焊缝缺陷识别装置的内部结构示意图；参考图3所示，为本发明一实施例提供的水下机器人液压油管道表面焊缝缺陷识别装置的装置示意图；The invention also provides an underwater robot hydraulic oil pipeline surface weld seam defect identification device. Referring to Figure 2, it is a schematic diagram of the internal structure of an underwater robot hydraulic oil pipeline surface weld defect identification device provided by an embodiment of the present invention; referring to Figure 3, it is an underwater robot hydraulic oil provided by an embodiment of the present invention. Schematic diagram of the device for detecting weld defects on the pipeline surface;

在本实施例中，所述水下机器人液压油管道表面焊缝缺陷识别装置1至少包括图像获取装置11、数据处理器12、缺陷识别装置13，通信总线14，以及网络接口15。In this embodiment, the underwater robot hydraulic oil pipeline surface weld defect identification device 1 at least includes an image acquisition device 11 , a data processor 12 , a defect identification device 13 , a communication bus 14 , and a network interface 15 .

其中，图像获取装置11包括移动光源、工业内窥镜、工业相机等。Wherein, the image acquisition device 11 includes a mobile light source, an industrial endoscope, an industrial camera, and the like.

数据处理器12至少包括一种类型的可读存储介质，所述可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、磁性存储器、磁盘、光盘等。数据处理器12在一些实施例中可以是水下机器人液压油管道表面焊缝缺陷识别装置1的内部存储单元，例如该水下机器人液压油管道表面焊缝缺陷识别装置1的硬盘。数据处理器12在另一些实施例中也可以是水下机器人液压油管道表面焊缝缺陷识别装置1的外部存储设备，例如水下机器人液压油管道表面焊缝缺陷识别装置1上配备的插接式硬盘，智能存储卡(SmartMedia Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，数据处理器12还可以既包括水下机器人液压油管道表面焊缝缺陷识别装置1的内部存储单元也包括外部存储设备。数据处理器12不仅可以用于存储安装于水下机器人液压油管道表面焊缝缺陷识别装置1的应用软件及各类数据，还可以用于暂时地存储已经输出或者将要输出的数据。The data processor 12 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card-type memory (eg, SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, and the like. In some embodiments, the data processor 12 may be an internal storage unit of the underwater robot hydraulic oil pipeline surface weld defect identification device 1 , such as a hard disk of the underwater robot hydraulic oil pipeline surface weld defect identification device 1 . In other embodiments, the data processor 12 can also be an external storage device of the underwater robot hydraulic oil pipeline surface weld defect identification device 1, such as the plug-in device 1 equipped on the underwater robot hydraulic oil pipeline surface weld defect identification device 1 Type hard disk, smart memory card (SmartMedia Card, SMC), secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc. Further, the data processor 12 may also include both an internal storage unit of the underwater robot hydraulic oil pipeline surface weld defect identification device 1 and an external storage device. The data processor 12 can not only be used to store the application software and various data installed in the underwater robot hydraulic oil pipeline surface weld defect identification device 1, but also can be used to temporarily store the data that has been output or will be output.

缺陷识别装置13在一些实施例中可以是一中央处理器(Central ProcessingUnit,CPU)、控制器、微控制器、微处理器或其他数据处理芯片，包括监控单元，用于运行数据处理器12中存储的程序代码或处理数据，例如缺陷识别程序指令16等。The defect identification device 13 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chips in some embodiments, including a monitoring unit, for running in the data processor 12 Stored program code or process data, such as defect identification program instructions 16, etc.

通信总线14用于实现这些组件之间的连接通信。The communication bus 14 is used to realize connection communication between these components.

网络接口15可选的可以包括标准的有线接口、无线接口(如WI-FI接口)，通常用于在该装置1与其他电子设备之间建立通信连接。The network interface 15 may optionally include standard wired interfaces and wireless interfaces (such as WI-FI interfaces), which are generally used to establish communication connections between the apparatus 1 and other electronic devices.

可选地，水下机器人液压油管道表面焊缝缺陷识别装置1还可以包括用户接口，用户接口可以包括显示器(Display)、输入单元比如键盘(Keyboard)，可选的用户接口还可以包括标准的有线接口、无线接口。可选地，在一些实施例中，显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode，有机发光二极管)触摸器等。其中，显示器也可以适当的称为显示屏或显示单元，用于显示在水下机器人液压油管道表面焊缝缺陷识别装置1中处理的信息以及用于显示可视化的用户界面。Optionally, the underwater robot hydraulic oil pipeline surface weld defect identification device 1 can also include a user interface, and the user interface can include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface can also include a standard Wired interface, wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, Organic Light-Emitting Diode) touch panel, and the like. Wherein, the display can also be appropriately referred to as a display screen or a display unit, and is used for displaying information processed in the underwater robot hydraulic oil pipeline surface weld defect identification device 1 and for displaying a visualized user interface.

图2仅示出了具有组件11-15以及水下机器人液压油管道表面焊缝缺陷识别装置1，本领域技术人员可以理解的是，图1示出的结构并不构成对水下机器人液压油管道表面焊缝缺陷识别装置1的限定，可以包括比图示更少或者更多的部件，或者组合某些部件，或者不同的部件布置。Fig. 2 only shows components 11-15 and the underwater robot hydraulic oil pipeline surface weld defect identification device 1, those skilled in the art can understand that the structure shown in Fig. The definition of the pipeline surface weld seam defect identification device 1 may include fewer or more components than shown in the figure, or combine some components, or arrange different components.

在图2所示的水下机器人液压油管道表面焊缝缺陷识别装置1实施例中，数据处理器12中存储有缺陷识别程序指令16；缺陷识别装置13执行数据处理器12中存储的缺陷识别程序指令16的步骤，与水下机器人液压油管道表面焊缝缺陷识别方法的实现方法相同，在此不作类述。In the embodiment of the underwater robot hydraulic oil pipeline surface weld defect identification device 1 shown in Figure 2, the defect identification program instruction 16 is stored in the data processor 12; the defect identification device 13 executes the defect identification stored in the data processor 12 The steps of the program instruction 16 are the same as the implementation method of the method for identifying weld defects on the surface of the hydraulic oil pipeline of the underwater robot, and will not be described here.

此外，本发明实施例还提出一种计算机可读存储介质，所述计算机可读存储介质上存储有缺陷识别程序指令，所述缺陷识别程序指令可被一个或多个处理器执行，以实现如下操作：In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which defect identification program instructions are stored, and the defect identification program instructions can be executed by one or more processors to achieve the following operate:

获取液压油管道图像，对液压油管道图像进行图像灰度化和灰度拉伸的预处理，得到液压油管道灰度图像；Obtain the hydraulic oil pipeline image, perform image grayscale and grayscale stretching preprocessing on the hydraulic oil pipeline image, and obtain the hydraulic oil pipeline grayscale image;

利用图像增强策略对液压油管道灰度图像进行图像增强处理；Using the image enhancement strategy to carry out image enhancement processing on the grayscale image of the hydraulic oil pipeline;

利用图像分割网络对增强后的图像进行分割，得到若干子图像；Use the image segmentation network to segment the enhanced image to obtain several sub-images;

利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征；Using the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the feature parameters of the sub-image to obtain the image features of the sub-image;

将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。The image features of the sub-image are used as the input of the convolutional neural network, and the position of the weld defect point on the surface of the hydraulic oil pipeline is identified by the convolutional neural network.

需要说明的是，上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。并且本文中的术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、装置、物品或者方法不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、装置、物品或者方法所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、装置、物品或者方法中还存在另外的相同要素。It should be noted that the serial numbers of the above embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments. And herein the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, apparatus, article or method comprising a set of elements includes not only those elements, but also includes the elements not expressly included. other elements listed, or also include elements inherent in the process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, apparatus, article or method comprising that element.

通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to enable a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

以上仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process conversion made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technical fields , are all included in the scope of patent protection of the present invention in the same way.

Claims (3)

1.一种水下机器人液压油管道表面焊缝缺陷识别方法，其特征在于，所述方法包括：1. An underwater robot hydraulic oil pipeline surface weld defect identification method is characterized in that, the method comprises: 获取液压油管道图像，对液压油管道图像进行图像灰度化和灰度拉伸的预处理，得到液压油管道灰度图像；Obtain the hydraulic oil pipeline image, perform image grayscale and grayscale stretching preprocessing on the hydraulic oil pipeline image, and obtain the hydraulic oil pipeline grayscale image; 利用图像增强策略对液压油管道灰度图像进行图像增强处理，所述图像增强策略流程为：Image enhancement processing is performed on the grayscale image of the hydraulic oil pipeline using an image enhancement strategy, and the process of the image enhancement strategy is as follows: 1)构建高斯滤波核函数矩阵，将高斯滤波核函数矩阵与液压油管道灰度图像进行卷积运算，得到高斯滤波后的液压油管道灰度图像；1) Construct the Gaussian filter kernel function matrix, and perform convolution operation on the Gaussian filter kernel function matrix and the grayscale image of the hydraulic oil pipeline to obtain the grayscale image of the hydraulic oil pipeline after the Gaussian filter; 2)对液压油管道灰度图像进行直方图均衡化处理，其步骤为：2) Perform histogram equalization processing on the grayscale image of the hydraulic oil pipeline, the steps are: 统计液压油管道灰度图像各个灰度级对应的像素数目，得到图像的直方图：Count the number of pixels corresponding to each gray level of the gray scale image of the hydraulic oil pipeline to obtain the histogram of the image:
Figure FDA0003834655620000011
Figure FDA0003834655620000011
 其中：in: k＝0,1,…,L-1，表示图像的灰度级；k=0,1,...,L-1, representing the gray level of the image; nk表示灰度级为k的像素数；n k represents the number of pixels with a gray level of k; n表示液压油管道灰度图像的像素总数；n represents the total number of pixels of the hydraulic oil pipeline grayscale image; 计算液压油管道灰度图像的累积直方图：Compute the cumulative histogram of a grayscale image of a hydraulic oil pipeline:
Figure FDA0003834655620000012
Figure FDA0003834655620000012
 对累积直方图进行映射变换处理，将原始累积直方图映射到灰度范围[L0,Lk]：Perform mapping transformation processing on the cumulative histogram, and map the original cumulative histogram to the gray range [L 0 ,L k ]: S＝L0+(Lk-L0)c(k)S＝L 0 +(L k -L 0 )c(k) 统计映射变换后S中各灰度级的像素个数，得到新的图像直方图；Count the number of pixels of each gray level in S after the mapping transformation, and obtain a new image histogram; 3)采用自适应的水下图像颜色矫正算法对直方图均衡化的液压油管道灰度图像进行颜色矫正，所述自适应的水下图像颜色矫正的公式为：3) Using an adaptive underwater image color correction algorithm to correct the color of the histogram-equalized hydraulic oil pipeline grayscale image, the formula for the adaptive underwater image color correction is:
Figure FDA0003834655620000013
Figure FDA0003834655620000013
 其中：in: I′(R,G,B)表示矫正后的液压油管道灰度图像在R,G,B三个颜色通道的和；I'(R, G, B) represents the sum of the three color channels of R, G, and B in the grayscale image of the hydraulic oil pipeline after correction; I(R,G,B)表示液压油管道灰度图像在R,G,B三个颜色通道的和；I(R,G,B) represents the sum of the three color channels of R, G, and B in the grayscale image of the hydraulic oil pipeline; μ表示图像颜色通道的闵可夫斯基距离均值；μ represents the mean value of the Minkowski distance of the image color channel; α表示液压油管道灰度图像在R,G,B三个颜色通道的最大值；α represents the maximum value of the grayscale image of the hydraulic oil pipeline in the three color channels of R, G, and B; β为修正参数，将其设置为0.2；β is a correction parameter, which is set to 0.2; 4)利用基于图像梯度的水下图像亮度增强算法对颜色矫正后的液压油管道灰度图像进行图像亮度增强处理，所述图像亮度增强的公式为：4) Using an image gradient-based underwater image brightness enhancement algorithm to perform image brightness enhancement processing on the color-corrected hydraulic oil pipeline grayscale image, the formula for image brightness enhancement is:
Figure FDA0003834655620000014
Figure FDA0003834655620000014
Figure FDA0003834655620000015
Figure FDA0003834655620000015
Figure FDA0003834655620000016
Figure FDA0003834655620000016
 其中：in: [Tmin,Tmax]表示图像亮度增强的范围；[T min , T max ] indicates the range of image brightness enhancement; s(x,y)表示图像像素(x,y)的亮度值；s(x,y) represents the brightness value of the image pixel (x,y); E(x,y)表示增强后图像像素(x,y)的亮度值；E(x, y) represents the brightness value of the enhanced image pixel (x, y); i表示梯度方向；i represents the gradient direction;
Figure FDA0003834655620000021
表示在不同梯度方向上的偏导数；
Figure FDA0003834655620000021
Represents the partial derivatives in different gradient directions; qi(x,y)表示在不同梯度方向上的目标梯度；q i (x, y) represents the target gradient in different gradient directions; wi(x,y)表示在不同梯度方向上梯度残差的影响权重，a表示梯度残差的灵敏度，其值范围在[-0.8,2]之间；w i (x, y) represents the influence weight of the gradient residual in different gradient directions, a represents the sensitivity of the gradient residual, and its value range is between [-0.8, 2]; g′(c,y)为亮度约束函数，将增强后的亮度限制在目标范围内；g'(c,y) is the brightness constraint function, which limits the enhanced brightness to the target range; 利用图像分割网络对增强后的图像进行分割，得到若干子图像，所述利用图像分割网络对增强后的图像进行分割，包括：The enhanced image is segmented by an image segmentation network to obtain several sub-images, and the enhanced image is segmented by the image segmentation network, including: 所述图像分割网络的目标函数为：The objective function of the image segmentation network is:
Figure FDA0003834655620000022
Figure FDA0003834655620000022
 其中：in: t′为预测到的图像分割边界；t' is the predicted image segmentation boundary; t为图像分割边界的二值化结果；t is the binarization result of the image segmentation boundary; M'为输入图像；M' is the input image; R为分割图像；R is the segmented image; D(t)表示分割边框的距离变换，即不同分割图像之间的距离图；D(t) represents the distance transformation of the segmentation border, that is, the distance map between different segmentation images; 将增强后的图像输入到图像分割网络中，根据预测所得到的分割边界t′进行图像分割，得到若干子图像；Input the enhanced image into the image segmentation network, perform image segmentation according to the predicted segmentation boundary t′, and obtain several sub-images; 利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，得到子图像的图像特征；Using the feature parameter extraction algorithm based on the gray level co-occurrence matrix to extract the feature parameters of the sub-image to obtain the image features of the sub-image; 将子图像的图像特征作为卷积神经网络的输入，利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置。The image features of the sub-image are used as the input of the convolutional neural network, and the position of the weld defect point on the surface of the hydraulic oil pipeline is identified by the convolutional neural network. 2.如权利要求1所述的一种水下机器人液压油管道表面焊缝缺陷识别方法，其特征在于，所述利用基于灰度共生矩阵的特征参数提取算法对子图像进行特征参数提取处理，包括：2. a kind of underwater robot hydraulic oil pipeline surface weld defect identification method as claimed in claim 1, is characterized in that, described utilization is based on the characteristic parameter extraction algorithm of gray level co-occurrence matrix to carry out characteristic parameter extraction process to sub-image, include: 1)对于子图像f(x,y)，其大小为M×N，存在k个灰度级，其中任意两个像素i和像素j之间的距离为
Figure FDA0003834655620000023
且两个像素连线同坐标轴形成的夹角为θ，则对于图像像素i以及图像像素j，其灰度共生矩阵值为Pij(d,θ)，子图像的灰度共生矩阵为：1) For a sub-image f(x,y), its size is M×N, there are k gray levels, and the distance between any two pixels i and pixel j is
Figure FDA0003834655620000023
And the angle formed by the connection line of two pixels with the coordinate axis is θ, then for image pixel i and image pixel j, the gray level co-occurrence matrix value is P ij (d, θ), and the gray level co-occurrence matrix of the sub-image is:
Figure FDA0003834655620000024
Figure FDA0003834655620000024
 2)提取灰度共生矩阵中的角二阶矩特征：2) Extract the second-order moment feature of the angle in the gray level co-occurrence matrix:
Figure FDA0003834655620000025
Figure FDA0003834655620000025
 3)提取灰度共生矩阵中的熵特征：3) Extract the entropy feature in the gray level co-occurrence matrix:
Figure FDA0003834655620000026
Figure FDA0003834655620000026
 4)提取灰度共生矩阵中的对比度特征：4) Extract the contrast feature in the gray level co-occurrence matrix:
Figure FDA0003834655620000027
Figure FDA0003834655620000027
 5)提取灰度共生矩阵中的聚类阴影特征：5) Extract the cluster shadow feature in the gray level co-occurrence matrix:
Figure FDA0003834655620000031
Figure FDA0003834655620000031
Figure FDA0003834655620000032
Figure FDA0003834655620000032
Figure FDA0003834655620000033
Figure FDA0003834655620000033
 对于每一幅子图像，将从灰度共生矩阵中所提取的特征作为子图像的特征。For each sub-image, the features extracted from the gray level co-occurrence matrix are used as the features of the sub-image. 3.如权利要求2所述的一种水下机器人液压油管道表面焊缝缺陷识别方法，其特征在于，所述利用卷积神经网络识别出液压油管道表面焊缝缺陷点的位置，包括：3. a kind of underwater robot hydraulic oil pipeline surface weld defect recognition method as claimed in claim 2, is characterized in that, described utilizing convolutional neural network to identify the position of hydraulic oil pipeline surface weld defect point, comprising: 将子图像的图像特征作为卷积神经网络的输入；卷积神经网络中的卷积层用于特征提取，公式为：The image features of the sub-image are used as the input of the convolutional neural network; the convolutional layer in the convolutional neural network is used for feature extraction, and the formula is:
Figure FDA0003834655620000034
Figure FDA0003834655620000034
 其中：in:
Figure FDA0003834655620000035
代表第n层的第i个特征图；
Figure FDA0003834655620000035
Represents the i-th feature map of the n-th layer; f()表示激活函数；f() represents the activation function; M”代表输入子图像集合；M" represents the input sub-image set;
Figure FDA0003834655620000036
表示第n-1层的j个特征；
Figure FDA0003834655620000036
Indicates the j features of the n-1th layer;
Figure FDA0003834655620000037
表示第n层第i个特征图与n-1层第j个特征图连接之间的卷积核；
Figure FDA0003834655620000037
Represents the convolution kernel between the i-th feature map of the n-th layer and the j-th feature map of the n-1 layer;
Figure FDA0003834655620000038
代表第n层的第i个特征的偏置；
Figure FDA0003834655620000038
Represents the bias of the i-th feature of the n-th layer; 池化层将输入的特征图进行特征压缩，简化网络复杂度；全连接层将获取的特征映射到原样本标记空间并将输出值送入分类器；The pooling layer compresses the input feature map to simplify the network complexity; the fully connected layer maps the acquired features to the original sample label space and sends the output value to the classifier; 在传播过程中，输入信息从输入层经隐含单元层逐层处理，并传向输出层，每一层神经元的输出只影响下一层神经元的输入；如果输出层不能得到期望的输出，则进入反向传播，反向传播通过误差反传算法，对网络的权值与阈值进行学习修正；In the propagation process, the input information is processed layer by layer from the input layer through the hidden unit layer, and then transmitted to the output layer. The output of neurons in each layer only affects the input of neurons in the next layer; if the output layer cannot get the desired output , it enters backpropagation, and backpropagation uses the error backpropagation algorithm to learn and correct the weights and thresholds of the network; 当网络的均方差小于给定值时，进入NMS算法运算，则确定缺陷点位置，即完成液压油管道表面缺陷识别。When the mean square error of the network is less than a given value, enter the NMS algorithm operation, then determine the position of the defect point, that is, complete the identification of the surface defect of the hydraulic oil pipeline.
CN202110697224.2A 2021-06-23 2021-06-23 A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots Active CN113436162B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110697224.2A CN113436162B (en) 2021-06-23 2021-06-23 A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110697224.2A CN113436162B (en) 2021-06-23 2021-06-23 A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots

Publications (2)

Publication Number Publication Date
CN113436162A CN113436162A (en) 2021-09-24
CN113436162B true CN113436162B (en) 2022-12-09

Family

ID=77753498

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110697224.2A Active CN113436162B (en) 2021-06-23 2021-06-23 A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots

Country Status (1)

Country Link
CN (1) CN113436162B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113763294B (en) * 2021-09-26 2023-08-08 上海航天精密机械研究所 Quick preprocessing method and system for weld image based on dynamic CLAHE
CN114359753B (en) * 2021-12-09 2025-01-28 中国科学院空天信息创新研究院 Water body identification method, device, electronic device and storage medium
CN114399432A (en) * 2021-12-13 2022-04-26 广西北投信创科技投资集团有限公司 Target recognition method, device, equipment, medium and product
CN114862775A (en) * 2022-04-21 2022-08-05 广州东焊智能装备有限公司 Visual identification system of welding robot
CN115598131B (en) * 2022-09-21 2024-10-18 江苏华电昆山热电有限公司 Positioning method suitable for welding lines in pipeline
CN117576105B (en) * 2024-01-17 2024-03-29 高科建材（咸阳）管道科技有限公司 Pipeline production control method and system based on artificial intelligence
CN117710365B (en) * 2024-02-02 2024-05-03 中国电建集团华东勘测设计研究院有限公司 Processing method and device for defective pipeline image and electronic equipment
CN117830300B (en) * 2024-03-04 2024-05-14 新奥新能源工程技术有限公司 Visual-based gas pipeline appearance quality detection method
CN118279314B (en) * 2024-06-04 2024-08-02 河南安盛技术管理有限公司 Water quality abnormality detection method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104408724A (en) * 2014-11-27 2015-03-11 中南大学 Depth information method and system for monitoring liquid level and recognizing working condition of foam flotation
CN110349114A (en) * 2019-05-24 2019-10-18 江西理工大学 Applied to the image enchancing method of AOI equipment, device and road video monitoring equipment
CN111360780A (en) * 2020-03-20 2020-07-03 北京工业大学 Garbage picking robot based on visual semantic SLAM

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10328514B2 (en) * 2011-11-07 2019-06-25 Lincoln Global, Inc. Use of mobile communications devices as user interface for welding equipment and systems
US10282914B1 (en) * 2015-07-17 2019-05-07 Bao Tran Systems and methods for computer assisted operation
CN104985290A (en) * 2015-08-04 2015-10-21 梁彦云 Neural-network-identification-based underwater weld joint tracking method
CN105938563A (en) * 2016-04-14 2016-09-14 北京工业大学 Weld surface defect identification method based on image texture
KR102125167B1 (en) * 2019-01-21 2020-06-19 한양대학교 에리카산학협력단 Automatic welding device, working method thereof
CN111932489B (en) * 2020-06-03 2024-02-20 西安电子科技大学 Weld defect detection method, weld defect detection system, storage medium, computer equipment and terminal
CN112396564A (en) * 2020-11-19 2021-02-23 汪金玲 Product packaging quality detection method and system based on deep learning
CN112734693B (en) * 2020-12-18 2024-06-07 平安科技（深圳）有限公司 Pipeline weld defect detection method and related device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104408724A (en) * 2014-11-27 2015-03-11 中南大学 Depth information method and system for monitoring liquid level and recognizing working condition of foam flotation
CN110349114A (en) * 2019-05-24 2019-10-18 江西理工大学 Applied to the image enchancing method of AOI equipment, device and road video monitoring equipment
CN111360780A (en) * 2020-03-20 2020-07-03 北京工业大学 Garbage picking robot based on visual semantic SLAM

Also Published As

Publication number Publication date
CN113436162A (en) 2021-09-24

Similar Documents

Publication Publication Date Title
CN113436162B (en) A method and device for identifying weld defects on the surface of hydraulic oil pipelines for underwater robots
CN111257341B (en) Crack detection method for underwater buildings based on multi-scale features and stacked fully convolutional network
TWI744283B (en) Method and device for word segmentation
CN111553929A (en) Mobile phone screen defect segmentation method, device and equipment based on fusion network
CN110766002B (en) Ship name character region detection method based on deep learning
CN110033040B (en) A flame identification method, system, medium and device
CN104299006A (en) Vehicle license plate recognition method based on deep neural network
CN113705461B (en) Face definition detection method, device, equipment and storage medium
CN111209858B (en) Real-time license plate detection method based on deep convolutional neural network
WO2022105197A1 (en) Systems and methods for image detection
CN114529946A (en) Pedestrian re-identification method, device, equipment and storage medium based on self-supervision learning
CN111539456B (en) Target identification method and device
CN112330619B (en) Method, device, equipment and storage medium for detecting target area
CN114782770A (en) A method and system for license plate detection and license plate recognition based on deep learning
CN117557565B (en) Detection method and device for lithium battery pole piece
CN113256572A (en) Gastroscope image analysis system, method and equipment based on restoration and selective enhancement
CN112258490A (en) Low-emissivity coating intelligent damage detection method based on optical and infrared image fusion
CN117218434A (en) Concrete structure surface defect classification method and system based on hybrid neural network
CN115049600A (en) Intelligent identification system and method for small sample pipeline defects
CN117409190A (en) Real-time infrared image target detection method, device, equipment and storage medium
CN117764977A (en) Magnetic shoe surface defect detection method based on global feature enhancement and multi-scale fusion
CN106127210A (en) A kind of significance detection method based on multiple features
CN116309245B (en) Underground drainage pipeline defect intelligent detection method and system based on deep learning
CN114926753B (en) Rapid target scene information extraction method under condition of massive images
CN117078629A (en) Glove defect detection method based on improved YOLOv5

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
PE01 Entry into force of the registration of the contract for pledge of patent right

Denomination of invention: A method and device for identifying surface weld defects in hydraulic oil pipelines of underwater robots

Granted publication date: 20221209

Pledgee: Huaxia Bank Co.,Ltd. Changsha Branch

Pledgor: HUNAN GUOTIAN ELECTRONIC TECHNOLOGY CO.,LTD.

Registration number: Y2024980057854

PE01 Entry into force of the registration of the contract for pledge of patent right