CN116243289A - Unmanned ship underwater target intelligent identification method based on imaging sonar - Google Patents

Abstract

The invention discloses an intelligent unmanned ship underwater target identification method based on imaging sonar, and relates to the field of unmanned operation platforms and target identification. Aiming at the requirements of unmanned ships on sailing and tasks, the invention considers the task characteristics of the unmanned ships, and specifically adopts forward-looking sonar to perform target ranging positioning and accurate identification, and for underwater targets, the invention realizes thorough water area sensing, target detection and accurate identification based on machine learning and. The invention can improve the underwater target recognition performance of the unmanned ship.

Description

Unmanned ship underwater target intelligent identification method based on imaging sonar

Technical Field

The invention relates to the technical fields of unmanned ship operation platforms, digital image processing for target identification and classification, underwater target identification and the like.

Background

Unmanned surface vessels (Unmanned Surface Vessels, USV) refer to intelligent unmanned platforms that can be deployed or recovered by land-based or surface vessels and that can achieve autonomous or semi-autonomous navigation. In the field of marine environment detection, unmanned surface vessels become a new detection means in the field of environment detection integration and intellectualization. Therefore, the unmanned surface vessel with sonar is often used for real-time detection of underwater environment, which also makes the unmanned surface vessel widely applied to the military field and the environmental exploration field, such as mine detection, information collection, monitoring, reconnaissance, offshore defense, marine hydrological meteorological information collection, submarine topography scanning and mapping and the like.

The unmanned surface vessel has small volume and shallow draft, and the executive tasks are mainly concentrated in shallow water areas. Underwater detection techniques mainly include optical imaging and sonar imaging. Unmanned ship underwater target recognition technology is different from land or air target recognition method. The target image information acquisition is difficult due to the influence of the environmental factors of the shallow water area, such as time-varying water flow, uneven illumination, turbid water quality and the like. Meanwhile, the method is greatly influenced by scattering effect of the aqueous medium on the target information, so that the target information is blurred or distorted. The traditional underwater target identification method of the unmanned ship is optical identification, the distance of optical imaging is generally between a few meters and tens of meters, the distance is relatively short, and the unmanned ship is basically invalid in a muddy water area. The sonar imaging has the advantages of long acting distance, strong penetrating capacity and the like, and is particularly suitable for muddy water areas.

In recent years, deep learning has made significant progress in target recognition and classification digital image processing, and recognition accuracy can be improved by training a training model of a large number of samples. Because the cost of the sonar target imaging test is high, and a large number of available actual images are difficult to obtain, the fact that overfitting occurs when the CNN is directly trained can lead to low sonar identification accuracy.

Disclosure of Invention

Aiming at the situation that the recognition target is near and the muddy water area is basically invalid in the traditional underwater target optical recognition method of the unmanned surface vessel; the invention provides an intelligent recognition method for underwater targets of an unmanned ship based on imaging sonar, which is characterized in that a sonar target imaging recognition training sample is difficult to obtain, so that the training result is easy to fit excessively, the recognition accuracy is low, the unmanned ship is used for carrying multi-beam forward-looking imaging sonar, a sonar image enhancement algorithm based on Curvelet-enhancement Curvelet transformation is combined with a convolutional neural network deep learning algorithm, detailed information such as the edges and textures of a sonar image is highlighted, the sonar imaging quality is improved, the target recognition requirement of common scenes of the unmanned ship is met, and the underwater target recognition accuracy of the unmanned ship is improved.

The technical solution of the invention comprises: utilizing a multi-beam forward-looking imaging sonar emission matrix carried by the unmanned surface vessel to emit a plurality of acoustic beams with the same horizontal opening angle and the same vertical opening angle at one time, scanning a sector three-dimensional area in front of the sonar, and collecting underwater target sonar images with multiple angles; and (3) carrying out Curvelet-enhancement Curvelet decomposition on the collected original sonar image, so that the overall contrast diagram of the sonar image is improved, and the overall imaging quality of the sonar image is enhanced. Establishing a deep learning neural network, establishing a convolution layer, extracting local features of an image, alternately establishing the convolution layer and a pooling layer, preventing data from being over-fitted, and improving the sample recognition accuracy through iterative training.

Compared with the prior art, the invention has the following advantages:

the traditional target identification method of the unmanned ship is optical identification, but the imaging effect of traditional optical imaging in a muddy water area can be influenced by water area waves and sunlight scattering and is basically ineffective. The image collected by the common imaging sonar has the characteristics of strong noise, serious distortion, blurred target edge, low resolution and the like. The invention adopts the intelligent underwater target recognition method based on the combination of the Curvelet-enhancement Curvelet transformation and the convolutional neural network deep learning algorithm, improves the display effect of the sonar image, improves the multi-scale characteristic and good directivity based on the Curvelet Curvelet transformation, provides the piecewise nonlinear enhancement method based on the Curvelet-enhancement transformation, can separate noise information and edge information well, can highlight detail information such as edges, textures and the like of the sonar image while improving contrast and suppressing noise, and purposefully strengthens the integral and local characteristics of the sonar image aiming at the application occasions such as the collected sonar image-rail , the fence, the fishing net and the like, enlarges the difference between different object characteristics in the image, and meets the requirement of intelligent underwater target recognition of an unmanned ship. The accuracy of target identification and the environmental adaptability are improved, and the false alarm and the false detection missing probability are reduced.

Drawings

Fig. 1 is a flow chart of the present invention.

Detailed Description

The invention provides an intelligent recognition method for an unmanned ship underwater target based on imaging sonar, wherein the implementation process is shown in fig. 1, and the preferred implementation process can be described as follows:

step 1: the front-view imaging sonar carried by the unmanned surface vessel is used for acquiring a sonar image, acquired data are transmitted to an information processing computer carried by the unmanned surface vessel, and a training and verification image and video database is built.

Step 2: curvelet-enhancement Curvelet decomposition is carried out on the collected original sonar image to obtain a low-frequency subband coefficient F ₀ And a high frequency subband coefficient F _s,n S denotes the scale and n denotes the subband direction.

Step 3: the S-shaped function algorithm is acted on the normalized low-frequency subband coefficient to improve the overall contrast of the sonar image:

f in the formula ₀ And F ₀ ' the low frequency subband coefficients before and after enhancement, respectively; m is M ₀ Is the maximum value of the low frequency coefficient; k (k) ₁ Is constant (k) ₁ > 1), wherein the sigmoid function algorithm is in the form of: y=vx/(x+exp #)a-Bx)), where v is the maximum gray value.

Step 4: in order to avoid amplification of noise coefficients, each high-frequency subband coefficient is subjected to nonlinear enhancement processing, and thresholding processing is performed in accordance with a set threshold, when abs (F _s,n ) When not less than T:

when abs (F) _s,n )<At T, F _s ' _,n =0; f in the formula _s,n And F _s ' _,n Each high-frequency subband coefficient before and after enhancement; m is M _s,n Is the maximum value of the layer coefficient; k (k) ₂ Is constant (k) ₂ > 1); wherein the nonlinear gain function is:

f(x)＝A[omicron(C(x-B)-omicron(-C(x+B))]×e ^(x-1)×D

in the middle of

The value of C is 20-50, the value of D is 1-0.05, the B parameter is used for controlling the enhancement range, and the C and D parameters are used for controlling the gain intensity.

Step 5: and (3) performing Curvelet-enhancement curved wave inverse transformation on all sub-band coefficients to obtain an enhanced sonar image.

Step 6: and (3) establishing a deep learning convolutional neural network for the enhanced sonar image subjected to Curvelet-enhancement Curvelet transformation in the step (1-5). And marking the characteristic region to be identified in the enhanced sonar image through manual screening.

Step 7: establishing a convolution layer, and setting a convolution kernel with m multiplied by n when an image is transmitted to the convolution layer

Multiplying each weight in the convolution kernel W with a corresponding pixel X covered in the enhanced sonar image X, and then summing, wherein the calculation formula is as follows:

the output is generated by adding a scalar offset to the operation result z. The convolution layer can extract local characteristics in the picture through the filtering of the convolution kernel, so that the data size is reduced, and the calculation consumption is reduced.

Step 8: and (3) carrying out maximum pooling layer operation on the region of interest, converting the image of the screened region into a feature vector with the fixed size of W multiplied by H, wherein the region of interest refers to a rectangular window in a convolution feature map, selecting and calculating a segmentation region by using a search algorithm, and each region of interest is represented by a quaternary vector, wherein (x, y) is used for identifying the upper left corner coordinates, and (W, H) is used for representing the height and the width of the rectangular window. The region of interest pooling layer divides the region of size (W, H) into w×h grid word windows, each window being approximately (W/W) × (H/H), and then maximizes the pooling of feature values in each sub-window into the corresponding output network. After this has been applied to each feature channel, h is the same as the maximum pooling of criteria.

Step 9: performing multi-task training, simultaneously taking target classification and candidate region frame regression calculation as two parallel output layers, outputting the distribution probability of each region of interest on K+1 class by the first task, wherein K is the number of target classification, adding the class of background, and calculating the probability by using a softmax function; the second task is to calculate the regression offset of the bounding box

The multi-task loss function is the joint training classification and candidate region frame regression calculation:

L(p,u,t ^u ,v)＝L _cls (p,u)+λ[u≥1]+L _loc (t ^u ,v)

the parameter u marks the true category of the candidate region content as a target, generally u is more than or equal to 1, and if u=0, the region content is represented as a background; l (L) _cis (p,u)＝-logP _u Is a loss function corresponding to category u, L _loc (t ^u V) by smoothing L ₁ Loss ofFunction calculation of the loss function of the frame position, t ^u Is the frame of category u prediction; square brackets [ u is greater than or equal to 1]If the mark meets the condition u in the square frame, the mark is 1, otherwise, the mark is 0; the parameter lambda controls the balance between the two loss functions; since both loss functions are equally important, in practice set to 1.

Step 10: the gradient descent method is used for parameter back propagation on the convolutional neural network. In the training process, the whole data set is completely sent into a network model for training and learning, so that the network calculates an iteratively updated gradient value by using all samples. The convolution characteristics are processed by the region-of-interest pooling layer, and the obtained characteristics are sent to two parallel computing tasks for training, classifying and positioning regression, so that the training of the target is better realized.

Step 11: and (3) through repeated iterative computation, the loss function value is smaller than 0.1, and the trained deep learning neural network is obtained.

Step 12: and (3) repeatedly training the deep learning neural network obtained in the step (6-10) by utilizing the enhanced sonar image database obtained in the step (1-5).

Step 13: and counting the accuracy of the recognition result obtained through training. If the accuracy is low, repeating the step 1, and increasing the diversity of the sonar collected samples. And (5) repeating the steps 1-10 on the sonar sample collected by the front vision multi-beam imaging sonar carried by the unmanned ship.

Step 14: and displaying the target identification type value at the display front end of the unmanned ship.

Claims (1)

1. An intelligent recognition method for an unmanned ship underwater target based on imaging sonar is characterized by comprising the following steps:

step 1: acquiring a sonar image by using a forward-looking imaging sonar carried by the unmanned surface vessel, transmitting acquired data to an information processing computer carried by the unmanned surface vessel, and constructing a training and verifying image and video database;

step 2: curvelet-enhancement Curvelet decomposition is carried out on the collected original sonar image to obtain a low-frequency subband coefficient F ₀ And a high frequency subband coefficient F _s,n S represents a scale, n represents a subband direction;

step 3: the S-shaped function algorithm is applied to normalized low-frequency subband coefficients:

f in the formula ₀ And F ₀ ' the low frequency subband coefficients before and after enhancement, respectively; m is M ₀ Is the maximum value of the low frequency coefficient; k (k) ₁ Is constant, k ₁ > 1, wherein the sigmoid function algorithm form: y=vx/(x+exp (a-Bx)), where v is the maximum gray value;

step 4: each high-frequency subband coefficient is subjected to nonlinear enhancement processing, and thresholding processing is performed simultaneously according to a set threshold, and when abs (F _s,n ) When not less than T:

when abs (F) _s,n )<And T is as follows:

F’ _s,n ＝0

f in the formula _s,n And F' _s,n Each high-frequency subband coefficient before and after enhancement; m is M _s,n Is the maximum value of the layer coefficient; k (k) ₂ Is constant, k ₂ > 1; wherein the nonlinear gain function is:

f(x)＝A[omicron(C(x-B)-omicron(-C(x+B))]×e ^(|x|-1)×D ；

wherein:

c takes the value between 20 and 50, D takes the value between 1 and 0.05, B is a control enhancement range parameter, and C and D are control gain intensity parameters;

step 5: performing Curvelet-enhancement curved wave inverse transformation on all sub-band coefficients to obtain an enhanced sonar image;

step 6: establishing a deep learning convolutional neural network for the enhanced sonar image subjected to Curvelet-enhancement Curvelet transformation in the step 1-step 5; marking a characteristic region to be identified in the enhanced sonar image through manual screening;

step 7: establishing a convolution layer, and setting a convolution kernel with m multiplied by n when an image is transmitted to the convolution layer

Multiplying each weight in the convolution kernel W with a corresponding pixel X covered in the enhanced sonar image X, and then summing, wherein the formula is as follows:

adding scalar bias to the operation result z to generate output; the convolution layer can extract local features in the picture through the filtering of the convolution kernel;

step 8: performing maximum pooling layer operation on a region of interest, and converting an image passing through a screening region into a feature vector with the size fixed as W multiplied by H, wherein the region of interest is a rectangular window in a convolution feature map; selecting to calculate a segmentation area by using a search algorithm, wherein each region of interest is represented by a quaternary vector (x, y, w, h), wherein (x, y) identifies the upper left corner coordinates and (w, h) represents the height and width of a rectangular window; the region of interest pooling layer divides the region with the size of (W, H) into W multiplied by H grid word windows, each window is (W/W) multiplied by (H/H), then the characteristic value in each sub-window is maximally pooled into a corresponding output network, and after the operation is carried out on each characteristic channel, H is the same as the standard maximum pooling;

step 9: performing multi-task training, simultaneously taking target classification and candidate region frame regression calculation as two parallel output layers, outputting the distribution probability of each region of interest on K+1 class by the first task, wherein K is the number of target classification, adding the class of background, and calculating the probability by using a softmax function; the second task is to calculate the regression offset of the bounding box

The multitasking loss function is combined training classification and candidate region frame regression calculation:

L(p,u,t ^u ,v)＝L _cls (p,u)+λ[u≥1]+L _loc (t ^u ,v)

the parameter u marks the real category of the candidate region content as a target, u is more than or equal to 1, and u=0 represents the region content as a background; l (L) _cis (p,u)＝-logP _u For the loss function corresponding to category u, L _loc (t ^u V) by smoothing L ₁ Loss function calculation of the loss function of the frame position, t ^u Is the frame of category u prediction; square brackets [ u is greater than or equal to 1]If the mark meets the condition u in the square frame, the mark is 1, otherwise, the mark is 0; the parameter lambda controls the balance between the two loss functions;

step 10: performing parameter back propagation on the convolutional neural network by using a gradient descent method; in the training process, the whole data set is completely sent into a network model for training and learning, so that the network calculates an iteratively updated gradient value by using all samples; the convolution characteristics are processed through a region-of-interest pooling layer, and the obtained characteristics are sent to two parallel computing tasks for training, classification and positioning regression;

step 11: through repeated iterative computation, the loss function value is smaller than 0.1, and a trained deep learning neural network is obtained;

step 12: repeatedly training the deep learning neural network obtained in the step 6-10 by utilizing the enhanced sonar image database obtained in the step 1-5;

step 13: counting the accuracy of the recognition result obtained through training; if the accuracy is low, repeating the step 1, and increasing diversity of the sonar collected samples; repeating the steps 1-10 on a sonar sample acquired by the front vision multi-beam imaging sonar carried by the unmanned ship;

step 14: and displaying the target identification type value at the display front end of the unmanned ship.

Images