CN110826575A - Underwater target identification method based on machine learning - Google Patents

Abstract

An underwater target recognition method based on machine learning belongs to the technical field of underwater machine vision detection and processing. The core of the underwater target recognition algorithm is the SSD target detection algorithm, which adopts a feedforward convolutional network structure, and uses convolution boxes of different sizes between different layers to obtain feature maps of different scales, and then performs regression according to the feature map. The results are obtained through the non-maximum value suppression algorithm. The SSD algorithm adopts multi-scale and anchor point methods to solve the low-precision problem of region proposal, and uses multi-scale feature vectors, which greatly improves the good effect on both small and large targets. , which is of great help to improve the overall recognition accuracy, and can obtain more accurate location information. The non-maximum suppression algorithm can not only realize the detection of the target, but also greatly improve the recognition accuracy of the underwater target, provide effective visual information for the underwater robot to observe and operate the underwater target, and improve the underwater target recognition accuracy. Intelligent recognition ability.

Description

An underwater target recognition method based on machine learning

technical field

The invention belongs to the technical field of underwater machine vision detection, in particular to an underwater target recognition method based on machine learning.

Background technique

The ocean covers most of the earth's area and possesses great resources and mysteries. Since modern times, with the advancement of marine equipment research and development technology, human beings have begun to further understand and develop marine resources, and all countries and regions in the world have begun to spare no effort in the research and development of marine equipment and the exploitation of underwater resources. my country has a coastline of nearly 20,000 kilometers. At the same time, the coastal economic sea areas are rich in marine resources, with good ocean development conditions and high demand. As one of the underwater sensing equipment, computer vision equipment is increasingly used. Into ocean exploration. Its complete vision system covers many disciplines and technologies such as optics, computer science control theory, etc., and has been widely used in underwater detection, operation and manned equipment. Among them, vision-based underwater tracking and identification technology has very important research value.

The underwater robot is equipped with visual equipment to better perceive the underwater environment, and the single-purpose installation can save more space for the underwater equipment, which provides effective conditions for the precise and maneuverable operation of the underwater equipment. The related research of the present invention is intended to provide target and space information for underwater equipment operation, underwater robot obstacle avoidance, path planning, etc. Therefore, the underwater monocular vision technology has practical research and practical application value.

The main shortcomings of the current underwater target recognition algorithms are:

First of all, the accuracy of the current underwater visual detection is not enough. The traditional recognition algorithm is not accurate enough for the target positioning. Therefore, it will have a great impact on the target detection. In the positioning, it is found that the positioning accuracy shows an oscillatory convergence and runs through In the whole process, this causes the blur of underwater imaging and affects the recognition accuracy of the target.

At present, there are two main types of algorithms based on machine learning, one is a two-stage method such as R-CNN, the other is one-stage, and SSD belongs to the latter. Compared with Yolo, the SSD algorithm has both accuracy and speed. Much better than that. Compared with Yolo, SSD uses CNN to detect directly, instead of doing detection after the fully connected layer like Yolo. In fact, the use of convolution for direct detection is only one of the differences between SSD and Yolo. There are also two important changes. One is that SSD extracts feature maps of different scales for detection. Large-scale feature maps (more advanced Feature map) can be used to detect small objects, while small-scale feature maps (later feature maps) are used to detect large objects; second, SSD uses prior boxes (Prior boxes, Default boxes of different scales and aspect ratios) , called anchors in Faster R-CNN). The disadvantage of the Yolo algorithm is that it is difficult to detect small targets and the positioning is inaccurate, but these important improvements enable SSD to overcome these shortcomings to a certain extent.

Compared with traditional CNN, neural network, that is, a network with input layer, output layer and multiple hidden layers, belongs to a subfield of machine learning. The main principle is that after the training data enters the input layer, it is processed by multiple hidden layers, and after a lot of training, some features of the input are extracted, and the model is used to classify or predict in practice. . Traditional methods have several problems in underwater target recognition, such as slow processing speed and low recognition rate.

Introduced SSD, a single-shot detector for multiple classes, which is faster than previous state-of-the-art detectors such as single-shot detectors (yolo) and has similar accuracy compared to recognition networks such as faster-Rcnn degrees, but faster. The core of SSD is to predict class scores and box offsets for a set of default bounding boxes using a small volume filter applied to the feature map. These design features lead to simple end-to-end training and high accuracy, even with low-resolution input images, which can meet the unsharp and low-resolution situations in underwater environments, further improving the speed-accuracy trade-off.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an underwater target recognition method based on machine learning, which can detect underwater target information more accurately.

The object of the present invention is achieved in this way:

An underwater target recognition method based on machine learning, comprising the following steps:

Step 1: Preprocess the underwater target image, and perform retinex restoration on the image to obtain the input image;

Step 2: Convolution between different layers using convolution boxes of different sizes to obtain feature maps of different scales;

Step 3: Perform regression according to the feature map obtained in step 2, normalize each feature map, and then obtain defualt boxes of different scales and offsets through detectors and classifiers of different sizes;

Step 4: Obtain the final result through the NMS suppression algorithm;

In the step 2, the region results obtained by the RPN calculation in the region selection algorithm are used, and the feature maps of different sizes are obtained by dividing them according to different scales, and then the RPN convolution kernel is used to move the upper region and obtain the confidence value of the region, A matrix with confidence is obtained by continuously moving the defualt boxes of different scales.

The width and height of the defualt box in the step 3 are:

和

分别为defualt box的宽和高，尺度采用a_r∈{1,2,1/2,3,1/3}，为不同层的尺度，其中m为个数，默认的尺度是通过m个各不相同的尺度特征图预测来完成，S_min为最小的尺度，S_max为最大的特征图的尺度。in

and

are the width and height of the defualt box, respectively, and the scale adopts a _r ∈ {1,2,1/2,3,1/3}, is the scale of different layers, where m is the number, and the default scale is completed by predicting m different scale feature maps, S _min is the smallest scale, and S _max is the largest feature map scale.

The center position of the selection box in the step 3 is:

where x and y represent the horizontal frame and the center point of the vertical axis, respectively, |f _k | represents the scale of the k-th feature map, ij∈[0,f _k ].

The method for NMS suppression in the step 4 includes:

Step 4.1: Arrange the elements in the matrix according to the conf size;

Step 4.2: Perform IoU calculation on all the intersection areas in descending order according to the results calculated in Step 4.1, and set the Th value, compare the IoU in order, and classify and classify according to its size;

Step 4.3: Go back to step 4.2 at the second largest frame position in the column;

Step 4.4: Repeat step 4.3 until all defualt boxes in this column are executed;

Step 4.5: After executing the traversal of the matrix, that is, executing all types of NMS;

Step 4.6: Perform further residual screening, and select all remaining categories based on confidence.

The loss function of the entire model in step 4 is

表示第i个盒是否与第p类物体的第j个目标边框相匹配，匹配为1，反之为0；若表示对于第j个目标边界框至少有一个盒与之匹配；N表示匹配和的数量；

用来衡量识别的性能；用来衡量边界框预测性能；其中

表示第j个目标的真实目标边框与特征抓取盒的边框之间的偏差，m∈{cx,cy,w,h}，其中(cx,cy)表示边框中心点坐标，(w,h)表示边框的宽和高。where x is used to judge whether the designed feature grab box has a corresponding target,

Indicates whether the i-th box matches the j-th target frame of the p-th object, the match is 1, otherwise it is 0; if Indicates that at least one box matches the j-th target bounding box; N represents the number of matching sums;

used to measure the performance of recognition; used to measure the bounding box prediction performance; where

Represents the deviation between the real target frame of the j-th target and the frame of the feature grab box, m∈{cx,cy,w,h}, where (cx,cy) represents the coordinates of the center point of the frame, (w,h) Indicates the width and height of the border.

The beneficial effects of the present invention are:

(1) The present invention adopts the method of multi-scale and anchor points to solve the low-precision problem of region proposal, and the multi-scale feature vector used greatly improves the good effect on both small targets and large targets, and at the same time The improvement of the overall recognition accuracy is of great help, and more accurate location information can be obtained compared to the previous suggested methods;

(2) This algorithm performs 20,000 iterations to calculate the error curve according to the loss function. The initial error is defined as 500, that is, the convergence range is (0,500), and the final convergence is about 20. The error rate is about less than 1%. It can be seen that the The algorithm has greatly improved in accuracy.

Description of drawings

Figure 1 is a diagram of the realization process of multi-scale detection;

Figure 2 shows the area selection method;

Fig. 3 is the flow chart of NMS algorithm;

Fig. 4 is the calculation diagram of stacking ratio;

Fig. 5 is the detection diagram of single fish and multiple fish under the multi-angle situation;

Figure 6 is a graph of error variation.

Detailed ways

Below in conjunction with the content of the invention, a detailed implementation and effects of the present invention are illustrated by the following examples.

Aiming at the deficiencies of the current prior art, the present invention aims to provide an underwater target detection algorithm with high reliability and good real-time performance, which can detect underwater target information more accurately. The algorithm can meet the needs of underwater observation and operation, and provide accurate identification of underwater targets for underwater robots. The invention can greatly improve the recognition accuracy of the underwater target, provide effective visual information for the underwater robot to observe and operate the underwater target, and improve the intelligent recognition ability of the underwater target.

Implementation 1: As shown in Figure 1, the invention implements an underwater target recognition algorithm based on machine learning according to the needs of underwater observation and operations. The core of the algorithm is the SSD target detection algorithm. SSD is a very high-precision target detection method. It uses a feedforward convolutional network structure. First, the underwater target image is preprocessed, and the image is retinex restored. Obtain the input image, and then use convolution boxes of different sizes between different layers to obtain feature maps of different scales, and then perform regression according to the obtained feature maps. Finally, the final result is obtained by the NMS suppression method, SSD The method of target detection uses multi-scale and anchor points to solve the low-precision problem of region proposal. The multi-scale feature vector used in it greatly improves the good effect of both small and large targets. At the same time, it is of great help to improve the overall recognition accuracy, and more accurate location information can be obtained compared to the previous suggested methods. The next convolutional network uses convolution templates of different scales for feature fusion, and then obtains default boxes with different scales and offsets. Finally, the final detection and classification results are achieved by adding an extreme value suppression algorithm. The specific steps are to use the standard VGG-16 network, perform feature detection through convolution templates of different sizes, obtain different feature maps, normalize each feature map, and then pass detectors and classifiers of different sizes. Then, defualt boxes with different scales and offsets are obtained, and finally the target detection is realized by the non-maximum suppression algorithm.

The implementation process of SSD multi-scale is shown in Figure 1. Here, the front-end network used for feature extraction uses the standard convolutional neural network VGG-16, and the subsequent convolutional networks use different scales of convolution templates to perform Feature fusion, and then obtain default boxes with different scales and offsets, and finally achieve the final detection and classification results by adding an extreme value suppression algorithm.

Implementation 2: As shown in Figure 2, in the convolutional network algorithm of SSD, RPN (Region Proposal Network) can be used to calculate the obtained regional results, and divided according to different scales to obtain a feature map size of 38 × 38 × 512 , 19 × 19 × 1024, 10 × 10 × 512, 5 × 5 × 256, 3 × 3 × 256, 1 × 1 × 256 and many other areas, according to Figure 2, which uses 5 × 5 × 256 The RPN process is described as an example. It can be seen that k default boxes are generated through different scales and ratios, where k=6 can generate 5 × 5 × 256 convolution boxes and 4 default boxes with bias at the same time. the corresponding confidence class.

The default scale here is done by predicting m different scale feature maps, where the minimum scale is set to S _min = 0.2, and the scale of the largest feature map is set to S _max = 0.95, then you can set the scale according to the corresponding scale. The scale of all layers is obtained by recursion, and the specific calculation is:

Here, a _r is paired according to different ratios, where the scale adopts a _r ∈ {1,2,1/2,3,1/3}, and the width and height of the corresponding default box can be obtained according to a _r .

和

分别为defualt box的宽和高，此外在radio＝1时，需要重新制定尺度为

至此可以得到6种不同尺度的default box。in

and

are the width and height of the defualt box, respectively. In addition, when radio=1, the scale needs to be re-scaled as

So far, 6 default boxes of different scales can be obtained.

At the same time, the center position of the selection box can be obtained by the following formula (3).

Among them, |f _k | represents the scale of the k-th feature map, x and y represent the center point of the horizontal and vertical axes of the frame, respectively, ij∈[0,f _k ].

The NMS suppression algorithm, in the convolutional neural network, cannot locate the real position very accurately. At this time, the method of NMS (Non-maximum suppression) needs to be used to reduce the occurrence of this situation.

Implementation 3: As shown in Figure 3 and Figure 4,

Step1: Arrange the elements in the 8732×21 matrix according to the size of Conf.

Step2: According to the results calculated in the first step from large to small, perform IoU calculation on all the intersection areas, and set the Th value, compare the IoU in order, and classify and classify according to their size.

Step3: Return to Step2 to execute at the second largest frame position in the column.

Step4: Repeat Step3 until all default boxes in this column are executed.

Step5: After executing the traversal of the 8732×21 matrix, all types of NMS are executed.

Step 6: Perform further residual screening, and select all the remaining categories based on confidence.

Features are extracted according to the feature capture box designed on each feature map coordinate point, and these features are used to predict the type and bounding box of the target. Here, a 3*3 convolution kernel is used to extract the special feature in each feature capture box. The convolution box used in each feature map is 3*3*6*(class+4), and 6 is The number of grab boxes at each feature map coordinate point, 4 is the deviation between the predicted target boundary and the target bounding box. If the size of a feature map is m*n, and there are 6 boxes on each coordinate, the output result of m*n*6*(class+4) is finally generated:

Loss function calculation, the loss function of the entire model is as follows:

表示第i个盒是否与第p类物体的第j个目标边框相匹配，匹配为1，反之为0。若

表示对于第j个目标边界框至少有一个盒与之匹配。式中N表示匹配和的数量。公式(5)中的第一部分式用来衡量识别的性能的，主要就是一个多类的softmax损失函数，细节where x is used to judge whether the designed feature grab box has a corresponding target,

Indicates whether the i-th box matches the j-th target bounding box of the p-th object, matching is 1, otherwise it is 0. like

Indicates that for the jth object bounding box at least one box matches it. where N represents the number of matching sums. The first part of formula (5) is used to measure the performance of recognition, mainly a multi-class softmax loss function.

公式(6)中，第二部分式用来衡量边界框预测性能的，用到的损失函数如下公式：in

In formula (6), the second part of the formula is used to measure the prediction performance of the bounding box, and the loss function used is as follows:

表示第j个目标的真实目标边框与特征抓取盒的边框之间的偏差，m∈{cx,cy,w,h}，其中(cx,cy)表示边框中心点坐标，(w,h)表示边框的宽和高。最后边框的位置信息可由下面公式(7)表示：in

Represents the deviation between the real target frame of the j-th target and the frame of the feature grab box, m∈{cx,cy,w,h}, where (cx,cy) represents the coordinates of the center point of the frame, (w,h) Indicates the width and height of the border. The position information of the final border can be represented by the following formula (7):

Meanwhile, the comprehensive confidence loss function of multiple regions can be expressed as:

Implementation 5: As shown in Figure 5, for the multi-angle identification of the underwater environment, for the identification of individual underwater fish, 200 underwater photos of different forms of fish and crabs were collected through the Internet and experimental pool experiments. The network is trained, and experiments are carried out from the aspects of loss function, recognition accuracy and positioning accuracy IOU. Here, the Conf of the target is set to be more than 0.5, and recursive detection is performed to exclude the 1/2 area of the center of the image area, and detection and identification are performed for complex real situations such as dense fish distribution and target intersection. It can more accurately identify the target fish in different poses and different positions, accurately identify the situation of far, near, multi-angle and complex parallel distribution of multiple fish, can realize multi-target recognition, and can achieve accurate identification of some partially occluded fish. identification. After 20,000 iterations, the error curve was calculated according to the loss function, as shown in Figure 6. The initial error is defined as 500, that is, the convergence range is (0,500), and the final convergence is about 20, and the error rate is about less than 1%. It is found in the positioning accuracy graph that the positioning accuracy converges in an oscillating manner and runs through the entire training process, which is caused by the blurring and phantom of underwater imaging. Finally, it is proved that it can well overcome the misrecognition caused by the low recognition accuracy caused by the blur of the existing underwater imaging.

Claims (6)

1. An underwater target identification method based on machine learning is characterized by comprising the following steps:

step1, preprocessing an underwater target image, and performing retinex restoration on the image to obtain an input image;

step2, carrying out convolution by adopting convolution boxes with different sizes and scales among different layers to obtain feature maps with different scales;

step3, performing regression according to the feature maps obtained in the step two, performing normalization processing on each feature map, and obtaining defuelt boxes with different scales and offset through detectors and classifiers with different sizes;

and 4, obtaining a final result through an NMS (network management system) inhibition algorithm.

2. The machine learning-based underwater target recognition method according to claim 1, characterized in that: in the step2, a region result obtained by RPN calculation in a region selection algorithm is adopted, the feature maps with different sizes are obtained by dividing according to different scales, then the RPN convolution core is used for moving the region on the feature map and obtaining the confidence value of the region, and a matrix with the confidence is obtained by continuously moving defualt boxes with different scales.

3. The machine learning-based underwater target recognition method according to claim 1, characterized in that: the width and height of the defuelt box in the step3 are as follows:

whereinAnd

width and height of defualt box, respectively, the scale being a_r∈{1,2,1/2,3,1/3}，

Is the scale of different layers, wherein m is the number, the default scale is realized by m different scale characteristic diagram predictions, S_minIs of the smallest scale, S_maxIs the scale of the largest feature map.

4. The machine learning-based underwater target recognition method according to claim 1, characterized in that: the central position of the selection frame in the step3 is as follows:

wherein x and y represent the horizontal frame, the central point of the longitudinal axis, | f_kI represents the k-th feature graph scale, i.j E [0, | f_k|]。

5. The machine learning-based underwater target recognition method according to claim 1, characterized in that: the method for NMS inhibition in step4 comprises

Step 4.1: arranging elements in the matrix according to the conf size;

step 4.2: IoU calculation is carried out on all the cross areas according to the sequence from large to small of the calculation result in the step 4.1, Th values are set, comparison is carried out on the Th values according to the sequence and IoU, and classification are carried out according to the sizes of the Th values;

step 4.3: the execution returns to the step 4.2 again at the second big frame position of the column;

step 4.4: repeating the step 4.3 until all the defuelt boxes in the row are executed;

step 4.5: performing traversal on the matrix, namely performing NMS of all categories;

step 4.6: and further residual screening is carried out, and all the final residual categories are selected according to the confidence coefficient.

6. The machine learning-based underwater target recognition method according to claim 1, characterized in that: the loss function of the whole model in the step4 is

Where x is used to determine whether the designed feature capture box has a corresponding target,

whether the ith box is matched with the jth target frame of the pth object or not is represented, the matching is 1, and otherwise, the matching is 0; if it is

Indicating that at least one box matches the jth target bounding box; n represents the number of matching sums;

to measure the performance of the recognition;

used to measure bounding box prediction performance; wherein

And (2) representing the deviation between the real target frame of the jth target and the frame of the characteristic grabbing box, wherein m belongs to { cx, cy, w, h }, wherein (cx, cy) represents the coordinate of the center point of the frame, and (w, h) represents the width and the height of the frame.

Images