CN117173552B - Underwater target detection method, system, electronic equipment and storage medium - Google Patents
Underwater target detection method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117173552B CN117173552B CN202311450871.9A CN202311450871A CN117173552B CN 117173552 B CN117173552 B CN 117173552B CN 202311450871 A CN202311450871 A CN 202311450871A CN 117173552 B CN117173552 B CN 117173552B
- Authority
- CN
- China
- Prior art keywords
- underwater target
- scale
- feature map
- convolution
- adaptive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 94
- 238000003860 storage Methods 0.000 title claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 55
- 238000009826 distribution Methods 0.000 claims abstract description 24
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 17
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 17
- 238000000605 extraction Methods 0.000 claims abstract description 16
- 230000008859 change Effects 0.000 claims abstract description 14
- 238000000034 method Methods 0.000 claims description 43
- 238000012937 correction Methods 0.000 claims description 41
- 230000003044 adaptive effect Effects 0.000 claims description 34
- 238000010586 diagram Methods 0.000 claims description 24
- 238000011176 pooling Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 6
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 18
- 230000008569 process Effects 0.000 description 18
- 238000012549 training Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 5
- 230000004927 fusion Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 241000251511 Holothuroidea Species 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 241000258957 Asteroidea Species 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 2
- 241000237503 Pectinidae Species 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 235000020637 scallop Nutrition 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an underwater target detection method, an underwater target detection system, electronic equipment and a storage medium, and belongs to the technical field of target detection. Comprises the steps of acquiring an underwater target image; inputting the underwater target image into a target detection model for processing to obtain an underwater target detection result; inputting an underwater target image into a target detection model for processing, and specifically comprising the following steps: inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images; inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a plurality of multi-scale feature maps; respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target; and the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive manner. The underwater target detection precision can be improved, and the problem of low underwater target detection precision is solved.
Description
Technical Field
The present invention relates to the field of target detection technologies, and in particular, to a method, a system, an electronic device, and a storage medium for detecting an underwater target.
Background
The statements in this section merely relate to the background of the present disclosure and may not necessarily constitute prior art.
The underwater image has the characteristics of low color distortion and low contrast, which weakens the characteristic information of the underwater target, thereby causing low detection precision of the underwater target. In addition, underwater targets are often densely distributed and have extremely large size differences, which further reduces detection accuracy.
The traditional underwater target detection network is difficult to adapt to the underwater targets with large scale difference, and the characteristics are processed by adopting the fixed convolution weights at each level. The constant convolution weight cannot adapt to input features with multiple scales, and has poor feature transformation capability, so that the detection precision of the underwater targets which are difficult to detect and have multiple scales and are dense is low.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides an underwater target detection method, an underwater target detection system, electronic equipment and a computer readable storage medium, wherein a weight self-adaptive correction layer is embedded in a plurality of layers, underwater target characteristics can be enhanced in a plurality of layers of a network, a double-branch self-adaptive detection module is designed, convolution weights which adaptively change along with multi-scale characteristics are obtained, the underwater target detection method, the system, the electronic equipment and the computer readable storage medium can adapt to underwater targets of various scales, and the underwater target detection precision is greatly improved.
In a first aspect, the present invention provides a method for detecting an underwater target;
an underwater target detection method comprising:
acquiring an underwater target image;
inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result;
the method for processing the underwater target image comprises the steps of:
inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images;
inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a plurality of multi-scale feature maps;
respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target;
the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive mode.
Further, the inputting the underwater target image into the feature extraction frame for processing specifically includes:
and inputting the underwater target image into a cascade convolution layer and a plurality of groups of stacked residual self-adaptive correction units for processing, and obtaining the final output characteristics of each group of residual self-adaptive correction units to serve as an underwater target characteristic diagram.
Preferably, the residual adaptive correction unit carries out convolution processing on the input features through a first convolution layer to obtain residual features; carrying out average pooling operation on residual error characteristics to obtain compression characteristics; and respectively encoding the compression features in the space dimension and the channel dimension, acquiring and fusing the space encoding features and the channel encoding features, and acquiring a convolution kernel calibration chart to calibrate the convolution kernel of the second convolution layer.
Further, the inputting the underwater target feature map into the multi-scale target distribution layer for processing comprises:
carrying out up-sampling treatment on the underwater target feature map, fusing the underwater target feature map after the up-sampling treatment with the underwater target feature map with the same scale, and obtaining a corresponding multi-scale feature map after the self-adaptive correction layer treatment;
based on the input underwater target feature map, a convolution weight correction map is obtained so as to adaptively correct the convolution weight of the convolution layer along with the change of feature information.
Further, the multi-scale target distribution layer comprises a plurality of parallel multi-scale target distribution branches, and the multi-scale target distribution branches comprise an adaptive correction layer and a convolution layer which are connected in sequence;
the self-adaptive correction layer comprises a global average pooling layer and a plurality of full-connection layers which are sequentially connected.
Furthermore, the parallel input of the multi-scale feature images to the dual-branch self-adaptive detection module for processing is specifically as follows:
and respectively inputting the multi-scale feature map into a category prediction branch and a position prediction branch for processing, and obtaining a category prediction result and a position prediction result of the underwater target in parallel.
Preferably, the category prediction branch and the position prediction branch generate self-adaptive convolution weights through convolution kernel synthesis factors which adaptively change along with multi-scale characteristics, and an adaptive convolution layer is constructed; based on the multi-scale feature map, the category prediction result and the position prediction result of the underwater target are obtained in parallel through the self-adaptive convolution layer.
In a second aspect, the present invention provides an underwater target detection system;
an underwater target detection system comprising:
an acquisition module configured to: acquiring an underwater target image;
an underwater target detection module configured to: inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result;
the method for processing the underwater target image comprises the steps of:
inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images;
inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a multi-scale feature map;
respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target;
the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive mode.
In a third aspect, the present invention provides an electronic device;
an electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of the above-described underwater target detection method.
In a fourth aspect, the present invention provides a computer-readable storage medium;
a computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of the above-described underwater target detection method.
Compared with the prior art, the invention has the beneficial effects that:
1. the technical scheme provided by the invention comprises three parts: (1) And the feature extraction frame is used for extracting feature information of the underwater target. Enhancing the feature expression in the layer by designing a residual error self-adaptive correction unit; (2) And the multi-scale target distribution layer is used for distributing the underwater targets with large scale difference to different characteristic layers, so as to avoid the omission phenomenon caused by the characteristic layers with single scale. The layer combines feature fusion and self-adaptive correction layer to enrich feature information. (3) And the position regression and category prediction double-branch self-adaptive correction module is used for mapping the characteristic information generated by the multi-scale target distribution layer into a detection result. The adaptive convolution kernel is obtained in the layer by designing a dual-branch adaptive correction module to enhance the feature expression.
2. According to the technical scheme provided by the invention, the weight self-adaptive correction layers are embedded in a plurality of layers, the underwater target characteristics can be enhanced in a plurality of layers of the network, the double-branch self-adaptive detection module is designed, the convolution weight which is adaptively changed along with the multi-scale characteristics is obtained, and the underwater target detection precision can be greatly improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
Fig. 1 is a schematic diagram of a network overall architecture according to an embodiment of the present invention;
fig. 2 (a) is a schematic overall architecture diagram of a feature extraction framework provided by an embodiment of the present invention, and fig. 2 (b) is a schematic architecture diagram of a residual adaptive correction unit provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a multi-scale target distribution layer according to an embodiment of the present invention;
fig. 4 is a schematic architecture diagram of a dual-branch adaptive detection module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an adaptive convolutional layer architecture according to an embodiment of the present invention;
FIG. 6 is an exemplary diagram of a relationship between a predicted frame and a real frame according to an embodiment of the present invention;
FIG. 7 is an exemplary diagram of an underwater target detection result provided in an embodiment of the present invention;
fig. 8 (a) is a diagram of examples of dense underwater target detection results provided by the embodiment of the present invention, and fig. 8 (b) is a diagram of examples of underwater target detection results provided by the embodiment of the present invention with different dimensions.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular forms also are intended to include the plural forms, and furthermore, it is to be understood that the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, such as, for example, processes, methods, systems, products or devices that comprise a series of steps or units, are not necessarily limited to those steps or units that are expressly listed, but may include other steps or units that are not expressly listed or inherent to such processes, methods, products or devices.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1
In the prior art, the underwater image has the defects of low color distortion, low contrast and the like, and the underwater targets are generally densely distributed and have extremely large size difference, so that the existing constant-weight underwater target detection network has low detection precision; therefore, the invention provides an underwater target detection method, which combines image characteristics to carry out self-adaptive correction on convolution kernel weights, and improves the detection precision of the underwater target.
Next, a detailed description will be given of an underwater target detection method disclosed in this embodiment with reference to fig. 1 to 8 (b). The underwater target detection method comprises the following steps:
s1, acquiring an underwater target image.
S2, inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result; the method specifically comprises the following steps:
s201, inputting the underwater target image into a feature extraction frame for processing, and obtaining a plurality of underwater target feature graphs with different scales.
Further, the feature extraction framework is used for converting the input underwater target image into an underwater target feature map.
Because the scale difference of the underwater targets is large and the definition of the underwater images is low, the characteristic extraction frame is designed to be a structure with a plurality of modules stacked from top to bottom, and the residual self-adaptive correction unit is innovatively designed to enhance the characteristic expression. After the image characteristics of the underwater target image are processed by a plurality of residual error self-adaptive correction units, the resolution ratio is gradually reduced.
Selecting the characteristic composition output characteristics of the last convolution layer of the residual error adaptive correction unit of different layersExpressed as:
（1）
wherein,Xrepresenting an input image of the underwater target,、/>、/>and->Respectively representing the underwater target feature graphs output by the feature extraction frame, outputting the underwater target feature graphs to a multi-scale target distribution layer, and representing scale levels by subscripts; />、/>、/>、/>Respectively representing the characteristic diagram +.>、/>、/>And->Corresponding residual adaptive correction unitSum up;、/>、/>、/>the convolution layers of 1×1 are respectively represented, the outputs are 256 channels, and the number of channels for unifying the output characteristics of each level is used.
In order to improve the propagation efficiency of the feature information, the feature extraction framework is designed as a residual adaptive correction unit stacked architecture, and as shown in fig. 2 (a), the framework includes 1 convolution layer and 4 groups of stacked residual adaptive correction units, each group includes 3 residual adaptive correction units.
The specific structure of the residual adaptive correction unit is shown in fig. 2 (b), in the residual adaptive correction unit, the output characteristic and the input characteristic are directly connected through jump connection, so as to prevent gradient dispersion and improve the network training efficiency.
Specifically, the input features areConvolution layer, batch normalization layer, reLu function, +.>The convolution layer and the batch normalization layer are connected with the acquired output features and input features in a jumping manner, and the final output features are obtained after the ReLu function.
To enhance feature expression, the present embodiment generates correction factors in both the convolution kernel channel and the spatial dimensions, for the secondThe convolution layer performs an adaptive convolution to enhance the feature expression. The specific flow is as follows:
reducing residual characteristics using averaging pooling operationsResolution of (2) to->Expressed as:
（2）
in the method, in the process of the invention,T c in order to be a characteristic diagram of an underwater target,PJC(.) is an average pooling operation,characteristic of compression->Representing the compressed feature size, wherein,Dindicating the number of channels.
The compressed features are then used to generate a compressed imageRearrange to +.>Is then rearranged with a 1X 1 convolutional layer>Coding in the spatial dimension:
（3）
in the method, in the process of the invention,for coding features in spatial dimension, +.>For a convolution operation of 1×1, +.>
Is the size of the encoded features in the spatial dimension.
In addition, features to be compressedTransposed into->Also using a 1 x 1 convolution to encode the features in the channel dimension:
（4）
in the method, in the process of the invention,for coding features in channel dimension, +.>For a convolution operation of 1 x 1,Tthe transpose of the matrix is represented,is the size of the encoded features in the channel dimension.
Finally, the coding features of the spatial dimensionCoding features of the channel dimension->Adding to obtain a convolution kernel calibration graph:
（5）
in the method, in the process of the invention,Sigmoid(.) represents a Sigmoid function, jz is an obtained convolution kernel correction diagram, the convolution kernel correction diagram is used to calibrate the convolution kernel of the 3 x 3 convolution layer to obtain an adaptive convolution kernel,the size of the chart is corrected for the convolution kernel.
S202, inputting the underwater target feature map into a multi-scale target distribution layer for processing, and obtaining a plurality of multi-scale feature maps.
Further, the multi-scale target distribution layer is used for detecting underwater targets with different scales. Thus, the multi-scale target distribution layer is designed to up-sample the underwater target feature map andthe same-level underwater target feature graphs in (a) are fused, and are expressed as follows:
（6）
（7）
（8）
（9）
where, U (-) represents upsampling,、/>、/>、/>
respectively representing 3×3 convolution layers for feature transformation, P 2 、P 3 Respectively, the characteristics of the intermediate layer.
From equation (9), the bottom layer of the multi-scale target distribution layerThe features contain high-level semantic features, so that the designed multi-scale target distribution layer can enrich the feature expression of multiple levels and improve the target detection precision. The detailed structure of the multi-scale target distribution layer is shown in fig. 3, and four-level multi-scale characteristic diagrams output by the multi-scale target distribution layer are respectively expressed as、/>、、/>Underwater target feature map output from feature extraction frame +.>、/>、/>、/>Corresponding to the above.
The embodiment innovatively designs the adaptive correction layer as a combination of global average pooling and full-connection layers, and realizes the adaptive convolution by generating a weight correction chart.
In the view of figure 3 of the drawings,QPCrepresenting a global average pooling layer,QL 1 ~ QL 8 indicating that the full-link layer is to be formed,QPCand (3) withQL 1 ~ QL 8 The structure is used for generating a weight correction chart, correcting a 3 multiplied by 3 convolution layer, and obtaining the self-adaptive convolution weight which changes along with the characteristic information so as to enrich the characteristic information of the underwater target.
Exemplary, for a characteristic map of an underwater targetUp-sampled and then compared with another underwater target characteristic diagram +.>Fusion, global average pooling layerQPCFull connection layerQL 3 Full connection layerQL 4 And processing, namely generating a weight correction chart and inputting a convolution layer of 3 multiplied by 3 to carry out convolution weight correction.
Finally, performing 3×3 adaptive convolution operation on the fused features to obtain a multi-scale feature map output by the multi-scale target distribution layer。
S203, respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target; the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive mode.
Furthermore, the convolution kernels used by the conventional underwater target detection network are fixed values, namely, the multi-scale features share fixed convolution kernels. The mode of sharing convolution kernels by the multi-scale features weakens the feature expression and reduces the underwater target detection precision.
Therefore, the present embodiment innovatively designs a dual-branch adaptive detection module, whose core is an adaptive convolution layer. The two information flows of the module are used for predicting category information, position information and deviation degree information of the underwater target, and the self-adaptive convolution layer is used for adaptively distributing different convolution kernels to the features of different scales so as to enhance feature expression and improve the detection precision of the underwater target.
The dual-branch adaptive detection module is shown in fig. 4, and the class prediction branch comprises 4 adaptive convolution layers and is used for outputting class information of the underwater target. The category information is a three-dimensional matrix, the number of channels is 4, and the probability that the output result belongs to one of the 4 categories is indicated. The position prediction branch comprises 5 self-adaptive convolution layers, and outputs position and deviation degree information of the underwater target. The position information is also a three-dimensional matrix, the number of channels is 4, and the distance from the output 4 bounding boxes to the central point of the underwater target is represented. The degree of deviation is a matrix with a channel number of 1, and represents the degree to which the bounding box deviates from the center point.
Multi-scale feature map to be inputSequentially inputting the two branches of the self-adaptive detection modules for processing.
Further, the adaptive convolution layer is configured to adaptively allocate convolution kernels to features of different scales, expressed as:
(10)
in the method, in the process of the invention,is a double-branch self-adaptive detection module +.>For the output detection information (generic term for category, position, degree of deviation), +.>For the input multi-scale feature map, each multi-scale feature map of the multi-scale feature map is sequentially input into a double-branch self-adaptive detection module,nrepresenting the first in a multiscale feature mapnLevel of->The convolution weights are represented, namely, the dynamic self-adaptive convolution weights, and the different convolution weights are distributed to different feature layers in the multi-scale features.
In order to obtain the self-adaptive convolution weight, the embodiment learns the convolution kernel synthesis factor from the multi-scale feature map, combines the Maxwell expansion formula to carry out nonlinear fusion on a single convolution kernel, achieves the purpose that the convolution kernel changes along with the multi-scale feature self-adaptation, and is expressed as follows:
（11）
wherein,、/>、/>respectively represent a convolution layer which learns different synthesis factor pairs convolution weight from the multi-scale feature map>Nonlinear fusion is performed, so that adaptive convolution weights are obtained.
Based on the design principle, an adaptive convolution layer is constructed, as shown in fig. 5, and the following steps are specifically executed, as shown in formula (12) to formula (16):
（12）
where QPC (-) represents global averaging pooling for mapping input multi-scale featuresFeature information compressed into one dimension->，/>Representation->Is +.>。
Furthermore, a 1×1 convolution operation is designedAnd->For->Executing feature transformation, establishing connection among feature information channels, and acquiring convolution kernel synthesis factors, wherein the convolution kernel synthesis factors are expressed as follows:
（13）
（14）
in the method, in the process of the invention,for->Characteristic information obtained after performing the first 1 x 1 convolution operation,/and>is thatSize of->For the learned convolution kernel synthesis factor, which varies with the level of the multi-scale feature map, sgmd (.) represents the Sigmoid function, +.>Is the size of the convolution kernel synthesis factor.
Finally, combining the Maclalin expansion formula and adopting convolution kernel synthesis factorsNonlinear fusion is carried out on the convolution kernel K to obtain a new convolution kernel +.>：
（15）
In the method, in the process of the invention,for the static convolution kernel created, +.>As the level of the multi-scale feature changes. Thus (S)>And also changes along with the hierarchy of the multi-scale features, namely the dynamic self-adaptive convolution kernel.
Finally, to further enhance the feature expression, a 1×1 convolution operation is designed to calibrate the input features and employ a dynamic adaptive convolution kernelConvolving the calibrated features to obtain output information:
（16）
in the method, in the process of the invention,for bias term, reLu { } represents ReLu activation function, BN { } represents bulk normalization layer, +.>Is a convolution operation of 1 x 1, ">"represents the corresponding multiplication of matrix elements,">To output informationDimension H is height, W is width, and C is channel number.
Further, in some embodiments, the target detection model is trained by using a loss function, and the output value of the underwater target detection network proposed in this embodiment is a target class, a target position, and a deviation, so the loss function is designed as a sum of loss values of three pieces of information, and is expressed as:
（17）
in the method, in the process of the invention,predicting loss for category->For deviation loss, ++>
In order to achieve the target position loss,and->And respectively adjusting factors for adjusting the weights of the three terms in the loss function, wherein the weights are respectively set to be 1.5 and 0.5.
(a) Loss of positionExpressed as:
（18）
in the formula, X represents an input image,representing all weights to be trained, +.>Representing the number of positive samples in the training data, +.>For predicted underwater target coordinates +.>For the reference value of the coordinates of the underwater target, four-dimensional vectors are usedRepresentation, defined as:
（19）
in the method, in the process of the invention,representing the length of the upper border of the underwater object from the center of the object> Indicating the length of the lower border of the underwater object from the center of the object, < +.>Represents the length of the left border of the underwater object from the center of the object, < >>Representing the length of the right frame of the underwater target from the center of the target; (x, y) is the pixel coordinates of the output information,>representing coordinates of upper left corner point of real frame in nth layer feature, < ->And the coordinates of the right lower corner point of the real frame in the nth layer of characteristics are represented.
In the formula (18), the number of the symbols,indicating function->Is a reference value for a category. When->If the function value is 1, otherwise, the function value is 0./>Representing the cross-ratio loss function, defined as:
（20）
wherein E represents a predicted underwater target frame, D represents a reference value of the target frame, L is a minimum frame of envelopes D and E, and the coordinates of E areD is +.>The relationship of which can be seen with reference to fig. 6.
(b) Degree of deviation lossExpressed as:
（21）
in the method, in the process of the invention,for the predicted degree of deviation +.>The reference value for the degree of deviation is expressed as:
（22）
in the method, in the process of the invention,the value range is 0, 1 as deviation degree information]。
In the formula (21)Representing a cross entropy loss function, expressed as:
（23）
(c) Class prediction lossThe definition is as follows:
（24）
in the method, in the process of the invention,class probability representing network output, +.>In order to output the pixel coordinates of the information,for reference value of category->For the Focal Loss function, expressed as:
（25）
in the method, in the process of the invention,and->As a regulatory factor, when->>When 0, the loss function assigns a larger weight to the indistinguishable samples, otherwise assigns a smaller weight to the easily separable samples, +.>And->Set to 0.3 and 1, respectively.
Then, the underwater robot is adopted to capture a training set disclosed by the large race as training data, a small-batch gradient descent algorithm is adopted to optimize the network weight provided by the invention, the number of samples input each time is 32, and the total iterative training is 2 ten thousand times. In the training process, the weight attenuation coefficient of the optimizer is set to be 0.0002, the initial learning rate is set to be 0.01, and when the iteration times reach 1 ten thousand times and 1.5 ten thousand times, the learning rate is attenuated to be 10% of the original value. The patent trains the network by adopting input images with various scales, and the short side size of the input images is a random integer of a [448, 640] interval.
The training network is tested by adopting an image library containing four kinds of underwater targets of sea cucumbers, scallops and starfish, the test results are shown in fig. 7, 8 (a) and 8 (b), fig. 7 shows that on a test set disclosed by the underwater robot grabbing a large race, the detection precision of the proposed network on the sea cucumbers, the scallops, the sea cucumbers and the starfish respectively reaches 84.4%, 70.2%, 78.7% and 85.1%, and fig. 8 (b) shows that the proposed network can accurately detect the underwater targets with different scales, and the proposed network also has a better detection effect on the dense underwater targets as shown in fig. 8 (a).
Example two
The embodiment discloses an underwater target detection system, comprising:
an acquisition module configured to: acquiring an underwater target image;
an underwater target detection module configured to: inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result;
the method for processing the underwater target image comprises the steps of:
inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images;
inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a multi-scale feature map;
respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target;
the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive mode.
It should be noted that, the above-mentioned obtaining module and the underwater target detecting module correspond to the steps in the first embodiment, and the above-mentioned modules are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above may be implemented as part of a system in a computer system, such as a set of computer-executable instructions.
Example III
The third embodiment of the invention provides an electronic device, which comprises a memory, a processor and computer instructions stored on the memory and running on the processor, wherein the steps of the underwater target detection method are completed when the computer instructions are run by the processor.
Example IV
A fourth embodiment of the present invention provides a computer readable storage medium storing computer instructions that, when executed by a processor, perform the steps of the above-described underwater target detection method.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing embodiments are directed to various embodiments, and details of one embodiment may be found in the related description of another embodiment.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. An underwater target detection method, comprising:
acquiring an underwater target image;
inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result;
the method for processing the underwater target image comprises the steps of:
inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images;
inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a plurality of multi-scale feature maps;
respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target;
the step of inputting the underwater target feature map into the multi-scale target distribution layer for processing comprises the following steps:
performing up-sampling treatment on the underwater target feature map, fusing the underwater target feature map after the up-sampling treatment with the underwater target feature map with the same scale, and performing convolution layer treatment after convolution weight correction to obtain a corresponding multi-scale feature map;
based on the input underwater target feature map, acquiring a convolution weight correction map so as to adaptively correct the convolution weight of a convolution layer along with the change of feature information;
the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive manner; the multi-scale feature map is respectively input into the double-branch self-adaptive detection module in parallel for processing specifically:
respectively inputting the multi-scale feature map into a category prediction branch and a position prediction branch for processing, and obtaining a category prediction result and a position prediction result of the underwater target in parallel;
the category prediction branches and the position prediction branches are combined with a Maxwell expansion formula through convolution kernel synthesis factors which adaptively change along with multi-scale characteristics to generate adaptive convolution weights, and an adaptive convolution layer is constructed; based on the multi-scale feature map, the category prediction result and the position prediction result of the underwater target are obtained in parallel through the self-adaptive convolution layer.
2. The underwater target detection method as claimed in claim 1, wherein the inputting the underwater target image into the feature extraction frame for processing is specifically:
and inputting the underwater target image into a cascade convolution layer and a plurality of groups of stacked residual self-adaptive correction units for processing, and obtaining the final output characteristics of each group of residual self-adaptive correction units to serve as an underwater target characteristic diagram.
3. The underwater target detection method as claimed in claim 2, wherein the residual adaptive correction unit performs convolution processing on the input feature through a first convolution layer to obtain a residual feature; carrying out average pooling operation on residual error characteristics to obtain compression characteristics; and respectively encoding the compression features in the space dimension and the channel dimension, acquiring and fusing the space encoding features and the channel encoding features, and acquiring a convolution kernel calibration chart to calibrate the convolution kernel of the second convolution layer.
4. The underwater target detection method as claimed in claim 1, wherein the multi-scale target allocation layer comprises a plurality of parallel multi-scale target allocation branches, the multi-scale target allocation branches comprising an adaptive correction layer and a convolution layer connected in sequence;
the self-adaptive correction layer comprises a global average pooling layer and a plurality of full-connection layers which are sequentially connected.
5. An underwater target detection system, comprising:
an acquisition module configured to: acquiring an underwater target image;
an underwater target detection module configured to: inputting the underwater target image into a target detection model for processing so as to obtain an underwater target detection result;
the method for processing the underwater target image comprises the steps of:
inputting the underwater target image into a feature extraction frame for processing to obtain a plurality of underwater target feature images;
inputting the underwater target feature map into a multi-scale target distribution layer for processing to obtain a multi-scale feature map;
respectively inputting the multi-scale feature images into a double-branch self-adaptive detection module in parallel for processing to obtain a category prediction result and a position prediction result of the underwater target;
the step of inputting the underwater target feature map into the multi-scale target distribution layer for processing comprises the following steps:
performing up-sampling treatment on the underwater target feature map, fusing the underwater target feature map subjected to the up-sampling treatment with the underwater target feature map of the same scale, and obtaining a corresponding multi-scale feature map after being subjected to convolution layer treatment after convolution weight correction;
based on the input underwater target feature map, acquiring a convolution weight correction map so as to adaptively correct the convolution weight of a convolution layer along with the change of feature information;
the double-branch self-adaptive detection module acquires a convolution kernel synthesis factor according to the multi-scale feature map so as to enable the convolution kernel to change along with the multi-scale feature map in a self-adaptive manner; the multi-scale feature map is respectively input into the double-branch self-adaptive detection module in parallel for processing specifically:
respectively inputting the multi-scale feature map into a category prediction branch and a position prediction branch for processing, and obtaining a category prediction result and a position prediction result of the underwater target in parallel;
the category prediction branches and the position prediction branches are combined with a Maxwell expansion formula through convolution kernel synthesis factors which adaptively change along with multi-scale characteristics to generate adaptive convolution weights, and an adaptive convolution layer is constructed; based on the multi-scale feature map, the category prediction result and the position prediction result of the underwater target are obtained in parallel through the self-adaptive convolution layer.
6. An electronic device comprising a memory and a processor and computer instructions stored on the memory and running on the processor, which when executed by the processor, perform the steps of the underwater target detection method as claimed in any of claims 1 to 4.
7. A computer readable storage medium storing computer instructions which, when executed by a processor, perform the steps of the underwater target detection method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311450871.9A CN117173552B (en) | 2023-11-03 | 2023-11-03 | Underwater target detection method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311450871.9A CN117173552B (en) | 2023-11-03 | 2023-11-03 | Underwater target detection method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117173552A CN117173552A (en) | 2023-12-05 |
| CN117173552B true CN117173552B (en) | 2024-02-20 |
Family
ID=88932141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311450871.9A Active CN117173552B (en) | 2023-11-03 | 2023-11-03 | Underwater target detection method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117173552B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112365540A (en) * | 2020-11-18 | 2021-02-12 | 北京观微科技有限公司 | Ship target positioning detection method and system suitable for multiple scales |
| CN113420643A (en) * | 2021-06-21 | 2021-09-21 | 西北工业大学 | Lightweight underwater target detection method based on depth separable cavity convolution |
| CN114170497A (en) * | 2021-11-03 | 2022-03-11 | 中国农业大学 | Multi-scale underwater fish school detection method based on attention module |
| CN114821286A (en) * | 2022-05-16 | 2022-07-29 | 西南石油大学 | Lightweight underwater target detection method and system based on image enhancement |
| CN115761467A (en) * | 2022-11-21 | 2023-03-07 | 大连海事大学 | Underwater target detection method based on high-low frequency characteristic difference pyramid neural network |
| CN116452966A (en) * | 2023-04-24 | 2023-07-18 | 北京理工大学珠海学院 | Target detection method, device and equipment for underwater image and storage medium |
| CN116740516A (en) * | 2023-05-19 | 2023-09-12 | 山东大学 | Target detection method and system based on multi-scale fusion feature extraction |
| CN116758407A (en) * | 2023-06-02 | 2023-09-15 | 南京信息工程大学 | Underwater small target detection method and device based on CenterNet |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9608738B2 (en) * | 2012-04-27 | 2017-03-28 | The Board Of Trustees Of The University Of Illinois | System and method for broadband doppler compensation |
-
2023
- 2023-11-03 CN CN202311450871.9A patent/CN117173552B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112365540A (en) * | 2020-11-18 | 2021-02-12 | 北京观微科技有限公司 | Ship target positioning detection method and system suitable for multiple scales |
| CN113420643A (en) * | 2021-06-21 | 2021-09-21 | 西北工业大学 | Lightweight underwater target detection method based on depth separable cavity convolution |
| CN114170497A (en) * | 2021-11-03 | 2022-03-11 | 中国农业大学 | Multi-scale underwater fish school detection method based on attention module |
| CN114821286A (en) * | 2022-05-16 | 2022-07-29 | 西南石油大学 | Lightweight underwater target detection method and system based on image enhancement |
| CN115761467A (en) * | 2022-11-21 | 2023-03-07 | 大连海事大学 | Underwater target detection method based on high-low frequency characteristic difference pyramid neural network |
| CN116452966A (en) * | 2023-04-24 | 2023-07-18 | 北京理工大学珠海学院 | Target detection method, device and equipment for underwater image and storage medium |
| CN116740516A (en) * | 2023-05-19 | 2023-09-12 | 山东大学 | Target detection method and system based on multi-scale fusion feature extraction |
| CN116758407A (en) * | 2023-06-02 | 2023-09-15 | 南京信息工程大学 | Underwater small target detection method and device based on CenterNet |
Non-Patent Citations (3)
| Title |
|---|
| FEATURE-TRANSFERABLE PYRAMID NETWORK FOR DENSE MULTI-SCALE OBJECT DETECTION IN SAR IMAGES;Zheng Zhou 等;《IGARSS 2022》;647-650 * |
| 基于改进SSD的合成孔径声呐图像水下多尺度目标轻量化检测模型;李宝奇 等;《电子与信息学报》;第43卷(第10期);2854-2862 * |
| 基于注意力机制的水下目标检测算法;赵晓飞 等;《扬州大学学报（自然科学版）》;第24卷(第1期);62-67 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117173552A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107945204B (en) | Pixel-level image matting method based on generation countermeasure network | |
| JP7325158B2 (en) | Data Representation for Dynamic Accuracy in Neural Network Cores | |
| US11514694B2 (en) | Teaching GAN (generative adversarial networks) to generate per-pixel annotation | |
| CN112287982A (en) | Data prediction method and device and terminal equipment | |
| CN109919183B (en) | Image identification method, device and equipment based on small samples and storage medium | |
| CN112862774B (en) | Accurate segmentation method for remote sensing image building | |
| CN112396645B (en) | Monocular image depth estimation method and system based on convolution residual learning | |
| CN111507521A (en) | Method and device for predicting power load of transformer area | |
| CN112184554A (en) | Remote sensing image fusion method based on residual mixed expansion convolution | |
| CN110427933A (en) | A kind of water gauge recognition methods based on deep learning | |
| CN109447897B (en) | Real scene image synthesis method and system | |
| CN113469074B (en) | Remote sensing image change detection method and system based on twin attention fusion network | |
| CN110956655B (en) | Dense depth estimation method based on monocular image | |
| CN115222946A (en) | Single-stage example image segmentation method and device and computer equipment | |
| CN112926607A (en) | Two-branch network image steganography framework and method based on convolutional neural network | |
| CN116075821A (en) | Form convolution and acceleration | |
| CN115457364A (en) | Target detection knowledge distillation method and device, terminal equipment and storage medium | |
| CN114283347A (en) | Target detection method, system, intelligent terminal and computer readable storage medium | |
| CN117173552B (en) | Underwater target detection method, system, electronic equipment and storage medium | |
| KR20200023695A (en) | Learning system to reduce computation volume | |
| CN111505706A (en) | Microseism P wave first arrival pickup method and device based on depth T-Net network | |
| CN112801201B (en) | Deep learning visual inertial navigation combined navigation design method based on standardization | |
| CN114913433A (en) | Multi-scale target detection method combining equalization feature and deformable convolution | |
| CN113868523A (en) | Recommendation model training method, electronic device and storage medium | |
| CN112348118A (en) | Image classification method based on gradient maintenance, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |