CN112200163B - Underwater benthos detection method and system - Google Patents

Underwater benthos detection method and system Download PDF

Info

Publication number
CN112200163B
CN112200163B CN202011393784.0A CN202011393784A CN112200163B CN 112200163 B CN112200163 B CN 112200163B CN 202011393784 A CN202011393784 A CN 202011393784A CN 112200163 B CN112200163 B CN 112200163B
Authority
CN
China
Prior art keywords
benthos
anchor frame
underwater
fusion
feature map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011393784.0A
Other languages
Chinese (zh)
Other versions
CN112200163A (en
Inventor
杨旭
万兆亮
黄海
张璐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN202011393784.0A priority Critical patent/CN112200163B/en
Publication of CN112200163A publication Critical patent/CN112200163A/en
Application granted granted Critical
Publication of CN112200163B publication Critical patent/CN112200163B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features

Abstract

The invention relates to a method and a system for detecting underwater benthos, wherein the method comprises the following steps: acquiring a plurality of underwater benthos images and corresponding characteristic information; according to each underwater benthos image and corresponding characteristic information, establishing a benthos detection model, comprising the following steps: extracting a multi-dimensional characteristic diagram for each underwater benthos image; performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map; generating an anchor frame according to the fusion characteristic diagram; adjusting the fusion characteristic diagram through an anchor frame to obtain an adjusted fusion characteristic diagram; training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN; determining an interested area based on the RPN and each anchor frame; generating a benthos detection model according to each region of interest and corresponding characteristic information; based on the benthos detection model, the to-be-detected characteristic information of the benthos to be detected can be accurately determined, and the detection precision is improved.

Description

Underwater benthos detection method and system
Technical Field
The invention relates to the technical field of robot vision, in particular to a method and a system for detecting underwater benthos.
Background
The general problem of target detection has been studied extensively in recent years and researchers have maintained a great deal of interest in its theory, algorithms and applications. However, the development of underwater target detection research is slow. The underwater target detection problem is a basic problem of visual perception of the underwater robot and a key premise of autonomous operation of the underwater robot, and the detection precision and timeliness directly influence whether an autonomous operation task of the robot succeeds or not.
The underwater robot needs to have higher requirements on the real-time performance of detection in practical application due to the actual requirements of autonomous operation, which is a balance between the problem solving precision and the problem solving speed. The existing underwater target detection work is mostly based on the traditional method, namely, shallow features of underwater pictures are manually extracted, and then whether the underwater targets are included is judged through a classifier. Although the underwater target detection method has been greatly developed for decades, many problems still exist, and for the detection task of benthic organisms (sea cucumbers, scallops and sea urchins), the most critical three problems are how to improve the detection precision of deformable organisms (such as the sea cucumbers and the scallops) to the greatest extent, how to alleviate the problem of unbalanced training samples caused by uneven growth distribution of the benthic organisms and how to alleviate the problem of limited underwater visibility on the premise of ensuring the real-time performance of the algorithm.
Disclosure of Invention
In order to solve the above problems in the prior art, i.e. to improve the detection accuracy while ensuring real-time detection, the invention aims to provide a method and a system for detecting underwater benthos.
In order to solve the technical problems, the invention provides the following scheme:
an underwater benthos detection method, comprising:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Optionally, the feature map is determined according to the following formula:
Figure 351858DEST_PATH_IMAGE001
；
wherein the content of the first and second substances,
Figure 766658DEST_PATH_IMAGE002
is thatk-thThe modulation factor of the location of the position,
Figure 991098DEST_PATH_IMAGE003
which represents the image after the normalization process,
Figure 960191DEST_PATH_IMAGE004
representing the feature map calculated by the modulated deformable convolution module,
Figure 911966DEST_PATH_IMAGE005
a weight that can be learned is represented,pthe location of the sample is represented by,
Figure 915694DEST_PATH_IMAGE006
for the firstkA predetermined offset from the position of the seat,
Figure 311035DEST_PATH_IMAGE007
a value representing the amount of shift that can be learned,k-this shown askOne position, K, represents the number of sample positions a convolution kernel has.
Optionally, generating an anchor frame according to the fused feature map specifically includes:
according to the fusion feature map, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution molecule;
and decoding the two shapes and the two positions to obtain the anchor frame corresponding to the fusion characteristic diagram.
Optionally, the Loss function of the first convolution branch is Focal local; the penalty function for the first convolution branch is BoundedIOU Loss.
Optionally, the training the RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network specifically includes:
determining the IOU (input output Unit) ratio of the intersection and the union of the two frames based on each fusion feature graph and the corresponding anchor frame;
classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU;
and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
Optionally, determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model, specifically including:
extracting a multi-dimensional characteristic graph to be detected of the benthic organism image to be detected;
determining an interested region from a multi-dimensional characteristic map to be detected based on the benthic organism detection model;
distributing each region of interest to the corresponding feature graph to be tested according to the area of the region of interest, and determining the corresponding coordinate region;
uniformly taking a plurality of reference points from the coordinate area aiming at each coordinate area;
for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected;
calculating an output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method;
and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
Optionally, the underwater benthos detection method further comprises:
connecting each piece of feature information to be tested to a fully connected network;
calculating an IOU value corresponding to each interested area;
classifying the corresponding region of interest into a positive sample region or a negative sample region according to the size of the IOU value;
determining a loss function of the benthic organism detection model from the positive and negative sample regions;
and correcting the benthos detection model according to the loss function of the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
an underwater benthos detection system, comprising:
the acquisition unit is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit is used for establishing a benthos detection model according to each underwater benthos image and corresponding characteristic information;
the modeling unit includes:
the extraction module is used for extracting a multi-dimensional feature map from each underwater benthos image;
the fusion module is used for carrying out feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
the generating module is used for generating an anchor frame according to the fusion characteristic diagram;
the adjusting module is used for adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
the training module is used for training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
the determining module is used for determining the region of interest based on the trained RPN and each anchor frame;
the modeling module is used for generating a benthos detection model according to each region of interest and the corresponding characteristic information;
and the detection unit is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
an underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In order to solve the technical problems, the invention also provides the following scheme:
a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
According to the embodiment of the invention, the invention discloses the following technical effects:
the benthos detection model is established based on the multiple underwater benthos images and the corresponding characteristic information, detection of benthos can be achieved in real time, the multi-dimensional characteristic diagram is extracted from the underwater benthos images, the dimension reduction characteristic fusion is carried out, the anchor frame is generated, the RPN network is trained, the region of interest is determined, the characteristic information to be detected of the benthos to be detected can be accurately determined, and the detection precision is high.
Drawings
FIG. 1 is a flowchart of the method for detecting underwater benthos according to the present invention;
FIG. 2 is a schematic block diagram of the underwater benthos detection system of the present invention.
Description of the symbols:
the system comprises an acquisition unit-1, a modeling unit-2, an extraction module-21, a fusion module-22, a generation module-23, an adjustment module-24, a training module-25, a determination module-26, a modeling module-27 and a detection unit-3.
Detailed Description
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
The invention aims to provide an underwater benthos detection method, which is characterized in that a benthos detection model is established based on a plurality of underwater benthos images and corresponding characteristic information, real-time detection of benthos can be realized, and the detection precision is high by extracting a multi-dimensional characteristic diagram from the underwater benthos images, reducing the dimension characteristic fusion, generating an anchor frame, training an RPN network and determining an interested region, so that the to-be-detected characteristic information of the benthos to be detected can be accurately determined.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
As shown in FIG. 1, the underwater benthos detection method of the present invention comprises:
step 100: acquiring a plurality of underwater benthos images and corresponding characteristic information;
step 200: according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
step 210: extracting a multi-dimensional feature map from each underwater benthos image;
step 220: performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
step 230: generating an anchor frame according to the fusion characteristic diagram;
step 240: adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
step 250: training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
step 260: determining an interested area based on the trained RPN and each anchor frame;
step 270: generating a benthos detection model according to each region of interest and corresponding characteristic information;
step 300: and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
In step 200, the acquired image of the underwater benthos is normalized to obtain dimensions of 256 × 512 × 1024 × 2048. In this embodiment, an image is read from a camera fixedly connected to a robot, and the image with the adjusted size passes through a backbone network. In order to relieve the problem of geometric deformation of non-rigid benthos (such as sea cucumber and scallop), the invention replaces the convolution layers in the third, fourth and fifth convolution groups with variability convolution.
Specifically, in step 210, a feature map is determined according to the following formula:
Figure 501844DEST_PATH_IMAGE001
；
wherein the content of the first and second substances,
Figure 257311DEST_PATH_IMAGE002
is thatk-thThe modulation factor of the location of the position,
Figure 849966DEST_PATH_IMAGE003
which represents the image after the normalization process,
Figure 665475DEST_PATH_IMAGE004
representing the feature map calculated by the modulated deformable convolution module,
Figure 359893DEST_PATH_IMAGE005
a weight that can be learned is represented,pthe location of the sample is represented by,
Figure 919050DEST_PATH_IMAGE006
for the firstkA predetermined offset from the position of the seat,
Figure 366212DEST_PATH_IMAGE007
a value representing the amount of shift that can be learned,k-this shown askOne position, K, represents the number of sample positions a convolution kernel has.
In step 220, to extract the multi-scale features, the present invention introduces the feature pyramid FPN method into the baseline method. Inputting feature maps with dimensions of 256 × 512 × 1024 × 2048, and obtaining 5 feature maps after passing through a feature pyramid network.
In step 230, generating an anchor frame according to the fused feature map, which specifically includes:
step 231: and respectively predicting the shape and the position of the anchor frame through the first convolution branch and the second convolution molecule according to the fusion feature map.
Wherein the Loss function of the first convolution branch is Focal local; the penalty function for the first convolution branch is BoundedIOU Loss.
Step 232: and decoding the two shapes and the two positions to obtain the anchor frame corresponding to the fusion characteristic diagram.
In step 240, the shape information of the anchor frame is merged into the merged feature map, and the branch result of the anchor frame shape obtained in step 230 is input to the bias term of the 3 × 3 deformable convolution after being subjected to a 1 × 1 convolution.
In step 250, the training an RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network specifically includes:
step 251: and determining the ratio IOU of the intersection and the union of the two frames based on each fusion feature graph and the corresponding anchor frame.
Step 252: and classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU.
Matching all Anchor frames with the labels in the characteristic information, and judging whether the Anchor frame Anchor contains a sample according to the size of the IOU:
when the IOU is more than or equal to 0.7, the Anchor frame Anchor contains samples, namely the Anchor frame Anchor is divided into positive sample Anchor frames; otherwise, the Anchor frame Anchor does not contain the sample, namely the Anchor frame Anchor is divided into negative sample Anchor frames.
Step 253: and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
In step 260, the information of RPN network prediction and the corresponding Anchor are decoded to generate a set of prediction boxes, i.e. ROI (region of interest). At the moment, the number of the prediction frames is huge, and in order to improve the quality of the prediction frames, the positive sample anchor frame and the negative sample anchor frame are randomly sampled, and 256 sample anchor frames are sampled.
Further, in step 300, in order to ensure that the feature dimensions corresponding to each ROI are consistent and eliminate the error of ROI Pooling during the two approximate roundings as much as possible, the present invention adds ROI Align Pooling to the baseline method.
Preferably, the determining the characteristic information to be detected of the benthos to be detected based on the benthos detection model according to the image of the benthos to be detected specifically includes:
step 310: and extracting a multi-dimensional characteristic diagram to be detected of the benthic organism image to be detected.
Step 320: and determining a region of interest from the multi-dimensional characteristic map to be detected based on the benthic organism detection model.
Step 330: and distributing each region of interest to the corresponding feature map to be measured according to the area of the region of interest, and determining the corresponding coordinate region.
Step 340: and uniformly taking a plurality of reference points from the coordinate area for each coordinate area.
Step 350: and for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected.
Step 360: and calculating the output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method.
Step 370: and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
In addition, the underwater benthos detection method of the invention further comprises:
step 410: and connecting each piece of feature information to be tested to the full-connection network.
Specifically, 256-dimensional characteristic information to be measured obtained after ROI Align Pooling is connected to a full-connection network.
Step 420: and calculating the IOU value corresponding to each interested area.
Step 410: the corresponding regions of interest are classified as either positive or negative sample regions depending on the size of the IOU value.
Specifically, if the IOU is greater than or equal to 0.5, the region of interest is a positive sample region, otherwise, the region of interest is a negative sample region.
Positive sample area sampling: sampling is carried out according to the proportion of 1:3, and 512 samples are sampled in total. For negative sample regions, the present invention employs an IOU balanced sampling strategy, as opposed to ordinary random sampling.
And dividing the screened negative samples into K buckets according to the corresponding IOU size.
Probability of selection for each sample
Figure 87044DEST_PATH_IMAGE008
The following can be calculated:
Figure 259968DEST_PATH_IMAGE009
，
Figure 357237DEST_PATH_IMAGE010
；
wherein the content of the first and second substances,Nwhich represents the number of selected samples, is,
Figure 190064DEST_PATH_IMAGE011
is shown as
Figure 816218DEST_PATH_IMAGE012
The number of candidate samples of the sub-iteration, and the maximum threshold value of the T iteration times. In the present embodiment, it is preferred that,Tthe value is 3.
Step 440: determining a loss function of the benthic organism detection model from the positive and negative sample regions.
Step 450: and correcting the benthos detection model according to the loss function of the benthos detection model.
The method adopts a method based on modularized deformable convolution to relieve the problem that the marine life is easy to deform after being disturbed. The underwater visibility of part of sea areas is limited, so that the robot cannot approach a target in a short distance in order to see clear sea creatures, and an overlong or too wide target appears in a visual field. Aiming at the problem of unbalanced distribution of underwater samples, the invention adopts barrel-dividing random sampling of collected samples according to IOU numerical values. According to the method, the GSDCN algorithm customized for the sea area of the yellow sea is adopted, and the highest detection precision of the URPC2018 data set is obtained on the premise of not adopting a universal skill.
In addition, the invention also provides an underwater benthos detection system which can ensure real-time detection and improve detection precision.
As shown in fig. 2, the underwater benthos detection system of the present invention includes an acquisition unit 1, a modeling unit 2, and a detection unit 3.
The acquisition unit 1 is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit 2 is used for establishing a benthos detection model according to the images of the underwater benthos and the corresponding characteristic information.
The detection unit 3 is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Further, the modeling unit 2 includes an extraction module 21, a fusion module 22, a generation module 23, an adjustment module 24, a training module 25, a determination module 26, and a modeling module 27.
Specifically, the extraction module 21 is configured to extract a multi-dimensional feature map from each underwater benthos image;
the fusion module 22 is configured to perform feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
the generating module 23 is configured to generate an anchor frame according to the fusion feature map;
the adjusting module 24 is configured to adjust the fusion feature map through the anchor frame to obtain an adjusted fusion feature map;
the training module 25 is configured to train the RPN network based on each fusion feature map and the corresponding anchor frame to obtain a trained RPN network;
the determining module 26 is configured to determine an area of interest based on the trained RPN network and each anchor frame;
the modeling module 27 is configured to generate a benthic organism detection model according to each region of interest and the corresponding feature information.
In addition, the invention also provides the following scheme:
an underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Further, the invention also provides the following scheme:
a computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map with reduced dimensions;
generating an anchor frame according to the fusion characteristic diagram;
adjusting the fusion characteristic diagram through the anchor frame to obtain an adjusted fusion characteristic diagram;
training the RPN based on each fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
Compared with the prior art, the underwater benthos detection system and the computer readable storage medium have the same beneficial effects as the underwater benthos detection method, and are not repeated herein.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims (9)

1. An underwater benthos detection method, comprising:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map after dimension reduction;
generating an anchor frame according to the fusion characteristic diagram after the dimension reduction; the method specifically comprises the following steps:
according to the fused feature map after dimensionality reduction, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution branch;
decoding the two shapes and the two positions to obtain an anchor frame corresponding to the fusion characteristic diagram after dimension reduction;
adjusting the fused feature map after dimension reduction through the anchor frame to obtain an adjusted fused feature map;
training the RPN based on each adjusted fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
2. The underwater benthos detection method according to claim 1, wherein the feature map is determined according to the following formula:
Figure DEST_PATH_IMAGE001
；
wherein the content of the first and second substances,
Figure 771084DEST_PATH_IMAGE002
is the firstkThe modulation factor for each of the sample positions,
Figure DEST_PATH_IMAGE003
which represents the image after the normalization process,
Figure 124443DEST_PATH_IMAGE004
representing the feature map calculated by the modulated deformable convolution module,
Figure DEST_PATH_IMAGE005
a weight that can be learned is represented,pthe location of the sample is represented by,
Figure 576284DEST_PATH_IMAGE006
for the firstkThe offset of each sampling position is preset,
Figure DEST_PATH_IMAGE007
a value representing the amount of shift that can be learned,kthe number of sample positions is indicated, and K indicates the number of sample positions that a convolution kernel has.
3. The method of detecting benthic organisms according to claim 1, wherein the Loss function of the first convolution branch is Focal local; the penalty function for the second convolution branch is BoundedIOU Loss.
4. The method for detecting underwater benthos according to claim 1, wherein the training of the RPN network based on each adjusted fusion feature map and the corresponding anchor frame to obtain the trained RPN network specifically comprises:
determining the ratio IOU of the intersection and the union of the two frames based on each adjusted fusion feature map and the corresponding anchor frame;
classifying the corresponding anchor frame into a positive sample anchor frame or a negative sample anchor frame according to the size of the IOU;
and respectively training the RPN according to the positive sample anchor frame and the negative sample anchor frame to obtain the trained RPN.
5. The method for detecting the underwater benthos according to claim 1, wherein the determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model specifically includes:
extracting a multi-dimensional characteristic graph to be detected of the benthic organism image to be detected;
determining an interested region from a multi-dimensional characteristic map to be detected based on the benthic organism detection model;
distributing each region of interest to the corresponding feature graph to be tested according to the area of the region of interest, and determining the corresponding coordinate region;
uniformly taking a plurality of reference points from the coordinate area aiming at each coordinate area;
for each reference point, determining four reference points closest to the reference point from the characteristic diagram to be detected;
calculating an output value of the reference point according to the four reference points and the reference point by adopting a bilinear interpolation method;
and averaging the output values of the reference points to obtain the output information of the coordinate area, wherein the output information of the coordinate area is the characteristic information to be detected of the benthic organisms to be detected.
6. The underwater benthos detection method according to claim 5, further comprising:
connecting each piece of feature information to be tested to a fully connected network;
calculating an IOU value corresponding to each interested area;
classifying the corresponding region of interest into a positive sample region or a negative sample region according to the size of the IOU value;
determining a loss function of the benthic organism detection model from the positive and negative sample regions;
and correcting the benthos detection model according to the loss function of the benthos detection model.
7. An underwater benthos detection system, comprising:
the acquisition unit is used for acquiring a plurality of underwater benthos images and corresponding characteristic information;
the modeling unit is used for establishing a benthos detection model according to each underwater benthos image and corresponding characteristic information;
the modeling unit includes:
the extraction module is used for extracting a multi-dimensional feature map from each underwater benthos image;
the fusion module is used for carrying out feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map after dimension reduction;
the generating module is used for generating an anchor frame according to the fusion characteristic diagram after the dimension reduction; the method specifically comprises the following steps:
according to the fused feature map after dimensionality reduction, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution branch;
decoding the two shapes and the two positions to obtain an anchor frame corresponding to the fusion characteristic diagram after dimension reduction;
the adjusting module is used for adjusting the fused feature map after the dimension reduction through the anchor frame to obtain an adjusted fused feature map;
the training module is used for training the RPN based on each adjusted fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
the determining module is used for determining the region of interest based on the trained RPN and each anchor frame;
the modeling module is used for generating a benthos detection model according to each region of interest and the corresponding characteristic information;
and the detection unit is used for determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
8. An underwater benthic organism detection system comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map after dimension reduction;
generating an anchor frame according to the fusion characteristic diagram after the dimension reduction; the method specifically comprises the following steps:
according to the fused feature map after dimensionality reduction, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution branch;
decoding the two shapes and the two positions to obtain an anchor frame corresponding to the fusion characteristic diagram after dimension reduction;
adjusting the fused feature map after dimension reduction through the anchor frame to obtain an adjusted fused feature map;
training the RPN based on each adjusted fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
9. A computer-readable storage medium storing one or more programs that, when executed by an electronic device including a plurality of application programs, cause the electronic device to:
acquiring a plurality of underwater benthos images and corresponding characteristic information;
according to each underwater benthos image and corresponding characteristic information, a benthos detection model is established, and the method specifically comprises the following steps:
extracting a multi-dimensional feature map from each underwater benthos image;
performing feature fusion on the multi-dimensional feature map based on the feature pyramid network to obtain a fusion feature map after dimension reduction;
generating an anchor frame according to the fusion characteristic diagram after the dimension reduction; the method specifically comprises the following steps:
according to the fused feature map after dimensionality reduction, respectively predicting the shape and the position of an anchor frame through a first convolution branch and a second convolution branch;
decoding the two shapes and the two positions to obtain an anchor frame corresponding to the fusion characteristic diagram after dimension reduction;
adjusting the fused feature map after dimension reduction through the anchor frame to obtain an adjusted fused feature map;
training the RPN based on each adjusted fusion characteristic diagram and the corresponding anchor frame to obtain a trained RPN;
determining an interested area based on the trained RPN and each anchor frame;
generating a benthos detection model according to each region of interest and corresponding characteristic information;
and determining the characteristic information to be detected of the benthos to be detected according to the image of the benthos to be detected based on the benthos detection model.
CN202011393784.0A 2020-12-03 2020-12-03 Underwater benthos detection method and system Active CN112200163B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011393784.0A CN112200163B (en) 2020-12-03 2020-12-03 Underwater benthos detection method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011393784.0A CN112200163B (en) 2020-12-03 2020-12-03 Underwater benthos detection method and system

Publications (2)

Publication Number Publication Date
CN112200163A CN112200163A (en) 2021-01-08
CN112200163B true CN112200163B (en) 2021-06-25

Family

ID=74033683

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011393784.0A Active CN112200163B (en) 2020-12-03 2020-12-03 Underwater benthos detection method and system

Country Status (1)

Country Link
CN (1) CN112200163B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112766340A (en) * 2021-01-11 2021-05-07 中山大学 Depth capsule network image classification method and system based on adaptive spatial mode
CN113343964B (en) * 2021-08-09 2022-06-10 湖南汇视威智能科技有限公司 Balanced underwater acoustic image target detection method
CN113762229B (en) * 2021-11-10 2022-02-08 山东天亚达新材料科技有限公司 Intelligent identification method and system for building equipment in building site

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109034245A (en) * 2018-07-27 2018-12-18 燕山大学 A kind of object detection method merged using characteristic pattern
CN110751185A (en) * 2019-09-26 2020-02-04 高新兴科技集团股份有限公司 Training method and device of target detection model
CN111126399A (en) * 2019-12-28 2020-05-08 苏州科达科技股份有限公司 Image detection method, device and equipment and readable storage medium
CN111242071A (en) * 2020-01-17 2020-06-05 陕西师范大学 Attention remote sensing image target detection method based on anchor frame

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9373059B1 (en) * 2014-05-05 2016-06-21 Atomwise Inc. Systems and methods for applying a convolutional network to spatial data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109034245A (en) * 2018-07-27 2018-12-18 燕山大学 A kind of object detection method merged using characteristic pattern
CN110751185A (en) * 2019-09-26 2020-02-04 高新兴科技集团股份有限公司 Training method and device of target detection model
CN111126399A (en) * 2019-12-28 2020-05-08 苏州科达科技股份有限公司 Image detection method, device and equipment and readable storage medium
CN111242071A (en) * 2020-01-17 2020-06-05 陕西师范大学 Attention remote sensing image target detection method based on anchor frame

Also Published As

Publication number Publication date
CN112200163A (en) 2021-01-08

Similar Documents

Publication Publication Date Title
CN112200163B (en) Underwater benthos detection method and system
CN109145830B (en) Intelligent water gauge identification method
CN104200495A (en) Multi-target tracking method in video surveillance
Lou et al. Accurate multi-view stereo 3D reconstruction for cost-effective plant phenotyping
CN112800964A (en) Remote sensing image target detection method and system based on multi-module fusion
CN111626993A (en) Image automatic detection counting method and system based on embedded FEFnet network
CN113052109A (en) 3D target detection system and 3D target detection method thereof
CN103198477A (en) Apple fruitlet bagging robot visual positioning method
CN108428220A (en) Satellite sequence remote sensing image sea island reef region automatic geometric correction method
CN110827312A (en) Learning method based on cooperative visual attention neural network
CN115187803B (en) Positioning method for picking process of famous tea tender shoots
CN109165603B (en) Ship detection method and device
CN110633711A (en) Computer device and method for training feature point detector and feature point detection method
CN108681691A (en) A kind of marine ships and light boats rapid detection method based on unmanned water surface ship
CN112287906A (en) Template matching tracking method and system based on depth feature fusion
CN116503760A (en) Unmanned aerial vehicle cruising detection method based on self-adaptive edge feature semantic segmentation
CN110345919A (en) Space junk detection method based on three-dimensional space vector and two-dimensional plane coordinate
CN115115601A (en) Remote sensing ship target detection method based on deformation attention pyramid
CN114463628A (en) Deep learning remote sensing image ship target identification method based on threshold value constraint
Le et al. A marine benthic detection algoritm based on improved lightweight network
CN110889418A (en) Gas contour identification method
CN115205853B (en) Image-based citrus fruit detection and identification method and system
CN117522950B (en) Geometric parameter measurement method for plant stem growth based on machine vision
Gaspar et al. Visual Place Recognition for Harbour Infrastructures Inspection
CN117495891B (en) Point cloud edge detection method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant