CN111062383A - Image-based ship detection depth neural network algorithm - Google Patents
Image-based ship detection depth neural network algorithm Download PDFInfo
- Publication number
- CN111062383A CN111062383A CN201911065233.9A CN201911065233A CN111062383A CN 111062383 A CN111062383 A CN 111062383A CN 201911065233 A CN201911065233 A CN 201911065233A CN 111062383 A CN111062383 A CN 111062383A
- Authority
- CN
- China
- Prior art keywords
- ship
- prediction
- image
- neural network
- box
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 42
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 28
- 238000005070 sampling Methods 0.000 claims abstract description 5
- 230000004927 fusion Effects 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims 1
- 230000006870 function Effects 0.000 abstract description 6
- 238000013527 convolutional neural network Methods 0.000 abstract description 3
- 238000011897 real-time detection Methods 0.000 abstract description 3
- 238000011160 research Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008570 general process Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image-based ship detection depth neural network algorithm, which comprises the following steps: firstly, sampling an image to enable the length and the width of the image to be twice of the original length and width; dividing the image into grids of S-S, predicting B boundary predictions for each grid, and giving 6 parameters including ship positions, confidence degrees and classification probabilities for each boundary prediction; thirdly, extracting the characteristics of each grid through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through characteristic fusion, and determining the position of a ship; and fourthly, designing a loss function, and balancing the prediction frame with the hull part and the prediction frame without the hull part by setting different scale factors. Based on the research of the convolutional neural network in the field of computer vision, the invention integrates classification and regression in a deep neural network for multi-target real-time detection on the basis of feature learning, and has strong accuracy and rapidity.
Description
Technical Field
The invention belongs to the field of ship detection, and particularly relates to an image-based ship detection deep neural network algorithm (SD-DCNN).
Background
The automatic detection of ships based on images is one of the basic problems of automatic driving and safe driving of ships and is also an important problem in the field of computer vision. In particular, since the small vessels are not equipped with equipment such as ais (automatic identification system), they cannot actively report information such as their own position, and are not easily sensed by other vessels, and thus, they are likely to cause a collision of the vessels. The traditional ship detection method adopts manual characteristics, the accuracy and robustness of the algorithm are insufficient, and false detection and missing detection can occur. The existing deep learning-based method can realize automatic detection of ships, but detection omission easily occurs to small ships, and driving safety is affected. Therefore, current ship driving mainly depends on manual observation to realize safe running of the ship.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the defects of the prior art, the invention provides an image-based ship detection deep neural network algorithm (SD-DCNN), which particularly considers small ships on the basis of detecting large and medium ships so as to realize the automatic perception of various ships.
The technical scheme is as follows: an image-based ship detection deep neural network algorithm is characterized in that: the method comprises the following detection processes:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids of S X S, predicting B ship boundaries for each grid, wherein each boundary prediction gives 6 parameters, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) is the central horizontal coordinate of a ship body prediction frame and the central vertical coordinate of the ship body prediction frame; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence=Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents area calculation;
thirdly, extracting features through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through feature fusion, and determining the position of a ship; the convolution operation with the length of 2 is adopted each time to realize downsampling;
and fourthly, designing a loss function, and setting different scale factors to make corresponding proportional balance for the prediction frame with the hull part and the prediction frame without the hull part.
As an optimization: and the third step is specifically realized by realizing a cascade cavity convolution neural network on the basis of the original deep convolution neural network to extract specific characteristics of the ship target and fusing the characteristics into a high-level network to extract high-level abstract characteristics.
As an optimization: the third step is specifically realized by giving three prediction results and adjusting the parameters of the network simultaneously in combination with different resolutions instead of giving one prediction result when the ship boundary is predicted.
As an optimization: in the fourth step, the loss function is as follows:
wherein,judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
Has the advantages that: based on the research of the convolutional neural network in the field of computer vision, the invention integrates classification and regression in a deep neural network for multi-target real-time detection on the basis of feature learning, and has strong accuracy and rapidity.
Different from the traditional method that the features (such as Local Binary Pattern features and Historic of Oriented Gradient features) are extracted firstly in target detection, a classification model (such as Support Vector Machine) is adopted to obtain a target, the SD-DCNN algorithm uses the regression idea, the features are not required to be designed manually, a classifier or a locator is not required to be trained independently, and only the target region prediction and the target category prediction are required to be integrated into a single neural network model. The method adopts a single neural network to unify candidate frame extraction, feature extraction, target classification and target positioning, and realizes end-to-end target detection. The SD-DCNN algorithm is compared with other object detection methods such as RCNN and Fast RCNN in the aspects of detection speed and accuracy, and the SD-DCNN has obvious advantages in the aspect of speed.
Drawings
FIG. 1 is a schematic diagram of a general process for SD-DCNN target detection according to the present invention;
FIG. 2 is a schematic diagram of the SD-DCNN network structure of the present invention; wherein, (a) the ship detects the deep neural network; (b) cascading a hole convolution module;
FIG. 3 is a diagram illustrating the SD-DCNN network results of the present invention; wherein, the rectangular box represents the detection result of the ship.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below so that those skilled in the art can better understand the advantages and features of the present invention, and thus the scope of the present invention will be more clearly defined. The embodiments described herein are only a few embodiments of the present invention, rather than all embodiments, and all other embodiments that can be derived by one of ordinary skill in the art without inventive faculty based on the embodiments described herein are intended to fall within the scope of the present invention.
Examples
Based on the research of the convolutional neural network in the field of computer vision, the classification and regression are fused in a deep neural network on the basis of feature learning to carry out multi-target real-time detection, and specifically, the invention relates to an image-based ship detection deep neural network algorithm (SD-DCNN).
SD-CDNN target detection Process
Since there are a plurality of ship targets in the image, each prediction frame needs to be discriminated. The specific detection process is as follows:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids (Grid cells) of S, predicting B Bounding Boxes (Bounding Boxes) for each Grid, wherein each frame gives 6 parameters, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) are the central horizontal coordinate of the ship body prediction box and the central vertical coordinate of the ship body prediction box; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence=Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents the area.
And thirdly, extracting features through a cascade cavity convolution neural network, and realizing ship boundary prediction through feature fusion. The specific implementation is that the cascaded cavity convolution neural network is realized on the basis of the original deep convolution neural network and is used for extracting specific characteristics of a ship target, and the characteristics are fused to predict the boundary of the ship in a multi-resolution mode so as to determine the position of the ship.
And fourthly, designing a loss function, wherein the effect of the loss function is to make corresponding proportional balance for the prediction box with the hull part and the prediction box without the hull part by setting different scale factors. The loss function is of the form:
wherein,judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
The general flow diagram of the SD-DCNN target detection is shown in FIG. 1 below.
SD-DCNN training network
The images are input into an SD-DCNN training set, the SD-DCNN automatically divides the input images into 13-13 grids, if the center of the ship body is located in the grids, the SD-DCNN considers the grids to be responsible for detecting the ship body part, each grid needs to process two pre-selection frames, 6 parameters in each pre-selection frame need to be determined, and the pre-selection frames are X, Y, W, H, SHIPConfigudence and SHIPPRO.
Two prediction boxes need to predict 12 parameters. Furthermore, as can be seen from equation 5, if it is necessary to predict the class probability Pr (class | ship) under the target condition, the probability of determining the ship hull by a certain prediction box can be obtained by multiplying Pr (ship) and Pr (class | ship). Where Pr (ship) is the probability of the hull type, and Pr (class | ship) is the determination probability of the presence of a ship in the present prediction box.
Pr(class)=Pr(class|ship)*Pr(ship) (5)
The design structure of the SD-DCNN network is shown in FIG. 2.
3. Testing SD-DCNN algorithm
In the process of predicting the SD-DCNN, the output type of the network needs to be judged to obtain the grid with the target, and the position information of the prediction frame corresponding to the grid is obtained, so that the corresponding target detection can be completed.
And (3) outputting two prediction boxes by each grid, predicting 6 parameters by each prediction box, and finally obtaining the output of 338 prediction box parameters, wherein the probability of the ship body is set to be 0.2 by formula 5, the ship body is not the ship body target when the probability is less than 0.2, and the ship body is the target ship body when the probability is more than 0.2. Arranging the ship positions in the prediction frames according to the descending order of the probability, combining the overlapped prediction frames, obtaining the final ship position through continuous repetition, and marking the ship position by a rectangular frame, as shown in figure 3.
Analysis of detection results of SD-DCNN algorithm
(1) Rapidity of operation
The Fast RCNN and SD-DCNN detectors were tested separately under CPU and GPU, and the results are shown in Table 1.
TABLE 1 detection speed contrast (frames/sec)
Under the CPU, the real-time performance of the CPU and the CPU cannot be met; under the GPU, the detection speed of Fast RCNN is far lower than twenty-four frames per second, while the detection performance of SD-DCNN is excellent.
(2) Accuracy of
The error rates of SD-DCNN and Fast RCNN were measured, and the results are shown in Table 2 below.
TABLE 2 detection accuracy comparison (error Rate)
As can be seen from the table, SD-DCNN has advantages in detection speed and higher accuracy.
The experimental environment of the invention is as follows: intel (R) core (TM) i5-5200U CPU @2.20GHz 2.20GHz4.00GB, NVIDIA TITAN XP, Ubuntu64 bit operating system. Under the experimental environment, the speed of system detection can meet the requirements of real-time performance and accuracy.
According to the invention, the method for detecting the multiple targets of the ship in real time is successful through actual video analysis, and has strong accuracy and rapidity.
Claims (6)
1. An image-based ship detection deep neural network algorithm is characterized in that: the method comprises the following detection processes:
firstly, sampling an image to enable the length and the width of the image to be 2 times of the original length and width, and extracting initial features by convolution operation;
secondly, dividing the image into grids of S X S, predicting B bounding boxes for each grid, and giving 6 parameters to each frame, namely X, Y, W, H, SHIPCFIdence and SHIPPPro, wherein (X, Y) is the central horizontal coordinate of the ship body prediction box and the central vertical coordinate of the ship body prediction box; (W, H) is the width of the hull prediction box and the height of the hull prediction box; SHIPPORO is the probability that the object in the prediction frame belongs to the ship; the SHIPCFIdence is the credibility of the ship body in the ship body prediction frame;
SHIPConfidence=Pr(ship)*IOU(pred|truth) (1)
wherein, Pr(ship) represents whether or not a hull exists in the prediction box:
IOU (pred | truth) represents the intersection ratio of the prediction box to the real box:
bb (pred) is a prediction box based on training data; BB (treth) is a real frame during target detection; area (·) represents area calculation;
thirdly, extracting features through a cascade cavity convolution neural network, realizing multi-resolution ship boundary prediction through feature fusion, and determining the position of a ship; the convolution operation with the length of 2 is adopted each time to realize downsampling;
and fourthly, designing a loss function, and carrying out corresponding proportional balance on the prediction frame with the hull part and the prediction frame without the hull part by setting different scale factors.
2. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the first step, in order to take the small ship which is difficult to detect into account, the image is sampled by linear interpolation in the specific implementation process, the image is used for amplifying the small ship, the initial characteristic is extracted, and the accurate detection of the subsequent ship is facilitated.
3. The image-based ship detection depth neural network algorithm of claim 1, wherein: and in the third step, in order to take small ships which are difficult to detect into account, the cascade cavity convolution neural network is realized on the basis of the original deep convolution neural network for extracting the specific characteristics of the ship target in the concrete implementation process, and the characteristics are fused into the high-level network to extract the high-level abstract characteristics, so that rich characteristic information is obtained.
4. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the third step, in order to take small ships which are difficult to detect into account, the extraction of ship features and the prediction of ship boundaries are realized under multiple resolutions in specific implementation.
5. The image-based ship detection depth neural network algorithm of claim 1, wherein: and in the third step, in order to take the small ships which are difficult to detect into account, the convolution operation with the step length of 2 is adopted to reduce the image size in each down-sampling process, instead of the simple pooling operation, so that the information of the small ships is kept as much as possible.
6. The image-based ship detection depth neural network algorithm of claim 1, wherein: in the fourth step, the loss function is as follows:
wherein,judging whether the jth prediction boundary box in the low i networks is responsible for judging the ship body target;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911065233.9A CN111062383A (en) | 2019-11-04 | 2019-11-04 | Image-based ship detection depth neural network algorithm |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911065233.9A CN111062383A (en) | 2019-11-04 | 2019-11-04 | Image-based ship detection depth neural network algorithm |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111062383A true CN111062383A (en) | 2020-04-24 |
Family
ID=70297584
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911065233.9A Pending CN111062383A (en) | 2019-11-04 | 2019-11-04 | Image-based ship detection depth neural network algorithm |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111062383A (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111738295A (en) * | 2020-05-22 | 2020-10-02 | 南通大学 | Image segmentation method and storage medium |
CN112348742A (en) * | 2020-11-03 | 2021-02-09 | 北京信工博特智能科技有限公司 | Image nonlinear interpolation obtaining method and system based on deep learning |
CN113158787A (en) * | 2021-03-11 | 2021-07-23 | 上海海事大学 | Ship detection classification method under complex marine environment |
CN113160253A (en) * | 2020-12-29 | 2021-07-23 | 南通大学 | Three-dimensional medical image segmentation method based on sparse mark and storage medium |
CN113763484A (en) * | 2021-09-17 | 2021-12-07 | 交通运输部水运科学研究所 | Ship target positioning and speed estimation method based on video image analysis technology |
CN113837086A (en) * | 2021-09-24 | 2021-12-24 | 南通大学 | Reservoir phishing person detection method based on deep convolutional neural network |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108256634A (en) * | 2018-02-08 | 2018-07-06 | 杭州电子科技大学 | A kind of ship target detection method based on lightweight deep neural network |
CN110147807A (en) * | 2019-01-04 | 2019-08-20 | 上海海事大学 | A kind of ship intelligent recognition tracking |
-
2019
- 2019-11-04 CN CN201911065233.9A patent/CN111062383A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108256634A (en) * | 2018-02-08 | 2018-07-06 | 杭州电子科技大学 | A kind of ship target detection method based on lightweight deep neural network |
CN110147807A (en) * | 2019-01-04 | 2019-08-20 | 上海海事大学 | A kind of ship intelligent recognition tracking |
Non-Patent Citations (1)
Title |
---|
杨名等: "甚高速区域卷积神经网络的船舶视频检测方法" * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111738295A (en) * | 2020-05-22 | 2020-10-02 | 南通大学 | Image segmentation method and storage medium |
CN111738295B (en) * | 2020-05-22 | 2024-03-22 | 南通大学 | Image segmentation method and storage medium |
CN112348742A (en) * | 2020-11-03 | 2021-02-09 | 北京信工博特智能科技有限公司 | Image nonlinear interpolation obtaining method and system based on deep learning |
CN112348742B (en) * | 2020-11-03 | 2024-03-26 | 北京信工博特智能科技有限公司 | Image nonlinear interpolation acquisition method and system based on deep learning |
CN113160253A (en) * | 2020-12-29 | 2021-07-23 | 南通大学 | Three-dimensional medical image segmentation method based on sparse mark and storage medium |
CN113160253B (en) * | 2020-12-29 | 2024-01-30 | 南通大学 | Three-dimensional medical image segmentation method based on sparse markers and storage medium |
CN113158787A (en) * | 2021-03-11 | 2021-07-23 | 上海海事大学 | Ship detection classification method under complex marine environment |
CN113158787B (en) * | 2021-03-11 | 2024-04-05 | 上海海事大学 | Ship detection and classification method under complex marine environment |
CN113763484A (en) * | 2021-09-17 | 2021-12-07 | 交通运输部水运科学研究所 | Ship target positioning and speed estimation method based on video image analysis technology |
CN113837086A (en) * | 2021-09-24 | 2021-12-24 | 南通大学 | Reservoir phishing person detection method based on deep convolutional neural network |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111062383A (en) | Image-based ship detection depth neural network algorithm | |
CN108961235B (en) | Defective insulator identification method based on YOLOv3 network and particle filter algorithm | |
CN110610166B (en) | Text region detection model training method and device, electronic equipment and storage medium | |
CN111784685A (en) | Power transmission line defect image identification method based on cloud edge cooperative detection | |
CN112633149B (en) | Domain-adaptive foggy-day image target detection method and device | |
CN111882620B (en) | Road drivable area segmentation method based on multi-scale information | |
CN109753949B (en) | Multi-window traffic sign detection method based on deep learning | |
CN112836639A (en) | Pedestrian multi-target tracking video identification method based on improved YOLOv3 model | |
CN110598698B (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
CN115205264A (en) | High-resolution remote sensing ship detection method based on improved YOLOv4 | |
CN114155527A (en) | Scene text recognition method and device | |
CN114332473B (en) | Object detection method, device, computer apparatus, storage medium, and program product | |
CN113487600B (en) | Feature enhancement scale self-adaptive perception ship detection method | |
CN113361528B (en) | Multi-scale target detection method and system | |
CN112101113B (en) | Lightweight unmanned aerial vehicle image small target detection method | |
CN112800934A (en) | Behavior identification method and device for multi-class engineering vehicle | |
CN115761513A (en) | Intelligent remote sensing identification method for mountain large landslide based on semi-supervised deep learning | |
CN117689943A (en) | Training method of screw detection model, screw detection method and device | |
CN111767919A (en) | Target detection method for multi-layer bidirectional feature extraction and fusion | |
CN115995075A (en) | Vehicle self-adaptive navigation method and device, electronic equipment and storage medium | |
CN112487920B (en) | Convolution neural network-based crossing behavior identification method | |
CN114898290A (en) | Real-time detection method and system for marine ship | |
CN114743045A (en) | Small sample target detection method based on double-branch area suggestion network | |
CN114463300A (en) | Steel surface defect detection method, electronic device, and storage medium | |
CN111178158A (en) | Method and system for detecting cyclist |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200424 |