CN111353459A - Ship target detection method under resource-limited condition - Google Patents

Ship target detection method under resource-limited condition Download PDF

Info

Publication number
CN111353459A
CN111353459A CN202010162239.4A CN202010162239A CN111353459A CN 111353459 A CN111353459 A CN 111353459A CN 202010162239 A CN202010162239 A CN 202010162239A CN 111353459 A CN111353459 A CN 111353459A
Authority
CN
China
Prior art keywords
network
target detection
target
ship
default
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010162239.4A
Other languages
Chinese (zh)
Inventor
汪磊
潘富成
李强
李健存
谢永虎
喻金桃
殷继先
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Guanwei Technology Co ltd
Original Assignee
Beijing Guanwei Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Guanwei Technology Co ltd filed Critical Beijing Guanwei Technology Co ltd
Priority to CN202010162239.4A priority Critical patent/CN111353459A/en
Publication of CN111353459A publication Critical patent/CN111353459A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Abstract

The invention discloses a ship target detection method under a resource-limited condition, which comprises the following steps: inputting the satellite image into a first area screening network to obtain a ship target candidate area image; inputting the ship target candidate area image into a pre-trained ship target detection model for target detection to obtain a preliminary target detection result; removing the duplicate of the primary target detection result by using an NMS method to obtain a target detection result; the ship target detection model under the resource-limited condition comprises a second area screening network, a network connection layer and a target detection network which are sequentially cascaded. The method can realize rapid detection of the ship target under the condition of no GPU and solve the problem of rapid target detection under the condition of limited resources.

Description

Ship target detection method under resource-limited condition
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a ship target detection method under a resource-limited condition.
Background
In recent years, with the progress of aerospace technology, the means for acquiring remote sensing images are becoming mature, and the resolution of the images, including temporal resolution, spatial resolution, radiation resolution and spectral resolution, is continuously improving. At present, remote sensing breaks through the bottleneck of data acquisition, moves to a new stage of comprehensive application, and lays a data foundation for the extraction of ocean offshore targets. The ship is used as an important marine target, is a key target for marine monitoring and wartime striking, effectively acquires basic information of the ship in real time, and has great significance in civil and military fields. In the civil field, rescue of ships in distress is assisted, illegal behaviors such as smuggling, illegal oil stain dumping, illegal fishing and pirate are struck, and ship information is required to be acquired when monitoring maritime transportation of a specific port or a sea area; in the military field, important information such as the type, the position and the like of ships is determined by detecting, monitoring and identifying the ships in key ports and sea areas, so that the analysis of the environment situation of a sea battlefield is facilitated, the sea fighting strength of the other party is mastered, the fighting effect on the sea during fighting is evaluated, the sea fighting information is formed, and the basis is provided for the decision support of the sea battlefield.
At present, methods for ship detection are mainly divided into two main categories: traditional methods and deep learning based methods. The traditional satellite image target detection method mainly adopts a multi-step strategy from coarse to fine, generally comprises the steps of image preprocessing, sea and land segmentation, regional feature extraction, target discrimination and the like, and the traditional method needs to manually design a feature extraction method, so that the adaptability is poor, and the detection result is inaccurate.
Due to the strong feature representation and the end-to-end training learning capability of the deep learning technology, the deep learning technology is widely applied to the field of target detection and recognition, and the detection performance is greatly improved. The target detection method based on deep learning includes the steps of extracting a candidate region from an image, and then identifying the candidate region and performing bounding box regression by using a deep Neural Network (CNN) and the like to realize target detection and identification.
The defects of the existing ship detection method mainly comprise: the calculation bottleneck of the CNN is mainly a high-dimensional full-link layer, which has many parameters and high calculation complexity, easily causes over-fitting, and needs image input of the same size, and a common target detection algorithm adopts a general convolution kernel, so that the aspect ratio target detection effect is poor, and the accuracy of the detection result is poor. The most important current target detection method based on deep learning needs GPU acceleration, and research and application under the condition of resource limitation are relatively few.
Therefore, how to provide a ship target detection method under the condition of resource limitation is a problem which needs to be solved urgently by the technical personnel in the field.
Disclosure of Invention
In view of this, the invention provides a method for detecting a ship target under a resource-limited condition, which can realize quick detection of the ship target under the condition without a GPU and solve the problem of quick detection of the target under the resource-limited condition.
In order to achieve the purpose, the invention adopts the following technical scheme:
a ship target detection method under a resource-limited condition comprises the following steps: inputting the satellite image into a first area screening network to obtain a ship target candidate area image; inputting the ship target candidate area image into a pre-trained ship target detection model for target detection to obtain a preliminary target detection result; removing the duplicate of the primary target detection result by using an NMS method to obtain a target detection result; the ship target detection model under the resource-limited condition comprises a second area screening network, a network connection layer and a target detection network which are sequentially cascaded.
The method comprises the steps of selecting a PNet network of a MTCNN model for real-time face detection as a first region screening network, screening candidate regions of ship targets, and accelerating search, wherein the MTCNN model is composed of three small convolutional neural networks with different scales, namely PNet, RNet and ONet, the PNet is a region suggestion network and used for generating candidate targets, the PNet is a shallow full convolutional network and comprises three convolutional layers and a pooling layer, and the size of an input image is 12 pixels × 12 pixels.
Further, the second regional screening network firstly adjusts the position of the default frame for the first time, so that the corrected default frame is provided for the target detection network, and better initialization regression is provided for subsequent detection; adopting a special lightweight network to replace a classification network as a backbone network for target detection, then connecting a Light-head to realize rapid calculation, then connecting a convolution layer for detection generated by a 3x5 long convolution kernel, and outputting whether each default frame contains a target and rough quadrilateral position offset information; the network connection layer transmits the characteristics in the second area screening network to the target detection network to predict the target position, size and class label.
Further, the network connection part is used for converting the feature diagram in the second area screening network into the target detection network to establish the connection between the second area screening network and the target detection network, so that the target detection network can share the features of the second area screening network, and the target detection network adopts the 5-layer feature diagram generated by the second area screening network for conversion as input to fuse the features of different layers.
Further, designing a ship target detection model:
(1) a special target detection Res2Block network is used as a backbone network, two backbone networks and feature maps with different scales generated by the additionally added convolutional layers are respectively selected for detection, and the generated feature maps are used for the second area screening network and the target detection network;
(2) in a second area screening network, 5 feature maps with different scales are used for default frame classification and default frame position adjustment; in the target detection, 5 scale features are converted through network connection and used as the input of a target detection network for multi-class prediction and target accurate position regression;
and generating convolution layers for detection by using a 3x5 long convolution kernel, and acquiring the type and position information of each default frame after the 5 feature maps are used for prediction output, wherein the position information is the offset information of the coordinates of four points of the ship target.
Further, the default box prediction process is as follows:
1) first assume b0＝{x0,y0,w0,h0Denotes a default frame and the corresponding quadrangle is expressed by
Figure BDA0002406214410000041
Wherein (x)0,y0) Represents the center point of the default box, (w)0,h0) Representing the width and height of the default box, the computation method of the quadrilateral representation is shown in formula (1):
Figure BDA0002406214410000042
2) the ship target probability and the position offset of each default frame are predicted in the detection layer after the 5 convolutional layers, and the output values are as follows: (Δ x, Δ y; Δ w; Δ h; Δ x)1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4(ii) a c) Where c is the confidence of the prediction, Δ x, Δ y are the deviation of the center point of the default box, Δ w is the deviation of the width of the default box, Δ h is the deviation of the height of the default box, Δ x1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4Is the offset of the default box four vertices; in the training stage, the default frame and the labeling quadrangle are calculated to obtain a true value, and then a loss value is calculated according to the difference value between the true value and the predicted value.
Further, the ship target detection loss function is shown in formula (2):
Figure BDA0002406214410000043
where i denotes the number of default boxes,
Figure BDA0002406214410000044
a category representing a real box matching the ith default box,
Figure BDA0002406214410000045
indicating the location and size, p, of the real box matching the ith default boxiIndicates confidence, xiCoordinates representing a default box in the second area screening network, ciIndicates the prediction class, tiRepresenting predicted coordinate information in the target detection network; n is a radical ofrpnAnd NodnRespectively representing the number of positive sample default frames in the second area screening network and the target detection network; l isbRepresenting a binary classification penalty, LmIndicates a multi-class loss, LrThe regression loss is expressed as a function of time,
Figure BDA0002406214410000046
indicating that if the confidence of the negative example is greater than a threshold, then 1 is returned, otherwise 0 is returned; if N is presentrpnSet up as 0
Figure BDA0002406214410000051
And
Figure BDA0002406214410000052
if N is presentodnWhen the value is 0, then set
Figure BDA0002406214410000053
And
Figure BDA0002406214410000054
further, inputting a candidate area obtained by the area screening and screening network as a ship target detection model, detecting whether a ship target exists in the area, if so, predicting four coordinates of the target, mapping the coordinates of the ship target in the candidate area to a large-scale satellite image, and repeating the process in all the candidate areas;
and after all the candidate areas are detected, if the detected target areas obtained on the large satellite image are overlapped, removing the duplicate by adopting an NMS algorithm to obtain the final target detection result.
Further, the method for removing duplicate by using NMS algorithm to obtain the final target detection result comprises: firstly, sequencing all detection results according to the probability, and traversing a prediction box from high to low; and for each prediction frame, removing other prediction frames with the IOU larger than 0.5 in the same category as that of the current prediction frame to obtain the detection result after the duplication is removed.
Further, before the satellite image is input into the first area screening network, after the ship target data in the high-resolution satellite image is labeled, generated and subjected to data amplification processing to obtain a ship target detection data set, the length, the width and the length-width ratio of the ship target are subjected to clustering analysis, and length-width ratio parameters suitable for the ship target are designed according to the length-width ratio clustering result.
Compared with the prior art, the invention discloses and provides a ship target detection method under the condition of resource limitation, which has the following technical advantages:
firstly, the invention adopts a lightweight dedicated network as a backbone network to replace the prior classification network so as to improve the target detection performance;
secondly, a Light-head is matched with the Light-weight backbone network, so that the calculation speed is improved;
thirdly, a long convolution kernel method is adopted, the default frame density is increased in the vertical direction, multi-direction detection is better adapted, ship target detection in any direction with a large length-width ratio is achieved, the position deviation of four point coordinates of a target area is predicted, and the position of the ship target is more accurately represented by the four point coordinates.
In conclusion, the ship target detection method under the resource limited condition can realize quick detection of the ship target under the condition without GPU, and solve the problem of quick target detection under the resource limited condition.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a ship target detection method under resource-constrained conditions.
Fig. 2 is a schematic diagram of a ship target detection network under the resource-limited condition.
Fig. 3 is a diagram illustrating a backbone network structure according to the present invention.
Fig. 4 is a schematic diagram of the backbone network infrastructure of the present invention.
FIG. 5 is a schematic view of a Light-head structure according to the present invention.
FIG. 6 is a schematic diagram of a feature transformation structure according to the present invention.
FIG. 7 is a diagram illustrating a default box prediction process according to the present invention.
FIG. 8 is a schematic diagram illustrating vertical offset of default boxes according to the present invention.
FIG. 9 is a schematic diagram of a MTCNN model PNet network structure according to the present invention.
FIG. 10 is a schematic diagram of a ship target detection result of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a ship target detection method under a resource-limited condition, which comprises the following steps: inputting the satellite image into a first area screening network to obtain a ship target candidate area image; inputting the ship target candidate area image into a pre-trained ship target detection model for target detection to obtain a preliminary target detection result; removing the duplicate of the primary target detection result by using an NMS method to obtain a target detection result; the ship target detection model under the resource-limited condition comprises a second area screening network, a network connection layer and a target detection network which are sequentially cascaded.
The specific method comprises the following steps:
1. labeling of satellite image ship data
And carrying out quadrilateral annotation on the ship target in the high-resolution satellite image by using a quadrilateral annotation tool by using a four-point method, and storing all labeled target information in the satellite image to the local in an XML (extensive markup language) file format. The stored information comprises the coordinates of four points of the quadrangle, the ship type and other information.
2. Generation of training data and data augmentation
2.1 Generation of training data
And (4) according to the required size input by the deep learning model, combining the marked XML file, and cutting the marked large-scene high-resolution satellite image into sample data with a fixed size. The specific method comprises the following steps: the size of the deep learning network input image is used as the size of a sliding window, and the deep learning network input image slides on a large satellite image in an overlapped mode. If the current sliding window contains the effective ship target, the image corresponding to the current sliding window is cut, meanwhile, the coordinate of the ship in the current window relative to the whole satellite image is updated to be the coordinate in the current window, and the coordinate is stored as an XML (extensive makeup language) label file corresponding to the cut image.
The effective target distinguishing mode of the ship is as follows: and taking the area of the sliding window and the ratio of the overlapping area of the ship target quadrilateral area to the area of the ship quadrilateral area as a criterion, and if the area ratio is more than 0.5, determining the ship target as an effective ship target.
2.2 data augmentation
Deep learning is a data-driven learning method, and data augmentation is performed on training data by the following method in order to meet learning training requirements, improve generalization capability of a target detection model, and prevent overfitting of the target detection model.
(1) And generating a multi-scale high-resolution image by adopting a super-resolution countermeasure generation network (SRGAN), and generating 2-time and 4-time resolution images to expand training and verification data.
(2) And further expanding training and verifying data by methods of rotation, turnover, brightness contrast adjustment and the like. The verification data refers to data for testing the accuracy of the model during training.
3. Ship target aspect ratio cluster analysis
The length-width ratio parameter of the universal target detection algorithm is simple to set, and is not suitable for ship targets with variable directions and large length-width ratios. Therefore, after the ship target detection data set is obtained through the steps of data labeling, data generation and the like, the length, the width and the length-width ratio of the ship target are firstly subjected to clustering analysis, and length-width ratio parameters suitable for the ship target are designed according to the length-width ratio clustering result. And setting the length-width ratio parameter of the ship target detection as follows according to the data clustering result: 3:1,5:1,7:1,9:1, 11:1.
4. Ship target detection model design and training
4.1 design idea of ship target detection model
The invention adopts an end-to-end target detection framework, combines a character scene detection thought and Light weight target detection to design a ship target detection model, adopts a special Light weight network to replace a classification network as a backbone network for target detection, and then connects a Light-head to realize rapid calculation. The detection network is divided into three parts: the second area screening network, the network connection layer and the target detection network are used for cascading the second area screening network and the target detection network for traditional target detection to form an end-to-end network, so that the accuracy of a two-stage (the second area screening network and the target detection network are independent) target detection method is maintained, and the efficiency of the one-stage (the network directly performs target detection) target detection method is also maintained. By adopting the lightweight backbone network and the Light-head, the network can realize the rapid detection of the ship target under the condition without GPU, and the problem of the rapid detection of the target under the condition of resource limitation is solved. The structure of the ship target detection network under the resource-limited condition is shown in fig. 2.
(1) The second regional screening network firstly adjusts the position of the default frame for the first time, so that the corrected default frame is provided for the target detection network, and better initialization regression is provided for subsequent detection. The method comprises the steps of adopting a special lightweight network to replace a classification network as a backbone network for target detection, then connecting a Light-head to realize rapid calculation, then connecting a convolution layer which is generated by a 3x5 long convolution kernel and used for detection, and outputting whether each default frame contains a target and rough quadrilateral position offset information. The backbone network and Light-head structures are shown in fig. 3 and 5, respectively.
As shown in fig. 4, the backbone network basic module represents multi-scale features with finer granularity, and increases the receptive field range of each network layer, and the module has stronger multi-scale feature extraction capability. The Light-head is used to increase the computation speed, and the structure thereof is shown in fig. 5.
(2) The network connection part transmits the characteristics in the second area screening network to the target detection network to predict the target position, size and class label. The structure shown in fig. 6 is adopted for feature conversion, on one hand, a Light-head structure is used more to improve the calculation speed, and on the other hand, the output feature map of the second area screening network is converted into the input of the target detection network.
In order to establish a connection between the second area screening network and the target detection network, the feature map in the second area screening network is converted into the target detection network through the network connection part, so that the target detection network can share the features of the second area screening network. Network connectivity is used only on the feature maps associated with the default box. Network connectivity integrates large-scale context by adding advanced features to improve detection accuracy. To match dimensions, deconvolution operations are used to make the feature maps of higher layers large, pixel-level addition is used, and convolution layers are added after summation to ensure the discriminativity of the features for detection.
(3) The target detection network and the second regional screening network share the characteristics, the target detection network adopts 5-layer characteristic graphs generated by the second regional screening network to convert and serve as input, the characteristics of different layers are fused, and regression is further improved and the multi-class labels are predicted.
4.2 Ship target detection model design and implementation
(1) And a special target detection Res2Block network is adopted as a backbone network, two backbone networks and feature maps with different scales generated by the additionally added convolutional layers are respectively selected for detection, and the generated feature maps are shared by the second area screening network and the target detection network.
(2) In the second area screening network, 5 different scale feature maps are used for default frame two classification (with/without target) and default frame position adjustment; in the target detection, 5 scale features are converted through network connection and used as the input of a target detection network for multi-class prediction and target accurate position regression.
(3) In order to detect the target with a large aspect ratio, a convolution layer for detection is generated by using a 3x5 long convolution kernel, the convolution layer for detection is used for prediction output after the 5 feature maps, and the category and the position information of each default frame are obtained, wherein the position information is the bias information of the coordinates of four points of the ship target.
The detection layer is the core of the network, the default box is rectangular, the output is a quadrilateral prediction box, and the predicted is the bias information relative to the default box. The default box learning process is shown in fig. 7. The solid white line is the real box, the dashed white line is the default box on the match, and the white arrow represents the learning process.
The default box prediction process is as follows:
1) first assume b0＝{x0,y0,w0,h0Denotes a default frame and the corresponding quadrangle is expressed by
Figure BDA0002406214410000101
Wherein (x)0,y0) Represents the center point of the default box, (w)0,h0) Representing the width and height of the default box, the computation method of the quadrilateral representation is shown in formula (1):
Figure BDA0002406214410000102
2) the ship target probability and the position offset of each default frame are predicted in the detection layer after the 5 convolutional layers, and the output values are as follows: (Δ x, Δ y; Δ w; Δ h; Δ x)1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4(ii) a c) Where c is the confidence of the prediction, Δ x, Δ y are the deviation of the center point of the default box, Δ w is the deviation of the width of the default box, Δ h is the deviation of the height of the default box, Δ x1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4Is the offset of the default box four vertices; in the training stage, the default frame and the labeling quadrangle are calculated to obtain a true value, and then a loss value is calculated according to the difference value between the true value and the predicted value.
(4) With a vertical offset, the default boxes of the present invention are elongated, which may result in the default boxes being horizontally dense and vertically sparse, resulting in inaccurate detection. Thus, setting the default box vertical offset in the vertical direction makes the default box vertically dense, and a white solid box alone without vertical offset will miss many consecutive vertical direction objects. The white dotted frame is added with vertical offset, and can surround the ship target information, as shown in fig. 8.
(5) And (3) removing the duplicate of the predicted candidate target areas of different characteristic layers by using an NMS (non-maximum suppression) algorithm to obtain a final prediction result of the ship target position and the type information. Firstly, all detection results are sorted according to the probability size, and a prediction box is traversed from high to low. And for each prediction frame, removing other prediction frames with the IOU larger than 0.5 in the same category as that of the current prediction frame to obtain the detection result after the duplication is removed.
The ship target detection loss function is shown in formula (2):
Figure BDA0002406214410000111
where i denotes the number of default boxes,
Figure BDA0002406214410000112
a category representing a real box matching the ith default box,
Figure BDA0002406214410000113
indicating the location and size, p, of the real box matching the ith default boxiIndicates confidence, xiCoordinates representing a default box in the second area screening network, ciIndicates the prediction class, tiRepresenting predicted coordinate information in the target detection network; n is a radical ofrpnAnd NodnRespectively representing the number of positive sample default frames in the second area screening network and the target detection network; l isbRepresenting a binary classification penalty (presence/absence of target), LmIndicates a multi-class loss, LrThe regression loss is expressed as a function of time,
Figure BDA0002406214410000121
indicating that if the confidence of the negative example is greater than a threshold, then 1 is returned, otherwise 0 is returned; if N is presentrpnSet up as 0
Figure BDA0002406214410000122
And
Figure BDA0002406214410000123
if N is presentodnWhen the value is 0, then set
Figure BDA0002406214410000124
And
Figure BDA0002406214410000125
5. large-amplitude satellite image ship target detection
A flow chart of large-scale satellite image ship target detection is shown in fig. 1.
5.1 candidate region screening
The traditional sliding window method has overlapped sliding on a large satellite image, then a sliding window area is used as the input of a target detection model for target detection, the whole image needs to be traversed, and the calculation efficiency is low, so the method adopts a PNet network of a real-time face detection MTCNN model as a first area screening network to screen a ship target candidate area and accelerate the search speed, and the network structure is shown in figure 9.
5.2 Ship target detection and repeat target removal
(1) And inputting a candidate area obtained by the area screening and screening network as a detection model, detecting whether a ship target exists in the area, if so, predicting four coordinates of the target, mapping the coordinates of the ship target in the candidate area to the large-scale satellite image, and repeating the process in all the candidate areas.
(2) After all candidate areas are detected, the detected target areas obtained on the large satellite images may overlap, so in order to obtain a unique target detection area, the NMS algorithm is used for deduplication, and a final target detection result is obtained. A schematic diagram of the detection result of the ship target detection is shown in fig. 10.
The technical scheme provided by the invention adopts an end-to-end target detection framework, a special lightweight network Res2Block is adopted to replace a classified network as a backbone network for target detection, and then a Light-head is connected to realize rapid calculation; the target position can be more accurately represented by representing the prediction regression result by a four-point method. By adopting the lightweight backbone network and the Light-head, the network can realize the rapid detection of the ship target under the condition without GPU, solve the problem of rapid target detection under the condition of resource limitation and more accurately represent the predicted target in a quadrilateral mode.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A ship target detection method under a resource-limited condition is characterized by comprising the following steps: inputting the satellite image into a first area screening network to obtain a ship target candidate area image; inputting the ship target candidate area image into a pre-trained ship target detection model for target detection to obtain a preliminary target detection result; removing the duplicate of the primary target detection result by using an NMS method to obtain a target detection result; the ship target detection model under the resource-limited condition comprises a second area screening network, a network connection layer and a target detection network which are sequentially cascaded.
2. The method as claimed in claim 1, wherein the method comprises the steps of adopting a PNet network of a real-time face detection MTCNN model as a first region screening network to screen candidate regions of the ship target and increase the search speed, wherein the MTCNN model comprises three small convolutional neural networks with different sizes, namely PNet, RNet and ONet, the PNet is a region suggestion network used for generating the candidate target, the PNet is a shallow full convolutional network comprising three convolutional layers and a pooling layer, and the size of an input image is 12 pixels × 12 pixels.
3. The method for detecting the ship target under the resource-constrained condition according to claim 1, wherein the second regional screening network firstly adjusts the position of the default frame for the first time, so that the corrected default frame is provided for the target detection network, and better initialization regression is provided for subsequent detection; adopting a special lightweight network to replace a classification network as a backbone network for target detection, then connecting a Light-head to realize rapid calculation, then connecting a convolution layer for detection generated by a 3x5 long convolution kernel, and outputting whether each default frame contains a target and rough quadrilateral position offset information; the network connection layer transmits the characteristics in the second area screening network to the target detection network to predict the target position, size and class label.
4. The method as claimed in claim 2, wherein the connection between the second area screening network and the target detection network is established by transforming the feature map in the second area screening network into the target detection network through the network connection part, so that the target detection network can share the features of the second area screening network, and the target detection network uses the 5-layer feature map generated by the second area screening network for transformation as input to merge the features of different layers.
5. The ship target detection method under the resource-constrained condition according to claim 1 or 4, characterized in that a ship target detection model is designed:
(1) a special target detection Res2Block network is used as a backbone network, two backbone networks and feature maps with different scales generated by the additionally added convolutional layers are respectively selected for detection, and the generated feature maps are used for the second area screening network and the target detection network;
(2) in a second area screening network, 5 feature maps with different scales are used for default frame classification and default frame position adjustment; in the target detection, 5 scale features are converted through network connection and used as the input of a target detection network for multi-class prediction and target accurate position regression;
and generating convolution layers for detection by using a 3x5 long convolution kernel, and acquiring the type and position information of each default frame after the 5 feature maps are used for prediction output, wherein the position information is the offset information of the coordinates of four points of the ship target.
6. The method of claim 5, wherein the default box prediction process comprises:
1) first assume b0＝{x0,y0,w0,h0Denotes a default frame and the corresponding quadrangle is expressed by
Figure FDA0002406214400000021
Wherein (x)0,y0) Represents the center point of the default box, (w)0,h0) Representing the width and height of the default box, the computation method of the quadrilateral representation is shown in formula (1):
Figure FDA0002406214400000022
2) the detection layer after 5 convolutional layers will predict the ship target probability and bits for each default frameSetting the offset, wherein the output value is as follows: (Δ x, Δ y; Δ w; Δ h; Δ x)1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4(ii) a c) (ii) a Where c is the confidence of the prediction, Δ x, Δ y are the deviation of the center point of the default box, Δ w is the deviation of the width of the default box, Δ h is the deviation of the height of the default box, Δ x1,Δy1；Δx2,Δy2；Δx3,Δy3；Δx4,Δy4Is the offset of the default box four vertices; in the training stage, the default frame and the labeling quadrangle are calculated to obtain a true value, and then a loss value is calculated according to the difference value between the true value and the predicted value.
7. The method of claim 6, wherein the ship target detection loss function is shown in formula (2):
Figure FDA0002406214400000031
where i denotes the number of default boxes,
Figure FDA0002406214400000032
a category representing a real box matching the ith default box,
Figure FDA0002406214400000033
indicating the location and size, p, of the real box matching the ith default boxiIndicates confidence, xiCoordinates representing a default box in the second area screening network, ciIndicates the prediction class, tiRepresenting predicted coordinate information in the target detection network; n is a radical ofrpnAnd NodnRespectively representing the number of positive sample default frames in the second area screening network and the target detection network; l isbRepresenting a binary classification penalty, LmIndicates a multi-class loss, LrThe regression loss is expressed as a function of time,
Figure FDA0002406214400000034
indicating that if the confidence of the negative example is greater than a threshold, then 1 is returned, otherwise 0 is returned; if N is presentrpnSet up as 0
Figure FDA0002406214400000035
And
Figure FDA0002406214400000036
if N is presentodnWhen the value is 0, then set
Figure FDA0002406214400000037
And
Figure FDA0002406214400000038
8. the method for detecting the ship target under the resource-constrained condition as claimed in claim 1, wherein the candidate region obtained by the region screening network is input as a ship target detection model, whether the ship target exists in the region is detected, if so, four coordinates of the target are predicted, the coordinates of the ship target in the candidate region are mapped to the large-scale satellite image, and the process is repeated in all the candidate regions;
and after all the candidate areas are detected, if the detected target areas obtained on the large satellite image are overlapped, removing the duplicate by adopting an NMS algorithm to obtain the final target detection result.
9. The method for detecting ship targets under the condition of limited resources according to claim 8, wherein the method for removing duplicate by using NMS algorithm to obtain the final target detection result comprises: firstly, sequencing all detection results according to the probability, and traversing a prediction box from high to low; and for each prediction frame, removing other prediction frames with the IOU larger than 0.5 in the same category as that of the current prediction frame to obtain the detection result after the duplication is removed.
10. The method for detecting the ship target under the resource-constrained condition as recited in claim 1, wherein before the satellite image is input to the first regional screening network, after the ship target detection data set is obtained by labeling, data generating and data amplifying the ship target data in the high-resolution satellite image, the length, width and aspect ratio of the ship target are subjected to cluster analysis, and an aspect ratio parameter suitable for the ship target is designed according to an aspect ratio clustering result.
CN202010162239.4A 2020-03-10 2020-03-10 Ship target detection method under resource-limited condition Pending CN111353459A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010162239.4A CN111353459A (en) 2020-03-10 2020-03-10 Ship target detection method under resource-limited condition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010162239.4A CN111353459A (en) 2020-03-10 2020-03-10 Ship target detection method under resource-limited condition

Publications (1)

Publication Number Publication Date
CN111353459A true CN111353459A (en) 2020-06-30

Family

ID=71194397

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010162239.4A Pending CN111353459A (en) 2020-03-10 2020-03-10 Ship target detection method under resource-limited condition

Country Status (1)

Country Link
CN (1) CN111353459A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110516605A (en) * 2019-08-28 2019-11-29 北京观微科技有限公司 Any direction Ship Target Detection method based on cascade neural network
CN110516606A (en) * 2019-08-28 2019-11-29 北京观微科技有限公司 High-resolution satellite image any direction Ship Target Detection method
CN110659601A (en) * 2019-09-19 2020-01-07 西安电子科技大学 Depth full convolution network remote sensing image dense vehicle detection method based on central point
CN110909797A (en) * 2019-11-22 2020-03-24 北京深睿博联科技有限责任公司 Image detection method and device, equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110516605A (en) * 2019-08-28 2019-11-29 北京观微科技有限公司 Any direction Ship Target Detection method based on cascade neural network
CN110516606A (en) * 2019-08-28 2019-11-29 北京观微科技有限公司 High-resolution satellite image any direction Ship Target Detection method
CN110659601A (en) * 2019-09-19 2020-01-07 西安电子科技大学 Depth full convolution network remote sensing image dense vehicle detection method based on central point
CN110909797A (en) * 2019-11-22 2020-03-24 北京深睿博联科技有限责任公司 Image detection method and device, equipment and storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SHANGHUA GAO: "《Res2Net: A New Multi-scale Backbone Architecture》", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 *
ZHI TIAN: "《FCOS: Fully Convolutional One-Stage Object Detection》", 《2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)》 *

Similar Documents

Publication Publication Date Title
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN109271856B (en) Optical remote sensing image target detection method based on expansion residual convolution
CN108460382B (en) Optical remote sensing image ship detection method based on deep learning single-step detector
CN109299688B (en) Ship detection method based on deformable fast convolution neural network
CN109711295B (en) Optical remote sensing image offshore ship detection method
CN110378308B (en) Improved port SAR image near-shore ship detection method based on fast R-CNN
CN108491854B (en) Optical remote sensing image target detection method based on SF-RCNN
CN108596055B (en) Airport target detection method of high-resolution remote sensing image under complex background
Schöller et al. Assessing deep-learning methods for object detection at sea from LWIR images
CN113436169B (en) Industrial equipment surface crack detection method and system based on semi-supervised semantic segmentation
CN110598609A (en) Weak supervision target detection method based on significance guidance
CN111753677B (en) Multi-angle remote sensing ship image target detection method based on characteristic pyramid structure
CN110516605A (en) Any direction Ship Target Detection method based on cascade neural network
CN110516606A (en) High-resolution satellite image any direction Ship Target Detection method
CN111462140B (en) Real-time image instance segmentation method based on block stitching
CN106845458B (en) Rapid traffic sign detection method based on nuclear overrun learning machine
CN112348758B (en) Optical remote sensing image data enhancement method and target identification method
CN114841972A (en) Power transmission line defect identification method based on saliency map and semantic embedded feature pyramid
CN109800714A (en) A kind of ship detecting system and method based on artificial intelligence
CN113469050A (en) Flame detection method based on image subdivision classification
CN111666801A (en) Large-scene SAR image ship target detection method
CN111582182B (en) Ship name recognition method, system, computer equipment and storage medium
Zhou et al. Building segmentation from airborne VHR images using Mask R-CNN
CN114529821A (en) Offshore wind power safety monitoring and early warning method based on machine vision
CN112784757B (en) Marine SAR ship target significance detection and identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200630