CN111027511A - Remote sensing image ship detection method based on region of interest block extraction - Google Patents

Remote sensing image ship detection method based on region of interest block extraction Download PDF

Info

Publication number
CN111027511A
CN111027511A CN201911338663.3A CN201911338663A CN111027511A CN 111027511 A CN111027511 A CN 111027511A CN 201911338663 A CN201911338663 A CN 201911338663A CN 111027511 A CN111027511 A CN 111027511A
Authority
CN
China
Prior art keywords
image
detection
area
land
remote sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911338663.3A
Other languages
Chinese (zh)
Other versions
CN111027511B (en
Inventor
侯彪
刘佳丽
焦李成
马文萍
马晶晶
杨淑媛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201911338663.3A priority Critical patent/CN111027511B/en
Publication of CN111027511A publication Critical patent/CN111027511A/en
Application granted granted Critical
Publication of CN111027511B publication Critical patent/CN111027511B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration by the use of local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Abstract

The invention discloses an optical remote sensing image ship detection method based on region of interest block extraction, which mainly solves the problems of low detection precision and more false alarms in the prior art. The implementation scheme is as follows: constructing an optical remote sensing image ship detection data set; downsampling and defogging enhancing the wide remote sensing image, and carrying out land and water segmentation by using context information and image global characteristics; training a target detection model based on the SCRDEt by using the constructed data set; according to the land and water segmentation result, scanning the original wide remote sensing image by using a partially overlapped sliding window to extract an interested area block as a to-be-detected area, and inputting the to-be-detected area image into a detection model to obtain an area detection result; mapping the region result to the original wide image scale, and performing improved non-maximum suppression to optimize the primary detection result; and optimizing the detection result again according to the structural characteristics of the ship. The method has high detection precision and low false alarm rate, and can be used for acquiring the ship target of interest and the position thereof in the large-format remote sensing image.

Description

Remote sensing image ship detection method based on region of interest block extraction
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method for detecting a ship by using an optical remote sensing image, which can be used for target identification in a large-format remote sensing image.
Background
Target detection of optical remote sensing images is one of important problems in the field of remote sensing image research, and ship target detection has extremely important application value in aspects of fishery management, military reconnaissance, strategic deployment and the like due to the particularity and the criticality. The ship target detection is to determine whether ships exist in a water area or on the shore from a complex scene and position the ships.
The traditional ship target detection method mainly comprises methods of sea-land segmentation and prior geographic information, a large number of features need to be manually designed to position a ship, and the adaptability is not strong when targets of two different scenes, namely an onshore ship and an offshore ship, are detected simultaneously.
In recent years, deep learning is rapidly developed, and a deep convolutional neural network can automatically extract shallow and deep features of an image, so that manual feature extraction of a traditional method is avoided, and an advanced result is obtained in the field of image processing. Currently, methods based on deep learning are the mainstream methods for target detection.
The ship target detection method based on deep learning can be divided into two types according to the detection frame form: a horizontal-based detection block and a tilt-based detection block. Wherein:
the method based on the horizontal detection frame comprises a classic two-stage detection frame fast-RCNN, and a single-stage detection frame Yolo, RetinaNet and SSD.
A method for detecting a frame based on tilt, comprising R2CNN, ROI-Transformer, SCRDEt, etc.
Due to the characteristics of random ship direction, dense ships approaching to the shore and the like, the inclined detection frame can be used for more accurately positioning the ships. In the method adopting the tilt Detection frame, the SCRDEt (means More Robust Detection for Small, clustered and tracked Objects) model can fully fuse the bottom-layer features and the high-layer features, and achieve a better Detection result for the dense object. However, the method based on deep learning does not perform land and water segmentation on the image, but directly inputs the wide image slice into a trained model to generate a result, inputs a complex land area not containing a water area into a detection model, not only reduces the detection efficiency, but also may cause an obvious land false alarm, and in addition, due to the fact that the incomplete hull caused by image segmentation also causes obvious missing detection, the detection precision is not high.
Disclosure of Invention
The invention aims to provide an optical remote sensing image ship detection method based on region-of-interest block extraction aiming at the defects of the prior art so as to optimize the positioning of ship targets in a remote sensing image and improve the detection efficiency and precision.
In order to achieve the purpose, the technical scheme of the invention comprises the following steps:
(1) constructing an optical remote sensing image ship detection data set G:
1a) downloading high-grade second optical remote sensing data, manually screening areas containing ship targets, cutting the areas in a partially overlapped mode and storing the areas;
1b) randomly turning or rotating all the images obtained in the step 1a up, down, left and right to obtain amplified images and storing the amplified images;
1c) marking the amplified images by inclined rectangular frames, storing marking information as an xml format file, and forming an optical remote sensing image ship detection data set G by using all the amplified images and the corresponding marking information;
(2) downsampling the wide optical remote sensing image, and reducing cloud and fog shielding on the image by using a defogging algorithm based on dark channel prior to obtain an enhanced image I;
(3) carrying out land and water segmentation on the enhanced image I by utilizing the context information and the image global features, extracting interested water area blocks, and obtaining a binary image:
(3a) performing initial threshold segmentation on the enhanced image I by using the context characteristic information of the pixels to obtain an initial land-water segmentation binary image I3；
(3b) Marking initial land and water segmentation binary image I3The set W ═ W of the connected domain of the middle water area1,...,wi,...,wnIn which wiRepresenting the communication area of the ith water area; marking initial land and water segmentation binary image I3Set of continental linking domains L ═ { L ═ L1,...,lj,...,lmIn which ljRepresents the jth land-based communication domain; extracting the region characteristics of each water area communication region and each land communication region, and re-calibrating the classification according to a judgment rule to obtain an optimized land and water segmentation binary image I of the enhanced image I4；
(3c) For optimizing land and water segmentation binary image I4After morphological dilation operation, the up-sampling is recovered to the original wide image size to obtain the final land and water segmentation binary image I5；
(4) Method for training SCRDet target detection model M based on convolutional neural network by adopting random multi-scale strategy0Obtaining a trained detection model M;
(5) using the final land-water segmentation binary map I5Extracting interested blocks on the original wide image by using partially overlapped sliding windows as a region to be detected to form a region image set to be detectedF＝{f1,...,fi,...,fnThe position set is S ═ S1,...,si,...,snIn which fiDenotes the i-th detection area, siIndicates the detection area fiThe coordinates of the upper left corner are input into a detection model M to obtain an image set F of each region to be detectediThe area detection result of (1);
(6) mapping the region detection result to the original wide image scale according to the position set S to obtain a primary detection result set
Figure BDA0002331656970000031
Wherein P isi、
Figure BDA0002331656970000032
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the preliminary detection; performing improved non-maximum suppression on the primary detection result set A to obtain a primary optimized detection result set
Figure BDA0002331656970000033
Wherein Qi、
Figure BDA0002331656970000034
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the primary optimization;
(7) carrying out secondary optimization on the primary optimization detection result set B according to the structural characteristics of the ship to obtain a final detection result set
Figure BDA0002331656970000035
Wherein R isi、
Figure BDA0002331656970000036
And respectively representing the category, position coordinates and confidence score of the ith detection frame finally generated.
Compared with the prior art, the invention has the following advantages:
1. according to the characteristics of complex scene and large size of the wide remote sensing image, the method utilizes context information and global characteristics of the down-sampled and defogged enhanced image to carry out land and water segmentation, and carries out morphological operation, so that land and water segmentation results can be more effectively optimized, and ship targets in an area of interest are reserved.
2. According to the invention, the whole image is scanned through the partially overlapped sliding window, and the region of interest is extracted according to the land and water segmentation result to be used as the region to be detected, so that the false alarm on land is reduced, and the target detection efficiency is effectively improved; in addition, the improved non-maximum suppression is performed on the detection result on the wide scale, the missing detection caused by image slices is effectively reduced, and the detection precision is improved.
Drawings
FIG. 1 is a schematic flow chart of an implementation of the present invention;
FIG. 2 is a simulation result image of land and water segmentation of a wide remote sensing image using the present invention;
FIG. 3 is a simulation result image of a ship test performed on a wide remote sensing image using a conventional method;
FIG. 4 is a simulation result image of a ship test performed on a wide remote sensing image using the present invention.
Detailed Description
The embodiments and effects of the present invention will be further explained below with reference to the drawings.
Referring to fig. 1, the implementation steps of this embodiment are as follows:
step 1, constructing an optical remote sensing image ship detection data set G.
1.1) downloading high-score second-order optical remote sensing data, manually screening areas containing ship targets, cutting the areas by using a part of overlapped sliding windows with the size of 832 multiplied by 832 and the step length of 416, and storing the areas;
1.2) randomly turning or rotating all the images obtained in the step 1.1 up, down, left and right to obtain amplified images and storing the amplified images;
1.3) labeling all the amplification images obtained in the step 1.2) by using inclined rectangular frames, saving the labeling information as an xml format file, and forming an optical remote sensing image ship detection data set G by using all the amplification images and the corresponding labeling information.
And 2, downsampling and defogging enhancement of the wide optical remote sensing image.
And carrying out down-sampling on the original wide optical remote sensing Image by 8 times, and reducing cloud and fog shielding on the original wide optical remote sensing Image according to a defogging algorithm based on Dark Channel Prior, which is proposed by He-Cacamme in Single Image HazeRemoval Using Dark Channel Prior, so as to obtain an enhanced Image I.
And 3, performing land and water segmentation by using the context information and the image global features.
3.1) carrying out preliminary threshold segmentation on the enhanced image I by utilizing the context characteristic information of the pixels;
3.1.a) performing mirror image expansion with the width of 3 on the edge of the enhanced image I to obtain an expanded image I ', taking each pixel point of the enhanced image I as the center of a 7 multiplied by 7 sliding window, extracting a corresponding window area on the expanded image I', and calculating a regional variance V as a context characteristic value of the central pixel point to obtain a characteristic image V of the enhanced image I, wherein the characteristic value of the enhanced image I at the coordinate (I, j) is V (I, j);
3.1.b) determining the corresponding peak variance V at the histogram peak with 1 as the distribution histogram of the interval statistical feature map VmaxCalculate [ v ]maxAnd +∞) of the variance g of the valley corresponding to the first valley of the histogramt；
3.1.c) feature value of the enhanced image I at coordinate (I, j) is V (I, j) and valley variance gtAnd (3) comparison:
if V (i, j) is less than or equal to gtThe updated V (i, j) is 1,
otherwise, updating V (i, j) to 0;
3.1.d) morphological dilation of the updated feature V to obtain a feature binary image I1；
3.1.e) with improved Laplace operator
Figure BDA0002331656970000041
Performing Laplace gray scale transformation on the enhanced image I, and segmenting the enhanced image I after gray scale transformation by using an OTSU (over the Top) adaptive threshold method to obtain a gray scale binary image I2；
F) mapping the characteristic binary image I1And gray scale binary image I2Performing logic and operation on the pixel-by-pixel correspondence to obtain a preliminary land-water segmentation binary image I3In which I3The middle pixel value 0 represents land, and the pixel value 1 represents water;
3.2) carrying out category recalibration on the communication area by utilizing the characteristics of the water area communication area and the land communication area:
3.2.a) marking the preliminary land-water segmentation binary map I3The set W ═ W of the connected domain of the middle water area1,...,wi,...,wnIn which wiRepresenting the communication area of the ith water area, calculating the area of the communication areas of the n water areas, taking the three communication areas with the largest areas as wide-width image water area characteristic templates, and calculating the variance sigma of the water area characteristic templates2；
3.2.b) initializing an area threshold t1＝35×35；
3.2.c) traversing the water area communication area set W, and calculating W in WiArea of (2)
Figure BDA0002331656970000051
Sum variance
Figure BDA0002331656970000052
If it is
Figure BDA0002331656970000053
And is
Figure BDA0002331656970000054
Then recalibrate wiFor water area, otherwise, recalibrate wiIs land;
3.2.d) marking the preliminary land-water segmentation binary map I3Set of continental linking domains L ═ { L ═ L1,...,lj,...,lmIn which ljRepresents the jth land-based communication domain;
3.2.e) traverse the set L of terrestrial communication domains, calculate L in LjHas an area of
Figure BDA0002331656970000055
If it is
Figure BDA0002331656970000056
Then recalibratejFor water area, otherwise, recalibrate ljLand;
f) respectively assigning 0 and 1 to all pixels in the land communication domain and the water communication domain after re-calibration to obtain an optimized land-water segmentation binary image I4；
3.3) segmentation of binary maps I with optimized land4Obtaining a final land and water segmentation binary image I5：
3.3.a) replication of an optimized amphibian-segmentation binary map I4Obtaining a copied result graph I4'；
3.3.b) optimization of land-water segmentation binary image I with 15 × 15 elliptical structuring elements by pixel-by-pixel scan4And structural elements are combined with I4And performing logic and operation on all pixel points in the corresponding region:
3.3.c) determining the result of each pixel point, updating I4' center pixel within corresponding region:
if the result of each pixel point is 0, updating the central pixel of the corresponding area to be 0;
otherwise, updating the central pixel of the corresponding area to be 1;
3.3.d) updating I4' Up-sampling by a factor of 8, and4' the size of the image is restored to the original wide image size to obtain the final land and water segmentation binary image I5。
Step 4, training the SCRDEt model M based on the convolution neural network at random and multiple scales0。
4.1) taking 90% of the optical remote sensing image ship detection data set G constructed in the step one as a training sample, and taking the rest 10% as a test sample;
4.2) use of ResNet-50 network as SCRdet model M0And pre-training model parameters obtained by ResNet-50 network by using data set ImageNet, and then using the model parameters to carry out model M on the SCRdet0Initializing parameters of the backbone network;
4.3) randomly selecting a training image and a corresponding label in the training sample, and randomly turning the image and the label up and down or left and right to obtain a transformation image X and a real label Y;
4.4) random slave [600,700,832]Selecting a certain scale as the length of the short edge of the image, scaling the transformed image X and the real label Y to the selected scale, and inputting the scaled transformed image X and the real label Y into the SCRdet model M0Obtaining a prediction result Y';
4.5) calculating the error between the predicted result Y' and the true annotation Y, minimizing the error using an optimizer Adam to update the SCRDEt model M0The weight parameter of (2);
4.6) repeating 4.3) -4.5), and obtaining the trained target detection model M when the number of training rounds reaches 200000.
And 5, extracting the region to be detected on the original wide image, and inputting the region to be detected into the trained target detection model M to detect the region.
5.1) initializing a window parameter p which is 832, and initializing an image set F and a position set S of a region to be detected as an empty set;
5.2) scanning the original wide remote sensing image by using a partially overlapped sliding window with the size of p multiplied by p and the step length of p/2, and calculating a final land and water segmentation binary image I corresponding to a window area f of the original wide remote sensing image5Water area S in the areaw；
5.3) according to the area S of the water areawJudging whether the window area f is an interested area block:
if Sw＞(p/8)2If so, judging that the window area F is an interested area block, adding the window area F into the image set F of the area to be detected, adding the upper left corner coordinate S of the window area F into the position set S, and obtaining the image set F of the area to be detected as { F ═ F {1,...,fi,...,fnAnd the set of positions S ═ S1,...,si,...,sn}；
Otherwise, judging that the window area f is a non-interesting area block and not carrying out any operation;
5.5) set F ═ F of the region images to be detected1,...,fi,...,fnInputting the data into a detection model M to obtain each area fiAll ofThe category, position coordinates, confidence score of the detection box.
And 6, performing improved non-maximum suppression on the primary detection result on the original wide image scale.
6.1) according to the set of positions S ═ { S ═ S1,...,si,...,snArea f to be detected iniCoordinate s ofiEach region fiMapping the detection result to the original wide image scale to obtain a primary detection result set:
Figure BDA0002331656970000071
wherein P isi、
Figure BDA0002331656970000072
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the preliminary detection;
6.2) initializing the detection result set B after one-time optimization to be null;
6.3) assuming the classes of all detection frames in the preliminary detection result set A as the same class, and making the index of the detection result with the highest confidence level in the A
Figure BDA0002331656970000073
Taking out the detection result from A according to the index r
Figure BDA0002331656970000074
After adding set B, and
Figure BDA0002331656970000075
delete from set A;
6.4) calculating the intersection ratio of each detection frame in the preliminary detection result set A and the detection frame with the highest confidence coefficient
Figure BDA0002331656970000076
Namely the ratio of the overlapping area of the two detection frames to the area of the union region;
6.5) determination of the intersection-to-parallel ratio
Figure BDA0002331656970000077
Cross-over ratio threshold t2The size relationship of (1):
if it is
Figure BDA0002331656970000078
Then the confidence score is updated
Figure BDA0002331656970000079
If not, then,
Figure BDA00023316569700000710
not updating;
6.6) repeating the operations of the steps 6.3) -6.5) until the preliminary detection result set A is empty, and obtaining a detection result set after primary optimization:
Figure BDA00023316569700000711
wherein Qi、
Figure BDA00023316569700000712
And respectively representing the category, position coordinates and confidence score of the ith detection frame after the primary optimization.
Step 7, according to the structural characteristics of ships in the optical remote sensing image, all detection results in the detection result set B after one-time optimization
Figure BDA00023316569700000713
And (5) carrying out secondary optimization.
7.1) judge confidence score
Figure BDA00023316569700000714
The size of (2): if it is
Figure BDA00023316569700000715
The class l of the ith detection boxiOptimizing as background, otherwise optimizing as ship target;
7.2) according to the position coordinates of the detection frame
Figure BDA00023316569700000716
Obtaining the longest edge x of the detection frame, and judging the size of x: if x > 450, the class Q of the ith detection frame is determinediOptimizing as a background, otherwise, optimizing as a ship target;
7.3) according to the position coordinates of the detection frame
Figure BDA00023316569700000717
Calculating the ratio y of the longest edge and the shortest edge of the detection frame, and judging the size of y: if y is more than 11, the type Q of the ith detection frame is determinediAnd optimizing as a background, otherwise, optimizing as a ship target.
The effect of the invention can be further illustrated by the following simulation experiment:
1. simulation conditions are as follows:
the simulation experiment adopts a full-color image of a main city in China shot by an optical remote sensing satellite with a ground resolution of 1 meter or 4 meters.
The CPU used for simulation is Intel (R) core (TM) i7-8750H, the main frequency is 2.20GHz, the GPU is 8G GeForceGTX1080, the simulation platform is a UBUNTU 16.04 operating system, a Tensorflow deep learning framework is used, and Python3.6 software is adopted for experiment.
2. Simulation content and results:
simulation 1, carrying out land and water segmentation on 10000 × 10000 wide remote sensing images by using the method, and shielding land according to land and water segmentation results, wherein the result is shown in fig. 2, wherein black represents land, and the rest are interested water areas. As can be seen from figure 2, the land-bound ship is well preserved.
Simulation 2, using the existing convolution neural network-based method to directly perform slice-type one-by-one detection on 10000 × 10000 wide remote sensing images, and the result is shown in fig. 3, wherein a green frame marks the detected ship, and blue numbers are time-shared as confidence. As can be seen from fig. 3, when the wide remote sensing image is directly sliced, the missing detection is obviously caused by the defect of the ship, because the distribution characteristics of the ship are not considered in the existing method.
Simulation 3, the invention is used for carrying out ship target detection on 10000 × 10000 wide remote sensing images, and the result is shown in fig. 4, wherein a green frame marks the detected ships, and a blue number is a confidence score.
Comparing simulation 2 with simulation 3, the method effectively performs land and water segmentation on the premise of retaining the information of the ship on the shore, extracts the region of interest according to the land and water segmentation result, optimizes the region of interest by adopting a partial overlap detection strategy and improving non-maximum inhibition, and finally optimizes the detection result again by combining the ship structure information.

Claims (7)

1.A remote sensing image ship detection method based on region of interest block extraction is characterized by comprising the following steps:
(1) constructing an optical remote sensing image ship detection data set G:
1a) downloading high-grade second optical remote sensing data, manually screening areas containing ship targets, cutting the areas in a partially overlapped mode and storing the areas;
1b) randomly turning or rotating all the images obtained in the step 1a up, down, left and right to obtain amplified images and storing the amplified images;
1c) marking the amplified images by inclined rectangular frames, storing marking information as an xml format file, and forming an optical remote sensing image ship detection data set G by using all the amplified images and the corresponding marking information;
(2) downsampling the wide optical remote sensing image, and reducing cloud and fog shielding on the image by using a defogging algorithm based on dark channel prior to obtain an enhanced image I;
(3) carrying out land and water segmentation on the enhanced image I by utilizing the context information and the image global features, extracting interested water area blocks, and obtaining a binary image:
(3a) performing initial threshold segmentation on the enhanced image I by using the context characteristic information of the pixels to obtain an initial land-water segmentation binary image I3；
(3b) Marking initial land and water segmentation binary image I3Middle water area communicating areaSet W ═ W1,…,wi,…,wnIn which wiRepresenting the communication area of the ith water area; marking initial land and water segmentation binary image I3Set of continental linking domains L ═ { L ═ L1,…,lj,…,lmIn which ljRepresents the jth land-based communication domain; extracting the region characteristics of each water area communication region and each land communication region, and re-calibrating the classification according to a judgment rule to obtain an optimized land and water segmentation binary image I of the enhanced image I4；
(3c) For optimizing land and water segmentation binary image I4After morphological dilation operation, the up-sampling is recovered to the original wide image size to obtain the final land and water segmentation binary image I5；
(4) Method for training SCRDet target detection model M based on convolutional neural network by adopting random multi-scale strategy0Obtaining a trained detection model M;
(5) using the final land-water segmentation binary map I5Extracting an interested area block as a to-be-detected area on the original wide image by using a partially overlapped sliding window, and forming an image set of the to-be-detected area as F ═ { F ═ F1,…,fi,…,fnIs assembled into positions
S＝{s1,…,si,…,snIn which fiDenotes the i-th detection area, siIndicates the detection area fiThe coordinates of the upper left corner are input into a detection model M to obtain an image set F of each region to be detectediThe area detection result of (1);
(6) mapping the region detection result to the original wide image scale according to the position set S to obtain a primary detection result set
Figure FDA0002331656960000021
Wherein P isi、
Figure FDA0002331656960000022
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the preliminary detection; improved non-maxima suppression for preliminary test result set AObtaining a set of optimized detection results
Figure FDA0002331656960000023
Wherein Qi、
Figure FDA0002331656960000024
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the primary optimization;
(7) carrying out secondary optimization on the primary optimization detection result set B according to the structural characteristics of the ship to obtain a final detection result set
Figure FDA0002331656960000025
Wherein R isi、
Figure FDA0002331656960000026
And respectively representing the category, position coordinates and confidence score of the ith detection frame finally generated.
2. The method according to claim 1, wherein the preliminary threshold segmentation is performed on the enhanced image I in (3a) by using the context feature information of the pixels, which is implemented as follows:
3a1) performing mirror image expansion with the width of 3 on the edge of the enhanced image I to obtain an expanded image I ', taking each pixel point of the enhanced image I as the center of a 7 multiplied by 7 sliding window, extracting a corresponding window area on the expanded image I', and calculating a regional variance V as a context characteristic value of the central pixel point to obtain a characteristic image V of the enhanced image I, wherein the characteristic value of the enhanced image I at the coordinate (I, j) is V (I, j);
3a2) determining the corresponding peak variance V at the histogram peak by taking 1 as the distribution histogram of the interval statistical feature graph VmaxCalculate [ v ]maxAnd +∞) of the variance g of the valley corresponding to the first valley of the histogramtIf V (i, j) is less than or equal to gtIf the updated V (I, j) is 1, otherwise, the updated V (I, j) is 0, and the feature map V after updating is subjected to morphological dilation operation to obtain a feature binary map I1；
3a3) Carrying out gray scale transformation on the enhanced image I by using an improved Laplacian operator, and segmenting the enhanced image I by using an OTSU (over the Top) adaptive threshold method to obtain a gray scale binary image I2；
3a4) The characteristic binary image I1And gray scale binary image I2Performing logic and operation on the pixel-by-pixel correspondence to obtain a preliminary land-water segmentation binary image I3，I3The middle pixel value 0 represents land and the pixel value 1 represents water.
3.A method according to claim 1 wherein an optimized land-water cut binary map I is utilized in (3c)4Obtaining a final land and water segmentation binary image I5The implementation is as follows:
3c1) duplication optimization land-water segmentation binary map I4Obtaining a copied result graph I4'；
3c2) Optimization of land-water segmentation binary image I by pixel-by-pixel scanning of 15 x 15 elliptical structural elements4And structural elements are combined with I4And performing logic and operation on all pixel points in the corresponding region:
3c3) judging the result of each pixel point, and updating I4' center pixel within corresponding region:
if the result of each pixel point is 0, updating the central pixel of the corresponding area to be 0;
otherwise, updating the central pixel of the corresponding area to be 1;
3c4) will update the I4' Up-sampling by a factor of 8, and4' the size of the image is restored to the original wide image size to obtain the final land and water segmentation binary image I5。
4. The method of claim 1, wherein random multi-scale strategy is adopted in (4) to train deep convolutional neural network-based SCRDEt model M0And obtaining a detection model M, which is specifically realized as follows:
4a) taking 90% of the constructed optical remote sensing image ship detection data set G as a training sample, and taking the rest 10% as a test sample;
4b) randomly selecting a training image and a corresponding label in a training sample, and randomly turning the image and the label up and down or left and right to obtain a transformation image X and a real label Y;
4c) random slave [600,700,832]Selecting a certain scale as the length of the short edge of the image, scaling the transformed image X and the real label Y to the selected scale, and inputting the scaled transformed image X and the real label Y into the SCRdet model M0Obtaining a prediction result Y';
4d) calculating the error between the predicted result Y' and the real annotation Y, and using an optimizer Adam to minimize the error so as to update the SCRDEt model M0The weight parameter of (2);
4e) and repeating 4b) -4d), and obtaining the detection model M after training when the number of the training rounds reaches 200000.
5. The method of claim 1, wherein the final land-water segmentation binary map I is utilized in (5)5Extracting an interested area block from an original wide image as a to-be-detected area, inputting an image set F of the to-be-detected area into a detection model M to obtain an area detection result, and specifically realizing the following steps:
5a) initializing an image set F and a position set S of a region to be detected as an empty set;
5b) scanning the original wide remote sensing image by using a partial overlapping window with the size of p multiplied by p and the step length of p/2, and calculating a final land and water segmentation binary image I corresponding to a window area f of the original wide remote sensing image5The water area in the region is Sw；
5c) According to SwJudging whether the window area f is an interested area block:
if Sw＞(p/8)2If so, judging that the window area F is an interesting area block, adding the window area F into an image set F of the area to be detected, and adding the upper left corner coordinate S of the window area F into a position set S;
otherwise, judging that the window area f is a non-interesting area block and not carrying out any operation;
5d) f & ltf & gt of regional image set to be detected1,…,fi,…,fnInputting the data into a detection model M to obtain each area fiClass, position coordinates, confidence of all detection boxesAnd (4) degree score.
6. The method of claim 1, wherein in (6), the region detection result is mapped to the original wide image scale, and the preliminary detection result set a is subjected to improved non-maximum suppression, which is implemented as follows:
6a) according to the position set S ═ S { (S)1,…,si,…,snArea f to be detected iniCoordinate s ofiEach region fiMapping the detection result to the original wide image scale to obtain a primary detection result set
Figure FDA0002331656960000041
Wherein P isi、
Figure FDA0002331656960000042
Respectively representing the category, position coordinates and confidence score of the ith detection frame of the preliminary detection;
6b) initializing the detection result set B after the primary optimization to be null;
6c) the categories of all detection frames in the preliminary detection result set A are assumed to be the same category, and the index of the detection result with the highest confidence level in the A is made
Figure FDA0002331656960000043
Taking out the detection result from A according to the index r
Figure FDA0002331656960000044
After adding set B, and
Figure FDA0002331656960000045
delete from set A;
6d) calculating the intersection ratio of each detection frame in the preliminary detection result set A and the detection frame with the highest confidence coefficient
Figure FDA0002331656960000046
6e) Cross is judgedIs to be compared with
Figure FDA0002331656960000047
Cross-over ratio threshold t2The size relationship of (1):
if it is
Figure FDA0002331656960000048
Then the confidence score is updated
Figure FDA0002331656960000049
If not, then,
Figure FDA00023316569600000410
not updating;
6f) repeating the operations of the steps 6c) to 6e) until the preliminary detection result set A is empty, and obtaining a detection result set after primary optimization:
Figure FDA0002331656960000051
wherein Qi、
Figure FDA0002331656960000052
And respectively representing the category, position coordinates and confidence score of the ith detection frame after the primary optimization.
7. The method according to claim 1, wherein in (7), all the detection results in the once-optimized detection result set B are determined according to the structural features of the ship
Figure FDA0002331656960000053
And performing secondary optimization, specifically realizing the following steps:
7a) determining confidence score
Figure FDA0002331656960000054
The size of (2): if it is
Figure FDA0002331656960000055
Then will be firstClass l of i detection boxesiOptimizing as background, otherwise optimizing as ship target;
7b) according to the position coordinates of the detection frame
Figure FDA0002331656960000056
Obtaining the longest edge of the detection frame as x, and judging the size of the x: if x > 450, the class Q of the ith detection frame is determinediOptimizing as a background, otherwise, optimizing as a ship target;
7c) according to the position coordinates of the detection frame
Figure FDA0002331656960000057
Calculating the ratio of the longest side to the shortest side of the detection frame as y, and judging the size of y: if y is more than 11, the type Q of the ith detection frame is determinediAnd optimizing as a background, otherwise, optimizing as a ship target.
CN201911338663.3A 2019-12-23 2019-12-23 Remote sensing image ship detection method based on region of interest block extraction Active CN111027511B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911338663.3A CN111027511B (en) 2019-12-23 2019-12-23 Remote sensing image ship detection method based on region of interest block extraction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911338663.3A CN111027511B (en) 2019-12-23 2019-12-23 Remote sensing image ship detection method based on region of interest block extraction

Publications (2)

Publication Number Publication Date
CN111027511A true CN111027511A (en) 2020-04-17
CN111027511B CN111027511B (en) 2022-04-29

Family

ID=70212653

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911338663.3A Active CN111027511B (en) 2019-12-23 2019-12-23 Remote sensing image ship detection method based on region of interest block extraction

Country Status (1)

Country Link
CN (1) CN111027511B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626176A (en) * 2020-05-22 2020-09-04 中国科学院空天信息创新研究院 Ground object target detection method and system of remote sensing image
CN112949389A (en) * 2021-01-28 2021-06-11 西北工业大学 Haze image target detection method based on improved target detection network
CN112967335A (en) * 2021-03-10 2021-06-15 精英数智科技股份有限公司 Bubble size monitoring method and device
CN113205469A (en) * 2021-06-04 2021-08-03 中国人民解放军国防科技大学 Single image defogging method based on improved dark channel
CN113781502A (en) * 2021-09-30 2021-12-10 浪潮云信息技术股份公司 Method for preprocessing image training data with ultra-large resolution
CN115294066A (en) * 2022-08-09 2022-11-04 重庆科技学院 Sandstone particle size detection method
CN116091451A (en) * 2023-01-09 2023-05-09 中国科学院苏州生物医学工程技术研究所 Retinal pigment epithelial cell image segmentation method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150243032A1 (en) * 2014-02-26 2015-08-27 Raytheon Company False alarm rejection for boat detection candidates
CN107784655A (en) * 2016-12-28 2018-03-09 中国测绘科学研究院 A kind of visual attention model SAR naval vessels detection algorithm of adaptive threshold
CN108133468A (en) * 2017-12-25 2018-06-08 南京理工大学 Auto-adaptive parameter enhances and the constant false alarm rate Ship Detection of tail auxiliary detection
CN108171193A (en) * 2018-01-08 2018-06-15 西安电子科技大学 Polarization SAR Ship Target Detection method based on super-pixel local message measurement
CN108921066A (en) * 2018-06-22 2018-11-30 西安电子科技大学 Remote sensing image Ship Detection based on Fusion Features convolutional network
CN109409285A (en) * 2018-10-24 2019-03-01 西安电子科技大学 Remote sensing video object detection method based on overlapping slice

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150243032A1 (en) * 2014-02-26 2015-08-27 Raytheon Company False alarm rejection for boat detection candidates
CN107784655A (en) * 2016-12-28 2018-03-09 中国测绘科学研究院 A kind of visual attention model SAR naval vessels detection algorithm of adaptive threshold
CN108133468A (en) * 2017-12-25 2018-06-08 南京理工大学 Auto-adaptive parameter enhances and the constant false alarm rate Ship Detection of tail auxiliary detection
CN108171193A (en) * 2018-01-08 2018-06-15 西安电子科技大学 Polarization SAR Ship Target Detection method based on super-pixel local message measurement
CN108921066A (en) * 2018-06-22 2018-11-30 西安电子科技大学 Remote sensing image Ship Detection based on Fusion Features convolutional network
CN109409285A (en) * 2018-10-24 2019-03-01 西安电子科技大学 Remote sensing video object detection method based on overlapping slice

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
章林: "高分三号SAR图像舰船目标检测研究及实现", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626176A (en) * 2020-05-22 2020-09-04 中国科学院空天信息创新研究院 Ground object target detection method and system of remote sensing image
CN112949389A (en) * 2021-01-28 2021-06-11 西北工业大学 Haze image target detection method based on improved target detection network
CN112967335A (en) * 2021-03-10 2021-06-15 精英数智科技股份有限公司 Bubble size monitoring method and device
CN113205469A (en) * 2021-06-04 2021-08-03 中国人民解放军国防科技大学 Single image defogging method based on improved dark channel
CN113781502A (en) * 2021-09-30 2021-12-10 浪潮云信息技术股份公司 Method for preprocessing image training data with ultra-large resolution
CN115294066A (en) * 2022-08-09 2022-11-04 重庆科技学院 Sandstone particle size detection method
CN116091451A (en) * 2023-01-09 2023-05-09 中国科学院苏州生物医学工程技术研究所 Retinal pigment epithelial cell image segmentation method and system

Also Published As

Publication number Publication date
CN111027511B (en) 2022-04-29

Similar Documents

Publication Publication Date Title
CN111027511B (en) Remote sensing image ship detection method based on region of interest block extraction
CN109977812B (en) Vehicle-mounted video target detection method based on deep learning
CN108961235B (en) Defective insulator identification method based on YOLOv3 network and particle filter algorithm
CN109583425B (en) Remote sensing image ship integrated recognition method based on deep learning
CN108573276B (en) Change detection method based on high-resolution remote sensing image
CN109409263B (en) Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network
CN109934200B (en) RGB color remote sensing image cloud detection method and system based on improved M-Net
CN107871119B (en) Target detection method based on target space knowledge and two-stage prediction learning
CN113160192B (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN108596055B (en) Airport target detection method of high-resolution remote sensing image under complex background
CN108615226B (en) Image defogging method based on generation type countermeasure network
CN103049763B (en) Context-constraint-based target identification method
WO2017071160A1 (en) Sea-land segmentation method and system for large-size remote-sensing image
CN110276264B (en) Crowd density estimation method based on foreground segmentation graph
CN112084869B (en) Compact quadrilateral representation-based building target detection method
CN109255350B (en) New energy license plate detection method based on video monitoring
CN109840483B (en) Landslide crack detection and identification method and device
CN111126335B (en) SAR ship identification method and system combining significance and neural network
CN113076871A (en) Fish shoal automatic detection method based on target shielding compensation
CN114677554A (en) Statistical filtering infrared small target detection tracking method based on YOLOv5 and Deepsort
CN109829423B (en) Infrared imaging detection method for frozen lake
CN112560675B (en) Bird visual target detection method combining YOLO and rotation-fusion strategy
CN112818905B (en) Finite pixel vehicle target detection method based on attention and spatio-temporal information
CN111091095A (en) Method for detecting ship target in remote sensing image
CN112836650A (en) Semantic analysis method and system for quality inspection report scanning image table

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant