CN111368599A - Remote sensing image sea surface ship detection method and device, readable storage medium and equipment - Google Patents
Remote sensing image sea surface ship detection method and device, readable storage medium and equipment Download PDFInfo
- Publication number
- CN111368599A CN111368599A CN201811599968.5A CN201811599968A CN111368599A CN 111368599 A CN111368599 A CN 111368599A CN 201811599968 A CN201811599968 A CN 201811599968A CN 111368599 A CN111368599 A CN 111368599A
- Authority
- CN
- China
- Prior art keywords
- sea surface
- image
- candidate region
- network
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 65
- 230000011218 segmentation Effects 0.000 claims abstract description 152
- 238000000034 method Methods 0.000 claims abstract description 113
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims description 51
- 230000015654 memory Effects 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 11
- 230000000877 morphologic effect Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 238000002372 labelling Methods 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 6
- 238000003909 pattern recognition Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 14
- 238000004364 calculation method Methods 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000003287 optical effect Effects 0.000 description 10
- 238000004590 computer program Methods 0.000 description 8
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 5
- 230000003628 erosive effect Effects 0.000 description 4
- 230000009467 reduction Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000009338 overlapping cropping Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Abstract
The invention discloses a method and a device for detecting a remote sensing image sea surface ship, a computer readable storage medium and equipment, belonging to the field of image processing and pattern recognition. The method comprises the following steps: inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph; according to the connectivity of the sea surface, a candidate region is positioned on the sea surface segmentation result graph, and a candidate region image is extracted from the image to be detected according to the positioned candidate region; and inputting the candidate area image into a VGG network after fine-tuning, and judging whether the candidate area image is a ship or not. The invention can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
Description
Technical Field
The invention relates to the field of image processing and pattern recognition, in particular to a method and a device for detecting remote sensing image sea surface ships, a computer readable storage medium and equipment.
Background
In recent years, with the development of remote sensing imaging technology, the resolution of remote sensing images is continuously improved, so that optical remote sensing image detection is possible. In military and civil fields, ship detection plays an important role, so that the optical remote sensing image is utilized to quickly and accurately detect ships, and the method has a huge application prospect.
The steps of ship detection of the existing optical remote sensing image are generally mainly divided into four steps: firstly, a land mask is used for shielding or removing a land area and cloud shielding in an image, so that ship detection only acts on an ocean area and does not process the land area; image preprocessing, namely, through a series of image processing operations, the purpose is to suppress clutter background, strengthen and highlight targets, remove interference on ship detection caused by various factors, and improve the accuracy and reliability of detection as much as possible; thirdly, image segmentation, namely segmenting the suspected target in the ocean area from the background by using a corresponding ship detection algorithm; and fourthly, deleting the false alarm, wherein the main purpose is to screen the suspected targets obtained by segmentation through priori knowledge and manual intervention and remove interference information from the suspected targets.
When the image is segmented in the prior art, the adopted method mainly comprises a global threshold algorithm, and the image is segmented by setting a fixed global threshold, so that the method has the advantages of simple threshold calculation and low operation complexity, and has the disadvantages that the global threshold cannot automatically adjust the threshold according to the change of a local area in the image, and a detection result is easy to cause a large amount of false alarms and missed detections due to local changes; the other method is a threshold detection algorithm based on a sliding window, the method considers the local change of the marine environment, the selected detection threshold can better accord with the statistical characteristics of the detected local area, but the method easily causes a large amount of false alarms when the spot noise is more and the sea surface storm is larger, and meanwhile, the calculation amount is extremely large and the processing speed is slow because the statistical parameters of the background area in the window need to be repeatedly calculated. Therefore, the existing ship detection method cannot well detect ships on the sea surface or has low detection speed.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method, a device, a computer readable storage medium and equipment for detecting a remote sensing image sea surface ship.
The technical scheme provided by the invention is as follows:
in a first aspect, the present invention provides a method for detecting a remote sensing image sea surface ship, the method comprising:
inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph;
according to the connectivity of the sea surface, a candidate region is positioned on the sea surface segmentation result graph, and a candidate region image is extracted from the image to be detected according to the positioned candidate region;
and inputting the candidate area image into a VGG network after fine-tuning, and judging whether the candidate area image is a ship or not.
Further, the U-net segmentation network is obtained by training through the following method:
labeling each pixel point of the training images in the training image set;
performing overlapping clipping on the marked training images;
and inputting the overlapped and cut images into a U-net segmentation network, and training by adopting a random gradient descent method.
Further, the inputting the image to be detected into the trained U-net segmentation network to obtain a sea segmentation result graph, which includes:
performing overlapping clipping on an image to be detected to obtain a sub-image;
inputting the subimages into the trained U-net segmentation network, and performing sea surface segmentation;
and splicing the subimages obtained after the sea surface segmentation to obtain a sea surface segmentation result graph.
Further, the step of locating a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface and extracting a candidate region image from the image to be detected according to the located candidate region includes:
carrying out binarization processing on the sea surface segmentation result graph;
performing morphological closing operation on the binarized image;
finding out all connected areas on the sea surface according to the connectivity of the sea surface, and obtaining the minimum circumscribed rectangle of the connected areas;
taking the minimum bounding rectangle with the area not smaller than a set threshold as the candidate region;
and extracting a candidate region image from the image to be detected according to the candidate region.
Further, the fine-tuning VGG network is obtained by the following method:
establishing a training sample set;
changing the number of output channels of the last full-connection layer of the VGG16 network by 2, and keeping other layers unchanged;
and performing fine-tuning on the changed VGG16 network by using the training sample set to obtain a fine-tuning VGG network.
In a second aspect, the present invention provides a remote sensing image sea surface ship detection device, comprising:
the sea surface segmentation module is used for inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph;
the candidate region module is used for positioning a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface and extracting a candidate region image from the image to be detected according to the positioned candidate region;
and the classification module is used for inputting the candidate area image into the VGG network after the fine-tuning and judging whether the candidate area image is a ship or not.
Further, the U-net segmentation network is obtained by training through the following modules:
the marking module is used for marking each pixel point of the training images in the training image set;
the overlap cropping module is used for performing overlap cropping on the marked training image;
and the training module is used for inputting the overlapped and cut images into the U-net segmentation network and training by adopting a random gradient descent method.
Further, the sea surface segmentation module comprises:
the overlap cropping unit is used for performing overlap cropping on the image to be detected to obtain a sub-image;
the sea surface segmentation unit is used for inputting the subimages into the trained U-net segmentation network to perform sea surface segmentation;
and the splicing unit is used for splicing the subimages obtained after the sea surface segmentation to obtain a sea surface segmentation result graph.
Further, the candidate region module includes:
the binarization unit is used for carrying out binarization processing on the sea surface segmentation result graph;
a morphology closing operation unit, which is used for performing morphology closing operation on the binarized image;
the device comprises a connected region acquisition unit, a communication unit and a communication unit, wherein the connected region acquisition unit is used for finding out all connected regions on the sea surface according to the connectivity of the sea surface and acquiring the minimum circumscribed rectangle of the connected regions;
a candidate region determination unit configured to determine, as the candidate region, a minimum bounding rectangle having an area not smaller than a set threshold;
and the candidate region image extraction unit is used for extracting a candidate region image from the image to be detected according to the candidate region.
Further, the fine-tuning VGG network is obtained by the following modules:
the training sample set module is used for establishing a training sample set;
the modification module is used for changing the output channel number of the last full connection layer of the VGG16 network by 2, and other layers are not changed;
and the fine adjustment module is used for performing fine-tuning on the VGG16 network by using the changed training sample set to obtain the VGG network after the fine-tuning.
In a third aspect, the present invention provides a computer readable storage medium for remote sensing image sea surface ship detection, comprising a processor and a memory for storing processor executable instructions, wherein the instructions, when executed by the processor, implement the steps of the remote sensing image sea surface ship detection method according to the first aspect.
In a fourth aspect, the present invention provides an apparatus for remote sensing image sea surface ship detection, comprising at least one processor and a memory storing computer executable instructions, wherein the processor executes the instructions to implement the steps of the remote sensing image sea surface ship detection method according to the first aspect.
The invention has the following beneficial effects:
the invention relates to a method for detecting ships on the sea surface, which can be used for segmenting the sea surface and ships. The method uses the U-net segmentation network to simply and effectively segment the sea surface, applies the U-net segmentation algorithm to the detection task of the optical remote sensing image, does not need complex image processing operation and manual feature extraction, has high segmentation speed and high segmentation accuracy, and can effectively reduce the false alarm rate; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
Drawings
FIG. 1 is a flow chart of a method for detecting a remote sensing image sea surface ship according to the present invention;
FIG. 2 is a schematic diagram of a U-net split network;
FIG. 3 is an exemplary diagram of images before and after sea segmentation;
FIG. 4 is an exemplary illustration of candidate regions located on a sea segmentation result graph;
FIG. 5 is an exemplary diagram of a single candidate region image extracted from an image to be detected;
FIG. 6 is an exemplary diagram of a ship test result;
FIG. 7 is a schematic diagram of the architecture of a VGG16 network;
FIG. 8 is an exemplary diagram of a positive sample;
FIG. 9 is an exemplary diagram of a negative example;
fig. 10 is a schematic view of the remote sensing image sea surface ship detection device of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
the embodiment of the invention provides a remote sensing image sea surface ship detection method, as shown in figure 1, the method comprises the following steps:
step S100: and inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph.
In the step, the image to be detected is input into the trained U-net segmentation network to obtain the classification probability of each pixel point in the image to be detected, and finally the pixel points segmented into the sea surface and the pixel points segmented into the non-sea surface in the image to be detected are set to be different values, so that the sea surface and the non-sea surface are distinguished, and a sea surface segmentation result graph is obtained.
The embodiment of the invention adopts the U-net segmentation network to carry out sea surface segmentation, and the network supports a small amount of data training, and has high speed and high segmentation accuracy.
The U-net segmentation network only consists of convolution layers and sampling layers, and comprises 19 convolution operations, 4 times of upsampling and 4 times of downsampling, and the specific network structure is shown in fig. 2, wherein the sampling layer is 2 x 2 maxporoling, the sizes of two copied feature maps in the network structure are the same, and the cropping operation is not carried out in all cropping and copying operations. The size of the resulting output image is the same as the size of the input image.
The image to be detected is input into a trained U-net segmentation network after being preprocessed, the U-net segmentation network can obtain sea surface probability and non-sea surface probability of the image to be detected pixel by pixel, when the sea surface probability obtained by the pixel is larger than the non-sea surface probability, the pixel is considered as a sea surface, when the sea surface probability obtained by the pixel is smaller than the non-sea surface probability, the pixel is considered as a non-sea surface, the pixels on the sea surface and the non-sea surface are set to be different values, and finally a sea surface segmentation result graph is obtained. The image to be detected and the sea surface segmented image are shown in fig. 3, wherein the left side is the original image to be detected, the right side is the segmented image, the gray part of the segmented image is the sea surface, and the white part is the non-sea surface.
Step S200: and positioning a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface, and extracting a candidate region image from the image to be detected according to the positioned candidate region.
After the sea surface is divided, the sea surface and the non-sea surface are distinguished, the non-sea surface outside the sea surface is a land part, the non-sea surface inside the sea surface is primarily considered as a ship, and the non-sea surface part inside the sea surface is selected as a candidate area. And because the whole sea surface is communicated, a candidate region can be extracted by a region communication method according to the connectivity of the sea surface, the candidate region is preferably represented by a square frame, and then a candidate region image is extracted from the image to be detected according to the candidate region and used for subsequent classification.
As shown in fig. 4 and 5, the two graphs of fig. 4 are candidate regions located on the sea surface segmentation result graph, and the series of graphs of fig. 5 are individual candidate region images extracted from the image to be detected (original image).
Step S300: and inputting the candidate area image into the VGG network after the fine-tuning, and judging whether the candidate area image is a ship or not.
From the foregoing, the candidate region is a non-sea surface portion within the sea surface, but the portion may be a ship, a small reef, and even a result caused by a sea surface segmentation error, and therefore the candidate region needs to be classified, and when the classification is performed, the classification is performed based on the original image, and therefore the candidate region image needs to be extracted from the original image according to the candidate region, and whether the candidate region image is a ship or not is determined.
The embodiment of the invention adopts the VGG network for classification, and the VGG16 network needs to be subjected to fine-tuning so as to be adapted to the method of the invention. After the candidate area image is input into the find-tuning VGG network, the find-tuning VGG network divides the candidate area image into two types, one type is a ship, the other type is a non-ship, the candidate area is reserved as a ship, and the candidate area is marked on the original image, so as to obtain a final ship detection result, as shown in fig. 6, the detected ship is in the box.
The invention relates to a method for detecting ships on the sea surface, which can be used for segmenting the sea surface and ships. The method uses the U-net segmentation network to simply and effectively segment the sea surface, applies the U-net segmentation algorithm to the detection task of the optical remote sensing image, does not need complex image processing operation and manual feature extraction, has high segmentation speed and high segmentation accuracy, and can effectively reduce the false alarm rate; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
In the embodiment of the invention, the U-net segmentation network is preferably obtained by training through the following method:
step 100': and marking each pixel point of the training images in the training image set.
In this step, labeling can be performed by a LableImg labeling tool, and the labeling rule is as follows: when the pixel belongs to the sea, the label is 1, and when the pixel does not belong to the sea, the label is 0.
Step 200': and performing overlapping cropping on the marked training images.
In this step, when the labeled training images are cut in an overlapping manner, a certain overlap between the horizontal direction and the vertical direction is ensured, the overlapping step length is preferably 40 pixels, and the cut images are uniform in size, preferably 160 × 160.
Step 300': and inputting the overlapped and cut images into a U-net segmentation network, and training by adopting a random gradient descent method. The parameters of the training process are set as follows: the blocksize is 16, namely 16 images are input into the U-net segmentation network each time, the iteration time is 10 ten thousand times, the gradient updating method is a small-batch random gradient descent algorithm (SGD), and finally the U-net segmentation network model for sea surface segmentation is obtained.
After the U-net segmentation network is trained, inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph, which specifically comprises the following steps:
step S110: and performing overlapping cropping on the image to be detected to obtain a sub-image.
In this step, the overlap clipping method is the same as the method in step 200 ', and can be understood with reference to step 200', which is not described herein again.
Step S120: and inputting the subimages into the trained U-net segmentation network to perform sea surface segmentation.
In this step, preprocessing is performed before the subimage is input into the U-net segmentation network, and the preprocessing includes mean value reduction and normalization processing: the mean value 128 is subtracted from each channel of the sub-image and multiplied by a scaling factor of 0.0078(1/128) to normalize the pixel values to [ -1,1 ].
And then inputting the subimage after the average value is reduced into a trained U-net segmentation network, wherein the U-net segmentation network can obtain sea surface probability and non-sea surface probability of pixel points of the subimage one by one, when the sea surface probability obtained by the pixel points is greater than the non-sea surface probability, the pixel points are considered as sea surfaces, when the sea surface probability obtained by the pixel points is smaller than the non-sea surface probability, the pixel points are considered as non-sea surfaces, preferably, the value of the pixel points of the sea surfaces can be set to be 1, and the value of the pixel points of the non-sea surfaces can be set to be 0.
Step S130: and splicing the sub-images after sea surface segmentation to obtain a large image with the same size as the original image, namely a sea surface segmentation result image, as shown in fig. 3.
As an improvement of the present invention, a candidate region is located on the sea surface segmentation result graph according to the connectivity of the sea surface, and a candidate region image is extracted from the image to be detected according to the located candidate region (step S200), including:
step S210: and converting the sea surface segmentation result image into a gray level image and carrying out binarization processing.
The pixel value of the pixel point segmented into the sea surface is preferably 0, and the pixel value of the pixel point segmented into the non-sea surface is preferably 1; that is, the value of 1 at the pixel point in the sea surface segmentation result map is set to 0, and the value of 0 at the pixel point is set to 1.
Step S220: and performing morphological closing operation on the binarized image.
The morphological closing operation is: the expansion operation is a process of expanding the boundary to the outside and filling up the cavity in the object, and the erosion operation is a process of contracting the boundary to the inside and eliminating small and meaningless objects. The embodiment of the invention can fill small holes by using morphological closing operation, so that some pixel points which are wrongly divided into sea surfaces are corrected.
In the present invention, the dilation operation is to scan each pixel of the image with a structuring element of 3 × 3, and the and operation is performed with the structuring element and the binary image it covers, if both are 0, then that pixel of the image is 0, otherwise 1, resulting in a one-round expansion of the binary image, the erosion operation is to scan each pixel of the image with a structuring element of 3 × 3, and the and operation is performed with the structuring element and the binary image it covers, if both are 1, resulting in that pixel of the image is 1, otherwise 0, resulting in a one-round reduction of the binary image.
Step S230: and finding out all connected areas on the sea surface according to the connectivity of the sea surface, and obtaining the minimum circumscribed rectangle of the connected areas, namely obtaining the object on the sea surface.
In this step, the connected region may be an eight-connected region or a four-connected region. The eight-connected region is a combination of movement in eight directions, namely, from one pixel in the region, and other pixel points in the region can be reached through the combination of movement in the eight directions, namely, up, down, left, right, up-left, up-right, down-left and down-right. The four-connected region is that starting from one pixel in the region, other pixel points in the region can be reached through the combination of movement in the upper direction, the lower direction, the left direction and the right direction.
Step S240: and taking the minimum bounding rectangle with the area not less than the set threshold as a candidate region.
The method comprises the following steps of performing primary screening according to the pixel point area of an object in an image, wherein the calculation method comprises the following steps: the length and the width of the rectangle are calculated according to the coordinates of the upper left corner and the lower right corner of the circumscribed rectangle, then the area of the rectangle is calculated, the minimum circumscribed rectangle with the area not smaller than a preset value (for example, the preset value is 20) is used as a candidate area, and the false detection object with extremely small area is reduced.
Step S250: and extracting a candidate region image from the image to be detected according to the candidate region to serve as a subsequent classification basis.
The embodiment of the invention obtains more accurate ship candidate areas by using morphological closing operation and an area communication method, and accurately identifies the ships.
In the invention, the VGG network after the fine-tuning is obtained by the following method:
first, a VGG network is introduced, which may be a VGG16 network, and the structure of the VGG16 network is shown in fig. 7: the convolutional neural network comprises 5 convolutional neural networks ConvNet connected in sequence, each ConvNet comprises a plurality of convolutional layers conv and a pooling layer pool, and after the convolutional layers conv and the pooling layer pool, the convolutional layers conv and the pooling layer pool are fully connected through 3 layers. The outputs of both the convolutional layer and the fully-connected layer in the network are activated via the ReLU. data is the input image, Conv1+ pool1 is the first convolutional neural network, Conv1 includes two convolutional layers, Conv2+ pool2 is the second convolutional neural network, and so on. fc6, fc7 and fc8 are three-layer full connections.
The VGG16 network of the invention firstly adopts the image of ImageNet database to be trained, then the embodiment of the invention carries out fine-tuning (fine-tuning) on the VGG16 network after being trained, and the fine-tuning method comprises the following steps:
step 100': and establishing a training sample set.
Wherein, the positive sample in the training sample set is shown in fig. 8, the positive sample can be preferably an image of a ship containing more than two thirds of the ship hull, the negative sample in the training sample set is shown in fig. 9, and the negative sample can be preferably an image containing less than two thirds of the ship hull or no ship.
Step 200': the number of output channels of the final full-connection layer fc8 of the VGG16 network is changed by 2, two classifications are carried out (even if the ship is not the ship), and other layers are not changed.
Step 300': and performing fine-tuning on the changed VGG16 network by using the training sample set to obtain a fine-tuning VGG network.
After the VGG network is finely adjusted, the candidate area image is adjusted to the required size (224 × 224) of the VGG16 network, and then the VGG network after the fine-tuning is input to judge whether the VGG network is a ship or not.
According to the method for detecting the ship on the sea surface, provided by the embodiment of the invention, the U-net segmentation network is used for simply and effectively segmenting the sea surface, complex image processing operation and manual feature extraction are not needed, the segmentation speed is high, the segmentation accuracy is high, and the false alarm rate can be effectively reduced; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
Example 2:
the embodiment of the invention provides a remote sensing image sea surface ship detection device, as shown in fig. 10, the device comprises:
and the sea surface segmentation module 10 is used for inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph.
And in the sea surface segmentation module, inputting the image to be detected into the trained U-net segmentation network to obtain the classification probability of each pixel point in the image to be detected, and finally setting the pixel points segmented into the sea surface and the pixel points segmented into the non-sea surface in the image to be detected to be different values, so that the sea surface and the non-sea surface are distinguished to obtain a sea surface segmentation result graph.
The embodiment of the invention adopts the U-net segmentation network to carry out sea surface segmentation, and the network supports a small amount of data training, and has high speed and high segmentation accuracy. The U-net split network architecture is described in example 1.
The image to be detected is input into a trained U-net segmentation network after being preprocessed, the U-net segmentation network can obtain sea surface probability and non-sea surface probability of the image to be detected pixel by pixel, when the sea surface probability obtained by the pixel is larger than the non-sea surface probability, the pixel is considered as a sea surface, when the sea surface probability obtained by the pixel is smaller than the non-sea surface probability, the pixel is considered as a non-sea surface, the pixels on the sea surface and the non-sea surface are set to be different values, and finally a sea surface segmentation result graph is obtained. The image to be detected and the sea surface segmented image are shown in fig. 3, wherein the left side is the original image to be detected, the right side is the segmented image, the gray part of the segmented image is the sea surface, and the white part is the non-sea surface.
And the candidate region module 20 is configured to locate a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface, and extract a candidate region image from the image to be detected according to the located candidate region.
After the sea surface is divided, the sea surface and the non-sea surface are distinguished, the non-sea surface outside the sea surface is a land part, the non-sea surface inside the sea surface is primarily considered as a ship, and the non-sea surface part inside the sea surface is selected as a candidate area. And because the whole sea surface is communicated, a candidate region can be extracted by a region communication method according to the connectivity of the sea surface, the candidate region is preferably represented by a square frame, and then a candidate region image is extracted from the image to be detected according to the candidate region and used for subsequent classification.
As shown in fig. 4 and 5, the two graphs of fig. 4 are candidate regions located on the sea surface segmentation result graph, and the series of graphs of fig. 5 are individual candidate region images extracted from the image to be detected (original image).
And the classification module 30 is configured to input the candidate area image into the fine-tuning VGG network, and determine whether the candidate area image is a ship.
From the foregoing, the candidate region is a non-sea surface portion within the sea surface, but the portion may be a ship, a small reef, and even a result caused by a sea surface segmentation error, and therefore the candidate region needs to be classified, and when the classification is performed, the classification is performed based on the original image, and therefore the candidate region image needs to be extracted from the original image according to the candidate region, and whether the candidate region image is a ship or not is determined.
The embodiment of the invention adopts the VGG network for classification, and the VGG16 network needs to be subjected to fine-tuning so as to be adapted to the method of the invention. After the candidate area image is input into the find-tuning VGG network, the find-tuning VGG network divides the candidate area image into two types, one type is a ship, the other type is a non-ship, the candidate area is reserved as a ship, and the candidate area is marked on the original image, so as to obtain a final ship detection result, as shown in fig. 6, the detected ship is in the box.
The invention relates to a device for detecting ships on the sea surface, which can be used for segmenting the sea surface and the ships. The method uses the U-net segmentation network to simply and effectively segment the sea surface, applies the U-net segmentation algorithm to the detection task of the optical remote sensing image, does not need complex image processing operation and manual feature extraction, has high segmentation speed and high segmentation accuracy, and can effectively reduce the false alarm rate; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
In the embodiment of the invention, the U-net segmentation network is obtained by training the following modules:
and the marking module is used for marking each pixel point of the training images in the training image set.
The labeling can be carried out through a LabLeImg labeling tool, and the labeling rule is as follows: when the pixel belongs to the sea, the label is 1, and when the pixel does not belong to the sea, the label is 0.
And the overlap cropping module is used for performing overlap cropping on the marked training image.
During overlapping cropping, certain overlapping between horizontal and vertical is guaranteed, overlapping step length is preferably 40 pixel points, and the size of the cropped images is uniform, preferably 160 × 160.
And the training module is used for inputting the overlapped and cut images into the U-net segmentation network and training by adopting a random gradient descent method.
The parameters of the exercise process are as follows: the blocksize is 16, namely 16 images are input into the U-net segmentation network each time, the iteration time is 10 ten thousand times, the gradient updating method is a small-batch random gradient descent algorithm (SGD), and finally the U-net segmentation network model for sea surface segmentation is obtained.
After the U-net segmentation network is trained, inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph, wherein the sea surface segmentation module specifically comprises:
and the overlap cropping unit is used for performing overlap cropping on the image to be detected to obtain a sub-image.
The method of the overlap cropping is the same as that of the overlap cropping module, and can be understood by referring to the overlap cropping module, which is not described herein again.
And the sea surface segmentation unit is used for inputting the subimages into the trained U-net segmentation network to perform sea surface segmentation.
Preprocessing neutron images in the sea surface segmentation unit before the neutron images are input into the U-net segmentation network, wherein the preprocessing comprises mean value reduction and normalization processing: the mean value 128 is subtracted from each channel of the sub-image and multiplied by a scaling factor of 0.0078(1/128) to normalize the pixel values to [ -1,1 ].
And then inputting the subimage after the average value is reduced into a trained U-net segmentation network, wherein the U-net segmentation network can obtain sea surface probability and non-sea surface probability of pixel points of the subimage one by one, when the sea surface probability obtained by the pixel points is greater than the non-sea surface probability, the pixel points are considered as sea surfaces, when the sea surface probability obtained by the pixel points is smaller than the non-sea surface probability, the pixel points are considered as non-sea surfaces, preferably, the value of the pixel points of the sea surfaces can be set to be 1, and the value of the pixel points of the non-sea surfaces can be set to be 0.
And the splicing unit is used for splicing the subimages after the sea surface segmentation, and a big image with the same size as the original image is a sea surface segmentation result image.
As an improvement of the present invention, the candidate region module includes:
and the binarization unit is used for carrying out binarization processing on the sea surface segmentation result graph.
The pixel value of the pixel point segmented into the sea surface is preferably 0, and the pixel value of the pixel point segmented into the non-sea surface is preferably 1; that is, the value of 1 at the pixel point in the sea surface segmentation result map is set to 0, and the value of 0 at the pixel point is set to 1.
And the morphological closing operation unit is used for performing the morphological closing operation on the binarized image.
The morphological closing operation is: the expansion operation is a process of expanding the boundary to the outside and filling up the cavity in the object, and the erosion operation is a process of contracting the boundary to the inside and eliminating small and meaningless objects. The embodiment of the invention can fill small holes by using morphological closing operation, so that some pixel points which are wrongly divided into sea surfaces are corrected.
In the present invention, the dilation operation is to scan each pixel of the image with a structuring element of 3 × 3, and the and operation is performed with the structuring element and the binary image it covers, if both are 0, then that pixel of the image is 0, otherwise 1, resulting in a one-round expansion of the binary image, the erosion operation is to scan each pixel of the image with a structuring element of 3 × 3, and the and operation is performed with the structuring element and the binary image it covers, if both are 1, resulting in that pixel of the image is 1, otherwise 0, resulting in a one-round reduction of the binary image.
And the connected region acquisition unit is used for finding out all connected regions on the sea surface according to the connectivity of the sea surface and acquiring the minimum external rectangle of the connected regions to obtain the object on the sea surface.
In this step, the connected region may be an eight-connected region or a four-connected region. The eight-connected region is a combination of movement in eight directions, namely, from one pixel in the region, and other pixel points in the region can be reached through the combination of movement in the eight directions, namely, up, down, left, right, up-left, up-right, down-left and down-right. The four-connected region is that starting from one pixel in the region, other pixel points in the region can be reached through the combination of movement in the upper direction, the lower direction, the left direction and the right direction.
And the candidate region determining unit is used for taking the minimum bounding rectangle with the area not smaller than a set threshold as the candidate region.
The eliminating unit performs primary screening according to the pixel point area of an object in an image, and the calculation method comprises the following steps: the length and the width of the rectangle are calculated according to the coordinates of the upper left corner and the lower right corner of the circumscribed rectangle, then the area of the rectangle is calculated, the minimum circumscribed rectangle with the area not smaller than a preset value (for example, the preset value is 20) is used as a candidate area, and the false detection object with extremely small area is reduced.
And the candidate region image extraction unit is used for extracting a candidate region image from the image to be detected according to the candidate region and using the candidate region image as a subsequent classification basis.
The embodiment of the invention obtains more accurate ship candidate areas by using morphological closing operation and an area communication method, and accurately identifies the ships.
In the invention, the VGG network after the fine-tuning is obtained by the following method:
the VGG16 network of the present invention was first trained using images of the ImageNet database, and the structure of the VGG16 network was as described in example 1. Then, the embodiment of the present invention performs fine-tuning (fine-tuning) on the trained VGG16 network through the following modules:
and the training sample set module is used for establishing a training sample set.
Wherein, the positive sample in the training sample set is shown in fig. 8, the positive sample can be preferably an image of a ship containing more than two thirds of the ship hull, the negative sample in the training sample set is shown in fig. 9, and the negative sample can be preferably an image containing less than two thirds of the ship hull or no ship.
And the modification module is used for changing the number of output channels of the final full-connection layer fc8 of the VGG16 network by 2, performing two-classification (even if the ship is not the ship), and keeping other layers unchanged.
And the fine adjustment module is used for performing fine-tuning on the changed VGG16 network by using the training sample set to obtain the fine-tuned VGG network.
After the VGG network is finely adjusted, the candidate area image is adjusted to the required size (224 × 224) of the VGG16 network, and then the VGG network after the fine-tuning is input to judge whether the VGG network is a ship or not.
According to the device for detecting the ship on the sea surface, provided by the embodiment of the invention, the U-net segmentation network is used for simply and effectively segmenting the sea surface, complex image processing operation and manual feature extraction are not needed, the segmentation speed is high, the segmentation accuracy is high, and the false alarm rate can be effectively reduced; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments. It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the apparatus and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Example 3:
the method or apparatus provided by the present specification and described in the foregoing embodiments may implement service logic through a computer program and record the service logic on a storage medium, where the storage medium may be read and executed by a computer, so as to implement the effect of the solution described in the embodiments of the present specification. Accordingly, the present invention also provides a computer readable storage medium for remote sensing image sea surface ship detection, comprising a processor and a memory for storing processor executable instructions, which when executed by the processor, implement the steps comprising the remote sensing image sea surface ship detection method of embodiment 1.
The invention relates to a computer readable storage medium for detecting ships on the sea surface, which can be used for realizing the detection method of the ships on the sea surface and can be used for segmenting the sea surface and ships. The method uses the U-net segmentation network to simply and effectively segment the sea surface, applies the U-net segmentation algorithm to the detection task of the optical remote sensing image, does not need complex image processing operation and manual feature extraction, has high segmentation speed and high segmentation accuracy, and can effectively reduce the false alarm rate; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
The storage medium may include a physical device for storing information, and typically, the information is digitized and then stored using an electrical, magnetic, or optical media. The storage medium may include: devices that store information using electrical energy, such as various types of memory, e.g., RAM, ROM, etc.; devices that store information using magnetic energy, such as hard disks, floppy disks, tapes, core memories, bubble memories, and usb disks; devices that store information optically, such as CDs or DVDs. Of course, there are other ways of storing media that can be read, such as quantum memory, graphene memory, and so forth.
The above description of the apparatus according to the method embodiment may also include other embodiments. The specific implementation manner may refer to the description of the related method embodiment, and is not described in detail herein.
Example 4:
the invention also provides equipment for remotely sensing the image and detecting the sea surface ships, which can be a single computer, and can also comprise an actual operation device and the like using one or more methods or one or more embodiment devices in the specification. The device for remotely sensed image sea surface ship detection may comprise at least one processor and a memory storing computer executable instructions, wherein the processor executes the instructions to implement the steps of the remotely sensed image sea surface ship detection method according to any one or more of embodiments 1.
The invention relates to a device for detecting a remote sensing image sea surface ship, which can segment the sea surface and the ship by a stored instruction. The method uses the U-net segmentation network to simply and effectively segment the sea surface, applies the U-net segmentation algorithm to the detection task of the optical remote sensing image, does not need complex image processing operation and manual feature extraction, has high segmentation speed and high segmentation accuracy, and can effectively reduce the false alarm rate; meanwhile, according to the connectivity of the sea surface, a candidate area can be directly obtained through a connected area method, and then the image of the candidate area is judged by using a fine-tuned VGG network without complex calculation of a constant false alarm method. Compared with the existing sea and land segmentation method and ship determination method, the method can effectively reduce the false alarm rate, reduce the complexity of sea surface ship detection, has high operation efficiency and improves the accuracy of detection results.
The above description of the device according to the method or apparatus embodiment may also include other embodiments, and specific implementation may refer to the description of the related method embodiment, which is not described herein in detail.
It should be noted that, the above-mentioned apparatus or system in this specification may also include other implementation manners according to the description of the related method embodiment, and a specific implementation manner may refer to the description of the method embodiment, which is not described herein in detail. The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class, storage medium + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant points, refer to the partial description of the method embodiment.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, when implementing one or more of the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, etc. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method or apparatus that comprises the element.
As will be appreciated by one skilled in the art, one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
One or more embodiments of the present description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description of the specification, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A remote sensing image sea surface ship detection method is characterized by comprising the following steps:
inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph;
according to the connectivity of the sea surface, a candidate region is positioned on the sea surface segmentation result graph, and a candidate region image is extracted from the image to be detected according to the positioned candidate region;
and inputting the candidate area image into a VGG network after fine-tuning, and judging whether the candidate area image is a ship or not.
2. The remote sensing image sea surface ship detection method according to claim 1, wherein the U-net segmentation network is trained by the following method:
labeling each pixel point of the training images in the training image set;
performing overlapping clipping on the marked training images;
and inputting the overlapped and cut images into a U-net segmentation network, and training by adopting a random gradient descent method.
3. The remote sensing image sea surface ship detection method of claim 2, wherein the inputting of the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result map comprises:
performing overlapping clipping on an image to be detected to obtain a sub-image;
inputting the subimages into the trained U-net segmentation network, and performing sea surface segmentation;
and splicing the subimages obtained after the sea surface segmentation to obtain a sea surface segmentation result graph.
4. The remote sensing image sea surface ship detection method of claim 3, wherein the step of locating a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface and extracting a candidate region image from the image to be detected according to the located candidate region comprises:
carrying out binarization processing on the sea surface segmentation result graph;
performing morphological closing operation on the binarized image;
finding out all connected areas on the sea surface according to the connectivity of the sea surface, and obtaining the minimum circumscribed rectangle of the connected areas;
taking the minimum bounding rectangle with the area not smaller than a set threshold as the candidate region;
and extracting a candidate region image from the image to be detected according to the candidate region.
5. The remote sensing image sea surface ship detection method according to any one of claims 1-4, wherein the fine-tuning VGG network is obtained by the following method:
establishing a training sample set;
changing the number of output channels of the last full-connection layer of the VGG16 network by 2, and keeping other layers unchanged;
and performing fine-tuning on the changed VGG16 network by using the training sample set to obtain a fine-tuning VGG network.
6. A remote sensing image sea surface ship detection device, the device comprising:
the sea surface segmentation module is used for inputting the image to be detected into the trained U-net segmentation network to obtain a sea surface segmentation result graph;
the candidate region module is used for positioning a candidate region on the sea surface segmentation result graph according to the connectivity of the sea surface and extracting a candidate region image from the image to be detected according to the positioned candidate region;
and the classification module is used for inputting the candidate area image into the VGG network after the fine-tuning and judging whether the candidate area image is a ship or not.
7. The remote sensing image sea surface ship detection device of claim 6, wherein the U-net segmentation network is trained by:
the marking module is used for marking each pixel point of the training images in the training image set;
the overlap cropping module is used for performing overlap cropping on the marked training image;
and the training module is used for inputting the overlapped and cut images into the U-net segmentation network and training by adopting a random gradient descent method.
8. The telemetric image sea surface vessel detection apparatus of claim 6 or 7, wherein the candidate region module comprises:
the binarization unit is used for carrying out binarization processing on the sea surface segmentation result graph;
a morphology closing operation unit, which is used for performing morphology closing operation on the binarized image;
the device comprises a connected region acquisition unit, a communication unit and a communication unit, wherein the connected region acquisition unit is used for finding out all connected regions on the sea surface according to the connectivity of the sea surface and acquiring the minimum circumscribed rectangle of the connected regions;
a candidate region determination unit configured to determine, as the candidate region, a minimum bounding rectangle having an area not smaller than a set threshold;
and the candidate region image extraction unit is used for extracting a candidate region image from the image to be detected according to the candidate region.
9. A computer readable storage medium for remote sensing image sea surface ship detection, comprising a processor and a memory for storing processor executable instructions, which when executed by the processor, implement steps comprising the remote sensing image sea surface ship detection method of any one of claims 1-5.
10. An apparatus for telemetric image sea surface ship detection, comprising at least one processor and a memory storing computer executable instructions, the processor implementing the steps of the telemetric image sea surface ship detection method according to any one of claims 1 to 5 when executing the instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811599968.5A CN111368599B (en) | 2018-12-26 | 2018-12-26 | Remote sensing image sea surface ship detection method and device, readable storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811599968.5A CN111368599B (en) | 2018-12-26 | 2018-12-26 | Remote sensing image sea surface ship detection method and device, readable storage medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111368599A true CN111368599A (en) | 2020-07-03 |
| CN111368599B CN111368599B (en) | 2023-11-07 |
Family
ID=71206025
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811599968.5A Active CN111368599B (en) | 2018-12-26 | 2018-12-26 | Remote sensing image sea surface ship detection method and device, readable storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111368599B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931688A (en) * | 2020-08-27 | 2020-11-13 | 珠海大横琴科技发展有限公司 | Ship recognition method and device, computer equipment and storage medium |
| CN112945014A (en) * | 2021-03-05 | 2021-06-11 | 山东中泰新能源集团有限公司 | Method and facility for protecting safety of territory |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104077777A (en) * | 2014-07-04 | 2014-10-01 | 中国科学院大学 | Sea surface vessel target detection method |
| WO2017071160A1 (en) * | 2015-10-28 | 2017-05-04 | 深圳大学 | Sea-land segmentation method and system for large-size remote-sensing image |
| CN107563303A (en) * | 2017-08-09 | 2018-01-09 | 中国科学院大学 | A kind of robustness Ship Target Detection method based on deep learning |
| CN107992818A (en) * | 2017-11-29 | 2018-05-04 | 长光卫星技术有限公司 | A kind of detection method of remote sensing image sea ship target |
| CN108090911A (en) * | 2018-01-08 | 2018-05-29 | 北京航空航天大学 | A kind of offshore naval vessel dividing method of remote sensing image |
| CN108734111A (en) * | 2018-04-26 | 2018-11-02 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | SAR image surface vessel recognition methods |
-
2018
- 2018-12-26 CN CN201811599968.5A patent/CN111368599B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104077777A (en) * | 2014-07-04 | 2014-10-01 | 中国科学院大学 | Sea surface vessel target detection method |
| WO2017071160A1 (en) * | 2015-10-28 | 2017-05-04 | 深圳大学 | Sea-land segmentation method and system for large-size remote-sensing image |
| CN107563303A (en) * | 2017-08-09 | 2018-01-09 | 中国科学院大学 | A kind of robustness Ship Target Detection method based on deep learning |
| CN107992818A (en) * | 2017-11-29 | 2018-05-04 | 长光卫星技术有限公司 | A kind of detection method of remote sensing image sea ship target |
| CN108090911A (en) * | 2018-01-08 | 2018-05-29 | 北京航空航天大学 | A kind of offshore naval vessel dividing method of remote sensing image |
| CN108734111A (en) * | 2018-04-26 | 2018-11-02 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | SAR image surface vessel recognition methods |
Non-Patent Citations (2)
| Title |
|---|
| 余东行;张保明;郭海涛;赵传;徐俊峰;: "联合显著性特征与卷积神经网络的遥感影像舰船检测", 中国图象图形学报, no. 12 * |
| 龚志成;曾惠翼;裴继红;: "基于邻域分析的海洋遥感图像舰船检测方法", 深圳大学学报(理工版), no. 06 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931688A (en) * | 2020-08-27 | 2020-11-13 | 珠海大横琴科技发展有限公司 | Ship recognition method and device, computer equipment and storage medium |
| CN112945014A (en) * | 2021-03-05 | 2021-06-11 | 山东中泰新能源集团有限公司 | Method and facility for protecting safety of territory |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111368599B (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112232349B (en) | Model training method, image segmentation method and device | |
| CN110310264B (en) | DCNN-based large-scale target detection method and device | |
| CN109711295B (en) | Optical remote sensing image offshore ship detection method | |
| Tong et al. | Saliency detection with multi-scale superpixels | |
| Yang et al. | Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and Kalman filter in time-lapse microscopy | |
| CN110838126B (en) | Cell image segmentation method, cell image segmentation device, computer equipment and storage medium | |
| CN110796048B (en) | Ship target real-time detection method based on deep neural network | |
| KR101896357B1 (en) | Method, device and program for detecting an object | |
| WO2021051604A1 (en) | Method for identifying text region of osd, and device and storage medium | |
| CN111079739B (en) | Multi-scale attention feature detection method | |
| CN110765865B (en) | Underwater target detection method based on improved YOLO algorithm | |
| CN111753682B (en) | Hoisting area dynamic monitoring method based on target detection algorithm | |
| CN111091095A (en) | Method for detecting ship target in remote sensing image | |
| CN111310622A (en) | Fish swarm target identification method for intelligent operation of underwater robot | |
| CN106845347A (en) | Marine oil film recognizes extracting method and system | |
| CN112818952A (en) | Coal rock boundary recognition method and device and electronic equipment | |
| CN111368599B (en) | Remote sensing image sea surface ship detection method and device, readable storage medium and equipment | |
| Zhao et al. | Research on detection method for the leakage of underwater pipeline by YOLOv3 | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN112825116B (en) | Method, device, medium and equipment for detecting and tracking human face of monitoring video image | |
| CN116797941A (en) | Marine oil spill risk source rapid intelligent identification and classification method for high-resolution remote sensing image | |
| CN112330659B (en) | Geometric tolerance symbol segmentation method combining LSD (least squares) linear detection and connected domain marking method | |
| CN116385915A (en) | Water surface floater target detection and tracking method based on space-time information fusion | |
| Zhu et al. | RHA-Net: An Encoder-Decoder Network with Residual Blocks and Hybrid Attention Mechanisms for Pavement Crack Segmentation | |
| CN111815629B (en) | Lining detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |