CN114627372A - Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning - Google Patents
Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning Download PDFInfo
- Publication number
- CN114627372A CN114627372A CN202210177141.5A CN202210177141A CN114627372A CN 114627372 A CN114627372 A CN 114627372A CN 202210177141 A CN202210177141 A CN 202210177141A CN 114627372 A CN114627372 A CN 114627372A
- Authority
- CN
- China
- Prior art keywords
- network
- image
- remote sensing
- sensing image
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method for quickly detecting a wide remote sensing image ship target based on intra-domain transfer learning, which belongs to the field of satellite image recognition, and comprises the steps of firstly, establishing a multi-spectral classification network model and pre-training a feature extraction network; secondly, establishing a single-stage rotating target detection model, finely adjusting a feature extraction network, and updating the weight of the target detection network; thirdly, performing thin cloud removal training on the remote sensing image by adopting a thin cloud removal network based on a style migration algorithm; then, in a prediction stage, sea and land segmentation is carried out on the remote sensing image by using a global 30-meter ground surface coverage fine classification product; and finally, performing light weight processing on the target detection network by combining sea surface and coastline masks and regional inactivation modes, and improving the detection speed. Compared with the conventional detection method, the method is improved at multiple angles of feature extractor training, thin cloud removal, model lightweight and the like, and wide remote sensing image ship target detection with higher precision and higher speed is realized.
Description
Technical Field
The invention belongs to the field of satellite image recognition, and particularly relates to a method for quickly detecting a wide remote sensing image ship target based on intra-domain transfer learning.
Background
With the rapid development of remote sensing technology, the time resolution, the spatial resolution and the spectral resolution of a satellite are greatly improved, so that the remote sensing data volume is increased explosively, and researchers at home and abroad make a great deal of research for extracting effective information from remote sensing big data, wherein the remote sensing image ship target detection is an important research direction. In recent years, deep learning is successfully applied to the field of computer vision, a new method is brought to remote sensing image processing, most of the existing remote sensing image target detection methods are based on a transfer learning method, namely, an image data set under a large-scale natural scene is adopted to pre-train a feature extractor, so that a convolutional neural network has the capability of extracting features such as edges, corners, textures and semantics, and then feature engineering design and special loss function design are carried out on the basis to achieve the capability of remote sensing target detection. However, the transfer learning is cross-domain, and is a feature extractor for training a remote sensing image domain by adopting images of a natural image domain, and two problems exist, one is that the natural image is stored by 8 bits, the remote sensing image is stored by 16 bits, and the 16-bit depth image contains much more information than the 8-bit depth image; secondly, the difference between the target mode of the natural image and the target mode of the remote sensing image is large, and the characteristic representation range is insufficient by adopting a characteristic extractor pre-trained by the natural image. In addition, the deep learning model has a large calculation amount and a low detection speed.
Disclosure of Invention
The invention aims to provide a method for quickly detecting a wide remote sensing image ship target based on intra-domain transfer learning, aiming at the problems of the defects of cross-domain transfer learning and low operation efficiency of a deep learning model.
The technical scheme adopted by the invention is as follows:
a method for quickly detecting a wide remote sensing image ship target based on intra-domain transfer learning comprises the following steps:
step 1: creating a multi-spectral-segment classification network model, and carrying out scene classification training on the multi-spectral-segment classification network model by using an original remote sensing image to obtain a pre-trained feature extraction network;
step 2: constructing a target detection network, taking the output characteristics of the pre-trained characteristic extraction network as the input of the target detection network, thereby establishing a single-stage rotating target detection model, and carrying out target detection training on the single-stage rotating target detection model by using a ship target detection data set to obtain a trained characteristic extraction network and a trained target detection network;
and step 3: constructing a thin cloud removal network based on a style migration algorithm, taking a non-cloud remote sensing image, a cloud-containing remote sensing image and a noise image as input of a trained feature extraction network, taking output features of the feature extraction network as input of the thin cloud removal network, and training a thin cloud removal network model to obtain the trained thin cloud removal network;
and 4, step 4: in the prediction stage, sea-land segmentation is carried out on the remote sensing image by utilizing a ground surface covering fine classification product to obtain a sea surface and a coastline mask;
and 5: cascading the trained feature extraction network, the thin cloud removal network and the target detection network, inputting the sea surface and coastline masks and the remote sensing image to be detected into the feature extraction network together, performing light weight processing on the feature extraction network by using a region inactivation mode, and performing feature extraction, thin cloud removal and target detection on ships near the coastline and on the sea surface.
Further, step 1 specifically comprises:
establishing a multi-spectrum segment classification network model, wherein except an input layer and an output layer, the multi-spectrum segment classification network model adopts a Darknet53 network structure; the input layer adopts 4 channels, and 16-bit original remote sensing images are input; and the output layer adopts 45-dimensional cross entropy loss, 45 represents the category number of the scene classification, and carries out smooth regularization processing on the scene labels to obtain a pre-trained feature extraction network.
Further, in step 2, the target detection network adopts a YOLOv4 detection algorithm, and a detection box description mode of the YOLOv4 detection algorithm is specifically as follows: the description of the detection box is performed using 4 vertices, in the form of 8 coordinate parameters: { x1,y1,x2,y2,x3,y3,x4,y4}; meanwhile, an arbitrary quadrilateral IOU calculation mode is adopted to replace a rectangular IOU calculation mode.
Further, step 3 specifically comprises:
constructing a thin cloud removal network based on a style migration algorithm, taking a cloud-containing remote sensing image as a style image a, taking a non-cloud remote sensing image as a content image p, taking the style image a, the content image p and a random noise image x as the input of a trained feature extraction network, and respectively extracting multi-scale style features of real ground features of the style image and fuzzy texture features A of cloud and foglSharpening texture feature P of content imagelAnd style characteristics G of random noise imagelAnd sharpening the texture feature FlAnd l represents a feature layer number; and establishing a loss function of the style characteristics between the random noise image and the style image
And a loss function of sharpening texture features between the random noise image and the content image
Training a thin cloud removal network by utilizing the output of the feature extraction network, and enabling a random noise image to have style features of a style image and sharpened texture features of a content image through iterative training, wherein the obtained random noise image is an image obtained by cloud-containing remote sensing image cloud removal; wherein, wlAnd vlThe weighting coefficients of the l-th layer of the loss function are represented.
Further, in step 5, the feature extraction network is subjected to lightweight processing by using a region deactivation mode, specifically:
and simultaneously taking the remote sensing image to be detected and the sea surface and coastline masks as the input of the feature extraction network, and performing regional inactivation treatment on the convolution layer and the pooling layer corresponding to the land part in the sea and land segmentation masks, thereby reducing the calculated amount of the feature extraction network.
Compared with the prior art, the invention has the advantages that:
(1) in the training stage of the feature extraction network, a domain transfer learning-based method is adopted, so that the features extracted by the feature extraction network are richer and are more suitable for remote sensing image target detection.
(2) In the thin cloud removing stage, the style migration network is adopted to remove the thin cloud, a cloud-free image pair does not need to be created, and the labeling cost is saved.
(3) In the prediction stage, the model is lightened by adopting a region inactivation mode, so that the detection precision is not lost, the false alarm can be reduced, and the detection speed can be increased.
Drawings
Fig. 1 is a general technical flowchart of the proposed method of the present invention.
FIG. 2 is a schematic diagram of a thin cloud removal algorithm used in the present invention.
Fig. 3(a) is a remote sensing image to be detected, fig. 3(b) is a 30-meter-surface-coverage fine classification product corresponding to the remote sensing image to be detected, fig. 3(c) is a result of registering the image to be detected and the 30-meter-surface-coverage fine classification product, fig. 3(d) is a schematic diagram of performing a region inactivation operation according to a land and sea segmentation mask, and fig. 3(e) is a schematic diagram of performing the region inactivation operation according to the land and sea segmentation mask and an effective range of the image to be detected.
Detailed Description
The invention provides a method for rapidly detecting a wide remote sensing image ship target based on intra-domain transfer learning, which is designed aiming at the problems of the defects of the cross-domain transfer learning and the low operation efficiency of a deep learning model, and the specific flow chart is shown in figure 1.
The following provides a further description of the embodiments and the basic principles of the present invention with reference to the accompanying drawings.
Step 1: creating a multi-spectral-segment classification network model, wherein the multi-spectral-segment classification network adopts a Darknet53 network structure except for an input layer and an output layer; the input layer adopts 4 channels and 16-bit original remote sensing images, so that information loss caused by channel rejection and image stretching is avoided; the output layer adopts 45-dimensional cross entropy loss, 45 represents the category number of scene classification, and carries out smooth regularization processing on scene labels, thereby reducing the difficulty of network learning and accelerating the convergence speed. Then, carrying out scene classification training on the multi-spectral classification network model by using the 16-bit original remote sensing image to obtain a pre-trained feature extraction network, so that the pre-trained feature extraction network has the remote sensing image feature extraction capability. The pre-trained feature extraction network will be used in subsequent steps, as shown in fig. 1 in particular;
and 2, step: constructing a target detection network, and taking the output characteristics of the pre-trained characteristic extraction network as the input of the target detection network, thereby establishing a single-stage rotating target detection model; the target detection network adopts an improved YOLOv4 detection algorithm, improves the description mode of the detection box in the target positioning stage, and uses the form of 4 vertexes and 8 coordinate parameters to describe the detection box: { x1,y1,x2,y2,x3,y3,x4,y4}. Meanwhile, an arbitrary quadrilateral IOU calculation mode is adopted to replace a rectangular IOU calculation mode. Then, using the ship targetThe detection data set carries out target detection training on a single-stage rotating target detection model, fine adjustment is carried out on a feature extractor, and weight updating is carried out on a target detection network, the feature extraction network after the fine adjustment training in the stage is still used in the subsequent steps, and the target detection network trained in the stage is directly used in a model prediction stage, as shown in figure 1;
step 3, constructing a thin cloud removal network based on a style migration algorithm, taking a non-cloud remote sensing image, a cloud-containing remote sensing image and a noise image as input of the trained feature extraction network, taking output features of the feature extraction network as input of the thin cloud removal network, and training a thin cloud removal network model to obtain the trained thin cloud removal network; as shown in fig. 2, a thin cloud removal network based on a style migration algorithm is constructed, a cloud-containing remote sensing image is used as a style image a, a non-cloud remote sensing image is used as a content image p, the style image a, the content image p and a random noise image x are used as input of a trained feature extraction network, and multi-scale style features of real ground features of the style image and fuzzy texture features a of cloud and mist are respectively extractedlSharpening texture feature P of content imagelAnd style characteristics G of random noise imagelAnd sharpening the texture feature FlL represents a feature layer number; and establishing a loss function of the style characteristics between the random noise image and the style image
And a loss function of sharpening texture features between the random noise image and the content image
Training a thin cloud removal network by utilizing the output of the feature extraction network, and enabling a random noise image to have style features of a style image and sharpened texture features of a content image through iterative training, wherein the obtained random noise image is an image obtained by cloud-containing remote sensing image cloud removal; wherein, wlAnd vlThe weighting coefficients of the l-th layer of the loss function are represented.
And 4, step 4: in the prediction stage, sea and land segmentation is carried out on the remote sensing image by using a global 30-meter ground surface covering fine classification product GLC _ FCS30 to obtain a sea surface and coastline mask, the remote sensing image to be detected is shown as a figure 3(a), the 30-meter ground surface covering fine classification product corresponding to the remote sensing image to be detected is shown as a figure 3(b), and a figure 3(c) is a result of registration of the image to be detected and the 30-meter ground surface covering fine classification product;
and 5: cascading the trained feature extraction network, the thin cloud removal network and the target detection network, inputting the sea surface and coastline masks and the remote sensing image to be detected into the feature extraction network together, performing light weight processing on the feature extraction network by using a region inactivation mode, and performing feature extraction, thin cloud removal and target detection on ships near the coastline and on the sea surface. As shown in fig. 1, the remote sensing image to be detected and the sea-land segmentation mask are simultaneously used as input; as shown in fig. 3, the region deactivation strategy firstly performs a region deactivation operation according to the land-sea division mask, and as shown in fig. 3(d), performs a deactivation process on convolution and pooling operations corresponding to the land mask so as not to participate in the calculation; secondly, performing region inactivation operation according to the land and sea segmentation mask and the effective range of the image to be detected, and performing inactivation processing on the invalid region in the input image as shown in fig. 3(e), thereby reducing the calculation amount, improving the prediction speed and reducing the false alarm.
The invention designs a wide remote sensing image ship target rapid detection method based on intra-domain transfer learning, aiming at the problems of the defects of cross-domain transfer learning and low operation efficiency of a deep learning model. The domain transfer learning-based method provided by the invention can enrich the characteristics extracted by the characteristic extraction network. Meanwhile, the thin cloud is removed by adopting the style migration network, a cloud-free image pair does not need to be created, and the labeling cost is saved. In addition, the invention adopts a region inactivation mode to carry out model lightweight processing, thereby not only not losing detection precision, but also reducing false alarm and simultaneously improving detection speed.
Claims (5)
1. A method for quickly detecting a wide remote sensing image ship target based on intra-domain transfer learning is characterized by comprising the following steps:
step 1: creating a multi-spectral-segment classification network model, and carrying out scene classification training on the multi-spectral-segment classification network model by using an original remote sensing image to obtain a pre-trained feature extraction network;
step 2: constructing a target detection network, taking the output characteristics of the pre-trained characteristic extraction network as the input of the target detection network, thereby establishing a single-stage rotating target detection model, and carrying out target detection training on the single-stage rotating target detection model by using a ship target detection data set to obtain a trained characteristic extraction network and a trained target detection network;
and step 3: constructing a thin cloud removal network based on a style migration algorithm, taking a non-cloud remote sensing image, a cloud-containing remote sensing image and a noise image as input of a trained feature extraction network, taking output features of the feature extraction network as input of the thin cloud removal network, and training a thin cloud removal network model to obtain the trained thin cloud removal network;
and 4, step 4: in the prediction stage, sea-land segmentation is carried out on the remote sensing image by utilizing a ground surface covering fine classification product to obtain a sea surface and a coastline mask;
and 5: cascading the trained feature extraction network, the thin cloud removal network and the target detection network, inputting the sea surface and coastline masks and the remote sensing image to be detected into the feature extraction network together, performing light weight processing on the feature extraction network by using a region inactivation mode, and performing feature extraction, thin cloud removal and target detection on ships near the coastline and on the sea surface.
2. The method for rapidly detecting the target of the wide remote sensing image ship based on the intra-domain transfer learning according to claim 1, wherein the step 1 specifically comprises the following steps:
establishing a multi-spectrum segment classification network model, wherein except an input layer and an output layer, the multi-spectrum segment classification network model adopts a Darknet53 network structure; the input layer adopts 4 channels, and 16-bit original remote sensing images are input; and the output layer adopts 45-dimensional cross entropy loss, 45 represents the category number of the scene classification, and carries out smooth regularization processing on the scene labels to obtain a pre-trained feature extraction network.
3. The method for rapidly detecting the target of the wide remote sensing image ship based on the intra-domain transfer learning as claimed in claim 1, wherein in the step 2, the target detection network adopts a YOLOv4 detection algorithm, and a detection frame description mode of the YOLOv4 detection algorithm is specifically as follows: the description of the detection box is performed using 4 vertices, in the form of 8 coordinate parameters: { x)1,y1,x2,y2,x3,y3,x4,y4}; meanwhile, an arbitrary quadrilateral IOU calculation mode is adopted to replace a rectangular IOU calculation mode.
4. The method for rapidly detecting the wide remote sensing image ship target based on the intra-domain transfer learning according to claim 1, wherein the step 3 specifically comprises the following steps:
constructing a thin cloud removal network based on a style migration algorithm, taking a cloud-containing remote sensing image as a style image a, taking a non-cloud remote sensing image as a content image p, taking the style image a, the content image p and a random noise image x as the input of a trained feature extraction network, and respectively extracting multi-scale style features of real ground features of the style image and fuzzy texture features A of cloud and foglSharpening texture feature P of content imagelAnd style characteristics G of random noise imagelAnd sharpening the texture feature FlAnd l represents a feature layer number; and establishing a loss function of the style characteristics between the random noise image and the style image
And a loss function of sharpening texture features between the random noise image and the content image
Training a thin cloud removal network by utilizing the output of the feature extraction network, enabling the random noise image to have the style features of the style image and the sharpened texture features of the content image through iterative training, and obtaining the random noise at the momentThe acoustic image is an image obtained by cloud-removing the cloud-containing remote sensing image; wherein, wlAnd vlThe weighting coefficients of the l-th layer of the loss function are represented.
5. The method for rapidly detecting the target of the wide remote sensing image ship based on the intra-domain transfer learning according to claim 1, wherein in the step 5, a light weight processing is performed on the feature extraction network by using a region inactivation mode, and specifically comprises the following steps:
and simultaneously taking the remote sensing image to be detected and the sea surface and coastline masks as the input of the feature extraction network, and performing regional inactivation treatment on the convolution layer and the pooling layer corresponding to the land part in the sea and land segmentation masks, thereby reducing the calculated amount of the feature extraction network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210177141.5A CN114627372A (en) | 2022-02-24 | 2022-02-24 | Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210177141.5A CN114627372A (en) | 2022-02-24 | 2022-02-24 | Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114627372A true CN114627372A (en) | 2022-06-14 |
Family
ID=81900803
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210177141.5A Pending CN114627372A (en) | 2022-02-24 | 2022-02-24 | Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627372A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115457396A (en) * | 2022-09-26 | 2022-12-09 | 河北省科学院地理科学研究所 | Surface target ground object detection method based on remote sensing image |
| CN116129145A (en) * | 2023-04-14 | 2023-05-16 | 广东海洋大学 | Method and system for extracting sandy coastline of high-resolution remote sensing image |
-
2022
- 2022-02-24 CN CN202210177141.5A patent/CN114627372A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115457396A (en) * | 2022-09-26 | 2022-12-09 | 河北省科学院地理科学研究所 | Surface target ground object detection method based on remote sensing image |
| CN116129145A (en) * | 2023-04-14 | 2023-05-16 | 广东海洋大学 | Method and system for extracting sandy coastline of high-resolution remote sensing image |
| CN116129145B (en) * | 2023-04-14 | 2023-06-23 | 广东海洋大学 | Method and system for extracting sandy coastline of high-resolution remote sensing image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yeh et al. | Multi-scale deep residual learning-based single image haze removal via image decomposition | |
| CN109961049B (en) | Cigarette brand identification method under complex scene | |
| CN111640125B (en) | Aerial photography graph building detection and segmentation method and device based on Mask R-CNN | |
| CN106897673B (en) | Retinex algorithm and convolutional neural network-based pedestrian re-identification method | |
| CN107145889B (en) | Target identification method based on double CNN network with RoI pooling | |
| CN109325484B (en) | Flower image classification method based on background prior significance | |
| CN114627372A (en) | Method for rapidly detecting wide remote sensing image ship target based on intra-domain transfer learning | |
| CN111899172A (en) | Vehicle target detection method oriented to remote sensing application scene | |
| CN108898145A (en) | A kind of image well-marked target detection method of combination deep learning | |
| CN110399840B (en) | Rapid lawn semantic segmentation and boundary detection method | |
| CN111310718A (en) | High-accuracy detection and comparison method for face-shielding image | |
| CN113744311A (en) | Twin neural network moving target tracking method based on full-connection attention module | |
| CN110458160A (en) | A kind of unmanned boat waterborne target recognizer based on depth-compression neural network | |
| CN114119586A (en) | Intelligent detection method for aircraft skin defects based on machine vision | |
| CN112396655A (en) | Point cloud data-based ship target 6D pose estimation method | |
| CN114612306A (en) | Deep learning super-resolution method for crack detection | |
| CN113888505B (en) | Natural scene text detection method based on semantic segmentation | |
| CN111612802B (en) | Re-optimization training method based on existing image semantic segmentation model and application | |
| CN113421210A (en) | Surface point cloud reconstruction method based on binocular stereo vision | |
| CN115909110A (en) | Lightweight infrared unmanned aerial vehicle target tracking method based on Simese network | |
| CN109064425B (en) | Self-adaptive non-local total variation image denoising method | |
| Jiao et al. | Research on convolutional neural network model for sonar image segmentation | |
| CN108256455B (en) | Road image segmentation method based on vanishing points | |
| CN112070689A (en) | Data enhancement method based on depth image | |
| CN117541969B (en) | Pornography video detection method based on semantics and image enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |