CN108549866B - Remote sensing airplane identification method based on dense convolutional neural network - Google Patents
Remote sensing airplane identification method based on dense convolutional neural network Download PDFInfo
- Publication number
- CN108549866B CN108549866B CN201810326575.0A CN201810326575A CN108549866B CN 108549866 B CN108549866 B CN 108549866B CN 201810326575 A CN201810326575 A CN 201810326575A CN 108549866 B CN108549866 B CN 108549866B
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- dense
- layer
- joint
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a joint supervision and identification method based on a dense convolutional neural network, which utilizes the advantages of feature reuse, dense connection and the like of a dense convolutional network structure to generate a highly parameterized high-efficiency model. And the loss function is improved, the original softmax loss function is changed into joint supervision of softmax loss and central loss, and through the joint supervision, the characteristic difference between different classes is enlarged, and the intra-class characteristic change of the same class is reduced. Therefore, the problem of low identification rate of shielding, noise and fuzzy conditions during airplane identification is solved.
Description
The technical field is as follows:
the invention relates to an airplane model identification technology for remotely sensing airplane images. The method is a joint supervision and identification method based on the dense convolutional neural network.
Background art:
the identification of the remote sensing airplane target has great research significance in both civil and military fields, and the accurate positioning identification of the airplane can be realized, so that the remote sensing airplane target is used for monitoring civil airplanes and accurate military striking. However, the remote sensing images have the problems of large information amount, unclear images and large influence of environmental factors, the convolutional neural network attracts wide attention as a method for solving the problem, and the method can automatically extract interesting features in the images by continuously training and adjusting weight parameters, so that accurate remote sensing airplane identification is realized.
At present, the types of airplane targets are various, the data volume is huge, and the collected images are greatly interfered by conditions such as environment, weather and the like, so that the research on the airplane identification method oriented to practical application is necessary. The traditional airplane identification algorithm mainly comprises template matching and a support vector machine. The early template matching algorithm principle is that the similarity between a template image and a target image is calculated to realize target identification, the identification accuracy is high, but the calculated amount is large, and the robustness to abnormal conditions such as shielding, blurring and rotation is poor. Later, with the development of machine learning, a support vector machine algorithm is provided, the identification accuracy is improved to a certain extent, the robustness is better than that of template matching, and however, a kernel function, a bandwidth and the like of the support vector machine are difficult to determine. In recent years, a Convolutional Neural Network (CNN) has become a leading sheep in the field of machine vision due to excellent performance, and can extract required features through multilayer convolutional networks and back propagation and then identify airplanes.
Based on the method, the invention provides a joint supervision and identification method based on the dense convolutional neural network, and a highly parameterized high-efficiency model is generated by utilizing the advantages of feature reuse, dense connection and the like of the dense convolutional network structure. The loss function uses the joint supervision of softmax loss and central loss, and through the joint supervision, the characteristic difference between different classes is enlarged, and the intra-class characteristic change of the same class is reduced. Therefore, the problem of low identification rate of shielding, noise and fuzzy conditions during airplane identification is solved.
The invention content is as follows:
the remote sensing airplane identification realized by using a deep learning method becomes an airplane identification research hotspot. Most of loss functions based on deep learning aircraft identification algorithms are softmax loss functions, and for remote sensing aircraft images affected by weather, noise, blur and other anomalies, the defects of the traditional softmax function mainly have three aspects: firstly, from the clustering perspective, the inter-class distance of the extracted features is larger than the inter-class distance in many cases, which is not beneficial to feature differentiation; the occupied area is large, and each class is expected to occupy a smaller part, because the aircraft has many classes, the model is expected to identify the classes which are not in the training data labels; third, softmax can make the model overly self-confident, and the classification result is basically not 1, i.e. 0. Therefore, in order to solve the problems, the invention adopts a joint supervision remote sensing airplane identification method based on a dense convolutional neural network, which mainly comprises the following steps:
the method comprises the following steps: training a dense convolutional neural network structure on a self-built remote sensing airplane database, wherein training samples are five types of remote sensing airplane images, and preprocessing is performed through rotation, noise addition, shielding at different degrees and motion blur of different pixels. The dense convolutional neural network structure is shown in fig. 2, a first layer convolutional layer of the network; followed by a maximum pooling layer P1; followed by 3 dense blocks D1, D2, D3 densely connected in the core part of the network, each dense block containing two convolutional layers; and transition layers T1 and T2 are used between dense blocks to reduce the output dimension; a pooling layer P2 and a full-link layer F; and finally, outputting the recognition result by using a joint supervision method.
Initializing parameter setting, setting the learning rate lr in the network training within the range of 0.009-0.02, setting the batchsize to ξ, namely, adjusting the weight once per ξ training samples, wherein the joint supervision adopted by the invention is to weight and sum the softmax loss function and the central loss function, balance the two loss functions by lambda, control the learning rate of the center by α, set the lambda within the range of 0.008-0.03, and set the α within the range of 0.2-0.5.
And thirdly, as shown in FIG. 2, firstly, a remote sensing image of 178 × 178 pixels passes through a first convolutional layer to obtain 16 characteristic maps of 89 × 89, then, the maximum pooling P1 is carried out to obtain 16 characteristic maps of 46 × 46, then, the 3 dense blocks D1, D2 and D3 are carried out to obtain 80 characteristic maps of 46 × 46, due to the dense connection characteristic, 80 is obtained from 16+16 × 4, then, the transition layer T1 containing the convolutional layer and the pooling layer is carried out to obtain 40 characteristic maps of 23 × 23, and similarly, after 14 dense blocks and one transition layer, 196 characteristic maps of 6 × 6 are obtained, and finally, after the average pooling layer, the 196 characteristic maps of 1 × 1 are obtained and sent to the full connection layer.
And step four, inputting the vector 1 × 10 output by the full connection layer into a joint supervision loss function L as formula 1, wherein Ls represents a softmax loss function, and Lc represents a central loss function.
Wherein: x is the number ofiRepresenting the ith deep feature in d-dimensional space, belonging to the yiClass, d, dimension of characteristic space, W, parameter matrix of full connection layer (W, d × n, d rows and n columns), Wj, j, size of batch processing, n, category number, b, offset, C, and CyiRepresents the y thiHeart-like with deep-like features. λ is used to balance the two loss functions. If λ is set to 0, this can be seen as a special case of such joint supervision, representing a softmax loss function only.
Step five: calculating the joint loss and comparing the joint loss with the set threshold value in the figure 3, solving the partial derivative of the joint loss to calculate the back propagation error if the joint loss is not met, and the equation is as follows
Step six, the equation of the updated weight and the depth feature class center after the error is obtained is as follows, in the training process, the updating of the depth feature class center adopts a training method for updating the deep feature class center based on a small batch of training sets, in addition, in order to avoid large disturbance caused by a small amount of error standard samples, α is used for controlling the learning rate of the center, and Lc is calculated for xiC of gradient sumyiUpdating an equation:
if the condition is satisfied, δ (condition) is 1, if the condition is not satisfied, δ (condition) is 0, α is in the range of [0,1], and then as shown in fig. 3, the weight and the centroid adjustment are circularly performed as the following equations until the calculated result of the loss function is output after the requirement is met.
Step seven: and (5) testing the network. In order to verify the effectiveness of the method, the robustness of the algorithm to three abnormal conditions of noise, fuzziness and shielding is verified in a self-built test set, and the remote sensing airplane identification result is predicted through a forward propagation algorithm.
Drawings
FIG. 1: training process diagram of joint supervision aircraft identification method based on dense convolutional neural network
FIG. 2: dense convolutional neural network structure schematic diagram
FIG. 3: combined supervision algorithm flow chart combining softmax loss function and central loss
Compared with the prior art, the invention has the advantages or positive effects that:
1. the advantages of the dense convolutional neural network: compared with the traditional convolution network, the method needs less parameters, because the redundant feature map does not need to be learnt again, the training speed is higher and the recognition accuracy is higher under the condition of the same convolution layer number. Dense convolutional networks clearly distinguish between information added to the network and information retained, which exploits the potential of the network through feature reuse, resulting in a compact model that is easy to train and highly parameterized.
2. Advantages of joint supervision: and (3) minimizing the distance between the depth feature and the depth feature class center according to a center loss formula on the basis of the softmax loss function through combined supervision, and punishing the feature far away from the depth feature class center. Effectively pulling the same class of deep features near its depth feature class center. Through the joint supervision, the characteristic difference between the five types of remote sensing airplanes is enlarged, and the intra-class characteristic change of each type of airplane is reduced, so that the discrimination capability of the dense convolutional network can be improved through the joint supervision, and the identification accuracy is improved.
The specific implementation mode is as follows:
the method comprises the following steps: training a dense convolutional neural network structure on a self-built remote sensing airplane database, wherein training samples are five types of remote sensing airplane images, and preprocessing is performed through rotation, noise addition, shielding at different degrees and motion blur of different pixels. The dense convolutional neural network structure is shown in fig. 2, a first layer convolutional layer of the network; followed by a maximum pooling layer P1; followed by 3 dense blocks D1, D2, D3 densely connected in the core part of the network, each dense block containing two convolutional layers; and transition layers T1 and T2 are used between dense blocks to reduce the output dimension; a pooling layer P2 and a full-link layer F; and finally, outputting the recognition result by using a joint supervision method.
Initializing parameter setting, setting the learning rate lr in the network training within the range of 0.009-0.02, setting the batchsize to ξ, namely, adjusting the weight once per ξ training samples, wherein the joint supervision adopted by the invention is to weight and sum the softmax loss function and the central loss function, balance the two loss functions by lambda, control the learning rate of the center by α, set the lambda within the range of 0.008-0.03, and set the α within the range of 0.2-0.5.
And thirdly, as shown in FIG. 2, firstly, a remote sensing image of 178 × 178 pixels passes through a first convolutional layer to obtain 16 characteristic maps of 89 × 89, then, the maximum pooling P1 is carried out to obtain 16 characteristic maps of 46 × 46, then, the 3 dense blocks D1, D2 and D3 are carried out to obtain 80 characteristic maps of 46 × 46, due to the dense connection characteristic, 80 is obtained from 16+16 × 4, then, the transition layer T1 containing the convolutional layer and the pooling layer is carried out to obtain 40 characteristic maps of 23 × 23, and similarly, after 14 dense blocks and one transition layer, 196 characteristic maps of 6 × 6 are obtained, and finally, after the average pooling layer, the 196 characteristic maps of 1 × 1 are obtained and sent to the full connection layer.
And step four, inputting the vector 1 × 10 output by the full connection layer into a joint supervision loss function L as formula 1, wherein Ls represents a softmax loss function, and Lc represents a central loss function.
Wherein: x is the number ofiRepresenting the ith deep feature in d-dimensional space, belonging to the yiClass, d, dimension of characteristic space, W, parameter matrix of full connection layer (W, d × n, d rows and n columns), Wj, j, size of batch processing, n, category number, b, offset, C, and CyiRepresents the y thiHeart-like with deep-like features. λ is used to balance the two loss functions. If λ is set to 0, this can be seen as a special case of such joint supervision, representing a softmax loss function only.
Step five: calculating the joint loss and comparing the joint loss with the set threshold value in the figure 3, solving the partial derivative of the joint loss to calculate the back propagation error if the joint loss is not met, and the equation is as follows
Step six: the equation of the updated weight and the depth feature class center after the error is obtained is as follows, and the deep feature class center is updated based on a small batch training set in the training processTraining method of symbolic class center, and to avoid large disturbance caused by small amount of error standard sample, we use α to control learning rate of center, calculate Lc to xiC of gradient sumyiUpdating an equation:
if the condition is satisfied, δ (condition) is 1, if the condition is not satisfied, δ (condition) is 0, α is in the range of [0,1], and then as shown in fig. 3, the weight and the centroid adjustment are circularly performed as the following equations until the calculated result of the loss function is output after the requirement is met.
Step seven: and (5) testing the network. In order to verify the effectiveness of the method, the robustness of the algorithm to three abnormal conditions of noise, fuzziness and shielding is verified in a self-built test set, and the remote sensing airplane identification result is predicted through a forward propagation algorithm.
In summary, the following steps: the method provided by the invention shows that the dense connection characteristic of the dense convolutional neural network can extract image characteristic information more completely, provide reliable basis for image identification and generate a highly-parameterized simplified model; by combining the combined supervision of softmax loss and central loss, intra-class aggregation and inter-class dispersion can be realized, and the robustness of the network to abnormal conditions such as fuzziness, shielding and noise is improved; therefore, the joint supervision method based on the dense convolutional neural network can realize accurate identification of the complex remote sensing airplane image.
Claims (1)
1. A joint supervision remote sensing airplane identification method based on a dense convolutional neural network is characterized by comprising the following steps:
the method comprises the following steps: training a dense convolutional neural network structure on a self-built remote sensing airplane database, wherein a training sample is a ten-class remote sensing airplane image and is subjected to preprocessing of rotation, noise addition, shielding at different degrees and motion blur of different pixels; the first layer of the network is a convolutional layer, then a maximum pooling layer, and then 3 dense blocks densely connected with the core part of the network, wherein each dense block comprises two convolutional layers, a transition layer is used between the dense blocks to reduce the output dimension, and then the pooling layer and a full connection layer are used, and finally a joint supervision method is used for outputting an identification result;
initializing parameter setting, wherein the learning rate lr in network training is set within the range of 0.009-0.02, the batchsize is set to ξ, namely, the weight is adjusted once every ξ samples are trained, the joint supervision adopted by the invention is to weight and sum a softmax loss function and a central loss function, balance the two loss functions by lambda, control the learning rate of a center by α, set the lambda within the range of 0.008-0.03, and set the α within the range of 0.2-0.5;
firstly, a remote sensing image of 178 × 178 pixels is subjected to a first convolutional layer to obtain 16 characteristic maps of 89 × 89, then the remote sensing image is subjected to maximum pooling to obtain 16 characteristic maps of 46 × 46, then the remote sensing image is subjected to 3 dense blocks to obtain 80 characteristic maps of 46 × 46, due to dense connection characteristics, 80 is obtained from 16+16 × 4, then the remote sensing image is subjected to a transition layer containing the convolutional layer and the pooling layer to obtain 40 characteristic maps of 23 × 23, similarly, 196 characteristic maps of 6 × 6 are obtained after the remote sensing image is subjected to 14 dense blocks and the transition layer, and finally the 196 characteristic maps of 1 × 1 are obtained through an average pooling layer and sent to a full connection layer;
inputting a 1 × 10 vector output by the full connection layer into a loss function L of joint supervision, wherein the formula (1) is as follows, Ls represents a softmax loss function, and Lc represents a central loss function;
wherein: x is the number ofiRepresenting the ith deep feature in d-dimensional space, belonging to the yiClass, d is the dimension of the feature space, W is a fully connected parameter matrix, W is { d × n }, d rows and n columns, Wj is the jth column of W, m is the size of batch processing, n is the number of classes, b is offset,
represents the y thiClass center of deep-like features, λ is used to balance two loss functions, if λ is set to 0, then it can be considered as a special case of such joint supervision, representing the use of softmax loss function only;
step five: calculating the joint loss and comparing the joint loss with a set threshold value, solving the partial derivative of the joint loss to calculate the back propagation error if the joint loss does not meet the requirement, wherein the equation is as follows
Step six, the equation of the updated weight and the depth feature class center after the error is obtained is as follows, in the training process, the deep feature class center is updated by adopting a training method for updating the deep feature class center based on a small-batch training set, in addition, in order to avoid large disturbance caused by a small amount of error standard samples, α is used for controlling the learning rate of the center, and Lc is calculated for xiGradient of (2) and
updating an equation:
wherein if y isiJ then δ (y)iJ) 1, if yiIs not equal to j, then delta (y)iJ) 0, α at [0,1]]Within the range, the weights and centroid adjustment are performed in subsequent cycles such asThe following equation is used for outputting a loss function calculation result until the requirement is met;
step seven: network testing: in order to verify the effectiveness of the method, the robustness of three abnormal conditions, namely noise, fuzziness and shielding, in the steps from one step to seven step of self-built test centralized verification is used for predicting the identification result of the remote sensing airplane through a forward propagation algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810326575.0A CN108549866B (en) | 2018-04-12 | 2018-04-12 | Remote sensing airplane identification method based on dense convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810326575.0A CN108549866B (en) | 2018-04-12 | 2018-04-12 | Remote sensing airplane identification method based on dense convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108549866A CN108549866A (en) | 2018-09-18 |
| CN108549866B true CN108549866B (en) | 2020-06-23 |
Family
ID=63514761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810326575.0A Active CN108549866B (en) | 2018-04-12 | 2018-04-12 | Remote sensing airplane identification method based on dense convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108549866B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829923B (en) * | 2018-12-24 | 2023-06-16 | 五邑大学 | Base station antenna downward inclination angle measurement system and method based on deep neural network |
| CN110059538B (en) * | 2019-02-27 | 2021-07-09 | 成都数之联科技有限公司 | Water body identification method based on deep dense neural network |
| CN110087097B (en) * | 2019-06-05 | 2021-08-03 | 西安邮电大学 | Method for automatically removing invalid video clips based on electronic endoscope |
| CN110232370B (en) * | 2019-06-21 | 2022-04-26 | 华北电力大学（保定） | Power transmission line aerial image hardware detection method for improving SSD model |
| CN111325239A (en) * | 2020-01-21 | 2020-06-23 | 上海眼控科技股份有限公司 | Image-based weather identification method and equipment |
| CN113705489B (en) * | 2021-08-31 | 2024-06-07 | 中国电子科技集团公司第二十八研究所 | Remote sensing image fine-granularity airplane identification method based on priori regional knowledge guidance |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6423926B1 (en) * | 2000-11-16 | 2002-07-23 | Joseph K. Kelly | Direct-metal-deposition (DMD) nozzle fault detection using temperature measurements |
| CN105512684B (en) * | 2015-12-09 | 2018-08-28 | 江苏航天大为科技股份有限公司 | Logo automatic identifying method based on principal component analysis convolutional neural networks |
| US10089717B2 (en) * | 2016-04-05 | 2018-10-02 | Flipboard, Inc. | Image scaling using a convolutional neural network |
| CN107330405A (en) * | 2017-06-30 | 2017-11-07 | 上海海事大学 | Remote sensing images Aircraft Target Recognition based on convolutional neural networks |
| CN107844743B (en) * | 2017-09-28 | 2020-04-28 | 浙江工商大学 | Image multi-subtitle automatic generation method based on multi-scale hierarchical residual error network |
| CN107818302A (en) * | 2017-10-20 | 2018-03-20 | 中国科学院光电技术研究所 | Non-rigid multi-scale object detection method based on convolutional neural network |
-
2018
- 2018-04-12 CN CN201810326575.0A patent/CN108549866B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108549866A (en) | 2018-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108549866B (en) | Remote sensing airplane identification method based on dense convolutional neural network | |
| US11195051B2 (en) | Method for person re-identification based on deep model with multi-loss fusion training strategy | |
| CN106599797B (en) | A kind of infrared face recognition method based on local parallel neural network | |
| CN104598890B (en) | A kind of Human bodys' response method based on RGB D videos | |
| CN110070074B (en) | Method for constructing pedestrian detection model | |
| CN103605972B (en) | Non-restricted environment face verification method based on block depth neural network | |
| CN110348399B (en) | Hyperspectral intelligent classification method based on prototype learning mechanism and multidimensional residual error network | |
| CN110287849A (en) | A kind of lightweight depth network image object detection method suitable for raspberry pie | |
| CN109800692B (en) | Visual SLAM loop detection method based on pre-training convolutional neural network | |
| CN109034376A (en) | A kind of unmanned plane during flying trend prediction method and system based on LSTM | |
| CN110781790A (en) | Visual SLAM closed loop detection method based on convolutional neural network and VLAD | |
| CN111461038A (en) | Pedestrian re-identification method based on layered multi-mode attention mechanism | |
| CN111460980B (en) | Multi-scale detection method for small-target pedestrian based on multi-semantic feature fusion | |
| CN110909762B (en) | Robot posture recognition method and device based on multi-sensor fusion | |
| CN106023257A (en) | Target tracking method based on rotor UAV platform | |
| CN106408030A (en) | SAR image classification method based on middle lamella semantic attribute and convolution neural network | |
| CN103065158A (en) | Action identification method of independent subspace analysis (ISA) model based on relative gradient | |
| CN112052772A (en) | Face shielding detection algorithm | |
| Wang et al. | Research on application of deep learning algorithm in image classification | |
| CN111062423B (en) | Point cloud classification method of point cloud graph neural network based on self-adaptive feature fusion | |
| CN107992850A (en) | A kind of outdoor scene three-dimensional colour point clouds sorting technique | |
| CN114329826A (en) | Plane cascade steady flow prediction method based on generative confrontation network | |
| CN107833239A (en) | A kind of searching of optimal matching method for tracking target based on weighted model constraint | |
| CN104268507A (en) | Manual alphabet identification method based on RGB-D image | |
| Raparthi et al. | Machine Learning Based Deep Cloud Model to Enhance Robustness and Noise Interference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |