CN107665351B - Airport detection method based on difficult sample mining - Google Patents
Airport detection method based on difficult sample mining Download PDFInfo
- Publication number
- CN107665351B CN107665351B CN201710314261.4A CN201710314261A CN107665351B CN 107665351 B CN107665351 B CN 107665351B CN 201710314261 A CN201710314261 A CN 201710314261A CN 107665351 B CN107665351 B CN 107665351B
- Authority
- CN
- China
- Prior art keywords
- candidate
- network
- region
- area
- airport
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
- G06F18/2111—Selection of the most significant subset of features by using evolutionary computational techniques, e.g. genetic algorithms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Abstract
The invention provides an airport detection method based on difficult sample mining, which comprises the following steps: taking the optical remote sensing image and the corresponding annotation truth value as training data of the optical remote sensing image; training a candidate area extraction network; training a regional classification network; extracting candidate regions and performing coupling training with a region classification network; fine tuning of an end-to-end deep convolutional network based on hard sample mining; airport detection of end-to-end deep convolutional neural networks. The deep convolutional neural network is introduced into the remote sensing image airport detection, the convolutional network is utilized to extract high-level semantic feature information of a target in the remote sensing image, airport candidate areas are screened through the high-level semantic features, whether the candidate areas are airports or not is secondarily confirmed, and the recall ratio and the accuracy rate of airport detection in the remote sensing image are improved.
Description
Technical Field
The invention relates to the technical field of digital image processing, in particular to an airport detection method based on difficult sample mining.
Background
In recent years, with the improvement of remote sensing imaging technology, the remote sensing data volume is increased explosively. Aiming at massive remote sensing data, the key information contained in the big data is automatically mined by a machine, so that people can be liberated from a fussy and repetitive discrimination task, wherein the airport detection problem is widely concerned due to strong applicability of the airport detection problem in both military and civilian aspects.
Airport detection remains a rather challenging problem due to the influence of rotation angle, scale, illumination and other factors in the remote sensing images. At present, most airport detection methods extract an airport candidate region from a full-width remote sensing image, and then extract bottom layer characteristics such as specific geometry and texture of an airport aiming at the candidate region to determine whether the candidate region is an airport or not. Meanwhile, in order to improve the speed and efficiency of airport detection in the large-format remote sensing image, a part of methods introduce a method for detecting the saliency in vision into the airport detection, and assume that an airport area has higher saliency in the remote sensing image, so that the positioning process of an airport candidate area is accelerated.
However, most of the current airport detection methods are based on the bottom layer features (such as scale invariant feature descriptors, SIFT, etc.) or the unique geometric features of artificially designed airports, and because the feature generalization capability of manual design is poor, the application requirements of airport detection under the multi-scale condition are difficult to meet.
Therefore, how to provide an airport detection method based on difficult sample mining is a problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of the above, the invention provides an airport detection method based on difficult sample mining, which introduces a deep convolutional neural network into a remote sensing image airport detection, not only extracts high-level semantic feature information of a target in the remote sensing image by using the convolutional network, screens an airport candidate region through the high-level semantic feature, but also performs secondary confirmation on whether the candidate region is an airport, and improves the recall ratio and the accuracy of the airport detection in the remote sensing image.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for airport detection based on difficult sample mining, the method comprising the steps of:
taking the optical remote sensing image and the corresponding annotation truth value as training data of the optical remote sensing image;
training a candidate area extraction network;
training a regional classification network;
extracting candidate regions and performing coupling training with a region classification network;
fine tuning of an end-to-end deep convolutional network based on difficult sample mining;
airport detection of end-to-end deep convolutional neural networks.
Preferably, in the airport detection method based on difficult sample mining, the step of using the optical remote sensing image and the corresponding annotation truth value as the training data of the optical remote sensing image specifically comprises:
carrying out image annotation on the full-width optical remote sensing image to obtain an annotation truth value in the full-width optical remote sensing image;
taking 250 pixels as step length in the marked full-width optical remote sensing image, extracting image data with a fixed window size in a sliding manner, and respectively recording the sliding times in the vertical direction and the horizontal direction as row and col; marking a truth value according to the full-width optical remote sensing image, and recording a truth value frame coordinate of airport area image data intercepted by a sliding window;
the relational expression of the truth value frame coordinates in the sliding window and the truth value marked in the full-width optical remote sensing image is as follows:
(x′ k ,y′ k )＝(x k -col×250,y k row x 250), wherein (x) k ,y k ) And (x' k ,y′ k ) And respectively marking a truth coordinate of the full-breadth optical remote sensing image and a truth frame coordinate in the sliding window, wherein k is equal to {1,2} and respectively represents an upper left corner point and a lower right corner point of the coordinate.
Preferably, in the airport detection method based on difficult sample mining, the training step of the candidate area extraction network specifically includes:
randomly generating a weight parameter of each convolution layer in the candidate area extraction network according to Gaussian distribution with zero mean standard deviation of 0.01;
extracting high-level semantic feature information from each image in the training data;
calculating loss functions of candidate regions in the image one by one according to the high-level semantic feature information;
randomly picking N from loss functions of all candidate regions batch Performing back propagation to update the weight parameters of each convolution layer in the candidate area extraction network;
when the candidate area is selected, the proportion of positive samples to negative samples is 1:1, and when the number of positive samples in the image is insufficient, the negative samples are added as the candidate area.
Preferably, in the airport detection method based on difficult sample mining, the training step of the area classification network specifically includes:
extracting a candidate region for each image in training data according to the trained region extraction network, taking the extracted image data of the candidate region and the labeling truth value of the candidate region as the training data of the region classification network, and training the region classification network.
Preferably, in the airport detection method based on difficult sample mining, the step of training the coupling between the candidate region extraction and the region classification network specifically includes:
initializing the corresponding first five convolutional layers in the candidate area network according to the parameters of the first five convolutional layers in the area classification network, and realizing the sharing and coupling of the candidate area extraction and the area classification network.
Preferably, in the above airport detecting method based on difficult sample mining, the step of training the candidate region extraction and the region classification network by coupling further includes:
fixing the shared convolutional layer weight parameters;
fine-tuning the specific convolutional layer weight parameters except the shared convolutional layer in the candidate region extraction network;
and generating different candidate regions according to the trimmed candidate region extraction network, inputting the generated candidate regions and the labeling truth values of the candidate regions into the region classification network, and trimming the specific full-connection layer weight parameters except the shared convolution layer in the region classification network.
Preferably, in the airport detection method based on difficult sample mining, the fine tuning step of the end-to-end deep convolutional network based on difficult sample mining specifically includes:
the difficulty degree of the training data differentiation is scored by utilizing the region extraction network, and the degree is scored according to a loss function
Selecting training data which are difficult to distinguish, and fine-tuning the regional extraction network again; loss function
The expression is as follows:
L cls (p,u)＝-logp u
wherein i represents the index number of the candidate region, p i Indicates the probability that the ith candidate region is the target region, p i * If the value is a truth value label, the value is {0,1}, and 1 indicates that the candidate area is a positive sample, namely an airport area, otherwise, the value is 0, and indicates that the candidate area is a negative sample, namely a background area; t is t i Predictor vector representing the position of the candidate region, t i * A vector representing a true position of the candidate region; l is a radical of an alcohol cls (p, u) represents a region classification loss function; l is a radical of an alcohol reg (t, u) represents a regional regression loss function;N cls and N reg Respectively representing the number of candidate areas to be classified and the number of regression candidate areas; λ is the balance factor.
Preferably, in the airport detection method based on difficult sample mining, the fine tuning step of the end-to-end deep convolutional network based on difficult sample mining further includes:
according to the candidate region obtained by the region extraction network after the fine tuning again and the labeling truth value of the candidate region, the fine tuning is carried out on the weight parameters of the full connection layer in the region classification network;
the difficulty of correctly classifying the candidate regions is scored, and a loss function L (P) of each candidate region is calculated i ,T i ) According to the maximum loss value
And establishing a hard sample set for each candidate area, and fine-tuning the area classification network again.
Preferably, in the airport detection method based on hard sample mining, after the fine tuning step of the end-to-end deep convolutional network based on hard sample mining, the method further includes:
and cascading the area extraction network and the area classification network to obtain the end-to-end convolution neural network.
Preferably, in the airport detection method based on hard sample mining, the airport detection step of the end-to-end deep convolutional neural network specifically includes:
inputting the optical remote sensing image into an end-to-end depth convolution neural network, outputting coordinates of upper left points and lower right points of a minimum circumscribed rectangular frame of an area where an airport is located by the end-to-end depth convolution neural network, and evaluating the probability that the optical remote sensing image is the airport area.
According to the technical scheme, compared with the prior art, the airport detection method based on the difficult sample mining is characterized in that a region extraction network is trained by utilizing training data, then a truth value is marked on a candidate region and a candidate region of an airport extracted by the region extraction network to train a region classification network, convolution layers of the region extraction network and the region classification network are shared, the coupling of the networks is realized, and the region extraction network and the region classification network are finely adjusted; the finely adjusted region extraction network and region classification network automatically mine difficult samples in training data, the region extraction network and the region classification network are subjected to secondary fine adjustment on the automatically constructed difficult sample set, an end-to-end deep convolution neural network is obtained, the recall ratio of the network to an airport target can be improved, and the probability of false alarm and missing detection is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic flow chart of an airport detection method based on difficult sample mining according to the present invention;
fig. 2 is a schematic diagram of a detection result of the airport detection method based on difficult sample mining according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides an airport detection method based on difficult sample mining, which comprises the following steps: taking the optical remote sensing image and the corresponding annotation truth value as training data of the optical remote sensing image; training a candidate area extraction network; training a regional classification network; extracting candidate regions and performing coupling training with a region classification network; fine tuning of an end-to-end deep convolutional network based on hard sample mining; airport detection of end-to-end deep convolutional neural networks. Compared with the prior art, the airport detection method based on difficult sample mining, which is provided by the invention, firstly trains the region extraction network by using training data, then marks a true value training region classification network on the airport candidate region and the candidate region extracted by using the region extraction network, initializes the first five layers of convolution layer weight parameters of the region classification network to extract the corresponding first five layers of convolution layers in the network, realizes network sharing and coupling, and finely adjusts the region extraction network and the region classification network; mining a difficult sample according to a loss function of the regional extraction network, and carrying out secondary fine adjustment on the regional extraction network according to the difficult sample; the method comprises the steps of obtaining regional classification network training data by using a trimmed regional extraction network, training the regional classification network again, automatically mining a difficult sample in the training data by the regional classification network, carrying out secondary trimming on the regional classification network on an automatically constructed difficult sample set, obtaining an end-to-end deep convolutional neural network, improving the recall ratio and detection precision of the network on an airport target, and reducing the probability of false alarm and missing detection.
Referring to fig. 1, fig. 1 is a schematic flow chart of an airport detection method based on difficult sample mining according to the present invention. The airport detection method based on difficult sample mining specifically comprises the following steps:
step S101: taking the optical remote sensing image and the corresponding annotation truth value as training data of the optical remote sensing image;
the specific execution method comprises the following steps: the optical remote sensing image has larger capacity required by storage in a computer due to wider breadth, larger field of view, richer texture information and the like, but the optical remote sensing image is difficult to be directly used as training data for optimizing the model due to the capacity limit of internal storage equipment of the computer such as a video memory, an internal memory and the like. Therefore, in order to segment the full-width optical remote sensing image data and extract an effective part from the segmented full-width optical remote sensing image data as training data, the method mainly comprises the following two steps:
a. carrying out image annotation on the full-width optical remote sensing image, labeling an external rectangular frame of the region where the airport is located in the image, and recording the upper left corner and the lower right corner of the rectangular frameThe point coordinates are respectively (x) 1 ,y 1 ) And (x) 2 ,y 2 ) Marking a true value as a full-width optical remote sensing image;
b. setting a sliding window size to be 500 multiplied by 600 pixels in a marked full-width optical remote sensing image by taking 250 pixels as a step length, sequentially sliding from right to left and from top to bottom to extract image data with a fixed window size, and respectively recording the sliding times in the vertical direction and the horizontal direction as row and col; marking a true value according to the full-width optical remote sensing image, if the window comprises an airport area, taking image data intercepted by the window as training data, and recording the position of a true value frame in the window data at the moment. The relation between the real value box coordinate in the sliding window and the labeled real value in the full-width optical remote sensing image is represented as follows:
(x′ k ,y′ k )＝(x k -col×250,y k -row×250)
wherein (x) k ,y k ) And (x' k ,y′ k ) Marking a true value coordinate of the full-breadth optical remote sensing image and a coordinate of a true value frame in a sliding window respectively, wherein k is an element {1,2} representing an upper left angular point and a lower right angular point.
Step S102: training the candidate area extraction network by using the training data to obtain an initialized area extraction network, and extracting the airport candidate area in the training data;
the specific execution method comprises the following steps: randomly generating weight parameters of each convolution layer in a candidate region extraction network by using Gaussian distribution with zero mean standard deviation of 0.01, extracting high-level semantic features from each image in training data, and calculating loss functions of candidate regions in the image one by one based on the semantic features; randomly picking N out of the loss of all candidate regions batch The counter-propagation is used to update the weight parameters of each convolutional layer in the network. When the candidate area is selected, the proportion of positive samples to negative samples is 1:1, and when the number of positive samples in the image is insufficient, the negative samples are added to ensure that the candidate area with a fixed number is selected. Loss function L (p) used in candidate area extraction network i ,t i ) Sorting the loss function L by region cls (p, u) and the local regression loss function L reg (t, u) two-part groupBecomes, defined as follows:
L cls (p,u)＝-logp u
wherein i represents the index number of the candidate region, p i Indicating the probability that the ith candidate region is the target region, the truth label p i * Whether the candidate area is a target area or not is represented, the value is {0,1}, 1 represents that the candidate area is a positive sample, namely an airport area, and otherwise, the value is 0, and represents that the candidate area is a negative sample, namely a background area; t is t i A predictor vector representing the position of the candidate region, t i * A vector representing a true position of the candidate region; l is a radical of an alcohol cls (p, u) represents a region classification loss function; l is a radical of an alcohol reg (t, u) represents a regional regression loss function; n is a radical of cls And N reg Respectively representing the number of candidate areas to be classified and the number of regression candidate areas; λ is a balance factor;
for the regression loss function L reg In (t, u), x, y, w, h respectively represent the x-axis coordinate value and the y-axis coordinate value of the center of the candidate region true value frame, and the width and the height of the candidate region true value frame.
And (4) carrying out iterative optimization solution on weight parameters of each layer in the candidate region extraction network by adopting a random gradient descent algorithm. The calculation process is
V t+1 ＝uV t -α▽L(W t )
W t+1 ＝W t +V t+1
Wherein W t Weight parameter, V, representing each convolutional layer of the network at the time of the t-th iterative update t Represents the t-th iterationAnd (5) trimming the updated weight value variable in time.
Step S103: training a region classification network by using an airport candidate region and a mark truth value, and sharing a convolution layer of the network with a region extraction network;
the specific execution method comprises the following steps: extracting a candidate region from each image in the training data by using the region extraction network trained in the step S102, marking truth values of the extracted candidate region and the candidate region as the training data of the region classification network, training the region classification network, and based on a loss function consisting of a region classification loss function and a region regression loss function, adopting a random gradient descent algorithm to optimize and solve weight parameters of each layer in the region classification network, wherein the specific solving mode is the same as that in the step S102;
when the training of the regional classification network reaches the maximum iteration times or the iteration cutoff condition, initializing the corresponding first five convolutional layers in the candidate regional network by using the parameters of the first five convolutional layers in the regional classification network, realizing the sharing and coupling of the two networks, and combining the candidate region extraction and the regional classification network into a uniform end-to-end deep convolutional neural network.
Step S104: fixing the convolution layer shared in the network, and finely adjusting the area extraction network and the area classification network;
the specific execution method comprises the following steps: under the condition that convolutional layer parameters shared by the area extraction network and the area classification network are kept unchanged, firstly, the weight parameters of the special convolutional layers except the shared convolutional layers in the candidate area extraction network are finely adjusted, then different candidate areas are generated by using the finely adjusted area extraction network, the generated candidate areas and the truth value labels of the candidate areas are input into the area classification network, the weight parameters of the special full-connection layers except the shared convolutional layers in the area classification network are finely adjusted, and the end-to-end deep convolutional network is obtained.
Step S105: the difficulty degree of distinguishing the training data is scored by utilizing the regional extraction network, and the data which are difficult to distinguish are selected for retraining the regional extraction network;
the specific execution method comprises the following steps: in the invention, the hard sample mining process is introduced into the coupling training of the area extraction network and the area classification network, and the hard sample mining is mainly carried out aiming at two parts of candidate area extraction and area classification.
Firstly, the regional extraction network obtained in the previous coupling training is used for scoring the difficulty degree of airport candidate region extraction in the training data, and the score is used as a loss function
For this reason, the larger the loss function value is, the more difficult it is to extract the airport candidate region in the image.
L cls (p,u)＝-logp u
Wherein, the values of the parameters are the same as those in step S102.
From the above formula, a loss function can be found
The main concern is the accuracy of extracting and locating the real region of the airport in the input image. Therefore, images which are relatively difficult to locate and extract to a real airport area can be screened from training data through the excavation process of the difficult samples, and secondary fine adjustment is carried out on the candidate area extraction network according to the batch of difficult sample data, so that the final area extraction network is obtained.
Step S106: labeling truth values for candidate regions and candidate regions obtained by a region extraction network, selecting the candidate regions which are difficult to be correctly classified as difficult samples through a region classification network, and finely adjusting the region classification network;
the specific execution method comprises the following steps: the method comprises the following steps that a difficult sample mining process of a regional classification network is divided into two steps, firstly, a candidate region extraction network obtained based on difficult sample mining training is used for extracting airport candidate regions in all training data, then all the candidate regions are used as the training data for fine adjustment of weight parameters of a full connection layer in the regional classification network, and any parameter of a shared convolutional layer is not changed in the fine adjustment process; then inputting the airport candidate regions into the fine-tuned region classification network again, and calculating the loss function L (p) of each candidate region by utilizing the forward propagation process of the network i ,t i ) And scoring the ease of correct classification of all candidate regions based on this loss function. Selecting the front with the largest loss value
And establishing a difficult sample set in each candidate area, and performing secondary fine adjustment on the classification network on the difficult sample set, so that the recall ratio of the network in airport detection is improved, and the probability of missed detection and false detection is reduced.
Step S107: and cascading the area extraction network and the area classification network to obtain an end-to-end convolutional neural network finally used for airport detection.
Step S108: detection of airports in optical remote sensing images through end-to-end convolutional neural network
The specific execution method comprises the following steps: and inputting the optical remote sensing image into an end-to-end depth convolution neural network, outputting coordinates of upper left and lower right points of a minimum circumscribed rectangular frame of an area where an airport is located by the end-to-end depth convolution neural network, and evaluating the probability that the optical remote sensing image is the airport area.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a detection result of the airport detection method based on difficult sample mining according to the present invention.
In the diagram, a first behavior does not have a network airport detection result of difficult sample mining, a second behavior mines a difficult sample and aims at an end-to-end deep convolution neural network airport detection result of the difficult sample data set after fine adjustment, a blue box represents a false alarm area, a red box represents a positive detection area, and a green box represents a truth-labeled airport area. And the average detection precision of the end-to-end deep convolutional neural network is 79.29% and 83.02% respectively after the hard sample mining network and the hard sample mining training are not carried out, and detection experiments prove that the detection precision of the network can be effectively improved by mining the hard samples and finely adjusting the end-to-end deep convolutional neural network based on the hard sample data set, and the probability of false alarm and missed detection is reduced.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed in the embodiment corresponds to the method disclosed in the embodiment, so that the description is simple, and the relevant points can be referred to the description of the method part.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. An airport detection method based on difficult sample mining, which is characterized by comprising the following steps:
taking the optical remote sensing image and the corresponding annotation truth value as training data of the optical remote sensing image;
training a candidate area extraction network;
training a regional classification network;
the coupled training of the candidate region extraction and the region classification network comprises the following steps: initializing the corresponding first five convolutional layers in the candidate area network according to the parameters of the first five convolutional layers in the area classification network, and realizing the sharing and coupling of the candidate area extraction and the area classification network;
fixing the shared convolutional layer weight parameters;
fine-tuning the specific convolutional layer weight parameters except the shared convolutional layer in the candidate region extraction network;
extracting a network according to the trimmed candidate region to generate different candidate regions, inputting the generated candidate regions and the labeling truth values of the candidate regions into the region classification network, and trimming the weight parameters of the specific full-connection layers except the shared convolution layer in the region classification network;
fine tuning of an end-to-end deep convolutional network difficult to sample mining, and fine tuning of full-link layer weight parameters in the regional classification network according to a candidate region obtained by the region extraction network after fine tuning again and a true labeling value of the candidate region as training data; scoring the difficulty degree of correct classification of the candidate regions, calculating a loss function of each candidate region, establishing a difficult sample set according to the candidate region with the maximum loss value, and fine-tuning the region classification network again;
airport detection of end-to-end deep convolutional neural networks.
2. The airport detection method based on difficult sample mining of claim 1, wherein the step of using the optical remote sensing image and the corresponding annotation truth value as the training data of the optical remote sensing image specifically comprises:
carrying out image annotation on the full-width optical remote sensing image to obtain an annotation truth value in the full-width optical remote sensing image;
taking 250 pixels as step length in the marked full-width optical remote sensing image, sliding and extracting image data with a fixed window size, and respectively recording the sliding times in the vertical direction and the horizontal direction as row and col; marking a truth value according to the full-width optical remote sensing image, and recording a truth value frame coordinate of airport area image data intercepted by a sliding window;
the relational expression of the truth frame coordinates in the sliding window and the truth values marked in the full-width optical remote sensing image is as follows:
(x′ k ,y′ k )＝(x k -col×250,y k -row×250)，
wherein (x) k ,y k ) And (x' k ,y′ k ) And marking a truth coordinate of the full-breadth optical remote sensing image and a truth frame coordinate in the sliding window respectively, wherein k is an element {1,2} which represents an upper left corner point and a lower right corner point of the coordinate respectively.
3. The airport detection method based on difficult sample mining of claim 1, wherein the training step of the candidate area extraction network specifically comprises:
randomly generating a weight parameter of each convolution layer in the candidate area extraction network according to Gaussian distribution with zero mean standard deviation of 0.01;
extracting high-level semantic feature information from each image in the training data;
calculating loss functions of candidate areas in the image one by one according to the high-level semantic feature information;
randomly picking N from loss functions of all candidate regions batch Performing back propagation to update the weight parameters of each convolution layer in the candidate area extraction network;
when the candidate area is selected, the proportion of positive samples to negative samples is 1:1, and when the number of positive samples in the image is insufficient, the negative samples are added as the candidate area.
4. The airport detection method based on difficult sample mining of claim 1, wherein the training step of the area classification network specifically comprises:
extracting a candidate region for each image in training data according to the trained region extraction network, taking the image data of the extracted candidate region and the labeling truth value of the candidate region as the training data of the region classification network, and training the region classification network.
5. The airport detection method based on difficult sample mining of claim 1, wherein the fine tuning step of the end-to-end deep convolutional network based on difficult sample mining specifically comprises:
the difficulty degree of the training data differentiation is scored by utilizing the regional extraction network, and the difficulty degree is scored according to a loss function
Selecting training data which are difficult to distinguish, and fine-tuning the area extraction network again; loss function
The expression is as follows:
L cls (p,u)＝-log p u
wherein i represents the index number of the candidate region, p i Indicates the probability that the ith candidate region is the target region, p i * If the value is a truth value label, the value is {0,1}, and 1 indicates that the candidate area is a positive sample, namely an airport area, otherwise, the value is 0, and indicates that the candidate area is a negative sample, namely a background area; t is t i A predictor vector representing the position of the candidate region, t i * A vector representing a true position of the candidate region; l is a radical of an alcohol cls (p, u) represents a region classification loss function; l is reg (t, u) represents a regional regression loss function; n is a radical of hydrogen cls And N reg Respectively representing the number of candidate areas to be classified and the number of regression candidate areas; λ is the balance factor.
6. The airport detection method based on difficult sample mining of claim 5, wherein the fine tuning step of the end-to-end deep convolutional network based on difficult sample mining further comprises:
fine-tuning the weight parameters of the full-connection layer in the regional classification network according to the candidate regions obtained by the regional extraction network after fine-tuning and the labeling truth values of the candidate regions as training data;
the difficulty of correctly classifying the candidate regions is scored, and a loss function L (P) of each candidate region is calculated i ,T i ) According to the maximum loss value before
And establishing a hard sample set for each candidate area, and fine-tuning the area classification network again.
7. The method for airport detection based on difficult sample mining of claim 1, wherein the step of fine tuning of the end-to-end deep convolutional network based on difficult sample mining further comprises:
and cascading the area extraction network and the area classification network to obtain the end-to-end convolution neural network.
8. The airport detection method based on difficult sample mining of claim 1, wherein the airport detection step of the end-to-end deep convolutional neural network specifically comprises:
inputting the optical remote sensing image into an end-to-end depth convolution neural network, outputting coordinates of upper left points and lower right points of a minimum circumscribed rectangular frame of an area where an airport is located by the end-to-end depth convolution neural network, and evaluating the probability that the area is the airport area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710314261.4A CN107665351B (en) | 2017-05-06 | 2017-05-06 | Airport detection method based on difficult sample mining |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710314261.4A CN107665351B (en) | 2017-05-06 | 2017-05-06 | Airport detection method based on difficult sample mining |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107665351A CN107665351A (en) | 2018-02-06 |
| CN107665351B true CN107665351B (en) | 2022-07-26 |
Family
ID=61122311
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710314261.4A Active CN107665351B (en) | 2017-05-06 | 2017-05-06 | Airport detection method based on difficult sample mining |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107665351B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416287B (en) * | 2018-03-04 | 2022-04-01 | 南京理工大学 | Pedestrian detection method based on missing negative sample mining |
| CN108549857B (en) * | 2018-03-30 | 2021-04-23 | 国信优易数据股份有限公司 | Event detection model training method and device and event detection method |
| CN108960314B (en) * | 2018-06-26 | 2022-02-11 | 北京旷视科技有限公司 | Training method and device based on difficult samples and electronic equipment |
| CN109117786B (en) * | 2018-08-09 | 2021-11-30 | 百度在线网络技术（北京）有限公司 | Data processing method and device based on neural network model and readable storage medium |
| CN109410220B (en) * | 2018-10-16 | 2019-12-24 | 腾讯科技（深圳）有限公司 | Image segmentation method and device, computer equipment and storage medium |
| CN109753910B (en) * | 2018-12-27 | 2020-02-21 | 北京字节跳动网络技术有限公司 | Key point extraction method, model training method, device, medium and equipment |
| CN110414587A (en) * | 2019-07-23 | 2019-11-05 | 南京邮电大学 | Depth convolutional neural networks training method and system based on progressive learning |
| CN110659680B (en) * | 2019-09-16 | 2022-02-11 | 西安电子科技大学 | Image patch matching method based on multi-scale convolution |
| CN111091164A (en) * | 2020-03-25 | 2020-05-01 | 魔视智能科技(上海)有限公司 | Semantic segmentation model training method and system based on oem |
| CN111738081A (en) * | 2020-05-20 | 2020-10-02 | 杭州电子科技大学 | Deep neural network sonar target detection method difficult for sample retraining |
| CN112215252B (en) * | 2020-08-12 | 2023-05-30 | 南强智视(厦门)科技有限公司 | Weak supervision target detection method based on-line difficult sample mining |
| CN112733860B (en) * | 2021-01-27 | 2021-09-10 | 上海微亿智造科技有限公司 | Method and system for mining difficult samples of two-classification segmentation network |
| CN113160175B (en) * | 2021-04-23 | 2022-06-14 | 杭州迪英加科技有限公司 | Tumor lymphatic vessel infiltration detection method based on cascade network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105740894A (en) * | 2016-01-28 | 2016-07-06 | 北京航空航天大学 | Semantic annotation method for hyperspectral remote sensing image |
| CN106022232A (en) * | 2016-05-12 | 2016-10-12 | 成都新舟锐视科技有限公司 | License plate detection method based on deep learning |
| CN106250812A (en) * | 2016-07-15 | 2016-12-21 | 汤平 | A kind of model recognizing method based on quick R CNN deep neural network |
| CN106570477A (en) * | 2016-10-28 | 2017-04-19 | 中国科学院自动化研究所 | Vehicle model recognition model construction method based on depth learning and vehicle model recognition method based on depth learning |
-
2017
- 2017-05-06 CN CN201710314261.4A patent/CN107665351B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105740894A (en) * | 2016-01-28 | 2016-07-06 | 北京航空航天大学 | Semantic annotation method for hyperspectral remote sensing image |
| CN106022232A (en) * | 2016-05-12 | 2016-10-12 | 成都新舟锐视科技有限公司 | License plate detection method based on deep learning |
| CN106250812A (en) * | 2016-07-15 | 2016-12-21 | 汤平 | A kind of model recognizing method based on quick R CNN deep neural network |
| CN106570477A (en) * | 2016-10-28 | 2017-04-19 | 中国科学院自动化研究所 | Vehicle model recognition model construction method based on depth learning and vehicle model recognition method based on depth learning |
Non-Patent Citations (1)
| Title |
|---|
| "Ship detection in SAR images based on an improved faster R-CNN";Jianwei Li et al;《2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA)》;20171114;正文第1-4页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107665351A (en) | 2018-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107665351B (en) | Airport detection method based on difficult sample mining | |
| CN107665498B (en) | Full convolution network aircraft detection method based on typical example mining | |
| EP2528017B1 (en) | Image identification device, image identification method, image identification program, and recording medium | |
| CN107833213B (en) | Weak supervision object detection method based on false-true value self-adaptive method | |
| WO2017190656A1 (en) | Pedestrian re-recognition method and device | |
| CN110580699A (en) | Pathological image cell nucleus detection method based on improved fast RCNN algorithm | |
| CN104978567B (en) | Vehicle checking method based on scene classification | |
| CN106372624B (en) | Face recognition method and system | |
| US20150332117A1 (en) | Composition modeling for photo retrieval through geometric image segmentation | |
| US20140270479A1 (en) | Systems and methods for parameter estimation of images | |
| CN114332650B (en) | Remote sensing image road identification method and system | |
| CN108154151B (en) | Rapid multi-direction text line detection method | |
| CN113095263B (en) | Training method and device for pedestrian re-recognition model under shielding and pedestrian re-recognition method and device under shielding | |
| CN103295013A (en) | Pared area based single-image shadow detection method | |
| CN112132004A (en) | Fine-grained image identification method based on multi-view feature fusion | |
| CN110909724B (en) | Thumbnail generation method of multi-target image | |
| CN110751619A (en) | Insulator defect detection method | |
| CN111882586A (en) | Multi-actor target tracking method oriented to theater environment | |
| Zhu et al. | Detecting natural scenes text via auto image partition, two-stage grouping and two-layer classification | |
| CN111160107B (en) | Dynamic region detection method based on feature matching | |
| CN108288061A (en) | A method of based on the quick positioning tilt texts in natural scene of MSER | |
| Hung et al. | Skyline localization for mountain images | |
| CN110390283B (en) | Cross-camera pedestrian re-retrieval method in commercial scene | |
| Wu et al. | Vehicle detection in high-resolution images using superpixel segmentation and CNN iteration strategy | |
| CN114267052A (en) | Pedestrian detection method based on yolov5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |