CN116630794A - Remote sensing image target detection method based on sorting sample selection and electronic equipment - Google Patents

Remote sensing image target detection method based on sorting sample selection and electronic equipment Download PDF

Info

Publication number
CN116630794A
CN116630794A CN202310457764.2A CN202310457764A CN116630794A CN 116630794 A CN116630794 A CN 116630794A CN 202310457764 A CN202310457764 A CN 202310457764A CN 116630794 A CN116630794 A CN 116630794A
Authority
CN
China
Prior art keywords
target
remote sensing
loss
sensing image
positive
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310457764.2A
Other languages
Chinese (zh)
Other versions
CN116630794B (en
Inventor
冯鹏铭
谢明杰
贺广均
关键
金世超
梁颖
上官博屹
张鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Engineering University
Beijing Institute of Satellite Information Engineering
Original Assignee
Harbin Engineering University
Beijing Institute of Satellite Information Engineering
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Engineering University, Beijing Institute of Satellite Information Engineering filed Critical Harbin Engineering University
Priority to CN202310457764.2A priority Critical patent/CN116630794B/en
Publication of CN116630794A publication Critical patent/CN116630794A/en
Application granted granted Critical
Publication of CN116630794B publication Critical patent/CN116630794B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/766—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using regression, e.g. by projecting features on hyperplanes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Abstract

The invention relates to a remote sensing image target detection method based on sequencing sample selection, and electronic equipment, which are used for acquiring a remote sensing image and a corresponding target label and preprocessing; obtaining a corresponding multi-scale feature map through a feature extraction backbone network and a feature pyramid network; constructing a classification branch network and a position and angle regression branch network, and predicting a multi-scale feature map to obtain a target predicted value; calculating an intersection ratio self-adaptive threshold value on the multi-scale feature map by utilizing the target label and the predicted value, and screening sample points to obtain positive and negative samples meeting the conditions; calculating sorting loss, positioning sorting loss and regression loss for network training; repeating the steps to train the detection model; and detecting by using a detection model. The method for detecting the target by the rotating frame of the high-resolution remote sensing image, disclosed by the invention, has the advantages of relieving the problem that the traditional classification capability is difficult to learn due to imbalance of positive and negative samples, promoting the improvement of the target detection performance and having great significance for detecting the target of the rotating frame of the high-resolution remote sensing image.

Description

Remote sensing image target detection method based on sorting sample selection and electronic equipment
Technical Field
The invention relates to the technical field of remote sensing image target detection and identification, in particular to a remote sensing image target detection method based on sequencing sample selection and electronic equipment.
Background
With the rapid development of remote sensing imaging technology and commercial aerospace application, remote sensing images with various time and space resolutions are continuously acquired, so that researchers can rapidly and comprehensively complete tasks such as earth observation. However, the increasing magnitude of remote sensing images bring serious challenges to the timeliness and application level of the traditional remote sensing data processing flow, and also face the embarrassment that data are massive but knowledge is difficult to find, so that the remote sensing field needs an intelligent image interpretation means to meet the application requirements of automatic information extraction. Under the background, remote sensing target detection is taken as a technology for automatically positioning the position of an interested target from a remote sensing image and classifying the category to which the target belongs, and is one of research hotspots and difficulties which are paid attention to in the field of remote sensing and image processing at present. At present, by utilizing a deep learning technology based on a convolutional neural network to automatically mine and learn target characteristics, a rotating target in a remote sensing image can be well detected.
Common remote sensing rotating target detection methods based on convolutional neural networks are mainly divided into two types: the first is a two-stage detector, which mainly extracts a feature map from a feature extraction backbone network, firstly explicitly acquires candidate regions possibly containing targets through a region suggestion network, and then performs more detailed target classification and regression on the output result of the region suggestion network, and the representative algorithm includes: roI transducer, SCRDet and Gliding Vertex; the second type is a single-stage detector, which is mainly Based on two types of Anchor frames (Anchor-Based) and non-Anchor frames (Anchor-Free), and predicts the target position offset and category on the feature map by using the Anchor frames, and then carries out regression and classification, and the representative algorithm comprises: r3Det and S2A-Net; the Anchor-Free single-stage detection method performs dense detection on each pixel position on the feature map, directly predicts the position of a target frame on the feature map after downsampling, then classifies and regresses the target, and the representative algorithm comprises: IENet, TOSO and BBAVectors.
In the prior art, most detection methods can mine and learn the characteristics of remote sensing targets and obtain good performance, but the characteristic that positive and negative samples are extremely unbalanced in the remote sensing images is usually ignored, so that the traditional classification task is difficult to learn target characteristics from positive samples effectively, meanwhile, the rotation targets have challenges of arbitrary angles and different shapes, the problems prevent the traditional remote sensing rotation target detection task from obtaining better performance, and in the field of natural image target detection, a large number of methods are used for solving the problems, such as a method for adopting more balanced loss functions and a method for re-modeling classification and regression tasks into new tasks. Specifically, a learner puts forward a method for converting a classification task into a sequencing task to learn, so that the problem of insufficient training caused by unbalanced positive and negative samples is effectively solved. However, the remote sensing image target distribution has overall sparse and local dense characteristics, and the characteristics are not considered in the above method, so that the above method cannot be suitable for remote sensing target detection tasks.
Disclosure of Invention
In order to solve the technical problems in the prior art, the invention aims to provide a remote sensing image target detection method and electronic equipment based on ordered sample selection, which are used for relieving the problem that the traditional classification capacity is difficult to learn due to imbalance of positive and negative samples, promoting the target detection performance to be improved and having important significance for detecting the target of a high-resolution remote sensing image rotating frame.
In order to achieve the above object, the present invention provides a remote sensing image target detection method based on ordered sample selection, comprising the following steps:
s1, acquiring a remote sensing image and a corresponding target label, and preprocessing;
s2, obtaining a corresponding multi-scale feature map through a feature extraction backbone network and a feature pyramid network;
s3, constructing a classification branch network and a position and angle regression branch network, and predicting the multi-scale feature map to obtain a target predicted value;
s4, calculating an intersection ratio self-adaptive threshold value on the multi-scale feature map by using the target label in the step S1 and the predicted value obtained in the step S3, and screening sample points to obtain positive and negative samples meeting the conditions;
s5, calculating sorting loss, positioning sorting loss and regression loss to perform network training;
s6, repeatedly executing the steps S1 to S5, and training a detection model;
and S7, detecting the remote sensing image by using the detection model obtained in the step S6.
According to an aspect of the present invention, in the step S3, specifically includes:
step S31, based on the multi-scale feature map obtained in the step S2, establishing a classification prediction network branch and a position regression prediction network branch;
and S32, predicting the multi-scale feature map by using the classification prediction network branch and the position regression prediction network branch to obtain a target prediction value, and calculating a classification prediction score and a target boundary frame coordinate.
According to an aspect of the present invention, in the step S4, specifically includes:
step S41, according to the coordinates (x) of the real rotation frame g in the target label obtained in the step S1 g ，y g W, h, θ) and the target predicted value obtained in the step S3At each layer, a distance from the center point (x g ，y g ) Nearest 15 prediction boxes->And calculates the cross ratio +.>
Wherein i is the number of feature layers, x g Is the x-axis coordinate, y of the center point of the rotating frame g For the center point y-axis coordinate of the rotating frame, w is the width of the rotating frame, h is the height of the rotating frame,as a predicted value, ioU () is an intersection ratio calculation function, and the value of the intersection ratio calculation function is the ratio of the intersection area and the union area of two rectangular frames;
step S42, using said step S41 the cross-over ratio of each layer obtainedCalculating statistics for each layerAnd the obtained threshold value is used as a threshold value for dividing positive and negative samples in the ith layer, and the positive and negative samples are adaptively divided from the ith layer prediction frame.
According to an aspect of the present invention, in the step S42, specifically includes:
step S421, traversing the cross-over ratio between the predicted frame of each layer obtained in step S41 and the real frame obtained in step S1Calculate the mean->Sum of variances->Obtaining positive and negative sample division threshold->
Step S422, traversing the prediction frame obtained in step S41If the intersection ratio of the predicted frame and the real frame is greater than or equal toThe sample is considered to belong to the positive sample P, otherwise the sample is considered to belong to the negative sample N.
According to an aspect of the present invention, in the step S5, specifically includes:
step S51, calculating a classification sorting loss L according to the classification prediction score obtained in the step S3 and the positive sample set P and the negative sample set N obtained in the step S4 cpx ；
Step S52, calculating the positioning and sequencing loss L according to the intersection ratio and the positive and negative samples obtained in the step S4 rpx ；
Step S53, calculating regression Loss L by using Smooth-L1 Loss according to the position regression prediction obtained in the step S3 and the target label obtained in the step S1 reg ；
Step S54, constructing total loss
wherein ,Np Is the positive sample number, lambda 1 and λ2 Is a balance coefficient for adjusting the weight between losses, L cpx Is a sort order loss, L rpx Is the loss of positioning and ordering, L reg Is the regression loss.
According to an aspect of the present invention, in step S51, specifically, it includes:
step S511, traversing positive sample points in P according to the classification prediction scores obtained in the step S3 and the positive sample set P and the negative sample set N obtained in the step S4Positive sample point->The cross-over ratio with the real frame g is +.>Screening out the cross-over ratio from P and N to be less than +.>Sample Point addition set +.>
Step S512, traversing the positive sample points in PClassification prediction score +.>Calculate its sorting order loss L cpx ：
Where H (-) is a distance function, the calculation formula is as follows:
H(x)＝-log(1-S(x))；
wherein S () represents a Sigmoid function:
according to an aspect of the present invention, in the step S52, specifically includes:
step S521, the cross-correlation obtained in said step S4And positive and negative sample sets for positive samples +.>Traversing the positive sample set P, classifying samples with an intersection ratio smaller than that of the positive sample set P into a set +.>
Step S522, calculating the positioning and sorting loss L rpx ：
wherein ,represents->And the intersection ratio between the predicted frame and the real frame g in the position.
According to an aspect of the present invention, in step S53, specifically, the method includes:
step S531, position regression prediction obtained in step S3Target label t obtained in step S1 * ＝(x * ，y * ，w * ，h * ，θ * )；
Step S532, calculate regression loss L reg ：
According to an aspect of the present invention, there is provided an electronic apparatus including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a remote sensing image target detection method selected based on the sorted samples according to any one of the above technical solutions.
According to an aspect of the present invention, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement a remote sensing image target detection method based on ordered sample selection as set forth in any one of the above technical solutions.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a remote sensing image target detection method and electronic equipment based on sequencing sample selection, which are characterized in that a remote sensing image is processed through a feature extraction main network and a feature pyramid network to obtain a corresponding multi-scale feature image, then a classification branch network and a position and angle regression branch network are utilized to predict the multi-scale feature image to obtain a target predicted value, and statistical information of an intersection ratio of given target labels (position labels) and predicted information contained in the target predicted value is calculated firstly to obtain an intersection ratio self-adaptive threshold value to divide positive and negative samples; and modeling a classification task and a positioning task as sequencing tasks by utilizing classification sequencing loss and positioning sequencing loss, and inputting the classification task and the positioning task into a total loss function together with the position regression loss based on Smooth L1, so that more accurate target characteristic information is learned, the problems of overall sparsity and local density of remote sensing image target distribution are solved, and the problem that the traditional classification capability is difficult to learn due to imbalance of positive and negative samples is further solved.
Further, the sequencing task is expanded into the positioning task, positioning sequencing learning is added by utilizing the thought of multi-task learning, and positioning capability is improved, so that the remote sensing target detection performance is improved, and the method has important significance for detecting the rotating frame target of the high-resolution remote sensing image.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 schematically illustrates a flow chart of a remote sensing image target detection method based on ordered sample selection provided in one embodiment of the invention;
FIG. 2 schematically illustrates a model training flow diagram of a method for detecting a rotational target of a remote sensing image according to an embodiment of the present invention;
FIG. 3 schematically illustrates a model training phase diagram according to one embodiment of the invention;
fig. 4 schematically shows a flow chart of step S4 according to an embodiment of the invention.
Detailed Description
The description of the embodiments of this specification should be taken in conjunction with the accompanying drawings, which are a complete description of the embodiments. In the drawings, the shape or thickness of the embodiments may be enlarged and indicated simply or conveniently. Furthermore, portions of the structures in the drawings will be described in terms of separate descriptions, and it should be noted that elements not shown or described in the drawings are in a form known to those of ordinary skill in the art.
Any references to directions and orientations in the description of the embodiments herein are for convenience only and should not be construed as limiting the scope of the invention in any way. The following description of the preferred embodiments will refer to combinations of features, which may be present alone or in combination, and the invention is not particularly limited to the preferred embodiments. The scope of the invention is defined by the claims.
As shown in fig. 1 to 4, the remote sensing image target detection method based on ordered sample selection of the present invention includes the following steps:
s1, acquiring a remote sensing image and a corresponding target label, and preprocessing;
s2, obtaining a corresponding multi-scale feature map through a feature extraction backbone network and a feature pyramid network;
s3, constructing a classification branch network and a position and angle regression branch network, and predicting the multi-scale feature map to obtain a target predicted value;
s4, calculating an intersection ratio self-adaptive threshold value on the multi-scale feature map by using the target label in the step S1 and the predicted value obtained in the step S3, and screening sample points to obtain positive and negative samples meeting the conditions;
s5, calculating sorting loss, positioning sorting loss and regression loss to perform network training;
s6, repeatedly executing the steps S1 to S5, and training a detection model;
and S7, detecting the remote sensing image by using the detection model obtained in the step S6.
In the embodiment, in the model training process, a remote sensing image is processed through a feature extraction main network and a feature pyramid network to obtain a corresponding multi-scale feature image, then a classification branch network and a position and angle regression branch network are utilized to predict the multi-scale feature image to obtain a target predicted value, and statistical information of an intersection ratio value of given target labels (position labels) and predicted information contained in the target predicted value is calculated first to obtain an intersection ratio self-adaptive threshold value to divide positive and negative samples; and modeling a classification task and a positioning task as sequencing tasks by utilizing classification sequencing loss and positioning sequencing loss, and inputting the classification task and the positioning task into a total loss function together with the position regression loss based on Smooth L1, so that more accurate target characteristic information is learned, the problems of overall sparsity and local density of remote sensing image target distribution are solved, and the problem that the traditional classification capability is difficult to learn due to imbalance of positive and negative samples is further solved.
Further, the sequencing task is expanded into the positioning task, positioning sequencing learning is added by utilizing the thought of multi-task learning, and positioning capability is improved, so that the remote sensing target detection performance is improved, and the method has important significance for detecting the rotating frame target of the high-resolution remote sensing image.
In one embodiment of the present invention, preferably, in step S1, after the remote sensing image is acquired, the remote sensing image is preprocessed, where the preprocessing includes one or more of random flipping, rotation, scaling, and cropping.
In the embodiment, operations such as cutting and overturning are performed on the remote sensing image, so that the robustness and universality of a model algorithm are enhanced, and the generalization capability is improved.
As shown in fig. 3, in one embodiment of the present invention, preferably, in step S2, further includes:
s21, extracting the characteristics of the input remote sensing image by utilizing a characteristic extraction backbone network to obtain a characteristic map;
and S22, carrying out up-sampling, transverse connection and feature fusion on the feature map obtained in the step S21 by utilizing a feature pyramid to obtain a multi-scale feature map.
In one embodiment of the present invention, preferably, in step S3, the method specifically includes:
step S31, based on the multi-scale feature map obtained in the step S2, establishing a classification prediction network branch and a position regression prediction network branch;
and S32, predicting the multi-scale feature map by using the classification prediction network branch and the position regression prediction network branch to obtain a target prediction value, and calculating a classification prediction score and a target boundary frame coordinate.
As shown in fig. 4, in one embodiment of the present invention, preferably, in step S4, the method specifically includes:
step S41, according to the coordinates (x) of the real rotation frame g in the target label obtained in step S1 g ，y g W, h, θ) and the target predicted value obtained in step S3At each layer, a distance from the center point (x g ，y g ) Nearest 15 prediction boxes->And calculates the cross ratio +.>
Wherein i is the number of feature layers, x g Is the x-axis coordinate, y of the center point of the rotating frame g For the center point y-axis coordinate of the rotating frame, w is the width of the rotating frame, h is the height of the rotating frame,as a predicted value, ioU () is an intersection ratio calculation function, and the value of the intersection ratio calculation function is the ratio of the intersection area and the union area of two rectangular frames;
step S42, utilizing the cross ratio of each layer obtained in step S41Calculate statistics for each layer->And the obtained threshold value is used as a threshold value for dividing positive and negative samples in the ith layer, and the positive and negative samples are adaptively divided from the ith layer prediction frame.
In one embodiment of the present invention, preferably, in step S42, statistics that need to be calculatedComprises a mean valueSum of variances->The specific calculation steps are as follows:
step S421, traversing the cross-over ratio between the predicted frame of each layer obtained in step S41 and the real frame obtained in step S1Calculate the mean->Sum of variances->Obtaining positive and negative sample division threshold->
Step S422, traversing the prediction frame obtained in step S41If predicting a frame and a real frameThe cross-over ratio is greater than or equal toThe sample is considered to belong to the positive sample P, otherwise the sample is considered to belong to the negative sample N.
In the embodiment, the dynamic positive and negative sample division threshold is realized by constructing the data statistics based on the intersection ratio between the prediction frame and the real frame, a large number of low-quality samples can be ignored, high-quality positive samples are selected, and the target characteristic information is extracted more accurately, so that the remote sensing target detection performance is improved.
As shown in fig. 1 and 2, in one embodiment of the present invention, preferably, in step S5, the method specifically includes:
step S51, calculating a classification sorting loss L according to the classification prediction score obtained in step S3 and the positive sample set P and the negative sample set N obtained in step S4 cpx ；
Step S52, calculating a positioning and sequencing loss L according to the intersection ratio and the positive and negative samples obtained in the step S4 rpx ；
Step S53, calculating regression Loss L by using Smooth-L1 Loss according to the position regression prediction obtained in step S3 and the target label obtained in step S1 reg ；
Step S54, constructing total loss
wherein ,Np Is the positive sample number, lambda 1 and λ2 Is a balance coefficient for adjusting the weight between losses, L cpx Is a sort order loss, L rpx Is the loss of positioning and ordering, L reg Is the regression loss.
In one embodiment of the present invention, preferably, in step S51, specifically includes:
step S511, traversing positive sample points in P according to the classification prediction scores obtained in step S3 and the positive sample set P and the negative sample set N obtained in step S4Positive sample point->The cross-over ratio with the real frame g is +.>Screening out the cross-over ratio from P and N to be less than +.>Sample Point addition set +.>
Step S512, traversing the positive sample points in PClassification prediction score +.>Calculate its sorting order loss L cpx ：
Where H (-) is a distance function, the calculation formula is as follows:
H(x)＝-log(1-S(x))；
wherein S () represents a Sigmoid function:
in this embodiment, through the above steps, the Anchor-Free single-stage detection framework can perform target detection of the rotating frame by converting classification and positioning tasks into sequencing tasks and learning the sequencing tasks, which is beneficial to alleviating the problem of imbalance between positive and negative samples.
In one embodiment of the present invention, preferably, in step S52, specifically includes:
step S521, using the cross-correlation ratio obtained in step S4And positive and negative sample sets for positive samples +.>Traversing the positive sample set P, classifying samples with an intersection ratio smaller than that of the positive sample set P into a set +.>
Step S522, calculating the positioning and sorting loss L rpx ：
wherein ,represents->And the intersection ratio between the predicted frame and the real frame g in the position.
In this embodiment, through the above steps, the network training is performed by using the positioning ordering loss, which is beneficial to improving the positioning accuracy.
In one embodiment of the present invention, preferably, in step S53, regression prediction learning is performed using a smoothl 1 loss, and the specific loss is calculated as follows:
step S531, position regression prediction obtained in step S3Target label t obtained in step S1 * ＝(x * ，y * ，w * ，h * ，θ * )；
Step S532, calculate regression loss L reg ：
In one embodiment of the present invention, it is preferable that in step S6, whether the training is ended is determined by judging whether the number of training iterations reaches a preset value.
According to an aspect of the present invention, there is provided an electronic apparatus including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a remote sensing image target detection method selected based on the sorted samples according to any one of the above technical solutions.
According to one aspect of the present invention, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement a remote sensing image target detection method according to any one of the above technical solutions, based on ordered sample selection.
Computer-readable storage media may include any medium that can store or transfer information. Examples of a computer readable storage medium include an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an Erasable ROM (EROM), a floppy disk, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a Radio Frequency (RF) link, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
The invention discloses a remote sensing image target detection method based on ordered sample selection and electronic equipment, wherein the remote sensing image target detection method based on ordered sample selection comprises the following steps: s1, acquiring a remote sensing image and a corresponding target label, and preprocessing; s2, obtaining a corresponding multi-scale feature map through a feature extraction backbone network and a feature pyramid network; s3, constructing a classification branch network and a position and angle regression branch network, and predicting the multi-scale feature map to obtain a target predicted value; s4, calculating an intersection ratio self-adaptive threshold value on the multi-scale feature map by using the target label in the step S1 and the predicted value obtained in the step S3, and screening sample points to obtain positive and negative samples meeting the conditions; step S5, calculating sorting loss, positioning sorting loss and regression loss for network training, and step S6, repeatedly executing the steps S1 to S5 to train a detection model; and S7, detecting the remote sensing image by using the detection model obtained in the step S6, so that the problems of overall sparsity and local density of target distribution of the remote sensing image are solved, and the problem that the traditional classification capability is difficult to learn due to imbalance of positive and negative samples is further solved.
Further, the sequencing task is expanded into the positioning task, positioning sequencing learning is added by utilizing the thought of multi-task learning, and positioning capability is improved, so that the remote sensing target detection performance is improved, and the method has important significance for detecting the rotating frame target of the high-resolution remote sensing image.
Furthermore, it should be noted that the present invention can be provided as a method, an apparatus, or a computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should also be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
It is finally pointed out that the above description of the preferred embodiments of the invention, it being understood that although preferred embodiments of the invention have been described, it will be obvious to those skilled in the art that, once the basic inventive concepts of the invention are known, several modifications and adaptations can be made without departing from the principles of the invention, and these modifications and adaptations are intended to be within the scope of the invention. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Claims (10)

1. The remote sensing image target detection method based on sequencing sample selection is characterized by comprising the following steps of:
s1, acquiring a remote sensing image and a corresponding target label, and preprocessing;
s2, obtaining a corresponding multi-scale feature map through a feature extraction backbone network and a feature pyramid network;
s3, constructing a classification branch network and a position and angle regression branch network, and predicting the multi-scale feature map to obtain a target predicted value;
s4, calculating an intersection ratio self-adaptive threshold value on the multi-scale feature map by using the target label in the step S1 and the predicted value obtained in the step S3, and screening sample points to obtain positive and negative samples meeting the conditions;
s5, calculating sorting loss, positioning sorting loss and regression loss to perform network training;
s6, repeatedly executing the steps S1 to S5, and training a detection model;
and S7, detecting the remote sensing image by using the detection model obtained in the step S6.
2. The method for detecting a target in a remote sensing image based on ordered sample selection according to claim 1, wherein in step S3, specifically comprising:
step S31, based on the multi-scale feature map obtained in the step S2, establishing a classification prediction network branch and a position regression prediction network branch;
and S32, predicting the multi-scale feature map by using the classification prediction network branch and the position regression prediction network branch to obtain a target prediction value, and calculating a classification prediction score and a target boundary frame coordinate.
3. The method for detecting a target in a remote sensing image based on ordered sample selection according to claim 1, wherein in step S4, specifically comprising:
step S41, according to the coordinates (x) of the real rotation frame g in the target label obtained in the step S1 g ,y g W, h, θ) and the target predicted value obtained in the step S3At each layer, a distance from the center point (x g ,y g ) Nearest 15 prediction boxes->And calculates the cross ratio +.>
Wherein i is the number of feature layers, x g Is the x-axis coordinate, y of the center point of the rotating frame g For the center point y-axis coordinate of the rotating frame, w is the width of the rotating frame, h is the height of the rotating frame,as a predicted value, ioU () is an intersection ratio calculation function, and the value of the intersection ratio calculation function is the ratio of the intersection area and the union area of two rectangular frames;
step S42, utilizing the cross ratio of each layer obtained in the step S41Calculate statistics for each layer->And the obtained threshold value is used as a threshold value for dividing positive and negative samples in the ith layer, and the positive and negative samples are adaptively divided from the ith layer prediction frame.
4. The method for detecting a target in a remote sensing image based on ordered sample selection according to claim 3, wherein in step S42, specifically comprising:
step S421, traversing the cross-over ratio between the predicted frame of each layer obtained in step S41 and the real frame obtained in step S1Calculate the mean->Sum of variances->Obtaining positive and negative sample division threshold->
Step S422, traversing the prediction frame obtained in step S41If the intersection ratio of the predicted frame and the real frame is equal to or greater than +.>The sample is considered to belong to the positive sample P, otherwise the sample is considered to belong to the negative sample N.
5. The method for detecting a target in a remote sensing image based on ordered sample selection according to claim 1, wherein in step S5, specifically comprising:
step S51, calculating a classification sorting loss L according to the classification prediction score obtained in the step S3 and the positive sample set P and the negative sample set N obtained in the step S4 cpx ；
Step S52, calculating the positioning and sequencing loss L according to the intersection ratio and the positive and negative samples obtained in the step S4 rpx ；
Step S53, calculating regression Loss L by using Smooth-L1 Loss according to the position regression prediction obtained in the step S3 and the target label obtained in the step S1 reg ；
Step S54, constructing total loss
wherein ,Np Is the positive sample number, lambda 1 and λ2 Is a balance coefficient for adjusting the weight between losses, L cpx Is a sort order loss, L rpx Is the loss of positioning and ordering, L reg Is the regression loss.
6. The method for detecting a target in a remote sensing image based on ordered sample selection according to claim 5, wherein in step S51, specifically comprising:
step S511, traversing positive sample points in P according to the classification prediction scores obtained in the step S3 and the positive sample set P and the negative sample set N obtained in the step S4Positive sample point->The cross-over ratio with the real frame g is +.>Screening out the cross-over ratio from P and N to be less than +.>Sample Point addition set +.>
Step S512, traversing the positive sample points in PClassification prediction score +.>Calculate its sorting order loss L cpx ：
Where H (-) is a distance function, the calculation formula is as follows:
H(x)＝-log(1-S(x))；
wherein S () represents a Sigmoid function:
7. the method according to claim 5, wherein in step S52, the method specifically comprises:
step S521, the cross-correlation obtained in said step S4And positive and negative sample sets for positive samples +.>Traversing the positive sample set P, classifying samples with an intersection ratio smaller than that of the positive sample set P into a set +.>
Step S522, calculating the positioning and sorting loss L rpx ：
wherein ,represents->And the intersection ratio between the predicted frame and the real frame g in the position.
8. The method according to claim 5, wherein in step S53, the method specifically comprises:
step S531, position regression prediction obtained by the step S3Target label t obtained in step S1 * ＝(x * ,y * ,w * ,h * ,θ * )；
Step S532, calculate regression loss L reg ：
9. An electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, the one or more computer programs being stored in the memory, which when executed by the electronic device, causes the electronic device to perform the remote sensing image object detection method selected based on the ranked samples as claimed in any one of claims 1 to 8.
10. A computer readable storage medium storing computer instructions which, when executed by a processor, implement the remote sensing image target detection method based on ordered sample selection of any one of claims 1 to 8.
CN202310457764.2A 2023-04-25 2023-04-25 Remote sensing image target detection method based on sorting sample selection and electronic equipment Active CN116630794B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310457764.2A CN116630794B (en) 2023-04-25 2023-04-25 Remote sensing image target detection method based on sorting sample selection and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310457764.2A CN116630794B (en) 2023-04-25 2023-04-25 Remote sensing image target detection method based on sorting sample selection and electronic equipment

Publications (2)

Publication Number Publication Date
CN116630794A true CN116630794A (en) 2023-08-22
CN116630794B CN116630794B (en) 2024-02-06

Family

ID=87591046

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310457764.2A Active CN116630794B (en) 2023-04-25 2023-04-25 Remote sensing image target detection method based on sorting sample selection and electronic equipment

Country Status (1)

Country Link
CN (1) CN116630794B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112732433A (en) * 2021-03-30 2021-04-30 骊阳(广东)节能科技股份有限公司 Data processing system capable of carrying out priority allocation
CN113887605A (en) * 2021-09-26 2022-01-04 中国科学院大学 Shape-adaptive rotating target detection method, system, medium, and computing device
CN113920377A (en) * 2021-11-05 2022-01-11 中国电子科技集团公司信息科学研究院 Method of classifying image, computer device, and storage medium
CN114170527A (en) * 2021-11-30 2022-03-11 航天恒星科技有限公司 Remote sensing target detection method represented by rotating frame
CN114445482A (en) * 2022-01-29 2022-05-06 福州大学 Method and system for detecting target in image based on Libra-RCNN and elliptical shape characteristics
CN114926500A (en) * 2022-05-20 2022-08-19 中国科学技术大学 Twin network target tracking method and system based on sorting
CN115019181A (en) * 2022-07-28 2022-09-06 北京卫星信息工程研究所 Remote sensing image rotating target detection method, electronic equipment and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112732433A (en) * 2021-03-30 2021-04-30 骊阳(广东)节能科技股份有限公司 Data processing system capable of carrying out priority allocation
CN113887605A (en) * 2021-09-26 2022-01-04 中国科学院大学 Shape-adaptive rotating target detection method, system, medium, and computing device
CN113920377A (en) * 2021-11-05 2022-01-11 中国电子科技集团公司信息科学研究院 Method of classifying image, computer device, and storage medium
CN114170527A (en) * 2021-11-30 2022-03-11 航天恒星科技有限公司 Remote sensing target detection method represented by rotating frame
CN114445482A (en) * 2022-01-29 2022-05-06 福州大学 Method and system for detecting target in image based on Libra-RCNN and elliptical shape characteristics
CN114926500A (en) * 2022-05-20 2022-08-19 中国科学技术大学 Twin network target tracking method and system based on sorting
CN115019181A (en) * 2022-07-28 2022-09-06 北京卫星信息工程研究所 Remote sensing image rotating target detection method, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN116630794B (en) 2024-02-06

Similar Documents

Publication Publication Date Title
CN109614985A (en) A kind of object detection method based on intensive connection features pyramid network
CN113705478B (en) Mangrove single wood target detection method based on improved YOLOv5
CN112084869B (en) Compact quadrilateral representation-based building target detection method
CN111368769B (en) Ship multi-target detection method based on improved anchor point frame generation model
CN112767357A (en) Yolov 4-based concrete structure disease detection method
CN111091095B (en) Method for detecting ship target in remote sensing image
CN111753682B (en) Hoisting area dynamic monitoring method based on target detection algorithm
CN112766184B (en) Remote sensing target detection method based on multi-level feature selection convolutional neural network
Zhu et al. Diverse sample generation with multi-branch conditional generative adversarial network for remote sensing objects detection
CN113468968B (en) Remote sensing image rotating target detection method based on non-anchor frame
CN107016403A (en) A kind of method that completed region of the city threshold value is extracted based on nighttime light data
CN110334594A (en) A kind of object detection method based on batch again YOLO algorithm of standardization processing
CN113487600B (en) Feature enhancement scale self-adaptive perception ship detection method
CN112241950A (en) Detection method of tower crane crack image
CN115359366A (en) Remote sensing image target detection method based on parameter optimization
Khodaverdizahraee et al. Segment-by-segment comparison technique for earthquake-induced building damage map generation using satellite imagery
CN114429577B (en) Flag detection method, system and equipment based on high confidence labeling strategy
CN112560895A (en) Bridge crack detection method based on improved PSPNet network
CN116168240A (en) Arbitrary-direction dense ship target detection method based on attention enhancement
Yang et al. Road crack detection using deep neural network with receptive field block
CN116630794B (en) Remote sensing image target detection method based on sorting sample selection and electronic equipment
CN114913504A (en) Vehicle target identification method of remote sensing image fused with self-attention mechanism
CN115273131A (en) Animal identification method based on dual-channel feature fusion
CN112465821A (en) Multi-scale pest image detection method based on boundary key point perception
CN113947723A (en) High-resolution remote sensing scene target detection method based on size balance FCOS

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant