CN113537098B - Method for detecting and identifying impact pit in landing image - Google Patents
Method for detecting and identifying impact pit in landing image Download PDFInfo
- Publication number
- CN113537098B CN113537098B CN202110826180.9A CN202110826180A CN113537098B CN 113537098 B CN113537098 B CN 113537098B CN 202110826180 A CN202110826180 A CN 202110826180A CN 113537098 B CN113537098 B CN 113537098B
- Authority
- CN
- China
- Prior art keywords
- pit
- impact
- impact pit
- detection
- pits
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000001514 detection method Methods 0.000 claims abstract description 98
- 238000004364 calculation method Methods 0.000 claims abstract description 5
- 238000000605 extraction Methods 0.000 claims description 21
- 230000004913 activation Effects 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 5
- 230000004927 fusion Effects 0.000 abstract description 3
- 239000011159 matrix material Substances 0.000 description 14
- 230000008569 process Effects 0.000 description 8
- 238000010276 construction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 description 2
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
A landing image crash pit detection and identification method, comprising the steps of: detecting the crash pit through the crash pit detection network DPCDN of the single-stage multiplication anchor point; the method comprises the steps of matching the impact pit targets output by the impact pit detection network DPCDN with a pre-established celestial body surface impact pit set, and identifying the impact pit targets output by the impact pit detection network DPCDN. The invention provides an operation for multiplying anchor points, which is helpful for detecting small-size collision pits and improving the network performance. Based on the detection result, a weighted bipartite graph-based collision pit matching recognition method is provided, a simple and efficient weight calculation method is designed, and information of collision pits to be matched and surrounding collision pits is fused. The invention has the characteristics of high average detection precision, strong detection capability for small-size dense impact pits and high recognition speed of fusion sequence image information, and achieves the leading performance level of impact pit detection and recognition.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a detection and recognition method of an impact pit in a landing image.
Background
Realizing soft landing on the surface of the planet is one of key technologies of space exploration tasks, and the soft landing requires a high-precision positioning technology. The problem that inertial navigation commonly used at present only can measure relative postures and has accumulated errors can be avoided by using the impact pit for positioning. The pose estimation method based on the impact pit comprises the following steps of processing an input image and detecting the impact pit; matching the detected crash pit data with a crash pit database, a process known as identification; finally, the pose is calculated by using the associated data.
The precision and the robustness of the detection of the impact pit are influenced by illumination conditions, noise, visual angles and noise, the dimensional change of the impact pit is large, and different impact pits are overlapped and nested, so that the detection algorithm is challenged. The existing impact pit detection algorithm is characterized in that the edge detection and the matching and fitting of the bright and dark features of the impact pit are carried out through the main process, the algorithm is complex, and the performance of the algorithm on a small-size dense impact pit scene is poor; the neural network is also used for detecting and outputting the impact pit, but the super parameters required to be designed are more, and the multi-scale detection is inflexible.
The identification is a process of associating the impact pits In the picture with the impact pits In a pre-established impact pit database, wherein each impact pit has a unique number In a global database, and the database also contains information such as three-dimensional coordinates under a celestial coordinate system and the like, and is divided into two scenes of identification with priori knowledge and Lost In Space (Lost In Space). The latter often requires that some constraints be assumed to accomplish the recognition, e.g. that the camera is shooting approximately vertically. Under the condition of inertial navigation, an initial pose is often known, and the accumulated error exists in the pose, so that the initial pose can be used as a priori pose input of the crash pit recognition, and the result of inertial navigation is corrected by using the results of the crash pit recognition and the pose calculation. The recognition algorithm should have strong robustness and real-time. During landing, the images are imaged as sequence images, and the information of the sequence images can also be used for assisting identification. At present, no method for detecting and identifying the impact pit can be realized at the same time and the image information of the sequence is fused.
Disclosure of Invention
In order to realize high-precision detection and identification of landing image impact pit targets, the invention provides a method for detecting and identifying landing image impact pits. The method is characterized in that the method is input into a sequence image, an initial pose of a first image and a pre-established collision pit database, and collision pits detected for each frame of image and matching recognition results of the collision pits and the database are output. The method consists of a single-stage multiplication anchor point impact pit detection network and two parts of 'frame-to-frame', 'frame-to-database' matching identification.
The invention provides a rapid feature extraction layer for helping a network to rapidly extract image features, provides operation of multiplying anchor points, helps to detect small-size collision pits, and improves network performance. Based on the detection result, a weighted bipartite graph-based collision pit matching recognition method is provided, a simple and efficient weight calculation method is designed, and information of collision pits to be matched and surrounding collision pits is fused. The invention further provides a matching algorithm between the frame and between the frame and the database. The invention has the characteristics of high average detection precision, strong detection capability for small-size dense impact pits and high recognition speed of fusion sequence image information, and achieves the leading performance level of impact pit detection and recognition.
The technical scheme of the invention is as follows:
A landing image crash pit detection and identification method comprising the steps of:
Step S100, detecting an impact pit through an impact pit detection network DPCDN of a single-stage multiplication anchor point;
Step S200, matching the impact pit target output by the impact pit detection network DPCDN with the pre-established celestial body surface impact pit set, so as to identify the impact pit target output by the impact pit detection network DPCDN.
Furthermore, the impact pit detection network rapidly extracts features through a rapid feature extraction layer, then uses a feature pyramid to enhance the features, detects and outputs the enhanced feature layer, and different layers of the feature pyramid are responsible for detecting impact pit targets with different sizes so as to realize rapid multi-scale impact pit detection.
Further, the detection of small-size impact pits is realized by using multiplication anchor point operation on a low-level feature layer.
Further, the step of the impact pit detection network for rapidly extracting features through the rapid feature extraction layer comprises the following steps:
(1) Characteristic input passes through a convolution layer, then results are inverted and spliced with results which are not inverted, and the spliced characteristic is input into a ReLU activation function;
(2) The fast feature extraction layer comprises 4 convolution layers, wherein the first two convolution layers use the operation of step (1), and feature extraction of the image is completed through the 4 convolution layers.
Further, the operation of the multiplication anchor point comprises the following steps:
Each point (x, y) on the feature map corresponds to the original image Wherein s is the scaling multiple of the original image to the layer of feature image, and each point on the feature image corresponds to a plurality of points of the original image under the operation of multiplying the anchor pointsI e Z and k i < s, k is the multiplication number, Z is an integer set, the network has multiple output detection layers, and the multiplication anchor operation is used on the first detection output.
Further, the step of matching and identifying the impact pit target output by the impact pit detection network DPCDN with the pre-established celestial body surface impact pit set includes:
step S210, projecting a pre-established celestial body surface impact pit set database to a current view field through an initial gesture to serve as a set to be matched; taking an impact pit target output by an impact pit detection network as a target set;
Step S220, after all the impact pits in the pre-established celestial body surface impact pit set database are projected to the current field of view, matching a set Θ global to be matched with a target set Θ k;
step S230, fusing landing image sequence information, and matching the k-th frame of impact pit with the impact pit which is completely identified in the previous k-1 frame so as to realize the matching identification of the impact pit target set output by the impact pit detection network and the impact pit in the pre-established celestial body surface impact pit set database.
Further, matching the set to be matched with the target set by using a weighted bipartite graph matching algorithm, wherein the weight is the cross-over ratio between the target frames of the impact pit and the Euclidean distance of the characteristic mode of the impact pit; the weights in the weighted bipartite graph optimal matching algorithm comprise two parts: (1) the cross-over ratio between rectangular frames; (2) The specific pattern of the impact pits to be matched and the impact pits surrounding the impact pits to be matched in the set to be matched.
Further, the step of calculating the Euclidean distance weight of the characteristic mode in the weighted bipartite graph optimal matching algorithm is as follows:
(1) Initializing feature vector v= (0, …, 0) T∈Rn×1, where E is a discrete factor and R is a real set.
(2) Selecting an impact pit to be matched as a central impact pit, forming a specific mode by the central impact pit and m impact pits around the central impact pit, calculating an angle theta between any two surrounding impact pits and the central impact pit, updating a characteristic vector V [ q ] =V [ q ] +1 by using theta epsilon [ qe (q+1) e), wherein the angle theta is an included angle, and q=theta/e is rounded downwards.
(3) Repeating the steps (1) and (2) until all the included angles are calculated.
Furthermore, in the step of matching and identifying the impact pit output by the impact pit detection network and the impact pit in the pre-established impact pit database, matching of the matched impact pit and the impact pit under the current field of view is realized by using a Kalman filter and a weighted bipartite graph optimal matching algorithm.
Compared with the prior art, the invention has the following beneficial technical effects:
the method has high average detection precision, strong detection capability on dense small-size impact pits and small network scale, and is suitable for detecting multi-scale impact pits;
Secondly, aiming at the scene of dense small-scale impact pits, the operation of multiplying anchor points is provided, and the average detection precision of the impact pits is improved;
thirdly, a backbone network is extracted by using the designed rapid characteristics, the network scale is compressed, and rapid detection is realized;
fourthly, the identification method combines the characteristics that the landing image is a sequence image, so that the identification matching among frames and between frames and a database is realized, and the combination of the frames and the database accelerates the speed of an identification algorithm;
Fifthly, characteristic descriptors of the collision pits are designed by fusing information of the collision pits to be matched and surrounding collision pits, and are identified by using a weighted bipartite graph matching method, so that accuracy and robustness of an identification algorithm are improved by the aid of information of the intersection ratio of the characteristic descriptors and the target frames.
Drawings
FIG. 1 is a flow chart of the method of detecting and identifying an impact pit according to the present invention;
FIG. 2 is a diagram of a DPCDN detection network framework in accordance with the present invention;
FIG. 3 is a view of the present invention before and after the multiplication anchor, centered in the impingement pit;
FIG. 4 is a flowchart of the method of identifying an impact pit according to the present invention;
FIG. 5 is a representation of the feature descriptor encoding scheme of the present invention;
Fig. 6 is a frame-to-frame bump pit state transition diagram of the present invention.
Detailed Description
The objects, technical solutions and advantages of the present invention will become more apparent by the following detailed description of the present invention with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.
The invention provides a method for detecting and identifying an impact pit of a landing image, which is used for detecting and identifying the impact pit in the landing image. As shown in fig. 1. Specifically, the method comprises the following steps:
step S100, detecting the crash pit through the crash pit detection network DPCDN of the single-stage multiplication anchor point.
Specifically, the impact pit detection network rapidly extracts features through a rapid feature extraction layer, and only comprises 4 convolution operations; and then, enhancing the characteristics by using the characteristic pyramid, detecting and outputting the enhanced characteristic layer, wherein different layers of the characteristic pyramid are responsible for detecting the impact pit targets with different sizes, so that the rapid multi-scale impact pit detection is realized. Detection of small-size crash pits can be achieved using multiplication anchor operations on low-level feature layers.
Specifically, the step of rapidly extracting features by the impact pit detection network through the rapid feature extraction layer comprises the following steps:
(1) The feature input passes through the convolution layer, then the result is inverted and then spliced with the result which is not inverted, and the spliced feature is input into the ReLU activation function, namely, one-time CReLU operation is performed.
(2) The fast feature extraction layer contains 4 convolution layers, with the first two convolution layers operating with CReLU, through which feature extraction of the image is accomplished.
Therefore, the number of model parameters can be reduced, and the feature extraction speed can be increased.
Specifically, the operation of multiplying the anchor point includes the following steps:
Each point (x, y) on the feature map corresponds to the original image Wherein s is the scaling multiple of the original image to the layer of feature image, and each point on the feature image corresponds to a plurality of points of the original image under the operation of multiplying the anchor pointsI e Z and k i < s, k is the multiplication number, Z is an integer set, the network has multiple output detection layers, and the multiplication anchor operation is used on the first detection output.
Step S200, matching the impact pit target output by the impact pit detection network DPCDN with the pre-established celestial body surface impact pit set, so as to identify the impact pit target output by the impact pit detection network DPCDN.
Specifically, the step of matching and identifying the impact pit target output by the impact pit detection network DPCDN with the pre-established celestial body surface impact pit set includes:
step S210, projecting a pre-established celestial body surface impact pit set database to a current view field through an initial gesture to serve as a set to be matched; taking an impact pit target output by an impact pit detection network as a target set;
Specifically, the method comprises the following steps:
(1) Frame 1 has a rough pose P initial, known as camera reference K, of projecting the impingement pits in the pre-established celestial surface impingement pit set database to the pixel plane:
(u,v)T＝Γ(X)＝KPXT
Γ (X) represents an equation for projecting the center point of the bump pit onto the camera plane, X is the center point of the bump pit, and is represented by three-dimensional coordinates, (u, v) T is the center point coordinates of the bump pit center point projected onto the pixel plane, and P is the camera pose.
(2) Taking 5 points on the edge of the impact pit X, obtaining corresponding points in 5 pixel planes through the projection, and fitting to obtain an elliptic equation and a corresponding circumscribed rectangle;
Step S220, after all the impact pits in the pre-established celestial body surface impact pit set database are projected to the current field of view, the set Θ global to be matched is matched with the target set Θ k.
Specifically, matching a set to be matched and a target set by using a weighted bipartite graph matching algorithm, wherein the weight is the cross-over ratio between the target frames of the impact pit and the Euclidean distance of the characteristic mode of the impact pit;
The weighted bipartite graph best matching algorithm is called KM algorithm. The weights in the algorithm consist of two parts: (1) the cross-over ratio between rectangular frames; (2) The specific pattern of the impact pits to be matched and the impact pits surrounding the impact pits to be matched in the set to be matched.
Specifically, the step of calculating the Euclidean distance weight of the characteristic mode in the weighted bipartite graph optimal matching algorithm is as follows:
(1) Initializing feature vector v= (0, …, 0) T∈Rn×1, where E is a discrete factor and R is a real set.
(2) Selecting an impact pit to be matched as a central impact pit, forming a specific mode by the central impact pit and m impact pits around the central impact pit, calculating an angle theta between any two surrounding impact pits and the central impact pit, updating a characteristic vector V [ q ] =V [ q ] +1 by using theta epsilon [ qe (q+1) e), wherein the angle theta is an included angle, and q=theta/e is rounded downwards.
(3) Repeating the steps (1) and (2) until all the included angles are calculated.
In the step of matching and identifying the impact pit output by the impact pit detection network and the impact pit in the pre-established impact pit database, the matched impact pit and the impact pit under the current field of view are matched by utilizing a Kalman filter and a weighted bipartite graph optimal matching algorithm, and the specific steps are as follows:
(1) The set of detection network output that the preamble has matched the pre-established celestial body surface set of impingement holes database is denoted Θ' =Θ {0,1,2,…,k-1}. The state of any crash pit in Θ' is expressed as:
where x, y is the center coordinates of the target frame, r is the aspect ratio, Is the variation of the aspect ratio, h is the height, the remaining variablesThe motion between two frames is modeled as a linear motion, and the predictive equation is:
Xk＝FXk-1
PK＝FPK-1FT+Q
Wherein X k is the predicted value of the state quantity of the kth frame, X k-1 is the predicted value of the state quantity of the kth-1 frame, F is the state transition matrix, P is the predicted value of the covariance matrix, and Q is the systematic error.
(2) And calculating the characteristic mode of each collision pit, matching by using a weighted bipartite graph optimal matching algorithm, updating the state, and taking the following equation as an updating equation of the Kalman filter.
y＝z-Hx′
K＝P′HT(HP′HT+R)-1
xk+1＝x′+Ky
P＝(I-KH)P′
Z is the detection result of the K frame merle, H is a measurement matrix, x 'is the state predicted value of the K frame, P' is the predicted covariance matrix of the K frame, the values of x 'and P' are obtained by the prediction equation, I is a unit matrix, R is a noise matrix, and K is Kalman filtering gain.
(3) The state of the set Θ 'is updated by the matching result, and the three states of the crash pit in the set Θ' are respectively a new crash pit S 1, a mature crash pit S 2 and an old crash pit S 3. The matching result updates the state of the Θ set. If a crash pit is detected for the first time, i.e. the crash pit is not associated with any previous crash pit, then this crash pit is added to the set Θ' and the state is "new" crash pit. If a certain crash pit is already a crash pit in the omega set, 2 consecutive frames are associated with crash pits in the theta set, the state thereof is updated to be a 'mature' crash pit, and if a crash pit of a certain crash pit in the theta set is not associated with 2 consecutive frames, the state thereof is changed to be an 'old' crash pit and is deleted.
Step S230, fusing landing image sequence information, and matching the k-th frame of impact pit with the impact pit which is completely identified in the previous k-1 frame so as to realize the matching identification of the impact pit target set output by the impact pit detection network and the impact pit in the pre-established celestial body surface impact pit set database.
The method for detecting and identifying landing image crash pits according to the present invention will be described in detail with reference to specific examples.
Method for detecting and identifying impact pit
The method for detecting and identifying the impact pit comprises two stages, wherein the first stage is impact pit detection, and the detection network is a full-connection convolutional neural network with single-stage multiplication anchor points; and step two, matching the collision pits in the frame and the database by a weighted bipartite graph matching algorithm, wherein the weight comprises the cross ratio of the target frame and the Euclidean distance of the feature descriptor.
In the embodiment of the invention, the method for detecting and identifying the impact pit receives landing sequence images with any resolution, outputs the detected position and the circumscribed rectangle side length of the impact pit, and identifies the number of the impact pit in a database. In the first stage, a single-stage multiplication anchor point detection network is adopted, and a rapid feature extraction layer is adopted to rapidly extract features and reduce the network scale. In order to realize the detection of the multi-scale impact pit, a plurality of detection layers are output, different detection layers are responsible for detecting the impact pits with different scales, multiplication anchor point operation is adopted on the detection layer with a low level, namely, a characteristic diagram corresponds to a plurality of points on an original image, and the proportion of detection points positioned at the center of the small-scale impact pit is improved.
And in the identification stage, the frame is matched with the frame and the frame is matched with the database, a weighted bipartite graph matching algorithm KM algorithm is used for matching, a characteristic mode consisting of the to-be-identified collision pit and surrounding collision pits is utilized to construct a characteristic descriptor as the weight of the KM algorithm, and the intersection ratio of the collision pits and the external rectangles is used as the weight of the KM algorithm for multiple matching, so that the success rate of identification and matching is improved. In order to realize the matching between the frame and the database, a reference pose is also required to be input, and the pose is derived from the input when the frame is the first frame; in the subsequent sequence image, the pose is output from the pose of the previous frame. To achieve "frame-to-frame" matching, it is necessary to predict the state of the pit at the current field of view of the previous frame, and to complete the prediction and update of the detected pit state using kalman filtering.
And finally, the network weight of the whole method is smaller than 10Mb, the detection rate reaches the current best, and the recognition speed reaches real-time (25 FPS) on the CPU.
Impact pit detection network construction
The invention constructs a high-performance impact pit detection network through the following three modules: a rapid feature extraction layer; a feature pyramid enhancement layer; and detecting an output layer. As shown in fig. 2.
① Fast feature extraction layer
FEL totally 4 characteristic compression modules, the step length between the previous layer and the next layer is 2, and a CReLU activation function is connected after conv1 and conv2, and the activation function is as follows:
CReLU(x)＝Concat[ReLU(x),ReLU(-x)]
relu (x) represents inputting feature x into Relu activation functions, relu (-x) represents inputting feature x back into Relu activation functions, resulting in two activated features that are stitched in the channel dimension (Concat).
Concat splicing two matrixes, wherein the splicing coordinate axis is the dimension of the channel, the activation function simultaneously passes the input characteristic x and the opposite number thereof through the activation function ReLU, and then the two are spliced to realize multiplication of the channel number. The network weights learned by the shallow network are approximately symmetrical, namely 128-channel weights, the weights of the front 64 channels and the rear 64 channels are approximately in the relation of opposite numbers, and the activation function can be adopted to reduce the network weights while increasing the channel number. Conv3 and Conv3 are shown in FIG. 2 as two parts, each of which is characterized by a convolution kernel enhancement of 1*1 and then a convolution kernel extraction of 3*3. Features can be extracted quickly and effectively by doubling the number of channels of CReLU and using a convolution operation with a step size of 2 continuously.
② Feature pyramid enhancement layer
In order to reasonably select the features output by the fast extraction layer, we calculate the receptive field of each feature map in the fast feature extraction layer, where the receptive field represents how large area of pixels on the original image can be sensed by one point of the feature map, and the receptive fields of Conv1, conv2, conv3, and Conv4 are 3,7, 15, and 31, respectively. The convolution layers conv2, conv3_2 and conv4_2 are selected as input features of the feature pyramid, output convolution layers P2, P3 and P4 are obtained, and the pyramid top layer P5 is obtained through the maximum pooling operation of the P4. P2-P5 will be input as crash pit features into the final classification and regression network. To reduce the number of parameters, the number of channels of the feature pyramid output is set to 128. In order to detect small-sized (diameter less than 12 pixels) impingement pits, a detection layer must be arranged on a convolution layer with a smaller step size to ensure sufficient detail information, the FPN structure helps to fuse high-level semantic information, and fused features can effectively detect impingement pits with various sizes.
③ Impact pit detection output layer
The P2, P3, P4 and P5 feature layers are connected with a detection output, and the output comprises a classification branch and a regression branch. And mapping each position on the feature map back to the original map, namely, a plurality of anchor points, namely, paving m/s multiplied by n/s anchor points on the corresponding original map, wherein the image of m multiplied by n pixels is changed into the feature map with the size of m/s multiplied by n/s through feature extraction with the step length of s. The classification branch consists of 4 convolution layers with the convolution kernel size of 3 and a prediction convolution layer, and is used for predicting that the output channel number of the convolution layer is 2. By means of this branch it can be determined whether these laid anchor points belong to the crash pit. The regression branch predicts the distances t, l, b, r of each anchor point from the four sides of the real target frame. Combining the classification branch and the regression branch to obtain whether each anchor point is positioned inside the impact pit or not, and if so, the anchor points are positioned inside the impact pit, and the anchor points are positioned at the distance from four sides of the impact pit.
④ Multiplication-based anchor operation
The P2 features output by feature pyramid FPN detect a large number of impingement pits with diameters of 12 to 16 pixels. The P2 layer has a step size of 4 and a 12-16 pixel diameter pit with 3*3 to 4*4 pit characteristics after feature compression can be used for classification and regression. The scaling of the P3 layer relative to the original input image is 4 times, namely, each position on the feature map corresponds to an image with the size of 4*4 on the original image, unlike the process of paving an anchor point responsible for detecting pixels smaller than 16 only at the center of an image with the size of 4*4, the anchor point is shifted leftwards and rightwards, and each point on the feature map corresponds to 4 anchor points. The point (x_p, y_p) at each position on the P2 feature map is mapped back to the upper left corner on the original image as (x_p 4/H, y_p 4/W), where H and W are the height and width of the image. The multiplication operation translates x_p downwards at equal intervals, translates y_p leftwards, multiplies x and y by 2 times, translates x and y once, and corresponds to 4 points of the original image at each point on the feature map. The dimension of the P3 layer is C p3×H×W,Cp3, the dimension of the classification branch weight is 4×C p3 ×3×3, and the output is 4×H×W.
Labeling as a positive sample point in a network requires that the following two conditions be met: (1) The anchor points of the feature point mapping return to the original image are in the real annotation frame; (2) The anchor point regression maximum value does not exceed the threshold value set by the feature point, and if the anchor point regression maximum value exceeds the threshold value, the detection layer with the larger threshold value detects the impact pit. But a large number of small-sized impingement pits exist, resulting in a high-level detection layer that does not have sufficient features to detect. Only the smallest P2 detection layer (stride=4) can be selected to detect an impingement pit of 12-14 diameter. A collision pit with the diameter of 12 corresponds to the P3 detection layerThe feature map features of the size, each feature corresponds to 9 detection anchor points, the anchor points positioned in the center of the impact pit are helpful for the impact pit detection, the quantitative estimation is calculated by using a centrality formula, and the centrality formula is that
L, r, t, b are distances from the center point to the left, right, upper and lower sides of the rectangular frame
As shown in fig. 3. Only one point with centrality larger than 0.5 before the multiplication anchor point is the most central point, only one point with the final detection function is achieved, and after multiplication, 9 points at the center can participate in classification and regression, namely, the proportion of the anchor points with high centrality is from 1:9, promote to 9:36=1:4 after multiplication anchor point.
Construction of impact pit identification method
The invention constructs a high-performance impact pit identification method by the following two aspects: the identification of matches between "frames and database" and the identification of matches between "frames". The identification method comprises the following steps: as shown in fig. 4.
Step a, set of crash pits already completed before the kth frame matching the database Θ' =Θ {0,1,2,…,k-1}, each crash pit having a corresponding statePredicting state/>, of impact pit under kth frame view field through Kalman filterThus the predicted composition set of all the impingement pits/>The parameter meaning theta' is a state set of an impact pit detected by a frame from 0 to k-1, and for any element x in the set, the state set is predicted by a Kalman filter prediction equation to obtain/>Pred (x) is the state of prediction x in the kth frame, and then the prediction value/>
Step b, calculatingFeature descriptors of all the impact pits in the (a);
c, the detection output of the kth frame image forms a set Θ k, and the feature descriptor of each impact pit in Θ k is calculated;
Step d, calculating each impact pit and each impact pit in Θ The intersection ratio and the feature coding distance of all the collision pits in the model (II) are taken as weights;
Step e, matching Θ k and Θ through a weighted bipartite graph matching algorithm Matching the impact pits in the test piece;
f (1), if the matching number is greater than a threshold value, inputting a recognition matching result into EFFICIENT PERSPECTIVE N point (EPnP) algorithm to obtain a kth frame pose;
F (2), if the number of matches is less than a threshold value, projecting an impact pit in the global database by using the pose of the previous frame to obtain a set Θ global, carrying out operations on Θ k and Θ global consistent with the steps (d) - (e) to obtain a frame-database matching result, and obtaining the current pose through EPnP algorithm;
Step g, updating the state of the Kalman filter by using the matching result;
and h, repeating the processes from the step a to the step g until the landing image sequence is processed.
① Feature descriptor encoding
In the above process, as shown in fig. 5, in a specific embodiment of the present invention, a method for performing feature construction by using structural information formed by the to-be-identified crash pit and surrounding crash pits is designed, and the specific steps are as follows:
(1) Initializing feature vector v= (0, …, 0) T∈Rn×1, where E is a discrete factor.
(2) Selecting an impact pit to be matched and m impact pits around the impact pit to form a specific mode, calculating the angle theta between the two impact pits around and the central impact pit, and updating v [ q ] = v [ q ] +1 by theta epsilon [ qe, (q+1) e), wherein q = theta/e is rounded downwards
(3) Repeating the steps (1) - (2) until all the included angles are calculated
② Identification matching between frames and database
The specific implementation process of the identification and matching of the frame and the database in the step f (2) is as follows: the identified match between the projected frame and the database requires an initial pose input when running the first frame of image, and then uses the pose output of the previous frame to project the impingement pit in the database to the current field of view. The method comprises the following specific steps:
(1) Camera internal reference K is known, which projects the impingement pits in the pre-established celestial body surface impingement pit set database to the pixel plane:
(u,v)T＝Γ(X)＝KPXT
Γ (X) represents an equation for projecting the center point of the impingement pit onto the camera plane, X is the center point of the impingement pit, represented by three-dimensional coordinates, (u, v) T is the center point coordinates of the projection of the center point of the impingement pit onto the pixel plane, and P is the camera outlier.
(2) And 5 points on the edge of the impact pit X are taken, corresponding points in 5 pixel planes are obtained through the projection, and an elliptic equation and a corresponding circumscribed rectangle are obtained through fitting.
(3) After all the crash pits in the database are projected to the current field of view, as a target set Θ global, Θ global is matched with the detected output crash pit set Θ k. The matching method is a weighted bipartite graph optimal matching algorithm, which is called KM algorithm. The weights in the algorithm consist of two parts: (1) the cross-over ratio between rectangular frames; (2) The specific pattern of the impingement pit and surrounding impingement pit is to be matched.
③ Identification matching from frame to frame
The specific implementation process of the steps a-f (1) is as follows:
(1) The Kalman filter calculates the predicted value of the preamble frame impingement pit C k-1 under the current view angle The state of any one of the crash pits is expressed as:
where x, y is the center coordinates of the target frame, r is the aspect ratio, Is the variation of the aspect ratio, h is the height, the remaining variablesThe derivatives of x, y, h, respectively, represent the speed of camera motion in the x, y, h directions:
x′＝Fxk-1
P′＝FPk-1FT+Q
Wherein x 'is the predicted value of the state quantity, x k-1 is the state of the k-1 frame, F is the state transition matrix, P' is the predicted value of the covariance matrix, P k-1 is the covariance matrix of the k-1 frame, and Q is the systematic error.
(2) The weights between the two impingement pits are calculated using a matrix-to-co-ratio (IOU) distance and a feature descriptor Euclidean distance.
(3) And calculating the feature codes of each impact pit according to the coding algorithm, and calculating the Euclidean distance between the features.
(4) The distance is input into a KM algorithm, two groups of impact pits are matched according to the matrix intersection ratio IOU distance, and the rest of the non-matched impact pits are matched by using the characteristic coding distance.
(5) Successfully matched impingement pits update the kalman filter parameters using the formula:
y＝z-Hx′
K＝P′HT(HP′HT+R)-1
xk+1＝x′+Ky
P＝(I-KH)P′
z is the detection result of the K frame merle, H is a measurement matrix, x 'is the state predicted value of the K frame, P' is the predicted covariance matrix of the K frame, the values of x 'and P' are obtained by the prediction equation, I is a unit matrix, R is a noise matrix, and K is Kalman filtering gain.
After the matching is completed, updating the state of the preamble frame impact pit set Θ': all the set of crash pits Θ k detected in the new picture are matched with the crash pits in the set of crash pits Θ' =Θ {0,1,…,k-1} for which the pre-sequence has been identified as complete. The impact pockets in the Θ' set have three states, namely a "new" impact pocket S 1, a "mature" impact pocket S 2, and an "old" impact pocket S 3. The matching result updates the state of the Θ set. If a crash pit is detected for the first time, i.e. the crash pit is not associated with any previous crash pit, then this crash pit is added to the set Θ' and the state is "new" crash pit. If a crash pit already in the Θ set is associated with a crash pit in the Θ set for 2 consecutive frames, its state is updated to "mature", and if a crash pit in the Θ set is not associated with 2 consecutive frames, its state is changed to "old" crash pit and it is deleted. As shown in fig. 6.
In summary, the present invention provides a method for detecting and identifying a landing image crash pit, which is characterized by comprising the following steps: detecting the crash pit through the crash pit detection network DPCDN of the single-stage multiplication anchor point; the method comprises the steps of matching the impact pit targets output by the impact pit detection network DPCDN with a pre-established celestial body surface impact pit set, and identifying the impact pit targets output by the impact pit detection network DPCDN. The invention provides an operation for multiplying anchor points, which is helpful for detecting small-size collision pits and improving the network performance. Based on the detection result, a weighted bipartite graph-based collision pit matching recognition method is provided, a simple and efficient weight calculation method is designed, and information of collision pits to be matched and surrounding collision pits is fused. The invention has the characteristics of high average detection precision, strong detection capability for small-size dense impact pits and high recognition speed of fusion sequence image information, and achieves the leading performance level of impact pit detection and recognition.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explanation of the principles of the present invention and are in no way limiting of the invention. Accordingly, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present invention should be included in the scope of the present invention. Furthermore, the appended claims are intended to cover all such changes and modifications that fall within the scope and boundary of the appended claims, or equivalents of such scope and boundary.
Claims (5)
1. A landing image crash pit detection and identification method, comprising the steps of:
Step S100, detecting an impact pit through an impact pit detection network DPCDN of a single-stage multiplication anchor point;
Step S200, matching the impact pit targets output by the impact pit detection network DPCDN with a pre-established celestial body surface impact pit set to realize the identification of the impact pit targets output by the impact pit detection network DPCDN;
The impact pit detection network rapidly extracts features through a rapid feature extraction layer, then uses a feature pyramid to enhance the features, performs detection output on the enhanced feature layer, and ensures that different layers of the feature pyramid are responsible for detecting impact pit targets with different sizes so as to realize rapid multi-scale impact pit detection;
the multiplication anchor point operation is used on the low-level characteristic layer, so that the detection of the small-size impact pit is realized;
the step of the impact pit detection network for rapidly extracting the characteristics through the rapid characteristic extraction layer comprises the following steps:
(1) Characteristic input passes through a convolution layer, then results are inverted and spliced with results which are not inverted, and the spliced characteristic is input into a ReLU activation function;
(2) The rapid feature extraction layer comprises 4 convolution layers, wherein the first two convolution layers complete feature extraction of the image through the 4 convolution layers by using the operation of the step (1);
the operation of the multiplication anchor comprises the following steps:
Each point on the feature map Corresponds to/>, on the original imageWherein s is the scaling multiple of the original image to the layer of feature image, and each point on the feature image corresponds to a plurality of points of the original image under the operation of multiplying the anchor pointsK is the multiplication number and Z is the integer set, the network has multiple output detection layers, and multiplication anchor operation is used on the first detection output.
2. The method of claim 1, wherein the step of matching and identifying the pool target output by the pool detection network DPCDN with a pre-established celestial surface pool set comprises:
step S210, projecting a pre-established celestial body surface impact pit set database to a current view field through an initial gesture to serve as a set to be matched; taking an impact pit target output by an impact pit detection network as a target set;
step S220, after all the impact pits in the pre-established celestial body surface impact pit set database are projected to the current field of view, the set to be matched is set With target set/>Matching;
step S230, fusing landing image sequence information, and matching the k-th frame of impact pit with the impact pit which is completely identified in the previous k-1 frame so as to realize the matching identification of the impact pit target set output by the impact pit detection network and the impact pit in the pre-established celestial body surface impact pit set database.
3. The method for detecting and identifying the impact pit of the landing image according to claim 2, wherein a weighted bipartite graph matching algorithm is used for matching the set to be matched and the target set, and the weight is the intersection ratio between the target frames of the impact pit and the Euclidean distance of the characteristic mode of the impact pit; the weights in the weighted bipartite graph optimal matching algorithm comprise two parts: (1) the cross-over ratio between rectangular frames; (2) The specific pattern of the impact pits to be matched and the impact pits surrounding the impact pits to be matched in the set to be matched.
4. A method of detecting and identifying landing image crash pits as claimed in claim 3, wherein the step of feature pattern euclidean distance weight calculation in the weighted bipartite graph best matching algorithm is as follows:
(1) Initializing feature vectors Wherein/>，/>R is a set of real numbers, which is a discrete factor;
(2) Selecting an impact pit to be matched as a central impact pit, forming a specific mode by the central impact pit and m impact pits around the central impact pit, and calculating the angle between any two impact pits around and the central impact pit By/>Update feature vector/>The angle/>I.e. is the included angle,/>Rounding downwards;
(3) Repeating the steps (1) and (2) until all the included angles are calculated.
5. The method for detecting and identifying the crash pit of the landing image according to claim 4, wherein in the step of matching and identifying the crash pit output by the crash pit detection network with the crash pit in the pre-established crash pit database, the matching of the crash pit which is already matched with the crash pit under the current field of view is realized by using a kalman filter and a weighted bipartite graph optimal matching algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110826180.9A CN113537098B (en) | 2021-07-21 | 2021-07-21 | Method for detecting and identifying impact pit in landing image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110826180.9A CN113537098B (en) | 2021-07-21 | 2021-07-21 | Method for detecting and identifying impact pit in landing image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113537098A CN113537098A (en) | 2021-10-22 |
| CN113537098B true CN113537098B (en) | 2024-06-21 |
Family
ID=78129150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110826180.9A Active CN113537098B (en) | 2021-07-21 | 2021-07-21 | Method for detecting and identifying impact pit in landing image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113537098B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734219A (en) * | 2018-05-23 | 2018-11-02 | 北京航空航天大学 | A kind of detection of end-to-end impact crater and recognition methods based on full convolutional neural networks structure |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109977812B (en) * | 2019-03-12 | 2023-02-24 | 南京邮电大学 | Vehicle-mounted video target detection method based on deep learning |
| JP2020203075A (en) * | 2019-05-12 | 2020-12-24 | オリジン ワイヤレス， インコーポレイテッドＯｒｉｇｉｎ Ｗｉｒｅｌｅｓｓ， Ｉｎｃ． | Method, apparatus, and system for wireless tracking, scanning, and monitoring |
-
2021
- 2021-07-21 CN CN202110826180.9A patent/CN113537098B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108734219A (en) * | 2018-05-23 | 2018-11-02 | 北京航空航天大学 | A kind of detection of end-to-end impact crater and recognition methods based on full convolutional neural networks structure |
Non-Patent Citations (1)
| Title |
|---|
| 应用灰度特征的行星陨石坑自主检测方法与着陆导航研究;陈建清;朱圣英;崔祜涛;崔平远;;宇航学报;20140830(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113537098A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108960211B (en) | Multi-target human body posture detection method and system | |
| CN111126472A (en) | Improved target detection method based on SSD | |
| CN111460926A (en) | Video pedestrian detection method fusing multi-target tracking clues | |
| CN113674328A (en) | Multi-target vehicle tracking method | |
| CN112132130B (en) | Real-time license plate detection method and system for whole scene | |
| CN112633220B (en) | Human body posture estimation method based on bidirectional serialization modeling | |
| CN102222341A (en) | Method and device for detecting motion characteristic point and method and device for detecting motion target | |
| Kuo et al. | Dynamic attention-based visual odometry | |
| Wang et al. | MCF3D: Multi-stage complementary fusion for multi-sensor 3D object detection | |
| Guindel et al. | Joint object detection and viewpoint estimation using CNN features | |
| CN113160283A (en) | Target tracking method based on SIFT under multi-camera scene | |
| CN115063447A (en) | Target animal motion tracking method based on video sequence and related equipment | |
| US20070223785A1 (en) | Image processor and method | |
| CN113297959A (en) | Target tracking method and system based on corner attention twin network | |
| CN115375736A (en) | Image-based pedestrian trajectory tracking method and device | |
| Wang et al. | Robust and fast vehicle turn-counts at intersections via an integrated solution from detection, tracking and trajectory modeling | |
| Kottath et al. | Mutual information based feature selection for stereo visual odometry | |
| CN116883457B (en) | Light multi-target tracking method based on detection tracking joint network and mixed density network | |
| CN113537098B (en) | Method for detecting and identifying impact pit in landing image | |
| Jo et al. | Mixture density-PoseNet and its application to monocular camera-based global localization | |
| CN116958927A (en) | Method and device for identifying short column based on BEV (binary image) graph | |
| Martelli et al. | FPGA-based robust ellipse estimation for circular road sign detection | |
| Sormann et al. | Dels-mvs: Deep epipolar line search for multi-view stereo | |
| Dong et al. | Polarmask-tracker: Lightweight multi-object tracking and segmentation model for edge device | |
| CN115953743A (en) | Parking space state identification method based on improved YOLO model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |